esoul-sdk 0.24.0 → 0.25.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/llms-full.txt CHANGED
@@ -20,7 +20,11 @@ platform's implementations.
20
20
  - [Getting started](docs/01-getting-started.md)
21
21
  - [Manifest reference](docs/02-manifest.md)
22
22
  - [Documentation index](#documentation)
23
+ - [A service on your computer, connected to your app](docs/20-a-service-on-your-computer.md)
24
+ - [CHANGELOG.md](CHANGELOG.md) — what each version added
25
+ - `api-reference.md` — every export with its signature and doc comment
23
26
  - `llms.txt` and `llms-full.txt` — the documentation in one file, for coding models
27
+ - The package on npm: <https://www.npmjs.com/package/esoul-sdk>
24
28
 
25
29
  ## Features
26
30
 
@@ -84,7 +88,8 @@ test helpers. Functions that need the platform (`pluginDb`, `viewerProfile`, `mi
84
88
  throw `host only` when called outside it; they are exercised through the test harness or in a
85
89
  Forge workbench.
86
90
 
87
- Requirements: Node.js 20 or newer, TypeScript 5, React 19 for the UI entry point.
91
+ Requirements: Node.js 20 or newer (22 for `esoul-device` on a computer), TypeScript 5, React 18 or
92
+ newer for the UI entry point.
88
93
 
89
94
  ## Concepts
90
95
 
@@ -124,11 +129,13 @@ application that provides the named contract.
124
129
  | Import | Contents |
125
130
  |---|---|
126
131
  | `esoul-sdk` | Schema and event types, `defineOps`, `handleOp`, `opTool`, `definePluginChannel`, `defineBindingEvent`, `checkBinding`, `resolveAppRole`, `chartSvg`, `incompleteStateNotice`, `deterministicReducerId`, `callPluginOp`, `kickPluginTask`, `pluginRouteUrl`, `nanoid`, the manifest schema, `compileEnvelope`, `compileCustomRole` |
127
- | `esoul-sdk/server` | `pluginDb`, `viewerProfile`, `sseStream`, `mintRouteToken`, `readAppState`, `callWorkspaceTool`, `computer`, `emitPluginAppEvent`, `generateAppImage`, `renderChartImage`, `setAppRole`, `listAppRoles`, `defineAppRole`, `removeAppRole`, `getPluginConnectionCredentials`, `pluginFiles`, and the context types (`PluginOpContext`, `PluginRouteContext`, `PluginViewer`, `PluginServerModule`) |
128
- | `esoul-sdk/react` | `useViewer`, `useSignInWall`, `useAppCanEdit`, `usePluginEventDispatch`, `usePluginRealtime`, `useWorkspaceTools`, `usePluginWorkspaceFiles`, `usePluginFileUpload`, `useFileSources`, `useFileSourceEntries`; the types `PluginViewerPublic`, `SignInWall` |
129
- | `esoul-sdk/testing` | `memoryDb`, `fakeViewer`, `runOp`, `fakeApps`, `capture`, `startMockOAuth` |
132
+ | `esoul-sdk/server` | `pluginDb`, `viewerProfile`, `sseStream`, `mintRouteToken`, `readAppState`, `callWorkspaceTool`, `computer`, `devices`, `questions`, `credentials`, `llm`, `filesForOp`, `pluginFiles`, `emitPluginAppEvent`, `generateAppImage`, `renderChartImage`, `setAppRole`, `listAppRoles`, `defineAppRole`, `removeAppRole`, `getPluginConnectionCredentials`, and the context types (`PluginOpContext`, `PluginRouteContext`, `PluginViewer`, `PluginServerModule`) |
133
+ | `esoul-sdk/react` | `useViewer`, `useSignInWall`, `useAppCanEdit`, `usePluginEventDispatch`, `runOptimistic`, `useAppNav`, `usePluginRealtime`, `useWorkspaceTools`, `useRemoteReconcile`, `useCredential`, `<ConnectAccount/>`, `<ConnectComputer/>`, `useDevices`, `useDeviceRun`, `useDeviceActions`, the file hooks (`useFileSources`, `useFileSourceEntries`, `useFolderAutocomplete`, `useFileUrls`, `useTransfer`, `usePluginWorkspaceFiles`, `usePluginFileUpload`), `ImageLabeler`; the types `PluginViewerPublic`, `SignInWall` |
134
+ | `esoul-sdk/machine` | what runs ON the person's computer: the `Program` type for a device program (types only), and the `esoul-device` runtime |
135
+ | `esoul-sdk/testing` | `memoryDb`, `fakeViewer`, `runOp`, `fakeApps`, `capture`, `memoryFiles`, `scriptedModel`, `simGmail`, `fakeCredentials`, `startMockOAuth`, `listFailedRequests` |
130
136
  | `esoul-sdk/schemas/plugin.schema.json` | The JSON Schema of the manifest |
131
137
  | `esoul-app validate <dir>` | Command-line validator for a package folder |
138
+ | `esoul-device` | The runtime on a person's computer: `connect`, `status`, `sh`, `config set`, `disconnect`, `uninstall` (docs/18) |
132
139
 
133
140
  ## Getting started
134
141
 
@@ -502,7 +509,8 @@ Platform columns on every table: `id`, `workspaceId`, `nodeId`, `ownerId`, `crea
502
509
  | `setAppRole(ctx, { email, role, attrs? })` | The owner gives an account one of the application's role words, or a composed role, with attribute values (`{ customer: "204" }`) validated against `roles.attributes`. Recorded as an event. |
503
510
  | `listAppRoles(ctx)` | `{ people, roles, custom, envelope }`. |
504
511
  | `defineAppRole(ctx, definition)`, `removeAppRole(ctx, name)` | The owner composes or removes a role inside `roles.custom`. |
505
- | `getPluginConnectionCredentials(ctx)` | The credentials of the instance's bound connection. |
512
+ | `credentials(ctx).slot(name)` | The person's account for a declared slot — Google, any OAuth 2.0 provider (Microsoft, Dropbox…) or a pasted API key: `status()` and `fetch(url, init?)`, with the token or key added by the platform and sent only where the slot may reach. Your code never holds it (docs/08). |
513
+ | `getPluginConnectionCredentials(connectionId, pluginId)` | Older: the token of one of the app's own `connections`, readable only from the app's own running call for the workspace owner's connection. Prefer a `credentials` slot (docs/08 §9). |
506
514
  | `pluginFiles(ctx)`, `filesForOp(ctx)` | Workspace files and file sources from server code: read by ref or address (`"workspace:/Mail/a.pdf"`), `save` to a path, and `transfer` a list of files into a folder with platform progress, Stop and a clean Retry (docs/09). |
507
515
 
508
516
  ### `esoul-sdk/react`
@@ -527,6 +535,7 @@ Platform columns on every table: `id`, `workspaceId`, `nodeId`, `ownerId`, `crea
527
535
  | `runOp(server, opName, { viewer, args?, db?, apps?, notifyFails? })` | Calls `pluginServer.ops[opName]` with a platform-shaped context. Returns `{ result, notified, emitted }`; throws the op's refusals. |
528
536
  | `fakeApps(fixtures)` | Bound applications for `ctx.apps`. |
529
537
  | `capture()` | A recorder for callbacks. |
538
+ | `fakeCredentials(slots)` | A stand-in for `credentials(ctx)`: each slot answers as the test says (`status`, `hosts`, `fetch`), the platform's refusals apply, and `calls` records every request (docs/08 §7). |
530
539
  | `startMockOAuth(opts?)` | A local OAuth server for connection tests. |
531
540
 
532
541
  ### Command line
@@ -569,7 +578,7 @@ Exits 0 when the folder is a well-formed application package; otherwise prints e
569
578
  | [5. The UI](docs/05-ui.md) | React, hooks, theme, responsive rules |
570
579
  | [6. The server](docs/06-server.md) | Ops, routes, webhooks, `computer`, charts, calling other applications |
571
580
  | [7. Background tasks](docs/07-background-tasks.md) | The durable executor, the replay model, concurrency |
572
- | [8. Connections](docs/08-connections.md) | OAuth and API-key connections held by the platform |
581
+ | [8. Accounts](docs/08-connections.md) | The person's Google, Microsoft or any OAuth 2.0 / API-key account, used through the platform without your code holding the token |
573
582
  | [9. Files](docs/09-files.md) | Workspace files, Drive, file providers |
574
583
  | [10. Testing](docs/10-testing.md) | The fold contract, ops, tasks and rules as tests |
575
584
  | [11. Shipping](docs/11-shipping.md) | The repository, review, release, install, dependencies |
@@ -578,12 +587,20 @@ Exits 0 when the folder is a well-formed application package; otherwise prints e
578
587
  | [14. Your own tables](docs/14-database.md) | `db`, rules, scopes, sealed fields, indexes, migrations |
579
588
  | [15. Realtime](docs/15-realtime.md) | Topics, audiences, addressing |
580
589
  | [16. Bindings](docs/16-bindings.md) | Slots, contracts, reaching another application |
590
+ | [17. Editing and merging](docs/17-editing-and-merging.md) | An editor that takes changes from other devices and agents; merges |
591
+ | [18. Your computer](docs/18-your-computer.md) | The device arm: commands, the workspace, programs, `esoul-device` |
581
592
  | [19. Model calls](docs/19-model-calls.md) | `llm(ctx)`: billed model calls, budgets, `scriptedModel()` for tests |
593
+ | [20. A service on your computer](docs/20-a-service-on-your-computer.md) | A walk-through: a program that keeps a service running on the person's computer, talking to the app both ways — built in the Forge, then installed |
582
594
 
583
595
  ## Versions
584
596
 
597
+ From 0.20.0 on, every version is in [CHANGELOG.md](CHANGELOG.md) (accounts through the platform — Google, then any OAuth 2.0 or API-key provider in 0.25.0,
598
+ model calls, file transfers, opened-from-outside navigation, server-only events, the questions bell,
599
+ mail waits). Earlier:
600
+
585
601
  | Version | Changes |
586
602
  |---|---|
603
+ | 0.16.0 – 0.19.1 | `runOptimistic` and the failed-request banner; stable list paging; editor sync (`useRemoteReconcile`, `mergeLines` / `mergeFields` / `mergeRecordsById`); the device arm — `devices(ctx)`, `<ConnectComputer/>`, the `esoul-device` runtime, declared commands, the shared command line, an app's own program on the computer (docs/17, 18). |
587
604
  | 0.15.0 | **Files work in a Forge box**, and do more: `FilesApi.list(…, opts)` narrowed at the source, `listAll`, `resolvePath`, `readMany`, `write` (make or replace by name — a label file beside its image; manifest `fileSources.write`), `readGrant` + `fileGrantUrl` (signed, expiring URLs for an `<img>` in a preview and for a machine). UI: `useFolderAutocomplete`, `useFileUrls`, `useFileSourceEntries(…, opts)` with `loadMore`, `listFileEntries`, `resolveFilePath`. `ImageLabeler` / `PolygonCanvas` — the Explorer's polygon editor as a component. Pure: `pairImagesWithLabels`, `serializeLabelMe`, `parseLabelMe`, `labelFileNameFor`. Testing: `memoryFiles`. |
588
605
  | 0.14.0 | Attribute-scoped access: `roles.attributes` (typed; validated when granted; `viewer.attrs`); scoped rule principals `{ role, where }` with `viewer.<attribute>`, applied to reads, aggregates, updates and deletes and filling or refusing creates, in both database clients; a rule combining `creator` with a scoped role is refused. `compileManifestRules` is the one reader of a manifest's rules. VIEW AS personas carry attributes (`visitor-a?customer=204`). |
589
606
  | 0.13.0 | Composed roles: `roles.custom` envelope; `defineAppRole`, `removeAppRole`; `setAppRole` with attributes; `listAppRoles` returns composed roles and the envelope. Reads, updates and deletes are narrowed by the composition in both database clients; surfaces not handed to the role are refused; `useViewer().customRole` and `can`. `access: [<roles>]` on a surface. Composed roles are personas in the Forge. |
@@ -771,14 +788,22 @@ schema is shipped at `schemas/plugin.schema.json`.
771
788
  | `entry` | the schema module without extension (`app`) |
772
789
  | `icon` | a lucide-react name from the curated set (the workbench lists them when yours is wrong); missing → the generic glyph, never a broken tile |
773
790
  | `author` | stamped by the workbench at scaffold: `url` is `esoul:user:<id>`, and it is what the release script and the id-collision check read. No email — a manifest may become public |
774
- | `ops` | server operations exported by `server.ts` (docs/06) |
791
+ | `ops` | server operations exported by `server.ts` (docs/06). A list of names (each `write`), or an object `name → { "access": "read" \| "public" \| "write" \| "roles", "roles"?, "requires"?: "account" }` (docs/13) |
792
+ | `routes` | HTTP routes `/api/plugins/<id>/route/<name>` served by `server.ts`, same object form; `"access": "token"` makes a door for a machine (`mintRouteToken`, docs/06) |
775
793
  | `webhooks` | inbound routes `POST /api/plugins/<id>/webhook/<hook>` served by `server.ts` (docs/06) |
776
- | `kickableTasks` | task names the **browser** may kick (docs/07) |
794
+ | `kickableTasks` | task names the **browser** may kick (docs/07); list or the object form |
777
795
  | `pollTasks` | tasks the platform kicks on a cadence, 5-minute granularity, minimum 5 (docs/07) |
796
+ | `roles.vocabulary` / `roles.default` | your app's own words for people (`["staff", "customer"]`) and which word each platform role starts as (docs/13) |
778
797
  | `roles.attributes` | what a grant may say about a person, typed: `{ "customer": "int" }` — rules scope a role by `viewer.<attribute>` (docs/13) |
779
798
  | `roles.custom` | the envelope an OWNER composes roles inside at runtime — fields a role may be scoped by / denied / change, transitions, surfaces, attributes (docs/13) |
780
799
  | `workspaceTools` | the grant wall for calling OTHER apps' tools — `<applicationType>:<tool base name>`; absent = no cross-app access (docs/04, 06) |
781
- | `connections` | OAuth2 / API-key connections the user grants once and the platform holds sealed; **names of env vars, never values**, and only the app's own: `PLUGIN_<ID>__<NAME>`; endpoints https (docs/08) |
800
+ | `db` | your own tables: fields, indexes, per-row `rules`, `sealed` fields — generated and scoped by the platform, additive on update (docs/14) |
801
+ | `channel` | realtime topics and who hears each (audiences) (docs/15) |
802
+ | `uses` / `provides` | binding slots another app fills by contract, and the contracts you provide (docs/16) |
803
+ | `credentials` | the person's accounts the app uses, one slot for now: `family: "google"` (`scopes`), `"oauth2"` (any OAuth 2.0 provider — `provider: { name, authorizeUrl, tokenUrl, scopes, hosts }`) or `"apiKey"` (`provider: { name, hosts, header?, prefix? }`), plus `why`, `label?`, `background?`. The person connects once in Account settings and assigns the account; your code calls through `credentials(ctx).slot(name).fetch` and never sees the token, key or client secret (docs/08) |
804
+ | `llm` | model calls billed to the owner: tiers, `dailyBudgetUsd`, `why`; called with `llm(ctx)` / `ctx.llm` — never a model SDK (docs/19) |
805
+ | `device` | parts that run on the person's computers: `commands`, `program` (`main`, `resident`, `setup`, `selfTest`), `config`, `redact`, `onAppDeleted` (docs/18, 20) |
806
+ | `connections` | older: OAuth2 / API-key connections whose token your code reads itself (`getPluginConnectionCredentials`); prefer a `credentials` slot. **Names of env vars, never values**, and only the app's own: `PLUGIN_<ID>__<NAME>`; endpoints https (docs/08 §9) |
782
807
  | `fileSources` / `fileProviders` | consent to read workspace files and providers; providers you contribute (docs/09) |
783
808
  | `fileProducers` | `[{ key, label }]` — your app's own bytes as a file-transfer source; implemented by `pluginServer.fileProducers[key]` (docs/09) |
784
809
  | `platformApi` | refuse install outside `[min, max]` of the platform contract version |
@@ -1031,6 +1056,12 @@ execute: async (args) => {
1031
1056
  }
1032
1057
  ```
1033
1058
 
1059
+ Wait for **one of**: `threadId` (the next message on that thread), or `fromEmail` (the next message
1060
+ from that sender, optionally with exactly that `subject`). `timeoutSeconds` defaults to 7 days
1061
+ (60 s – 30 days); `onResume` / `onTimeout` replace the lines the run reads when it wakes
1062
+ (`{{messageId}}`, `{{from}}`, `{{subject}}`, `{{threadId}}` are filled in). The wake is scoped to
1063
+ this workspace and this inbox, and never to the account's own sends.
1064
+
1034
1065
  The run wakes on `workspace/email.message_arrived` for this inbox (the platform sends it for the
1035
1066
  Gmail app's live mail) and first checks the inbox's folded `messages` for an arrival newer than
1036
1067
  `afterTimestamp`, so a reply that landed before the wait is not missed.
@@ -1294,6 +1325,31 @@ inline SVG pattern and the thing's icon reads as "not photographed yet"; a grey
1294
1325
  rectangle reads as broken. Fix the aspect ratio in both branches and nothing
1295
1326
  moves when a photograph arrives.
1296
1327
 
1328
+ ## Images, films and fonts the app ships with — assets
1329
+
1330
+ An app's package holds text only. Its media is an **asset**: the bytes go to a content-addressed
1331
+ store the platform runs, the package carries `assets.json` (written FOR you), and the app renders
1332
+ one immutable URL that is the same in a Forge preview, installed, and on a custom domain.
1333
+
1334
+ Put one with the board's `put_app_asset {pluginId, name, fileId | contentBase64 | sourceUrl}` (a
1335
+ workspace file — what `generate_image` / `generate_video` made —, a public https URL ≤ 64 MB, or
1336
+ ≤ 3 MB of base64 a model wrote itself), or over the local `esoul-mcp` with `forge_put_asset(...,
1337
+ file=…)` for a file on the person's disk (≤ 64 MB). Names: `webp png jpg jpeg gif avif svg mp4 webm
1338
+ mp3 m4a woff2 woff pdf json` — never a script or a page.
1339
+
1340
+ ```tsx
1341
+ import assets from "./assets.json";
1342
+ import { assetUrl, assetUrlOrNull } from "esoul-sdk";
1343
+
1344
+ <video src={assetUrl(assets, "hero.mp4")} poster={assetUrl(assets, "hero.jpg")} playsInline muted loop />
1345
+ <img src={assetUrl(assets, "logo.webp")} alt="" />
1346
+ const maybe = assetUrlOrNull(assets, "intro.mp4"); // null when the asset is not there
1347
+ ```
1348
+
1349
+ `assetUrl` never throws: a name missing from `assets.json` logs once and returns an address that
1350
+ 404s, so the page stays up. The same bytes twice are one object; `remove_app_asset` takes a name
1351
+ out. URLs are `/pa/<pluginId>/<sha256>.<ext>`, served immutable.
1352
+
1297
1353
 
1298
1354
 
1299
1355
  ==============================================================================
@@ -1357,8 +1413,21 @@ folds to one item. Fail CLOSED when no secret is configured.
1357
1413
  ## `readAppState(nodeId)`
1358
1414
 
1359
1415
  One app, folded to head: `{ nodeId, workspaceId, applicationType, foldedSeq, state }`, or `null`
1360
- when absent. Use it for your own instance and, with care, for other apps in the same workspace.
1361
- Never a raw database read — the import wall refuses `prisma` and the state column lags the log.
1416
+ when absent. Never a raw database read — the import wall refuses `prisma` and the state column lags
1417
+ the log.
1418
+
1419
+ It reads for the call the platform is running, and only what that call may see:
1420
+
1421
+ - your own app type in your workspace — always;
1422
+ - another app type in your workspace — only when your manifest grants one of its tools in
1423
+ `workspaceTools` (`"spreadsheet:add_row"` lets you read spreadsheets); otherwise it is refused,
1424
+ naming the line to add;
1425
+ - an app in another workspace — `null`, the same answer as an app that does not exist;
1426
+ - from a webhook (which runs for no instance) — only your own type, in any workspace, to find the
1427
+ instance to route to;
1428
+ - outside an op, route, task or webhook — it throws.
1429
+
1430
+ A Forge box applies the same rule.
1362
1431
 
1363
1432
  ## Asking the person — `questions(ctx)`
1364
1433
 
@@ -1397,9 +1466,12 @@ const r = await callWorkspaceTool({
1397
1466
  ```
1398
1467
 
1399
1468
  Gated exactly like the UI: the manifest must declare `"my_computer:claude_task"` in
1400
- `workspaceTools`, and the target must live in the calling app's own workspace. Name a specific
1469
+ `workspaceTools`, and the target must live in the calling app's own workspace. `pluginId` and
1470
+ `nodeId` must be the running call's OWN app instance — any other pair is refused. Name a specific
1401
1471
  instance with `targetNodeId` instead of `appType`. For a `my_computer`, prefer `computer(ctx, nodeId)`
1402
- below, which parses the answers and knows the waits. This is how an app runs a `my_computer`
1472
+ below, which parses the answers and knows the waits. (For a part of YOUR app that runs on the
1473
+ person's computer — a service, a program, declared commands — use the device arm instead:
1474
+ [18](18-your-computer.md), [20](20-a-service-on-your-computer.md).) This is how an app runs a `my_computer`
1403
1475
  (commands, Claude Code sessions, results), fills a spreadsheet from a job, or books a calendar.
1404
1476
 
1405
1477
  ## `computer(ctx, machineNodeId)` — a paired machine, from server code
@@ -1510,10 +1582,14 @@ Emit ANOTHER app's own events (its `dataCreator` shape) into the same workspace,
1510
1582
  your plugin — for mirroring into a native app whose processors you cannot call directly.
1511
1583
  Same-workspace only.
1512
1584
 
1513
- ## Connections in server code
1585
+ ## Accounts in server code
1514
1586
 
1515
- `ctx.cloudConnectionId` is the instance's bound connection; resolve it with
1516
- `getPluginConnectionCredentials(connectionId, pluginId)` (docs/08).
1587
+ The person's accounts — Google, Microsoft or any OAuth 2.0 provider, or a pasted API key — are
1588
+ reached through `credentials(ctx).slot(name).fetch(url)`: the platform adds the token or key, sends
1589
+ it only where the slot may reach, and your code never holds it (docs/08). `ctx.cloudConnectionId`
1590
+ is the id of the account assigned to the instance (an id, not a secret). The older
1591
+ `getPluginConnectionCredentials(connectionId, pluginId)` reads one of the app's own `connections`
1592
+ (docs/08 §9).
1517
1593
 
1518
1594
  ## What server code may import
1519
1595
 
@@ -1593,6 +1669,34 @@ the example stops streaming — nothing durable happened unless the handler wrot
1593
1669
  task (docs/07) survives everything and costs seconds per hop. The stopwatch ships both and
1594
1670
  records which one drove each run; measure before you choose.
1595
1671
 
1672
+ ## The headless bridge — a browser on the platform, for a page-shaped engine
1673
+
1674
+ Some engines exist only as a web page: a WebAssembly CAD kernel, a canvas renderer. The
1675
+ person's open tab can run it; `headlessBridge(ctx, …)` runs it on the platform too, so an agent
1676
+ with nobody's tab open (a phone with the app closed, an agent network, MCP) gets the same work
1677
+ done — no computer of the person's involved.
1678
+
1679
+ ```ts
1680
+ import { headlessBridge } from "esoul-sdk/server";
1681
+
1682
+ const r = await headlessBridge(ctx, {
1683
+ url: "https://my-runtime.example/page.html?parent=https%3A%2F%2Fmy-runtime.example",
1684
+ boot: { field: "type", oneOf: ["ready", "error"] }, // the page's own ready message
1685
+ calls: [{ id: "replay", message: { myBridge: 1, id: "replay", method: "replay", args: { … } } }],
1686
+ });
1687
+ r.boot; // the ready message
1688
+ r.replies[0]; // the first message whose `id` equals "replay"
1689
+ r.timings; // launchMs / bootMs (cold only), per-call ms, totalMs
1690
+ r.warm; // true when the page was already open on the platform
1691
+ ```
1692
+
1693
+ The page is loaded top-level, so it is its own `parent`: a bridge written for an iframe answers
1694
+ unchanged when `?parent=` names the page's own origin. The browser and the loaded page stay warm
1695
+ on the platform between calls for the same url; the first call pays the launch and the page's
1696
+ download, later calls only the calls. `url` must be https and public; at most 32 calls; replies
1697
+ up to 24 MB. A Forge box has no browser: there the call throws and says so — keep the app open
1698
+ in the preview and let the tab do the work.
1699
+
1596
1700
 
1597
1701
 
1598
1702
  ==============================================================================
@@ -1688,170 +1792,252 @@ Unit-test the handler with a fake `ctx` (`step.run` that just calls the function
1688
1792
 
1689
1793
 
1690
1794
  ==============================================================================
1691
- # 8. Connections and OAuth — tokens the platform holds for you
1795
+ # 8. Accounts — the person's logins, used by your app without your app holding them
1692
1796
 
1693
- An app that talks to an outside service declares the connection; the person grants it once in
1694
- Account settings; the platform stores the tokens sealed and refreshes them; your code asks for a
1695
- credential when it needs one. **Secrets never live in your package**: the manifest names
1696
- environment variables, not values.
1797
+ An app that works with a person's account at another service (Gmail, Outlook, OneNote, Dropbox,
1798
+ Notion, Spotify, GitHub…) declares a **credential slot** in `plugin.json`. The person connects the
1799
+ account once, at the user level, in Account settings, and assigns it to the app. Your server code
1800
+ calls the service **through the platform**: `credentials(ctx).slot(name).fetch(url)`. The platform
1801
+ checks who is calling and where the request goes, then adds the token or key itself.
1697
1802
 
1698
- ```json
1699
- "connections": [{
1700
- "key": "github", "kind": "oauth2", "label": "GitHub",
1701
- "authorizeUrl": "https://github.com/login/oauth/authorize",
1702
- "tokenUrl": "https://github.com/login/oauth/access_token",
1703
- "oauthScopes": ["repo"],
1704
- "clientIdEnv": "PLUGIN_MY_APP__GITHUB_CLIENT_ID", "clientSecretEnv": "PLUGIN_MY_APP__GITHUB_CLIENT_SECRET"
1705
- }]
1706
- ```
1803
+ **The token, the key and the OAuth client secret never reach your code, your package, your logs or
1804
+ you, the app's developer.** They are sealed (AES-256-GCM) on the server and sent only to the hosts
1805
+ the slot declares, which the person saw before connecting.
1707
1806
 
1708
- **An app names only its own environment variables**: `PLUGIN_<ID>__<NAME>` — the app id upper-cased
1709
- with `-` as `_`, two underscores, then upper-case words joined by single underscores (app `my-app` →
1710
- `PLUGIN_MY_APP__GITHUB_CLIENT_ID`). The platform reads these values and sends them to the URLs your
1711
- manifest names, so any other name is refused at validation, at install and again every time the
1712
- platform would read it: an app that could name `DATABASE_URL` would be sent the platform's database.
1713
- The double underscore gives every name exactly one owner (a single one would give app `shop` the
1714
- names of `shop-demo`); a name ending in `_WEBHOOK_TOKEN` is the platform's webhook secret and is
1715
- refused too. `authorizeUrl` and `tokenUrl` must be https — plain http only to `127.0.0.1` /
1716
- `localhost`, where the mock provider from `esoul-sdk/testing` listens. The operator of the platform
1717
- sets the variables; `pluginEnvPrefix("my-app")` and `pluginEnvNameProblem(id, name)` from `esoul-sdk`
1718
- give the rule in code.
1807
+ Three families:
1719
1808
 
1720
- `kind: "apiKey"` instead declares `headerNames` — the header names the app will send; the person
1721
- pastes the key once.
1809
+ | `family` | For | The person connects in | The platform enforces |
1810
+ |---|---|---|---|
1811
+ | `google` | Gmail, Calendar, Contacts, Drive through the platform's verified Google client | Account settings → **Google** | the endpoints your declared Google scopes reach |
1812
+ | `oauth2` | any OAuth 2.0 provider: Microsoft (Outlook, OneNote, OneDrive, Teams), Dropbox, Spotify, GitHub, Slack… | Account settings → **Accounts** | the hosts your provider block declares; the provider enforces the scopes |
1813
+ | `apiKey` | a key or personal access token the person pastes (Notion, OpenWeather, Airtable…) | Account settings → **Accounts** | the hosts your provider block declares |
1722
1814
 
1723
- ## Using a credential
1815
+ One account serves every app of the same provider: a person who connects Microsoft once can assign
1816
+ that account to an Outlook app and a OneNote app. Each assignment is the person's act, recorded on
1817
+ the app's timeline (`workspace/credential_bound`), like Gmail's Google account.
1724
1818
 
1725
- > **Deprecated for Google.** `getPluginConnectionCredentials` hands your code the token itself. For
1726
- > the person's Google account use a platform account slot instead (below): your code never holds
1727
- > the token. It stays for an app's own oauth2/apiKey connections until those move to slots too.
1819
+ ## 1. Declare the slot
1728
1820
 
1729
- ```ts
1730
- import { getPluginConnectionCredentials } from "esoul-sdk/server";
1821
+ ### `google`
1731
1822
 
1732
- async function listRepos(ctx: PluginOpContext) {
1733
- if (!ctx.cloudConnectionId) throw new Error("connect GitHub in Account settings first");
1734
- const cred = await getPluginConnectionCredentials(ctx.cloudConnectionId, "my-app");
1735
- if (cred.kind !== "oauth2") throw new Error("expected an OAuth connection");
1736
- const r = await fetch("https://api.github.com/user/repos", { headers: { Authorization: `Bearer ${cred.accessToken}` } });
1737
- // …
1823
+ ```json
1824
+ "credentials": {
1825
+ "mailbox": {
1826
+ "family": "google",
1827
+ "scopes": ["gmail.modify"],
1828
+ "label": "Mailbox",
1829
+ "why": "Reads, labels and sends mail in the inbox you choose.",
1830
+ "background": true
1831
+ }
1738
1832
  }
1739
1833
  ```
1740
1834
 
1741
- The platform unseals, checks expiry and refreshes with the stored refresh token before handing
1742
- you `accessToken`. Never log it, never put it in an event, never return it from a tool.
1835
+ `scopes` are Google's, short (`gmail.modify`) or full
1836
+ (`https://www.googleapis.com/auth/gmail.modify`), from: `gmail.readonly`, `gmail.modify`,
1837
+ `gmail.send`, `gmail.compose`, `gmail.labels`, `calendar.events`, `calendar.readonly`, `contacts`,
1838
+ `contacts.readonly`, `drive.readonly`, `drive.file`. `openid`, `email` and `profile` are implicit.
1743
1839
 
1744
- An instance is **bound** to one connection (`ctx.cloudConnectionId`); a person with two GitHub
1745
- accounts adds two instances. File providers you contribute receive `ctx.connectionId` per active
1746
- connection (docs/09).
1840
+ ### `oauth2` — any OAuth 2.0 provider
1841
+
1842
+ ```json
1843
+ "credentials": {
1844
+ "microsoft": {
1845
+ "family": "oauth2",
1846
+ "label": "Microsoft account",
1847
+ "why": "Reads and sends mail in your Outlook mailbox.",
1848
+ "background": true,
1849
+ "provider": {
1850
+ "name": "Microsoft",
1851
+ "authorizeUrl": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
1852
+ "tokenUrl": "https://login.microsoftonline.com/common/oauth2/v2.0/token",
1853
+ "scopes": ["offline_access", "User.Read", "Mail.ReadWrite", "Mail.Send"],
1854
+ "hosts": ["graph.microsoft.com"],
1855
+ "accountUrl": "https://graph.microsoft.com/v1.0/me",
1856
+ "accountField": "userPrincipalName",
1857
+ "authorizeParams": { "prompt": "select_account" }
1858
+ }
1859
+ }
1860
+ }
1861
+ ```
1747
1862
 
1748
- ## Platform accounts (Google)
1863
+ | `provider` field | Meaning |
1864
+ |---|---|
1865
+ | `name` | The service as the person knows it. Shown on every screen and in every refusal. |
1866
+ | `authorizeUrl`, `tokenUrl` | The provider's endpoints (authorization code + PKCE S256). https only (plain http only to `127.0.0.1` / `localhost`, for the mock provider). Accounts are "of the same provider" when both URLs are the same. |
1867
+ | `scopes` | The provider's own scope words. Ask for `offline_access` (or the provider's equivalent) so the platform gets a refresh token. |
1868
+ | `hosts` | **The only hosts the token is ever sent to**: DNS names, no scheme, port, path or IP address. 1–10. |
1869
+ | `accountUrl` + `accountField` | A GET on one of `hosts` that names the account (e.g. Graph's `/me` → `userPrincipalName`). Declare it, or ask for `openid` so the provider's id token names the account: **an account the platform cannot name is refused** (two unnamed accounts would overwrite each other). |
1870
+ | `authorizeParams` (optional) | Extra sign-in parameters (`prompt`, `login_hint`…). `client_id`, `redirect_uri`, `scope`, `state` and PKCE are the platform's. |
1871
+ | `tokenAuth` (optional) | `"body"` (default: client id and secret in the form) or `"basic"` (HTTP Basic). |
1872
+ | `clientIdEnv`, `clientSecretEnv` (optional) | A client **the platform's operator** holds for your app (`PLUGIN_<ID>__…` env vars, the rule in §9). Without them, each person registers their own client once (§2). |
1749
1873
 
1750
- The person connects a Google account once, in **Account settings → Google**, and assigns it to your
1751
- app. Your app declares what it needs in `plugin.json`, and calls Google **through the platform**:
1874
+ ### `apiKey` — a key the person pastes
1752
1875
 
1753
1876
  ```json
1754
1877
  "credentials": {
1755
- "mailbox": {
1756
- "family": "google",
1757
- "scopes": ["gmail.modify"],
1758
- "label": "Mailbox",
1759
- "why": "Reads, labels and sends mail in the inbox you choose.",
1760
- "background": true
1878
+ "notion": {
1879
+ "family": "apiKey",
1880
+ "why": "Reads the pages you share with the integration.",
1881
+ "provider": {
1882
+ "name": "Notion",
1883
+ "hosts": ["api.notion.com"],
1884
+ "header": "Authorization",
1885
+ "prefix": "Bearer ",
1886
+ "keyHelp": "Notion → Settings → Connections → Develop or manage integrations → Internal integration secret.",
1887
+ "verifyUrl": "https://api.notion.com/v1/users/me"
1888
+ }
1761
1889
  }
1762
1890
  }
1763
1891
  ```
1764
1892
 
1893
+ `header` defaults to `Authorization`, `prefix` to `"Bearer "` for that header and `""` for any
1894
+ other (`"X-Api-Key"`, say). `verifyUrl`, when declared, is called with the key before it is kept: a
1895
+ 401/403 refuses it. `Cookie`, `Host`, `Content-Length` and similar headers cannot carry a key.
1896
+
1897
+ ### Rules for every family
1898
+
1765
1899
  - The slot name is lower-case words joined by `-`. **One slot per app for now**; several are coming.
1766
- - `scopes` are Google's, short (`gmail.modify`) or full
1767
- (`https://www.googleapis.com/auth/gmail.modify`), from: `gmail.readonly`, `gmail.modify`,
1768
- `gmail.send`, `gmail.compose`, `gmail.labels`, `calendar.events`, `calendar.readonly`,
1769
- `contacts`, `contacts.readonly`, `drive.readonly`, `drive.file`. `openid`, `email` and `profile`
1770
- are implicit.
1771
1900
  - `why` is the sentence the person reads before assigning an account.
1772
- - `background: true` lets the app's own work with nobody present (a scheduled sync) use the
1773
- account. Without it only the account's owner, acting through the app, can.
1774
-
1775
- ### Server: `credentials(ctx).slot(name)`
1901
+ - `background: true` lets the app's own work with nobody present (a scheduled sync, a task kicked by
1902
+ the app) use the account. Without it only the account's owner, acting through the app, can.
1903
+ - `label` (optional) names the slot on screen.
1904
+
1905
+ ## 2. How the person connects
1906
+
1907
+ **Google**: Account settings → Google → Connect. One consent covers what the platform offers.
1908
+
1909
+ **`oauth2`**: Account settings → **Accounts** lists every installed app that asks for an account,
1910
+ with the hosts the token will be sent to, the scopes asked, and where the sign-in and the token
1911
+ exchange happen. If the platform holds no client for your app, the person registers **their own
1912
+ app with the provider once** (Microsoft: an Entra app registration; GitHub: an OAuth app…), gives it
1913
+ the redirect URI the screen shows — `https://<the platform's address>/api/account/credentials/callback`
1914
+ — and enters its client id and secret. The secret is sealed and never shown again; a second account
1915
+ of the same provider reuses it. Then **Connect** → the provider's sign-in → back, connected, and
1916
+ assigned to the app chosen in *Assign to*.
1917
+
1918
+ **`apiKey`**: the same screen, a key field (with your `keyHelp`), **Save key**.
1919
+
1920
+ Later:
1921
+
1922
+ - A `connectUrl` never signs anyone in by itself: it opens Account settings → Accounts on your
1923
+ app's slot, where the person sees the hosts and presses **Connect**. (A link that signed in
1924
+ silently would let any page have a person approve hosts they never saw.)
1925
+ - An app of the same provider that needs **more scopes** than the account granted gets
1926
+ `needs_consent`; connecting again from its card asks for the union, so the apps already using
1927
+ the account keep working.
1928
+ - **Each app gets only its own scopes.** When one account was granted more than an app asks for
1929
+ (OneNote's `Notes.ReadWrite` beside Outlook's `Mail.ReadWrite`), the platform refreshes a token
1930
+ narrowed to the calling slot's scopes for that app's calls — OneNote's code cannot send mail. A
1931
+ provider that will not narrow (or issued no refresh token) gets the call refused with
1932
+ `not_allowed`, never the whole grant.
1933
+ - An assignment ends when the account's owner can no longer write the app's workspace (removed
1934
+ from a shared workspace): the app answers `not_bound` until someone assigns another account.
1935
+ - An app that sends the token to a **host the account was not connected for** gets
1936
+ `needs_consent` with `missingHosts`; the person approves the host on the Accounts screen
1937
+ ("Use ann@contoso.com and allow teams.microsoft.com").
1938
+ - **Disconnect** releases every app (recorded on each app's timeline) and deletes the token or key.
1939
+
1940
+ From your app's own screen, `<ConnectAccount/>` / `useCredential().connect()` (§5) does the right
1941
+ thing for each state: assigns the person's only fitting account in place, sends them to sign in and
1942
+ assign in one trip, or opens Account settings.
1943
+
1944
+ ## 3. Server: `credentials(ctx).slot(name)`
1776
1945
 
1777
1946
  ```ts
1778
1947
  import { credentials, CredentialUnavailable } from "esoul-sdk/server";
1779
1948
 
1780
- export const listLabels: PluginOpHandler = async (ctx) => {
1781
- const g = credentials(ctx).slot("mailbox");
1782
- const s = await g.status(); // { state: "ready", account: { email } } for the owner
1783
- if (s.state !== "ready") return { ok: false, state: s.state, connectUrl: s.connectUrl };
1784
- const r = await g.fetch("https://gmail.googleapis.com/gmail/v1/users/me/labels");
1785
- return await r.json();
1949
+ export const inbox: PluginOpHandler = async (ctx) => {
1950
+ const ms = credentials(ctx).slot("microsoft");
1951
+ const s = await ms.status(); // { state: "ready", provider: { name: "Microsoft", … }, account: { email } } for the owner
1952
+ if (s.state !== "ready") return { ok: false, state: s.state, reason: s.reason, connectUrl: s.connectUrl };
1953
+ const r = await ms.fetch("https://graph.microsoft.com/v1.0/me/messages?$top=25&$select=subject,from,receivedDateTime");
1954
+ if (!r.ok) return { ok: false, status: r.status };
1955
+ return { ok: true, messages: (await r.json()).value };
1786
1956
  };
1787
1957
  ```
1788
1958
 
1789
- `fetch(url, init?)` is the standard `fetch`; the platform adds the token. Pass `ctx` itself — the
1790
- context your op, route or task was called with. A copy (`{ ...ctx }`), a context kept from an
1791
- earlier call, or one used after its call finished is refused. In a task, call it inside
1792
- `ctx.step.run(…)` as well as outside; a webhook runs for no app instance, so kick a task for the
1793
- instance and use the account there.
1959
+ `fetch(url, init?)` is the standard `fetch`; the platform adds the token (`Authorization: Bearer …`)
1960
+ or the key (in its declared header). Pass `ctx` itself — the context your op, route or task was
1961
+ called with. A copy (`{ ...ctx }`), a context kept from an earlier call, or one used after its call
1962
+ finished is refused. A **task** (in `app.tsx`, which cannot import `esoul-sdk/server`) uses
1963
+ `ctx.credentials("microsoft")` — the same slot, for that run — inside `ctx.step.run(…)` as well as
1964
+ outside; a webhook runs for no app instance, so kick a task for the instance and use the account
1965
+ there.
1966
+
1967
+ ## 4. The checks
1794
1968
 
1795
1969
  A request goes out only when **every** check passes, in this order:
1796
1970
 
1797
1971
  1. the context is the platform's, for the call running now;
1798
1972
  2. the slot is declared in your manifest;
1799
- 3. the URL is `https://`, on a Google API host, with a clean path (no `..`, no encoded `/` or `.`,
1800
- no credentials or port), and one of the slot's declared scopes reaches that method and path —
1801
- `gmail.readonly` reaches only `GET`/`HEAD` under `gmail.googleapis.com/gmail/v1/users/me/`,
1802
- `gmail.send` only `POST …/messages/send`;
1803
- 4. an account is assigned to this app instance;
1973
+ 3. the URL is `https://`, with a clean path (no `..`, no encoded `/` or `.`, no credentials or
1974
+ port, no method-override parameter), and
1975
+ - `google`: on a Google API host, and one of the slot's scopes reaches that method and path
1976
+ (`gmail.readonly` reaches only `GET`/`HEAD` under `gmail.googleapis.com/gmail/v1/users/me/`,
1977
+ `gmail.send` only `POST …/messages/send`);
1978
+ - `oauth2` / `apiKey`: on one of the slot's `hosts`;
1979
+ 4. an account of this provider is assigned to this app instance, and its owner can still write
1980
+ the instance's workspace;
1804
1981
  5. the caller is the account's owner (or the app's own background work, when the slot says
1805
1982
  `background: true`);
1806
- 6. the account has granted every declared scope, and its grant is alive.
1983
+ 6. the account has granted every declared scope, was connected for every declared host, and its
1984
+ grant is alive;
1985
+ 7. `oauth2` / `apiKey`: the host is also one the account was connected for, and resolves only to
1986
+ public addresses.
1807
1987
 
1808
- Any `Authorization`, `Cookie`, method-override or `X-Goog-*` header you set is dropped (except
1809
- `X-Goog-Upload-*` on an upload path). Redirects are followed only to an allowed URL on the same
1810
- host. Response bodies are capped at 35 MB.
1988
+ Any `Authorization`, `Cookie`, `Host`, hop-by-hop, method-override or key header you set is dropped (for Google also
1989
+ `X-Goog-*`, except `X-Goog-Upload-*` on an upload path). Redirects are followed only for GET/HEAD,
1990
+ on the same host, to a URL the slot may reach. Response bodies are capped at 35 MB. An `oauth2`
1991
+ token about to expire is refreshed first (rotated refresh tokens are kept; the token endpoint's
1992
+ redirects are never followed); a 401 is retried once with a fresh token; a refresh the provider
1993
+ refuses marks the account for reconnecting. A token is narrowed to the calling slot's scopes when
1994
+ the account was granted more (§2).
1811
1995
 
1812
- ### Refusals
1996
+ ## 5. Refusals, status, and the UI
1813
1997
 
1814
1998
  A refused call throws `CredentialUnavailable` with a `code`:
1815
1999
 
1816
2000
  | `code` | Meaning | What to do |
1817
2001
  |---|---|---|
1818
- | `not_bound` | No account is assigned to this app. | The owner assigns one in Account settings → Google. |
1819
- | `needs_consent` | The account has not granted a declared scope. `missingScopes` names them. | Send the owner to `connectUrl`. |
2002
+ | `not_bound` | No account of this provider is assigned to this app. | The owner assigns one (Account settings → Google, or → Accounts), or `<ConnectAccount/>`. |
2003
+ | `needs_consent` | The account has not granted a declared scope (`missingScopes`), or was not connected for a declared host (`missingHosts`). | Send the owner to `connectUrl` (or the Accounts screen, for hosts). |
1820
2004
  | `reconnect` | The account's grant was revoked or expired. | Send the owner to `connectUrl`. |
1821
- | `not_allowed` | This caller may not use the account, or the URL is outside the declared scopes. | Nothing, for another person; fix the URL, for your code. |
2005
+ | `not_allowed` | This caller may not use the account, or the URL is outside what the slot may reach. | Nothing, for another person; fix the URL, for your code. |
1822
2006
  | `not_declared` | The manifest has no slot of that name. | Declare it. |
1823
- | `rate_limited` | Google asked us to slow down. | Retry after `retryAfterMs`. |
1824
- | `provider_unavailable` | Google could not be reached. | Retry later. |
2007
+ | `rate_limited` | The provider asked us to slow down. | Retry after `retryAfterMs`. |
2008
+ | `provider_unavailable` | The provider could not be reached or refused the platform itself. | Retry later. |
1825
2009
 
1826
2010
  `status()` answers the same questions without throwing:
1827
- `{ state: "ready" | "not_bound" | "needs_consent" | "reconnect" | "not_allowed", account?, missingScopes?, connectUrl?, reason? }`.
1828
- `account` (the email) is present only when the caller owns the account.
1829
-
1830
- ### UI: `useCredential` and `<ConnectAccount/>`
2011
+ `{ state: "ready" | "not_bound" | "needs_consent" | "reconnect" | "not_allowed", provider?, account?, missingScopes?, missingHosts?, connectUrl?, reason? }`.
2012
+ `provider` is `{ name, family }`. `account` (the email or login) is present only when the caller
2013
+ owns the account. Every `reason` is a sentence you can show or relay; none contains a secret.
1831
2014
 
1832
2015
  ```tsx
1833
2016
  import { useCredential, ConnectAccount } from "esoul-sdk/react";
1834
2017
 
1835
- const gm = useCredential("mailbox", { nodeId: identifier.nodeId });
1836
- if (gm.state !== "ready") return <ConnectAccount slot="mailbox" nodeId={identifier.nodeId} />;
1837
- return <p>Mail for {gm.account?.email}</p>;
2018
+ const ms = useCredential("microsoft", { nodeId: identifier.nodeId });
2019
+ if (ms.state !== "ready") return <ConnectAccount slot="microsoft" nodeId={identifier.nodeId} />;
2020
+ return <p>Mail for {ms.account?.email}</p>;
1838
2021
  ```
1839
2022
 
1840
2023
  The hook asks the platform the same question `status()` answers, for the person looking at the
1841
- screen, and asks again when the window regains focus. `gm.connect()` opens the platform's Google
1842
- consent for missing permissions or a reconnect, and Account settings → Google to assign an account.
2024
+ screen, and asks again when the window regains focus or an account is assigned anywhere.
2025
+ `ms.connect()` assigns, signs in, grants or reconnects, whichever the state needs.
2026
+ `<ConnectAccount/>` is one line and one button that does it, worded with the provider's name.
1843
2027
 
1844
- ### Why there is no token
2028
+ ## 6. Why there is no token in your code
1845
2029
 
1846
- The platform's Google client is verified for Gmail, Calendar, Contacts and Drive together, and
1847
- the token it holds carries all of it. Google cannot narrow a token to one app's scopes, and your
1848
- server code runs inside the platform's process. So the token stays with the platform: the
1849
- scopes you declare are enforced at the proxy, per request, and a mistake in an app cannot leak
1850
- the person's mail to somewhere it should not go.
2030
+ - **Google**: the platform's Google client is verified for Gmail, Calendar, Contacts and Drive
2031
+ together, and its token carries all of it; Google cannot narrow it per app. The scopes you
2032
+ declare are enforced at the proxy, per request.
2033
+ - **Everyone else**: your server code runs inside the platform's process, and the person trusts the
2034
+ platform with their login, not every app they install. A token in app code could be logged,
2035
+ stored, or sent anywhere; through the proxy it goes only to the hosts the person approved, and
2036
+ only when the person (or, for `background` slots, the app's own work) is the caller.
1851
2037
 
1852
- ### In a Forge box: one simulated Google account
2038
+ ## 7. In a Forge box, and in tests
1853
2039
 
1854
- A Forge box holds no credential, so it holds a world instead: one simulated Gmail account,
2040
+ **`google`: one simulated Google account.** A Forge box holds no credential, so it holds a world instead: one simulated Gmail account,
1855
2041
  `owner@sim.mail.example`, with a seeded mailbox of 300 synthetic messages (`simGmail()` from
1856
2042
  `esoul-sdk/testing`). Every instance's slot is bound to it, so `status()` answers
1857
2043
  `{ state: "ready", account: { email: "owner@sim.mail.example" } }` for the owner and
@@ -1876,6 +2062,141 @@ The world is deterministic (same seed, same mailbox) and survives preview restar
1876
2062
  In the app's own tests (`esoul-box test`) every test file gets a fresh world of the same seed
1877
2063
  and nothing is written. The installed app uses the person's real account and never the sim.
1878
2064
 
2065
+ **`oauth2` / `apiKey`: no outside account exists in a box.** `status()` answers
2066
+ `{ state: "not_bound", provider, reason }` with the reason "no Microsoft account in the Forge
2067
+ preview: connect one to the INSTALLED app…", and `fetch()` refuses with `not_bound` — after
2068
+ production's host wall, so a URL the installed app could never reach is refused as `not_allowed`
2069
+ here too. Build it in this order:
2070
+
2071
+ 1. the code that calls the service, tested with `fakeCredentials` from `esoul-sdk/testing`:
2072
+
2073
+ ```ts
2074
+ import { fakeCredentials } from "esoul-sdk/testing";
2075
+
2076
+ const creds = fakeCredentials({
2077
+ microsoft: {
2078
+ hosts: ["graph.microsoft.com"],
2079
+ fetch: (url) => Response.json(url.pathname === "/v1.0/me/messages" ? { value: [{ subject: "Hi" }] } : {}),
2080
+ },
2081
+ });
2082
+ jest.mock("esoul-sdk/server", () => ({ ...jest.requireActual("esoul-sdk/server"), credentials: (ctx: object) => creds.credentials(ctx) }));
2083
+ // … run the op (runOp), then:
2084
+ expect(creds.calls[0]).toMatchObject({ slot: "microsoft", method: "GET", url: "https://graph.microsoft.com/v1.0/me/messages?$top=25" });
2085
+ ```
2086
+
2087
+ It applies the platform's rules (declared slot, https, `hosts`, `Authorization`/`Cookie`
2088
+ dropped, a not-ready `status` thrown as its code) and records every call. A task test sets
2089
+ `ctx.credentials = (slot) => creds.credentials(ctx).slot(slot)`.
2090
+ 2. the screen's not-connected state, which the box shows as it is;
2091
+ 3. install the app, then the person connects the account (§2) and the app runs against it.
2092
+
2093
+ ## 8. Worked example: Outlook and OneNote on one Microsoft account
2094
+
2095
+ Two apps, one account. Each declares the same provider (the same `authorizeUrl` and `tokenUrl`) with its own scopes:
2096
+
2097
+ ```jsonc
2098
+ // outlook-lite/plugin.json
2099
+ "credentials": { "microsoft": { "family": "oauth2", "why": "Reads and sends your Outlook mail.", "background": true,
2100
+ "provider": { "name": "Microsoft",
2101
+ "authorizeUrl": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
2102
+ "tokenUrl": "https://login.microsoftonline.com/common/oauth2/v2.0/token",
2103
+ "scopes": ["offline_access", "User.Read", "Mail.ReadWrite", "Mail.Send"],
2104
+ "hosts": ["graph.microsoft.com"], "accountUrl": "https://graph.microsoft.com/v1.0/me", "accountField": "userPrincipalName" } } }
2105
+
2106
+ // onenote-lite/plugin.json — the same provider block, scopes ["offline_access", "User.Read", "Notes.ReadWrite"]
2107
+ ```
2108
+
2109
+ The person, once:
2110
+
2111
+ 1. **Register an app with Microsoft**: <https://entra.microsoft.com> (or portal.azure.com) →
2112
+ App registrations → New registration. Supported account types: *any organizational directory
2113
+ and personal Microsoft accounts* (that is what `/common/` serves). Redirect URI: platform
2114
+ **Web**, the URI Account settings → Accounts shows.
2115
+ 2. Certificates & secrets → New client secret → copy its **Value**. (API permissions need no
2116
+ change: Microsoft asks for the scopes at sign-in.)
2117
+ 3. In ExternalSoul: Account settings → Accounts → *Outlook lite · Microsoft account* → enter the
2118
+ client id (the registration's *Application (client) ID*) and the secret → Connect → sign in.
2119
+ 4. *OneNote lite* asks for `Notes.ReadWrite` the account has not granted: its **Connect** signs in
2120
+ again for the union (no new client needed), and both apps use the one account.
2121
+
2122
+ The apps' code:
2123
+
2124
+ ```ts
2125
+ // outlook-lite/server.ts
2126
+ const ms = credentials(ctx).slot("microsoft");
2127
+ await ms.fetch("https://graph.microsoft.com/v1.0/me/sendMail", {
2128
+ method: "POST",
2129
+ headers: { "Content-Type": "application/json" },
2130
+ body: JSON.stringify({ message: { subject, body: { contentType: "Text", content: text }, toRecipients: [{ emailAddress: { address: to } }] } }),
2131
+ });
2132
+
2133
+ // onenote-lite/server.ts
2134
+ const pages = await (await credentials(ctx).slot("microsoft").fetch("https://graph.microsoft.com/v1.0/me/onenote/pages?$top=20")).json();
2135
+ await credentials(ctx).slot("microsoft").fetch(`https://graph.microsoft.com/v1.0/me/onenote/sections/${sectionId}/pages`, {
2136
+ method: "POST", headers: { "Content-Type": "text/html" },
2137
+ body: `<!DOCTYPE html><html><head><title>${title}</title></head><body>${html}</body></html>`,
2138
+ });
2139
+ ```
2140
+
2141
+ Neither app ever sees the token, the refresh token or the client secret.
2142
+
2143
+ ## 9. The app's own connections (`connections`) — older, discouraged
2144
+
2145
+ Before credential slots, an app declared `connections`: the person connects in Account settings →
2146
+ Plugins, and server code reads the token itself with `getPluginConnectionCredentials`. **Use a
2147
+ credential slot instead** — with `connections` the token reaches your code, the client must be set
2148
+ by the platform's operator, nothing assigns a connection to an app instance, and it does not work in
2149
+ a Forge box. It remains for existing apps, restricted: the call must come from your own op, route
2150
+ or task, and only connections of the owner of the workspace that call runs in are readable.
2151
+
2152
+ ```json
2153
+ "connections": [{
2154
+ "key": "github", "kind": "oauth2", "label": "GitHub",
2155
+ "authorizeUrl": "https://github.com/login/oauth/authorize",
2156
+ "tokenUrl": "https://github.com/login/oauth/access_token",
2157
+ "oauthScopes": ["repo"],
2158
+ "clientIdEnv": "PLUGIN_MY_APP__GITHUB_CLIENT_ID", "clientSecretEnv": "PLUGIN_MY_APP__GITHUB_CLIENT_SECRET"
2159
+ }]
2160
+ ```
2161
+
2162
+ ### Environment variables an app may name
2163
+
2164
+ **An app names only its own environment variables**: `PLUGIN_<ID>__<NAME>` — the app id upper-cased
2165
+ with `-` as `_`, two underscores, then upper-case words joined by single underscores (app `my-app` →
2166
+ `PLUGIN_MY_APP__GITHUB_CLIENT_ID`). The platform reads these values and sends them to the URLs your
2167
+ manifest names, so any other name is refused at validation, at install and again every time the
2168
+ platform would read it: an app that could name `DATABASE_URL` would be sent the platform's database.
2169
+ The double underscore gives every name exactly one owner (a single one would give app `shop` the
2170
+ names of `shop-demo`); a name ending in `_WEBHOOK_TOKEN` is the platform's webhook secret and is
2171
+ refused too. `authorizeUrl` and `tokenUrl` must be https — plain http only to `127.0.0.1` /
2172
+ `localhost`, where the mock provider from `esoul-sdk/testing` listens. The operator of the platform
2173
+ sets the variables; `pluginEnvPrefix("my-app")` and `pluginEnvNameProblem(id, name)` from `esoul-sdk`
2174
+ give the rule in code.
2175
+
2176
+ `kind: "apiKey"` instead declares `headerNames` — the header names the app will send; the person
2177
+ pastes the key once.
2178
+
2179
+ ### Reading a connection
2180
+
2181
+ The connection's id reaches your code where the platform hands it over: a file provider you
2182
+ contribute receives `ctx.connectionId` per active connection (docs/09).
2183
+
2184
+ ```ts
2185
+ import { getPluginConnectionCredentials } from "esoul-sdk/server";
2186
+
2187
+ async function listRepos(ctx: { connectionId: string }) {
2188
+ const cred = await getPluginConnectionCredentials(ctx.connectionId, "my-app");
2189
+ if (cred.kind !== "oauth2") throw new Error("expected an OAuth connection");
2190
+ const r = await fetch("https://api.github.com/user/repos", { headers: { Authorization: `Bearer ${cred.accessToken}` } });
2191
+ // …
2192
+ }
2193
+ ```
2194
+
2195
+ The platform unseals, checks expiry and refreshes with the stored refresh token before handing
2196
+ you `accessToken`. Never log it, never put it in an event, never return it from a tool. Called
2197
+ outside your own running op, route or task, or for a connection of anyone but the owner of the
2198
+ workspace your call runs in, it throws.
2199
+
1879
2200
 
1880
2201
 
1881
2202
  ==============================================================================
@@ -2170,6 +2491,35 @@ it("the description never claims an emptiness it could not read", () => {
2170
2491
  A UI test renders once with a fixed state and checks the visible chrome and the empty state.
2171
2492
  Keep suites fast; they run beside the live preview on the same machine.
2172
2493
 
2494
+ ## The testing kit — `esoul-sdk/testing`
2495
+
2496
+ Worlds your tests run in, so nothing real is touched and every run is the same:
2497
+
2498
+ | Helper | Gives you | Chapter |
2499
+ |---|---|---|
2500
+ | `memoryDb(manifest)` + `fakeViewer(kind, opts)` | your declared tables in memory, with your REAL row rules compiled; a viewer of any kind (`owner`, `member`, `visitor`, `anonymous`, `agent`, roles and attributes) | 14 |
2501
+ | `runOp(pluginServer, "op", { viewer, args, db, apps })` | calls one op with a real-shaped `ctx`; returns its result plus what it `emit`ted and `notify`d | 14 |
2502
+ | `fakeApps({ slot: { tools } })`, `capture()` | bound apps for `ctx.apps.<slot>` (calls recorded); a recorder for any callback | 16 |
2503
+ | `memoryFiles(tree)` | the files door (`filesForOp`) over an in-memory tree: sources, list, read, write, transfers — with `crashAt` / `failItem` to prove a retry leaves no residue | 09 |
2504
+ | `scriptedModel(policy)` | `llm(ctx)` answering from your script; `calls` records every request | 19 |
2505
+ | `simGmail(opts)` | a Gmail API at the HTTP level (`fetch`, `install()`), with accounts, `deliver(...)`, personas that reply (`addPersona`), a cold-campaign `crowd(n)`, a virtual clock (`advance(ms)`), `faults` (429/5xx/401/network), `snapshot()` / restore; `sent` and `outbox` say what left | 08 |
2506
+ | `fakeCredentials(slots)` | `credentials(ctx)` without accounts: per slot a `status`, the `hosts` it may reach and a `fetch` handler; the platform's refusals (undeclared slot, plain http, a host off the list, a slot not ready) and `calls` recording every request | 08 |
2507
+ | `startMockOAuth()` | a local OAuth 2 provider for a connection's sign-in and refresh | 08 |
2508
+ | `listFailedRequests()`, `subscribeFailedRequests`, `resetFailedRequests` | what `runOptimistic` reverted and named on the banner | 05 |
2509
+
2510
+ ```ts
2511
+ import { memoryDb, fakeViewer, runOp, scriptedModel } from "esoul-sdk/testing";
2512
+ import manifest from "./plugin.json";
2513
+ import { pluginServer } from "./server";
2514
+
2515
+ it("a customer sees only their own orders", async () => {
2516
+ const db = memoryDb(manifest);
2517
+ await runOp(pluginServer, "place-order", { viewer: fakeViewer("visitor", { userId: "a" }), args: { sku: "x" }, db });
2518
+ const b = await runOp(pluginServer, "list-orders", { viewer: fakeViewer("visitor", { userId: "b" }), db });
2519
+ expect(b.result).toEqual({ orders: [] });
2520
+ });
2521
+ ```
2522
+
2173
2523
  ## Tools
2174
2524
 
2175
2525
  Call `execute` with a recording `eventCallback` and assert the events it emitted and the text it
@@ -3276,11 +3626,37 @@ recorded as a conflict. `baseMerge` runs in every fold: keep it pure, exactly li
3276
3626
  - Device view state (which page is open, scroll, selection) belongs in component state or
3277
3627
  `sessionStorage`, never in a shared event another device would follow.
3278
3628
 
3629
+ ## 4. Declare `resolveConcurrent` for patch-shaped events
3630
+
3631
+ `baseMerge` is for events that write one whole value. An event that writes a PATCH — a list of sub-items it
3632
+ changed, each with the version it built on — can do better: declare `merge` with `scope` (the sub-item ids),
3633
+ and `resolveConcurrent`. The platform calls it only when the patch would land as a conflict (a writer it never
3634
+ saw changed one of its sub-items), with the state it lands on; it merges what it can against the bases it
3635
+ carries and returns the rewritten event:
3636
+
3637
+ ```ts
3638
+ merge: { item: (_d, ctx) => `${ctx.applicationId}:cad:model`, scope: (d) => d.delta.ops.map(keyOf), describe: (d) => d.label },
3639
+ resolveConcurrent: (state, eventData, { theirsSess }) => {
3640
+ const r = resolveEdit(state, eventData); // pure: three-way per sub-item against the base each op carries
3641
+ if (!r) return null; // cannot reason about it → the plain conflict (applied and marked)
3642
+ return r.clashes.length === 0
3643
+ ? { eventData: r.eventData } // everything merged: applied, no marker
3644
+ : { eventData: r.eventData, detail: { kind: "cad-edit", clashes: r.clashes, theirsSess } };
3645
+ },
3646
+ ```
3647
+
3648
+ What it returns is what the fold applies, on every device and on the server. Keep it pure and symmetric; a
3649
+ property test over randomised edit pairs is the cheapest proof.
3650
+
3279
3651
 
3280
3652
 
3281
3653
  ==============================================================================
3282
3654
  # 18 · Parts of your app that run on the person's computer
3283
3655
 
3656
+ > Building a service that runs on the computer and talks to your app? Start with the
3657
+ > walk-through, [20 · A service on your computer](20-a-service-on-your-computer.md); this page is
3658
+ > the reference.
3659
+
3284
3660
  Some apps need the person's own machine: a GPU for training, files that never
3285
3661
  leave a laptop, a tool installed there, a local network. The **device arm**
3286
3662
  lets an app run commands on a computer the owner connects, with the output
@@ -3340,12 +3716,17 @@ The panel lists connected computers (online, last seen, Disconnect) and a
3340
3716
  **Connect a computer** button. It shows one command to paste in a terminal:
3341
3717
 
3342
3718
  ```
3343
- npx -y -p esoul-sdk@0.19.0 esoul-device connect https://externalsoul.com/api/device/m/offers/off_…
3719
+ npx -y -p esoul-sdk@0.19.1 esoul-device connect https://externalsoul.com/api/device/m/offers/off_…
3344
3720
  ```
3345
3721
 
3346
3722
  The runtime ships inside `esoul-sdk` (the `esoul-device` command, and
3347
3723
  `esoul-sdk/machine` for code that runs on the computer). It uses only Node
3348
- built-ins, so a computer needs nothing but Node 22.
3724
+ built-ins, so a computer needs nothing but Node 22. The version in the command
3725
+ is the runtime's own, pinned by the platform (it moves more slowly than the
3726
+ package). A computer without Node uses the panel's other line, which brings a
3727
+ pinned Node first, without sudo:
3728
+ `curl -fsSL https://externalsoul.com/api/device/install | sh -s -- <link>`.
3729
+ The link expires in 10 minutes.
3349
3730
 
3350
3731
  The command holds no secret. The computer makes its own key, prints a code
3351
3732
  (`K7Q-4XM`) and the list of commands; the page shows the same code with
@@ -3415,8 +3796,10 @@ const { run, finished } = useDeviceRun({ workspaceId, nodeId, commandId });
3415
3796
  // run.output streams in as the command prints (first 1 MB kept, then run.tail)
3416
3797
  ```
3417
3798
 
3418
- `useDeviceActions({ nodeId })` gives the owner's buttons from the UI (run,
3419
- cancel, disconnect, approve an update) without writing an op.
3799
+ `useDeviceActions({ nodeId })` gives the owner's buttons from the UI (`run`,
3800
+ `cancel`, `disconnect`, `approveUpdate`, `setWorkspace` for the reach, `sh`,
3801
+ and `recent` runs) without writing an op. `useMyComputers()` and
3802
+ `<YourComputers/>` list the person's computers across apps.
3420
3803
 
3421
3804
  ## 6. Local settings stay local
3422
3805
 
@@ -3425,7 +3808,7 @@ manifest declares under `device.config` are set on the computer and never sent
3425
3808
  to the platform:
3426
3809
 
3427
3810
  ```
3428
- npx -y -p esoul-sdk@0.19.0 esoul-device config set "My app" HF_TOKEN
3811
+ npx -y -p esoul-sdk@0.19.1 esoul-device config set "My app" HF_TOKEN
3429
3812
  ```
3430
3813
 
3431
3814
  The command receives them as environment variables. `redact` patterns are
@@ -3444,9 +3827,12 @@ applied on the computer before any output leaves it.
3444
3827
 
3445
3828
  ## 8. Testing
3446
3829
 
3447
- In a Forge preview, `devices(ctx)` answers with a clear refusal and
3448
- `<ConnectComputer/>` says computers connect to the installed app. Mock
3449
- `esoul-sdk/server` in unit tests, as for any host function.
3830
+ Computers connect to INSTALLED apps. In a Forge preview `devices(ctx).list()`
3831
+ answers `[]`, every other `devices()` call throws "computers connect to
3832
+ INSTALLED apps", `useDevices`/`useDeviceRun` stay empty, `useDeviceActions`
3833
+ throws, and `<ConnectComputer/>` says so. Mock `esoul-sdk/server` in unit tests,
3834
+ as for any host function, and test a program with a fake `p`
3835
+ ([20 §5](20-a-service-on-your-computer.md)).
3450
3836
 
3451
3837
  ## 9. The workspace: agents working on the computer
3452
3838
 
@@ -3462,9 +3848,9 @@ dev server open — the owner gives the app a **workspace**:
3462
3848
  Two parties agree. **The computer** sets a ceiling when it connects:
3463
3849
 
3464
3850
  ```
3465
- npx -y -p esoul-sdk@0.19.0 esoul-device connect <link> # this folder (the default)
3466
- npx -y -p esoul-sdk@0.19.0 esoul-device connect <link> --folder ~/proj # another folder
3467
- npx -y -p esoul-sdk@0.19.0 esoul-device connect <link> --scope computer # allow the whole computer
3851
+ npx -y -p esoul-sdk@0.19.1 esoul-device connect <link> # this folder (the default)
3852
+ npx -y -p esoul-sdk@0.19.1 esoul-device connect <link> --folder ~/proj # another folder
3853
+ npx -y -p esoul-sdk@0.19.1 esoul-device connect <link> --scope computer # allow the whole computer
3468
3854
  ```
3469
3855
 
3470
3856
  **The owner** grants each app at most that ceiling from the app's page (the
@@ -3601,8 +3987,16 @@ The rules that keep it safe and quiet:
3601
3987
 
3602
3988
  Where each program stands on each computer — installing, each setup step,
3603
3989
  self-test, running, the program's own `p.status` words, why it failed —
3604
- comes back on `devices(ctx).list()` as `programState`, and `<ConnectComputer/>`
3605
- shows it. The reference app is **CPU monitor** (`src/plugins/cpu-monitor`).
3990
+ comes back on `devices(ctx).list()` as `programState` (one computer:
3991
+ `devices(ctx).program(linkId).state()`), and `<ConnectComputer/>` shows it.
3992
+
3993
+ Limits: a message (`send`) and its answer are each at most 1 MB of JSON; a
3994
+ topic matches `[a-zA-Z][a-zA-Z0-9_.:-]{0,63}`; a message for an offline
3995
+ computer expires (600 s default). `p.op` calls are resent when no answer has
3996
+ come in about 4 s (up to 3 tries) — make those ops idempotent and pass
3997
+ `{ key }`. `p.workspaceRoot` is null when the whole computer is granted.
3998
+ Nothing reaches INTO the computer: there is no tunnel or open port; a program
3999
+ reaches its own localhost and calls out. The reference app is **CPU monitor** (`src/plugins/cpu-monitor`).
3606
4000
 
3607
4001
  ### An agent on each computer
3608
4002
 
@@ -3784,3 +4178,298 @@ expect(model.calls).toHaveLength(1);
3784
4178
  In a **Forge box**, `llm(ctx)` and `ctx.llm` refuse what the platform refuses (`not_declared`,
3785
4179
  `not_allowed`, `model_not_allowed`) and answer everything else from `scriptedModel()` — the box
3786
4180
  holds no model key and never spends a token. Credits and the daily budget are not simulated there.
4181
+
4182
+
4183
+
4184
+ ==============================================================================
4185
+ # 20 · A service on your computer, connected to your app
4186
+
4187
+ A guide, start to finish: an app built in the Forge that runs its own
4188
+ long-lived service on the person's computer — a local model server, a
4189
+ sampler, a bridge to a device on the desk, a trainer's supervisor — and
4190
+ talks to it both ways. The reference for every piece is
4191
+ [18 · Parts of your app that run on the person's computer](18-your-computer.md);
4192
+ this page is the path through it.
4193
+
4194
+ ## The shape
4195
+
4196
+ ```
4197
+ your app on ExternalSoul the person's computer
4198
+ ───────────────────────── ─────────────────────────────
4199
+ server.ts ── devices(ctx).program(linkId) device/main.ts (your PROGRAM,
4200
+ .send("ask", data) ─────────▶ p.on("ask", …) kept running,
4201
+ ◀───────── answer (≤ 1 MB) sandboxed)
4202
+ ops.ts ◀── p.op("report", data) ──────── p.every / events │
4203
+ (the op sees viewer.device) ▼
4204
+ your service on localhost
4205
+ (started by the program,
4206
+ installed by setup steps)
4207
+ ```
4208
+
4209
+ - **The program is the service's keeper.** Your app ships a `device/` folder;
4210
+ the person approves it once on their computer; the runtime installs it,
4211
+ runs your setup steps (a Python venv, `npm ci`, a model download), checks it
4212
+ with your self-test, and keeps it running (`"resident": true`), restarting
4213
+ it when it exits.
4214
+ - **App → service:** your server code sends a message to the program; the
4215
+ program's handler does the work — usually a `fetch("http://localhost:…")`
4216
+ to the service it started — and returns the answer.
4217
+ - **Service → app:** the program calls one of your app's own ops with
4218
+ `p.op(...)`. The op knows it came from that computer (`ctx.viewer.device`).
4219
+ - **Nothing reaches INTO the computer.** There is no tunnel and no open port:
4220
+ the computer only calls out. The program can reach its own `localhost` and
4221
+ the internet; your app reaches the program only through `send`.
4222
+
4223
+ ## 1. Declare the program (`plugin.json`)
4224
+
4225
+ ```json
4226
+ {
4227
+ "ops": ["ask", "report", "computers"],
4228
+ "device": {
4229
+ "program": {
4230
+ "main": "device/main.ts",
4231
+ "resident": true,
4232
+ "setup": [
4233
+ { "id": "venv", "describe": "Python environment", "run": ["python3", "-m", "venv", ".venv"] },
4234
+ {
4235
+ "id": "deps", "describe": "Install the server",
4236
+ "run": [".venv/bin/pip", "install", "-r", "{program}/device/requirements.txt"],
4237
+ "inputs": ["device/requirements.txt"]
4238
+ }
4239
+ ],
4240
+ "selfTest": { "run": [".venv/bin/python", "{program}/device/self_test.py"] }
4241
+ },
4242
+ "config": { "MODEL_DIR": { "describe": "Where the model weights are on this computer" } },
4243
+ "onAppDeleted": "kill"
4244
+ }
4245
+ }
4246
+ ```
4247
+
4248
+ - `setup` runs once per step and per change of its `inputs` — an app update
4249
+ that leaves `requirements.txt` alone does not reinstall. Steps are argv, never
4250
+ a shell line; `{program}` is the program's folder. Relative paths (`.venv`)
4251
+ live in the program's data folder, which is kept across versions.
4252
+ - `selfTest` must exit 0 before the program counts as ready; make it the
4253
+ thing your service needs (`claude --version`, a GPU check, an import).
4254
+ - `config` names values that stay on the computer (`esoul-device config set
4255
+ "<app name>" MODEL_DIR ~/models`); the program reads them as `p.config`.
4256
+
4257
+ ## 2. Write the program (`device/main.ts`)
4258
+
4259
+ ```ts
4260
+ import type { Program } from "esoul-sdk/machine"; // types only — nothing is installed beside the program
4261
+ import { spawn, type ChildProcess } from "node:child_process";
4262
+ import { join } from "node:path";
4263
+
4264
+ const PORT = 8711;
4265
+ let server: ChildProcess | null = null;
4266
+
4267
+ async function healthy(): Promise<boolean> {
4268
+ try { return (await fetch(`http://127.0.0.1:${PORT}/health`)).ok; } catch { return false; }
4269
+ }
4270
+
4271
+ export default {
4272
+ async start(p) {
4273
+ // Start the service: argv, no shell; its output goes to the program's log.
4274
+ server = spawn(join(p.dataDir, ".venv/bin/python"), [join(p.programDir, "device/serve.py"), "--port", String(PORT)], {
4275
+ cwd: p.dataDir,
4276
+ env: { ...process.env, MODEL_DIR: p.config.MODEL_DIR ?? "" },
4277
+ stdio: ["ignore", "inherit", "inherit"],
4278
+ });
4279
+ for (let i = 0; i < 60 && !(await healthy()); i++) await new Promise((r) => setTimeout(r, 1000));
4280
+ p.status((await healthy()) ? "Model server ready" : "Model server did not start", { ready: await healthy() });
4281
+
4282
+ // App → service: the answer of the handler is the answer of send().
4283
+ p.on("ask", async (data) => {
4284
+ const r = await fetch(`http://127.0.0.1:${PORT}/ask`, { method: "POST", body: JSON.stringify(data), headers: { "content-type": "application/json" } });
4285
+ return await r.json();
4286
+ });
4287
+
4288
+ // Long work: answer at once, report when done (the app's op stores it).
4289
+ p.on("job", async (data) => {
4290
+ const { requestId } = data as { requestId: string };
4291
+ void (async () => {
4292
+ const r = await fetch(`http://127.0.0.1:${PORT}/job`, { method: "POST", body: JSON.stringify(data) });
4293
+ await p.op("report", { requestId, result: await r.json() }, { key: requestId });
4294
+ })();
4295
+ return { accepted: true };
4296
+ });
4297
+
4298
+ // Service → app on a clock.
4299
+ p.every(60_000, async () => p.op("report", { kind: "heartbeat", ok: await healthy() }));
4300
+ },
4301
+ async stop() { server?.kill(); },
4302
+ } satisfies Program;
4303
+ ```
4304
+
4305
+ What a program may import: Node built-ins (`node:fs`, `node:child_process`, …),
4306
+ files of its own `device/` folder, and `esoul-sdk/machine` **types only**. No
4307
+ packages — install what the service needs in a setup step. At most 2 MB and
4308
+ 200 files; download weights and datasets in setup or at start. Node 22.6+
4309
+ runs a `.ts` entry directly (no enums, no namespaces).
4310
+
4311
+ ## 3. Talk to it from the app (`server.ts`)
4312
+
4313
+ ```ts
4314
+ import { devices, type PluginOpContext } from "esoul-sdk/server";
4315
+
4316
+ // App → service. linkId names one connected computer (devices(ctx).list()).
4317
+ async function ask(ctx: PluginOpContext, args: { linkId: string; prompt: string }) {
4318
+ const r = await devices(ctx).program(args.linkId).send("ask", { prompt: args.prompt }, { waitSeconds: 30 });
4319
+ if (!r.ok) return { ok: false, error: r.error ?? "the computer did not answer" }; // offline, not approved, handler threw
4320
+ return { ok: true, answer: r.result };
4321
+ }
4322
+
4323
+ // Service → app. Only the program on a computer may report.
4324
+ async function report(ctx: PluginOpContext, args: { requestId?: string; result?: unknown; kind?: string }) {
4325
+ const device = ctx.viewer.device;
4326
+ if (!device) throw new Error("report comes from the program on a computer");
4327
+ await ctx.emit("result_reported", { machineId: device.machineId, hostname: device.hostname, ...args });
4328
+ return { ok: true };
4329
+ }
4330
+
4331
+ // Which computers there are, and where each program stands.
4332
+ async function computers(ctx: PluginOpContext) {
4333
+ return (await devices(ctx).list()).map((d) => ({
4334
+ linkId: d.linkId, hostname: d.machine?.hostname, online: d.machine?.online ?? false,
4335
+ program: d.programState?.phase ?? (d.program ? "approved" : "not approved"),
4336
+ status: d.programState?.status?.text ?? null,
4337
+ }));
4338
+ }
4339
+ ```
4340
+
4341
+ Rules that matter:
4342
+
4343
+ - **`send` waits at most 55 s** (`waitSeconds`; `0` returns at once with the
4344
+ `commandId`). For anything longer, use the job pattern above: answer
4345
+ `accepted`, finish in the background, `p.op` the result.
4346
+ - **A message and an answer are each at most 1 MB of JSON.** Topics match
4347
+ `[a-zA-Z][a-zA-Z0-9_.:-]{0,63}`. For files, hand the program a link
4348
+ (`(await filesForOp(ctx)).readGrant(...)` + `fileGrantUrl`) and let it
4349
+ download; for results, store them through ops or save files.
4350
+ - **A queued message for an offline computer expires** (600 s by default) —
4351
+ a click on Monday never runs on Thursday.
4352
+ - **`p.op` must be quick and idempotent.** The computer resends a call whose
4353
+ answer has not arrived in about 4 s (up to 3 tries). Pass `{ key }` with a
4354
+ stable id and make the op a no-op the second time (store by `requestId`).
4355
+ - **Who can do what:** `send`, `run`, `sh` and `cancel` need a caller who may
4356
+ change the workspace (`ctx.viewer.canWrite`). The program's `p.op` calls run
4357
+ as the owner who approved that computer (`viewer.kind === "agent"`,
4358
+ `viewer.device` set) through your app's own op and its own rules.
4359
+
4360
+ `devices(ctx).program(linkId).state()` gives that program's state on that
4361
+ computer (installing, a setup step, self-test, running, its last `p.status`
4362
+ words, why it failed) — the same that `list()` returns as `programState`.
4363
+
4364
+ ## 4. The page: connect a computer
4365
+
4366
+ ```tsx
4367
+ import { ConnectComputer, useDevices } from "esoul-sdk/react";
4368
+
4369
+ export function Computers({ workspaceId, nodeId }: { workspaceId: string; nodeId: string }) {
4370
+ const { devices } = useDevices({ workspaceId, nodeId }); // pick a linkId for ask()
4371
+ return <ConnectComputer workspaceId={workspaceId} nodeId={nodeId} title="Computers" />;
4372
+ }
4373
+ ```
4374
+
4375
+ `<ConnectComputer/>` lists the computers (online, last seen, the program's
4376
+ progress and words, Disconnect) and a **Connect a computer** button.
4377
+
4378
+ ## 5. Build it in the Forge — what you can prove before installing
4379
+
4380
+ The Forge box runs your app in a preview. **Computers connect to installed
4381
+ apps only**: in the box `devices(ctx).list()` answers `[]`, every other
4382
+ `devices()` call throws "computers connect to INSTALLED apps", and
4383
+ `<ConnectComputer/>` shows a note saying so. Plan the work in that order:
4384
+
4385
+ 1. **Prove the program without a computer.** It is a plain object with
4386
+ `start(p)`; give it a fake `p` in a test and call its handlers:
4387
+
4388
+ ```ts
4389
+ import program from "../device/main";
4390
+
4391
+ test("ask goes to the local service and comes back", async () => {
4392
+ const handlers: Record<string, (d: unknown) => unknown> = {};
4393
+ const ops: Array<[string, unknown]> = [];
4394
+ const p = {
4395
+ computer: { hostname: "test", platform: "linux", arch: "x64", cpus: 4, memoryGb: 8 },
4396
+ appName: "Model", programDir: __dirname, dataDir: "/tmp/model-test", workspaceRoot: null, config: {},
4397
+ status: () => {}, log: () => {}, every: () => () => {},
4398
+ on: (t: string, h: (d: unknown) => unknown) => { handlers[t] = h; },
4399
+ op: async (name: string, args: unknown) => { ops.push([name, args]); return { ok: true }; },
4400
+ };
4401
+ // stub the service: global.fetch = jest.fn(...) answering /health and /ask
4402
+ await program.start(p as never);
4403
+ expect(await handlers.ask({ prompt: "hi" })).toEqual({ text: "hello" });
4404
+ });
4405
+ ```
4406
+
4407
+ 2. **Prove the app side** with `esoul-sdk/server` mocked (`devices` returning a
4408
+ fake whose `program().send` answers), and call `report` with a faked
4409
+ `viewer.device` to see the result land in the fold.
4410
+ 3. **Run the service itself** in the box if it can run there (`run_in_app`:
4411
+ `python3 device/serve.py --port 8711 &` then `curl localhost:8711/health`) —
4412
+ the same code the program will start.
4413
+ 4. `check_app` packs `device/` and refuses a program over 2 MB / 200 files or
4414
+ one that imports a package.
4415
+
4416
+ What only the installed app can prove: the program installed and running on
4417
+ a real computer, `send`, `p.op`, the approval card and the reach choice.
4418
+
4419
+ ## 6. Install, connect, approve
4420
+
4421
+ 1. `install_app` (the board's tool, or Install on the board). The app is now
4422
+ on the platform.
4423
+ 2. Open the app, press **Connect a computer**. The panel shows one command to
4424
+ paste in a terminal on that computer:
4425
+
4426
+ ```
4427
+ npx -y -p esoul-sdk@0.19.1 esoul-device connect https://externalsoul.com/api/device/m/offers/off_…
4428
+ ```
4429
+
4430
+ No Node there? The panel's other line installs a pinned Node first, without
4431
+ sudo: `curl -fsSL https://externalsoul.com/api/device/install | sh -s -- <link>`.
4432
+ The link expires in 10 minutes and holds no secret.
4433
+ 3. The computer prints a code (`K7Q-4XM`); the page shows the same code with
4434
+ **Approve**, the program's size and version, its setup steps and
4435
+ self-test, and **what the app may reach there**: commands only, its
4436
+ folder, or the whole computer (never more than the computer allowed with
4437
+ `--scope` / `--folder` when it connected).
4438
+ 4. The runtime installs itself as a user service (systemd on Linux, a
4439
+ LaunchAgent on macOS, a plain background service where neither exists),
4440
+ so it comes back after a reboot. Linux and macOS; Windows via WSL2.
4441
+ 5. The card shows setup progress, then the self-test, then your `p.status`
4442
+ words. Your app's `computers` op (or `list()`) says `running`.
4443
+
4444
+ **Updates.** A new version that changes the program (any file, the setup, the
4445
+ self-test) has a new digest and waits for **Approve** on the card; until then
4446
+ the computer keeps running the version it has. Changing what the app may
4447
+ reach there restarts the program under the new reach.
4448
+
4449
+ **Where it runs.** Unconfined only with the whole computer granted (needed
4450
+ when the service uses the person's own logins, like Claude Code or git).
4451
+ Otherwise in an OS sandbox (bubblewrap on Linux, Seatbelt on macOS): it reads
4452
+ the system, writes only `p.dataDir` and the granted folder, and its home and
4453
+ `/tmp` are private. No sandbox on the computer → refused, never run
4454
+ unconfined. The network is not confined either way.
4455
+
4456
+ ## 7. Other ways in, and when to use them
4457
+
4458
+ | You need | Use |
4459
+ |---|---|
4460
+ | A long-lived service your app owns, two-way | **A program** (this page) |
4461
+ | A known tool run with checked parameters (`nvidia-smi`, a training script) | **Declared commands** — `device.commands` + `devices(ctx).run` ([18 §1–4](18-your-computer.md)) |
4462
+ | Agents that work IN a project folder (read, edit, patch, run tests, keep a dev server in a terminal) | **The workspace** — `devices(ctx).sh("term new dev npm run dev", { agent })` ([18 §9](18-your-computer.md)) |
4463
+ | A process that is not your program pushes data to your app (a script, a device, a CI job) | **A token route** — declare the route `"access": "token"`, mint with `mintRouteToken(ctx, { route, ttlSeconds })`, the process POSTs with `Authorization: Bearer <token>` ([06 · Routes](06-server.md)). Works in the Forge box too (the box's public preview URL). |
4464
+ | The person already runs the My Computer app and the work is one-off shell commands | **`computer(ctx, machineNodeId)`** — shell strings the owner approves, output ≤ 8,000 characters ([06](06-server.md)); works in the box through the board's tab |
4465
+
4466
+ ## 8. When it does not work
4467
+
4468
+ | You see | It means |
4469
+ |---|---|
4470
+ | `send` → `ok: false`, `expired` | the computer was offline for the message's whole life (600 s) |
4471
+ | `send` → error "not approved" / no program in `list()` | the person has not approved the program (or its new version) on the card |
4472
+ | `programState.phase === "failed"` | a setup step or the self-test failed, or the program exited five times in five minutes — the card shows the last log lines |
4473
+ | "refused: no sandbox" on the card | the computer has neither bubblewrap nor Seatbelt; grant the whole computer or install bubblewrap |
4474
+ | the same report stored twice | the op is not idempotent — key it by `requestId` (`p.op(..., { key })`) |
4475
+ | `send` answers `the answer was larger than 1 MB` | return a summary; put the bulk in a file and send a link or an id |