@collective-intelligence/extensions-sdk-backend 0.2.6 → 0.2.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,276 @@
1
+ # The solution template
2
+
3
+ What you get from next-effect-app, where each piece of code goes, and how to take a new demo from local dev to an installed package.
4
+
5
+ `next-effect-app` is the starting point for every demo. It is a bun and Turborepo monorepo with one Next.js 16 App Router web app (`apps/web`) and one Extension Package declaration (`apps/extensions`). You start a demo by generating a new repository from it with **Use this template** on GitHub. Its example features are patterns to copy: a demo keeps their shape and replaces their content with the customer's domain.
6
+
7
+ ## What you get
8
+
9
+ ```bash
10
+ apps/
11
+ web/ # @acme/web Next app: routes, composition, Data Sources, app API and BFF instances
12
+ extensions/ # @acme/extensions Extension Package declaration (composition root)
13
+ packages/
14
+ organization/acme/ # @acme/organization package identity, access, groups, external application
15
+ features/intake/ # @acme/intake lead intake: API, store, XState flow, views
16
+ features/lead-conversion/ # @acme/lead-conversion convert-lead workflow and browser flow
17
+ features/lead-qualification/ # @acme/lead-qualification lead qualifier agent, Skill, tool, Studio widget
18
+ entities/lead/ # @acme/lead Lead contract, CI schema, Lead API, Read Port, Realtime, views
19
+ entities/opportunity/ # @acme/opportunity Opportunity contract, CI schema, views
20
+ modules/identity/ # @acme/identity CI federated auth, CI runtime, principal, login providers
21
+ modules/app-api/ # @acme/app-api typed package API client, QueryClient, ApiPrefetch
22
+ modules/ui/ # @acme/ui shadcn/ui primitives, AI Elements, theme, workspace-kit
23
+ tooling/config/ # @acme/config shared TypeScript base config
24
+ ```
25
+
26
+ Intake is a form with an XState flow and an app API that keeps its data in an in-memory store. Lead conversion is a Fixed workflow started by a single-action task. Lead qualification is an agent with a Skill, a tool and a Studio widget. The repository has no docs wiki, no infra folder and no generated client code, because the browser derives each typed client from its package `HttpApi` at runtime. It also ships an agent harness: an `AGENTS.md` per scope, and `.agents/` with skills, rules and lint scripts.
27
+
28
+ ## Tiers and the import rule
29
+
30
+ Every package lives at `packages/<tier>/<name>`, keeps a flat name (`@acme/<name>`) and has exactly one `tier:*` tag in its own `turbo.json`. The six tiers, from top to bottom, are `app` (the composition roots), `organization` (package identity, access, groups, external application and deploy order), `features` (one user-visible capability each), `entities` (one business entity each), `modules` (business-agnostic machinery) and `tooling` (the shared tsconfig base).
31
+
32
+ ```mermaid
33
+ flowchart TD
34
+ web["apps/web"]
35
+ ext["apps/extensions"]
36
+ org["organization<br/>@acme/organization"]
37
+ subgraph features["features"]
38
+ f1["intake"]
39
+ f2["lead-conversion"]
40
+ f3["lead-qualification"]
41
+ end
42
+ subgraph entities["entities"]
43
+ e1["lead"]
44
+ e2["opportunity"]
45
+ end
46
+ subgraph modules["modules"]
47
+ m1["identity"]
48
+ m2["app-api"]
49
+ m3["ui"]
50
+ end
51
+ tooling["tooling<br/>config"]
52
+ web --> org
53
+ ext -->|"passes packageComponents to declarePackage"| org
54
+ org -->|"composes CI declarations in deploy order"| features
55
+ features --> entities
56
+ entities --> modules
57
+ modules --> tooling
58
+ f1 -.-|"never imports"| f2
59
+ e1 -.-|"never imports"| e2
60
+ ```
61
+
62
+ Imports point down the tiers and may skip a tier. Inside one tier, only modules and tooling packages import each other. A feature never imports another feature and an entity never imports another entity. When two of them must work together, a page in `apps/web` connects them.
63
+
64
+ The arrows are real configuration. The root `turbo.json` lists, for each tag, the tags it may depend on, and `bunx turbo boundaries` fails the build on a wrong import:
65
+
66
+ ```json
67
+ "tier:features": { "dependencies": { "allow": ["tier:entities", "tier:modules", "tier:tooling"] } },
68
+ "tier:entities": { "dependencies": { "allow": ["tier:modules", "tier:tooling"] } },
69
+ "tier:modules": { "dependencies": { "allow": ["tier:modules", "tier:tooling"] } }
70
+ ```
71
+
72
+ Workspaces import each other by package name and exported subpath, such as `@acme/lead/views`, and never by relative path.
73
+
74
+ ## Package subpaths
75
+
76
+ A feature or entity package exposes only the subpaths it needs. Each subpath is a folder under `src/` with one `index.ts` and one `exports` entry in `package.json`.
77
+
78
+ | Subpath | Holds | Imported by |
79
+ | ---------- | ----------------------------------------------------------------------------------------------------------- | --------------------------------------------- |
80
+ | `domain` | Effect Schema models, resource contracts, schema FQNs, pure rules | every other subpath and other packages |
81
+ | `ci` | CI declarations (extensions SDK) as factories over organization handles | `@acme/organization` |
82
+ | `api` | the package `HttpApi` under `/api/app`, its groups and tagged errors | `server`, `headless`, the app API composition |
83
+ | `server` | the route Layer of the package `HttpApi` and its services | `apps/web/src/server` only |
84
+ | `headless` | typed client, query options, XState machines, entity bindings, Realtime topics and hooks, simulated sources | `views` and the app |
85
+ | `views` | wired React views on `@acme/ui`; feature parts enter through slots | the app |
86
+
87
+ Inside a package the subpaths also have a direction. `domain` imports no other subpath, `ci` imports neither `headless` nor `views`, and `headless` and `views` never import `server`. That keeps server code out of the browser bundle and keeps CI declarations free of React.
88
+
89
+ Each template package exposes a different set:
90
+
91
+ | Package | Subpaths |
92
+ | -------------------------- | ------------------------------------- |
93
+ | `@acme/lead` | `domain ci api server headless views` |
94
+ | `@acme/opportunity` | `domain ci headless views` |
95
+ | `@acme/intake` | `domain api server headless views` |
96
+ | `@acme/lead-conversion` | `ci headless views` |
97
+ | `@acme/lead-qualification` | `ci` |
98
+
99
+ `@acme/lead-qualification` adds an agent, a Skill, a tool and a Studio widget, and all of that is CI-side behavior. Its single `./ci` subpath shows that you can ship an AI capability without writing any web code.
100
+
101
+ ## Features take their audience as a parameter
102
+
103
+ A feature package must not import `@acme/organization`, because organization sits above it in the tiers. A feature that offers something to a group of users therefore takes that `audience`, and the application handle, as parameters. The organization package owns the groups and passes them in, then lists every component in deploy order, where each component follows the components it references:
104
+
105
+ ```ts
106
+ export const packageComponents = [
107
+ crmDomain,
108
+ ...leadComponents({ domain: crmDomain, version: VERSION }),
109
+ ...opportunityComponents({ domain: crmDomain, version: VERSION }),
110
+ salesGroup,
111
+ acmeApplication,
112
+ crmCapabilities,
113
+ ...crmPolicies,
114
+ repProfile,
115
+ managerProfile,
116
+ ...salesGrants,
117
+ applicationTokens,
118
+ ...convertLeadComponents({ application: acmeApplication, version: VERSION }),
119
+ ...leadQualificationComponents({
120
+ application: acmeApplication,
121
+ audience: salesMembers,
122
+ version: VERSION
123
+ }),
124
+ salesDeskApp
125
+ ]
126
+ ```
127
+
128
+ Because the organization package owns the content, `apps/extensions/src/index.ts` only declares the package:
129
+
130
+ ```ts
131
+ export default declarePackage({
132
+ components: packageComponents,
133
+ hostedContributions: [/* the Lead score card Studio widget */],
134
+ manifest: packageManifest
135
+ })
136
+ ```
137
+
138
+ [Extensions](../develop/extensions.md) describes each kind of declaration, and [Access](../platform/access.md) explains the Access Profiles and Access Grants in that list.
139
+
140
+ ## The web app layers
141
+
142
+ Entity and feature code lives in packages. The web app keeps only routes, Composition, the sign-in gate, Data Sources and server instances, arranged in a fixed set of layers where each layer imports only the layers below it:
143
+
144
+ | Layer in `apps/web/src` | Holds |
145
+ | ----------------------- | --------------------------------------------------------------------------------------------------------------------- |
146
+ | `app/` | route files only; each re-exports a Composition export, and `app/api/**` forwards to `server/<domain>/server` |
147
+ | `composition/` | `providers/` (global providers, Realtime providers) and `routing/` (access, shells, pages) |
148
+ | `features/` | app-local capabilities: their Data Sources (`remote/server.ts`) and `page-link/` |
149
+ | `entities/` | app-local entity machinery that several features need (the CI session), Data Sources of entity packages, `page-link/` |
150
+ | `server/` | beside the layers: one folder per domain, where `server.ts` (`server-only`) holds the process instance |
151
+
152
+ Only Composition imports a Data Source, and only Data Sources and `app/api/**` import a `server/<domain>/server.ts`. Peers still meet without importing each other: the CRM page template passes `ConvertLead` into the `selectionActions` slot of `LeadWorkspace` and `LeadAttachment` into its `editorActions` slot. Each page's owner gives its `{ label, path }` in `page-link/`, and shells and links read it, so no route path is written twice.
153
+
154
+ One Next app serves two server surfaces:
155
+
156
+ | Path | Handler | What it is |
157
+ | ------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------ |
158
+ | `/api/app/*` | `server/app-api/server` | the app API: the packages' Effect `HttpApi`s merged, OpenAPI at `/api/app/openapi.json`, Scalar docs at `/api/app/docs` |
159
+ | `/api/*` | `server/ci/server` | the CI SDK BFF: sign-in with a sealed cookie, Lead and Opportunity resource proxy, Realtime ticket and revalidate routes |
160
+ | `/api/leads/[id]/convert` | `server/ci/server` | the Fixed convert-lead workflow route |
161
+
162
+ The BFF holds the CI session in a sealed cookie and proxies resource calls, so the browser never holds a CI secret. [External applications](../develop/external-apps.md) covers the BFF in depth. The intake app API keeps its data in `IntakeStore.layerMemory`, and you replace that layer when a demo needs its own database.
163
+
164
+ ## Start a new demo
165
+
166
+ 1. Generate a repository with **Use this template**. In the new repository, delete the README "Contributing" section and `.github/workflows/close-pull-requests.yml`, which only matter in the template repository itself.
167
+ 2. Rename the template for the customer. Replace the `@acme/` package scopes and the `acme.app.` FQNs, replace `https://acme.localhost` in `packages/organization/acme/src/application.ts` and `.env.example`, and replace the theme tokens and the sidebar brand in `apps/web/src/composition/routing/shells/workspace/app-sidebar.tsx`. Theme tokens live once, in `packages/modules/ui/src/styles/globals.css`.
168
+ 3. Run it locally. The Overview and Lead intake pages work with no configuration, because the intake API keeps requests in memory:
169
+
170
+ ```bash
171
+ bun install
172
+ bun run dev:web # http://localhost:3000
173
+ ```
174
+
175
+ The CRM page needs a CI tenant.
176
+
177
+ 4. Deploy the package to an Organization. `ci deploy` runs the whole release path in one command, as [Extensions](../develop/extensions.md) describes. The long path runs each step by hand:
178
+
179
+ ```bash
180
+ ci package build --declaration ./apps/extensions/src/index.ts --out apps/extensions/dist/release
181
+ ci package validate --archive apps/extensions/dist/release/release.tar \
182
+ --manifest apps/extensions/dist/release/manifest.json \
183
+ --package-manifest apps/extensions/dist/release/package-manifest.json
184
+ ci package publish --backend-url <url> \
185
+ --archive apps/extensions/dist/release/release.tar \
186
+ --manifest apps/extensions/dist/release/manifest.json \
187
+ --package-manifest apps/extensions/dist/release/package-manifest.json \
188
+ --idempotency-key <key>
189
+ ci package promote --backend-url <url> --release <release-id> --idempotency-key <key>
190
+ ci package install --backend-url <url> --organization <organization-id> \
191
+ --package acme.app.knowledge --release <release-id> --idempotency-key <key>
192
+ ```
193
+
194
+ Running `package build` into a scratch folder and then `package validate` is the dry run for any change to the package.
195
+
196
+ 5. Rotate the application secret. The package installs a confidential application without a secret, so an Organization administrator rotates one:
197
+
198
+ ```bash
199
+ curl -sS -X POST \
200
+ -H "authorization: Bearer <administrator-token>" \
201
+ -H "x-organization-id: <organization-id>" \
202
+ <url>/admin/organization/external-apps/<application-id>/credential/rotate
203
+ ```
204
+
205
+ Copy `apps/web/.env.example` to `apps/web/.env.local`. Store `clientSecret` as `CI_CLIENT_SECRET`, `application.clientId` as `CI_CLIENT_ID` and the application id as `CI_APPLICATION_ID`. Then set `APP_ORIGIN`, `CI_TENANT_URL`, `CI_ISSUER`, `CI_AUDIENCE`, `CI_ORGANIZATION_ID`, `SESSION_SECRETS` and `NEXT_PUBLIC_APPLICATION_ID`, which equals `CI_APPLICATION_ID` and is the only value the browser sees. `ci apps enroll --env-file apps/web/.env.local` writes the enrollment values for you. Management tokens never go to Next.
206
+
207
+ 6. Add users to the Sales group. CI accepts a user's token only when the user holds the application gate, so nobody can sign in yet. An administrator adds each user to the Sales group (Sales rep profile) or to its `managers` subgroup (Sales manager profile).
208
+ 7. Sign in over HTTPS. The sealed session cookie requires HTTPS, so serve the dev server through a local HTTPS proxy:
209
+
210
+ ```bash
211
+ portless acme bun run dev:web # https://acme.localhost
212
+ ```
213
+
214
+ > **Warning:** The `./` in `--declaration ./apps/extensions/src/index.ts` is required.
215
+ > Without it, the CLI resolves the path like a module name and fails.
216
+
217
+ > **Note:** After each npm release of the CI packages, the template gets the released SDK
218
+ > versions pinned and one sync commit. A repository you generated earlier does
219
+ > not receive these updates by itself.
220
+
221
+ ## What to change and what to keep
222
+
223
+ A new demo changes the organization package, the entities, the features, the theme and the origins. It keeps `modules/identity`, `modules/app-api`, the BFF and server wiring, the tier tags, the lint configuration and the harness files, because they hold no business logic. A new environment variable goes into `apps/web/.env.example`, `packages/modules/identity/src/server/env.ts`, `turbo.json` `globalEnv` and the README tenant section together.
224
+
225
+ ## Checks that keep the structure honest
226
+
227
+ `@collective-intelligence/lint` is one Oxlint plugin named `ci-solution`. Extending `recommended` turns every rule on as an error:
228
+
229
+ | Rule | Checks |
230
+ | -------------------------------- | ---------------------------------------------------------------------------------- |
231
+ | `ci-solution/package-entrypoint` | an import of another package targets an entrypoint in its `package.json` `exports` |
232
+ | `ci-solution/subpath-imports` | imports between subpath folders follow the allowed direction |
233
+ | `ci-solution/subpath-entrypoint` | an import of another subpath folder targets its index file |
234
+ | `ci-solution/subpath-index` | every subpath folder has `index.ts` or `index.tsx` |
235
+ | `ci-solution/effect-subpath` | SDK Effect APIs come from the SDK `/effect` subpath |
236
+ | `ci-solution/no-run-in-view` | a view module calls no `fetch` and runs no Effect |
237
+ | `ci-solution/effect-actors` | XState side-effect actors are `fromEffect` actors |
238
+ | `ci-solution/effect-writes` | no module uses TanStack Query `useMutation`; writes run in `fromEffect` actors |
239
+
240
+ Each package has an `oxlint.local.ts` that extends the preset:
241
+
242
+ ```ts
243
+ import { recommended as ciSolution } from "@collective-intelligence/lint"
244
+ import { defineConfig } from "oxlint"
245
+
246
+ export default defineConfig({
247
+ extends: [ciSolution],
248
+ overrides: [
249
+ {
250
+ files: ["**/*.proof.ts"],
251
+ rules: { "eslint/curly": "off", "jsdoc/check-tag-names": "off" }
252
+ }
253
+ ]
254
+ })
255
+ ```
256
+
257
+ The web app also runs a `frontend-layers` lint for layer order, domain entrypoints, folder names and the `no-peer-feature` and `no-peer-entity` rules. For coding agents, the template ships Codebase Rules in `.agents/rules/`, such as `web-layer-imports`, `ci-sdk-boundaries` and `ci-realtime-topics`, each with a path trigger and a task trigger. The day-to-day checks are:
258
+
259
+ ```bash
260
+ bun run check-types
261
+ bun run test
262
+ bunx turbo boundaries
263
+ bun run lint
264
+ bun run format:check
265
+ bun run build
266
+ ```
267
+
268
+ When you change the Lead cache, a Lead write or Lead Realtime, also run `bun run check:models` in `packages/entities/lead`. [State machines](state-machines.md) explains what that command checks.
269
+
270
+ ## Related pages
271
+
272
+ - [State machines](state-machines.md)
273
+ - [SDKs](../develop/sdks.md)
274
+ - [Extensions](../develop/extensions.md)
275
+ - [External applications](../develop/external-apps.md)
276
+ - [Access](../platform/access.md)
@@ -0,0 +1,294 @@
1
+ # State machines
2
+
3
+ How next-effect-app models UI flows as XState machines with Effect actors, keeps server data in the QueryCache, resends uncertain writes safely, and checks hard flows with Models.
4
+
5
+ In `next-effect-app`, a UI flow that carries business logic is an XState machine in the `headless/` folder of its package, at `headless/<flow>-machine.ts`. Flows that belong only to the web app live at `apps/web/src/{features,entities}/<domain>/<flow>-machine.ts`. The machine decides where the flow is: editing, reviewing, submitting, failed. Every side effect is an Effect program that the machine runs as an actor, and TanStack Query owns the remote data that those programs read and write.
6
+
7
+ The machine context holds flow state only and never an API response. Two views that show the same record therefore read one cache entry, and no machine keeps a copy that can go stale. Server Components never run machines or Effects; they await a Promise boundary, and machines exist only in client code.
8
+
9
+ ## Machine, task or hook
10
+
11
+ Most of the template does not use a hand-written machine. Pick the smallest tool that covers the flow:
12
+
13
+ | Pick | When | Template examples |
14
+ | -------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
15
+ | XState machine | The flow has steps the user moves through, a lifecycle, or error branches that go back to an earlier step. | Lead intake (edit, review, submit), the CI session (initialize, sign in, refresh, sign out) |
16
+ | `effectTask` | One button runs one Effect and shows one result. | Convert lead, attach a file, sign out |
17
+ | Plain hook | The SDK hooks already cover it, or nothing writes. | `useEntityList`, `useEntityRecord`, `useEntityUpdate`, `useLeadRealtime`, the Lead score card |
18
+
19
+ Plain navigation and synchronous state changes need no actor at all.
20
+
21
+ ## Side effects are Effect actors
22
+
23
+ Each side effect is a `fromEffect(program)` actor from `@collective-intelligence/external-apps-client-sdk-react/effect`, declared in `setup({ actors })`. The actor runs on the `runtime` that the machine gets as input, and a view gets that runtime with `useEffectRuntime()` from the `EffectRuntimeProvider` in `composition/providers`. The intake submit actor calls the app API and then invalidates the intake list, so every view that shows the list refetches:
24
+
25
+ ```ts
26
+ export const submitIntakeActor = (submit: SubmitIntake) =>
27
+ fromEffect(({ draft }: IntakeInput & Readonly<{ draft: IntakeDraft }>) =>
28
+ submit(draft).pipe(
29
+ Effect.tap(() =>
30
+ QueryCache.use((cache) =>
31
+ cache.invalidate(intakeRequestsQuery.queryKey)
32
+ )
33
+ )
34
+ )
35
+ )
36
+ ```
37
+
38
+ Remote data moves only through the `QueryCache` service of the runtime, with `fetch`, `invalidate`, `setData`, `update` and `remove`. Three lint rules enforce this. `ci-solution/effect-actors` requires `fromEffect` actors, and `ci-solution/effect-writes` forbids TanStack `useMutation` everywhere, including hooks and helpers. `ci-solution/no-run-in-view` forbids `fetch` and running an Effect in a view module.
39
+
40
+ ## A machine in the template
41
+
42
+ The intake machine has four steps and a terminal state:
43
+
44
+ - `editing` goes to `reviewing` on `intake.review`.
45
+ - `reviewing` goes back to `editing` on `intake.edit`, or to `submitting` on `intake.confirm`.
46
+ - `submitting` goes to `done` when the actor succeeds, or back to `reviewing` with an error message when it fails.
47
+ - `done` goes to `editing` on `intake.reset`.
48
+
49
+ The `submitting` state invokes the actor. Its `onError` receives the typed failure, so the machine branches on it with `Schema.is`. `IntakeDuplicate` keeps its `_tag` from the server `HttpApi` to the browser, which is why `isDuplicate` works on the client:
50
+
51
+ ```ts
52
+ submitting: {
53
+ invoke: {
54
+ input: ({ context }) => ({ draft: context.draft, runtime: context.runtime }),
55
+ onDone: {
56
+ actions: { params: ({ event }) => event.output, type: "submitted" },
57
+ target: "done",
58
+ },
59
+ onError: {
60
+ actions: assign({
61
+ error: ({ event }) =>
62
+ isDuplicate(event.error)
63
+ ? "This email already has an open intake request."
64
+ : "The request could not be sent. Try again.",
65
+ }),
66
+ target: "reviewing",
67
+ },
68
+ src: "submitIntake",
69
+ },
70
+ },
71
+ ```
72
+
73
+ Leaving the invoking state interrupts the Effect fiber. The machine has no UI of its own: `submitted` is a typed action, and the view provides the toast with `machine.provide({ actions })`. The view runs `useMachine(intakeMachine.provide(...), { input: { runtime } })` and renders by `snapshot.matches("editing")`, `matches("submitting")` and `matches("done")`.
74
+
75
+ The same `provide` call swaps the data source for a demo without a backend. A simulated source lives in `headless/simulated-*.ts`, built with `simulate("<pace>", effect)` from `@acme/app-api`, and only Composition or a test selects it. Tests use the same mechanism. A test drives the machine on a `ManagedRuntime` of the SDK `makeTestQueryCache` layer, never a hand-made `QueryCache` double, and asserts on cache entries or recorded calls:
76
+
77
+ ```ts
78
+ const cache = makeTestQueryCache({ seed: [[intakeRequestsQuery.queryKey, []]] })
79
+ const runtime = ManagedRuntime.make(cache.layer)
80
+ const actor = createActor(
81
+ intakeMachine.provide({
82
+ actors: { submitIntake: submitIntakeActor(submit) }
83
+ }),
84
+ { input: { runtime } }
85
+ ).start()
86
+ // ...
87
+ expect(cache.calls).toEqual([
88
+ { member: "invalidate", queryKey: intakeRequestsQuery.queryKey }
89
+ ])
90
+ ```
91
+
92
+ Under `bun test` the simulated pace is instant.
93
+
94
+ ## Single actions with effectTask
95
+
96
+ For a single action, the SDK `effectTask(run)` wraps one Effect in a small machine with fixed states:
97
+
98
+ - `idle` goes to `running` on `RUN` with an input.
99
+ - `running` goes to `done` on success, with the result in `context.output`, or to `failed` on failure, with the error in `context.error`.
100
+ - `running` goes to `failed` on `CANCEL`, with a `Cause.Interrupt` reason as the error.
101
+ - `done` and `failed` go to `idle` on `RESET`, or back to `running` on `RUN`.
102
+
103
+ A `RUN` or `RESET` that arrives while the task is running is ignored. The convert-lead button is an `effectTask`. Its Effect starts the workflow through the BFF, follows the run to its end, and then marks every CI entity read stale:
104
+
105
+ ```ts
106
+ export const convertLead =
107
+ (start: StartConvert) =>
108
+ ({ followRun, leadId }: ConvertLeadRequest) =>
109
+ Effect.tryPromise({
110
+ catch: () => new ConvertLeadFailed({ step: "start" }),
111
+ try: async () => await start(leadId)
112
+ }).pipe(
113
+ Effect.flatMap(({ runId }) =>
114
+ Effect.tryPromise({
115
+ catch: () => new ConvertLeadFailed({ step: "follow" }),
116
+ try: async () => {
117
+ await followRun(runId)
118
+ }
119
+ })
120
+ ),
121
+ Effect.andThen(
122
+ QueryCache.use((cache) => cache.invalidate(entityReadsKey))
123
+ )
124
+ )
125
+
126
+ export const convertLeadTask = effectTask(convertLead(startConvertLive))
127
+ ```
128
+
129
+ The failure records which step failed, and `convertFailureMessage(snapshot)` maps it to a message for the view. The view calls `useMachine(convertLeadTask, { input: { runtime: useEffectRuntime() } })`, sends `{ type: "RUN", input: { followRun, leadId } }` from the button, and disables the button while `state.matches("running")`. On the server, the BFF route checks `convertLeadRequirement` on every call and starts the run with a fixed idempotency key per Lead:
130
+
131
+ ```ts
132
+ idempotencyKey: ({ leadId }) => `convert-lead:${leadId}`,
133
+ ```
134
+
135
+ ## Uncertain writes and idempotency keys
136
+
137
+ A write is uncertain when the request left the browser and no answer came back. CI may or may not have applied it, and only a resend of the same action settles the question. The Lead attachment flow shows how the template handles this. `attachLeadFile` is an `effectTask` whose Effect checks the file's size and type locally, asks CI for an upload slot, PUTs the bytes straight to storage (never through the BFF), writes `{ attachment: { upload } }` at the Lead's revision with an idempotency key, and then invalidates the entity reads.
138
+
139
+ ```mermaid
140
+ stateDiagram-v2
141
+ [*] --> idle
142
+ idle --> running: RUN with a new file
143
+ state running {
144
+ [*] --> checkFile
145
+ checkFile: Check size and type locally
146
+ askSlot: Ask CI for an upload slot
147
+ putBytes: PUT bytes to storage
148
+ writeLead: Write the Lead with an idempotency key
149
+ invalidateReads: Invalidate entity reads
150
+ checkFile --> askSlot
151
+ askSlot --> putBytes
152
+ putBytes --> writeLead
153
+ writeLead --> invalidateReads
154
+ }
155
+ running --> done: CI confirms the write
156
+ running --> failed: too large, wrong type, storage, conflict, refused
157
+ running --> failed: no answer, reason uncertain, write kept
158
+ failed --> running: Resubmit with the kept write, same key
159
+ failed --> running: RUN with a new file
160
+ done --> running: RUN with a new file
161
+ ```
162
+
163
+ The steps inside `running` are steps of the Effect program, and the machine itself sees one `running` state. A resubmit skips the upload and sends only the final write. The size and type checks use `leadAttachmentAccepts` and `leadAttachmentMaxBytes` (5 MiB) from `@acme/lead/domain`. The CI schema declares the same values on the File field, so CI refuses a larger upload too, while a file that fails the local check sends nothing.
164
+
165
+ The write that might need a resend is a value the failure carries:
166
+
167
+ ```ts
168
+ export const PendingAttachment = Schema.Struct({
169
+ expectedRevision: Schema.String,
170
+ idempotencyKey: Schema.String,
171
+ upload: Schema.String
172
+ })
173
+ ```
174
+
175
+ When the write ends with a rejected promise or an SDK `uncertain` outcome, the task fails with reason `uncertain` and keeps that `PendingAttachment`. Sent again with the same idempotency key, CI applies the write at most once and returns its first answer. `pendingAttachment(snapshot)` reads the kept write from a failed task, and the `LeadAttachment` view shows a "Resubmit attachment" button only when it is not null. The button sends `RUN` with `{ pending }` in place of a new file, which reuses the same key and uploads nothing again. Every failure reason maps to one message, and the uncertain one reads "No answer came from CI, so the file may be attached. Resubmit to finish."
176
+
177
+ > **Warning:** A retry after an uncertain write must resend the same action with the same
178
+ > idempotency key. A new key makes it a new action, and CI could apply the
179
+ > change twice.
180
+
181
+ The SDK entity hooks follow the same rule. Their writes report `conflict`, `rejected` and `uncertain` in `status`; `submit` creates a new action with one operation key, and `resubmit` sends the previous action exactly as it was. Queries and mutations never retry by themselves. While a write is uncertain, `EntityForm` shows every field read-only, so Resubmit sends the draft that the form shows. [SDKs](../develop/sdks.md) lists these hooks.
182
+
183
+ ## Realtime refreshes the cache
184
+
185
+ Realtime reaches the views through the cache. It never sends events into a machine. `useLeadRealtime` subscribes to the Lead topic, and each batch of changes refreshes the SDK entity reads and marks the app API read of each changed id stale:
186
+
187
+ ```ts
188
+ export const leadsTopic: RealtimeTopic = { schema: leadSchemaFqn }
189
+
190
+ export const useLeadRealtime = () => {
191
+ const queryClient = useQueryClient()
192
+ const refreshEntityReads = useEntityChanges(leadContract)
193
+ useRealtimeTopic(leadsTopic, (changes) => {
194
+ refreshEntityReads(changes)
195
+ void refreshLeadReads(queryClient, changes)
196
+ })
197
+ }
198
+ ```
199
+
200
+ Changes carry ids only, and the reads reload. Every write in the template ends the same way, with the cache:
201
+
202
+ | Writer | What it refreshes |
203
+ | -------------------------------------------------------------- | ------------------------------------------------------------------------ |
204
+ | Intake actor | invalidates the intake list query |
205
+ | Convert and attach actors | invalidate `entityReadsKey` |
206
+ | SDK entity hooks | store the confirmed record and reload lists |
207
+ | A write the BFF confirms | expires the Next cache tags of the record |
208
+ | A write outside the BFF (workflow, Studio, agent, another app) | reaches the Realtime Revalidate Route, which expires the Next cache tags |
209
+
210
+ Invalidation is enough for almost every flow, but one case can go wrong. A Realtime change can arrive while this tab's optimistic write is still pending, and a naive refetch would then show the old server value and hide the user's edit. The SDK entity read prevents this by waiting for the writes of its key to settle (`writesSettled`) before it fetches. The template checks that interleaving with a Model.
211
+
212
+ ## Models
213
+
214
+ A Model is a `*.proof.ts` TypeScript module that states the state, actions and invariants of one flow. `tsprover` compiles it to TLA+, and the `tla-ts` checker visits every reachable state. When a rule breaks, the checker prints a trace: the sequence of actions, with the state after each one. An XState machine describes what one user sees step by step in one tab, while a Model describes what can happen when several writers interleave, and the template uses each for its own question.
215
+
216
+ | Construct | Meaning |
217
+ | ---------------------------------------- | ------------------------------------------------------ |
218
+ | top-level `let` | a state variable |
219
+ | exported function that changes state | an action |
220
+ | exported function that returns a boolean | an invariant |
221
+ | guard `if (<condition>) return;` | a precondition |
222
+ | `/** @fair */` | this action must eventually run |
223
+ | `/** @leadsTo target */` | a liveness property |
224
+ | `/** @constraint */` | a bound, so the search ends |
225
+ | `ghost` prefix on a `let` | records what a consumer saw, only to state a guarantee |
226
+
227
+ The Lead Realtime Model in `packages/entities/lead` is trimmed here. The full file also has `writeConfirm`, `otherWrite`, `realtimeChange` and `neverBehindConfirmed`:
228
+
229
+ ```ts
230
+ let server: Revision = 0
231
+ let cached: Revision = 0
232
+ let writePending = false
233
+ let refetchDue = false
234
+ let ghostConfirmed: Revision = 0
235
+ let ghostPredicted: Revision = 0
236
+
237
+ export const writeStart = () => {
238
+ if (writePending) return
239
+ if (server === 2) return
240
+ writePending = true
241
+ ghostPredicted = server === 0 ? 1 : 2
242
+ cached = ghostPredicted
243
+ }
244
+
245
+ /** @fair */
246
+ export const refetch = () => {
247
+ if (!refetchDue) return
248
+ if (writePending) return
249
+ cached = server
250
+ refetchDue = false
251
+ }
252
+
253
+ export const pendingWriteShown = () =>
254
+ !writePending || cached === ghostPredicted
255
+
256
+ /** @leadsTo isFresh */
257
+ export const isRefetchDue = () => refetchDue && !isFresh()
258
+ ```
259
+
260
+ The checker visits 22 states and proves two things. A Realtime refetch never hides the tab's pending optimistic write (`pendingWriteShown`, `neverBehindConfirmed`), and after a Realtime change the views eventually reach the stored revision (`@leadsTo isFresh`). The package runs it with one script:
261
+
262
+ ```bash
263
+ cd packages/entities/lead
264
+ bun run check:models
265
+ ```
266
+
267
+ With no file arguments, `tsprover` checks every `*.proof.ts` below the package and exits with code 1 on any failure. Run it when you change the Lead cache, a Lead write or Lead Realtime.
268
+
269
+ > **Note:** Pin `tsprover` exactly, as the template does with `"tsprover": "0.0.2"`. It
270
+ > and its checker are 0.0.x releases, so any release can change the Model
271
+ > language or the checker.
272
+
273
+ ### When a flow deserves a Model
274
+
275
+ Most demo flows need no Model. Add a `*.proof.ts` only when the flow has one of these:
276
+
277
+ - two or more writers on shared state
278
+ - a background job or retry that touches client-written state
279
+ - recovery or resync
280
+ - a delivery guarantee such as "eventually delivered"
281
+
282
+ In the template, only the optimistic write against a Realtime refetch met that bar. Write one Model per service: the state the service owns becomes a top-level `let`, and each dependency stays outside and enters only as an action. The SDKs ship their own Models for the optimistic write coordinator, the session, the Realtime machine, change-log resync and topic grant tokens, so a demo inherits those guarantees and models only its own flows.
283
+
284
+ ### Sensitivity Proofs
285
+
286
+ A Model that passes may simply never reach the dangerous case. A Sensitivity Proof shows that it does. You break the Model on purpose with one named Mutant, by removing one guard for an invariant or removing `@fair` from one action for a `@leadsTo` property. The checker must report a violation on the Mutant and pass again after you revert it. If the Mutant passes with the same state count, the guarded case is unreachable, and you widen the bounds or add an actor before you prove again.
287
+
288
+ A worked example: a client save and a background hash job both write `hashed`. Without the guard `if (job !== content) return;`, the invariant `hashMatchesContent` fails on the trace `save -> save -> hashJob`, because a stale job writes the hash of old content. With the guard, 11 distinct states pass. Both guards of the Lead Realtime Model have Sensitivity Proofs.
289
+
290
+ ## Related pages
291
+
292
+ - [The solution template](index.md)
293
+ - [SDKs](../develop/sdks.md)
294
+ - [External applications](../develop/external-apps.md)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@collective-intelligence/extensions-sdk-backend",
3
- "version": "0.2.6",
3
+ "version": "0.2.8",
4
4
  "description": "Framework-neutral executable contracts for Collective Intelligence extension packages, with an Effect adapter at ./effect.",
5
5
  "type": "module",
6
6
  "sideEffects": false,
@@ -44,7 +44,7 @@
44
44
  "access": "public"
45
45
  },
46
46
  "dependencies": {
47
- "@collective-intelligence/extensions-sdk": "0.2.5"
47
+ "@collective-intelligence/extensions-sdk": "0.2.7"
48
48
  },
49
49
  "peerDependencies": {
50
50
  "effect": "^4.1.0"
@@ -55,7 +55,7 @@
55
55
  "@types/bun": "1.4.2",
56
56
  "@typescript-eslint/utils": "8.22.0",
57
57
  "bun-types": "^1.4.2",
58
- "effect": "4.1.0",
58
+ "effect": "4.1.2",
59
59
  "eslint": "9.39.5",
60
60
  "oxfmt": "0.66.0",
61
61
  "oxlint": "1.90.0",
@@ -63,5 +63,5 @@
63
63
  "typescript": "7.0.2",
64
64
  "ultracite": "7.10.7"
65
65
  },
66
- "gitHead": "a1356612c41288b3d0e8e580f0964a975d58905d"
66
+ "gitHead": "b2a4e19aabe96ba0b9285a911f59d40d1d111a4d"
67
67
  }