@collective-intelligence/external-apps-server-sdk 0.0.0-stage → 0.1.0

package/README.md

@@ -1,3 +1,1076 @@
-# Temporary Holding Version
+# `@collective-intelligence/external-apps-server-sdk`
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Confidential server client and proxy boundary for Collective Intelligence
+external applications. Import administrative APIs only from `/admin`; import
+the optional Express runtime adapter from `/runtime/express`.
+
+The `/admin` client exposes knowledge connector administration through
+`knowledgeConnectors` (formerly `connectors`). It calls the backend's
+`/admin/extensions/knowledge-connector-*` resources. Upgrade callers and the
+backend together; the previous namespace and API routes have no compatibility
+aliases. Request and response shapes are unchanged.
+
+The customer-hosted knowledge gateway is intentionally split into isolated
+subpaths:
+
+- `/gateway` owns deterministic `defineGateway` manifests, explicit one-level
+  operation domains, proposal projection, and the Web `Request`/`Response`
+  policy kernel;
+- `/auth` verifies short-lived ES256 CI caller JWTs through tenant-bound JWKS,
+  including audience, installation, organization, capability binding, expiry,
+  and atomic `jti` replay consumption; and
+- `/resources` declares promise-based resource handlers over the portable
+  client resource contract.
+
+## Choose authentication by boundary
+
+The SDK has three separate authentication boundaries:
+
+- `createServerClient` uses confidential service credentials for requests from
+  the customer server to CI. Keep those credentials on the server and use the
+  only supported client mode, `service`. Its `tenantUrl` must be a
+  credential-free HTTPS origin; for local development it also accepts an
+  `http` origin whose host is exactly `localhost`, a `127.0.0.0/8` address, or
+  `[::1]`. Any other `http` host is rejected before credentials are sent.
+- Browser-facing application routes use either CI federation from
+  `/auth/federated` or local email/password sessions from `/auth/better-auth`.
+  Select one mode for a deployment and authorize those sessions at the
+  application route boundary.
+- Gateway routes accept short-lived ES256 caller tokens signed by CI. The
+  customer server verifies those tokens and their live connection grant through
+  `/auth`; browser cookies do not authorize Gateway calls.
+
+Administrative CI APIs use a separately supplied administrator bearer token
+through `/admin`. None of these credentials or grants belongs in a Gateway
+manifest.
+
+A confidential application created by package install has no `clientSecret`
+until an administrator with `public_applications:manage` rotates its credential
+(`POST /admin/organization/public-applications/{id}/credential/rotate`, or
+**Rotate secret** in the Dashboard). CI returns the secret once; store it
+server-side. See
+[Provision a secret for a package-created application](../../../docs/reference/package-release-cli.md#provision-a-secret-for-a-package-created-application).
+
+Gateway manifests contain only serializable declarations. Handlers,
+credentials, signing keys, resolved identities, and customer authorization
+callbacks stay in the runtime definition. Public principals are the closed
+`detached_user | service_principal` union. A delegated customer identity is
+retained separately as the stable external issuer/subject audit context and
+never changes caller authority.
+
+Gateway protocol version 2 exposes Knowledge resources as remote source
+metadata only. A resource definition describes the external application's own
+record shape and capabilities; it never declares a CI organization domain,
+entity schema, relation schema, schema FQN, or field mapping. Organization
+administrators select exposed resources and map them to organization-owned
+schemas when configuring the corresponding CI Knowledge connector.
+Push ingestion follows the same boundary: records contain source identity,
+revision, timestamps, deletion state, and the remote value only. They cannot
+declare organization relation edges or relation fields; CI derives those
+semantics through the organization-owned connector mapping.
+`createGatewayIngestionClient` from `/gateway` accepts the same `tenantUrl`
+values as `createServerClient`, including a loopback `http` origin for local
+development.
+
+Each `defineOperation` must fully materialize the policy CI reviews: portable
+input/output schemas, `effects`, `idempotency`, `retry`, `authorization`,
+`execution`, `limits`, and `version`. `retry` is either `{ mode: "never" }` or
+an `idempotent_only` policy with 2–10 total attempts. Execution is synchronous
+and declares a finite timeout. `maxAttempts` counts the initial call and every
+permitted retry. `limits` bounds the request body and serialized result.
+`defineGateway` rejects non-integer or unsafe bounds before hashing the
+manifest: timeouts are at most one hour and request and response payloads at
+most 16 MiB. Background execution is rejected at registration.
+
+The kernel validates an optional `x-idempotency-key` once and forwards the
+exact value to the operation handler. A `requires_key` operation is
+never called without one. Replay lookup and durable result storage belong to
+the handler and the customer's persistence layer; the framework-neutral kernel
+does not claim generic deduplication.
+
+`createGatewayKernel` serves the closed versioned discovery, manifest, health,
+resource and domain-qualified synchronous operation routes. It does not expose
+background execution handles, polling, cancellation or event-stream routes.
+The kernel applies each operation's request, timeout and result bounds; the
+gateway-global body limit remains an additional ceiling. An already-aborted
+request never starts an operation. A live abort signals the handler and returns
+a cancellation response; completed requests detach their abort listener. A
+handler must cooperate with its signal to stop its own work. Cancellation does
+not undo an already-committed mutation. Runtime adapters translate framework
+values to that kernel instead of reimplementing routing, authorization, or error
+policy.
+Provider and framework peers are optional and isolated to their declared
+`/knowledge-sources/*`, `/persistence/*`, `/runtime/*`, and `/tools/*` exports; importing the
+package root or the gateway/auth/resource subpaths does not load them.
+
+The configured `tenantUrl` must be a credential-free HTTPS origin, without a
+path, query, or fragment. Mount `createExternalApplicationBff` (or
+`createExternalApplicationBffNextHandlers`) once for federated auth, Resource
+proxy, and dedicated Capability forwarding. Pass `ciResources` to bind one
+Resource proxy mount per entity contract by `schemaFqn`; the BFF authorizes
+those mounts with `authorizeWithUserAuth`. Dedicated Capability forwarding sends
+the signed-in user's delegated token and the caller's `x-organization-id` by
+default. For an application without federated CI users, pass
+`capabilities: { authentication: { mode: "service", credentials },
+getAuditSubject, organizationId, tenantUrl }`: each call carries the
+application's Basic credentials, the configured organization, and the
+`x-external-audit-issuer` / `x-external-audit-subject` headers from
+`getAuditSubject(request)`. A request without an Audit subject receives `401`
+and is never forwarded. To bring anonymous visitors to a declared Agent, add
+`caller: "anonymous_visitor"`: each call also carries
+`x-external-caller-kind: anonymous_visitor` and never `x-external-ci-user-id`,
+so the backend runs only the Agent's own bindings and never the application's
+service authority. An Audit subject with `mappedUserId` is refused with `401`
+for a visitor caller. Forwarding covers `context`, `knowledge`, `agents`,
+`conversations` (continue and history), `skills`, `workflows`, and `runs/*`.
+Reconstruct the public origin from
+`x-forwarded-host` / `x-forwarded-proto`. Compose a Fixed workflow route with
+`createFixedWorkflowRoute`: it rewrites that public origin, authorizes
+`getEntityContext` plus `requiredScopes`, stamps `x-organization-id` from the
+session, POSTs `/application/v1/workflows/:id/runs`, and returns `{ runId }`.
+Configure exactly one server-owned target: `workflowId` for a workflow UUID or
+`workflowFqn` for its stable fully qualified name. The route rejects either
+field in browser input and encodes the configured target as one URL segment.
+Override `validate`, `idempotencyKey`, `authorize`, or `getAccessToken` when
+the application needs different policy. The deprecated `./express` entry
+re-exports only `createGatewayExpressMiddleware` from `./runtime/express`.
+
+## Reuse Nest operations and local AI tools
+
+Use Node 22+ with ESM for the AI SDK 7 adapter. The inspected and tested baseline on 6 October 2026
+is `ai@7.0.128`, `@ai-sdk/provider-utils@5.0.54`, and Nest `11.2.4`, with
+Zod `4.4.3` in workspace source checks and the Nest demo, and `4.6.5` in the
+installed AI consumer; recheck the stable AI patch when installing. Declared optional peer
+ranges are Nest `>=10 <12`, AI `>=7.0.128 <8`, provider-utils `>=5.0.54 <6`,
+and Zod `>=4 <5`. A peer range is not proof of every version combination.
+Install only the peers required by the subpaths you use. Nest applications also
+need their HTTP platform adapter, `reflect-metadata`, and `rxjs`.
+
+Workspace SDK, backend, and Nest demo dependencies align on `zod@4.4.3`. Bun
+resolves separate Drizzle and AI provider-utils installations for different peer
+versions; aligning Zod preserves their shared database and schema type identities.
+The installed AI consumer continues to verify `zod@4.6.5` in one peer context.
+
+- `/runtime/nestjs` exports `bindNestOperation`, `GatewayNestModule`, and
+  `createGatewayNestHandler`. It composes supplied application providers over
+  the existing Fetch kernel.
+- `/tools/ai-sdk` exports asynchronous `defineAiSdkOperation`. It imports a
+  supplied local tool; it does not generate tools or execute model generation.
+
+The package root and `/gateway` do not load these optional peers. Use public
+subpaths rather than private implementation imports. For runnable Nest DI,
+caller verification, manifests, and CI Connection/binding setup, follow the
+[Nest demo](../../../demo/external-app-nest/README.md).
+
+### Bind an existing Nest callable
+
+`bindNestOperation({ declaration, normalize, authorize, execute })` returns a
+Gateway Operation. All three callbacks are required. `declaration` contains the
+same metadata and portable input/output projections as `defineOperation`;
+`normalize` uses your native schema, `authorize` receives the normalized input
+and verified invocation context, and `execute` calls your existing injected
+service. The invocation also carries `signal` and an optional validated
+`idempotencyKey`. Bind instance methods through a closure to retain `this`.
+
+```ts
+import { bindNestOperation } from "@collective-intelligence/external-apps-server-sdk/runtime/nestjs"
+
+const operation = bindNestOperation({
+  declaration: greetingDeclaration,
+  normalize: ({ input }) => greetingInput.parse(input),
+  authorize: ({ input, context }) => permissions.canGreet(context, input),
+  execute: ({ input, signal }) => greetings.greet(input, { signal })
+})
+```
+
+`greetingDeclaration`, `greetingInput`, `permissions`, and `greetings` are
+application-owned dependencies; the demo supplies a concrete composition.
+The binding does not inherit controller guards, middleware, pipes, interceptors,
+or serializers. Reuse business policy explicitly. Connection admission is
+followed by application/resource authorization; an External Audit Subject does
+not supply an application user or access rights.
+
+### Reuse an existing AI SDK tool
+
+This complete tool binding demonstrates native defaults/transforms, a validated
+context factory, an explicit missing-output override, and business admission.
+The supplied permission service is the application integration point.
+
+```ts
+import { defineAiSdkOperation } from "@collective-intelligence/external-apps-server-sdk/tools/ai-sdk"
+import type { GatewayInvocationContext } from "@collective-intelligence/external-apps-server-sdk/auth"
+import { tool } from "ai"
+import { z } from "zod"
+
+export async function bindGreetingTool(permissions: {
+  canGreet(context: GatewayInvocationContext, name: string): Promise<boolean>
+}) {
+  const existing = tool({
+    description: "Greet an authorized name",
+    inputSchema: z.object({
+      name: z.string().transform((value) => value.trim()),
+      prefix: z.string().default("Hello")
+    }),
+    contextSchema: z.object({ organizationId: z.string().min(1) }),
+    execute: async ({ name, prefix }, options) => ({
+      greeting: `${prefix} ${name}`,
+      organizationId: options.context.organizationId
+    })
+  })
+  return await defineAiSdkOperation({
+    id: "greet",
+    title: "Greet",
+    version: "1.0.0",
+    tool: existing,
+    effects: "read",
+    idempotency: "safe",
+    retry: { mode: "never" },
+    authorization: {
+      organization: "required",
+      delegatedSubject: "allowed",
+      principalKinds: ["service_principal"],
+      scopes: ["greetings:read"]
+    },
+    execution: { mode: "synchronous", timeoutMs: 1000 },
+    limits: { maxRequestBytes: 1024, maxResponseBytes: 1024 },
+    authorize: ({ input, context }) =>
+      permissions.canGreet(context, input.name),
+    context: ({ context }) => ({ organizationId: context.organizationId }),
+    output: {
+      dialect: "https://json-schema.org/draft/2020-12/schema",
+      fidelity: "exact",
+      document: {
+        type: "object",
+        additionalProperties: false,
+        properties: {
+          greeting: { type: "string" },
+          organizationId: { type: "string" }
+        },
+        required: ["greeting", "organizationId"]
+      }
+    }
+  })
+}
+```
+
+Await the operation before putting it in `defineOperationDomain` and
+`defineGateway`, then mount `createGatewayKernel({ gateway, verifyCaller })`
+through `GatewayNestModule.registerAsync({ imports, inject, useFactory })`.
+Keep handlers, runtime context, secrets, and DI instances out of the manifest.
+Configure the trusted CI issuer, audience, atomic replay store, live Connection
+grant check, and accepted operation binding before invoking it. The demo
+explains those prerequisites and the existing administrative setup seams.
+
+Admission requires a local `execute` callback and one bounded JSON result.
+Provider/client-only tools, streaming/preliminary results, generation lifecycle
+callbacks (`onInputStart`, `onInputDelta`, `onInputAvailable`), approval needs,
+and `toModelOutput` are unsupported. Registration rejects unsupported tool
+features; a streamed/non-JSON execution result produces a bounded capability
+diagnostic. A context-dependent description needs a fixed `description`
+override. Missing input/output projections prevent publication.
+
+The adapter projects supported Zod 4, Standard JSON Schema, and AI JSON Schema
+inputs/outputs into Draft2020-12. Native input parsing still supplies defaults
+and transforms before authorization and execution. `input` and `output`
+overrides are portable `PackageSchemaProjection` values for schema gaps;
+input must satisfy `ai_function_input`, output must satisfy `runtime_json`.
+Nested `true`/`false` subschemas are preserved under supported keywords;
+projection roots remain objects, and AI-input profile restrictions and native
+validation still apply.
+Do not overstate refinements or transforms that JSON Schema cannot encode.
+Overrides retain native input validation. Output is checked against its
+published projection and byte limit, not passed through a native output
+transform. Unknown/unregistered format constraints and unsupported conversions
+require a supported explicit projection; do not silence them with an empty
+schema. See [portable schemas](../extensions-sdk/README.md).
+
+A tool with context needs an application-owned `context(request)` factory and
+native validator, from `tool.contextSchema` or the adapter's `contextSchema`
+option. Construct context from the verified caller and authorized resources.
+The factory result is validated before execution; invalid context never
+executes the tool. The adapter supplies `toolCallId` from CI's correlation ID,
+`abortSignal` from the invocation, and `messages: []`. It cannot reconstruct
+original model messages or approval authority. Cancellation is cooperative;
+ignored signals can leave work running and cannot roll back writes. Arbitrary
+exceptions remain private; use `GatewayOperationError` from `/gateway` only
+for deliberately safe caller-facing failures.
+
+For multi-step CV work, use application-owned start/status bindings rather
+than waiting for a workflow in this bounded tool adapter. The
+[CV integration guide](../../../demo/external-app-nest/docs/cv-integration.md)
+shows distinct receipts and final results, independent resource authorization,
+and application-owned execution, recovery, failure, and duplicate protection.
+
+## Agent runs and conversations
+
+`agents.invoke({ agentId, input })` starts one Agent run and returns
+`{ runId }`. Read it with `runs.get` (a `ServerRun`; a succeeded Agent run's
+`result` is a `ServerAgentRunResult`, `{ message, output? }`) or page its events
+with `runs.stream`. `message` is the Agent's normal final answer. `output` is the
+structured Agent Output the run saved, already valid against the Agent's
+declared output schema; it is absent when the run saved none, such as a
+clarifying question. Reading a run again returns the same saved result.
+
+For a multi-turn chat, use `conversations` instead. The first message starts a
+conversation; each later message continues it:
+
+```ts
+const ci = runAsApplication({
+  auditSubject: { issuer: "apito.it", subjectId: memberId },
+  credentials,
+  tenantUrl
+})
+
+const first = await ci.conversations.continue({
+  agentId,
+  input: "Find suppliers of recycled packaging",
+  organizationId
+})
+// first.value.result: { conversationId, runId }
+
+const next = await ci.conversations.continue({
+  conversationId,
+  input: "Tell me more about the second one",
+  organizationId
+})
+
+const history = await ci.conversations.history({
+  conversationId,
+  organizationId
+})
+// history.value.result: { agentId, conversationId, createdAt, turns }
+```
+
+Each turn is an ordinary Agent run: follow it with `runs.get` / `runs.stream`
+using its `runId`. The run of a continued turn receives the earlier turns as
+the model's own conversation history: each earlier message and answer, plus
+any Agent Output that turn saved, before the new message. Only turns that
+succeeded before the new turn started count. A turn is
+`{ runId, status, input, message, output?, createdAt, completedAt }`:
+`message` is the Agent's final answer once the run succeeded, otherwise
+`null`; `output` is present when the run saved an Agent Output.
+
+- A conversation belongs to the application, organization and Audit subject
+  that started it. Another subject (or a call without that subject) gets
+  `not_found` for both `continue` and `history`.
+- Wait until the previous turn finished: continuing while it still runs returns
+  `conflict`.
+- `idempotencyKey` (at most 128 characters; generated when omitted) makes a
+  retried message replay the same turn instead of adding one. Use each key for
+  one message only; reusing it for a different message or conversation returns
+  `conflict`.
+- The application's Agent capability gates conversations, with the `ai:invoke`
+  scope.
+- The Agent's current audience gates every turn and every `history` read. When
+  an administrator removes the caller from the audience, the same caller gets
+  `not_found` for its own conversations too.
+- For an anonymous visitor, pass `callerKind: "anonymous_visitor"` with the
+  visitor's `auditSubject` (no `mappedUserId`). Only a declared Agent with
+  `audience: { anonymous: true }` that lists this application admits it. The
+  visitor's turns run with the Agent's own bindings under an anonymous visitor
+  subject, never with the application's scopes. Visitors are separate owners:
+  one visitor never reads another visitor's conversation.
+
+## Gateway registry and orchestration
+
+Build one registry with `defineGateway`. Its `knowledge`, `datasets`, and
+`operationDomains` arrays are the complete allowlist projected into discovery.
+`defineGateway` snapshots those declarations, derives the deterministic
+manifest and resolves the executable binding for each incoming route.
+
+Pass that registry and one machine caller verifier to `createGatewayKernel`.
+The kernel owns discovery, connection enrollment, authentication,
+authorization, bounded JSON handling and dispatch. A runtime adapter then lifts
+the same Fetch `Request`/`Response` seam into the host framework. Application
+routes can reuse the same CRUD resource with their own user-session policy.
+
+```ts
+const gateway = defineGateway({
+  id: "crm",
+  version: "1.0.0",
+  knowledge: [{ resource: leadResource, access: "read_write" }],
+  datasets: [leadPipeline],
+  operationDomains: [crmOperations]
+})
+
+const kernel = createGatewayKernel({
+  gateway,
+  verifyCaller: connectionAuth.verifyCaller,
+  verifySetupCredential: connectionAuth.verifySetupCredential,
+  enrollConnection: connectionAuth.enrollConnection
+})
+```
+
+This package exposes Knowledge, Dataset, and declared operation tools through
+the Gateway protocol. An MCP server or MCP transport is outside this SDK.
+
+### Enroll and verify connections
+
+`createGatewayConnectionAuthentication` composes a durable connection-grant
+store, a replay store, trusted CI issuers, and a server-only setup credential.
+Wire its three returned functions into the kernel as shown above.
+
+```ts
+const connectionAuth = createGatewayConnectionAuthentication({
+  audience: "https://crm.example/gateway",
+  trustedIssuers: [
+    { issuer: "https://ci.example", tenantUrl: "https://ci.example" }
+  ],
+  grants: connectionGrantStore,
+  replayStore,
+  setupCredentials: loadActiveSetupCredentials
+})
+```
+
+CI starts enrollment with a signed discovery token and the configured setup
+credential. The verifier validates the token signature, issuer, audience,
+connection, organization, capability binding, scopes, expiry and unique `jti`,
+then confirms the connection with CI before persisting the grant. Normal calls
+must match an enrolled grant and repeat that live validation. Store grants and
+replay identifiers durably and atomically when the application runs more than
+one process. `createInMemoryReplayStore` is suitable only for a single-process
+fixture. The SDK verifies CI signatures; it does not issue caller tokens.
+
+### Package-managed enrollment
+
+For a declared Gateway, configure `/auth`'s `createGatewayPackageEnrollment`
+with the exact audience, explicitly trusted CI issuers, a local `allowedPackages`
+list of `{ organizationId, packageFqn, gatewayFqn }` tuples, and an atomic
+`GatewayPackageEnrollmentStore`. Pass its `enrollPackageConnection` callback to
+`createGatewayKernel` alongside the normal invocation verifier. The kernel
+supplies its runtime manifest hash to the callback; do not substitute a hash
+from request input.
+
+`POST /v1/connections/package-enroll` accepts a narrow signed Bearer token and
+returns `{ enrolled: true, manifest }`. This route requires no setup credential.
+The token has purpose `package_gateway_enrollment`, binding `connection:enroll`,
+scope `gateway:enroll`, an exact audience, an issued-at time and a lifetime of at
+most 300 seconds. Its `org_id`, `package_installation_id`, `package_fqn`,
+`gateway_fqn`, `installation_id` (the Connection ID), `manifest_hash` and `jti`
+are required. Configured keys, local package policy and the complete runtime
+manifest hash must agree. The verifier also checks live CI enrollment authority
+before storage, with bounded requests and no redirects. Enrollment tokens never
+authorize operation invocation.
+
+`makeGatewayPackageEnrollmentStore` from `/persistence/drizzle` commits nonce
+consumption and the complete grant in one PostgreSQL transaction. It is also
+available as `makePlatformPersistence(...).gateway.enrollment`. App-owned
+migrations must add nullable text columns `package_installation_id`,
+`package_fqn`, `gateway_fqn`, `audience`, and `manifest_hash` to
+`gateway_connections` before using this managed route. Legacy grant
+`has`/`put`/`remove` works with the original three-column table. The SDK executes
+no implicit migration.
+
+The [Nest server integration](../../../demo/external-app-nest/apps/server/src/integrations/collective-intelligence/gateway-security.ts) supplies configured trust and allowed identities; [exact deployment](../../../docs/reference/package-release-cli.md#deploy-an-exact-package-and-its-declared-gateways) uses this route without broad credentials reaching the server.
+
+Concurrent delivery of one token admits one request; a later grant write failure
+rolls back nonce consumption. Retry a lost acknowledgement with a fresh signed
+token for the same owning installation and Connection. A different package
+owner cannot replace that grant. Existing setup enrollment remains available
+through `enrollConnection`; normal calls still require an enrolled grant, live
+CI authority, operation scopes, replay protection and application authorization.
+
+## Bounded custom operations
+
+Define each operation once and include it in a named operation domain. CI admits
+only explicitly selected operations from the accepted connection revision. Its
+generic tool input uses the existing `ai-function-input` schema profile: an
+object root with supported keywords. Incompatible inputs are rejected at
+admission; their schemas are not rewritten. Declared JSON output may be an
+object, array or scalar and is validated before returning a successful result.
+
+```ts
+import {
+  defineOperation,
+  defineOperationDomain
+} from "@collective-intelligence/external-apps-server-sdk/gateway"
+
+const quote = defineOperation<{ quantity: number }, { total: number }>({
+  id: "quote",
+  version: "1.0.0",
+  title: "Quote",
+  description: "Calculate a quote for an opportunity.",
+  input: {
+    dialect: "https://json-schema.org/draft/2020-12/schema",
+    fidelity: "exact",
+    document: {
+      type: "object",
+      properties: { quantity: { type: "number", minimum: 1 } },
+      required: ["quantity"],
+      additionalProperties: false
+    }
+  },
+  output: {
+    dialect: "https://json-schema.org/draft/2020-12/schema",
+    fidelity: "exact",
+    document: {
+      type: "object",
+      properties: { total: { type: "number" } },
+      required: ["total"],
+      additionalProperties: false
+    }
+  },
+  effects: "read",
+  idempotency: "safe",
+  retry: { mode: "never" },
+  authorization: {
+    organization: "required",
+    principalKinds: ["service_principal"],
+    delegatedSubject: "forbidden",
+    scopes: ["quotes:read"]
+  },
+  execution: { mode: "synchronous", timeoutMs: 5000 },
+  limits: { maxRequestBytes: 4096, maxResponseBytes: 4096 },
+  handler: async ({ input }) => ({ total: input.quantity * 42 })
+})
+
+const crm = defineOperationDomain({
+  id: "crm",
+  title: "CRM",
+  operations: [quote]
+})
+// Include crm in defineGateway({ ...definition, operationDomains: [crm] }).
+```
+
+Mutations that require replay protection declare `idempotency: "requires_key"`.
+The host must atomically associate the key with its mutation and original result
+in its own database. CI persists its invocation receipt and only permits retries
+under the declared policy; that receipt cannot replace application-side replay
+protection after an uncertain network outcome. The declared timeout bounds CI's
+whole invocation attempt sequence, including caller authentication and retries.
+
+## Identifiers
+
+Import `uuidV7` from the package root when an application needs a portable,
+time-ordered UUID default. It uses the host's Web Crypto implementation and
+does not load any optional framework peer.
+
+## External application modules
+
+`ExternalAppSubmodule<T>` validates an application-owned module with a required
+`gateway` declaration. That nested declaration must provide `knowledge`, a
+non-empty `datasets` list, or a non-empty `operations` list; application-owned
+fields such as `resource` and `conversion` remain alongside it. Pass the
+factory's inferred return type so those additional fields remain strongly typed.
+`ExternalAppModule<Submodules>` validates the named submodule map without
+widening those submodules:
+
+```ts
+import type {
+  ExternalAppModule,
+  ExternalAppSubmodule
+} from "@collective-intelligence/external-apps-server-sdk/gateway"
+
+type LeadsSubmodule = ExternalAppSubmodule<
+  ReturnType<typeof makeCrmLeadsSubmodule>
+>
+type OpportunitiesSubmodule = ExternalAppSubmodule<
+  ReturnType<typeof makeCrmOpportunitiesSubmodule>
+>
+
+type CrmModule = ExternalAppModule<{
+  leads: LeadsSubmodule
+  opportunities: OpportunitiesSubmodule
+}>
+```
+
+## Effect authoring
+
+Install the optional `effect` peer and import helpers from `/runtime/effect`.
+The executable consumer is tested with `effect@4.0.1`. Core SDK imports
+do not load Effect. `DateTimeString` and `UuidV7` provide reusable wire schemas;
+`jsonSchema` projects an Effect schema into the JSON Schema document accepted
+by gateway resource and operation definitions. The projection rejects unknown
+properties for `Schema.Struct` objects while preserving the value
+constraints of open `Schema.Record` objects.
+
+`makeEffectCrudResource` acquires the supplied repository Effect once when the
+composition program runs. It returns the same portable `CrudResource` used by
+application HTTP and gateway knowledge bindings. Its acquisition error and
+service requirements remain in the Effect type; provide the application's
+Layer and keep its scope open for the lifetime of the handlers.
+
+```ts
+import * as Effect from "effect/Effect"
+import * as Schema from "effect/Schema"
+import {
+  createResourceEffectHandler,
+  DateTimeString,
+  jsonSchema,
+  makeEffectCrudResource,
+  UuidV7
+} from "@collective-intelligence/external-apps-server-sdk/runtime/effect"
+
+const leadSchema = Schema.Struct({
+  createdAt: DateTimeString,
+  id: UuidV7,
+  name: Schema.NonEmptyString
+})
+const leadJsonSchema = jsonSchema(leadSchema)
+
+const application = Effect.gen(function* application() {
+  const resource = yield* makeEffectCrudResource({
+    definition: leadDefinition,
+    authorization: leadAuthorization,
+    repository: leadRepositoryEffect
+  })
+  const handle = createResourceEffectHandler({
+    resource,
+    basePath: "/api/leads",
+    authorize: authorizeApplicationRequest
+  })
+  return { resource, handle }
+})
+```
+
+Here `leadRepositoryEffect` acquires the host's standard repository from its
+Layer; `authorizeApplicationRequest` returns an Effect containing `true` or an
+explicit denial `Response`. Typed policy failures and service requirements
+remain visible to the host. Bind the returned `resource` to `defineGateway` and
+lift its Fetch kernel with `createGatewayEffectHandler`. These adapters use the
+same validation, CRUD behavior, SQL and authorization boundaries as the native
+Fetch handlers.
+
+Already-aborted requests never start host work. Live request aborts interrupt
+the Effect fiber, and fiber interruption aborts the forwarded Request signal.
+Interruption waits for finalizers within the authorization Effect (including
+its explicitly scoped resources) before the handler exits. A standard database
+operation already submitted to PostgreSQL may still commit; cancellation does
+not promise SQL rollback. The host owns database and server shutdown through
+its Layer/scope.
+
+`test/consumer/effect/run.ts` is an executable example using a Layer-owned
+PostgreSQL fixture, one Drizzle repository, and actual application/gateway HTTP
+requests. `bun run test:pack` validates it from the published tarball and removes
+its owned database and server afterward.
+
+## NestJS authoring
+
+Import `GatewayNestModule` and `CrudResourceNestModule` from `/runtime/nestjs`.
+They create native Nest dynamic modules, controllers and factory providers.
+Use ordinary Nest `imports`, `inject`, and `useFactory` options; asynchronous
+providers are supported. The host's module owns and exports one CRUD resource,
+and both registrations inject that same resource. No global registry is used.
+
+```ts
+import type { CrudResource } from "@collective-intelligence/external-apps-server-sdk/resources"
+import {
+  CrudResourceNestModule,
+  GatewayNestModule
+} from "@collective-intelligence/external-apps-server-sdk/runtime/nestjs"
+
+const applicationRoutes = CrudResourceNestModule.registerAsync({
+  imports: [LeadsModule],
+  inject: [LEAD_RESOURCE],
+  path: "/api/leads",
+  useFactory: (resource: CrudResource) => ({
+    resource,
+    authorize: authorizeApplicationRequest
+  })
+})
+const gatewayRoutes = GatewayNestModule.registerAsync({
+  imports: [LeadsModule],
+  inject: [LEAD_RESOURCE],
+  useFactory: (resource: CrudResource) =>
+    createGatewayKernel({
+      gateway: defineGateway({
+        ...gatewayDefinition,
+        knowledge: [{ resource, access: "read_write" }]
+      }),
+      verifyCaller: verifyMachineCaller
+    })
+})
+```
+
+Add both returned modules to the application's `@Module({ imports: [...] })`.
+`LeadsModule` owns the repository, SQL driver, migrations, and `LEAD_RESOURCE`
+provider. `authorizeApplicationRequest` returns `true` or a denial `Response`;
+`verifyMachineCaller` supplies the separately configured gateway authority.
+Set a knowledge binding to `access: "read"` to limit its gateway operations
+without removing the application's own authorized CRUD capabilities.
+
+The gateway uses its canonical `/v1/*` and discovery routes at the application
+origin; do not add a Nest global prefix to those routes. Start Nest with
+`NestFactory.create(AppModule, { bodyParser: false })` so the SDK owns bounded
+JSON parsing. Other application routes can install their own body parser.
+The executable consumer uses Nest's Express platform; its native wildcard
+syntax also remains compatible with Nest 10 routing.
+
+`test/consumer/nestjs/application.ts` contains the complete typed application
+composition. `bun test src/adapters/runtime/nestjs.test.ts` runs it against isolated
+PostgreSQL and real HTTP. `bun run test:pack` compiles and runs the installed
+Nest consumer and verifies package-root imports without Nest peers. The host
+owns application shutdown and database cleanup.
+
+## Next.js, Web and Express hosting
+
+`createResourceNextRouteHandlers` from `/runtime/next` wraps the same resource
+HTTP handler used by other hosts. In an App Router route such as
+`app/api/leads/[...path]/route.ts`, export its method handlers:
+
+```ts
+import { createResourceNextRouteHandlers } from "@collective-intelligence/external-apps-server-sdk/runtime/next"
+import { leadResource, authorizeApplicationRequest } from "../../../../server"
+
+export const runtime = "nodejs"
+export const dynamic = "force-dynamic"
+export const { GET, POST, PATCH, DELETE } = createResourceNextRouteHandlers({
+  resource: leadResource,
+  basePath: "/api/leads",
+  authorize: authorizeApplicationRequest
+})
+```
+
+Bind that same resource to the gateway and export
+`createGatewayNextRouteHandlers(kernel)` from `app/v1/[...path]/route.ts`.
+The application policy authenticates browser requests; gateway routes retain
+machine authentication and their declared read or write exposure. A writable
+application resource can have a read-only gateway binding.
+
+For Fetch hosts, `createGatewayWebHandler(kernel)` returns the canonical Gateway
+`(Request) => Promise<Response>` handler. A resource HTTP handler already uses
+that same Fetch shape. Express adapts either one:
+
+```ts
+app.use(
+  "/api",
+  createGatewayExpressMiddleware({
+    handle: createResourceHttpHandler({
+      resource: leadResource,
+      basePath: "/leads",
+      authorize: authorizeApplicationRequest
+    })
+  })
+)
+app.use("/ci", createGatewayExpressMiddleware(kernel))
+```
+
+Express removes its mount prefix, so these routes are `/api/leads/search` and
+`/ci/v1/entities/leads/search`. Mount the middleware before a body parser to let
+the shared handler enforce its byte limit while reading the request stream.
+Oversized uploads receive a bounded error without waiting for EOF; the adapter
+flushes that response before closing the unread connection. Client disconnects
+abort the forwarded request signal. The host owns server and database shutdown.
+
+`test/consumer/next` builds and starts a real Next application, then exercises
+Next, Express and Web against one PostgreSQL resource. `bun run test:pack`
+compiles and runs that consumer from the installed package.
+
+## Shared CRUD resources
+
+Expose a remote resource once with `defineGateway({ knowledge: [{ resource,
+access: "read_write" }], ... })`. The exposure supplies discovery and dispatch;
+`createGatewayKernel` no longer accepts a separate resource list. Use `access:
+"read"` to expose only search/get. Every exposed operation requires an explicit
+authorization declaration. Registrations are copied and frozen at construction.
+All connections see the same source records. The `installed_organization` marker
+identifies CI's destination organization; it never adds a source-row predicate.
+
+`defineCrudResource` accepts a portable resource definition, a `CrudRepository`,
+and gateway authorization policies. Its validated `executeCrud` service is also
+used by `createResourceHttpHandler({ resource, basePath, authorize })`; the host
+must authorize browser requests explicitly. A browser session is not a machine
+caller identity. Both transports enforce declared capabilities, schemas,
+revision preconditions and idempotency keys.
+
+The application routes are `POST /search`, `POST /records`, and
+`GET|PATCH|DELETE /records/:id`, relative to `basePath`. Search accepts nested
+`and`, `or`, and `not` filters, declared field comparisons, membership, null and
+text predicates, sorting, text search, and page/pageSize pagination. Page size
+is bounded to 100. The PostgreSQL repository adds identity fields as stable
+sorting tie-breakers and reads totals in the same database snapshot.
+
+Import Drizzle persistence from
+`@collective-intelligence/external-apps-server-sdk/persistence/drizzle`.
+`definePlatformTables()` declares the SDK-owned federated-session, Gateway
+Connection, replay, CRUD-journal, and CRUD-idempotency tables in the PostgreSQL
+`platform` schema. Export those models from the application's Drizzle schema so
+the host's normal migration workflow discovers them. The SDK never creates
+tables at runtime.
+
+```ts
+export const platformTables = definePlatformTables()
+const platformPersistence = makePlatformPersistence({
+  database,
+  tables: platformTables
+})
+
+const repository = new DrizzlePostgresRepository({
+  database,
+  table: leads,
+  definition: leadDefinition,
+  revisionField: "revision",
+  persistence: platformPersistence.resources
+})
+const resource = defineCrudResource({
+  definition: leadDefinition,
+  repository,
+  authorization: leadAuthorization
+})
+```
+
+`makePlatformPersistence()` wires all SDK-owned tables to the federated-session,
+Gateway grant, replay, and CRUD persistence contracts in one call. Use that
+bundle at the application composition root so generic plumbing stays in the SDK
+while authentication policy and configuration remain app-owned. The same
+entrypoint also exports the individual store factories for hosts that need
+custom composition.
+
+Effect applications may import `DrizzlePostgres`, `layerFromDatabase`, and
+`layerNodePostgres` from
+`@collective-intelligence/external-apps-server-sdk/persistence/drizzle/effect`.
+The Node PostgreSQL layer owns the pool lifetime and accepts an explicit
+`migrate` hook. It never discovers a migration directory or runs hidden DDL.
+
+The generic machine resource API retains its explicit hard-delete behavior.
+Updates and hard deletes compare and replace the revision atomically. Stale
+writes return a nonretryable conflict (HTTP 409). Identity and revision fields
+cannot be supplied in update values. Creates persist the idempotency key,
+input fingerprint and original result in the same transaction as the entity
+and change journal. Reusing a key with different input conflicts. Committed
+upserts and hard-delete tombstones are available through `/changes` with a
+source-bound watermark. Generic cursor pagination and restore are not implemented
+by this PostgreSQL adapter and cannot be advertised. Browser entity CRUD uses
+the separate policy-aware port below.
+
+When `entities.deletedAtField` is configured, generic `get`, `list`, `scan`,
+`update` and `hardDelete` also exclude soft-deleted rows. Repositories without
+`entities` retain their existing generic behavior.
+
+Repository results must contain JSON data. For native driver values, provide a
+typed `project` callback, for example `project: row => ({ ...row, createdAt:
+row.createdAt.toISOString() })`. Unmapped native values are rejected, ensuring
+the initial result and persisted idempotent retry have identical representations.
+
+Run `bun run test:postgres` for the isolated real PostgreSQL contract suite;
+Docker must be available. The runner owns a unique container and verifies its
+removal. `bun run test:pack` also compiles and runs installed Drizzle and Effect
+consumers through the published persistence entrypoints.
+
+### Authorized browser entity CRUD
+
+`defineEntityResource({ definition, repository })` binds the common page-based
+entity protocol to `EntityRepository`. Configure `entities` on the existing
+`DrizzlePostgresRepository` and pass `repository.entities` to the binding:
+
+```ts
+const repository = new DrizzlePostgresRepository({
+  database,
+  table: leads,
+  definition: leadDefinition,
+  revisionField: "revision",
+  persistence: platformPersistence.resources,
+  entities: {
+    storeId: "customer-crm",
+    deletedAtField: "deletedAt",
+    policy: {
+      authorizeOperation: authorizeLeadOperation,
+      rowPredicate: authorizedLeadRows,
+      authorizeResult: authorizeLeadDisclosure
+    }
+  }
+})
+const resource = defineEntityResource({
+  definition: leadDefinition,
+  repository: repository.entities
+})
+const handle = createResourceHttpHandler({
+  basePath: "/api/entities/leads",
+  resource,
+  authorize: async (request) =>
+    (await auth.getEntityContext(request, "leads")) ??
+    new Response(null, { status: 401 })
+})
+```
+
+`auth` is the existing `createFederatedAuth(...)` instance. Its server-only
+`getEntityContext` resolves current CI application, user, organization, scopes,
+roles and team IDs; browser session JSON remains identity and expiry only.
+Missing CI policy context denies entity access. Local Better Auth sessions do
+not establish CI entity authority. Browser framework adapters require this
+verified context instead of boolean authorization. Machine gateway calls retain
+their separate caller policy.
+
+All three local policy callbacks are required. `rowPredicate` applies before
+selection, counting and writes; explicitly return `undefined` for shared local
+rows. No CI organization partition is inferred. `authorizeResult` checks current
+disclosure policy for fresh results and historical receipt snapshots, including
+deleted records. `storeId` is a stable canonical store identity, not a request
+URL. Migrate the nullable text or timestamp-with-timezone deletion column through
+the application's normal schema workflow.
+
+Common create, update and delete require an operation key. The common definition
+advertises `delete: true`; deletion uses `POST /records/:id/soft-delete`.
+Existing HTTP `DELETE` retains its generic hard-delete meaning and is unavailable
+through the common entity binding. Missing or deleted targets return `not_found`;
+a stale live revision returns `conflict`.
+
+The canonical write, new revision, existing change journal and original-result
+receipt commit atomically. Matching replay returns the original result even
+after later edits or deletion, subject to current authorization. A changed intent
+conflicts. The existing receipt table and retention remain in use, without a
+second store or automatic expiry. Synchronization and derived delivery consume
+the committed journal separately; later failure cannot undo the save or cause
+replay to write again.
+
+Every application-owned writer, including custom operations and imports, must
+participate in the same revision and transactional journal authority. Raw writes
+that bypass it invalidate stale-write protection. Local relationship constraints
+remain application-owned; soft deletion does not cascade to adjacent records.
+Patches replace supplied top-level fields, validate the full candidate, reject
+empty patches and preserve omitted fields. Nonempty equal-value updates still
+advance the revision.
+
+Queries use coherent page/count snapshots, application visibility before
+pagination, literal text search, Unicode code-point string ordering, nulls last
+in both directions and identity tie-breakers. Normal reads exclude tombstones.
+The isolated PostgreSQL suite proves these behaviors; the installed package
+consumer also checks common create/update/delete, original-result replay and
+denied replay through public entrypoints.
+
+## Dataset queries
+
+Register a `GatewayDatasetBinding` beside Knowledge resources and operation
+domains. The binding declares one Dataset definition, authorization policy and
+query callback. Gateway protocol version 2 exposes the definition and bounded
+read-only records query:
+
+```ts
+const leadPipeline: GatewayDatasetBinding = {
+  id: "lead-pipeline",
+  title: "Lead pipeline",
+  definition: leadPipelineDefinition,
+  authorization: {
+    organization: "required",
+    principalKinds: ["service_principal"],
+    scopes: ["datasets:read"]
+  },
+  query: async (input, context, { signal }) =>
+    queryLeadPipeline({ input, context, signal })
+}
+```
+
+Clients read `GET /v1/datasets/:id/definition` and submit
+`POST /v1/datasets/:id/query`. A query carries the admitted definition revision,
+source location, projection, ordering and row/byte/time bounds. The kernel
+rejects stale revisions, unsupported aggregate or time-series operations,
+undeclared fields and malformed results. Version 2 does not advertise Dataset cursor
+pagination, predicates, grouping, aggregates or managed extraction.
+
+## CI-federated user authentication
+
+Import `createFederatedAuth` from
+`@collective-intelligence/external-apps-server-sdk/auth/federated`. Supply the
+browser-visible HTTPS application origin, CI issuer, OAuth client, organization,
+audience, scopes, and an application-owned atomic session store. The facade owns
+PKCE login, callback, session inspection and logout under `/api/auth` by default.
+
+The store persists only hashed opaque-cookie state, token grants and revisions;
+the browser receives a `Secure`, `HttpOnly`, `SameSite=Lax` host cookie. Every
+session lookup revalidates the short-lived CI identity. Revoked or expired CI
+access ends application authorization on the next validation. A confidential
+OAuth client may supply `clientSecret`; a public PKCE client omits it.
+
+### Forwarding a bound entity to CI
+
+Import `defineCiEntityResource` from `/resources` and use it with the same
+`createResourceHttpHandler` authorization callback as a local entity binding:
+
+```ts
+const resource = defineCiEntityResource({
+  definition: leadDefinition,
+  tenantUrl: "https://ci.example.com",
+  organizationId,
+  schemaFqn
+})
+```
+
+The tenant HTTPS origin, organization, and schema FQN are server configuration.
+Pass the original context returned by `auth.getEntityContext` to the handler;
+copying or serializing it intentionally loses its private forwarding capability.
+The verified grant stays server-side. CI rechecks current application/user policy
+and owns canonical writes, revisions, and receipts; the gateway keeps no entity
+replica. Keys, edited revisions, record IDs, payloads, and correlation IDs survive
+the hop unchanged, so direct and proxied replay share the same CI receipt scope.
+The adapter never retries or follows redirects. Cancellation or a lost response
+after dispatch remains an uncertain outcome; retain the original action for
+explicit resubmission.
+
+## Optional local user authentication
+
+Import `createBetterAuth` only from
+`@collective-intelligence/external-apps-server-sdk/auth/better-auth` and install
+its optional `better-auth` peer (tested with 1.7.5). Supply the application's
+database and own its migrations and model names. Enable password login
+explicitly with `emailAndPassword: { enabled: true }` when required.
+
+The adapter verifies sessions against SQL on every lookup; cookie session caches
+and secondary storage are disabled. `getSession` returns a session or `null`,
+and reports database outages through a typed unavailable error. Local user
+identity grants no CI organization or gateway machine authority. The CRM demo
+uses federated authentication.
+
+## Run the Effect CRM demo
+
+The repository's [Effect CRM demo](../../../demo/external-app-effect/README.md)
+composes the published seams in one application: CI-federated login, shared
+Lead and Opportunity CRUD, a read-only Dataset, an idempotent conversion tool,
+durable connection grants, and an Effect server behind a TanStack Start client.
+
+Follow its environment template, then start PostgreSQL, the server and the HTTPS
+client in separate terminals:
+
+```bash
+docker run --rm --name external-app-effect-db \
+  -e POSTGRES_PASSWORD=postgres \
+  -e POSTGRES_DB=external_app_effect \
+  -p 127.0.0.1:55432:5432 postgres:17
+
+cd demo/external-app-effect/apps/server
+set -a && source .env && set +a
+bun run dev
+
+cd demo/external-app-effect/apps/client
+DEMO_APPLICATION_ORIGIN=https://external-app-effect.localhost \
+DEMO_SERVER_INTERNAL_ORIGIN=http://127.0.0.1:4118 \
+portless external-app-effect bun run dev
+```
+
+Open `https://external-app-effect.localhost`. Configure CI's reusable Gateway
+connection with `https://external-app-effect.localhost/gateway` and the same
+server-only setup token. The demo README lists the public Gateway and
+application routes and the package-specific verification commands.
+
+## Source layout
+
+For maintainers, `src/entrypoints` mirrors every declared package export and is
+the only source of published JavaScript and declarations. Confidential CI
+clients live under `src/clients`; application login belongs to
+`src/application-auth`; gateway machine authorization, definitions, and HTTP
+routing stay under `src/gateway`; portable resource behavior stays under
+`src/resources`; and proxy policy stays under `src/proxy`.
+
+Framework runtimes, provider mappings, persistence adapters, and private
+generated-client bridges are isolated under `src/adapters`. Optional peer
+dependencies must remain behind their matching runtime, persistence, or
+knowledge-source entrypoint. Add or remove public subpaths through
+`package.json`; the build derives its complete facade set from that export map
+and rejects missing entrypoints.
+
+Each export also declares a `ci-source` condition that points to its TypeScript
+facade. Workspace consumers opt into that condition with TypeScript
+`customConditions`, Bun's `--conditions=ci-source`, or their bundler's
+equivalent.
+Without that explicit local condition, including in packed and published npm
+consumers, resolution continues to use the generated `dist` JavaScript and
+declarations.
+
+## Enum value compatibility
+
+Repository-owned enum values use lower `snake_case`. Update gateway manifests,
+request payloads and saved resource filters together with their consumers. For
+example, resource filters use `starts_with` and `is_null`, lifecycle operation
+values use `soft_delete` and `hard_delete`, and gateway principal kinds use
+`detached_user` and `service_principal`. Public method and capability property
+names such as `softDelete` remain unchanged. External HTTP, OAuth, JSON Schema
+and provider values retain their required spelling at their adapter boundaries.