@collective-intelligence/extensions-sdk-backend 0.2.7 → 0.2.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +4 -0
- package/README.md +7 -0
- package/dist/effect/datasets/provider-implementations.d.ts +14 -2
- package/dist/effect.js +1 -1
- package/docs/effect.md +3 -0
- package/docs/guide/ai/index.md +349 -0
- package/docs/guide/data/connectors.md +204 -0
- package/docs/guide/data/datasets.md +156 -0
- package/docs/guide/data/knowledge.md +211 -0
- package/docs/guide/develop/extensions.md +208 -0
- package/docs/guide/develop/external-apps.md +245 -0
- package/docs/guide/develop/sdks.md +233 -0
- package/docs/guide/index.md +78 -0
- package/docs/guide/platform/access.md +244 -0
- package/docs/guide/platform/architecture.md +142 -0
- package/docs/guide/platform/entrypoints.md +236 -0
- package/docs/guide/scenarios/index.md +107 -0
- package/docs/guide/template/index.md +276 -0
- package/docs/guide/template/state-machines.md +294 -0
- package/package.json +4 -4
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# Platform architecture
|
|
2
|
+
|
|
3
|
+
The parts of Collective Intelligence, the three integration models, and how a demo package moves from build to install.
|
|
4
|
+
|
|
5
|
+
A demo built on Collective Intelligence (CI) produces two artifacts and points both at a CI deployment that already runs. The first is an extension package that CI installs into an Organization. The second is a web app that runs on your own host and calls CI. Your team does not run the backend, its workers, Compute, Media, the Dashboard or Studio; the CI deployment provides all of them.
|
|
6
|
+
|
|
7
|
+
## Tenant and Organization
|
|
8
|
+
|
|
9
|
+
A tenant is one CI deployment and its top-level security boundary, so one installation of CI has exactly one tenant. An Organization is a data and access partition inside a tenant, and each Organization belongs to one tenant. For a demo, the customer is almost always an Organization inside an existing tenant. Do not use "tenant" as another word for the customer.
|
|
10
|
+
|
|
11
|
+
Platform-wide choices live at tenant scope: tenant users, oversight of Organizations, feature flags, AI model availability and defaults, maintenance and monitoring. Most of what a demo touches lives at Organization scope: users and Groups, installed packages, connector instances, Knowledge, tools, Skills, workflows and Studio apps.
|
|
12
|
+
|
|
13
|
+
CI never infers either scope. Code installed inside CI receives the tenant and Organization from CI on each call. An external application names the tenant URL in its configuration and the Organization in each data operation.
|
|
14
|
+
|
|
15
|
+
## Surfaces
|
|
16
|
+
|
|
17
|
+
| Surface | Who uses it | What it does for a demo |
|
|
18
|
+
| ----------- | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
19
|
+
| Dashboard | Administrators | Configures the Organization: users, Groups, access, packages, connectors, Knowledge schemas, agents, Skills, tools, workflows, external applications and MCP. |
|
|
20
|
+
| Studio | End users | Desktop app where people chat with agents, open Studio apps, fill agent forms and read Documents. Each tenant has an install page with a Connect Studio button, so nobody types a backend URL. |
|
|
21
|
+
| Backend API | Dashboard, Studio, CLI, your app | Serves every HTTP route. Its OpenAPI document is the authority that the SDKs and the CLI are generated from. |
|
|
22
|
+
| Workers | The backend | Background workers run long jobs, and interactive workers run the agent and tool work that users wait for. |
|
|
23
|
+
| Compute | The backend | Runs transcription, transcoding and conversion from a queue. Each capability has its own slots, so a long transcode does not block transcription. |
|
|
24
|
+
| Media | Browsers | Serves file bytes and image transforms from URLs that the backend signs. No file byte passes through a backend endpoint. |
|
|
25
|
+
|
|
26
|
+
Your code never calls Compute. For files, the backend decides who can see each one and signs its URL, and Media only checks the signature. A principal can fetch a file if, and only if, it can see the item the file is attached to.
|
|
27
|
+
|
|
28
|
+
> **Note:** Some Dashboard areas, such as Organization onboarding, Knowledge setup and
|
|
29
|
+
> apps setup, start disabled. A tenant administrator turns them on per
|
|
30
|
+
> Organization with feature flags. If a demo Organization lacks a menu item,
|
|
31
|
+
> check the tenant feature flags first.
|
|
32
|
+
|
|
33
|
+
## Three ways to integrate
|
|
34
|
+
|
|
35
|
+
CI supports three integration models, and none of them depends on another.
|
|
36
|
+
|
|
37
|
+
```mermaid
|
|
38
|
+
flowchart LR
|
|
39
|
+
subgraph tenant["CI tenant (one deployment)"]
|
|
40
|
+
dashboard["Dashboard"]
|
|
41
|
+
studio["Studio"]
|
|
42
|
+
api["Backend API"]
|
|
43
|
+
workers["Workers"]
|
|
44
|
+
compute["Compute"]
|
|
45
|
+
media["Media"]
|
|
46
|
+
subgraph org["Organization"]
|
|
47
|
+
pkg["Installed extension package"]
|
|
48
|
+
end
|
|
49
|
+
dashboard --> api
|
|
50
|
+
studio --> api
|
|
51
|
+
api --> workers
|
|
52
|
+
api --> compute
|
|
53
|
+
api --- pkg
|
|
54
|
+
end
|
|
55
|
+
webapp["Your web app (BFF)"] -->|"Application API"| api
|
|
56
|
+
api -->|"signed calls"| gateway["Your Gateway"]
|
|
57
|
+
browser["Browser"] -->|"signed URLs"| media
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The first model installs code inside CI. An extension package declares schemas, Groups, access, agents, Skills, tools, workflows and Studio apps, and it can add React widgets to Studio and the Dashboard. CI runs this code in its own hosts and gives it the current tenant and Organization on each call. See [Extensions](../develop/extensions.md).
|
|
61
|
+
|
|
62
|
+
The second model is an external application: software your team hosts that calls CI over HTTP. The template's web app is one. A server-side BFF holds the app's confidential credentials, signs users in with their CI identity and calls the Application API. See [External apps](../develop/external-apps.md).
|
|
63
|
+
|
|
64
|
+
In the third model, CI calls your server. A Gateway, installed in your server with the server SDK, exposes your own resources, datasets and business operations to CI. Your application keeps its database, its business rules and its user sessions, and CI can import your resources into Knowledge, query your datasets and call your operations as tools.
|
|
65
|
+
|
|
66
|
+
Choose the direction by asking who owns the records. When CI owns them, as with the template's Leads that people edit from Studio and from the web app, call CI from your app. When your app's database stays the source of truth and CI must read or act on it, expose the data through a Gateway. The template uses only the outbound direction and needs no Gateway connection to run.
|
|
67
|
+
|
|
68
|
+
## Packages
|
|
69
|
+
|
|
70
|
+
A package in CI distributes platform capabilities, and code is only one part of it. One package can carry Knowledge domains and schemas, connectors, tools, Skills, agents, workflows, Studio apps and widgets, Groups, Access Profiles and external application registrations. A customer solution package usually holds the shared pieces for one Organization's initiative. One package per Organization is a convention, and CI does not enforce it.
|
|
71
|
+
|
|
72
|
+
CI's own built-in features ship as packages with the same manifest, bundle and runtime contracts, so a demo package uses the same mechanism as the platform's core packages.
|
|
73
|
+
|
|
74
|
+
Items that a package declares belong to the package installation. A Declared Studio App, for example, is read-only in the Studio builder and changes only when you deploy a new release of its package. Ordinary saves in the Dashboard never install or change packages.
|
|
75
|
+
|
|
76
|
+
### Package lifecycle
|
|
77
|
+
|
|
78
|
+
```mermaid
|
|
79
|
+
flowchart LR
|
|
80
|
+
declare["Declarations in TypeScript"] --> build["ci package build"]
|
|
81
|
+
build --> archive["Immutable archive and release manifest"]
|
|
82
|
+
archive --> validate["Validate"]
|
|
83
|
+
validate --> publish["Publish"]
|
|
84
|
+
publish --> promote["Promote"]
|
|
85
|
+
promote --> install["Install or update in one Organization"]
|
|
86
|
+
install --> desired["Desired release"]
|
|
87
|
+
desired -->|"prepared on every host"| serving["Serving release"]
|
|
88
|
+
desired -.->|"preparation fails"| previous["Previous release keeps serving"]
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Each published version is an immutable release. Installing or updating a package in an Organization selects a desired release, and CI then prepares it on every host that needs it: the backend, the Dashboard and Studio. The serving release changes only when every host is ready. If preparation fails, the previous compatible release continues to serve. Backend code and React widgets from one release always move together and cannot drift to different versions.
|
|
92
|
+
|
|
93
|
+
Rollback selects an earlier release through the same lifecycle checks. It does not restore records, and it does not write inverse changes to definitions.
|
|
94
|
+
|
|
95
|
+
The CLI and the Dashboard reach the same backend operation for each package action. The CLI also has `ci deploy`, which publishes the release only when it is absent, promotes it only when needed, and then installs or updates it. Copying a package archive somewhere gives it no authority; only an installation makes a package active in an Organization.
|
|
96
|
+
|
|
97
|
+
## How the template maps onto CI
|
|
98
|
+
|
|
99
|
+
The `next-effect-app` template has two composition roots, one for each side of the boundary.
|
|
100
|
+
|
|
101
|
+
| Folder | Runs where | SDKs | Reaches CI by |
|
|
102
|
+
| ----------------- | ---------------------------------------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------ |
|
|
103
|
+
| `apps/extensions` | Inside CI, installed in the Organization | `extensions-sdk`, `extensions-sdk-backend`, `extensions-sdk-frontend-react`, `ui-react` | `ci package build`, publish, promote and install |
|
|
104
|
+
| `apps/web` | Your host, as a Next.js app | `external-apps-server-sdk` on the server, `external-apps-client-sdk-react` in the browser | The BFF calls the Application API |
|
|
105
|
+
|
|
106
|
+
The package root in `apps/extensions` is short. It combines the declared components with the Lead score card widget for Studio:
|
|
107
|
+
|
|
108
|
+
```ts
|
|
109
|
+
export default declarePackage({
|
|
110
|
+
components: packageComponents,
|
|
111
|
+
hostedContributions: [
|
|
112
|
+
declareFrontendCatalogContribution({
|
|
113
|
+
compatibility: { react: "^19.0.0" },
|
|
114
|
+
fqn: leadScoreCardContributionFqn,
|
|
115
|
+
host: "studio",
|
|
116
|
+
load: async () => await import("./studio")
|
|
117
|
+
})
|
|
118
|
+
],
|
|
119
|
+
manifest: packageManifest
|
|
120
|
+
})
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
When installed, that package creates the CRM domain with the Lead and Opportunity schemas, the Sales Groups and their access, the app registration, the convert-lead workflow, the lead qualifier agent, the Sales desk Studio app and the widget. The web app has no database of its own. Users sign in with their CI identity, Leads and Opportunities live in CI Knowledge, and lead conversion runs as a CI workflow.
|
|
124
|
+
|
|
125
|
+
> **Warning:** The web app serves two route families that start with `/api`. The `/api/app/*`
|
|
126
|
+
> routes are the app's own HTTP API, merged from its packages. The `/api/*`
|
|
127
|
+
> routes are the BFF that forwards to CI. The browser calls only these
|
|
128
|
+
> same-origin routes and never holds the client secret.
|
|
129
|
+
|
|
130
|
+
The Overview and Lead intake pages of the template work with no CI at all, because intake data stays in memory. Only the CRM pages need a connected tenant.
|
|
131
|
+
|
|
132
|
+
## Where data lives
|
|
133
|
+
|
|
134
|
+
Knowledge holds durable business concepts such as customers, leads and assets as records that follow declared schemas, and it is what agents, Skills, workflows and Studio apps query. Datasets hold typed, high-volume records such as financial measures or sensor readings, which Knowledge does not fit. Documents are a separate surface for files and Document Collections, with their own page in Studio. Read [Knowledge](../data/knowledge.md), [Datasets](../data/datasets.md) and [Connectors](../data/connectors.md) for the details.
|
|
135
|
+
|
|
136
|
+
## Next pages
|
|
137
|
+
|
|
138
|
+
- [Entrypoints](entrypoints.md): How people, apps and agents reach CI, and what your code imports.
|
|
139
|
+
- [Access](access.md): Access Profiles, Access Grants and Max Privileges.
|
|
140
|
+
- [Extensions](../develop/extensions.md): Write the package that CI installs.
|
|
141
|
+
- [External apps](../develop/external-apps.md): Build the web app that calls CI.
|
|
142
|
+
- [Template](../template/index.md): Start from the `next-effect-app` template.
|
|
@@ -0,0 +1,236 @@
|
|
|
1
|
+
# Entrypoints
|
|
2
|
+
|
|
3
|
+
The ways people, apps and agents enter CI at runtime, and the package subpaths that demo code imports.
|
|
4
|
+
|
|
5
|
+
In this guide, "entrypoint" has two meanings. Runtime entrypoints are the ways people, apps and agents enter CI: the Dashboard, Studio, the CLI, the HTTP API, MCP, workflow triggers and Realtime. Code entrypoints are the package subpaths that demo code imports, both from the CI SDKs and from the template's own packages.
|
|
6
|
+
|
|
7
|
+
## Runtime entrypoints
|
|
8
|
+
|
|
9
|
+
```mermaid
|
|
10
|
+
flowchart LR
|
|
11
|
+
admin["Administrator"] --> dashboard["Dashboard"]
|
|
12
|
+
user["End user"] --> studio["Studio"]
|
|
13
|
+
dev["Developer or pipeline"] --> cli["ci CLI"]
|
|
14
|
+
script["Script with an API key"] --> http["HTTP API"]
|
|
15
|
+
browser["Your browser code"] --> bff["Your BFF"]
|
|
16
|
+
bff --> appapi["Application API"]
|
|
17
|
+
ai["AI client"] --> mcp["MCP endpoint"]
|
|
18
|
+
system["External system"] --> trigger["Workflow trigger"]
|
|
19
|
+
clock["Schedule"] --> trigger
|
|
20
|
+
subgraph ci["CI"]
|
|
21
|
+
dashboard
|
|
22
|
+
studio
|
|
23
|
+
http
|
|
24
|
+
appapi
|
|
25
|
+
mcp
|
|
26
|
+
trigger
|
|
27
|
+
realtime["Realtime events"]
|
|
28
|
+
end
|
|
29
|
+
cli --> http
|
|
30
|
+
realtime -->|"ids of changed records"| browser
|
|
31
|
+
http -->|"signed calls"| gateway["Your Gateway"]
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
| Entrypoint | Who uses it | Authenticates with | Typical demo use |
|
|
35
|
+
| --------------------- | ---------------------------------------------- | ----------------------------------------------------------- | ----------------------------------------------------------------------------- |
|
|
36
|
+
| Dashboard | Administrators | Organization sign-in, including SSO | Install the package, add users to Groups, manage access, rotate an app secret |
|
|
37
|
+
| Studio | End users | CI sign-in through the tenant's Connect Studio link | Chat with agents, open a Studio app, use a widget |
|
|
38
|
+
| CLI (`ci`) | Developers and pipelines | CLI Session from `ci login`, or `CI_MANAGEMENT_TOKEN` | Build and deploy the package, create API keys |
|
|
39
|
+
| HTTP API | Scripts and unattended runs | API key of a service principal, limited by Max Privileges | Script demo setup, run agent conversations without a person |
|
|
40
|
+
| External applications | Your web app's server | OAuth client credentials, member tokens, server credentials | The template's CRM pages and lead conversion |
|
|
41
|
+
| MCP | AI clients and assistants | Bearer session token or API key | Let an assistant run Skills and workflows |
|
|
42
|
+
| Workflow triggers | Other systems, the clock, apps, record changes | Depends on the trigger; the webhook trigger uses an API key | Start a workflow from a webhook or on a schedule |
|
|
43
|
+
| Realtime | Your browser, through your BFF | Single-use ticket that your server requests | Refresh lists when a record changes |
|
|
44
|
+
|
|
45
|
+
### Dashboard and Studio
|
|
46
|
+
|
|
47
|
+
Administrators configure an Organization in the Dashboard. Its Organization admin area holds accounts, Groups, Access Profiles, capabilities, Max Privileges, access explanations, packages and a Developers area; separate areas hold AI (agents, Skills, tools, workflows, approvals), Apps (Studio, external applications, MCP), datasets and Documents. The Developers area has tabs for the CLI, connections, service principals and variables, and its CLI tab shows the exact `ci login` command for the Organization.
|
|
48
|
+
|
|
49
|
+
End users work in Studio. They chat with agents, open Studio apps, fill agent forms and read Documents. A package can ship a whole Studio app, which CI creates and publishes on install; the template's Sales desk is one. A Studio widget never holds CI credentials. It calls CI through the Studio API client from `@collective-intelligence/extensions-sdk-frontend-react/studio-api`, which Studio binds to its own signed-in transport, so each request runs with the current user's permissions.
|
|
50
|
+
|
|
51
|
+
### CLI
|
|
52
|
+
|
|
53
|
+
The `ci` CLI runs on Bun and calls the backend only through its generated OpenAPI operations. Install it globally and sign in against the API URL of your tenant:
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
npm install -g @collective-intelligence/cli
|
|
57
|
+
ci login <api-url> --organization <organization-id>
|
|
58
|
+
ci whoami
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
`ci login` asks you to approve the device in the Dashboard and stores a 30-day CLI Session in your user profile. `ci upgrade` updates a global install.
|
|
62
|
+
|
|
63
|
+
> **Note:** `ci login` needs the API URL. It reads the tenant's discovery document at
|
|
64
|
+
> `/.well-known/ci-configuration` and rejects a Dashboard URL as "not a CI API
|
|
65
|
+
> URL".
|
|
66
|
+
|
|
67
|
+
The CLI takes credentials from a flag first, then from the environment (`CI_BACKEND_URL`, `CI_MANAGEMENT_TOKEN`), then from the stored profile. Developers at a terminal use `ci login`, and pipelines set `CI_MANAGEMENT_TOKEN`. Publishing reads `CI_PUBLISHER_TOKEN` from the environment only.
|
|
68
|
+
|
|
69
|
+
For a package, a demo uses `ci package build`, `validate`, `publish`, `promote` and `install`, or the single `ci deploy` command, which publishes only when the release is absent, promotes only when needed, and then installs or updates. `ci package dev` runs a development session that serves your package modules from your machine to real Dashboard and Studio hosts. `ci package codegen` generates TypeScript types from the package schemas, and `ci doctor` checks discovery, signing keys, Organization access and redirect URIs.
|
|
70
|
+
|
|
71
|
+
### HTTP API and API keys
|
|
72
|
+
|
|
73
|
+
The backend publishes its OpenAPI document at `/openapi.json`. Its wire schemas are the authority, and the SDKs and the CLI are generated from them. A demo meets these public route families:
|
|
74
|
+
|
|
75
|
+
- `/application/v1/*`, the Application API for external applications
|
|
76
|
+
- `/oauth/*`, the authorization server for app sign-in
|
|
77
|
+
- `/.well-known/ci-configuration`, the tenant discovery document
|
|
78
|
+
- `/mcp`, the MCP endpoint
|
|
79
|
+
- `/openapi.json`, the API description
|
|
80
|
+
|
|
81
|
+
Other route families serve the Dashboard and Studio. A generated operation is not safe for a client just because it exists in the OpenAPI document.
|
|
82
|
+
|
|
83
|
+
Each credential is a separate authority:
|
|
84
|
+
|
|
85
|
+
| Credential | Held by | Used for |
|
|
86
|
+
| ------------------------------------------- | -------------------------------- | -------------------------------------------------------------------------------- |
|
|
87
|
+
| CLI Session | A person | Interactive CLI work for 30 days, from device approval |
|
|
88
|
+
| Management token (`CI_MANAGEMENT_TOKEN`) | A pipeline | CLI work with an Organization API key or a session token |
|
|
89
|
+
| Publisher credential (`CI_PUBLISHER_TOKEN`) | A release pipeline | Publish and promote for one package only |
|
|
90
|
+
| Client id and client secret | An external application | OAuth sign-in; only a confidential app has a secret |
|
|
91
|
+
| Server credential | An external application's server | Calls that act with the app's own grants |
|
|
92
|
+
| API key | A service principal | Calls that act with the principal's grants, limited by a Max Privileges template |
|
|
93
|
+
|
|
94
|
+
API keys have no scopes. A key never acts beyond its Max Privileges template, and creating, rotating or revoking a key needs manage access to API keys, which no API key has:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
ci apps credentials upsert <app-fqn-or-id> --name pipeline --env-file .env.server
|
|
98
|
+
ci api-keys upsert --principal <principal-fqn> --name pipeline \
|
|
99
|
+
--max-privileges <template-fqn> --env-file .env.pipeline
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
A service principal key can run Studio agent conversations without a person. It can list apps, open and read conversations, send agent turns, and cancel or close a conversation. Approvals, memory, preferences and answers to questions stay with people.
|
|
103
|
+
|
|
104
|
+
> **Warning:** A service principal run asks no questions and has no Studio memory. When a
|
|
105
|
+
> tool binding has an approval gate, the unattended run approves by default and
|
|
106
|
+
> records the approval without an actor. Plan scripted demos around that
|
|
107
|
+
> behavior.
|
|
108
|
+
|
|
109
|
+
### External applications
|
|
110
|
+
|
|
111
|
+
An external application signs users in with the OAuth authorization code flow and S256 PKCE, through `/oauth/authorize` and `/oauth/token`. The Dashboard shows the sign-in and consent page with the Organization's branding and login methods, including SSO. CI publishes OAuth 2.0 authorization server metadata and is not an OpenID Connect provider.
|
|
112
|
+
|
|
113
|
+
A public client runs only in the browser and has a client id without a secret. A confidential client is a backend that keeps a client secret. Access tokens last 300 seconds, and only a confidential app can renew one without a browser redirect or exchange a member session for a token. The template's app is confidential. Because its package declares the app, you take the client id from the Dashboard and rotate the secret there to get the client secret.
|
|
114
|
+
|
|
115
|
+
In the template, the browser calls only same-origin `/api/*` routes. The BFF behind them, built with `createExternalApplicationBffNextHandlers` from the server SDK, signs users in with a sealed cookie, forwards Resource calls per entity contract, requests Realtime tickets and forwards capabilities. Only `NEXT_PUBLIC_APPLICATION_ID` is visible to the browser.
|
|
116
|
+
|
|
117
|
+
An app can start only a workflow whose active External application trigger names that app. The template's `POST /api/leads/{id}/convert` route uses `createFixedWorkflowRoute`, which starts exactly one workflow that the server configures (`acme.app.workflows.convert-lead`) with the idempotency key `convert-lead:{leadId}`. The route rejects a workflow name or key in browser input. A call on behalf of a user runs as that user; a service call runs as the service principal that the workflow selects.
|
|
118
|
+
|
|
119
|
+
### MCP
|
|
120
|
+
|
|
121
|
+
The backend serves MCP at `/mcp` and at `/mcp/o/<organizationId>`. Every MCP request needs a valid Bearer session token or API key, including `initialize` and `tools/list`. The server exposes these tools: `list_memberships`, `list_skills`, `run_skill`, `get_skill_run`, `list_operations`, `invoke_operation`, `get_operation`, `cancel_operation` and `explain_access`. `run_skill` runs in the background, so a client polls `get_skill_run` for the result.
|
|
122
|
+
|
|
123
|
+
A workflow becomes visible to MCP clients through an MCP operation trigger. The caller gets an opaque handle and reads the result with `get_operation`. An MCP Profile, managed in the Dashboard under Apps > MCP, defines what an MCP principal can see and do, and it denies everything by default. The built-in Studio user profile grants Studio and MCP use.
|
|
124
|
+
|
|
125
|
+
### Workflow triggers
|
|
126
|
+
|
|
127
|
+
A trigger is the entrypoint of a workflow. The core workflows package ships these triggers:
|
|
128
|
+
|
|
129
|
+
| Trigger | Starts a run when |
|
|
130
|
+
| -------------------- | ------------------------------------------------- |
|
|
131
|
+
| On demand | A user starts the workflow explicitly |
|
|
132
|
+
| Schedule | A cron expression fires |
|
|
133
|
+
| Webhook | Another system sends an authenticated HTTP `POST` |
|
|
134
|
+
| External application | A bound external application starts it |
|
|
135
|
+
| MCP operation | An MCP client invokes it |
|
|
136
|
+
| Entity created | A Knowledge record is created |
|
|
137
|
+
| Entity updated | A Knowledge record changes |
|
|
138
|
+
| Entity deleted | A Knowledge record is deleted |
|
|
139
|
+
|
|
140
|
+
Integration packages add message and file triggers for Slack, Microsoft 365 email, Teams and Drive, Telegram, WhatsApp, Twilio SMS and WhatsApp, and Notion.
|
|
141
|
+
|
|
142
|
+
A webhook trigger authenticates the caller with an API key, accepts `POST`, and returns JSON. The caller waits for the workflow's response until a deadline and then receives `202` with the run id. CI binds event triggers when it publishes the workflow, and it refuses to publish an enabled trigger with an invalid activation declaration. In the template, the convert-lead workflow is declared in the package, and its trigger names the app's FQN, which install resolves to the app id.
|
|
143
|
+
|
|
144
|
+
### Realtime
|
|
145
|
+
|
|
146
|
+
Realtime tells open pages that records changed. Your server requests a single-use ticket from `POST /application/v1/realtime/tickets`, and the browser opens an event stream at `GET /application/v1/realtime/events` with that ticket. Events carry ids only, so the page reloads the records through its own data routes. A subscription names a schema and an optional field-equals filter:
|
|
147
|
+
|
|
148
|
+
```ts
|
|
149
|
+
const realtime = createRealtimeClient({
|
|
150
|
+
ticketUrl: "/api/realtime/ticket",
|
|
151
|
+
document,
|
|
152
|
+
window,
|
|
153
|
+
onReset: () => queryClient.invalidateQueries(),
|
|
154
|
+
onAccessChanged: () => session.refresh()
|
|
155
|
+
})
|
|
156
|
+
realtime.subscribe(
|
|
157
|
+
{ schema: "crm.lead", filter: { field: "ownerId", eq: userId } },
|
|
158
|
+
(changes) => invalidateLeads(changes.map((change) => change.id))
|
|
159
|
+
)
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
On a reset, the client drops its local view and refetches every query that a subscription feeds.
|
|
163
|
+
|
|
164
|
+
### When CI calls you
|
|
165
|
+
|
|
166
|
+
A Gateway reverses the direction. You register a Gateway connection with an HTTPS endpoint and a setup credential, and after enrollment CI calls your server with short-lived ES256 tokens and a replay check. Your operations then become CI tools for chat, Skills and workflows. [External apps](../develop/external-apps.md) covers the setup.
|
|
167
|
+
|
|
168
|
+
## Code entrypoints
|
|
169
|
+
|
|
170
|
+
Demo code imports CI packages and its own workspaces by package name and exported subpath, for example `@acme/lead/views`, and never by relative path. Each subpath has one folder under `src/` with one `index.ts` and one `exports` entry in `package.json`.
|
|
171
|
+
|
|
172
|
+
### SDK subpaths
|
|
173
|
+
|
|
174
|
+
The SDKs split by where the code runs. [SDKs](../develop/sdks.md) has the full map; these are the subpaths a demo meets most.
|
|
175
|
+
|
|
176
|
+
| Package | Runs | Holds | Main subpaths |
|
|
177
|
+
| --------------------------------------------------------- | ------------ | ------------------------------------------------------------------------ | ---------------------------------------------------------- |
|
|
178
|
+
| `@collective-intelligence/extensions-sdk` | Inside CI | Declarations such as `declarePackage`, `declareAgent`, `declareWorkflow` | `.`, `./effect` |
|
|
179
|
+
| `@collective-intelligence/extensions-sdk-backend` | Inside CI | Contracts for backend tools and connectors | `.`, `./effect` |
|
|
180
|
+
| `@collective-intelligence/extensions-sdk-frontend` | Inside CI | Framework-neutral frontend catalog | `.`, `./effect` |
|
|
181
|
+
| `@collective-intelligence/extensions-sdk-frontend-react` | Inside CI | React bindings for Studio and Dashboard contributions | `./studio`, `./dashboard`, `./studio-api`, `./effect` |
|
|
182
|
+
| `@collective-intelligence/external-apps-server-sdk` | Your server | Server client, BFF, authentication, Gateway | `.`, `./auth/federated`, `./effect`, `./next`, `./gateway` |
|
|
183
|
+
| `@collective-intelligence/external-apps-client-sdk` | Your browser | React-free client and Realtime | `.`, `./resources`, `./effect` |
|
|
184
|
+
| `@collective-intelligence/external-apps-client-sdk-react` | Your browser | React hooks, including the entity hooks in `./resources` | `.`, `./resources`, `./effect` |
|
|
185
|
+
|
|
186
|
+
Every SDK puts its Promise API at the root and its Effect services, Layers, errors and Schemas at `/effect`. `effect` is a required peer of every SDK, so install it next to the SDK and keep one `effect` version in the app. Optional peers such as Next.js, Express, Drizzle, Better Auth and the AI SDK sit behind their own subpaths, so importing the server SDK root never loads them. In the template, `/next` is imported only inside `apps/web/src/server`, because `/next` loads Next.js and `/effect` does not.
|
|
187
|
+
|
|
188
|
+
### Template package subpaths
|
|
189
|
+
|
|
190
|
+
Each template package exposes up to six subpaths, and each subpath holds one kind of code:
|
|
191
|
+
|
|
192
|
+
| Subpath | Holds | Imported by |
|
|
193
|
+
| ---------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------- |
|
|
194
|
+
| `domain` | Effect Schema models, resource contracts, schema FQNs, pure rules | Every other subpath and other packages |
|
|
195
|
+
| `ci` | CI declarations, written as factories over Organization handles | `@acme/organization` |
|
|
196
|
+
| `api` | The package `HttpApi` under `/api/app`, its groups and tagged errors | `server`, `headless`, app API composition |
|
|
197
|
+
| `server` | The route Layer of the package `HttpApi` and its services | `apps/web/src/server` only |
|
|
198
|
+
| `headless` | Typed client, query options, XState machines, entity bindings, Realtime hooks, simulated sources | `views`, the app |
|
|
199
|
+
| `views` | Wired React views on `@acme/ui`, with slots for feature parts | The app |
|
|
200
|
+
|
|
201
|
+
Imports point one way. `domain` imports no other subpath, `ci` imports neither `headless` nor `views`, and `headless` and `views` never import `server`. The `ci-solution/subpath-imports` lint rule enforces these rules.
|
|
202
|
+
|
|
203
|
+
A package exposes only the subpaths it needs. `@acme/lead` has all six, while `@acme/lead-qualification` has only `./ci`, which holds the lead qualifier agent, its Skill, its tool and its Studio widget. The `ci` subpath is where solution code becomes code that CI installs.
|
|
204
|
+
|
|
205
|
+
### Template routes and widgets
|
|
206
|
+
|
|
207
|
+
The web app has three route files, and each only re-exports handlers from `server/` and the packages:
|
|
208
|
+
|
|
209
|
+
| Route file | Serves |
|
|
210
|
+
| ------------------------------------- | ------------------------------------------------------- |
|
|
211
|
+
| `app/api/app/[[...path]]/route.ts` | The app's own API, composed from the package `HttpApi`s |
|
|
212
|
+
| `app/api/[...path]/route.ts` | The CI BFF: sign-in, Resources, Realtime ticket |
|
|
213
|
+
| `app/api/leads/[id]/convert/route.ts` | The fixed convert-lead workflow |
|
|
214
|
+
|
|
215
|
+
Inside CI, a Studio or Dashboard widget enters through a hosted contribution. You write `declareFrontendCatalogContribution({ fqn, host: "studio", load: () => import("./studio") })` and pass it to `declarePackage({ hostedContributions })`. The `load` function stays out of the serialized package manifest, and the loaded module exports `loadFrontendCatalog` for the host.
|
|
216
|
+
|
|
217
|
+
## Pick an entrypoint
|
|
218
|
+
|
|
219
|
+
| I want to | Use |
|
|
220
|
+
| ------------------------------------ | ---------------------------------------------------------- |
|
|
221
|
+
| Let users chat with an agent | Studio |
|
|
222
|
+
| Let users work in my web app | The Application API through my BFF |
|
|
223
|
+
| Start a workflow from my app | A fixed workflow route and an External application trigger |
|
|
224
|
+
| Start a workflow from another system | A webhook trigger |
|
|
225
|
+
| Run a workflow every night | A schedule trigger |
|
|
226
|
+
| Let an AI assistant call CI | MCP |
|
|
227
|
+
| Script a demo setup | The CLI with a service principal API key |
|
|
228
|
+
| Let CI call my system | A Gateway |
|
|
229
|
+
|
|
230
|
+
## Related pages
|
|
231
|
+
|
|
232
|
+
- [Platform architecture](architecture.md): Tenants, Organizations, surfaces and the package lifecycle.
|
|
233
|
+
- [Access](access.md): Access Profiles, Access Grants and Max Privileges.
|
|
234
|
+
- [SDKs](../develop/sdks.md): Every CI package and its subpaths.
|
|
235
|
+
- [External apps](../develop/external-apps.md): The BFF, sign-in and the Gateway.
|
|
236
|
+
- [Template](../template/index.md): The packages and routes of the template.
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# Choosing primitives
|
|
2
|
+
|
|
3
|
+
Questions, criteria and scenario patterns for picking the CI primitives a demo needs.
|
|
4
|
+
|
|
5
|
+
A customer scenario reaches the demo team as a story, such as a sales desk that qualifies leads or a back office that reads uploaded documents. This page gives the criteria that turn such a story into CI primitives. The [template](../template/index.md) is one finished answer, a small sales CRM with Lead and Opportunity schemas in Knowledge, an external application for the web app, a convert-lead workflow, a lead qualifier agent and a Sales desk Studio app.
|
|
6
|
+
|
|
7
|
+
Some screens need no primitive at all. The template's public intake form posts to an API inside the web app, and that API keeps requests in memory until you replace its store with a database layer.
|
|
8
|
+
|
|
9
|
+
## Decision path
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
flowchart TD
|
|
13
|
+
S[Demo scenario] --> Q1{Where does the data live?}
|
|
14
|
+
Q1 -->|In CI| Q2{Who edits it?}
|
|
15
|
+
Q2 -->|People, agents and workflows| K[Knowledge schema]
|
|
16
|
+
Q2 -->|Nobody, rows are read and aggregated| MD[Managed dataset]
|
|
17
|
+
Q1 -->|Customer database or warehouse| Q3{Must it stay there?}
|
|
18
|
+
Q3 -->|Yes| XD[External dataset or Gateway]
|
|
19
|
+
Q3 -->|Agents need a few fields| K
|
|
20
|
+
Q1 -->|Only the app needs it| APP[App-owned storage]
|
|
21
|
+
S --> Q4{Is it files?}
|
|
22
|
+
Q4 -->|People browse, agents search| DOC[Documents]
|
|
23
|
+
Q4 -->|One extraction step| WF[Workflow]
|
|
24
|
+
S --> Q5{Does it come from a SaaS?}
|
|
25
|
+
Q5 -->|Recurring import| CON[Connector]
|
|
26
|
+
S --> Q6{Deterministic or open-ended?}
|
|
27
|
+
Q6 -->|Fixed steps| WF
|
|
28
|
+
Q6 -->|Open-ended| AG["Agent + Skill"]
|
|
29
|
+
Q6 -->|One fixed rule| T[Tool]
|
|
30
|
+
S --> Q7{Does a side effect need approval?}
|
|
31
|
+
Q7 -->|In chat or a Skill form| TA[Tool with approval]
|
|
32
|
+
Q7 -->|Inside a process| AN[Workflow approval node]
|
|
33
|
+
S --> Q8{Who uses the UI?}
|
|
34
|
+
Q8 -->|Customers outside CI| EA[External app]
|
|
35
|
+
Q8 -->|Staff inside Studio| SA[Studio app]
|
|
36
|
+
S --> Q9{Must agents call existing backend logic?}
|
|
37
|
+
Q9 -->|Yes| GW["Gateway + remote tool collection"]
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The questions are independent, so one scenario answers several of them and the demo combines the results. The sales CRM answers the data question with Knowledge schemas, the work question with a workflow and an agent, and the UI question twice: customers use the external app and staff use the Studio app.
|
|
41
|
+
|
|
42
|
+
## Criteria
|
|
43
|
+
|
|
44
|
+
| Signal in the scenario | Primitive | Why | Watch out for |
|
|
45
|
+
| ----------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
|
|
46
|
+
| People in the app or Studio, agents and workflows all read and edit the records | [Knowledge schema](../data/knowledge.md) | One typed record per real thing, with relations, search and data policies that every surface shares | Measurements, events and other high-volume rows belong in a dataset, so do not create one record per row |
|
|
47
|
+
| Rows that people read and aggregate, such as a CSV or Excel export with trends | [Managed dataset](../data/datasets.md) | CI stores the typed rows and answers records, aggregate and time-series queries | A dataset declared with no data policy grants access to nobody, and deploy does not import the file |
|
|
48
|
+
| A warehouse such as Microsoft Fabric or Databricks holds the data, and it must stay there | [External dataset](../data/datasets.md) | CI queries the source live and keeps no copy | Queries take no SQL, and a provider outage fails the query with no fallback |
|
|
49
|
+
| The customer's own app and database keep the records | [Gateway](../develop/external-apps.md) | CI reaches the resources, dataset queries and operations that the customer server declares | The Gateway exposes only what its definition lists and never creates CI schemas |
|
|
50
|
+
| The app database is the authority, but agents need some of its fields | [Knowledge schema](../data/knowledge.md) with a projection | Only the needed, non-personal fields leave the app | Decide field by field what may leave; personal data stays in the app database |
|
|
51
|
+
| Files that people browse and agents search | [Documents](../data/knowledge.md#documents-and-knowledge) in a Document Collection | Collections hold folders, versions and permissions per path, and a connected collection stays in sync with a drive | A [Documents connector](../data/connectors.md) sync creates no Knowledge records, and new files become searchable only after indexing |
|
|
52
|
+
| A file that feeds one extraction step and nothing else | Stored file passed to a [workflow](../ai/index.md) | The workflow takes the file as `storageFileId` and returns extracted values | Use Documents instead when people must find the file later |
|
|
53
|
+
| A SaaS tool or SQL database that the demo imports from on a schedule | [Knowledge connector](../data/connectors.md) | A connector instance maps source fields to Knowledge schemas and runs full or delta imports | Installing the instance imports nothing until a schedule or a manual trigger runs, and SQL delta imports need a watermark column |
|
|
54
|
+
| A fixed sequence such as extract, check, create and notify | [Workflow](../ai/index.md) | A graph of steps with triggers, approvals and typed output, where a prompt node covers one bounded AI step | Every triggered workflow needs a service principal as its run-as identity |
|
|
55
|
+
| Open-ended work such as answering questions, finding matches or qualifying | [Agent with a Skill](../ai/index.md) | The agent picks its path from what it finds and returns typed output inside stated boundaries | A Skill can use only tools that its agent also allows, so the agent lists the Skill's tools again |
|
|
56
|
+
| A rule that must give the same answer every time | [Tool](../ai/index.md) | Code computes the result, and the agent explains it | An agent can call the tool only when its `usageScopes` include `"automation"` |
|
|
57
|
+
| A side effect that a person must approve | [Tool binding with approval](../ai/index.md), or a workflow approval node | `approval: "always"` or a named rule pauses the call until someone decides | A named rule must exist in the Organization before install, and an unattended agent run approves by default |
|
|
58
|
+
| Customers or members use the UI outside CI | [External application](../develop/external-apps.md) | The web app keeps its own UI and reaches CI through its BFF with a sealed-cookie session | The app takes no OAuth scopes; its app gate and Max Privileges limit what users can do |
|
|
59
|
+
| Staff work inside Studio | [Studio app](../develop/extensions.md) | Built-in views show schema records without code, and custom widgets cover what they cannot show | A Declared Studio App is read-only in the Studio builder and changes only with a new package release |
|
|
60
|
+
| Agents must call logic that an existing backend already implements | [Gateway with a remote tool collection](../develop/external-apps.md) | The logic stays in the customer service and reaches agents as tools | CI signs a short-lived token for every Gateway call, and browser cookies never authorize one |
|
|
61
|
+
| Some people may see or do things that others may not | [Access Profiles and Access Grants](../platform/access.md) | Capabilities and Knowledge data policies decide actions and visible records, and grants give them to Groups | Knowledge data policies do not cover datasets, which list their principals in their own policy |
|
|
62
|
+
| Logs, counters or sessions that only the app needs | App-owned storage behind the app API | No CI user or agent reads this data | A staff page that shows it needs its own credential in the app |
|
|
63
|
+
|
|
64
|
+
## Workflow, agent or tool
|
|
65
|
+
|
|
66
|
+
A workflow prompt node and an agent both call a model, which makes this the choice newcomers most often get wrong. A prompt node fits a step with fixed input and output shapes at a fixed place in a sequence, while an agent fits work whose path depends on what it finds. A rule that must give the same answer every time belongs in a tool that the agent or Skill calls. The template's lead score tool adds named points ("Work email" 40, "Email domain matches the company" 30, "Qualified by sales" 30), and the lead qualifier agent submits that score with a one-sentence rationale. In the demos built so far, document extraction, enrichment and approvals ran as workflows, and question answering, matching and qualification ran as agents.
|
|
67
|
+
|
|
68
|
+
An agent that must not change data says so in `behavioralBoundaries`. The lead qualifier declares "Never change a Lead or create an Opportunity". When an agent should help with a change, let it propose the change and let the person apply it with a button in the app.
|
|
69
|
+
|
|
70
|
+
A workflow's trigger follows from who or what starts the work. A `schedule` trigger runs it on a clock, `entity-created` and `entity-updated` react to Knowledge changes, `external-application` lets the web app start it, and `on-demand` lets staff start it from Studio. The template's convert-lead workflow uses the `external-application` trigger and runs as the `convert-lead-runner` service principal. See [AI](../ai/index.md) for the full comparison of tools, Skills, agents and workflows.
|
|
71
|
+
|
|
72
|
+
## Approvals depend on the caller
|
|
73
|
+
|
|
74
|
+
In chat or a Skill form, a tool binding with `approval: "always"` or a named rule pauses the call until a person decides. A workflow pauses at an explicit approval node, and the approver decides from Studio notifications.
|
|
75
|
+
|
|
76
|
+
> **Warning:** An unattended agent run has no person to ask. When one of its tool bindings
|
|
77
|
+
> requires approval, CI approves the call by default and records that decision.
|
|
78
|
+
> If the demo must show a person approving, use chat, a Skill form or a workflow
|
|
79
|
+
> approval node.
|
|
80
|
+
|
|
81
|
+
## Scenario patterns
|
|
82
|
+
|
|
83
|
+
| Pattern | Primitives | Why |
|
|
84
|
+
| ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
85
|
+
| Sales CRM: staff qualify and convert leads that arrive through a public site | Lead and Opportunity [Knowledge schemas](../data/knowledge.md), an [external application](../develop/external-apps.md) with a BFF, the convert-lead [workflow](../ai/index.md), a lead qualifier agent with a Skill and a scoring tool, a Sales desk [Studio app](../develop/extensions.md) | People edit leads in the web app, staff read them in Studio, and the agent reads the same records. This is the [template](../template/index.md). |
|
|
86
|
+
| Data kept in the customer's database: the customer will not move its records | An external application and a [Gateway](../develop/external-apps.md) that exposes entity CRUD, a read-only dataset query and one business operation | CI reaches the records in place, so nothing is copied into Knowledge |
|
|
87
|
+
| Existing backend logic for agents: a service already implements the business rule | A Gateway with a remote tool collection | One service answers its own API, a CI operation binding and an agent tool, so its code never moves into a package |
|
|
88
|
+
| Document intake: people upload a document and expect structured data back | A stored file, a workflow started with `storageFileId`, an approval node, a Studio upload form | Extraction is a fixed sequence, so a workflow runs it. A website upload and a Studio upload form start the same workflow with the same input, and the app server settles the result. |
|
|
89
|
+
| Member portal: members sign in on a public site, and staff run the back office in Studio | Knowledge schemas with data policies, [Access Profiles](../platform/access.md), an external application, Studio apps, agents for member questions and matching, workflows for enrichment, scheduled jobs and approvals, app-owned storage for activity logs | Records that members and staff both edit live in Knowledge, where a data policy can hide a flagged record from everyone. An `entity-updated` workflow asks the app to re-read changed records, and activity logs stay in the app database. |
|
|
90
|
+
| Collaborative tracker: several people edit plans, actions and comments at the same time | Knowledge schemas with relations, Realtime and optimistic writes in the external application | An edit in one window appears in the others without a reload ([state machines](../template/state-machines.md) shows how), and the first demo needs no AI primitive |
|
|
91
|
+
| App database as the authority: the app owns the records, but staff want AI advice | A Knowledge schema that holds a projection of non-personal fields, an advisory agent | Contacts and other personal data never leave the app, and staff decide what to do with the agent's advice |
|
|
92
|
+
| Pure mock demo: a first visual pitch before any integration | No CI primitive; a state machine with simulated saves in Effect | The screens can be shown before CI work starts, and CI primitives come later |
|
|
93
|
+
|
|
94
|
+
Connectors, datasets declared in a package and Document Collections appear in the criteria above because the platform supports them, but no shipped demo pattern uses them yet.
|
|
95
|
+
|
|
96
|
+
Demos built from the template started with Knowledge schemas, access and an external application, and they added agents and workflows only where the scenario asked for them; one demo shipped with no CI primitive at all.
|
|
97
|
+
|
|
98
|
+
## Related pages
|
|
99
|
+
|
|
100
|
+
- [Knowledge](../data/knowledge.md)
|
|
101
|
+
- [Datasets](../data/datasets.md)
|
|
102
|
+
- [Connectors](../data/connectors.md)
|
|
103
|
+
- [AI](../ai/index.md)
|
|
104
|
+
- [External apps](../develop/external-apps.md)
|
|
105
|
+
- [Extensions](../develop/extensions.md)
|
|
106
|
+
- [Access](../platform/access.md)
|
|
107
|
+
- [The template](../template/index.md)
|