@collective-intelligence/extensions-sdk-frontend 0.2.6 → 0.2.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,156 @@
1
+ # Datasets
2
+
3
+ Query typed tables of measurements, events and warehouse data, and control who can read them.
4
+
5
+ A Dataset is a typed collection of rows that belongs to an Organization. Each Dataset has a stable identity, a versioned definition, a policy that says who may query it, and provenance on every result. Datasets hold data that has many rows and few edits, such as meter readings, monthly sales figures or a table in a customer's data warehouse. Agents, workflows and Studio query that data in bounded, typed requests.
6
+
7
+ ## Dataset or Knowledge schema
8
+
9
+ Knowledge models things that people talk about, edit, link and search, with one record per thing. A Dataset models observations about those things, with one row per measurement, event or transaction. The platform does not create a Knowledge record for each row.
10
+
11
+ | | Knowledge schema | Dataset |
12
+ | -------------------- | ------------------------------------------------------- | -------------------------------------------------------------- |
13
+ | One row or record is | A real thing: a customer, a Lead, a goal | An observation: a reading, a transaction, an event |
14
+ | People edit rows | Yes, in forms, Studio and the API | No; rows come from file imports or from the source system |
15
+ | Links | Relation fields between records | An optional binding from a Dataset field to a Knowledge schema |
16
+ | Queries | Field filters, text and semantic search, relation paths | `records`, `aggregate` and `timeSeries` |
17
+ | Access | Knowledge data policies across schemas and domains | One policy per Dataset with an explicit list of principals |
18
+
19
+ A strategy Metric shows the split. The Metric itself is a Knowledge record with an owner, a target, a unit, a status and links to goals, and its history of measured values is a Dataset. A Dataset field such as `customerId` can bind to the Customer schema in Knowledge. That binding links the two without copying rows into Knowledge, and deleting one side does not delete the other.
20
+
21
+ ```mermaid
22
+ flowchart LR
23
+ subgraph K[Knowledge]
24
+ KS[Customer schema]
25
+ end
26
+ subgraph D[Datasets]
27
+ DS[Orders Dataset]
28
+ end
29
+ K --- KC[Records, relation fields, semantic search]
30
+ D --- DC[Typed rows, measures, aggregates, time buckets]
31
+ DS -. optional binding on customerId .-> KS
32
+ ```
33
+
34
+ ## Managed and External Datasets
35
+
36
+ A Dataset has one of two backings. CI stores the rows of a Managed Dataset in PostgreSQL, in the Organization Dataset Store. The rows of an External Dataset stay in the customer's system, and CI queries them live. Both backings use the same query contract, so tools, workflows and Studio do not need to know which one they read. The backing shows only in the provenance, the capabilities and the freshness of a result.
37
+
38
+ CI keeps no hidden copy of External data. Every query reaches the provider and CI caches no results, so when the provider is down the query fails with no fallback.
39
+
40
+ | Provider | Backing | Source |
41
+ | ----------------------------------------------------------------------- | -------- | ---------------------------------------------------------- |
42
+ | `builtin.core.datasets.providers.postgres` | Managed | Rows stored in CI |
43
+ | Microsoft Fabric (package `builtin.integrations.microsoft-fabric`) | External | Fabric Warehouses and Lakehouse SQL analytics endpoints |
44
+ | Databricks (package `builtin.integrations.databricks`) | External | Unity Catalog tables and views through one SQL Warehouse |
45
+ | Remote Gateway (`builtin.core.external-apps.providers.remote-external`) | External | A customer server that exposes rows through the server SDK |
46
+
47
+ The Fabric and Databricks packages do not install automatically, so install the package before you declare a Dataset that uses it. A Fabric connection holds one service principal from the customer's Microsoft Entra tenant, with a tenant ID, a client ID and a client secret or certificate. A Databricks connection uses service principal OAuth and rejects personal access tokens. A Fabric source location has exactly three parts: workspace, item and object. To combine several sources, combine them in the customer's warehouse and point the Dataset at the result. New Lakehouse tables can take a few minutes to appear on the Fabric SQL analytics endpoint, so refresh the metadata in Fabric if a new table is missing.
48
+
49
+ A Remote Gateway Dataset answers `records` queries only, with at most 1,000 rows, 1 MiB and 60 seconds per request. CI checks the Dataset policy first, and the customer server can narrow the result further but cannot widen it.
50
+
51
+ ## Load a Managed Dataset
52
+
53
+ In the Dashboard you create a Dataset in one of two ways: import a structured file as a Managed Dataset, or connect a compatible External source. A Managed import accepts CSV, Excel `.xlsx`, JSON arrays of records, JSON Lines and Parquet. CI inspects the file and proposes field types, identity fields and candidate time fields, and a person reviews and approves the proposal before CI loads any rows. There is no spreadsheet-style row editor.
54
+
55
+ Ingestion runs in the background. The API answers `202 Accepted` with a run ID, and you follow the run until it finishes. Each run uses one mode: `append` adds rows, `upsert` inserts or updates rows by their identity fields, and `replace` swaps all visible rows in one step. If an `upsert` run contains the same identity twice, CI rejects the run and does not pick one row for you.
56
+
57
+ The table template of the Dataset decides which modes it allows.
58
+
59
+ | Template | Purpose | Modes |
60
+ | ----------------- | -------------------------------- | ----------------------------- |
61
+ | `generic-table` | General typed records | `append`, `upsert`, `replace` |
62
+ | `entity-snapshot` | Current state keyed by entity | `upsert`, `replace` |
63
+ | `time-series` | Time-indexed measures | `append` |
64
+ | `event-log` | Append-only events | `append` |
65
+ | `fact-table` | Dimensions, measures and periods | `append`, `upsert` |
66
+ | `status-update` | Append-only status history | `append` |
67
+
68
+ ## Query a Dataset
69
+
70
+ A Dataset query has one of three kinds. `records` returns selected rows with filters, sort and pagination. `aggregate` groups measures by dimensions with `count`, `countDistinct`, `sum`, `average`, `minimum` and `maximum`. `timeSeries` buckets measures over a time field, with a grain from `minute` to `year`.
71
+
72
+ ```ts
73
+ type RecordsQuery = Readonly<{
74
+ kind: "records"
75
+ datasetId: string
76
+ definitionRevision?: string
77
+ select: readonly string[]
78
+ filters?: DatasetFilter
79
+ sort?: readonly DatasetSort[]
80
+ page: DatasetPageRequest
81
+ }>
82
+ ```
83
+
84
+ A Managed `records` query returns at most 50 fields, 100 rows and 1 MiB per response. Filters are combined with AND and use `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `contains` and `isNull`. CI returns 64-bit integers and decimals as exact JSON strings, so parse them before you do arithmetic.
85
+
86
+ Datasets accept no SQL. Calculated measures use a constrained expression model, and an operation that the provider cannot run fails with a typed capability error. CI never loads a whole table to compute a result in memory.
87
+
88
+ > **Note:** The External providers that ship today answer `records` and `aggregate`, and
89
+ > the Remote Gateway answers `records` only. Use a Managed Dataset when the demo
90
+ > needs `timeSeries`.
91
+
92
+ Every result carries its provenance: the Dataset ID, the definition revision, the query kind, the backing, the freshness when the provider knows it, a correlation ID, the timing, and warnings for partial or stale results. Provenance never contains credentials or physical table names.
93
+
94
+ ## Access
95
+
96
+ Each Dataset has its own policy, and the policy lists its principals by ID. It holds `principals` (each a `user` or `service` with an `id`), `allowedOperations` (`describe`, `records`, `aggregate`, `time_series`), `allowedFields`, `sensitiveFields`, an optional `rowFilter`, and the limits `maxRows`, `maxBytes` and `maxTimeoutMs`. A user also needs the Organization permission to query Datasets, and a Dataset grant does not give Organization membership or capabilities.
97
+
98
+ Knowledge access works differently: Knowledge data policies apply across schemas and domains. The [Access](../platform/access.md) page explains both models.
99
+
100
+ > **Warning:** A package-declared Dataset without a `dataPolicy` grants access to nobody,
101
+ > including the person who deployed it. Always declare the principals that the
102
+ > demo needs.
103
+
104
+ ## Declare a Dataset in a package
105
+
106
+ `declareDatasetConnectorInstance` declares a Dataset in the demo package. An External declaration names the provider, a connection slot, the source location and the policy:
107
+
108
+ ```ts
109
+ const orders = declareDatasetConnectorInstance({
110
+ manifest: { fqn: "com.acme.sales.instances.orders", version: "1.0.0" },
111
+ presentation: { displayName: "Warehouse orders" },
112
+ definition: {
113
+ lifecycle: { deployMode: "init_only", readonly: false },
114
+ bindingMode: "external",
115
+ providerFqn: "com.acme.integrations.warehouse",
116
+ connection: { setupKey: "warehouse" },
117
+ sourceLocation: {
118
+ segments: [
119
+ { kind: "container", opaqueId: "sales", label: "Sales" },
120
+ { kind: "dataset", opaqueId: "table:orders", label: "Orders" }
121
+ ]
122
+ },
123
+ dataPolicy: {
124
+ principals: [{ kind: "user", id: "7f824c46-..." }],
125
+ allowedOperations: ["describe", "records"],
126
+ allowedFields: ["id", "total"],
127
+ maxRows: 1000,
128
+ maxBytes: 1048576
129
+ }
130
+ }
131
+ })
132
+ ```
133
+
134
+ A deploy configures Datasets and loads no data. An External Dataset is ready to query live once its connection works, and its optional schedules run health checks or propose definition changes for review. A Managed declaration creates an empty Dataset, and loading rows stays a manual import. CI refuses a Managed declaration when the Organization Dataset Store is not enabled. Removing the package never deletes Managed rows.
135
+
136
+ ## Datasets for agents and workflows
137
+
138
+ Agents use one tool, Query Dataset (`builtin.core.datasets.tools.query`). It lists, describes and queries the Datasets that an agent may use, with bounded `records`, `aggregate` and `timeSeries` requests. You choose those Datasets by ID in the Skill binding, and an empty selection gives the agent no Datasets. The agent never passes SQL, provider names, connections or table locations. The [AI](../ai/index.md) page explains Skills and tool bindings.
139
+
140
+ Workflows have one node for each operation: describe, query records, query aggregate and query time series. Studio shows Datasets in their own area, apart from the Knowledge views.
141
+
142
+ ## Pick the right tool for a demo
143
+
144
+ - The customer has a warehouse in Fabric or Databricks: use an External Dataset and query it live.
145
+ - The customer has a CSV or Excel export and wants to see trends: import it into a Managed Dataset.
146
+ - Users must edit records and link them to each other: use Knowledge.
147
+
148
+ Datasets are sized for about 100,000 to 1,000,000 data points per Organization and do not fit a high-volume telemetry stream. Joins across Datasets, writes back to External Datasets, streaming ingestion and copying External data into a Managed Dataset are not available. A route for external applications to query Datasets directly is planned; today agents, workflows and Studio run the queries.
149
+
150
+ ## Related pages
151
+
152
+ - [Knowledge](knowledge.md)
153
+ - [Connectors](connectors.md)
154
+ - [Access](../platform/access.md)
155
+ - [AI](../ai/index.md)
156
+ - [Extensions](../develop/extensions.md)
@@ -0,0 +1,211 @@
1
+ # Knowledge
2
+
3
+ Declare schemas, store typed records, query them, and read and watch them from a demo app.
4
+
5
+ Knowledge stores the business records of a demo, such as Leads and Opportunities. A schema defines the fields that a record type can have, and a record is one instance that matches that schema. Schemas live in a Knowledge domain, a named container that the Organization owns. In the template, the extension package `acme.app.knowledge` declares one domain with a Lead schema and an Opportunity schema, and the demo app reads and writes Lead and Opportunity records through the SDKs.
6
+
7
+ ```mermaid
8
+ flowchart LR
9
+ P[Extension package] --> D[Knowledge domain]
10
+ D --> L[Lead schema]
11
+ D --> O[Opportunity schema]
12
+ L --> LRec[Lead records]
13
+ O --> ORec[Opportunity records]
14
+ ```
15
+
16
+ Schemas that a package or a CLI deploy creates have the definition type `custom`, and a custom schema can point a relation field at one of the platform's `builtin` schemas.
17
+
18
+ ## Field kinds
19
+
20
+ A schema combines ordinary value fields with three special kinds. Each special kind has its own declaration block, and the platform generates its value schema.
21
+
22
+ | Kind | Declared in | Stored value | Use it for |
23
+ | -------------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
24
+ | Value field | `schema` properties | `string`, `number` (with `numberKind: "integer"` for whole numbers), `datetime`, `string_select` with `options` | Names, amounts, dates, status values |
25
+ | Relation field | `fieldSemantics["/<key>"].relation` | One record ID or an array of record IDs | Links between records, such as Opportunity to Lead |
26
+ | User field | `userFields` | One user ID or an ordered list of user IDs | Owners, reviewers, the users who act for a record |
27
+ | File field | `fileFields` | A File value with `id`, `url`, `filename`, `mimeType`, `size` and `source` | Attachments such as a signed PDF |
28
+
29
+ ## Declare a schema in a package
30
+
31
+ A demo declares its schemas in code, inside its extension package. This is the template's Lead schema, shortened:
32
+
33
+ ```ts
34
+ declareKnowledgeEntitySchema({
35
+ definition: {
36
+ domainFqn: domain.manifest.fqn,
37
+ fileFields: {
38
+ attachment: {
39
+ accepts: ["application/pdf", "image/jpeg", "image/png"],
40
+ maxBytes: 5 * 1024 * 1024,
41
+ multiple: false,
42
+ visibility: "private"
43
+ }
44
+ },
45
+ lifecycle: { deployMode: "upsert", readonly: true },
46
+ schema: knowledgeRecordSchema({
47
+ additionalProperties: false,
48
+ properties: {
49
+ company: { required: true, type: "string" },
50
+ email: { required: true, type: "string" },
51
+ name: { minLength: 1, required: true, type: "string" },
52
+ score: { numberKind: "integer", type: "number" },
53
+ status: {
54
+ options: ["converted", "open", "qualified"],
55
+ type: "string_select"
56
+ },
57
+ createdAt: { type: "datetime" }
58
+ }
59
+ })
60
+ },
61
+ manifest: { fqn: "acme.app.schemas.lead", version },
62
+ presentation: {
63
+ displayName: "Lead",
64
+ description: "A prospective Acme customer."
65
+ }
66
+ })
67
+ ```
68
+
69
+ Each declared item needs a `lifecycle`. With `deployMode: "upsert"`, every deploy reconciles the schema with the code, and `init_only` creates the schema once and then leaves it alone. With `readonly: true`, Dashboard administrators cannot edit the definition, so the code stays the only source of truth. Choose `readonly: true` when the demo code owns the schema and you redeploy to change it, and `readonly: false` when a customer administrator should keep editing the schema in the Dashboard after the deploy.
70
+
71
+ The template also has a Lead entity contract in `domain/lead.ts`. That contract types the resource for the demo app (its capabilities, the fields you can filter, sort and search on, and the create and update shapes) and never reaches CI. CI receives only the schema above, so when you add a field, change both files.
72
+
73
+ The CLI puts the schema into an Organization in five steps: `ci package build`, `validate`, `publish`, `promote` and `install --organization <id>`. The [Extensions](../develop/extensions.md) page shows the full commands.
74
+
75
+ ## Relation fields
76
+
77
+ A relation field holds references to records of exactly one target schema. Declare it under `fieldSemantics` with a target schema and a cardinality, and leave the property out of the value schema because the platform generates its value schema:
78
+
79
+ ```ts
80
+ fieldSemantics: {
81
+ "/companies": {
82
+ relation: {
83
+ targetSchemaFqn: "core.schemas.crm.company",
84
+ cardinality: "many_to_many",
85
+ onDelete: "restrict",
86
+ },
87
+ },
88
+ }
89
+ ```
90
+
91
+ Cardinality reads from the source record to the target record. "One" means at most one, so a link is never required.
92
+
93
+ | Cardinality | Targets per source | Sources per target |
94
+ | -------------- | ------------------ | ------------------ |
95
+ | `one_to_one` | At most one | At most one |
96
+ | `one_to_many` | Many | At most one |
97
+ | `many_to_one` | At most one | Many |
98
+ | `many_to_many` | Many | Many |
99
+
100
+ To show the same links from the other side, declare an inverse field on the target schema with `inverseOf: { schemaFqn, field }`. An inverse field is read-only, and a write to it fails, even a write of `null` or an empty array. `onDelete` controls what happens when someone physically deletes a target record: `clear` (the default) removes the reference, and `restrict` refuses the delete. A logical delete keeps the references.
101
+
102
+ > **Warning:** If you declare `Company.contacts` and `Contact.company` as two forward fields,
103
+ > you get two separate relationships that do not stay in sync. Declare one
104
+ > forward field and make the other side its inverse.
105
+
106
+ In the template, Opportunity stores `leadId` as a plain required string. That is enough to display the ID. If your demo needs to navigate from an Opportunity to its Lead, expand the Lead in one query, or filter on `lead.company`, replace `leadId` with a `many_to_one` relation field that targets the Lead schema.
107
+
108
+ ## User fields and file fields
109
+
110
+ A User field stores the ID of one Organization member or External App User, or an ordered list of them. The platform resolves names, emails and avatars when it displays the field and never stores them. Selecting a user in the field does not assign work, send a notification or grant access.
111
+
112
+ ```ts
113
+ userFields: {
114
+ owner: { multiple: false },
115
+ reviewers: { multiple: true },
116
+ }
117
+ ```
118
+
119
+ One User field per schema can set `accounts: true`. The users in that field act for the record, and access rules can test it with `isMe("accounts")` or `includesMe("accounts")`, for example to let a user write only the records that list them. The platform owns this field: package policies, app users and agent tools cannot change it, but a workflow `add-account` step can. The [Access](../platform/access.md) page explains the rules that read it.
120
+
121
+ A file field stores files. Its read value is `{ id, url, filename, mimeType, size, source }`, where `source` is `managed` when CI stores the bytes and `external_url` when the file stays at a remote URL. To upload, the app asks CI for an upload slot. The slot is single use and belongs to one file field. CI refuses an upload larger than `maxBytes`, and the template also checks the size in the browser so that it sends nothing too large. To update the Lead attachment, the template sends `{ upload }` for a new file, `{ id }` to keep the current file, or `null` to remove it.
122
+
123
+ ## Query records
124
+
125
+ Field filters narrow records with `{ field, operator, value }` and operators such as `eq`, `in`, `exists` and `is_empty`, and text search matches plain text. Semantic search ranks records by embeddings, so it works only on a schema with embeddings enabled, and each result carries its `match.semantic_score`.
126
+
127
+ The schema controls embeddings. `embedEntities` turns on one combined vector per record, `fieldSemantics["/field"].includeInItemEmbedding` selects the fields that go into that vector, and `fieldSemantics["/field"].embedded` gives a long text field its own passage vectors. When you change these settings through a deploy or an admin edit, CI reprocesses the existing records automatically. You do not need a new import or an Organization reset.
128
+
129
+ > **Warning:** `embedEntities` alone gives no semantic search over records. Select at least
130
+ > one field with `includeInItemEmbedding`, or the schema has no record vector
131
+ > and shows a configuration status instead.
132
+
133
+ A filter path such as `company.industry` follows declared relation fields for at most three hops, and `include` returns the related records in an `included` tree beside the references. Includes go at most three levels deep and return at most 100 readable records per field, with a `truncated` marker when they stop. Each included record passes the same read check as a direct read, so `include` never shows a record that the caller cannot read. Relevance-ranked search rejects filters on relation fields.
134
+
135
+ > **Note:** Knowledge has no graph database and no graph traversal. Older material that
136
+ > describes graph edges or relation bindings refers to a retired model. Relation
137
+ > paths and `include` follow the relation fields that the schema declares.
138
+
139
+ ## Retrieval for agents
140
+
141
+ Agents and workflows read Knowledge through built-in tools. Knowledge Query (`builtin.core.knowledge.tools.query`) reads Knowledge only, and Documents Query (`builtin.core.documents.tools.query`) searches Documents only. Advanced Information Retrieval (`builtin.core.information.tools.advanced-retrieval`) is the only tool that searches both, and it does so by default.
142
+
143
+ ```mermaid
144
+ flowchart LR
145
+ K[Knowledge Query] --> KP[Knowledge policy and record filters]
146
+ D[Documents Query] --> DP[Documents policy and collection paths]
147
+ A[Advanced Information Retrieval] --> KP
148
+ A --> DP
149
+ KP --> R[PostgreSQL ranking]
150
+ DP --> R
151
+ R --> C[Live authorization recheck]
152
+ C --> E[Results]
153
+ ```
154
+
155
+ PostgreSQL holds the source of truth and runs semantic search with pgvector. The tool input has no field for a provider, a vector or a ranking setting, so an agent cannot change how CI routes the query. Before CI returns a result, it checks the current policies again, which means that a revoked grant takes effect on the next query. The [AI](../ai/index.md) page covers tools and Skills.
156
+
157
+ ## Documents and Knowledge
158
+
159
+ Knowledge holds typed records with fields. Documents holds files in collections and folders, with versions and permissions per path. A Document Collection is either Local, where people upload files, or Connected, where a Documents connector keeps it in sync with folders in Microsoft 365 or Google Drive.
160
+
161
+ The two areas share storage and processing. A Knowledge file field stores its file through Documents, in an internal collection that Documents browse and search do not show. A reader reaches that file through the Knowledge record, and Advanced Information Retrieval returns it when the reader can read the record. Both areas also use one extraction and embedding pipeline, and CI reuses one processing result when the same bytes appear twice in an Organization.
162
+
163
+ A finished sync does not mean that the new files are searchable yet, because indexing runs after the sync. A workflow Wait at level `sync` waits for the import only, while level `indexing` also waits for indexing. Use `indexing` before a demo step that searches the new content. The [Connectors](connectors.md) page compares Knowledge sync with Documents sync.
164
+
165
+ ## Read and write from an app
166
+
167
+ A demo app reaches records in two ways, and the template uses both.
168
+
169
+ | | Browser | Server |
170
+ | ----------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
171
+ | API | Entity hooks from `defineEntity({ contract, config })` in `@collective-intelligence/external-apps-client-sdk-react/resources` | `ServerClient.records` from `@collective-intelligence/external-apps-server-sdk/effect` |
172
+ | Acts as | The signed-in user | The app credential (`CI_CLIENT_ID`, `CI_CLIENT_SECRET`) |
173
+ | Operations | Whatever the entity contract allows: create, get, search, update, delete, upload | `create`, `get`, `search`, `update`, `delete` |
174
+ | Typical use | Forms, lists and tables | Server routes and the template's Read Port |
175
+
176
+ The server client takes the Organization and the schema on each call, and the schema FQN works as `schemaId`. A missing record fails with the `CiError` code `not_found`. This is the template's server read, without the wrapper that turns `not_found` into an empty result:
177
+
178
+ ```ts
179
+ const record =
180
+ yield *
181
+ client.records.get({
182
+ id,
183
+ organizationId: options.organizationId,
184
+ schemaId: leadSchemaFqn
185
+ })
186
+ const fields = yield * decodeLead(record.value)
187
+ return { ...fields, id: record.id }
188
+ ```
189
+
190
+ `records.search` accepts a filter, text search, sort, pagination and `include`. App filtering goes through `POST /application/v1/resources/:schemaId/search`, and relation field pickers use `POST /application/v1/knowledge/autocomplete`. The SDKs do not wrap semantic search; agents use the retrieval tools for that.
191
+
192
+ > **Warning:** The server reads with the app credential, so that credential itself needs read
193
+ > access to Leads. CI accepts a user's token only when the user holds the
194
+ > application gate. See [Access](../platform/access.md).
195
+
196
+ ## Change events
197
+
198
+ Every write to a record produces a change event, whether the write comes from the demo app, Studio, an agent or a workflow. Database triggers write a change log entry for each changed record, with one entry per record per transaction. A sync that changes nothing writes no entry.
199
+
200
+ An app subscribes to a topic per schema, for example `{ schema: "acme.app.schemas.lead" }`. `useRealtimeTopic(topic, changes => ...)` receives batches of changes that carry record IDs only, and `useEntityChanges(contract)` gives a function that refreshes the entity reads for those IDs. CI checks read access for each change before it sends it. When CI cannot decide whether a reader may see a delete, it sends that reader a `reset` instead of the ID, and the client must then refetch.
201
+
202
+ To see this in the template, open `/crm` in two browser windows and edit a Lead in one of them as a Sales manager. The other window shows the change without a reload, and it does the same when the change comes from Studio, an agent or the convert-lead workflow.
203
+
204
+ ## Related pages
205
+
206
+ - [Datasets](datasets.md)
207
+ - [Connectors](connectors.md)
208
+ - [Access](../platform/access.md)
209
+ - [SDKs](../develop/sdks.md)
210
+ - [Extensions](../develop/extensions.md)
211
+ - [AI](../ai/index.md)
@@ -0,0 +1,208 @@
1
+ # Extensions
2
+
3
+ What a package declares, how the template wires it, how to deploy it with ci deploy, and the rules its code runs under.
4
+
5
+ A package is a TypeScript module whose default export is `declarePackage({ manifest, components, hostedContributions })`. The `ci` CLI builds that module into a release, which is one immutable published version of the package. An administrator then installs the release into one Organization, and the installation records which release that Organization runs.
6
+
7
+ The template's package declares the Lead and Opportunity schemas, the Sales group tree, the external application of the web app, Sales Access Profiles and grants, the Max Privileges of the application's tokens, the convert-lead workflow, the lead qualifier Agent with its Skill and tool, and the Sales desk Studio app. [SDKs](sdks.md) explains which `@collective-intelligence/*` package each part imports.
8
+
9
+ ## What a package can declare
10
+
11
+ | Declaration | What it creates in the Organization |
12
+ | ---------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
13
+ | `declareDomain`, `declareKnowledgeEntitySchema` | A Knowledge domain and the schemas of its records |
14
+ | `declareTool`, `declareToolCollection` | Tools that Agents and workflows call, grouped in collections |
15
+ | `declareSkill`, `declareAgent` | Skills and the Agents that hold them |
16
+ | `declareWorkflow`, `declareWorkflowNode` | Workflows and custom workflow nodes |
17
+ | `declareStudioApp` | A Declared Studio App that CI creates and publishes on install |
18
+ | `declareStudioAppTemplate` | A starting point that a builder copies into a new Studio app |
19
+ | `declareStudioComponent` | A Studio component, such as a widget of category `widget` |
20
+ | `declareConnection`, `declareConnectionInstance` | Connections to outside systems and their configured instances |
21
+ | `declareKnowledgeConnector`, `declareKnowledgeConnectorInstance` | Knowledge connectors and configured connector instances |
22
+ | `declareDocumentCollection` | One connected document collection, such as a Microsoft 365 drive folder |
23
+ | Dataset parsers, providers and blueprints | The dataset building blocks of the package |
24
+ | `declareOrganizationGroup` | A group tree with subgroups, created empty |
25
+ | `declareAccessCapabilities`, `declareAccessProfile`, `declareAccessProfileGrant` | Capabilities, Access Profiles, and Access Grants of a profile to a group |
26
+ | `declareAccessMaxPrivileges` | The Max Privileges of the application's user tokens |
27
+ | `declareOrganizationSettings` | Identity, branding, authentication, locale and governance settings (at most one per package) |
28
+ | `declareExternalApplication` | The OAuth client of your web app, with origins and redirect URIs |
29
+ | Event listeners, notification channels, email providers, Organization SSO, Organization Instructions | The matching Organization features |
30
+
31
+ Executable kinds also take an `implementation` class from `@collective-intelligence/extensions-sdk-backend`. Domains, schemas, policies, collections and Studio apps carry no class.
32
+
33
+ | Declaration | Class method that CI calls |
34
+ | ---------------------------- | ------------------------------------------- |
35
+ | `declareTool` | `PackageTool.execute` |
36
+ | `declareConnection` | `PackageConnection.test` |
37
+ | `declareKnowledgeConnector` | `pull` and `push` |
38
+ | `declareWorkflowNode` | `execute` |
39
+ | `declareEventListener` | Selectors plus `normalizeWebhook` or `poll` |
40
+ | `declareNotificationChannel` | `deliver` |
41
+
42
+ A component refers to another component of the same package through the handle that its `declare*` call returned. For a built-in component or one from another package, pass the FQN string instead, for example `"builtin.core.knowledge.tools.query"`. TypeScript does not check these strings. Install resolves them and rolls the package back when one does not exist, and the SDK publishes no list of built-in FQNs, so the template pins its FQNs in a `bun test` that catches a typo before deploy.
43
+
44
+ A package that declares Agents or Skills needs an administrator to install it. A package can depend on another package by FQN only, for example `dependencies: [{ fqn: "com.acme.base" }]`, and each install picks the latest eligible release of the dependency by promotion order.
45
+
46
+ ## How the template wires the package
47
+
48
+ ```mermaid
49
+ flowchart TD
50
+ ext["apps/extensions (@acme/extensions)<br/>index.ts: declarePackage<br/>studio.ts: loadFrontendCatalog"]
51
+ org["@acme/organization<br/>package identity, VERSION, domain,<br/>Sales group, external application,<br/>access, Studio app, packageComponents"]
52
+ lead["@acme/lead/ci<br/>Lead schema"]
53
+ opp["@acme/opportunity/ci<br/>Opportunity schema"]
54
+ qual["@acme/lead-qualification/ci<br/>tool, Skill, Agent, widget"]
55
+ conv["@acme/lead-conversion/ci<br/>convert-lead workflow"]
56
+ ext --> org
57
+ org --> lead
58
+ org --> opp
59
+ org --> qual
60
+ org --> conv
61
+ ```
62
+
63
+ The template keeps its CI code in three layers. `apps/extensions` is the composition root. It imports `@acme/organization` and one FQN from a `./ci` subpath. `@acme/organization` holds the package identity and everything shared across features, and it lists `packageComponents` in deploy order. Each entity or feature package exports a factory from its `./ci` subpath, for example `leadQualificationComponents({ application, audience, version })`, which takes the organization handles and the package version and returns its components in deploy order. A lower layer never imports `@acme/organization`.
64
+
65
+ This is the whole package entry:
66
+
67
+ ```ts
68
+ import { leadScoreCardContributionFqn } from "@acme/lead-qualification/ci"
69
+ import { packageComponents, packageManifest } from "@acme/organization"
70
+ import { declarePackage } from "@collective-intelligence/extensions-sdk"
71
+ import { declareFrontendCatalogContribution } from "@collective-intelligence/extensions-sdk-frontend"
72
+
73
+ export default declarePackage({
74
+ components: packageComponents,
75
+ hostedContributions: [
76
+ declareFrontendCatalogContribution({
77
+ compatibility: { react: "^19.0.0" },
78
+ fqn: leadScoreCardContributionFqn,
79
+ host: "studio",
80
+ load: async () => await import("./studio")
81
+ })
82
+ ],
83
+ manifest: packageManifest
84
+ })
85
+ ```
86
+
87
+ `apps/extensions/src/studio.ts` contains one line, `export { loadFrontendCatalog } from "@acme/organization"`. `ci package build` ships every export of that module to the browser, so it exports only the catalog loader.
88
+
89
+ One entity package such as `@acme/lead` holds the CI schema in `./ci`, the BFF code in `./server` and the React views in `./views`. The lint rule `ci-solution/subpath-imports` keeps React out of the CI release: `domain/` imports no other subpath, `ci/` imports no `headless/` or `views/`, and `headless/` and `views/` import no `server/`.
90
+
91
+ The template names components `acme.app.<kind>.<name>`, for example `acme.app.tools.lead-score` or `acme.app.workflows.convert-lead`, and the package itself is `acme.app.knowledge`. Each `./ci` subpath exports its FQN constants, and other code imports the constant instead of typing the string again.
92
+
93
+ A widget needs two declarations. On the backend side, `declareStudioComponent` with `category: "widget"`, a `propsSchema` and a `sourceRef` goes into `components`. On the frontend side, `declareFrontendComponent` with a `load` callback and a `studio_component` target is returned from `loadFrontendCatalog`. The `sourceRef` names the Studio hosted contribution of the package.
94
+
95
+ To add a component to the template:
96
+
97
+ 1. Declare it in the `./ci` subpath of the package that owns it.
98
+ 2. Add it to `packageComponents` after every component it references.
99
+ 3. Bump `VERSION` in `packages/organization/acme/src/package.ts`. The package and every component share this one version.
100
+
101
+ ## Lifecycle
102
+
103
+ Every content component sets `lifecycle: { deployMode, readonly }`.
104
+
105
+ | Option | Effect |
106
+ | ------------------------- | -------------------------------------------------------------------------------------------- |
107
+ | `deployMode: "upsert"` | Each deploy brings the definition back in line with the package. |
108
+ | `deployMode: "init_only"` | The first deploy creates the definition. Later deploys leave an administrator's edits alone. |
109
+ | `readonly: true` | Nobody can edit the definition by hand. Records of a schema stay editable. |
110
+
111
+ The template uses `upsert` with `readonly: true` for almost everything. The convert-lead workflow and the Sales reps grant use `init_only` with `readonly: false`, so an administrator can change them after install and an update keeps that change.
112
+
113
+ ## Package variables
114
+
115
+ `packageVariables` declares values that change between environments and returns one typed handle per variable. You put the handle in a component definition where a value would go. The manifest stores `{ "$variable": "<name>" }` in its place, so the release digest is the same in every environment and CI substitutes the value at deploy.
116
+
117
+ ```ts
118
+ const variables = {
119
+ collection: {
120
+ type: "string",
121
+ default: "customers",
122
+ description: "Remote CRM collection"
123
+ },
124
+ companyDomains: { type: "string[]" },
125
+ crmToken: { type: "secret" }
126
+ } as const satisfies PackageVariablesDeclaration
127
+
128
+ const vars = packageVariables(variables)
129
+ // remoteCollectionId: vars.collection
130
+
131
+ export default declarePackage({ components, manifest, variables })
132
+ ```
133
+
134
+ A variable has a type of `string`, `number`, `boolean`, `string[]`, `json` or `secret`. A variable without a default is required at deploy, and a `secret` cannot have a default. When a deploy provisions a secret without a value, the install still succeeds, but runtime use fails until an administrator sets it.
135
+
136
+ A handle cannot go in `fqn`, `version`, `kind`, `lifecycle`, a schema projection, or a reference to another component by FQN such as `schemaFqn` or `templateFqn`. The package FQN cannot be a variable either.
137
+
138
+ Connection slots work the same way at deploy. A component names a `{ setupKey }`, and the Inputs File gives either an existing `connectionId` or a `newConnection` with a name and fields. Installing a connector instance creates no records and starts no import. [Connectors](../data/connectors.md) covers instances and slots in detail.
139
+
140
+ ## Deploy with ci deploy
141
+
142
+ `ci deploy` runs the whole release path for one declaration:
143
+
144
+ 1. Build and validate the declaration.
145
+ 2. Find the exact immutable release that matches the build.
146
+ 3. Publish and promote the release only when CI does not have it yet.
147
+ 4. Install it into the Organization, or update the existing installation.
148
+ 5. Follow every operation until it succeeds.
149
+
150
+ ```bash
151
+ ci deploy --backend-url https://ci.example.com --organization <organization-id> \
152
+ --declaration ./src/package.ts --non-interactive --json
153
+ ```
154
+
155
+ `ci deploy` reads two credentials. `CI_PUBLISHER_TOKEN` publishes and promotes, and `CI_MANAGEMENT_TOKEN` installs and updates in the Organization. If `CI_MANAGEMENT_TOKEN` is an Organization API key (prefix `dpk_`), you can omit `--organization`. `ci deploy --inputs-template` prints every connection slot and variable without contacting the backend.
156
+
157
+ A repeated deploy skips work that is already complete, and a new version updates the existing installation. CI rejects changed bytes under the same version and silent downgrades. Pass `--accept-breaking-changes` to acknowledge a breaking change. The default deadline is 300,000 ms, and `--timeout-ms` changes it.
158
+
159
+ > **Warning:** `--declaration` must start with `./` or be an absolute path, because the CLI
160
+ > resolves a bare `packages/...` like a module name and fails. A package with a
161
+ > Studio contribution waits until a Studio reports ready, and an unattended
162
+ > pipeline has no Studio, so pass `--skip-studio-readiness` there.
163
+
164
+ The long way runs each step by hand:
165
+
166
+ ```bash
167
+ ci package build --declaration ./apps/extensions/src/index.ts --out apps/extensions/dist/release
168
+ ci package validate --archive .../release.tar --manifest .../manifest.json --package-manifest .../package-manifest.json
169
+ ci package publish --backend-url <url> --archive ... --manifest ... --package-manifest ... --idempotency-key <key>
170
+ ci package promote --backend-url <url> --release <release-id> --idempotency-key <key>
171
+ ci package install --backend-url <url> --organization <organization-id> --package acme.app.knowledge --release <release-id> --idempotency-key <key>
172
+ ```
173
+
174
+ `ci package build` writes `release.tar`, `manifest.json`, `package-manifest.json` and `private-source-maps/`. The source maps hold your full source, so keep them off any public origin. Publish, promote and install each return an Operation ID, and the step is done only when the operation reports `succeeded`. `ci package releases`, `update`, `rollback`, `sync` and `operation reconnect` cover the rest of the lifecycle.
175
+
176
+ Install records a Desired Release. CI makes it the Serving Release only after every required Host (Backend, Studio, Dashboard) has prepared it, so the backend and React parts of one package never run different versions. A failed generation never becomes Serving and leaves the previous Serving Release in place.
177
+
178
+ For a fast loop without releases, `ci package dev --backend-url <url> --declaration ./src/package.ts --host-origin <studio-or-dashboard-origin>` starts a Development Session. It serves modules with Vite and rebuilds on save until the command stops. It needs a `CI_MANAGEMENT_TOKEN` with `ai:manage`, and only one live command can own a package at a time.
179
+
180
+ ## Runtime rules for backend code
181
+
182
+ Built-in packages ship with each CI version, use FQNs under `builtin.`, and run their backend code inside the CI backend process. Third-party backend code runs in Bun Workers. Each backend process keeps one Worker Pool per package release, and every Organization that uses that release shares it. Workers start on demand and stop after an idle period. Built-in and third-party packages use the same public SDK contracts.
183
+
184
+ > **Warning:** Never keep tenant data in module-level variables. Read the Organization,
185
+ > capabilities and configuration from the per-call `context`, because the next
186
+ > call in the same Worker can belong to a different Organization.
187
+
188
+ Every call gets an `AbortSignal` in `context.signal` and can have a deadline in `context.deadlineAt`. The host does not retry a failed call. A Worker that passes its heap limit is recycled without cutting the calls in flight. CI treats third-party code as trusted code: a valid publisher credential admits it to hosted execution, and Workers do not sandbox it.
189
+
190
+ ## After install
191
+
192
+ Some parts of the demo need a person after the first deploy:
193
+
194
+ 1. Rotate the client secret. A confidential application has no secret after install. An administrator with `external_apps:manage` runs `ci apps rotate-secret <application-id>` or clicks Rotate secret on the Dashboard External Apps page. CI shows the secret once, and the web app stores it as `CI_CLIENT_SECRET`.
195
+ 2. Add users to the groups. The package creates groups empty and grants profiles to groups. In the template, an administrator adds users to the Sales group or its `managers` subgroup.
196
+ 3. Set every secret variable that the deploy left without a value.
197
+ 4. Enroll the web app with `ci apps enroll --env-file apps/web/.env.local`, which writes the `CI_*` values the server SDK reads.
198
+
199
+ The Dashboard Extensions page lists packages on Installed and Available tabs. Install and uninstall show a spinner until the real outcome, and leaving the page does not cancel the operation. A configured Knowledge connector instance, even a disabled one, or a saved connection blocks uninstall, and the Dashboard shows the error next to the package.
200
+
201
+ ## Related pages
202
+
203
+ - [SDKs](sdks.md)
204
+ - [External apps](external-apps.md)
205
+ - [Access](../platform/access.md)
206
+ - [Template](../template/index.md)
207
+ - [Knowledge](../data/knowledge.md)
208
+ - [AI](../ai/index.md)