@collective-intelligence/extensions-sdk-frontend 0.2.6 → 0.2.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -21,3 +21,7 @@ API.
21
21
  - [skills/extensions-sdk-frontend-move-to-effect/SKILL.md](skills/extensions-sdk-frontend-move-to-effect/SKILL.md):
22
22
  run when a frontend extension package adds `effect`, has more than one
23
23
  `effect` version, or lint reports `ci-solution/effect-subpath`.
24
+
25
+ ## Platform guide
26
+
27
+ - [docs/guide/index.md](docs/guide/index.md): read before choosing CI primitives for a solution or demo, or when a task needs the platform model (architecture, access, data, AI, template).
@@ -0,0 +1,349 @@
1
+ # AI primitives
2
+
3
+ Tools, Skills, Agents, workflows and Studio chat, how they compose, and how audiences, approvals, models, runs and timeouts behave.
4
+
5
+ The `next-effect-app` template ships two AI features. Its Lead qualifier Agent scores how ready a Lead is to convert, and its convert-lead workflow marks a Lead converted when the web app asks. Both are built from the same small set of AI primitives, and a new demo feature usually starts as a copy of one of them. This page explains each primitive through those two features and ends with a table that helps you choose between them.
6
+
7
+ ## The building blocks
8
+
9
+ | Building block | What it is | Model inside | How it starts |
10
+ | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------- |
11
+ | Tool | A typed, callable capability, such as a Knowledge query, a message send, a web search or your own package code. Tools are grouped in tool collections. | No | A Skill, an Agent, a workflow node or a chat turn calls it. |
12
+ | Skill | Versioned instructions with model settings, permissions and a bounded set of tools, for one AI task. | Yes | Studio chat, a Studio Skill Form, the public Skill API, MCP, a workflow Skill node, or an Agent that holds it. |
13
+ | Agent | An Organization-owned actor with a Role, held Skills, its own tools and an optional typed output. | Yes | Your server invokes it, a conversation continues it, or an event, a delegation, a schedule or a heartbeat triggers it. |
14
+ | Workflow | A durable graph of steps with triggers, versions, runs, pauses and approvals. | Only in Prompt and Skill nodes | A trigger fires: webhook, schedule, Knowledge record event, external application, MCP operation or on demand. |
15
+ | Studio chat | The Studio conversation in which Organization members use the tools and Skills offered to them. | Yes | A member sends a message. |
16
+
17
+ The template's Lead qualifier composes three of these. A package declares the Lead score tool, a Lead scoring Skill that binds it, and an Agent that holds the Skill and is offered to the external application:
18
+
19
+ ```mermaid
20
+ flowchart LR
21
+ tool["Lead score tool"] -->|is created in| coll["Sales tool collection"]
22
+ skill["Lead scoring Skill"] -->|binds| tool
23
+ agent["Lead qualifier Agent"] -->|holds| skill
24
+ agent -->|lists directly| tool
25
+ agent -->|lists directly| kq["Knowledge query tool"]
26
+ agent -->|offered to| app["External application"]
27
+ ```
28
+
29
+ > **Warning:** A held Skill can use only the tools that its Agent also allows, which is why
30
+ > the Lead qualifier lists the Lead score tool again. When you add a tool to a
31
+ > Skill, add it to the Agent's `tools` list too.
32
+
33
+ ## Tools
34
+
35
+ Put deterministic logic in a tool and let the model orchestrate. The Lead score tool computes the 0 to 100 score from fixed points: 40 for a work email, 30 when the email domain matches the company, 30 when sales has qualified the Lead, and 100 when the Lead is already converted. The Skill tells the model to use that score as is, so the model never produces the number itself.
36
+
37
+ A package tool is a runtime class plus a declaration:
38
+
39
+ ```ts
40
+ export class LeadScoreTool extends PackageTool<LeadScoreInput, LeadScore> {
41
+ override async execute(lead: LeadScoreInput) {
42
+ return await Effect.runPromise(scoreLead(lead));
43
+ }
44
+ }
45
+
46
+ declareTool({
47
+ definition: {
48
+ audience,
49
+ autoActivate: true,
50
+ autoInstantiate: true,
51
+ bindingScopes: ["organization"],
52
+ capabilityType: "read",
53
+ effectKind: "read_only",
54
+ input: effectSchema(leadScoreInputSchema),
55
+ output: effectSchema(leadScoreSchema, { codec: true }),
56
+ toolGuidance: {
57
+ inputRequirementsSummary: "Needs the Lead's company and email, and its status when known.",
58
+ useWhen: "Score how ready one Lead is to convert, from 0 to 100.",
59
+ },
60
+ toolCollectionFqns: [salesToolCollectionFqn],
61
+ usageScopes: ["automation", "chat", "skill"],
62
+ },
63
+ implementation: LeadScoreTool,
64
+ ...
65
+ });
66
+ ```
67
+
68
+ `usageScopes` decides where the tool can appear: in a Skill, in chat, or in automation. The template needs `automation` so that the Lead qualifier Agent can use the tool. When you omit `usageScopes`, the tool appears in all three places, and an explicit empty list makes it unavailable everywhere. The model reads `toolGuidance.useWhen` and `toolGuidance.inputRequirementsSummary` to decide when to call the tool and what to pass, so write both for the model.
69
+
70
+ A demo can also bind built-in tools for web search (Exa or Tavily, which need an API key), website scraping, image generation, email, notifications, OCR, speech transcription and synthesis, and report building. External applications can use `publish-run-event`.
71
+
72
+ ## Skills
73
+
74
+ A Skill's instructions are plain Markdown. The template's Lead scoring Skill uses numbered steps and one rule that stops the model from guessing:
75
+
76
+ ```ts
77
+ const instructions = `# Lead scoring
78
+
79
+ Score one Lead with the Lead score tool. Never guess a score.
80
+
81
+ 1. Read: Take the Lead's company, email, and status from Knowledge.
82
+ 2. Score: Call the Lead score tool once with those fields.
83
+ 3. Report: Use the tool's score as is. Explain it in one sentence from the tool's reasons.
84
+ `
85
+ ```
86
+
87
+ The package passes these instructions to `declareSkill({ definition: { audience, instructions, lifecycle, tools } })`, and the template's `tools` list holds only the Lead score tool. CI stores each Skill as a versioned package, and runtime always uses the active approved version.
88
+
89
+ The same Skill can run in Studio chat, in a Studio Skill Form, through the public Skill API (`skills_invoke`, then `runs_get` or `runs_stream`), through MCP `run_skill`, in a workflow Skill node, and inside an Agent that holds it. A Skill version can pin a model. Without a pin, it uses the model that the running Organization sets for Skill turns, which defaults to the `balanced` tier. A pinned model that does not resolve is a configuration error, and CI does not fall back to another model.
90
+
91
+ ## Agents
92
+
93
+ The template declares its Agent like this:
94
+
95
+ ```ts
96
+ declareAgent({
97
+ definition: {
98
+ audience: { applications: [application] },
99
+ behavioralBoundaries: ["Never change a Lead or create an Opportunity"],
100
+ lifecycle: { deployMode: "upsert", readonly: true },
101
+ objectives: ["Score how ready a Lead is to convert, from 0 to 100"],
102
+ operatingInstructions:
103
+ "Read the Lead from Knowledge, score it with the Lead scoring Skill, then submit the score with a one-sentence rationale.",
104
+ output: { schema: effectSchema(leadQualificationSchema, { codec: true }) },
105
+ responsibility: "Qualify Leads for the sales team",
106
+ skills: [skill],
107
+ tools: ["builtin.core.knowledge.tools.query", tool]
108
+ },
109
+ manifest: { fqn: leadQualifierFqn, version },
110
+ presentation: {
111
+ description: "Scores a Lead's readiness to convert.",
112
+ displayName: "Lead qualifier"
113
+ }
114
+ })
115
+ ```
116
+
117
+ CI turns the Role into the run's system message in a fixed order: `responsibility`, `objectives`, `operatingInstructions`, `behavioralBoundaries` and the declared output contract. A catalog of the held Skills follows, with the reference, name and description of each. The task and its context, for example today's date, become the user message.
118
+
119
+ Held Skills load on demand. At first the Agent sees only the catalog. When it calls the CI-owned `load-skill` tool, it receives the Skill's instructions, and only then does CI offer that Skill's tool bindings to the model. A second tool, `read-skill-resource`, returns a reference, asset or script file of a loaded Skill.
120
+
121
+ `output: { schema }` takes a codec, so the app can keep one runtime schema in its contracts package. A succeeded run returns `{ message, output? }`. `message` is the final answer, and `output` is the structured Agent Output, already valid against the schema. `output` is absent when the run saved none, for example when the Agent asked a clarifying question.
122
+
123
+ An Agent without `activity` is idle and runs only when something triggers it: an invoke call, a conversation turn, an event or a delegation. `activity: { heartbeat, schedule }` turns on either source when you set it to `true`. Every Agent that a package declares is an Organization Agent.
124
+
125
+ FQN string references are not type-checked when you build. Install resolves them and rolls the package back when one is missing: each held Skill must be approved, each Agent or Skill tool must be an enabled tool of the Organization, and each audience reference must exist. Only an administrator can install a package that declares Agents or Skills.
126
+
127
+ ## Audiences
128
+
129
+ An audience is an allow-list with five keys:
130
+
131
+ - `organization: true` for the whole Organization
132
+ - `groups` for members of the listed Groups, including subgroups unless you set `coverage: "direct"`
133
+ - `profiles` for the listed Access Profiles
134
+ - `applications` for the listed external applications
135
+ - `anonymous` for anonymous visitors of the listed applications
136
+
137
+ An omitted or empty audience offers the item to nobody. Packages never name individual users; an admin adds users in the Dashboard. The template passes `salesMembers = { groups: [salesGroup] }` as the audience of the Sales tools, the Skill and the Studio app, so every Sales member gets them, and offers the Agent to the external application only.
138
+
139
+ A tool collection does not offer its tools to anyone, even when the collection has an audience. Each tool names its own audience, so set `audience` on every tool you declare.
140
+
141
+ Whether an admin's Dashboard edit to an audience survives the next deploy depends on the item. The deploy that creates a tool or tool collection writes its audience once, and later admin edits stay. Agents and Skills follow their `lifecycle`: `upsert` rewrites the declared audience on each deploy, `init_only` keeps admin edits, and `readonly` refuses them.
142
+
143
+ The declaration of an Agent sets the most it can do: its tools, scopes, approval controls and audience. An admin can only narrow that policy, for example by removing a tool or a Skill or by narrowing a binding's settings. These restrictions survive package updates and apply from the next run. [Access](../platform/access.md) explains Groups, Access Profiles and Access Grants.
144
+
145
+ ## Approvals
146
+
147
+ An entry in a `tools` list can be a `ToolBinding` instead of a bare reference. A binding can rename the tool, set its settings, narrow its Knowledge and choose an approval mode, and you can bind one tool several times under different names:
148
+
149
+ ```ts
150
+ tools: [
151
+ {
152
+ tool: "builtin.core.knowledge.tools.query",
153
+ name: "member-query",
154
+ settings: { limit: 10 },
155
+ knowledge: { domains: { include: [membersDomain] } },
156
+ approval: { rule: "Escalations" }
157
+ }
158
+ ]
159
+ ```
160
+
161
+ `approval` has three modes. `"never"` makes no approval request for the binding. `"always"` pauses for an explicit decision by the user who started the call, and that user's eligibility to approve does not approve the call. `{ rule: "<rule name>" }` uses an approval rule of the installing Organization, with that rule's reviewers and expiry. Install fails when no rule has the name, so create the rule in the target Organization before you install a demo package that uses one.
162
+
163
+ Approval pauses the call and leaves the tool visible. With `"always"`, the model still sees the tool definition and can decide to call it, and the call then waits for a person.
164
+
165
+ ### Approvals by caller
166
+
167
+ The same binding behaves differently for each kind of caller:
168
+
169
+ | Caller | What happens when a call needs approval |
170
+ | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
171
+ | Studio chat or a Skill run | The run pauses until a person decides. |
172
+ | Studio Skill Form | `ask_always` forces approval of every installed tool call, even on `never` bindings. `automatic` skips approval. Other authorization checks still apply. |
173
+ | Agent run | Nobody is there to approve. CI rechecks the binding, and unless the binding is denied, it approves the call by default, records it with the reason `unattended_default`, and runs it once. |
174
+ | Workflow | An Approval node pauses the workflow for a person. Approve and Reject both resume it with the decision data, so a reject branch can continue, for example to send a rejection email with the approver's `comment`. Reviewers see a preview of up to 32 next nodes for each decision. |
175
+
176
+ > **Note:** `"always"` does not stop an autonomous Agent the way it stops a chat user.
177
+ > When a demo must show a person approving, use Studio chat, a Skill Form or a
178
+ > workflow Approval node.
179
+
180
+ ### What approval does not do
181
+
182
+ An approval authorizes one attempt. It does not check the tool input and cannot make the call succeed: an approved Knowledge Query `search` call still fails when it has no `entitySchemaId`.
183
+
184
+ The reviewer never becomes the identity that runs the tool. After approval, the call resumes as the original invoker, and CI rechecks that user's current Organization authority, so a changed action, a rejection, an expiry or revoked access blocks the call. When the recipients or destination of a send change between approval and send, CI does not send. It opens a fresh approval request under the original user, and that request needs a new decision. Reviewers see the facts of the pending action in Studio Notifications. CI freezes those facts when the request opens and shows secrets as omitted.
185
+
186
+ ## Models
187
+
188
+ An Organization configures one model for each standard tier: `complex`, `balanced`, `fast` and `super_fast`. Agents, Prompt nodes and other AI operations name a tier, never a vendor model. When the default model of a tier is missing, disabled or archived, every run that needs that tier is unavailable.
189
+
190
+ An Agent uses the tier in its `modelTier`, or `balanced` when it sets none. Its step limit is its `maxSteps`, or else the Organization setting `agentRunMaxSteps` (default 25, maximum 100), and a run that uses all its steps fails with `step_limit_reached`. A Skill's own model and iteration settings never apply inside an Agent run. Agents have no model preset, and CI rejects the old `modelPreset` field.
191
+
192
+ ## Runs are asynchronous
193
+
194
+ The gateway in front of CI cuts a request that stays silent for 120 seconds and caps every response at 10 minutes. Anything that can run longer than about 100 seconds must therefore return an id for the caller to poll, which is why Agent, Skill and workflow invocations all return a run id.
195
+
196
+ Studio chat turns, Skill runs, approval continuations, public Skill runs and MCP `run_skill` never execute inside the HTTP request. The request admits the run and returns its id, and a worker executes the run from a durable command queue. A client follows the run's stream, and a stream that ends cleanly does not mean the run has finished. Reconnect with the last sequence (`Last-Event-ID` or `afterSequence`) until the closing frame arrives.
197
+
198
+ Inside a run, a known tool error returns to the model as a tool result. The other calls in the same batch still run and the model can try again, so a successful Skill run can show failed calls in its trace.
199
+
200
+ ### Timeouts
201
+
202
+ | Limit | Default | Effect |
203
+ | ---------------------------------------------------------- | ------------------------------------------- | ----------------------------------------------------------------------------------------- |
204
+ | AI run deadline, `aiRunDeadlineSeconds` | 300 s, minimum 30 s | The run stops and fails with "The AI run timed out after N seconds." |
205
+ | Model call, `aiLlmCallTimeoutSeconds` | 180 s | Applies to each model call. Tool execution between calls does not count. |
206
+ | Interactive timeout on synchronous routes | 75 s | Failures surface before the 90 s request deadline. |
207
+ | Webhook response, `workflowWebhookResponseDeadlineSeconds` | 25 s, maximum 100 s | The webhook answers 202 with an id to monitor. |
208
+ | `await_result` trigger, `timeoutMs` | Set on each trigger; the template uses 90 s | The caller waits up to this long. Nodes never run inside the request. |
209
+ | Agent steps, `agentRunMaxSteps` | 25, maximum 100 | The run fails with `step_limit_reached`. |
210
+ | Workflow scheduler tick | 30 s | Each node-to-node continuation can wait for the next tick, so plan demo timing around it. |
211
+
212
+ ### Calling an Agent from your server
213
+
214
+ Your server invokes an Agent with `agents.invoke({ agentId, input })`, which returns `{ runId }`. Read the result with `runs.get`, or page through live events with `runs.stream`. The input carries `{ task, context?, attachments }`, with up to five files of 5 MiB each in PNG, JPEG, WebP, GIF, PDF, text, CSV, Markdown or JSON format. CI does not accept Office files.
215
+
216
+ `runs.stream` returns three kinds of event. `agent.tool` gives the tool name and its state (running, completed or failed) without arguments or results. `agent.reasoning` gives the provider's reasoning-summary text in batches, and `agent.context` gives input and output token counts for each model step. A page holds at most 100 events, so keep polling with the last cursor. Filter the reasoning text on your server before you show it to visitors.
217
+
218
+ For a multi-turn chat, call `conversations.continue({ agentId | conversationId, input, organizationId })` and read earlier turns with `conversations.history`. Each turn is an ordinary Agent run, and the earlier successful turns, with their saved Agent Output, become the model's conversation history. A continue call made while the previous turn still runs returns `conflict`, and an `idempotencyKey` replays the same turn on retry.
219
+
220
+ Anonymous visitors can chat with an Agent only when its audience has `anonymous: [app]` for that application and the call passes `callerKind: "anonymous_visitor"`. Their turns run with the Agent's own bindings, never with the application's grants, and visitors never see each other's conversations.
221
+
222
+ ## Workflows
223
+
224
+ A workflow graph runs from a trigger to a Workflow input node, through action nodes, to a Workflow output node. A trigger connects only to the Workflow input, and the Workflow output receives connections only from actions. The Workflow input and the Workflow output each carry a JSON Schema contract. The template's convert-lead workflow is the smallest useful example:
225
+
226
+ ```mermaid
227
+ flowchart LR
228
+ trigger["External application trigger"] -->|event| input["Workflow input"]
229
+ input --> mark["Set Variable: mark-converted"]
230
+ mark --> output["Workflow output"]
231
+ ```
232
+
233
+ Its one action node sets a fixed patch and declares how it behaves on dry runs, errors and retries:
234
+
235
+ ```ts
236
+ {
237
+ config: {},
238
+ id: "mark-converted",
239
+ inputs: { patch: { mode: "fixed", value: { converted: true } } },
240
+ metadata: {},
241
+ settings: {
242
+ dryRun: { mode: "allow" },
243
+ onError: { mode: "fail_run" },
244
+ retry: { mode: "disabled" },
245
+ },
246
+ templateFqn: "builtin.core.workflows.nodes.actions.set-variable",
247
+ templateVersion: "1.0.0",
248
+ }
249
+ ```
250
+
251
+ A triggered workflow needs an identity to run as, and CI publishes it only with one. The template declares `servicePrincipal({ fqn: convertLeadRunnerFqn, groups: [], name: "Convert lead runner", version })` and sets `servicePrincipalFqn` on the workflow, so every tool call in the workflow runs as that principal. A service principal has no user identity in the tool catalog, which means user-scoped tool bindings never match it. Give it a tool through an Organization binding or through a Group it belongs to.
252
+
253
+ Each trigger has an invocation mode: `attach`, `detach`, or `await_result` with a `timeoutMs`. `detach` answers at once with a durable 202, and `await_result` waits up to its timeout. The convert-lead trigger uses `{ mode: "await_result", timeoutMs: 90_000 }`. The web app starts it from its server with `createFixedWorkflowRoute` from `@collective-intelligence/external-apps-server-sdk/next`. The server owns the workflow FQN, and the browser sends only the Lead id:
254
+
255
+ ```ts
256
+ const handle = createFixedWorkflowRoute<{ leadId: string }, { leadId: string }>(
257
+ {
258
+ ...options,
259
+ idempotencyKey: ({ leadId }) => `convert-lead:${leadId}`,
260
+ // The Workflows feature in the user's slice of the app's access; CI decides the run again.
261
+ requirement: convertLeadRequirement,
262
+ resourceId: "leads",
263
+ validate: (body, { leadId }) => {
264
+ const decoded = decodeConvertBody(body)
265
+ if (Option.isNone(decoded) || decoded.value.leadId !== leadId) {
266
+ throw new TypeError("invalid_request")
267
+ }
268
+ return { leadId }
269
+ }
270
+ }
271
+ )
272
+ ```
273
+
274
+ Call a workflow by `workflowId` or `workflowFqn`, never by slug. A package workflow keeps its declaration FQN in every install, while its id differs from one install to the next.
275
+
276
+ The core catalog has these triggers: Entity Created, Entity Updated (with optional watched fields), Entity Deleted, External application, MCP operation, On-demand, Schedule (cron) and Webhook. Its action nodes are Adapter, Approval, Code Block, File Content, File URL, HTTP Request, If, Invoke Workflow, Loop, Merge, Persist Artifact, Prompt, Set Variable, Skill, Stop Error, Stored File Link, Switch and Wait. Other core packages add nodes for Documents, speech and image generation.
277
+
278
+ Two of these action nodes run a model. Prompt generates text or structured JSON with a governed model: `mode` is `"text"` or `"structured"`, `model` optionally names a tier, and `responseSchema` shapes the structured result. It returns `{ kind: "text", text }` or `{ kind: "structured", object }`. The Skill node invokes a governed Skill by `skillId` and passes it an `inputMessage`. For code without a model, Code Block runs restricted JSON-only Python or JavaScript in an approved Docker runtime.
279
+
280
+ A field that is read at runtime can take an expression instead of a fixed value: `{ mode: "expression", languageVersion: 1, source }`. Structural fields stay fixed. Expressions use a closed, Mustache-shaped language that is not JavaScript. `{{ input.customer.email }}` reads the workflow input, and `{{ nodes["Fetch customer"].result.customer.email }}` reads an upstream node. A source that is exactly one interpolation keeps its JSON type, while any text around it makes the result a string. To convert a value, add a pipe such as `{{ input.amount | number }}`. The available pipes are `string`, `number`, `boolean`, `trim`, `lower`, `upper`, `length`, `first`, `last`, `floor`, `ceil` and `datetime`.
281
+
282
+ A package workflow has a `lifecycle` with `deployMode` set to `upsert` or `init_only`, plus `readonly`. The builder shows a readonly workflow as a view and accepts only layout changes. The template uses `init_only` with `readonly: false`, so admins can edit the deployed graph and later releases do not overwrite it.
283
+
284
+ In the builder, Save accepts an incomplete draft. Test and Publish validate the full graph. A full-workflow test needs a trigger and a Workflow input connected to an action, and otherwise fails with `trigger-required`, `input-target-required` or `node-not-found`. You can test a single node without a trigger.
285
+
286
+ ### Webhook triggers
287
+
288
+ A Webhook trigger accepts `POST` requests authenticated with an API key (`authentication: "api_key"`) on a `path` made of letters, digits and `._~-`. It passes the JSON body to the workflow as `event` and answers with JSON. The call waits for the workflow result for up to `workflowWebhookResponseDeadlineSeconds`, 25 seconds by default and 100 at most, and the response depends on the workflow's state at that point:
289
+
290
+ | Workflow state | Response |
291
+ | ----------------------------- | --------------------------------------------- |
292
+ | Completed | 200 with the workflow output JSON |
293
+ | Failed | 500 with `{ error, runId, status }` |
294
+ | Still running at the deadline | 202 with a `runId` or `deliveryId` to monitor |
295
+
296
+ ### Output contracts
297
+
298
+ A workflow's public result contains only what its declared output contract binds. CI never falls back to the output of an intermediate node, and it reports success only after it has validated and stored the declared result. `runs.get` returns `result: { nativeStatus: "completed", output }`. When reading the result fails, read the same `runId` again and do not send the business command a second time.
299
+
300
+ > **Warning:** The template's convert-lead workflow declares an empty output contract (`bindings: []`, `jsonSchema: { type: "object" }`), so it returns no business fields. Add output bindings when your workflow must return data.
301
+
302
+ ## How agents reach Knowledge, Documents and datasets
303
+
304
+ An agent reads data only through the tools it is bound to. The core packages provide these tools:
305
+
306
+ | Source | Tools |
307
+ | ----------- | ----------------------------------------------------------- |
308
+ | Knowledge | `catalog`, `query`, `mutate`, `knowledge-document-read` |
309
+ | Documents | `query` |
310
+ | Information | `advanced-retrieval`, `expand-passage`, `read-source-range` |
311
+ | Datasets | `query` |
312
+
313
+ Knowledge Query has seven intents: `autocomplete`, `search`, `facets`, `facet_values`, `count`, `get` and `relation_members`. `search`, `count` and the two facet intents need an `entitySchemaId`, and `get` needs an `entityId`. A missing id fails with `invalid_input`, so tell the model in the Skill or the Role to find schema ids with Knowledge `catalog` first and never to invent one. When a schema has record or indexed-file embeddings, Query also runs semantic retrieval on PostgreSQL with pgvector, and the model never sees the vectors or the embedding provider.
314
+
315
+ Two settings decide what these tools can read. A binding's `knowledge` field narrows a tool to included or excluded domains and entity schemas, as the `member-query` binding above does. `dataAccess` on `declareAgent`, `declareSkill` or `declareWorkflow` decides, separately for `knowledge`, `documents` and `datasets`, whose permission applies. With `"user"`, the run reads what the acting person can read. With the declaration's own kind (`"agent"`, `"skill"` or `"workflow"`), which is the default, the run reads its configured resources without the caller's permission on the source. `dataAccess` grants no extra tool bindings.
316
+
317
+ ```ts
318
+ const dataAccess = {
319
+ knowledge: "user",
320
+ documents: "agent"
321
+ } satisfies DataAccess<"agent">
322
+ ```
323
+
324
+ [Knowledge](../data/knowledge.md) and [Datasets](../data/datasets.md) explain the data that these tools read.
325
+
326
+ ## Tool, Skill, Agent or workflow
327
+
328
+ | | Tool | Skill | Agent | Workflow |
329
+ | --------------------- | --------------------------------------------------------------------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
330
+ | Model inside | No | Yes | Yes | Only in Prompt and Skill nodes |
331
+ | Who decides the steps | Your code | The model, guided by the instructions | The model, guided by the Role and held Skills | The graph you build |
332
+ | How it starts | Another primitive or a chat turn calls it | Chat, Skill Form, API, MCP, workflow node or Agent | Your server, a conversation, an event, a delegation, a schedule or a heartbeat | Webhook, schedule, record event, external application, MCP or on demand |
333
+ | Human approval | Set on its binding | The run pauses for a decision | Approved by default and recorded | Approval node |
334
+ | Output | Typed by its output schema | The run's result | `message` plus typed `output` | The declared output contract |
335
+ | Pick it when | You need one deterministic, typed action such as score, fetch or send | One AI task with a bounded tool set must run in several places | You need a long-lived Role with an audience, typed output, conversation history or unattended runs | The steps are fixed and you need triggers, approvals or a typed result |
336
+
337
+ Most demo features combine these primitives. The Lead qualifier uses a deterministic tool, a Skill that tells the model how to use it, and an Agent that holds the Skill for the external application. Convert-lead is a workflow because its one step is fixed and the web app waits for the result, so it needs no model. When a workflow step needs AI, use a Prompt or Skill node. [Scenarios](../scenarios/index.md) walks through more demo cases.
338
+
339
+ ## Not covered here
340
+
341
+ This page does not cover Personal Agents (admins create them in the Dashboard), Studio chat Projects, running Skill scripts, or the planned cancel and retry controls for Agent runs.
342
+
343
+ ## Related pages
344
+
345
+ - [Knowledge](../data/knowledge.md)
346
+ - [Datasets](../data/datasets.md)
347
+ - [Extensions](../develop/extensions.md)
348
+ - [Access](../platform/access.md)
349
+ - [Scenarios](../scenarios/index.md)
@@ -0,0 +1,204 @@
1
+ # Connectors
2
+
3
+ Connect outside systems, map their fields to Knowledge schemas, and run and watch imports.
4
+
5
+ A connector imports records from an outside system, such as Notion, HubSpot or a SQL database, into Knowledge records. A Documents connector does the same for files, which it keeps in a Documents collection. An import passes through several stages: CI reads the source page by page, stages the records, commits the accepted ones into Knowledge, and then indexes them for search. A demo that shows imported data has to wait for the stage it needs.
6
+
7
+ ## Connections, templates and instances
8
+
9
+ Three objects work together, and a demo usually declares two of them.
10
+
11
+ A connection holds credentials. A connection template, such as `builtin.integrations.notion.connections.internal-integration-token`, defines the credential form and a test, and an Organization connection holds the actual credentials. Connectors, tools and Dataset providers each point at a connection through a slot, so one connection can serve all of them.
12
+
13
+ A connector template is the import capability that a package ships, for example the "Notion API" connector `builtin.integrations.notion.connectors.api`. A connector instance is one configured use of that template in an Organization, with its settings, connection slots, entity bindings, pipelines, schedules and run state. Notion, for example, allows several instances in one Organization.
14
+
15
+ ```mermaid
16
+ flowchart LR
17
+ CT[Connection template] --> OC[Organization connection with credentials]
18
+ KT[Connector template] --> CI[Connector instance]
19
+ CI -- slot --> OC
20
+ CI -- entity bindings --> KS[Knowledge schemas]
21
+ CI -- pipelines --> SC[Schedules: full or delta]
22
+ ```
23
+
24
+ ## Credentials
25
+
26
+ A package declares a connection with `declareConnectionInstance`. The secret comes from a package variable, so the release itself never contains it:
27
+
28
+ ```ts
29
+ const variables = { notionToken: { type: "secret" } } as const
30
+ const vars = packageVariables(variables)
31
+
32
+ const notionConnection = declareConnectionInstance({
33
+ manifest: { fqn: "acme.connections.notion", version: "1.0.0" },
34
+ presentation: { displayName: "Company Notion" },
35
+ definition: {
36
+ templateFqn:
37
+ "builtin.integrations.notion.connections.internal-integration-token",
38
+ lifecycle: { deployMode: "upsert", readonly: false },
39
+ credentials: { integrationToken: vars.notionToken }
40
+ }
41
+ })
42
+ ```
43
+
44
+ At deploy time, the Inputs File links the variable to an Organization secret:
45
+
46
+ ```json
47
+ {
48
+ "variables": {
49
+ "notionToken": { "link": "NOTION_INTEGRATION_TOKEN" }
50
+ }
51
+ }
52
+ ```
53
+
54
+ `ci deploy --inputs-template` prints every connection slot and variable that the Inputs File must fill. CI stores the link itself as the credential, so a later rotation of the secret reaches the connection. When the linked secret does not exist yet, CI creates it as pending, and the connection makes no provider calls until someone fills in the value. A field that the template declares as secret stays secret even when the setup marks it otherwise.
55
+
56
+ ## Declare a connector instance
57
+
58
+ `declareKnowledgeConnectorInstance` names the template, fills its connection slots, binds remote collections to Knowledge schemas and sets the import pipelines. Each connection slot takes exactly one of `{ setupKey }`, which an administrator fills at install, or `{ connectionInstanceFqn }`, which points at a connection that the package declares. The package must also list the integration, such as `{ fqn: "builtin.integrations.notion" }`, in its `dependencies`.
59
+
60
+ ```ts
61
+ const customers = declareKnowledgeConnectorInstance({
62
+ manifest: { fqn: "com.acme.sales.instances.customers", version: "1.0.0" },
63
+ presentation: { displayName: "CRM customers" },
64
+ definition: {
65
+ templateFqn: "com.acme.integrations.crm",
66
+ lifecycle: { deployMode: "upsert", readonly: false },
67
+ enabled: true,
68
+ connections: { crm: { setupKey: "crm" } },
69
+ entityBindings: [
70
+ {
71
+ key: "customers",
72
+ schemaFqn: "com.acme.sales.customer",
73
+ bindingMode: "managed",
74
+ remoteCollectionId: "customers",
75
+ fieldMap: [
76
+ {
77
+ op: "path",
78
+ remote: "name",
79
+ local: "name",
80
+ direction: "inbound",
81
+ skipIfNull: false
82
+ }
83
+ ]
84
+ }
85
+ ],
86
+ pipelines: [
87
+ {
88
+ key: "nightly",
89
+ definition: {
90
+ steps: [{ tasks: [{ kind: "entity", bindingKey: "customers" }] }]
91
+ },
92
+ schedules: [
93
+ {
94
+ key: "nightly_delta",
95
+ cronExpression: "0 2 * * *",
96
+ timezone: "Europe/Rome",
97
+ importMode: "delta",
98
+ enabled: true
99
+ }
100
+ ]
101
+ }
102
+ ]
103
+ }
104
+ })
105
+ ```
106
+
107
+ > **Warning:** Deploying a package that declares a connector instance creates no records and
108
+ > starts no import. Data arrives only when a schedule fires or someone starts
109
+ > the pipeline by hand, so plan for that step before you open the demo.
110
+
111
+ A redeploy follows the `lifecycle` of the instance. `init_only` keeps the existing configuration. `upsert` merges the new declaration, the previous declaration and the live configuration, so a manual edit to an unrelated field survives, and CI refuses changes that compete for the same field. Imported records, run history and cursors are runtime state, and a redeploy never resets them. When you remove a binding from the declaration, CI archives it; when you remove a pipeline, CI disables it. Imported records stay in both cases.
112
+
113
+ ## Discovery and field mapping
114
+
115
+ During setup, a connector reports the remote collections and fields it finds, each with a type hint such as `{ kind: "datetime" }`. Knowledge setup uses the hint to choose a local field type. Notion dates, for example, arrive as `datetime` because a Notion date can carry a time.
116
+
117
+ Running setup again does not repair a field that already exists. Setup skips remote models that are already bound, and an existing field keeps its type. To change a `date` field to `datetime` later, update both the schema property and the field definition.
118
+
119
+ A binding's `fieldMap` lists the rules that copy remote values into local fields.
120
+
121
+ | Rule | What it does |
122
+ | ----------- | -------------------------------------------------------------------------------------------------------------- |
123
+ | `path` | Copies one remote value into a local field |
124
+ | `reference` | Stores remote IDs as relation field references to records of another binding of the same connector instance |
125
+ | `embedded` | Maps a nested remote object into a record of another binding, creates that record when needed, and links to it |
126
+
127
+ The `remote` value is a dot path with at most one `[]` segment, which takes every item of an array. This `reference` rule fills a `companies` relation field from HubSpot-style associations:
128
+
129
+ ```ts
130
+ {
131
+ op: "reference",
132
+ local: "companies",
133
+ remote: "associations.companies.results[].id",
134
+ direction: "inbound",
135
+ skipIfNull: true,
136
+ }
137
+ ```
138
+
139
+ Each entity binding has a binding mode, `managed` or `external`, and the template declares which modes it supports. Notion supports both. An `external` binding refuses the managed-only `updateMode` and `outboundStrategy` options.
140
+
141
+ ## Imports
142
+
143
+ ```mermaid
144
+ flowchart LR
145
+ T[Schedule or manual start] --> P[Pull source pages]
146
+ P --> S[Staged records]
147
+ S --> C[Commit into Knowledge]
148
+ C --> I[Embedding and indexing]
149
+ ```
150
+
151
+ A FULL import lists everything in the source. A DELTA import reads only what changed since the Delta Since time, which is the start of the binding's last successful phase minus the connector's overlap. A failed phase does not move Delta Since. The overlap (`deltaOverlapSeconds`, 300 seconds by default and 120 for Notion) reads the edge of the window again to catch late changes. The Notion connector recommends a delta schedule every five minutes. A SQL connector can run a delta import only when you choose a watermark column for the binding, the column that records when a row changed; without one, it runs full imports only.
152
+
153
+ The import writes staged records first and then commits the accepted records into Knowledge. Treat an import as finished only when CI has committed the staged records and the indexing work that follows. Before each remote page, the import saves its cursor and counters. When a worker stops in the middle of a call, the next attempt repeats at most that one page, and repeating a page is safe because page commits are idempotent. Full imports resume from their cursor as well, so a restart does not scan the whole source again.
154
+
155
+ When a source value does not fit the schema field type, for example a Notion date with a time going into a `date` field, the batch fails with a non-retryable `entity_data` error that names the record. The run stays failed until you fix the data or the field type.
156
+
157
+ ## Deletions
158
+
159
+ Every Knowledge connector supports deletion on full runs. A full run deletes the records that are missing from a complete listing of the source, and a page that cannot prove the listing complete deletes nothing.
160
+
161
+ > **Note:** Deletion on delta runs is planned. Until then, count on full runs to remove
162
+ > records that were deleted at the source.
163
+
164
+ For a Notion demo, this means that a page deleted in Notion leaves CI on the next full run and stays through the five-minute delta runs.
165
+
166
+ ## Health
167
+
168
+ Health belongs to the Organization connection, and CI copies it onto each connector that uses that connection. A failed connection test marks the connection `unhealthy` and records the error. In Knowledge setup, **Refresh data sources** runs the connection refresh again before it updates the connector.
169
+
170
+ An operator follows an import in several places: the connection test result, the readiness of each binding, the status of each pipeline run, the counts of staged records, and the embedding work that follows the commit.
171
+
172
+ ## Built-in integrations
173
+
174
+ The platform ships these integrations. Some import data, and others add only connections, tools and workflow nodes.
175
+
176
+ | Role | Integrations |
177
+ | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
178
+ | Knowledge connectors | Notion, HubSpot, Monday, SQL (PostgreSQL and MySQL), Structured File (with an FTP source), the external application Knowledge gateway |
179
+ | Documents connectors | Microsoft 365 Drive, Google Drive |
180
+ | Dataset providers | Microsoft Fabric, Databricks |
181
+ | Connections, tools and actions only | Apollo, Lusha, Slack, Telegram, Twilio, WhatsApp |
182
+
183
+ Lusha shows the difference. It has an API key connection and prospecting tools and workflow actions, and it imports nothing. Use it to enrich a Lead from a button, and use a connector such as HubSpot to sync a CRM. The [Datasets](datasets.md) page covers Fabric and Databricks.
184
+
185
+ ## Knowledge sync and Documents sync
186
+
187
+ Knowledge connectors and Documents connectors have different owners and different rules. A Documents connector never creates Knowledge records, and it does not use the Knowledge import stages or the Knowledge deletion rules.
188
+
189
+ | | Knowledge connector | Documents connector |
190
+ | ------------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------- |
191
+ | Writes | Knowledge records in bound schemas | Files and versions in one Connected collection |
192
+ | Configured by | A connector instance with entity bindings and pipelines | One Documents connector instance per Connected collection |
193
+ | Schedule | Cron schedules per pipeline, FULL or DELTA | CI checks every minute and starts a sync five minutes after the last attempt finished |
194
+ | Package declaration | `declareKnowledgeConnectorInstance` | `declareDocumentCollection` with a connection slot and remote folders |
195
+ | Folder exclusions | Not applied | Excluded Folder Names skip whole folders |
196
+
197
+ A Documents sync moves its provider checkpoint forward only after it applies the changes. Two Connected collections on the same provider account cannot overlap their root folders. A finished Documents sync still has to index the files before search finds them, as the [Knowledge](knowledge.md) page explains.
198
+
199
+ ## Related pages
200
+
201
+ - [Knowledge](knowledge.md)
202
+ - [Datasets](datasets.md)
203
+ - [Extensions](../develop/extensions.md)
204
+ - [Access](../platform/access.md)