@lionhummer/constellation 0.6.0 → 0.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +327 -160
- package/dist/index.js +443 -281
- package/dist/types/client-core/src/agent.d.ts +81 -0
- package/dist/types/client-core/src/index.d.ts +102 -131
- package/dist/types/client-core/src/input-receiver.d.ts +24 -0
- package/dist/types/sdk/src/index.d.ts +5 -7
- package/examples/quickstart.mjs +3 -3
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,231 +1,398 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Constellation TypeScript SDK
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Server-side client for Constellation. You send a model request; Constellation chooses a provider, runs the call durably (with retries and fallbacks), and tells your server the result. If the model asks for tools, your server runs them and sends the next input.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
- Node.js 22 or newer.
|
|
6
|
+
- Server-side only. API keys never belong in browser code, source control or logs.
|
|
7
|
+
|
|
8
|
+
```sh
|
|
6
9
|
npm install @lionhummer/constellation
|
|
7
10
|
```
|
|
8
11
|
|
|
9
|
-
##
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
12
|
+
## How it works
|
|
13
|
+
|
|
14
|
+
A **request** is one task. It can take several **model steps**, and each step is one model call. Constellation keeps the request ID the same across all steps.
|
|
15
|
+
|
|
16
|
+
```text
|
|
17
|
+
Your server Constellation
|
|
18
|
+
─────────── ─────────────
|
|
19
|
+
requests.submit(input) ─────────────────────▶ chooses a provider, calls the model
|
|
20
|
+
(retries or falls back if a provider fails)
|
|
21
|
+
◀───── webhook ─────── model.completed: assistant message + tool calls
|
|
22
|
+
run the tools
|
|
23
|
+
submit the next input ──────────────────────▶ step 2: calls the model again
|
|
24
|
+
(inReplyTo: that event)
|
|
25
|
+
◀───── webhook ─────── model.completed: final answer
|
|
26
|
+
stop: submit "observe" ─────────────────────▶ request completed
|
|
27
|
+
```
|
|
20
28
|
|
|
21
|
-
|
|
29
|
+
Five rules explain almost everything else:
|
|
22
30
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
31
|
+
1. **You own the conversation.** Every input contains the complete message history. Constellation does not remember earlier messages for you.
|
|
32
|
+
2. **Every event gets exactly one answer.** You answer a `model.completed` event either by *continuing*, which sends the next complete input, or by *stopping*, which sends an `observe` response. A second, different answer to the same event is rejected.
|
|
33
|
+
3. **Nothing waits for you between steps.** After a step completes, the request is in `client_processing` and uses no Constellation capacity until you answer. The request deadline (`timeoutMs`, default 30 minutes) still runs and includes your processing time.
|
|
34
|
+
4. **Webhooks arrive at least once.** Store each event before you acknowledge it, and expect duplicates.
|
|
35
|
+
5. **Retries must be identical.** Save every body you send together with its idempotency key. If a call fails or times out, resend exactly the same body. Constellation returns the original result instead of doing the work twice.
|
|
26
36
|
|
|
27
|
-
|
|
37
|
+
## Quick start: one model call
|
|
28
38
|
|
|
29
|
-
The
|
|
39
|
+
The simplest use needs no webhook: submit an input and read the result.
|
|
30
40
|
|
|
31
|
-
```
|
|
41
|
+
```ts
|
|
32
42
|
import { Constellation, uuid7 } from "@lionhummer/constellation";
|
|
33
43
|
|
|
34
|
-
function required(name) {
|
|
35
|
-
const value = process.env[name];
|
|
36
|
-
if (!value) throw new Error(`Set ${name} in your server environment.`);
|
|
37
|
-
return value;
|
|
38
|
-
}
|
|
39
|
-
|
|
40
|
-
const workspaceId = required("CONSTELLATION_WORKSPACE_ID");
|
|
41
|
-
const userId = process.env.CONSTELLATION_USER_ID || undefined;
|
|
42
|
-
const planId = process.env.CONSTELLATION_PLAN_ID || undefined;
|
|
43
44
|
const client = new Constellation({
|
|
44
|
-
baseUrl:
|
|
45
|
-
apiKey:
|
|
45
|
+
baseUrl: process.env.CONSTELLATION_BASE_URL!, // host root, e.g. https://constellation.example.com
|
|
46
|
+
apiKey: process.env.CONSTELLATION_EXECUTION_KEY!,
|
|
46
47
|
});
|
|
47
48
|
|
|
48
|
-
//
|
|
49
|
-
const catalog = await client.models.list({ workspaceId
|
|
50
|
-
console.log("Available models:", JSON.stringify(catalog, null, 2));
|
|
51
|
-
const model = process.env.CONSTELLATION_MODEL;
|
|
52
|
-
if (!model) {
|
|
53
|
-
console.log("Set CONSTELLATION_MODEL to one of the returned models[].id values, then rerun.");
|
|
54
|
-
} else {
|
|
55
|
-
const input = {
|
|
56
|
-
idempotencyKey: uuid7(), // Save this input/key if submission needs retrying.
|
|
57
|
-
workspaceId,
|
|
58
|
-
userId,
|
|
59
|
-
planId,
|
|
60
|
-
model,
|
|
61
|
-
messages: [{ role: "user", text: "Say hello in one short sentence." }],
|
|
62
|
-
maxOutputTokens: 256,
|
|
63
|
-
timeoutMs: 120_000,
|
|
64
|
-
// No tools or wait checkpoints: no webhook or input consumer is needed.
|
|
65
|
-
};
|
|
66
|
-
const { id } = await client.requests.create(input);
|
|
67
|
-
console.log("Request ID:", id);
|
|
68
|
-
for await (const event of client.requests.streamEvents(id, {
|
|
69
|
-
signal: AbortSignal.timeout(150_000),
|
|
70
|
-
})) {
|
|
71
|
-
console.log(event.seq, event.type);
|
|
72
|
-
}
|
|
73
|
-
const { request } = await client.requests.getSummary(id);
|
|
74
|
-
console.log("Final status:", request.status);
|
|
75
|
-
console.log("Output:", request.output?.text ?? "");
|
|
76
|
-
if (request.status !== "completed") {
|
|
77
|
-
throw new Error(`Request ${id} ended as ${request.status}: ${request.reason ?? "inspect the request error and attempts"}`);
|
|
78
|
-
}
|
|
79
|
-
}
|
|
80
|
-
```
|
|
49
|
+
// Find the models this key may use. This does not call a provider.
|
|
50
|
+
const catalog = await client.models.list({ workspaceId: "workspace-123" });
|
|
81
51
|
|
|
82
|
-
|
|
52
|
+
const input = {
|
|
53
|
+
idempotencyKey: uuid7(), // save input + key so a retry sends the same body
|
|
54
|
+
workspaceId: "workspace-123",
|
|
55
|
+
model: "catalog/example-model", // one of catalog.models[].id
|
|
56
|
+
messages: [{ role: "user" as const, text: "Say hello in one sentence." }],
|
|
57
|
+
};
|
|
58
|
+
const { id } = await client.requests.submit(input);
|
|
83
59
|
|
|
84
|
-
|
|
60
|
+
// Live output (optional). Closing the stream does not cancel the request.
|
|
61
|
+
for await (const event of client.requests.streamEvents(id)) {
|
|
62
|
+
console.log(event.sequence, event.type);
|
|
63
|
+
}
|
|
85
64
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
| 403 | Execution permission, matching workspace/user scope and model access rules. Management/telemetry permissions do not imply execution. |
|
|
90
|
-
| No models | Operator has enabled a provider, model and offering with an eligible route/access rule. Discovery uses your workspace/user scope. |
|
|
91
|
-
| 409 | Read the error code/message. Reuse the exact original input for an idempotency retry; renew an expired input claim before answering. |
|
|
92
|
-
| 429 / 503 | Respect `Retry-After`. Retry uncertain admission with the saved input/key. |
|
|
93
|
-
| `queued` | A worker must be running and connected to the same PostgreSQL/Redis environment. |
|
|
94
|
-
| `waiting` | Handle the pending input. Tools always require a consumer; wait checkpoints do too. SSE observation does not answer inputs. |
|
|
95
|
-
| `failed` | Inspect `request.reason`, `request.error`, attempts and timeline for provider, routing or deadline failure. |
|
|
65
|
+
const { request } = await client.requests.getSummary(id);
|
|
66
|
+
console.log(request.status, request.output?.text);
|
|
67
|
+
```
|
|
96
68
|
|
|
97
|
-
|
|
69
|
+
`events.terminal` defaults to `"off"`, so this request completes after one model step. The package ships a runnable version of this code: set the `CONSTELLATION_*` variables and run `node node_modules/@lionhummer/constellation/examples/quickstart.mjs`.
|
|
98
70
|
|
|
99
|
-
|
|
71
|
+
## Build an agent with tools
|
|
100
72
|
|
|
101
|
-
|
|
73
|
+
A tool-using agent needs four pieces:
|
|
102
74
|
|
|
103
|
-
|
|
75
|
+
1. **A webhook** that receives events and stores them.
|
|
76
|
+
2. **Hooks** that run your tools and build the next input.
|
|
77
|
+
3. **An `EventStore`**: a table in your database that remembers each event and the response you prepared for it.
|
|
78
|
+
4. **`processEvent`**, which ties the other three together.
|
|
104
79
|
|
|
105
|
-
|
|
80
|
+
### 1. Turn on events
|
|
106
81
|
|
|
107
|
-
|
|
82
|
+
An operator sets a webhook URL and a webhook token on your execution key, in the console under **Keys**. Then request events on every input:
|
|
108
83
|
|
|
109
|
-
|
|
84
|
+
```ts
|
|
85
|
+
const first = {
|
|
86
|
+
idempotencyKey: uuid7(),
|
|
87
|
+
workspaceId: "workspace-123",
|
|
88
|
+
externalId: "conversation-42", // optional: your own ID, useful for searching
|
|
89
|
+
model: "catalog/example-model",
|
|
90
|
+
instructions: "You are a helpful assistant.",
|
|
91
|
+
messages: [{ role: "user" as const, text: "What's the weather in Paris?" }],
|
|
92
|
+
tools: [{
|
|
93
|
+
name: "get_weather",
|
|
94
|
+
description: "Current weather for a city",
|
|
95
|
+
inputSchema: { type: "object", properties: { city: { type: "string" } }, required: ["city"] },
|
|
96
|
+
}],
|
|
97
|
+
events: { terminal: "notify" as const }, // send model.completed / failed / cancelled to the webhook
|
|
98
|
+
};
|
|
99
|
+
await saveConversation(first); // your storage: you need the history for step 2
|
|
100
|
+
await client.requests.submit(first);
|
|
101
|
+
```
|
|
110
102
|
|
|
111
|
-
|
|
103
|
+
If the key has no webhook configured, admission rejects `terminal: "notify"`.
|
|
104
|
+
|
|
105
|
+
### 2. Write the hooks
|
|
112
106
|
|
|
113
107
|
```ts
|
|
114
|
-
import type {
|
|
108
|
+
import type { AgentHooks } from "@lionhummer/constellation";
|
|
115
109
|
|
|
116
|
-
const
|
|
117
|
-
|
|
110
|
+
const hooks: AgentHooks = {
|
|
111
|
+
// One handler per tool name. Calls from one model step run in parallel (toolConcurrency, default 8).
|
|
118
112
|
tools: {
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
afterStep: async (payload, state, { inputId, parentSpanId, signal }) => {
|
|
123
|
-
await settleOnce(inputId, payload, { parentSpanId, signal })
|
|
124
|
-
return shouldContinue(state)
|
|
125
|
-
? { action: "continue", stepInput: await buildCompleteNextStep(state) }
|
|
126
|
-
: { action: "complete" }
|
|
113
|
+
get_weather: async (args, { effectId, signal }) => {
|
|
114
|
+
// effectId is stable for this tool call. Use it to make side effects idempotent.
|
|
115
|
+
return weatherApi.current((args as { city: string }).city, { requestId: effectId, signal });
|
|
127
116
|
},
|
|
128
117
|
},
|
|
129
|
-
}
|
|
130
118
|
|
|
131
|
-
|
|
119
|
+
// Optional. Default: continue if the model called tools, otherwise stop.
|
|
120
|
+
onStepComplete: async ({ event, toolResults }) => (toolResults.length ? "continue" : "stop"),
|
|
121
|
+
|
|
122
|
+
// Required to continue. Return the COMPLETE next input.
|
|
123
|
+
prepareRequest: async ({ event, toolResults }) => {
|
|
124
|
+
const conversation = await loadConversation(event.requestId);
|
|
125
|
+
const messages = [
|
|
126
|
+
...conversation.messages,
|
|
127
|
+
event.data.message, // the assistant message, including its tool calls
|
|
128
|
+
{ role: "tool" as const, toolResults }, // your results, matched by tool-call id
|
|
129
|
+
];
|
|
130
|
+
await saveMessages(event.requestId, messages);
|
|
131
|
+
return { ...conversation.settings, messages };
|
|
132
|
+
},
|
|
133
|
+
|
|
134
|
+
// Optional. Called when a hook throws and when the model step failed or was cancelled.
|
|
135
|
+
onError: async (error, { event }) => {
|
|
136
|
+
logger.warn({ requestId: event.requestId, error }, "agent step failed");
|
|
137
|
+
},
|
|
138
|
+
};
|
|
132
139
|
```
|
|
133
140
|
|
|
134
|
-
`
|
|
141
|
+
`saveConversation`, `loadConversation`, `saveMessages`, `weatherApi` and `logger` stand for your own code.
|
|
135
142
|
|
|
136
|
-
|
|
143
|
+
`conversation.settings` must repeat the first request's `workspaceId`, `userId`, `externalId`, `model`, `planId`, `events` and `timeoutMs`. A continuation that changes any of them is rejected. You may change `messages`, `instructions`, `tools`, `parameters` and `cache`.
|
|
137
144
|
|
|
138
|
-
|
|
145
|
+
### 3. Implement the `EventStore`
|
|
139
146
|
|
|
140
|
-
|
|
141
|
-
const event = await client.webhooks.verify(request, webhookToken)
|
|
142
|
-
if ("inputId" in event) {
|
|
143
|
-
await client.inputs.handle(event.requestId, event.inputId, handlers)
|
|
144
|
-
} else if (event.kind === "request.events") {
|
|
145
|
-
await storeEventBatchOnce(event.requestId, event.afterSeq, event.events)
|
|
146
|
-
} else {
|
|
147
|
-
await finalizeOnce(event.requestId, event.eventSeq,
|
|
148
|
-
await client.requests.getSummary(event.requestId))
|
|
149
|
-
}
|
|
150
|
-
```
|
|
147
|
+
The store is a small table in your own database. It guarantees two things:
|
|
151
148
|
|
|
152
|
-
|
|
149
|
+
- an event you acknowledged is never lost;
|
|
150
|
+
- a response you prepared is sent unchanged, even after a crash.
|
|
153
151
|
|
|
154
|
-
|
|
152
|
+
Do not use an in-memory map in production. Here is a complete PostgreSQL version using [`pg`](https://node-postgres.com/):
|
|
155
153
|
|
|
156
|
-
|
|
154
|
+
```sql
|
|
155
|
+
CREATE TABLE constellation_events (
|
|
156
|
+
event_id text PRIMARY KEY,
|
|
157
|
+
payload jsonb NOT NULL,
|
|
158
|
+
prepared boolean NOT NULL DEFAULT false,
|
|
159
|
+
response jsonb, -- the exact body we send (or resend)
|
|
160
|
+
state text NOT NULL DEFAULT 'pending', -- pending | done | rejected
|
|
161
|
+
rejection jsonb,
|
|
162
|
+
failures integer NOT NULL DEFAULT 0,
|
|
163
|
+
last_error jsonb,
|
|
164
|
+
first_failure_at timestamptz,
|
|
165
|
+
last_failure_at timestamptz,
|
|
166
|
+
received_at timestamptz NOT NULL DEFAULT now()
|
|
167
|
+
);
|
|
168
|
+
```
|
|
157
169
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
170
|
+
```ts
|
|
171
|
+
import pg from "pg";
|
|
172
|
+
import { PermanentEventError, type EventStore, type WebhookEvent } from "@lionhummer/constellation";
|
|
173
|
+
|
|
174
|
+
const db = new pg.Pool();
|
|
175
|
+
|
|
176
|
+
export const store: EventStore & { pending(): Promise<WebhookEvent[]> } = {
|
|
177
|
+
// Save the event before acknowledging the webhook. Duplicates must match.
|
|
178
|
+
async accept(event) {
|
|
179
|
+
await db.query(
|
|
180
|
+
`INSERT INTO constellation_events (event_id, payload) VALUES ($1, $2)
|
|
181
|
+
ON CONFLICT (event_id) DO NOTHING`, [event.id, event]);
|
|
182
|
+
const { rows } = await db.query(
|
|
183
|
+
`SELECT payload = $2::jsonb AS same FROM constellation_events WHERE event_id = $1`, [event.id, event]);
|
|
184
|
+
if (!rows[0]?.same) throw new Error(`event ${event.id} redelivered with a different payload`);
|
|
185
|
+
},
|
|
163
186
|
|
|
164
|
-
|
|
187
|
+
// Run `prepare` at most once per event at a time, and save its result before returning it.
|
|
188
|
+
async prepare(event, prepare) {
|
|
189
|
+
const tx = await db.connect();
|
|
190
|
+
try {
|
|
191
|
+
await tx.query("BEGIN");
|
|
192
|
+
const { rows } = await tx.query(
|
|
193
|
+
`SELECT prepared, response, state, rejection FROM constellation_events
|
|
194
|
+
WHERE event_id = $1 FOR UPDATE`, [event.id]); // serializes concurrent processing
|
|
195
|
+
const row = rows[0];
|
|
196
|
+
if (row.state === "rejected") {
|
|
197
|
+
const r = row.rejection;
|
|
198
|
+
throw new PermanentEventError(r.status, r.code, r.message);
|
|
199
|
+
}
|
|
200
|
+
if (row.prepared) { await tx.query("COMMIT"); return row.response; }
|
|
201
|
+
const response = await prepare(); // runs your tools and hooks
|
|
202
|
+
await tx.query(
|
|
203
|
+
`UPDATE constellation_events SET prepared = true, response = $2 WHERE event_id = $1`,
|
|
204
|
+
[event.id, response]);
|
|
205
|
+
await tx.query("COMMIT");
|
|
206
|
+
return response;
|
|
207
|
+
} catch (error) {
|
|
208
|
+
await tx.query("ROLLBACK");
|
|
209
|
+
throw error;
|
|
210
|
+
} finally {
|
|
211
|
+
tx.release();
|
|
212
|
+
}
|
|
213
|
+
},
|
|
165
214
|
|
|
166
|
-
|
|
215
|
+
async complete(eventId) {
|
|
216
|
+
await db.query(`UPDATE constellation_events SET state = 'done' WHERE event_id = $1`, [eventId]);
|
|
217
|
+
},
|
|
167
218
|
|
|
168
|
-
|
|
219
|
+
// A permanent rejection: never resend this response.
|
|
220
|
+
async reject(eventId, error) {
|
|
221
|
+
await db.query(
|
|
222
|
+
`UPDATE constellation_events SET state = 'rejected', rejection = $2 WHERE event_id = $1`,
|
|
223
|
+
[eventId, { status: error.status, code: error.code, message: error.message }]);
|
|
224
|
+
},
|
|
169
225
|
|
|
170
|
-
|
|
226
|
+
// Optional: a retryable failure. Alert when failures pile up or get old.
|
|
227
|
+
async retry(eventId, error) {
|
|
228
|
+
await db.query(
|
|
229
|
+
`UPDATE constellation_events SET failures = failures + 1, last_error = $2,
|
|
230
|
+
first_failure_at = coalesce(first_failure_at, now()), last_failure_at = now()
|
|
231
|
+
WHERE event_id = $1`, [eventId, error]);
|
|
232
|
+
},
|
|
171
233
|
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
234
|
+
// Not part of EventStore: used by the recovery sweep below.
|
|
235
|
+
async pending() {
|
|
236
|
+
const { rows } = await db.query(
|
|
237
|
+
`SELECT payload FROM constellation_events WHERE state = 'pending' ORDER BY received_at LIMIT 100`);
|
|
238
|
+
return rows.map((row) => row.payload);
|
|
239
|
+
},
|
|
240
|
+
};
|
|
176
241
|
```
|
|
177
242
|
|
|
178
|
-
`
|
|
243
|
+
`prepare` keeps a row lock while your tools run. That is acceptable for tools that finish in seconds or minutes. For very long work, replace the lock with a lease column.
|
|
179
244
|
|
|
180
|
-
|
|
245
|
+
### 4. Receive webhooks and process events
|
|
181
246
|
|
|
182
|
-
|
|
247
|
+
Acknowledge quickly, then process in the background. The webhook handler below works with any framework that uses the web `Request`/`Response` types (Hono, Next.js route handlers, Bun, Deno):
|
|
183
248
|
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
249
|
+
```ts
|
|
250
|
+
import { processEvent, PermanentEventError } from "@lionhummer/constellation";
|
|
251
|
+
|
|
252
|
+
export async function webhook(request: Request): Promise<Response> {
|
|
253
|
+
let event;
|
|
254
|
+
try {
|
|
255
|
+
event = await client.webhooks.verify(request, process.env.CONSTELLATION_WEBHOOK_TOKEN!);
|
|
256
|
+
} catch {
|
|
257
|
+
return new Response(null, { status: 401 });
|
|
258
|
+
}
|
|
259
|
+
await store.accept(event); // if this throws, the 5xx makes Constellation redeliver
|
|
260
|
+
void handle(event); // don't make the webhook wait for your tools
|
|
261
|
+
return new Response(null, { status: 202 });
|
|
189
262
|
}
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
263
|
+
|
|
264
|
+
async function handle(event: WebhookEvent) {
|
|
265
|
+
try {
|
|
266
|
+
await processEvent(client, store, event, hooks);
|
|
267
|
+
} catch (error) {
|
|
268
|
+
if (error instanceof PermanentEventError) {
|
|
269
|
+
await markConversationFailed(event.requestId, error); // will not be retried
|
|
270
|
+
} else {
|
|
271
|
+
logger.warn({ eventId: event.id, error }, "event processing will be retried");
|
|
272
|
+
}
|
|
273
|
+
}
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
// Optional: resume unfinished events right after a restart.
|
|
277
|
+
for (const event of await store.pending()) void handle(event);
|
|
197
278
|
```
|
|
198
279
|
|
|
199
|
-
|
|
280
|
+
Don't run tools inside the webhook call. Constellation waits 5 seconds for the acknowledgement by default, and nothing in Constellation waits for your tools.
|
|
200
281
|
|
|
201
|
-
|
|
282
|
+
If your process dies while handling an event, you don't need a poller to recover it. Constellation sends an unanswered `model.completed` event again after the response window (10 minutes by default), up to 3 times, and stops as soon as your response arrives. A re-sent event has the same `id`, so `store.accept` and `store.prepare` resume it from your saved checkpoint. If the event is still being processed when the re-send arrives, acknowledge it anyway. Keep your processing lease shorter than the response window and renew it while work continues, so a re-send after a crash can take the event over.
|
|
202
283
|
|
|
203
|
-
|
|
284
|
+
## What `processEvent` does
|
|
204
285
|
|
|
205
|
-
|
|
286
|
+
For each event:
|
|
206
287
|
|
|
207
|
-
|
|
288
|
+
1. **Prepare the response, at most once.** `store.prepare` returns the saved response if one exists. Otherwise the SDK builds one:
|
|
289
|
+
- For `model.completed`, it runs every tool call in parallel and calls `onStepComplete`. To continue, it calls `prepareRequest`, which produces a continuation. To stop, it produces an `observe` response.
|
|
290
|
+
- For `model.failed` and `model.cancelled`, it calls `onError(event.data)` and produces an `observe` response.
|
|
208
291
|
|
|
209
|
-
|
|
292
|
+
The response carries the idempotency key `event.id`, `inReplyTo: { requestId, eventId }` and the telemetry recorded while preparing. The store saves it before anything is sent.
|
|
293
|
+
2. **Submit it** through `requests.submit`. Retries send the saved body unchanged.
|
|
294
|
+
3. **Record the outcome**:
|
|
210
295
|
|
|
211
|
-
|
|
296
|
+
| Server answer | Meaning | `processEvent` |
|
|
297
|
+
| --- | --- | --- |
|
|
298
|
+
| `200`, `outcome: "accepted"` | Your response was accepted. A continuation starts the next step (`scheduled: true`). | `store.complete`, resolves |
|
|
299
|
+
| `200`, `outcome: "already_accepted"` | A retry of a response that was already accepted, for example after a lost reply. Nothing runs twice. | `store.complete`, resolves |
|
|
300
|
+
| `200`, `outcome: "event_closed"` | The request was cancelled or expired before your continuation arrived. Nothing continues; your telemetry is still recorded (`observationsRecorded`). | `store.complete`, resolves |
|
|
301
|
+
| `400 bad_request` | The response itself is invalid, for example a changed model or a bad message. | `store.reject`, throws `PermanentEventError` |
|
|
302
|
+
| `409 response_conflict` | A *different* response to this event was already accepted. | `store.reject`, throws `PermanentEventError` |
|
|
303
|
+
| `410 observations_expired` | The request ended more than 24 hours ago. | `store.reject`, throws `PermanentEventError` |
|
|
304
|
+
| Anything else: network error, timeout, `5xx`, `429`, `409 conflict`, `401`, `403` | Temporary. | `store.retry`, throws. The next attempt resends the same body. |
|
|
212
305
|
|
|
213
|
-
|
|
306
|
+
The permanent codes are exported as `PERMANENT_RESPONSE_CODES`. The receipt passed to `store.complete` includes `outcome`, so your store can record that a request ended without continuing.
|
|
214
307
|
|
|
215
|
-
|
|
308
|
+
Fix `401` and `403` by correcting the key: the saved response is still sent unchanged afterwards.
|
|
216
309
|
|
|
217
|
-
|
|
310
|
+
## When your code fails
|
|
218
311
|
|
|
219
|
-
|
|
312
|
+
| What happened | Without `onError` | With `onError` |
|
|
313
|
+
| --- | --- | --- |
|
|
314
|
+
| A tool or hook throws | `processEvent` throws and the event is retried later. Tools may run again, so use `effectId`. | `onError` runs. If it returns, the request stops and the error is recorded in telemetry. If it throws, the event is retried. |
|
|
315
|
+
| No handler for a tool the model called, or `prepareRequest` is missing when continuing | The request is marked **failed** and `HookConfigurationError` (a `PermanentEventError`) is thrown. | `onError` runs, as above. |
|
|
316
|
+
| The model step failed or was cancelled (`model.failed`, `model.cancelled`) | An `observe` response records your telemetry. | `onError(event.data)` runs first. |
|
|
220
317
|
|
|
221
|
-
##
|
|
318
|
+
## Side effects and duplicates
|
|
222
319
|
|
|
223
|
-
|
|
320
|
+
Hooks can run more than once. A crash after a tool ran, but before the prepared response was saved, means the tool runs again. Constellation cannot make your side effects exactly-once; your code must:
|
|
224
321
|
|
|
225
|
-
|
|
322
|
+
- pass `effectId` to anything with side effects (payments, emails, writes). It is stable for one tool call of one event;
|
|
323
|
+
- settle per-step billing or usage once per `event.id`;
|
|
324
|
+
- honour the `signal` you receive and put timeouts on external calls.
|
|
226
325
|
|
|
227
|
-
|
|
326
|
+
## Live output
|
|
228
327
|
|
|
229
|
-
`
|
|
328
|
+
`requests.streamEvents(id, { after, signal })` streams `model.delta` events (text and reasoning as it is generated), followed by each step's final event. It reconnects automatically. To resume after a restart, pass the last `event.sequence` you processed as `after`. Streaming is for display only: run tools from the webhook, never from the stream. Stopping the stream does not cancel the request; `requests.cancel(id)` does.
|
|
230
329
|
|
|
231
|
-
|
|
330
|
+
## Telemetry
|
|
331
|
+
|
|
332
|
+
Hooks receive a `telemetry` object. Spans and logs you record are attached to the step that produced the event, so they appear in the request's timeline in the console next to Constellation's own spans:
|
|
333
|
+
|
|
334
|
+
```ts
|
|
335
|
+
tools: {
|
|
336
|
+
search: async (args, { telemetry }) =>
|
|
337
|
+
telemetry.span("search.query", { index: "docs" }, async (span) => {
|
|
338
|
+
span.log("info", "querying index");
|
|
339
|
+
return searchIndex(args);
|
|
340
|
+
}),
|
|
341
|
+
},
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
The SDK records `client.processing` and one span per tool automatically. A response carries at most 512 spans, 128 logs and 1 MiB of telemetry; the SDK trims larger batches and records what it omitted. Do not put secrets or personal data in span names, attributes or log messages.
|
|
345
|
+
|
|
346
|
+
## Limits and timeouts
|
|
347
|
+
|
|
348
|
+
| Setting | Default | Notes |
|
|
349
|
+
| --- | --- | --- |
|
|
350
|
+
| `timeoutMs` | 30 minutes | Deadline for the whole request, including your processing between steps. |
|
|
351
|
+
| `compressRequests` client option | `true` | Request bodies of 16 KiB or more are sent gzip-compressed. Every step resends the full conversation, so this cuts upload time several-fold. |
|
|
352
|
+
| `attemptTimeoutMs`, `firstByteTimeoutMs`, `idleTimeoutMs` | not set | Optional per-provider-call limits. Operators also run stall watchdogs: by default 120 s to the first response byte and 300 s with no data. Keepalives count as data. |
|
|
353
|
+
| `maxProviderAttempts` | 32 | Provider attempts per step, including fallbacks (1–256). |
|
|
354
|
+
| `toolConcurrency` hook option | 8 | Parallel tool calls per event (1–128). |
|
|
355
|
+
| Telemetry per response | 512 spans, 128 logs, 1 MiB | Larger batches are trimmed by the SDK. |
|
|
356
|
+
| Late observations | 24 hours | After the request ends, `observe` responses are accepted for 24 hours. |
|
|
357
|
+
| Retry identity | 7 days (operator setting) | After a request ends, an identical retry of its `idempotencyKey` returns the original request for this long. Later, the same key starts a new request. |
|
|
358
|
+
|
|
359
|
+
A failed provider call falls back to the next provider only before any output has been streamed. After that, the step fails instead of mixing output from two models.
|
|
360
|
+
|
|
361
|
+
## Client reference
|
|
362
|
+
|
|
363
|
+
| Method | What it does |
|
|
364
|
+
| --- | --- |
|
|
365
|
+
| `models.list(scope)` | Models this key can use, with supported parameters. No provider call. |
|
|
366
|
+
| `models.resolve(input)` | Which providers would serve a model, without calling them. |
|
|
367
|
+
| `requests.submit(input)` | Start a request, continue it (`inReplyTo`), or stop it (`action: "observe"`). |
|
|
368
|
+
| `requests.preview(input)` | Validate a complete input and show routing, without calling a provider. |
|
|
369
|
+
| `requests.cancel(id)` | Cancel a request. |
|
|
370
|
+
| `requests.getState(id)` | Committed state: status, steps, attempt outcomes, usage and cost. |
|
|
371
|
+
| `requests.getSummary(id)`, `requests.get(id)` | Request detail, without or with spans and logs. |
|
|
372
|
+
| `requests.list(query)` | Page through requests, e.g. by `externalId` or `status`. |
|
|
373
|
+
| `requests.listEvents(id, after)`, `requests.streamEvents(id)` | Replay events after a sequence number, or stream them live. |
|
|
374
|
+
| `requests.listRecords(id, kind)` | Page steps, attempts, spans or logs. |
|
|
375
|
+
| `requests.getTimeline(id)`, `requests.getTimings(id)` | Trace and timing breakdown. |
|
|
376
|
+
| `requests.getHttpExchange(id, attemptId)`, `requests.getHttpBody(...)` | The provider HTTP call captured for one attempt. |
|
|
377
|
+
| `webhooks.verify(request, token)` | Authenticate and parse an incoming webhook. |
|
|
378
|
+
| `telemetry.*` | Read-only reporting. Returns metadata only unless you pass `include: "content"`. Needs telemetry permission. |
|
|
379
|
+
| `management.registry.*` | Edit the model registry. Needs management permission. |
|
|
380
|
+
| `discover()` | The OpenAPI document for exactly what this key may call. |
|
|
381
|
+
|
|
382
|
+
Exports for event processing: `processEvent`, `prepareResponse` (build a response without submitting it), `EventReceiver`, `ClientTelemetry`, `PermanentEventError`, `HookConfigurationError`, `PERMANENT_RESPONSE_CODES`, `ConstellationError` and `uuid7`.
|
|
383
|
+
|
|
384
|
+
## Troubleshooting
|
|
385
|
+
|
|
386
|
+
| Symptom | What to check |
|
|
387
|
+
| --- | --- |
|
|
388
|
+
| `models.list` is empty | The key's workspace/user scope, an access rule for that workspace, and an enabled model and offering. |
|
|
389
|
+
| `terminal: "notify"` is rejected | The execution key has no webhook URL configured. |
|
|
390
|
+
| The same webhook keeps arriving | Your endpoint returned non-2xx or took longer than the delivery timeout (5 s by default). Return 202 after `store.accept` and process in the background. |
|
|
391
|
+
| `400` on a continuation | Model, workspace, user, `externalId`, plan, events or `timeoutMs` differ from the first request, or the message history is malformed. |
|
|
392
|
+
| `409 response_conflict` | Your code produced a different response for an event that was already answered. Answers must come from the saved response. |
|
|
393
|
+
| Tools ran twice | Expected after a crash. Make side effects idempotent with `effectId`. |
|
|
394
|
+
| A step finished with `outcome: "event_closed"` | The request was cancelled or its `timeoutMs` passed while your code was working. |
|
|
395
|
+
| Client spans missing from the timeline | Telemetry is sent with the response. Check that `processEvent` reached the server. |
|
|
396
|
+
| Request expired while tools were running | `timeoutMs` covers your processing time too. Raise it on the first request. |
|
|
397
|
+
|
|
398
|
+
More detail: [application integration](../../docs/integration.md), [HTTP API](../../docs/api.md) and the [OpenAPI document](../../docs/registry-api.openapi.json).
|