@trigger.dev/sdk 0.0.0-prerelease-20260908122921 → 0.0.0-prerelease-streamfix-20260909094302
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/commonjs/imports/ai-runtime-cjs.cjs.map +1 -1
- package/dist/commonjs/imports/ai-runtime.js +0 -2
- package/dist/commonjs/v3/ai.d.ts +16 -199
- package/dist/commonjs/v3/ai.js +102 -983
- package/dist/commonjs/v3/ai.js.map +1 -1
- package/dist/commonjs/v3/chat-client.d.ts +2 -3
- package/dist/commonjs/v3/chat-client.js +5 -31
- package/dist/commonjs/v3/chat-client.js.map +1 -1
- package/dist/commonjs/v3/chat-react.d.ts +0 -34
- package/dist/commonjs/v3/chat-react.js +1 -47
- package/dist/commonjs/v3/chat-react.js.map +1 -1
- package/dist/commonjs/v3/chat-server.d.ts +6 -42
- package/dist/commonjs/v3/chat-server.js +7 -52
- package/dist/commonjs/v3/chat-server.js.map +1 -1
- package/dist/commonjs/v3/chat.d.ts +10 -81
- package/dist/commonjs/v3/chat.js +46 -292
- package/dist/commonjs/v3/chat.js.map +1 -1
- package/dist/commonjs/v3/sessions.d.ts +2 -15
- package/dist/commonjs/v3/sessions.js +1 -12
- package/dist/commonjs/v3/sessions.js.map +1 -1
- package/dist/commonjs/v3/shared.js +36 -30
- package/dist/commonjs/v3/shared.js.map +1 -1
- package/dist/commonjs/v3/test/mock-chat-agent.d.ts +0 -43
- package/dist/commonjs/v3/test/mock-chat-agent.js +0 -90
- package/dist/commonjs/v3/test/mock-chat-agent.js.map +1 -1
- package/dist/commonjs/v3/test/test-session-handle.js +0 -6
- package/dist/commonjs/v3/test/test-session-handle.js.map +1 -1
- package/dist/commonjs/version.js +1 -1
- package/dist/esm/imports/ai-runtime.d.ts +2 -2
- package/dist/esm/imports/ai-runtime.js +2 -2
- package/dist/esm/imports/ai-runtime.js.map +1 -1
- package/dist/esm/v3/ai.d.ts +16 -199
- package/dist/esm/v3/ai.js +103 -984
- package/dist/esm/v3/ai.js.map +1 -1
- package/dist/esm/v3/chat-client.d.ts +2 -3
- package/dist/esm/v3/chat-client.js +5 -31
- package/dist/esm/v3/chat-client.js.map +1 -1
- package/dist/esm/v3/chat-react.d.ts +0 -34
- package/dist/esm/v3/chat-react.js +1 -46
- package/dist/esm/v3/chat-react.js.map +1 -1
- package/dist/esm/v3/chat-server.d.ts +6 -42
- package/dist/esm/v3/chat-server.js +8 -53
- package/dist/esm/v3/chat-server.js.map +1 -1
- package/dist/esm/v3/chat.d.ts +10 -81
- package/dist/esm/v3/chat.js +47 -293
- package/dist/esm/v3/chat.js.map +1 -1
- package/dist/esm/v3/sessions.d.ts +2 -15
- package/dist/esm/v3/sessions.js +1 -11
- package/dist/esm/v3/sessions.js.map +1 -1
- package/dist/esm/v3/shared.js +23 -17
- package/dist/esm/v3/shared.js.map +1 -1
- package/dist/esm/v3/test/mock-chat-agent.d.ts +0 -43
- package/dist/esm/v3/test/mock-chat-agent.js +2 -92
- package/dist/esm/v3/test/mock-chat-agent.js.map +1 -1
- package/dist/esm/v3/test/test-session-handle.js +0 -6
- package/dist/esm/v3/test/test-session-handle.js.map +1 -1
- package/dist/esm/version.js +1 -1
- package/docs/ai-chat/actions.mdx +23 -55
- package/docs/ai-chat/anatomy.mdx +3 -3
- package/docs/ai-chat/backend.mdx +48 -125
- package/docs/ai-chat/background-injection.mdx +19 -67
- package/docs/ai-chat/client-protocol.mdx +4 -5
- package/docs/ai-chat/compaction.mdx +7 -11
- package/docs/ai-chat/custom-agents.mdx +0 -23
- package/docs/ai-chat/fast-starts.mdx +20 -27
- package/docs/ai-chat/frontend.mdx +14 -17
- package/docs/ai-chat/migrating-from-a-route-handler.mdx +14 -16
- package/docs/ai-chat/patterns/skills.mdx +10 -7
- package/docs/ai-chat/patterns/version-upgrades.mdx +6 -79
- package/docs/ai-chat/pending-messages.mdx +3 -3
- package/docs/ai-chat/prompt-caching.mdx +25 -23
- package/docs/ai-chat/quick-start.mdx +11 -11
- package/docs/ai-chat/reference.mdx +5 -12
- package/docs/ai-chat/sessions.mdx +1 -6
- package/docs/ai-chat/testing.mdx +1 -2
- package/docs/ai-chat/tools.mdx +13 -18
- package/docs/ai-chat/upgrade-guide.mdx +2 -2
- package/docs/apikeys.mdx +45 -27
- package/docs/deployment/overview.mdx +8 -4
- package/docs/deployment/preview-branches.mdx +4 -4
- package/docs/deployment/version-skew-protection.mdx +0 -62
- package/docs/manual-setup.mdx +7 -7
- package/docs/mcp-tools.mdx +0 -9
- package/docs/quick-start.mdx +3 -3
- package/docs/realtime/auth.mdx +1 -1
- package/docs/self-hosting/security.mdx +0 -5
- package/docs/tasks/scheduled.mdx +0 -24
- package/docs/triggering.mdx +1 -1
- package/package.json +4 -4
- package/skills/trigger-authoring-chat-agent/SKILL.md +27 -38
- package/skills/trigger-chat-agent-advanced/SKILL.md +12 -31
- package/dist/commonjs/v3/chatVersionSkew.d.ts +0 -12
- package/dist/commonjs/v3/chatVersionSkew.js +0 -30
- package/dist/commonjs/v3/chatVersionSkew.js.map +0 -1
- package/dist/commonjs/v3/externalDeploymentId.d.ts +0 -23
- package/dist/commonjs/v3/externalDeploymentId.js +0 -43
- package/dist/commonjs/v3/externalDeploymentId.js.map +0 -1
- package/dist/esm/v3/chatVersionSkew.d.ts +0 -12
- package/dist/esm/v3/chatVersionSkew.js +0 -27
- package/dist/esm/v3/chatVersionSkew.js.map +0 -1
- package/dist/esm/v3/externalDeploymentId.d.ts +0 -23
- package/dist/esm/v3/externalDeploymentId.js +0 -38
- package/dist/esm/v3/externalDeploymentId.js.map +0 -1
- package/docs/ai-chat/patterns/native-compaction.mdx +0 -310
- package/docs/reports.mdx +0 -157
- package/docs/troubleshooting-zod.mdx +0 -158
|
@@ -143,12 +143,6 @@ await waitUntilComplete();
|
|
|
143
143
|
5s timeout, before `onTurnComplete`). `chat.inject(messages)` queues `ModelMessage[]` that drain at
|
|
144
144
|
the next turn start or `prepareStep` boundary.
|
|
145
145
|
|
|
146
|
-
Two lanes, decided by role. A `role: "system"` message goes to the model's instructions, where it is
|
|
147
|
-
trusted like the system prompt, and applies to the next turn only. Any other role joins the
|
|
148
|
-
conversation and is untrusted by construction, so put checkable facts there and directives in the
|
|
149
|
-
system lane. The instructions lane reaches the model only through the managed `streamText` (or a
|
|
150
|
-
`chat.toStreamTextOptions()` spread), since that is where the SDK can set instructions.
|
|
151
|
-
|
|
152
146
|
```ts
|
|
153
147
|
export const myChat = chat.agent({
|
|
154
148
|
id: "my-chat",
|
|
@@ -160,9 +154,8 @@ export const myChat = chat.agent({
|
|
|
160
154
|
})()
|
|
161
155
|
);
|
|
162
156
|
},
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
streamText({ messages, abortSignal: signal, stopWhen: stepCountIs(15) }),
|
|
157
|
+
run: async ({ messages, signal }) =>
|
|
158
|
+
streamText({ ...chat.toStreamTextOptions({ registry }), messages, abortSignal: signal, stopWhen: stepCountIs(15) }),
|
|
166
159
|
});
|
|
167
160
|
```
|
|
168
161
|
|
|
@@ -170,9 +163,8 @@ export const myChat = chat.agent({
|
|
|
170
163
|
|
|
171
164
|
`compaction.shouldCompact` decides when, `summarize` produces the summary that replaces the model
|
|
172
165
|
messages. UI messages are preserved by default (customize via `compactUIMessages`). The `prepareStep`
|
|
173
|
-
that performs inner-loop compaction
|
|
174
|
-
you pass after
|
|
175
|
-
switching compaction off.
|
|
166
|
+
that performs inner-loop compaction is auto-injected by `chat.toStreamTextOptions()`; a `prepareStep`
|
|
167
|
+
you pass after the spread wins.
|
|
176
168
|
|
|
177
169
|
```ts
|
|
178
170
|
compaction: {
|
|
@@ -190,14 +182,7 @@ compaction: {
|
|
|
190
182
|
`actionSchema` validates; `onAction` mutates via `chat.history` (`slice`, `replace`, `rollbackTo`,
|
|
191
183
|
`remove`, `getPendingToolCalls`, `extractNewToolResults`). Actions fire `hydrateMessages` and
|
|
192
184
|
`onAction` only, never `run()` or the turn hooks. Return a `StreamTextResult`, string, or `UIMessage`
|
|
193
|
-
to also emit a model response
|
|
194
|
-
carries the agent's prompt and tools like any other turn.
|
|
195
|
-
|
|
196
|
-
Persistence splits by model. Without `hydrateMessages` the runtime snapshots the conversation after
|
|
197
|
-
an action that changed it, so a rollback or a returned response survives the run ending. With
|
|
198
|
-
`hydrateMessages` your store is the source of truth and the runtime does not write, so mirror every
|
|
199
|
-
mutation yourself: a regenerate is a delete and an insert, and `chat.pipeAndCapture` hands back the
|
|
200
|
-
same assistant message the runtime would have captured.
|
|
185
|
+
to also emit a model response.
|
|
201
186
|
|
|
202
187
|
```ts
|
|
203
188
|
export const myChat = chat.agent({
|
|
@@ -210,8 +195,7 @@ export const myChat = chat.agent({
|
|
|
210
195
|
if (action.type === "undo") chat.history.slice(0, -2);
|
|
211
196
|
if (action.type === "rollback") chat.history.rollbackTo(action.targetMessageId);
|
|
212
197
|
},
|
|
213
|
-
run: async ({ messages, signal
|
|
214
|
-
streamText({ model: anthropic("claude-sonnet-4-5"), messages, abortSignal: signal }),
|
|
198
|
+
run: async ({ messages, signal }) => streamText({ model: anthropic("claude-sonnet-4-5"), messages, abortSignal: signal }),
|
|
215
199
|
});
|
|
216
200
|
```
|
|
217
201
|
|
|
@@ -226,19 +210,18 @@ must be **schema-only** (a module importing `ai` + `zod` only); heavy executes s
|
|
|
226
210
|
|
|
227
211
|
```ts
|
|
228
212
|
import { chat } from "@trigger.dev/sdk/chat-server";
|
|
213
|
+
import { streamText, stepCountIs } from "ai";
|
|
229
214
|
import { anthropic } from "@ai-sdk/anthropic";
|
|
230
215
|
import { headStartTools } from "@/lib/chat-tools/schemas";
|
|
231
216
|
|
|
232
217
|
export const chatHandler = chat.headStart({
|
|
233
218
|
agentId: "my-chat",
|
|
234
|
-
|
|
235
|
-
// and `abortSignal`: the handover needs `stopWhen: stepCountIs(1)` so the agent,
|
|
236
|
-
// not this handler, runs step 2 onward. Passing any of them is a type error.
|
|
237
|
-
run: async ({ streamText }) =>
|
|
219
|
+
run: async ({ chat: helper }) =>
|
|
238
220
|
streamText({
|
|
221
|
+
...helper.toStreamTextOptions({ tools: headStartTools }),
|
|
239
222
|
model: anthropic("claude-sonnet-4-6"),
|
|
240
223
|
system: "You are helpful.",
|
|
241
|
-
|
|
224
|
+
stopWhen: stepCountIs(15),
|
|
242
225
|
}),
|
|
243
226
|
});
|
|
244
227
|
// Next.js: export const POST = chatHandler; Transport: headStart: "/api/chat"
|
|
@@ -265,10 +248,8 @@ export const myChat = chat.agent({
|
|
|
265
248
|
### 8. Pending messages (mid-stream user input)
|
|
266
249
|
|
|
267
250
|
A message sent while a turn is streaming should NOT cancel the stream. Configure
|
|
268
|
-
`pendingMessages` (`shouldInject`, `prepare`, `onReceived`, `onInjected`) on the agent so the
|
|
269
|
-
|
|
270
|
-
of the conversation your hooks see, so it arrives in `uiMessages` and `newUIMessages` at
|
|
271
|
-
`onTurnComplete` and an app persisting from there stores it without extra work. On the frontend, `usePendingMessages`
|
|
251
|
+
`pendingMessages` (`shouldInject`, `prepare`, `onReceived`, `onInjected`) on the agent so the SDK's
|
|
252
|
+
auto-injected `prepareStep` folds them in at the next boundary. On the frontend, `usePendingMessages`
|
|
272
253
|
returns `pending`, `steer(text)`, `queue(text)`, and `promoteToSteering(id)`; send via
|
|
273
254
|
`transport.sendPendingMessage(chatId, uiMessage, metadata?)`.
|
|
274
255
|
|
|
@@ -1,12 +0,0 @@
|
|
|
1
|
-
export type ChatVersionSkewPolicy = "follow" | "hold";
|
|
2
|
-
export type SessionVersionPin = {
|
|
3
|
-
externalDeploymentId?: string | null;
|
|
4
|
-
lockToVersion?: string;
|
|
5
|
-
};
|
|
6
|
-
export type ResolvePinToFollowOptions = {
|
|
7
|
-
policy: ChatVersionSkewPolicy | undefined;
|
|
8
|
-
deployedExternalId: string | undefined;
|
|
9
|
-
upgradeAlreadyRequested: boolean;
|
|
10
|
-
readPin: () => Promise<SessionVersionPin | undefined>;
|
|
11
|
-
};
|
|
12
|
-
export declare function resolvePinToFollow(options: ResolvePinToFollowOptions): Promise<string | undefined>;
|
|
@@ -1,30 +0,0 @@
|
|
|
1
|
-
"use strict";
|
|
2
|
-
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
-
exports.resolvePinToFollow = resolvePinToFollow;
|
|
4
|
-
const v3_1 = require("@trigger.dev/core/v3");
|
|
5
|
-
async function resolvePinToFollow(options) {
|
|
6
|
-
if (options.policy === "hold") {
|
|
7
|
-
return undefined;
|
|
8
|
-
}
|
|
9
|
-
if (options.upgradeAlreadyRequested) {
|
|
10
|
-
return undefined;
|
|
11
|
-
}
|
|
12
|
-
if (!options.deployedExternalId) {
|
|
13
|
-
return undefined;
|
|
14
|
-
}
|
|
15
|
-
const [error, pin] = await (0, v3_1.tryCatch)(options.readPin());
|
|
16
|
-
if (error || !pin) {
|
|
17
|
-
return undefined;
|
|
18
|
-
}
|
|
19
|
-
if (!pin.externalDeploymentId) {
|
|
20
|
-
return undefined;
|
|
21
|
-
}
|
|
22
|
-
if (pin.lockToVersion) {
|
|
23
|
-
return undefined;
|
|
24
|
-
}
|
|
25
|
-
if (pin.externalDeploymentId === options.deployedExternalId) {
|
|
26
|
-
return undefined;
|
|
27
|
-
}
|
|
28
|
-
return pin.externalDeploymentId;
|
|
29
|
-
}
|
|
30
|
-
//# sourceMappingURL=chatVersionSkew.js.map
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"chatVersionSkew.js","sourceRoot":"","sources":["../../../src/v3/chatVersionSkew.ts"],"names":[],"mappings":";;;AAAA,6CAAgD;AAgBzC,KAAK,6BACV,OAAkC;IAElC,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QAC9B,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,OAAO,CAAC,uBAAuB,EAAE,CAAC;QACpC,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,CAAC,OAAO,CAAC,kBAAkB,EAAE,CAAC;QAChC,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,CAAC,KAAK,EAAE,GAAG,CAAC,GAAG,MAAM,IAAA,aAAQ,EAAC,OAAO,CAAC,OAAO,EAAE,CAAC,CAAC;IAEvD,IAAI,KAAK,IAAI,CAAC,GAAG,EAAE,CAAC;QAClB,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,CAAC,GAAG,CAAC,oBAAoB,EAAE,CAAC;QAC9B,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,GAAG,CAAC,aAAa,EAAE,CAAC;QACtB,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,GAAG,CAAC,oBAAoB,KAAK,OAAO,CAAC,kBAAkB,EAAE,CAAC;QAC5D,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,OAAO,GAAG,CAAC,oBAAoB,CAAC;AAClC,CAAC"}
|
|
@@ -1,23 +0,0 @@
|
|
|
1
|
-
import { type SessionTriggerConfig } from "@trigger.dev/core/v3";
|
|
2
|
-
/** Reads an env var unless the scope opted out of ambient context (`inheritContext: false`). */
|
|
3
|
-
export declare function scopedEnvVar(name: string): string | undefined;
|
|
4
|
-
/**
|
|
5
|
-
* Precedence: an explicit value, the client config, `TRIGGER_EXTERNAL_DEPLOYMENT_ID`, then
|
|
6
|
-
* platform/CI commit vars when automatic skew protection is on. `null` is an explicit opt-out.
|
|
7
|
-
*/
|
|
8
|
-
export declare function resolveTriggerExternalDeploymentId(explicit?: string | null): string | undefined;
|
|
9
|
-
/** A session trigger config as callers write it: the pin is optional, and `null` opts out. */
|
|
10
|
-
type TriggerConfigInput = Omit<SessionTriggerConfig, "externalDeploymentId"> & {
|
|
11
|
-
externalDeploymentId?: string | null;
|
|
12
|
-
};
|
|
13
|
-
/**
|
|
14
|
-
* Fill in `triggerConfig.externalDeploymentId` on an outgoing session-create body, at each point
|
|
15
|
-
* one leaves the SDK. Discovery has to happen here rather than server-side: the commit SHA lives
|
|
16
|
-
* in the calling application's runtime, so the caller's build is what selects the agent version.
|
|
17
|
-
*/
|
|
18
|
-
export declare function withResolvedExternalDeploymentId<TBody extends {
|
|
19
|
-
triggerConfig: TriggerConfigInput;
|
|
20
|
-
}>(body: TBody): TBody & {
|
|
21
|
-
triggerConfig: SessionTriggerConfig;
|
|
22
|
-
};
|
|
23
|
-
export {};
|
|
@@ -1,43 +0,0 @@
|
|
|
1
|
-
"use strict";
|
|
2
|
-
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
-
exports.scopedEnvVar = scopedEnvVar;
|
|
4
|
-
exports.resolveTriggerExternalDeploymentId = resolveTriggerExternalDeploymentId;
|
|
5
|
-
exports.withResolvedExternalDeploymentId = withResolvedExternalDeploymentId;
|
|
6
|
-
const v3_1 = require("@trigger.dev/core/v3");
|
|
7
|
-
/** Reads an env var unless the scope opted out of ambient context (`inheritContext: false`). */
|
|
8
|
-
function scopedEnvVar(name) {
|
|
9
|
-
const scope = v3_1.sdkScope.getStore();
|
|
10
|
-
if (scope && !scope.inheritContext)
|
|
11
|
-
return undefined;
|
|
12
|
-
return (0, v3_1.getEnvVar)(name);
|
|
13
|
-
}
|
|
14
|
-
/**
|
|
15
|
-
* Precedence: an explicit value, the client config, `TRIGGER_EXTERNAL_DEPLOYMENT_ID`, then
|
|
16
|
-
* platform/CI commit vars when automatic skew protection is on. `null` is an explicit opt-out.
|
|
17
|
-
*/
|
|
18
|
-
function resolveTriggerExternalDeploymentId(explicit) {
|
|
19
|
-
if (explicit === null)
|
|
20
|
-
return undefined;
|
|
21
|
-
return (0, v3_1.resolveExternalDeploymentId)({
|
|
22
|
-
explicit,
|
|
23
|
-
clientConfig: v3_1.apiClientManager.externalDeploymentId,
|
|
24
|
-
read: scopedEnvVar,
|
|
25
|
-
});
|
|
26
|
-
}
|
|
27
|
-
/**
|
|
28
|
-
* Fill in `triggerConfig.externalDeploymentId` on an outgoing session-create body, at each point
|
|
29
|
-
* one leaves the SDK. Discovery has to happen here rather than server-side: the commit SHA lives
|
|
30
|
-
* in the calling application's runtime, so the caller's build is what selects the agent version.
|
|
31
|
-
*/
|
|
32
|
-
function withResolvedExternalDeploymentId(body) {
|
|
33
|
-
const resolved = resolveTriggerExternalDeploymentId(body.triggerConfig.externalDeploymentId);
|
|
34
|
-
const { externalDeploymentId: _omit, ...rest } = body.triggerConfig;
|
|
35
|
-
return {
|
|
36
|
-
...body,
|
|
37
|
-
triggerConfig: {
|
|
38
|
-
...rest,
|
|
39
|
-
...(resolved ? { externalDeploymentId: resolved } : {}),
|
|
40
|
-
},
|
|
41
|
-
};
|
|
42
|
-
}
|
|
43
|
-
//# sourceMappingURL=externalDeploymentId.js.map
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"externalDeploymentId.js","sourceRoot":"","sources":["../../../src/v3/externalDeploymentId.ts"],"names":[],"mappings":";;;;;AAAA,6CAM8B;AAE9B,gGAAgG;AAChG,sBAA6B,IAAY;IACvC,MAAM,KAAK,GAAG,aAAQ,CAAC,QAAQ,EAAE,CAAC;IAClC,IAAI,KAAK,IAAI,CAAC,KAAK,CAAC,cAAc;QAAE,OAAO,SAAS,CAAC;IACrD,OAAO,IAAA,cAAS,EAAC,IAAI,CAAC,CAAC;AACzB,CAAC;AAED;;;GAGG;AACH,4CAAmD,QAAwB;IACzE,IAAI,QAAQ,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAExC,OAAO,IAAA,gCAA2B,EAAC;QACjC,QAAQ;QACR,YAAY,EAAE,qBAAgB,CAAC,oBAAoB;QACnD,IAAI,EAAE,YAAY;KACnB,CAAC,CAAC;AACL,CAAC;AAOD;;;;GAIG;AACH,0CAEE,IAAW;IACX,MAAM,QAAQ,GAAG,kCAAkC,CAAC,IAAI,CAAC,aAAa,CAAC,oBAAoB,CAAC,CAAC;IAC7F,MAAM,EAAE,oBAAoB,EAAE,KAAK,EAAE,GAAG,IAAI,EAAE,GAAG,IAAI,CAAC,aAAa,CAAC;IAEpE,OAAO;QACL,GAAG,IAAI;QACP,aAAa,EAAE;YACb,GAAG,IAAI;YACP,GAAG,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,oBAAoB,EAAE,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;SACxD;KACF,CAAC;AACJ,CAAC"}
|
|
@@ -1,12 +0,0 @@
|
|
|
1
|
-
export type ChatVersionSkewPolicy = "follow" | "hold";
|
|
2
|
-
export type SessionVersionPin = {
|
|
3
|
-
externalDeploymentId?: string | null;
|
|
4
|
-
lockToVersion?: string;
|
|
5
|
-
};
|
|
6
|
-
export type ResolvePinToFollowOptions = {
|
|
7
|
-
policy: ChatVersionSkewPolicy | undefined;
|
|
8
|
-
deployedExternalId: string | undefined;
|
|
9
|
-
upgradeAlreadyRequested: boolean;
|
|
10
|
-
readPin: () => Promise<SessionVersionPin | undefined>;
|
|
11
|
-
};
|
|
12
|
-
export declare function resolvePinToFollow(options: ResolvePinToFollowOptions): Promise<string | undefined>;
|
|
@@ -1,27 +0,0 @@
|
|
|
1
|
-
import { tryCatch } from "@trigger.dev/core/v3";
|
|
2
|
-
export async function resolvePinToFollow(options) {
|
|
3
|
-
if (options.policy === "hold") {
|
|
4
|
-
return undefined;
|
|
5
|
-
}
|
|
6
|
-
if (options.upgradeAlreadyRequested) {
|
|
7
|
-
return undefined;
|
|
8
|
-
}
|
|
9
|
-
if (!options.deployedExternalId) {
|
|
10
|
-
return undefined;
|
|
11
|
-
}
|
|
12
|
-
const [error, pin] = await tryCatch(options.readPin());
|
|
13
|
-
if (error || !pin) {
|
|
14
|
-
return undefined;
|
|
15
|
-
}
|
|
16
|
-
if (!pin.externalDeploymentId) {
|
|
17
|
-
return undefined;
|
|
18
|
-
}
|
|
19
|
-
if (pin.lockToVersion) {
|
|
20
|
-
return undefined;
|
|
21
|
-
}
|
|
22
|
-
if (pin.externalDeploymentId === options.deployedExternalId) {
|
|
23
|
-
return undefined;
|
|
24
|
-
}
|
|
25
|
-
return pin.externalDeploymentId;
|
|
26
|
-
}
|
|
27
|
-
//# sourceMappingURL=chatVersionSkew.js.map
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"chatVersionSkew.js","sourceRoot":"","sources":["../../../src/v3/chatVersionSkew.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,sBAAsB,CAAC;AAgBhD,MAAM,CAAC,KAAK,UAAU,kBAAkB,CACtC,OAAkC;IAElC,IAAI,OAAO,CAAC,MAAM,KAAK,MAAM,EAAE,CAAC;QAC9B,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,OAAO,CAAC,uBAAuB,EAAE,CAAC;QACpC,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,CAAC,OAAO,CAAC,kBAAkB,EAAE,CAAC;QAChC,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,MAAM,CAAC,KAAK,EAAE,GAAG,CAAC,GAAG,MAAM,QAAQ,CAAC,OAAO,CAAC,OAAO,EAAE,CAAC,CAAC;IAEvD,IAAI,KAAK,IAAI,CAAC,GAAG,EAAE,CAAC;QAClB,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,CAAC,GAAG,CAAC,oBAAoB,EAAE,CAAC;QAC9B,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,GAAG,CAAC,aAAa,EAAE,CAAC;QACtB,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,IAAI,GAAG,CAAC,oBAAoB,KAAK,OAAO,CAAC,kBAAkB,EAAE,CAAC;QAC5D,OAAO,SAAS,CAAC;IACnB,CAAC;IAED,OAAO,GAAG,CAAC,oBAAoB,CAAC;AAClC,CAAC"}
|
|
@@ -1,23 +0,0 @@
|
|
|
1
|
-
import { type SessionTriggerConfig } from "@trigger.dev/core/v3";
|
|
2
|
-
/** Reads an env var unless the scope opted out of ambient context (`inheritContext: false`). */
|
|
3
|
-
export declare function scopedEnvVar(name: string): string | undefined;
|
|
4
|
-
/**
|
|
5
|
-
* Precedence: an explicit value, the client config, `TRIGGER_EXTERNAL_DEPLOYMENT_ID`, then
|
|
6
|
-
* platform/CI commit vars when automatic skew protection is on. `null` is an explicit opt-out.
|
|
7
|
-
*/
|
|
8
|
-
export declare function resolveTriggerExternalDeploymentId(explicit?: string | null): string | undefined;
|
|
9
|
-
/** A session trigger config as callers write it: the pin is optional, and `null` opts out. */
|
|
10
|
-
type TriggerConfigInput = Omit<SessionTriggerConfig, "externalDeploymentId"> & {
|
|
11
|
-
externalDeploymentId?: string | null;
|
|
12
|
-
};
|
|
13
|
-
/**
|
|
14
|
-
* Fill in `triggerConfig.externalDeploymentId` on an outgoing session-create body, at each point
|
|
15
|
-
* one leaves the SDK. Discovery has to happen here rather than server-side: the commit SHA lives
|
|
16
|
-
* in the calling application's runtime, so the caller's build is what selects the agent version.
|
|
17
|
-
*/
|
|
18
|
-
export declare function withResolvedExternalDeploymentId<TBody extends {
|
|
19
|
-
triggerConfig: TriggerConfigInput;
|
|
20
|
-
}>(body: TBody): TBody & {
|
|
21
|
-
triggerConfig: SessionTriggerConfig;
|
|
22
|
-
};
|
|
23
|
-
export {};
|
|
@@ -1,38 +0,0 @@
|
|
|
1
|
-
import { apiClientManager, getEnvVar, resolveExternalDeploymentId, sdkScope, } from "@trigger.dev/core/v3";
|
|
2
|
-
/** Reads an env var unless the scope opted out of ambient context (`inheritContext: false`). */
|
|
3
|
-
export function scopedEnvVar(name) {
|
|
4
|
-
const scope = sdkScope.getStore();
|
|
5
|
-
if (scope && !scope.inheritContext)
|
|
6
|
-
return undefined;
|
|
7
|
-
return getEnvVar(name);
|
|
8
|
-
}
|
|
9
|
-
/**
|
|
10
|
-
* Precedence: an explicit value, the client config, `TRIGGER_EXTERNAL_DEPLOYMENT_ID`, then
|
|
11
|
-
* platform/CI commit vars when automatic skew protection is on. `null` is an explicit opt-out.
|
|
12
|
-
*/
|
|
13
|
-
export function resolveTriggerExternalDeploymentId(explicit) {
|
|
14
|
-
if (explicit === null)
|
|
15
|
-
return undefined;
|
|
16
|
-
return resolveExternalDeploymentId({
|
|
17
|
-
explicit,
|
|
18
|
-
clientConfig: apiClientManager.externalDeploymentId,
|
|
19
|
-
read: scopedEnvVar,
|
|
20
|
-
});
|
|
21
|
-
}
|
|
22
|
-
/**
|
|
23
|
-
* Fill in `triggerConfig.externalDeploymentId` on an outgoing session-create body, at each point
|
|
24
|
-
* one leaves the SDK. Discovery has to happen here rather than server-side: the commit SHA lives
|
|
25
|
-
* in the calling application's runtime, so the caller's build is what selects the agent version.
|
|
26
|
-
*/
|
|
27
|
-
export function withResolvedExternalDeploymentId(body) {
|
|
28
|
-
const resolved = resolveTriggerExternalDeploymentId(body.triggerConfig.externalDeploymentId);
|
|
29
|
-
const { externalDeploymentId: _omit, ...rest } = body.triggerConfig;
|
|
30
|
-
return {
|
|
31
|
-
...body,
|
|
32
|
-
triggerConfig: {
|
|
33
|
-
...rest,
|
|
34
|
-
...(resolved ? { externalDeploymentId: resolved } : {}),
|
|
35
|
-
},
|
|
36
|
-
};
|
|
37
|
-
}
|
|
38
|
-
//# sourceMappingURL=externalDeploymentId.js.map
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"externalDeploymentId.js","sourceRoot":"","sources":["../../../src/v3/externalDeploymentId.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,gBAAgB,EAChB,SAAS,EACT,2BAA2B,EAC3B,QAAQ,GAET,MAAM,sBAAsB,CAAC;AAE9B,gGAAgG;AAChG,MAAM,UAAU,YAAY,CAAC,IAAY;IACvC,MAAM,KAAK,GAAG,QAAQ,CAAC,QAAQ,EAAE,CAAC;IAClC,IAAI,KAAK,IAAI,CAAC,KAAK,CAAC,cAAc;QAAE,OAAO,SAAS,CAAC;IACrD,OAAO,SAAS,CAAC,IAAI,CAAC,CAAC;AACzB,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,kCAAkC,CAAC,QAAwB;IACzE,IAAI,QAAQ,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAExC,OAAO,2BAA2B,CAAC;QACjC,QAAQ;QACR,YAAY,EAAE,gBAAgB,CAAC,oBAAoB;QACnD,IAAI,EAAE,YAAY;KACnB,CAAC,CAAC;AACL,CAAC;AAOD;;;;GAIG;AACH,MAAM,UAAU,gCAAgC,CAE9C,IAAW;IACX,MAAM,QAAQ,GAAG,kCAAkC,CAAC,IAAI,CAAC,aAAa,CAAC,oBAAoB,CAAC,CAAC;IAC7F,MAAM,EAAE,oBAAoB,EAAE,KAAK,EAAE,GAAG,IAAI,EAAE,GAAG,IAAI,CAAC,aAAa,CAAC;IAEpE,OAAO;QACL,GAAG,IAAI;QACP,aAAa,EAAE;YACb,GAAG,IAAI;YACP,GAAG,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,oBAAoB,EAAE,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;SACxD;KACF,CAAC;AACJ,CAAC"}
|
|
@@ -1,310 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
title: "Native compaction & provider fallback"
|
|
3
|
-
sidebarTitle: "Native compaction"
|
|
4
|
-
description: "Persist provider-native compaction (Anthropic context editing, OpenAI stored responses) across chat.agent turns so history is never re-sent, and fall back between providers without losing the conversation."
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
Providers compact a conversation within a single request. Anthropic's [context editing](https://docs.anthropic.com/en/docs/build-with-claude/context-editing) clears old tool-use blocks server-side, and OpenAI's [stored responses](https://platform.openai.com/docs/guides/conversation-state) keep the thread server-side so you only send the delta. Neither changes what your agent has accumulated, so on its own the next turn re-sends the whole transcript again and the token saving is lost.
|
|
8
|
-
|
|
9
|
-
This is the gap this page closes. After each turn, mirror what the provider compacted into the agent's stored history with [`chat.history.set()`](/ai-chat/reference#chat-namespace), so the next turn is derived from the already-reduced conversation. And because a native handle is provider-specific, this page also shows how a provider-agnostic [Trigger.dev compaction](/ai-chat/compaction) summary lets you fall back between providers without re-expanding the context.
|
|
10
|
-
|
|
11
|
-
<Note>
|
|
12
|
-
The full runnable example is [`triggerdotdev/resilient-chat-example`](https://github.com/triggerdotdev/resilient-chat-example). See `native-persist.ts` for the Anthropic persistence flow and `resilient-chat.ts` for OpenAI stored responses plus provider fallback.
|
|
13
|
-
</Note>
|
|
14
|
-
|
|
15
|
-
## Two kinds of compaction
|
|
16
|
-
|
|
17
|
-
They are not competing; they compose. Native compaction is the per-turn optimization, and Trigger.dev compaction is the durable, portable checkpoint.
|
|
18
|
-
|
|
19
|
-
| | Native (provider) | Trigger.dev `compaction` |
|
|
20
|
-
| --- | --- | --- |
|
|
21
|
-
| Runs | Inside one provider request | Between steps / turns, in your run |
|
|
22
|
-
| Scope | Provider-specific (Anthropic edits, OpenAI stored thread) | Provider-agnostic |
|
|
23
|
-
| Portable across a provider switch | No, the handle is a cache miss on the other provider | Yes, `summarize` returns a plain string |
|
|
24
|
-
| Persisted by default | No, you mirror it in `onTurnComplete` | Yes, replaces model messages and keeps UI messages |
|
|
25
|
-
|
|
26
|
-
## Persist Anthropic native context editing
|
|
27
|
-
|
|
28
|
-
Anthropic's `contextManagement` clears old tool-use/tool-result blocks server-side per request, and reports how many it cleared in `providerMetadata.anthropic.contextManagement.appliedEdits` (`clearedToolUses`, `clearedInputTokens`). It does not touch your accumulated history, so on its own the next turn still re-sends everything.
|
|
29
|
-
|
|
30
|
-
The fix: read the `appliedEdits` counts as they stream in `onStepFinish`, then after the turn mirror that clearing into stored history with `chat.history.set()`. No custom summarizer is involved, since the provider's native editing drives what gets persisted.
|
|
31
|
-
|
|
32
|
-
```ts /trigger/native-persist.ts
|
|
33
|
-
import { chat } from "@trigger.dev/sdk/ai";
|
|
34
|
-
import { streamText, stepCountIs, tool, type UIMessage } from "ai";
|
|
35
|
-
import { anthropic } from "@ai-sdk/anthropic";
|
|
36
|
-
import { z } from "zod";
|
|
37
|
-
|
|
38
|
-
const fetchRecord = tool({
|
|
39
|
-
description: "Fetch the full text of a record by its numeric id.",
|
|
40
|
-
inputSchema: z.object({ id: z.number() }),
|
|
41
|
-
execute: async ({ id }) => ({ id, text: `RECORD ${id}: ...` }),
|
|
42
|
-
});
|
|
43
|
-
|
|
44
|
-
// How many tool-uses Anthropic cleared this turn, per chat. Captured in run(),
|
|
45
|
-
// applied in onTurnComplete. An in-memory Map is enough because the run stays
|
|
46
|
-
// alive across turns (idleTimeoutInSeconds).
|
|
47
|
-
const clearedByChat = new Map<string, number>();
|
|
48
|
-
|
|
49
|
-
const isToolPart = (p: { type?: string }) =>
|
|
50
|
-
typeof p?.type === "string" && (p.type.startsWith("tool-") || p.type === "dynamic-tool");
|
|
51
|
-
|
|
52
|
-
// Drop the oldest n tool parts, mirroring what the provider cleared. A tool call
|
|
53
|
-
// and its result live in one part, so pairing stays intact.
|
|
54
|
-
function pruneOldestToolParts(messages: UIMessage[], n: number): UIMessage[] {
|
|
55
|
-
let toRemove = n;
|
|
56
|
-
const out: UIMessage[] = [];
|
|
57
|
-
for (const m of messages) {
|
|
58
|
-
if (toRemove <= 0 || m.role !== "assistant" || !m.parts) {
|
|
59
|
-
out.push(m);
|
|
60
|
-
continue;
|
|
61
|
-
}
|
|
62
|
-
const kept = m.parts.filter((p) => {
|
|
63
|
-
if (toRemove > 0 && isToolPart(p)) {
|
|
64
|
-
toRemove--;
|
|
65
|
-
return false;
|
|
66
|
-
}
|
|
67
|
-
return true;
|
|
68
|
-
});
|
|
69
|
-
if (kept.length > 0) out.push({ ...m, parts: kept });
|
|
70
|
-
}
|
|
71
|
-
return out;
|
|
72
|
-
}
|
|
73
|
-
|
|
74
|
-
export const nativePersist = chat.agent({
|
|
75
|
-
id: "native-persist",
|
|
76
|
-
idleTimeoutInSeconds: 120,
|
|
77
|
-
tools: { fetchRecord },
|
|
78
|
-
run: async ({ messages, chatId, tools, signal }) => {
|
|
79
|
-
return streamText({
|
|
80
|
-
model: anthropic("claude-sonnet-4-5"),
|
|
81
|
-
messages,
|
|
82
|
-
tools,
|
|
83
|
-
abortSignal: signal,
|
|
84
|
-
stopWhen: stepCountIs(12),
|
|
85
|
-
providerOptions: {
|
|
86
|
-
anthropic: {
|
|
87
|
-
contextManagement: {
|
|
88
|
-
edits: [
|
|
89
|
-
{
|
|
90
|
-
type: "clear_tool_uses_20250919",
|
|
91
|
-
trigger: { type: "tool_uses", value: 2 },
|
|
92
|
-
keep: { type: "tool_uses", value: 1 },
|
|
93
|
-
clearToolInputs: true,
|
|
94
|
-
},
|
|
95
|
-
],
|
|
96
|
-
},
|
|
97
|
-
},
|
|
98
|
-
},
|
|
99
|
-
onStepFinish: ({ providerMetadata }) => {
|
|
100
|
-
const cm = providerMetadata?.anthropic?.contextManagement as
|
|
101
|
-
| { appliedEdits?: Array<{ type?: string; clearedToolUses?: number }> }
|
|
102
|
-
| undefined;
|
|
103
|
-
let stepCleared = 0;
|
|
104
|
-
for (const e of cm?.appliedEdits ?? []) {
|
|
105
|
-
if (e.type === "clear_tool_uses_20250919") stepCleared += e.clearedToolUses ?? 0;
|
|
106
|
-
}
|
|
107
|
-
if (stepCleared > 0) {
|
|
108
|
-
clearedByChat.set(chatId, (clearedByChat.get(chatId) ?? 0) + stepCleared);
|
|
109
|
-
}
|
|
110
|
-
},
|
|
111
|
-
});
|
|
112
|
-
},
|
|
113
|
-
// After the turn, mirror the server-side clearing into stored history.
|
|
114
|
-
onTurnComplete: async ({ chatId, uiMessages }) => {
|
|
115
|
-
const cleared = clearedByChat.get(chatId) ?? 0;
|
|
116
|
-
if (cleared <= 0) return;
|
|
117
|
-
chat.history.set(pruneOldestToolParts(uiMessages, cleared));
|
|
118
|
-
clearedByChat.set(chatId, 0);
|
|
119
|
-
},
|
|
120
|
-
});
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
Turn 1 sends the user message and accumulates six tool results. Anthropic clears four of them server-side. `onTurnComplete` prunes those four from stored history, so turn 2 re-sends the smaller conversation (one tool result, not six) instead of the full transcript.
|
|
124
|
-
|
|
125
|
-
<Note>
|
|
126
|
-
`onTurnComplete` is where persistence happens. Action turns fire `onAction` only, and a `chat.history.set()` inside `run()` is overwritten by the accumulator at turn end. See [Persistence and replay](/ai-chat/patterns/persistence-and-replay#action-turns-no-snapshot-write).
|
|
127
|
-
</Note>
|
|
128
|
-
|
|
129
|
-
## Persist OpenAI stored responses
|
|
130
|
-
|
|
131
|
-
OpenAI's `store: true` keeps the thread server-side and returns a `responseId`. Pass that back as `previousResponseId` on the next turn and send only the messages since the last assistant reply; everything before it lives on OpenAI's side.
|
|
132
|
-
|
|
133
|
-
```ts /trigger/openai-store.ts
|
|
134
|
-
import { chat } from "@trigger.dev/sdk/ai";
|
|
135
|
-
import { streamText, stepCountIs, type ModelMessage } from "ai";
|
|
136
|
-
import { openai } from "@ai-sdk/openai";
|
|
137
|
-
|
|
138
|
-
// Persist the stored-response handle between turns. Replace with your database.
|
|
139
|
-
const nativeStore = new Map<string, { previousResponseId: string }>();
|
|
140
|
-
|
|
141
|
-
// When OpenAI already holds the thread, send only what is new since the last
|
|
142
|
-
// assistant reply. Everything before that lives server-side.
|
|
143
|
-
function messagesSinceLastAssistant(messages: ModelMessage[]): ModelMessage[] {
|
|
144
|
-
let last = -1;
|
|
145
|
-
for (let i = 0; i < messages.length; i++) {
|
|
146
|
-
if (messages[i]!.role === "assistant") last = i;
|
|
147
|
-
}
|
|
148
|
-
return last === -1 ? messages : messages.slice(last + 1);
|
|
149
|
-
}
|
|
150
|
-
|
|
151
|
-
export const openaiStore = chat.agent({
|
|
152
|
-
id: "openai-store",
|
|
153
|
-
idleTimeoutInSeconds: 120,
|
|
154
|
-
run: async ({ messages, chatId, signal }) => {
|
|
155
|
-
const native = nativeStore.get(chatId);
|
|
156
|
-
const outbound = native ? messagesSinceLastAssistant(messages) : messages;
|
|
157
|
-
|
|
158
|
-
const result = streamText({
|
|
159
|
-
model: openai("gpt-4o"),
|
|
160
|
-
messages: outbound,
|
|
161
|
-
abortSignal: signal,
|
|
162
|
-
stopWhen: stepCountIs(5),
|
|
163
|
-
providerOptions: {
|
|
164
|
-
openai: native ? { store: true, previousResponseId: native.previousResponseId } : { store: true },
|
|
165
|
-
},
|
|
166
|
-
});
|
|
167
|
-
|
|
168
|
-
// Capture the response id off the metadata for the next turn.
|
|
169
|
-
void result.providerMetadata.then((meta) => {
|
|
170
|
-
const rid = typeof meta?.openai?.responseId === "string" ? meta.openai.responseId : undefined;
|
|
171
|
-
if (rid) nativeStore.set(chatId, { previousResponseId: rid });
|
|
172
|
-
});
|
|
173
|
-
|
|
174
|
-
return result;
|
|
175
|
-
},
|
|
176
|
-
});
|
|
177
|
-
```
|
|
178
|
-
|
|
179
|
-
Turn 1 stores the thread and sends all three messages. Turn 2 sends only the new user message (`1/3`), because OpenAI already has the rest.
|
|
180
|
-
|
|
181
|
-
## Fall back between providers without losing history
|
|
182
|
-
|
|
183
|
-
A native handle is a per-provider cache. An OpenAI `previousResponseId` means nothing to Anthropic, and Anthropic's server-side edits don't exist on OpenAI. So when a provider is down and you fall back to another, the native optimization is a cache miss, and a naive fallback re-sends the entire raw transcript to the new provider.
|
|
184
|
-
|
|
185
|
-
[Trigger.dev's `compaction`](/ai-chat/compaction) is the portable checkpoint that closes this gap. `summarize` returns a plain string and `compactModelMessages` returns neutral `ModelMessage[]`, so the summary survives any provider switch. Tag each native handle with the provider that produced it. On a switch it's a cache miss, and you rebuild from the summary instead of re-expanding the context.
|
|
186
|
-
|
|
187
|
-
```ts /trigger/resilient-chat.ts
|
|
188
|
-
import { chat } from "@trigger.dev/sdk/ai";
|
|
189
|
-
import { streamText, generateText, stepCountIs, generateId, type ModelMessage } from "ai";
|
|
190
|
-
import { anthropic } from "@ai-sdk/anthropic";
|
|
191
|
-
import { openai } from "@ai-sdk/openai";
|
|
192
|
-
|
|
193
|
-
type Provider = "anthropic" | "openai";
|
|
194
|
-
const FALLBACK_ORDER: Provider[] = ["anthropic", "openai"];
|
|
195
|
-
|
|
196
|
-
// Native handle, tagged with the provider that produced it. Replace with your DB.
|
|
197
|
-
type NativeState = { provider: "openai"; previousResponseId: string };
|
|
198
|
-
const nativeStore = new Map<string, NativeState>();
|
|
199
|
-
|
|
200
|
-
// Provider-agnostic summary: a plain string, portable across any provider.
|
|
201
|
-
async function summarizeConversation(messages: ModelMessage[]): Promise<string> {
|
|
202
|
-
const { text } = await generateText({
|
|
203
|
-
model: openai("gpt-4o-mini"),
|
|
204
|
-
messages: [
|
|
205
|
-
...messages,
|
|
206
|
-
{
|
|
207
|
-
role: "user",
|
|
208
|
-
content:
|
|
209
|
-
"Summarize this conversation so it can continue with ANY model. " +
|
|
210
|
-
"Preserve decisions made, facts established, open questions, and the user's intent.",
|
|
211
|
-
},
|
|
212
|
-
],
|
|
213
|
-
});
|
|
214
|
-
return text;
|
|
215
|
-
}
|
|
216
|
-
|
|
217
|
-
export const resilientChat = chat.agent({
|
|
218
|
-
id: "resilient-chat",
|
|
219
|
-
idleTimeoutInSeconds: 120,
|
|
220
|
-
|
|
221
|
-
compaction: {
|
|
222
|
-
shouldCompact: ({ totalTokens }) => (totalTokens ?? 0) > 80_000,
|
|
223
|
-
summarize: ({ messages }) => summarizeConversation(messages),
|
|
224
|
-
compactModelMessages: ({ modelMessages, summary }) => [
|
|
225
|
-
{ role: "user", content: `Summary of the conversation so far:\n\n${summary}` },
|
|
226
|
-
...modelMessages.slice(-2),
|
|
227
|
-
],
|
|
228
|
-
compactUIMessages: ({ uiMessages, summary }) => [
|
|
229
|
-
{
|
|
230
|
-
id: generateId(),
|
|
231
|
-
role: "assistant",
|
|
232
|
-
parts: [{ type: "text", text: `[Conversation summary]\n\n${summary}` }],
|
|
233
|
-
},
|
|
234
|
-
...uiMessages.slice(-2),
|
|
235
|
-
],
|
|
236
|
-
},
|
|
237
|
-
|
|
238
|
-
// A Trigger.dev compaction is the reset point: the provider's server-side thread
|
|
239
|
-
// no longer matches the compacted baseline, so invalidate the native handle.
|
|
240
|
-
onCompacted: async ({ chatId }) => {
|
|
241
|
-
if (chatId) nativeStore.delete(chatId);
|
|
242
|
-
},
|
|
243
|
-
|
|
244
|
-
run: async ({ messages, chatId, signal }) => {
|
|
245
|
-
let lastError: unknown;
|
|
246
|
-
for (const providerId of FALLBACK_ORDER) {
|
|
247
|
-
const native = nativeStore.get(chatId);
|
|
248
|
-
try {
|
|
249
|
-
if (providerId === "openai") {
|
|
250
|
-
// On a switch to OpenAI with no matching handle, `messages` is already the
|
|
251
|
-
// compacted baseline (summary + recent), so raw history is not re-sent.
|
|
252
|
-
const useHandle = native?.provider === "openai";
|
|
253
|
-
const result = streamText({
|
|
254
|
-
model: openai("gpt-4o"),
|
|
255
|
-
messages,
|
|
256
|
-
abortSignal: signal,
|
|
257
|
-
stopWhen: stepCountIs(5),
|
|
258
|
-
providerOptions: {
|
|
259
|
-
openai: useHandle
|
|
260
|
-
? { store: true, previousResponseId: native!.previousResponseId }
|
|
261
|
-
: { store: true },
|
|
262
|
-
},
|
|
263
|
-
});
|
|
264
|
-
void result.providerMetadata.then((meta) => {
|
|
265
|
-
const rid = typeof meta?.openai?.responseId === "string" ? meta.openai.responseId : undefined;
|
|
266
|
-
if (rid) nativeStore.set(chatId, { provider: "openai", previousResponseId: rid });
|
|
267
|
-
});
|
|
268
|
-
return result;
|
|
269
|
-
}
|
|
270
|
-
|
|
271
|
-
return streamText({
|
|
272
|
-
model: anthropic("claude-sonnet-4-5"),
|
|
273
|
-
messages,
|
|
274
|
-
abortSignal: signal,
|
|
275
|
-
stopWhen: stepCountIs(5),
|
|
276
|
-
providerOptions: {
|
|
277
|
-
anthropic: {
|
|
278
|
-
contextManagement: {
|
|
279
|
-
edits: [{ type: "clear_tool_uses_20250919", trigger: { type: "input_tokens", value: 80_000 }, keep: { type: "tool_uses", value: 3 } }],
|
|
280
|
-
},
|
|
281
|
-
},
|
|
282
|
-
},
|
|
283
|
-
});
|
|
284
|
-
} catch (error) {
|
|
285
|
-
lastError = error; // Provider failed, try the next one in the order.
|
|
286
|
-
}
|
|
287
|
-
}
|
|
288
|
-
throw lastError;
|
|
289
|
-
},
|
|
290
|
-
});
|
|
291
|
-
```
|
|
292
|
-
|
|
293
|
-
When Anthropic is down, the loop falls through to OpenAI. Because `compaction` has already reduced `messages` to a summary plus the last couple of exchanges, the switch sends the portable baseline, not megabytes of raw transcript.
|
|
294
|
-
|
|
295
|
-
<Warning>
|
|
296
|
-
Fallback here retries a turn that hasn't started streaming yet. Once a response is streaming to the client, a mid-stream provider failure can't be swapped transparently. Surface the error and let the frontend regenerate the turn. See [Error handling](/ai-chat/error-handling).
|
|
297
|
-
</Warning>
|
|
298
|
-
|
|
299
|
-
## Production notes
|
|
300
|
-
|
|
301
|
-
- **Persist the handles.** The `Map`s above (`nativeStore`, `clearedByChat`) work in the example because the run stays alive across turns, but they don't survive a run boundary. Store native handles and summaries in your database keyed by `chatId`, alongside your [message persistence](/ai-chat/patterns/database-persistence).
|
|
302
|
-
- **No cross-provider translation.** Native compaction from one provider never transfers to another. The Trigger.dev `compaction` summary is the only portable baseline across a switch.
|
|
303
|
-
- **Native compaction is opt-in per turn.** It applies only for the provider whose `providerOptions` you set on that turn's `streamText` call.
|
|
304
|
-
|
|
305
|
-
## See also
|
|
306
|
-
|
|
307
|
-
- [Compaction](/ai-chat/compaction): the provider-agnostic `compaction` option, `onCompacted`, and manual `chat.compact()`.
|
|
308
|
-
- [Prompt caching](/ai-chat/prompt-caching): the other per-turn token optimization, and how it interacts with a growing history.
|
|
309
|
-
- [Database persistence](/ai-chat/patterns/database-persistence): where to store native handles and summaries for real.
|
|
310
|
-
- [Lifecycle hooks](/ai-chat/lifecycle-hooks): `onTurnComplete` and `onCompacted` in the broader hook taxonomy.
|