eve 0.90.0 → 0.91.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (185) hide show
  1. package/CHANGELOG.md +34 -23
  2. package/dist/src/channel/auth/http-basic.js +1 -1
  3. package/dist/src/channel/channel-operations.d.ts +1 -1
  4. package/dist/src/channel/types.d.ts +2 -0
  5. package/dist/src/cli/dev/tui/prompt-commands.d.ts +1 -1
  6. package/dist/src/client/session.d.ts +1 -1
  7. package/dist/src/compiled/.vendor-stamp.json +1 -1
  8. package/dist/src/execution/dynamic-workflow/tool.js +1 -1
  9. package/dist/src/execution/proxied-deliver-step.js +1 -1
  10. package/dist/src/execution/sandbox/bash.js +2 -2
  11. package/dist/src/execution/session/active-turn.js +1 -1
  12. package/dist/src/execution/session/admission.d.ts +6 -0
  13. package/dist/src/execution/session/admission.js +1 -1
  14. package/dist/src/execution/session/delivery-facts.js +1 -1
  15. package/dist/src/execution/session/next-input.d.ts +2 -2
  16. package/dist/src/execution/session/next-input.js +1 -1
  17. package/dist/src/execution/session/program.js +1 -1
  18. package/dist/src/execution/session-inbox/inbox.d.ts +2 -2
  19. package/dist/src/execution/session-inbox/inbox.js +1 -1
  20. package/dist/src/execution/session-inbox/protocol.js +1 -1
  21. package/dist/src/execution/tools/agent-router-workflow.js +1 -1
  22. package/dist/src/harness/compaction/engine.js +2 -2
  23. package/dist/src/harness/current-messages.js +1 -1
  24. package/dist/src/harness/emission.d.ts +3 -0
  25. package/dist/src/harness/emission.js +1 -1
  26. package/dist/src/harness/facts.d.ts +5 -2
  27. package/dist/src/harness/facts.js +1 -1
  28. package/dist/src/harness/model-call/call.d.ts +7 -1
  29. package/dist/src/harness/model-call/call.js +3 -1
  30. package/dist/src/harness/provider-tool-history.d.ts +8 -0
  31. package/dist/src/harness/provider-tool-history.js +1 -1
  32. package/dist/src/harness/session-machine/transitions.d.ts +3 -2
  33. package/dist/src/harness/step/after-model.js +1 -1
  34. package/dist/src/harness/step/context.js +1 -1
  35. package/dist/src/harness/step/intake.js +1 -1
  36. package/dist/src/harness/step/prompt.js +1 -1
  37. package/dist/src/harness/tool-history-repair.d.ts +2 -0
  38. package/dist/src/harness/tool-history-repair.js +1 -1
  39. package/dist/src/harness/tool-loop.js +1 -1
  40. package/dist/src/internal/application/stamped-package-version.js +1 -1
  41. package/dist/src/protocol/session-events/families/delivery.d.ts +5 -3
  42. package/dist/src/protocol/session-events/families/delivery.js +1 -1
  43. package/dist/src/protocol/session-projection/fold.js +1 -1
  44. package/dist/src/public/channels/discord/hitl.js +1 -1
  45. package/dist/src/public/channels/slack/hitl.d.ts +2 -1
  46. package/dist/src/public/channels/slack/hitl.js +3 -3
  47. package/dist/src/public/channels/slack/interaction-cards.js +1 -1
  48. package/dist/src/public/channels/slack/request-card-render.js +1 -1
  49. package/dist/src/public/channels/teams/hitl.js +1 -1
  50. package/dist/src/public/channels/teams/verify.js +1 -1
  51. package/dist/src/public/definitions/hook.d.ts +90 -7
  52. package/dist/src/public/definitions/hook.js +1 -1
  53. package/dist/src/public/hooks/index.d.ts +3 -2
  54. package/dist/src/reactions/continuation.d.ts +19 -0
  55. package/dist/src/reactions/continuation.js +1 -0
  56. package/dist/src/reactions/intents.d.ts +10 -4
  57. package/dist/src/reactions/intents.js +1 -1
  58. package/dist/src/reactions/kinds/hook.js +1 -1
  59. package/dist/src/reactions/runner.js +1 -1
  60. package/dist/src/reactions/state.d.ts +15 -2
  61. package/dist/src/reactions/state.js +1 -1
  62. package/dist/src/runtime/resolve-connection.js +1 -1
  63. package/dist/src/runtime/subagents/workflow.js +1 -1
  64. package/dist/src/setup/primitives/open-url.js +1 -1
  65. package/dist/src/setup/scaffold/create/project.js +1 -1
  66. package/dist/src/shared/value-delta.d.ts +5 -4
  67. package/dist/src/shared/value-delta.js +1 -1
  68. package/dist/src/tools/provided/ask-question-workflow.js +1 -1
  69. package/dist/src/tools/provided/sleep-workflow.js +1 -1
  70. package/dist/src/tools/provided/web-fetch.js +1 -1
  71. package/docs/agent-config.md +199 -222
  72. package/docs/channels/chat-sdk.mdx +60 -64
  73. package/docs/channels/custom.mdx +264 -281
  74. package/docs/channels/discord.mdx +176 -123
  75. package/docs/channels/eve.mdx +136 -206
  76. package/docs/channels/github.mdx +24 -22
  77. package/docs/channels/linear.mdx +30 -30
  78. package/docs/channels/linq.mdx +25 -13
  79. package/docs/channels/mcp.mdx +154 -116
  80. package/docs/channels/overview.mdx +31 -32
  81. package/docs/channels/photon.mdx +32 -56
  82. package/docs/channels/slack.mdx +538 -375
  83. package/docs/channels/teams.mdx +40 -26
  84. package/docs/channels/telegram.mdx +13 -13
  85. package/docs/channels/twilio.mdx +11 -11
  86. package/docs/code-extension.mdx +40 -42
  87. package/docs/concepts/built-in-tools.md +89 -57
  88. package/docs/concepts/context-control.md +20 -20
  89. package/docs/concepts/default-harness.md +20 -36
  90. package/docs/concepts/execution-model-and-durability.mdx +96 -160
  91. package/docs/concepts/project-structure.mdx +21 -14
  92. package/docs/concepts/security-model.md +27 -27
  93. package/docs/concepts/sessions-runs-and-streaming.md +184 -198
  94. package/docs/concepts/state.md +7 -15
  95. package/docs/connections/mcp.mdx +109 -93
  96. package/docs/connections/openapi.mdx +57 -59
  97. package/docs/connections/overview.mdx +175 -170
  98. package/docs/evals/assertions.mdx +36 -29
  99. package/docs/evals/cases.mdx +38 -42
  100. package/docs/evals/judge.mdx +21 -29
  101. package/docs/evals/overview.mdx +42 -38
  102. package/docs/evals/reporters.mdx +54 -27
  103. package/docs/evals/running.mdx +15 -8
  104. package/docs/evals/targets.mdx +16 -19
  105. package/docs/evals/tool-stubs.mdx +111 -97
  106. package/docs/experimental/dynamic-schedules.mdx +118 -63
  107. package/docs/extensions.md +58 -65
  108. package/docs/getting-started.mdx +4 -6
  109. package/docs/guides/auth-and-route-protection.md +87 -70
  110. package/docs/guides/client/continuations.mdx +23 -26
  111. package/docs/guides/client/messages.mdx +47 -66
  112. package/docs/guides/client/output-schema.mdx +8 -8
  113. package/docs/guides/client/overview.mdx +45 -43
  114. package/docs/guides/client/streaming.mdx +62 -71
  115. package/docs/guides/decide.md +24 -79
  116. package/docs/guides/deployment/meta.json +1 -1
  117. package/docs/guides/deployment/overview.md +27 -18
  118. package/docs/guides/deployment/self-hosted-operations.md +99 -0
  119. package/docs/guides/deployment/self-hosting.md +63 -80
  120. package/docs/guides/deployment/vercel.mdx +31 -27
  121. package/docs/guides/deployment/workflow-worlds.md +100 -0
  122. package/docs/guides/dev-tui.md +48 -41
  123. package/docs/guides/dynamic-capabilities.md +49 -50
  124. package/docs/guides/frontend/nextjs.mdx +2 -2
  125. package/docs/guides/frontend/nuxt.mdx +11 -11
  126. package/docs/guides/frontend/overview.mdx +1 -1
  127. package/docs/guides/frontend/sveltekit.mdx +11 -11
  128. package/docs/guides/frontend/tanstack.mdx +16 -16
  129. package/docs/guides/frontend/use-eve-agent-svelte.mdx +21 -21
  130. package/docs/guides/frontend/use-eve-agent-vue.mdx +22 -22
  131. package/docs/guides/frontend/use-eve-agent.mdx +25 -25
  132. package/docs/guides/hooks.md +152 -106
  133. package/docs/guides/instrumentation/agent-runs.mdx +26 -30
  134. package/docs/guides/instrumentation/instrumentation.mdx +49 -76
  135. package/docs/guides/instrumentation/meta.json +1 -1
  136. package/docs/guides/instrumentation/otel.mdx +113 -179
  137. package/docs/guides/remote-agents.md +125 -187
  138. package/docs/guides/self-modification.md +18 -18
  139. package/docs/guides/session-context.md +16 -16
  140. package/docs/install-integrations.mdx +26 -24
  141. package/docs/instructions.mdx +37 -42
  142. package/docs/memory/custom-provider.md +35 -41
  143. package/docs/memory/file.md +93 -76
  144. package/docs/memory/overview.mdx +145 -156
  145. package/docs/patterns/durable-cross-channel-notifications.md +15 -19
  146. package/docs/patterns/dynamic-scheduling.md +19 -23
  147. package/docs/patterns/multi-tenant-approvals.md +17 -22
  148. package/docs/patterns/multi-tenant-auth.md +43 -42
  149. package/docs/patterns/multi-tenant-memory.md +39 -36
  150. package/docs/patterns/voice-notes.md +3 -3
  151. package/docs/protocols/acp.md +13 -13
  152. package/docs/protocols/ucp.mdx +7 -7
  153. package/docs/reference/agent-files.md +6 -6
  154. package/docs/reference/cli.md +99 -98
  155. package/docs/reference/telemetry.md +1 -1
  156. package/docs/reference/typescript-api.md +52 -53
  157. package/docs/responsible-use.md +2 -2
  158. package/docs/sandbox/custom-providers.mdx +60 -56
  159. package/docs/sandbox/default.mdx +23 -24
  160. package/docs/sandbox/docker.mdx +16 -10
  161. package/docs/sandbox/index.mdx +93 -61
  162. package/docs/sandbox/just-bash.mdx +17 -11
  163. package/docs/sandbox/troubleshooting.mdx +29 -23
  164. package/docs/sandbox/vercel.mdx +45 -52
  165. package/docs/schedules.mdx +65 -54
  166. package/docs/skills.mdx +74 -49
  167. package/docs/subagents/index.mdx +97 -136
  168. package/docs/tools/human-in-the-loop.md +160 -90
  169. package/docs/tools/meta.json +1 -1
  170. package/docs/tools/overview.mdx +229 -237
  171. package/docs/tools/tasks.md +118 -117
  172. package/docs/tools/workflows.mdx +447 -543
  173. package/docs/tutorial/connect-a-warehouse.mdx +11 -13
  174. package/docs/tutorial/first-agent.mdx +14 -16
  175. package/docs/tutorial/guard-the-spend.mdx +6 -8
  176. package/docs/tutorial/how-it-runs.mdx +8 -8
  177. package/docs/tutorial/query-sample-data.mdx +8 -8
  178. package/docs/tutorial/remember-definitions.mdx +4 -4
  179. package/docs/tutorial/run-analysis.mdx +20 -22
  180. package/docs/tutorial/ship-it.mdx +19 -19
  181. package/docs/tutorial/team-playbooks.mdx +9 -9
  182. package/docs/use-cases.mdx +13 -11
  183. package/package.json +1 -1
  184. package/docs/guides/instrumentation/migration.md +0 -96
  185. package/docs/tools/tasks-upgrade.md +0 -218
@@ -4,27 +4,18 @@ description: "Configure lifecycle instrumentation, handle runtime events, and co
4
4
  url: "/observability/instrumentation"
5
5
  ---
6
6
 
7
- Add lifecycle instrumentation as files under `agent/instrumentation/`. Each
8
- file handles eve runtime events independently. Configure trace exporters and
9
- shared span settings with the [built-in OpenTelemetry APIs](./otel).
10
-
11
- The instrumentation directory replaces `agent/instrumentation.ts`. See
12
- [Migrate instrumentation](/docs/observability/instrumentation-migration) when
13
- upgrading from `eve < 0.62.0`.
7
+ An instrumentation file is a file under `agent/instrumentation/` that receives eve runtime events, such as a tool call starting or a turn ending. To export traces instead, use the [built-in OpenTelemetry APIs](./otel).
14
8
 
15
9
  ## Add instrumentation
16
10
 
17
- The filename identifies the instrumentation slot. A lifecycle-event file must
18
- default-export `defineInstrumentation(...)` or `disableInstrumentation()`.
19
- See [OpenTelemetry](./otel) for the supported OTel declarations.
11
+ eve names each file by its filename. A lifecycle-event file must default-export `defineInstrumentation(...)` or `disableInstrumentation()`. See [OpenTelemetry](./otel) for the OTel files.
20
12
 
21
13
  ```text
22
14
  agent/instrumentation/
23
15
  audit.ts lifecycle instrumentation
24
16
  ```
25
17
 
26
- This instrumentation measures the time from a call being requested until it
27
- settles, including any approval wait, without receiving arguments or results:
18
+ This file measures how long each call takes from request to finish, including any approval wait. It doesn't receive arguments or results:
28
19
 
29
20
  ```ts title="agent/instrumentation/audit.ts"
30
21
  import { defineInstrumentation } from "eve/instrumentation";
@@ -48,20 +39,50 @@ export default defineInstrumentation({
48
39
  });
49
40
  ```
50
41
 
51
- `ctx.state` is JSON storage scoped to this file and operation. It survives
52
- durable suspension and is released after the terminal event.
42
+ `ctx.state` stores JSON for one operation in this file, such as one tool call. It is kept while the session pauses, such as during an approval wait. eve releases it after the operation's last event.
43
+
44
+ Handlers from different files run at the same time, in no fixed order. A failure in one file doesn't affect the others.
45
+
46
+ | Field | Purpose |
47
+ | ------------- | -------------------------------------------------------------------------------------------------------------------------- |
48
+ | `events` | Handlers keyed by event name. See [Lifecycle events](#lifecycle-events). |
49
+ | `tracePolicy` | Whether this file receives a trace, and with which content. See [Control inputs and outputs](#control-inputs-and-outputs). |
50
+ | `setup` | Asynchronous initialization. Runs once at startup, before any event arrives. |
51
+ | `flush` | Sends buffered records. eve calls it before a session goes idle. |
52
+ | `shutdown` | Releases resources when the process stops. |
53
+
54
+ A `setup` failure stops startup. Modules that an instrumentation file imports load before `setup`, so keep dependencies that need initialization out of those imports.
55
+
56
+ ## Lifecycle events
57
+
58
+ Events that also appear in the session stream use the same names and outcomes there. Each operation emits one final event; read its `outcome` to tell completion from failure. A start event and its final event share an `idempotencyKey`, so match on it when you upsert records across replays.
59
+
60
+ | Kind | Events | Meaning |
61
+ | ---------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
62
+ | Session | `session.started`, `session.ended` | Session start and end. `session.ended.outcome` is `completed` or `failed`. |
63
+ | Turn | `turn.started`, `turn.settled` | Turn work. `outcome` is `completed`, `cancelled`, or `failed`. A cancelled turn is not a failure. |
64
+ | Message | `delivery.consumed`, `delivery.settled` | An inbound message eve recorded. The final `outcome` is `handled`, `awaiting-input`, `applied`, `ignored`, `refused`, or `failed`. `delivery.consumed` carries channel metadata when available. |
65
+ | Model run | `model.started`, `model.retried`, `model.settled` | Each attempt of a model run. `model.retried` ends a failed attempt before another try. `model.settled.outcome` is `completed`, `failed`, `interrupted`, `abandoned`, or `cancelled` (the turn was cancelled). |
66
+ | Provider call | `model.call.started`, `model.call.settled` | One provider call; a model attempt can make several. The final `outcome` is `"completed"` or `"failed"`. Completion carries usage, finish reason, and optional `providerMetadata`. |
67
+ | Call | `call.requested`, `call.settled` | A tool, skill, or agent call, including direct `invokeTool` calls. `capability` names its kind and name. The final `outcome` is `completed`, `failed`, `rejected`, `interrupted`, or `abandoned`. |
68
+ | User input | `interaction.opened`, `interaction.settled` | A question, approval, or budget prompt, keyed by `interactionId`. The final event carries `outcome`, `responder` (who answered), and, when recorded, `answer`. |
69
+ | Memory operation | `memory.operation.started`, `memory.operation.settled` | A memory search or write, with operation name, phase, and slot. The final `outcome` is `"completed"` or `"failed"`. |
70
+
71
+ A message that pauses for a person ends with `awaiting-input`, and the work after the answer belongs to the answer's message. OTel links a turn to a message only when the turn handled exactly one message.
72
+
73
+ Each call emits one `call.*` pair, and SDK callbacks don't add a second one. A subagent's `call.settled` includes its reported usage.
74
+
75
+ The event's `scope` field holds the session and run IDs the event belongs to. `turnId` can be absent for compaction or direct calls. Treat `idempotencyKey` as opaque. `runId` and `attempt` identify a run; runtime-context resolvers receive them as `run: { id, attempt }`.
76
+
77
+ eve rebuilds a thrown tool error from its saved record, so check `error.name`, not `instanceof`.
53
78
 
54
- Use `setup` for asynchronous initialization. In `eve dev` and built servers,
55
- eve awaits every provider's setup before loading the server entry. A setup
56
- failure stops startup. Modules imported by an instrumentation file load before
57
- setup, so keep dependencies that need initialization out of those imports.
79
+ Use `InstrumentationEvent<"model.call.settled">` from `eve/instrumentation` to type a standalone handler. Model input content is `unknown`, because it holds the provider's request messages, not eve conversation messages. Narrow it before you read it.
80
+
81
+ The OpenTelemetry trace schema is version 4.
58
82
 
59
83
  ## Control inputs and outputs
60
84
 
61
- Each lifecycle file created with `defineInstrumentation(...)` has an
62
- independent `tracePolicy`. It decides whether that file receives a trace and
63
- whether its events include input or output content. OpenTelemetry destinations
64
- share the process-wide policy declared by `otel(...)`.
85
+ Each file created with `defineInstrumentation(...)` has its own `tracePolicy`. It decides whether that file receives a trace and whether its events include input or output content. OpenTelemetry destinations share the process-wide policy declared by `otel(...)`.
65
86
 
66
87
  ```ts title="agent/instrumentation/audit.ts"
67
88
  import { defineInstrumentation } from "eve/instrumentation";
@@ -83,13 +104,9 @@ export default defineInstrumentation({
83
104
  });
84
105
  ```
85
106
 
86
- The function receives `agentName`, `channel`, `audience`, `environment`, and
87
- `principalType`. `audience` is `"public"`, `"private"`, or
88
- `"unknown"`; `environment` is `"development"`, `"preview"`, or
89
- `"production"`.
107
+ The function receives `agentName`, `channel`, `audience`, `environment`, and `principalType`, the kind of caller. `audience` is `"public"`, `"private"`, or `"unknown"`; `environment` is `"development"`, `"preview"`, or `"production"`.
90
108
 
91
- Without a policy, or when it returns `true`, eve sends metadata for every
92
- trace. It includes input and output content in these cases:
109
+ Return `{ emit: false }` to skip the trace for this file, or `{ emit: true, recordInputs, recordOutputs }` to choose each content direction. Without a policy, or when it returns `true`, eve sends metadata for every trace and includes content in these cases:
93
110
 
94
111
  | Environment | Audience | Content |
95
112
  | --------------------- | ---------------------- | ------------------ |
@@ -97,22 +114,13 @@ trace. It includes input and output content in these cases:
97
114
  | Preview or production | `public` | Inputs and outputs |
98
115
  | Preview or production | `private` or `unknown` | Metadata only |
99
116
 
100
- Return `{ emit: false }` to not sample the trace for this file. Return
101
- `{ emit: true, recordInputs, recordOutputs }` to choose the two content
102
- directions explicitly. Inputs include model prompts, tool arguments, call labels,
103
- channel input, and user responses. Labels follow the input policy because they can contain
104
- input-derived text. Outputs include model responses, tool results,
105
- requests for user input, model provider metadata, and error details. Omitted
106
- content fields are unavailable to the handler.
117
+ Inputs include model prompts, tool arguments, call labels (which can contain input-derived text), channel input, and user responses. Outputs include model responses, tool results, requests for user input, model provider metadata, and error details. Content that isn't recorded is absent from the event.
107
118
 
108
- The channel assigns the audience once when it creates the session. It controls
109
- content capture, not access. See [Audience](../channels/overview#audience) for
110
- how the channel derives `public`, `private`, and `unknown`.
119
+ The channel sets the audience once, when it creates the session. The audience controls content capture, not access. See [Audience](../channels/overview#audience) for how the channel picks `public`, `private`, or `unknown`.
111
120
 
112
121
  ## Redact fields
113
122
 
114
- Lifecycle events are immutable snapshots. Copy the fields you need into a
115
- destination-specific payload and redact that copy before sending it:
123
+ Events are read-only, so copy the fields you need into a payload and redact the copy before sending it:
116
124
 
117
125
  ```ts title="agent/instrumentation/audit.ts"
118
126
  import { defineInstrumentation } from "eve/instrumentation";
@@ -148,45 +156,10 @@ async function sendAuditRecord(record: unknown): Promise<void> {
148
156
  }
149
157
  ```
150
158
 
151
- Prefer `recordInputs: false` or `recordOutputs: false` when the destination does not need an entire content direction. Use field-level redaction only when the destination needs part of that content.
152
-
153
- ## Lifecycle events
154
-
155
- Lifecycle events derived from session facts use the same names and outcomes as
156
- the stream. For example, `turn.settled` carries the turn's `outcome`. Match an
157
- operation's start and terminal events by their shared `idempotencyKey` when
158
- upserting records across replays.
159
-
160
- | Lifecycle | Events | Meaning |
161
- | ---------------- | ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
162
- | Session | `session.started`, `session.ended` | Session creation and end. `session.ended.outcome` is `completed` or `failed`. |
163
- | Turn | `turn.started`, `turn.settled` | Turn work. `outcome` is `completed`, `cancelled`, or `failed`; a cancelled turn is not a failure. |
164
- | Delivery | `delivery.consumed`, `delivery.settled` | The committed delivery facts. Settlement includes `outcome`: `handled`, `awaiting-input`, `applied`, `ignored`, `refused`, or `failed`. Channel metadata is on consumption when available. |
165
- | Model run | `model.started`, `model.retried`, `model.settled` | Each attempt of a model run. `model.retried` ends a failed attempt before another try; `model.settled.outcome` is `completed`, `failed`, `interrupted`, `abandoned`, or `cancelled` when the turn was cancelled. |
166
- | Provider call | `model.call.started`, `model.call.settled` | An individual provider call; one model attempt can make several. Settlement has `outcome: "completed"` or `"failed"`; completion carries usage, finish reason, and optional `providerMetadata`. |
167
- | Call | `call.requested`, `call.settled` | A tool, skill, or agent call, including direct `invokeTool` calls. `capability` identifies its kind and name. Settlement carries `completed`, `failed`, `rejected`, `interrupted`, or `abandoned`. |
168
- | User input | `interaction.opened`, `interaction.settled` | A question, approval, or budget prompt, keyed by `interactionId`. Settlement includes the interaction's `outcome`, `responder` (who decided it), and, when recorded, its `answer`. |
169
- | Memory operation | `memory.operation.started`, `memory.operation.settled` | A memory search or write, with operation name, phase, and slot. Settlement has `outcome: "completed"` or `"failed"`. |
170
-
171
- A delivery that pauses for a person settles `awaiting-input`; resumed work belongs to the answer's delivery. OTel attributes a turn to a delivery only when exactly one delivery was consumed. Event `scope` includes session and run coordinates; `turnId` can be absent for compaction or direct calls. Treat `idempotencyKey` as opaque.
172
-
173
- Use `InstrumentationEvent<"model.call.settled">` from `eve/instrumentation` when typing a
174
- standalone handler. Model input content is `unknown`: the messages are the provider's request
175
- messages, not an eve conversation-message schema. Narrow that content before reading it.
176
-
177
- Each call emits one `call.*` lifecycle; SDK callbacks do not duplicate it. A subagent's `call.settled` includes its reported usage. A thrown tool error is reconstructed from the durable record, so don't use `instanceof` to classify it; inspect `error.name` instead.
178
-
179
- When upgrading, use each operation's terminal event and inspect its `outcome` instead of subscribing to separate completion and failure events. See the [v27 event mapping](/docs/concepts/upgrade-to-v27-events#event-mapping). Provider metadata moved to `model.call.settled`. Replace step indices and encoded attempt IDs with `runId` and `attempt`; use `capability` for call kind and name. Channel types and `isChannel` now come from `eve/channels`. Runtime-context resolvers receive `run: { id, attempt }` instead of `step.index`.
180
-
181
- The OpenTelemetry trace schema remains version 4: span names, attributes, and span events are unchanged.
182
-
183
- Handlers from different files run concurrently and are failure-isolated. Do
184
- not depend on execution order. Use `flush` to drain buffered records and
185
- `shutdown` to release resources.
159
+ When the destination needs no content in one direction, set `recordInputs: false` or `recordOutputs: false` instead.
186
160
 
187
161
  ## What to read next
188
162
 
189
- - [Migrate instrumentation](/docs/observability/instrumentation-migration): replace `agent/instrumentation.ts`.
190
163
  - [OpenTelemetry](/docs/observability/otel): configure process-wide settings and third-party destinations.
191
164
  - [Agent Runs](/docs/observability/agent-runs): configure Vercel's built-in trace destination.
192
165
  - [Local development](../guides/dev-tui): inspect local traces in the TUI
@@ -1,5 +1,5 @@
1
1
  {
2
2
  "title": "Observability",
3
3
  "defaultOpen": false,
4
- "pages": ["instrumentation", "otel", "agent-runs", "migration"]
4
+ "pages": ["instrumentation", "otel", "agent-runs"]
5
5
  }
@@ -4,43 +4,11 @@ description: "Configure OpenTelemetry destinations, content capture, and third-p
4
4
  url: "/observability/otel"
5
5
  ---
6
6
 
7
- `eve dev` records local traces by default. Use `otel()` for process-wide
8
- settings and `otelIntegration()` for third-party destinations.
9
-
10
- ## Session handover events
11
-
12
- Existing Workflow step spans include these events when tracing is active:
13
-
14
- - `eve.session.handover.forwarding_started`: the previous owner starts submitting its buffered messages, including an empty buffer.
15
- - `eve.session.handover.part_submitted`: a buffered part was durably submitted.
16
- - `eve.session.handover.final_submitted`: the final marker was durably submitted. This does not confirm that the successor has processed it.
17
- - `eve.session.handover.inbox_unblocked`: an existing turn step observes that the inbox applied every buffered part and the final marker.
18
-
19
- Join both owners with `eve.session.id` and `eve.handoff.attempt_key`.
20
- `workflow.run.id` identifies the emitting owner; submission events also include
21
- `eve.handoff.destination_run_id`. Counts and part indexes describe the transfer
22
- without recording message content. Step IDs and `eve.handoff.step_attempt`
23
- distinguish retries; these events are not exactly-once counters.
24
-
25
- The receiver records `eve.handoff.applied_at_ms` using Workflow's replay-stable
26
- clock, identified by `eve.handoff.applied_at_clock: workflow`. This is logical
27
- event time, not the host time when replay applied the marker. The event's own
28
- timestamp is when the next turn step exports the observation. No receiver
29
- observation is exported if no subsequent turn step runs. An abandoned handover
30
- does not emit a successful receipt. No additional Workflow steps or writes are
31
- created for tracing.
7
+ eve emits OpenTelemetry traces for agent turns, and `eve dev` records them locally by default. Configure OpenTelemetry with files under `agent/instrumentation/`: `otel()` for process-wide settings, and `otelIntegration()` for each third-party destination.
32
8
 
33
9
  ## Configure OpenTelemetry
34
10
 
35
- `otel()` and `otelIntegration()` are optional:
36
-
37
- - Use `otel()` only when you need process-wide settings such as the resource,
38
- sampler, propagators, or trace capture policy. Declare it at most once.
39
- - Use `otelIntegration()` only when adding a third-party OpenTelemetry
40
- destination. Declare one file per exporter or processor chain.
41
-
42
- Add `agent/instrumentation/otel.ts` when you need process-wide settings or
43
- want to control the content eve writes to OpenTelemetry spans:
11
+ Both are optional. Declare `otel()` at most once, in `agent/instrumentation/otel.ts`, for settings such as the resource, sampler, propagators, or trace capture policy. Declare `otelIntegration()` in one file per third-party exporter or processor chain.
44
12
 
45
13
  ```ts title="agent/instrumentation/otel.ts"
46
14
  import { otel } from "eve/instrumentation/otel";
@@ -55,21 +23,84 @@ export default otel({
55
23
  });
56
24
  ```
57
25
 
58
- The OpenTelemetry `tracePolicy` is a capture ceiling shared by every OTel
59
- destination. A destination cannot restore content excluded by this policy. It
60
- does not affect lifecycle instrumentation created with `defineInstrumentation()`.
61
- See [Audience](../channels/overview#audience) for how channels classify the
62
- conversation passed to this policy.
63
-
64
- The session's resolved capture decision also selects its default
65
- [Workflow run-data retention](/docs/agent-config#run-data-retention). Allowing
66
- both inputs and outputs uses the World's default; dropping the trace or excluding
67
- either direction requests deletion of durable payloads and streams when the run
68
- ends. This makes completed private and unknown-audience transcripts unreadable
69
- under the default policy outside development. Set
70
- `experimental.workflow.retention: "default"` on the agent to retain completed
71
- sessions independently of tracing. Destination `exportPolicy` filters do not
72
- change Workflow retention, and Workflow deletion does not delete exported traces.
26
+ `otel({ tracePolicy })` sets the most content any OTel destination can receive. No destination can add back content it excludes. It doesn't affect instrumentation files created with `defineInstrumentation()`. See [Audience](../channels/overview#audience) for how channels classify the conversation passed to this policy.
27
+
28
+ The same capture decision sets the session's default [Workflow run-data retention](/docs/agent-config#run-data-retention):
29
+
30
+ - Allowing both inputs and outputs keeps the World's default retention.
31
+ - Dropping the trace or excluding either direction deletes stored payloads and streams when the run ends.
32
+
33
+ So under the default policy, completed private and unknown-audience transcripts are unreadable outside development. Set `experimental.workflow.retention: "default"` on the agent to keep completed sessions regardless of tracing. Destination `exportPolicy` filters don't change Workflow retention, and Workflow deletion doesn't delete exported traces.
34
+
35
+ ## Add an OpenTelemetry destination
36
+
37
+ To send traces to a third-party destination, install `@vercel/otel` for its exporters:
38
+
39
+ ```bash
40
+ pnpm add @vercel/otel
41
+ ```
42
+
43
+ ```ts title="agent/instrumentation/honeycomb.ts"
44
+ import { OTLPHttpProtoTraceExporter } from "@vercel/otel";
45
+ import { otelIntegration } from "eve/instrumentation/otel";
46
+
47
+ export default otelIntegration({
48
+ exportPolicy: {
49
+ span: () => ({ redact: true, inputs: true, outputs: true }),
50
+ },
51
+ traceExporter: new OTLPHttpProtoTraceExporter({
52
+ url: "https://api.honeycomb.io/v1/traces",
53
+ headers: {
54
+ "x-honeycomb-team": process.env.HONEYCOMB_API_KEY!,
55
+ },
56
+ }),
57
+ });
58
+ ```
59
+
60
+ The example's `exportPolicy` keeps span metadata but removes inputs and outputs from this destination. Pass `spanProcessors` to `otelIntegration()` when a custom destination needs filtering or transformation. Processors and exporters from any `@opentelemetry/*` build satisfy the `SpanProcessor` and `SpanExporter` types from `eve/instrumentation/otel`. `SpanProcessorOrName` also accepts `"auto"`.
61
+
62
+ ## Filter a destination
63
+
64
+ `otelIntegration()` and `localTraces()` accept one `exportPolicy` object or an array applied in order. Each policy filters spans and attributes before that destination's processors receive them:
65
+
66
+ ```ts title="agent/instrumentation/local.ts"
67
+ import { localTraces } from "eve/instrumentation/otel";
68
+
69
+ export default localTraces({
70
+ exportPolicy: {
71
+ span: ({ name }) => ({ emit: name !== "internal.cache.refresh" }),
72
+ attribute: ({ key }) => (key === "customer.id" ? { emit: false } : { emit: true }),
73
+ },
74
+ });
75
+ ```
76
+
77
+ | Callback | Return | Effect |
78
+ | ----------- | ---------------------------------------------------------------- | -------------------------------------- |
79
+ | `span` | `{ emit: true }` | Keep the span unchanged. |
80
+ | `span` | `{ emit: false }` | Omit the span from this destination. |
81
+ | `span` | `{ redact: true }` with `inputs: true`, `outputs: true`, or both | Keep the span and remove that content. |
82
+ | `span` | Throws | Drop the span from this destination. |
83
+ | `attribute` | `{ emit: true }` | Keep the attribute. |
84
+ | `attribute` | `{ emit: false }` | Remove the attribute. |
85
+ | `attribute` | `{ replace: true, value }` | Change the attribute's value. |
86
+
87
+ A redaction decision implies emission and needs at least one direction.
88
+
89
+ ## Local traces
90
+
91
+ Omit `agent/instrumentation/local.ts` to keep `eve dev`'s local traces. Export
92
+ `localTraces(...)` from that file to reconfigure them, or `disableInstrumentation()` to turn them off:
93
+
94
+ ```ts title="agent/instrumentation/local.ts"
95
+ import { disableInstrumentation } from "eve/instrumentation";
96
+
97
+ export default disableInstrumentation();
98
+ ```
99
+
100
+ ## Enable tracing on Vercel
101
+
102
+ For Agent Runs sampling defaults and existing-deployment setup, see
103
+ [Enable tracing on Vercel](./agent-runs#enable-tracing-on-vercel).
73
104
 
74
105
  ## Instrument Node.js dependencies
75
106
 
@@ -101,38 +132,6 @@ export default defineAgent({
101
132
  });
102
133
  ```
103
134
 
104
- ## Add an OpenTelemetry destination
105
-
106
- Use this section when sending traces to a third-party destination. Install
107
- `@vercel/otel` before using its exporters:
108
-
109
- ```bash
110
- pnpm add @vercel/otel
111
- ```
112
-
113
- ```ts title="agent/instrumentation/honeycomb.ts"
114
- import { OTLPHttpProtoTraceExporter } from "@vercel/otel";
115
- import { otelIntegration } from "eve/instrumentation/otel";
116
-
117
- export default otelIntegration({
118
- exportPolicy: {
119
- span: () => ({ redact: true, inputs: true, outputs: true }),
120
- },
121
- traceExporter: new OTLPHttpProtoTraceExporter({
122
- url: "https://api.honeycomb.io/v1/traces",
123
- headers: {
124
- "x-honeycomb-team": process.env.HONEYCOMB_API_KEY!,
125
- },
126
- }),
127
- });
128
- ```
129
-
130
- Pass `spanProcessors` to `otelIntegration()` when a custom destination needs
131
- filtering or transformation. Processors and exporters from any `@opentelemetry/*` build
132
- satisfy the `SpanProcessor` and `SpanExporter` types from `eve/instrumentation/otel`;
133
- `SpanProcessorOrName` also accepts `"auto"`. The example's `exportPolicy` preserves span
134
- metadata while removing inputs and outputs from this destination.
135
-
136
135
  ## Add runtime context
137
136
 
138
137
  Use `runtimeContext` to add JSON values to AI SDK spans for each model attempt.
@@ -157,20 +156,13 @@ export default otelIntegration({
157
156
  });
158
157
  ```
159
158
 
160
- The resolver receives the channel, session, final model input, `run: { id, attempt }`, and
161
- optional turn. Its input type is `InstrumentationModelStartedInput` from `eve/instrumentation`.
162
- Model input content is `unknown`; narrow provider-specific messages before reading them.
163
- Keys beginning with `eve.` are reserved. Content excluded by
164
- `otel({ tracePolicy })` is not available to the resolver.
159
+ The resolver receives the channel, session, final model input, `run: { id, attempt }`, and optional turn. Its input type is `InstrumentationModelStartedInput` from `eve/instrumentation`. Model input content is `unknown`, so narrow provider-specific messages before you read them. Content that `otel({ tracePolicy })` excludes isn't available to the resolver.
165
160
 
166
- Authored values appear under `ai.settings.context.*`. Framework channel, session,
167
- step, turn, and version identifiers use structural span attributes instead of
168
- runtime-context copies. `ai.settings.context.eve.environment` remains available.
161
+ Your values appear under `ai.settings.context.*`, next to `ai.settings.context.eve.environment`. Keys that begin with `eve.` are reserved. Channel, session, step, turn, and version IDs are regular span attributes, not runtime context.
169
162
 
170
163
  ## Trace topology
171
164
 
172
- OpenTelemetry destinations receive the following trace topology for an ordinary
173
- agent turn:
165
+ An ordinary agent turn produces this trace:
174
166
 
175
167
  ```text
176
168
  invoke_agent <agent>
@@ -179,53 +171,28 @@ invoke_agent <agent>
179
171
  └── execute_tool <tool>
180
172
  ```
181
173
 
182
- Each root agent turn starts a new trace. If eve receives a local caller's trace
183
- context, the first local subagent turn uses that trace. Its span is a
184
- child of the dispatch span and has a different span ID. Without caller trace
185
- context, the first subagent turn starts a new trace. Later turns in the child
186
- session start new traces. They do not use the original dispatch span as their
187
- parent.
188
-
189
- Use `gen_ai.tool.call.id` to correlate tool and approval spans. Tool failures use
190
- `error.type`. These attributes replace the duplicate `agent.action.call_id` and
191
- `agent.action.error.code` fields without changing trace schema version 4.
192
-
193
- Model and activation spans record token counts under `gen_ai.usage.*`, without
194
- duplicate `agent.usage.*` token fields. Step and tool spans retain `agent.usage.*`.
195
- Cost values remain unchanged.
196
-
197
- A remote child starts a separate trace with a root `invoke_agent` span. Its first
198
- turn links to the caller with `eve.link.type=agent.dispatch`. The receiving
199
- deployment applies its sampler. Queries for local dispatches must use
200
- `parentSpanId` or `agent.parent_run.id` and `agent.parent_call.id` instead of a
201
- dispatch link.
202
-
203
- Use `gen_ai.conversation.id` to find the root session and its child sessions
204
- across traces. eve adds this ID to spans created during model, tool, and memory
205
- execution. This includes spans from third-party instrumentation. `agent.run.id`
206
- identifies one agent session, not the full conversation.
207
-
208
- The shared conversation ID does not give access to message history, agent state,
209
- or credentials. It does not give authorization or require the same worker for
210
- each turn. Remote requests carry the conversation ID in W3C Baggage.
211
- `trustedForwarders` controls whether the receiver accepts parent session data
212
- and the trace-content policy. See
213
- [Remote agents](../guides/remote-agents#preserving-trace-content).
214
- Gateway-routed model calls propagate the configured OpenTelemetry context at
215
- each provider request. Eve-owned baggage is removed before forwarding. Gateway
216
- controls whether its spans are sampled and which destinations receive them.
217
-
218
- When AI Gateway reports
219
- [session affinity](https://vercel.com/docs/ai-gateway/models-and-providers/session-affinity)
220
- for a model call, the `agent.step` span records it. These attributes appear even
221
- when the trace content policy omits outputs.
222
-
223
- `vercel.ai_gateway.client_session_id` is the session ID AI Gateway used for
224
- affinity. It is not the eve session ID that `vercel.session_id` records. It
225
- equals `gen_ai.conversation.id` unless you set `gateway.sessionId` yourself or
226
- the ID is longer than 256 characters, in which case AI Gateway hashes it. An
227
- authored `gateway.sessionId` is recorded verbatim, so don't put personal data in
228
- it.
174
+ Each root agent turn starts a new trace. A local subagent's first turn joins the caller's trace, as a child of the dispatch span, when eve receives the caller's trace context. Otherwise it starts a new trace. Later turns in the child session start new traces without the dispatch span as parent.
175
+
176
+ A remote child starts a separate trace with a root `invoke_agent` span. Its first turn links to the caller with `eve.link.type=agent.dispatch`, and the receiving deployment applies its own sampler. Local dispatches have no dispatch link, so query them by `parentSpanId`, or by `agent.parent_run.id` and `agent.parent_call.id`.
177
+
178
+ | Attribute | Use |
179
+ | ------------------------ | ----------------------------------------------------------- |
180
+ | `gen_ai.conversation.id` | Find the root session and its child sessions across traces. |
181
+ | `agent.run.id` | Identify one agent session, not the full conversation. |
182
+ | `gen_ai.tool.call.id` | Match tool spans with their approval spans. |
183
+ | `error.type` | Classify tool failures. |
184
+ | `gen_ai.usage.*` | Token counts on model and activation spans. |
185
+ | `agent.usage.*` | Token counts on step and tool spans. |
186
+
187
+ The trace schema is version 4.
188
+
189
+ eve adds `gen_ai.conversation.id` to spans created during model, tool, and memory work, including spans from third-party instrumentation. The conversation ID grants no access to message history, agent state, or credentials, and no authorization. Remote requests carry it in W3C Baggage. `trustedForwarders` controls whether the receiver accepts parent session data and the trace-content policy. See [Remote agents](../guides/remote-agents#preserving-trace-content).
190
+
191
+ Gateway-routed model calls pass the configured OpenTelemetry context on each provider request. eve removes its own baggage first. AI Gateway decides whether its spans are sampled and which destinations receive them.
192
+
193
+ When AI Gateway reports [session affinity](https://vercel.com/docs/ai-gateway/models-and-providers/session-affinity) for a model call, the `agent.step` span records it, even when the trace content policy omits outputs.
194
+
195
+ `vercel.ai_gateway.client_session_id` is the session ID AI Gateway used for affinity. It isn't the eve session ID, which `vercel.session_id` records. Unless you set `gateway.sessionId` yourself, it equals `gen_ai.conversation.id`. AI Gateway hashes a conversation ID longer than 256 characters. eve records an authored `gateway.sessionId` exactly as written, so don't put personal data in it.
229
196
 
230
197
  | Attribute | Value |
231
198
  | ----------------------------------------------- | --------------------------------------------------------------- |
@@ -235,54 +202,21 @@ it.
235
202
  | `vercel.ai_gateway.affinity.pinned_provider` | Provider the session's pin named |
236
203
  | `vercel.ai_gateway.affinity.displaced_provider` | Provider that would have gone first, when the pin moved it |
237
204
 
238
- ## Local traces
239
-
240
- `eve dev` records local traces by default. Omitting
241
- `agent/instrumentation/local.ts` preserves that default. Export
242
- `localTraces(...)` from that file to reconfigure it, or disable it explicitly:
243
-
244
- ```ts title="agent/instrumentation/local.ts"
245
- import { disableInstrumentation } from "eve/instrumentation";
246
-
247
- export default disableInstrumentation();
248
- ```
249
-
250
- ## Enable tracing on Vercel
251
-
252
- For Agent Runs sampling defaults and existing-deployment setup, see
253
- [Enable tracing on Vercel](./agent-runs#enable-tracing-on-vercel).
254
-
255
- ## Filter a destination
205
+ ## Session handover events
256
206
 
257
- `otelIntegration()` and `localTraces()` accept one
258
- `exportPolicy` object or an array applied in order. Each policy filters spans
259
- and attributes before that destination's processors receive them:
207
+ When a session moves to a new workflow run, after a deployment change or [compaction](../concepts/execution-model-and-durability#compaction-handoff), Workflow step spans include these events:
260
208
 
261
- ```ts title="agent/instrumentation/local.ts"
262
- import { localTraces } from "eve/instrumentation/otel";
209
+ - `eve.session.handover.forwarding_started`: the previous run starts sending its buffered messages, even when the buffer is empty.
210
+ - `eve.session.handover.part_submitted`: eve saved one buffered part for the new run.
211
+ - `eve.session.handover.final_submitted`: eve saved the final marker. The new run may not have processed it yet.
212
+ - `eve.session.handover.inbox_unblocked`: a turn step in the new run sees that every buffered part and the final marker were applied.
263
213
 
264
- export default localTraces({
265
- exportPolicy: {
266
- span: ({ name }) => ({ emit: name !== "internal.cache.refresh" }),
267
- attribute: ({ key }) => (key === "customer.id" ? { emit: false } : { emit: true }),
268
- },
269
- });
270
- ```
214
+ Join the two runs with `eve.session.id` and `eve.handoff.attempt_key`. `workflow.run.id` identifies the run that emitted the event, and submission events also include `eve.handoff.destination_run_id`. Counts and part indexes describe the transfer without message content. Step retries can repeat events, so tell them apart by step ID and `eve.handoff.step_attempt` instead of counting events.
271
215
 
272
- Return `{ emit: true }` from `span` to retain a span unchanged, or
273
- `{ emit: false }` to omit it from this destination. To retain the span while
274
- redacting content, return `{ redact: true }` with `inputs: true`,
275
- `outputs: true`, or both. A redaction decision implies emission and requires
276
- at least one direction. For example, return
277
- `{ redact: true, inputs: true, outputs: true }` to redact both directions.
278
- A throwing `span` callback drops the span from this destination. Return
279
- `{ emit: true }`, `{ emit: false }`, or
280
- `{ replace: true, value }` from `attribute` to retain, remove, or change
281
- one attribute.
216
+ The new run records `eve.handoff.applied_at_ms` on Workflow's replay-stable clock (`eve.handoff.applied_at_clock: workflow`). That value is logical event time, not host time. The event's own timestamp is when the next turn step exports it, so the new run emits no event until a later turn step runs. An abandoned handover emits no success event.
282
217
 
283
218
  ## What to read next
284
219
 
285
220
  - [Instrumentation](/docs/observability/instrumentation): handle eve lifecycle events.
286
221
  - [Agent Runs](./agent-runs): configure Vercel's built-in trace destination.
287
- - [Migrate instrumentation](/docs/observability/instrumentation-migration): replace `agent/instrumentation.ts`.
288
222
  - [Local development](../guides/dev-tui): inspect local traces in the TUI.