eve 0.71.3 → 0.72.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +37 -0
- package/README.md +22 -6
- package/dist/src/acp/adapter.js +1 -1
- package/dist/src/ai/decide.d.ts +7 -0
- package/dist/src/ai/decide.js +1 -0
- package/dist/src/chunks/{use-eve-agent-CFCaYnrI.js → use-eve-agent-C1GYGCtb.js} +698 -388
- package/dist/src/chunks/{use-eve-agent-DcDhXNv1.js → use-eve-agent-hZJ7CB8t.js} +698 -388
- package/dist/src/cli/commands/integration-setup.js +1 -1
- package/dist/src/cli/commands/registry.js +1 -1
- package/dist/src/cli/commands/trace-detail.d.ts +1 -1
- package/dist/src/cli/commands/trace-detail.js +1 -1
- package/dist/src/cli/dev/tui/runner.js +1 -1
- package/dist/src/cli/dev/tui/setup-flow.d.ts +1 -0
- package/dist/src/cli/dev/tui/setup-panel.d.ts +1 -0
- package/dist/src/cli/dev/tui/setup-panel.js +2 -1
- package/dist/src/cli/dev/tui/traces/trace-conversation.d.ts +2 -2
- package/dist/src/cli/dev/tui/traces/trace-conversation.js +2 -2
- package/dist/src/cli/dev/tui/transcript-parts.d.ts +2 -1
- package/dist/src/cli/dev/tui/transcript-parts.js +1 -1
- package/dist/src/cli/dev/tui/transcript.js +1 -1
- package/dist/src/cli/dev/tui/tui-prompter.js +1 -1
- package/dist/src/client/conversation-projection.d.ts +4 -0
- package/dist/src/client/conversation-projection.js +1 -0
- package/dist/src/client/conversation-reducer.js +1 -1
- package/dist/src/client/conversation-state.d.ts +5 -26
- package/dist/src/client/conversation-state.js +1 -1
- package/dist/src/client/index.d.ts +1 -0
- package/dist/src/client/index.js +1 -1
- package/dist/src/client/message-action-parts.d.ts +1 -11
- package/dist/src/client/message-action-parts.js +1 -1
- package/dist/src/client/message-reducer.d.ts +3 -1
- package/dist/src/client/message-reducer.js +1 -1
- package/dist/src/client/session-utils.js +1 -1
- package/dist/src/client/session.js +1 -1
- package/dist/src/client/tool-call-state.d.ts +23 -0
- package/dist/src/client/tool-call-state.js +1 -0
- package/dist/src/client/tool-part-state.d.ts +7 -0
- package/dist/src/client/tool-part-state.js +1 -0
- package/dist/src/compiled/.vendor-stamp.json +10 -10
- package/dist/src/compiled/@ai-sdk/anthropic/index.d.ts +1453 -1358
- package/dist/src/compiled/@ai-sdk/anthropic/index.js +1 -2
- package/dist/src/compiled/@ai-sdk/code-mode/index.d.ts +172 -163
- package/dist/src/compiled/@ai-sdk/code-mode/index.js +15 -20
- package/dist/src/compiled/@ai-sdk/google/index.d.ts +772 -733
- package/dist/src/compiled/@ai-sdk/google/index.js +6 -8
- package/dist/src/compiled/@ai-sdk/mcp/index.d.ts +887 -790
- package/dist/src/compiled/@ai-sdk/mcp/index.js +1 -1
- package/dist/src/compiled/@ai-sdk/openai/index.d.ts +1722 -1691
- package/dist/src/compiled/@ai-sdk/openai/index.js +3 -14
- package/dist/src/compiled/@ai-sdk/otel/index.d.ts +216 -159
- package/dist/src/compiled/@ai-sdk/otel/index.js +3 -3
- package/dist/src/compiled/@ai-sdk/provider/index.d.ts +7542 -7271
- package/dist/src/compiled/@ai-sdk/provider/index.js +1 -3
- package/dist/src/compiled/@ai-sdk/provider-utils/index.d.ts +1799 -1600
- package/dist/src/compiled/@eve/catalog/index.js +1 -1
- package/dist/src/compiled/_chunks/workflow/dist-DXLdlucM.js +15 -0
- package/dist/src/compiled/_chunks/workflow/experimental-decision-BrfWf6-n.js +3 -0
- package/dist/src/compiler/extension-compatibility.d.ts +36 -20
- package/dist/src/compiler/extension-compatibility.js +1 -1
- package/dist/src/compiler/manifest.js +1 -1
- package/dist/src/context/build-dynamic-tools.js +1 -1
- package/dist/src/context/dynamic-tool-lifecycle.js +1 -1
- package/dist/src/context/dynamic-tool-metadata.d.ts +1 -0
- package/dist/src/context/providers/session.js +1 -1
- package/dist/src/evals/define-eval.d.ts +1 -1
- package/dist/src/evals/judge.d.ts +1 -1
- package/dist/src/evals/judge.js +2 -2
- package/dist/src/evals/mock-model.js +1 -1
- package/dist/src/evals/types.d.ts +8 -8
- package/dist/src/execution/cancel-descendant-turns-step.d.ts +1 -0
- package/dist/src/execution/cancel-descendant-turns-step.js +1 -1
- package/dist/src/execution/cancelled-model-call-batch.js +1 -1
- package/dist/src/execution/coordination-dispatch-shared.js +1 -1
- package/dist/src/execution/coordination-dispatch-step.js +1 -1
- package/dist/src/execution/durable-session-read.d.ts +5 -0
- package/dist/src/execution/durable-session-read.js +1 -0
- package/dist/src/execution/durable-session-store.d.ts +4 -10
- package/dist/src/execution/durable-session-store.js +1 -1
- package/dist/src/execution/dynamic-connections.d.ts +3 -3
- package/dist/src/execution/dynamic-connections.js +1 -1
- package/dist/src/execution/dynamic-workflow/tool.js +1 -1
- package/dist/src/execution/legacy-session/interrupt-step.js +1 -1
- package/dist/src/execution/legacy-session/snapshot.d.ts +3 -0
- package/dist/src/execution/legacy-session/snapshot.js +1 -1
- package/dist/src/execution/model-call-batching.js +1 -1
- package/dist/src/execution/node-step.d.ts +2 -0
- package/dist/src/execution/node-step.js +1 -1
- package/dist/src/execution/proxied-deliver-step.js +1 -1
- package/dist/src/execution/publish-session-events.d.ts +7 -2
- package/dist/src/execution/publish-session-events.js +1 -1
- package/dist/src/execution/session/active-turn.js +1 -1
- package/dist/src/execution/session/answer-caller.js +1 -1
- package/dist/src/execution/session/checkpoint-migrations.js +1 -1
- package/dist/src/execution/session/finalization.js +1 -1
- package/dist/src/execution/session/handoff-steps.d.ts +3 -1
- package/dist/src/execution/session/handoff-steps.js +1 -1
- package/dist/src/execution/session/handoff.d.ts +1 -1
- package/dist/src/execution/session/handoff.js +1 -1
- package/dist/src/execution/session/pending-turn-state.d.ts +8 -3
- package/dist/src/execution/session/pending-turn-state.js +1 -1
- package/dist/src/execution/session/task-tool-tracing-step.d.ts +5 -0
- package/dist/src/execution/session/task-tool-tracing-step.js +1 -0
- package/dist/src/execution/session/turn-step-result.js +1 -1
- package/dist/src/execution/session/turn-step.js +1 -1
- package/dist/src/execution/session/turn-waiting-step.js +1 -1
- package/dist/src/execution/session/turn.d.ts +1 -0
- package/dist/src/execution/session/turn.js +1 -1
- package/dist/src/execution/settle-cancelled-turn-step.d.ts +4 -6
- package/dist/src/execution/settle-cancelled-turn-step.js +1 -1
- package/dist/src/execution/tasks/start.d.ts +3 -3
- package/dist/src/execution/tasks/start.js +1 -1
- package/dist/src/execution/tasks/steps.d.ts +1 -1
- package/dist/src/execution/tasks/steps.js +1 -1
- package/dist/src/execution/terminal-session-completion-step.d.ts +5 -1
- package/dist/src/execution/terminal-session-completion-step.js +1 -1
- package/dist/src/execution/terminal-session-failure-step.js +1 -1
- package/dist/src/execution/tools/agent-router-workflow.js +1 -1
- package/dist/src/execution/tools/connection-tools.js +2 -2
- package/dist/src/execution/tools/workflow/harness-definition.js +1 -1
- package/dist/src/execution/tools/workflow/step.js +1 -1
- package/dist/src/execution/tools/workflow/withdraw-step.d.ts +7 -9
- package/dist/src/execution/tools/workflow/withdraw-step.js +1 -1
- package/dist/src/execution/workflow-trace-context.d.ts +2 -2
- package/dist/src/harness/authorization.d.ts +4 -2
- package/dist/src/harness/authorization.js +1 -1
- package/dist/src/harness/{compaction.js → compaction/engine.js} +1 -1
- package/dist/src/harness/compaction/step.d.ts +46 -0
- package/dist/src/harness/compaction/step.js +1 -0
- package/dist/src/harness/coordination.d.ts +32 -68
- package/dist/src/harness/coordination.js +1 -1
- package/dist/src/harness/current-messages.d.ts +1 -3
- package/dist/src/harness/current-messages.js +1 -1
- package/dist/src/harness/emission.d.ts +4 -52
- package/dist/src/harness/emission.js +1 -1
- package/dist/src/harness/execute-tool.d.ts +1 -0
- package/dist/src/harness/final-output.d.ts +5 -0
- package/dist/src/harness/final-output.js +1 -1
- package/dist/src/harness/hitl/approvals.d.ts +115 -0
- package/dist/src/harness/hitl/approvals.js +1 -0
- package/dist/src/harness/hitl/approved-call-callers.d.ts +14 -0
- package/dist/src/harness/hitl/approved-call-callers.js +1 -0
- package/dist/src/harness/hitl/approved-calls.d.ts +37 -0
- package/dist/src/harness/hitl/approved-calls.js +1 -0
- package/dist/src/harness/hitl/budget.d.ts +21 -0
- package/dist/src/harness/hitl/budget.js +1 -0
- package/dist/src/harness/{approval-candidates.d.ts → hitl/candidates.d.ts} +9 -18
- package/dist/src/harness/hitl/candidates.js +1 -0
- package/dist/src/harness/hitl/coordinator.js +1 -0
- package/dist/src/harness/hitl/delivery.d.ts +37 -0
- package/dist/src/harness/hitl/delivery.js +1 -0
- package/dist/src/harness/hitl/index.d.ts +49 -0
- package/dist/src/harness/hitl/index.js +1 -0
- package/dist/src/harness/hitl/intake.d.ts +67 -0
- package/dist/src/harness/hitl/intake.js +1 -0
- package/dist/src/harness/{stale-input-responses.d.ts → hitl/stale-responses.d.ts} +5 -3
- package/dist/src/harness/hitl/stale-responses.js +3 -0
- package/dist/src/harness/inline-tool-authorization.d.ts +4 -0
- package/dist/src/harness/inline-tool-authorization.js +1 -1
- package/dist/src/harness/input-extraction.d.ts +1 -10
- package/dist/src/harness/input-extraction.js +1 -1
- package/dist/src/harness/input-request-resolution.d.ts +1 -5
- package/dist/src/harness/input-request-resolution.js +1 -1
- package/dist/src/harness/model-call/call.d.ts +78 -0
- package/dist/src/harness/model-call/call.js +1 -0
- package/dist/src/harness/{model-call-error.d.ts → model-call/errors.d.ts} +1 -1
- package/dist/src/harness/model-call/failure.d.ts +52 -0
- package/dist/src/harness/model-call/failure.js +1 -0
- package/dist/src/harness/model-call/model.d.ts +18 -0
- package/dist/src/harness/model-call/model.js +1 -0
- package/dist/src/harness/model-call/recovery.d.ts +52 -0
- package/dist/src/harness/model-call/recovery.js +1 -0
- package/dist/src/harness/model-call/request.d.ts +30 -0
- package/dist/src/harness/model-call/request.js +3 -0
- package/dist/src/harness/model-call/response.d.ts +39 -0
- package/dist/src/harness/model-call/response.js +1 -0
- package/dist/src/harness/model-call/retry.d.ts +11 -0
- package/dist/src/harness/model-call/retry.js +1 -0
- package/dist/src/harness/model-call/run.d.ts +38 -0
- package/dist/src/harness/model-call/run.js +1 -0
- package/dist/src/harness/model-call/tools.d.ts +48 -0
- package/dist/src/harness/model-call/tools.js +1 -0
- package/dist/src/harness/model-call/usage.d.ts +23 -0
- package/dist/src/harness/model-call/usage.js +1 -0
- package/dist/src/harness/nested-actions.d.ts +2 -2
- package/dist/src/harness/provider-tools.d.ts +2 -2
- package/dist/src/harness/provider-tools.js +1 -1
- package/dist/src/harness/proxy-input-requests.d.ts +4 -12
- package/dist/src/harness/proxy-input-requests.js +1 -1
- package/dist/src/harness/session-machine/commit.d.ts +28 -0
- package/dist/src/harness/session-machine/commit.js +1 -0
- package/dist/src/harness/session-machine/current.d.ts +29 -0
- package/dist/src/harness/session-machine/current.js +1 -0
- package/dist/src/harness/session-machine/events.d.ts +6 -0
- package/dist/src/harness/session-machine/events.js +1 -0
- package/dist/src/harness/session-machine/resolver-events.d.ts +12 -0
- package/dist/src/harness/session-machine/resolver-events.js +1 -0
- package/dist/src/harness/session-machine/state.d.ts +47 -0
- package/dist/src/harness/session-machine/state.js +1 -0
- package/dist/src/harness/session-machine/transitions.d.ts +188 -0
- package/dist/src/harness/session-machine/transitions.js +1 -0
- package/dist/src/harness/session-machine/view.d.ts +58 -0
- package/dist/src/harness/session-machine/view.js +1 -0
- package/dist/src/harness/step/after-model.d.ts +7 -0
- package/dist/src/harness/step/after-model.js +1 -0
- package/dist/src/harness/step/context.d.ts +59 -0
- package/dist/src/harness/step/context.js +1 -0
- package/dist/src/harness/step/intake.d.ts +34 -0
- package/dist/src/harness/step/intake.js +1 -0
- package/dist/src/harness/step/prompt.d.ts +37 -0
- package/dist/src/harness/step/prompt.js +1 -0
- package/dist/src/harness/step-hooks.d.ts +8 -13
- package/dist/src/harness/step-hooks.js +1 -1
- package/dist/src/harness/tool-loop.d.ts +7 -22
- package/dist/src/harness/tool-loop.js +1 -3
- package/dist/src/harness/tools.d.ts +13 -0
- package/dist/src/harness/tools.js +1 -1
- package/dist/src/harness/turn-tag-state.d.ts +1 -1
- package/dist/src/harness/types.d.ts +6 -0
- package/dist/src/instrumentation/ai-sdk-hook-bridge.d.ts +1 -1
- package/dist/src/instrumentation/ai-sdk-hook-bridge.js +1 -1
- package/dist/src/instrumentation/content.js +1 -1
- package/dist/src/instrumentation/dispatch.js +1 -1
- package/dist/src/instrumentation/lifecycle.d.ts +28 -56
- package/dist/src/instrumentation/native-events.d.ts +2 -1
- package/dist/src/instrumentation/native-events.js +1 -1
- package/dist/src/instrumentation/provider.d.ts +1 -1
- package/dist/src/instrumentation/runtime-context.d.ts +2 -3
- package/dist/src/instrumentation/runtime-context.js +1 -1
- package/dist/src/instrumentation/runtime.d.ts +11 -1
- package/dist/src/instrumentation/runtime.js +1 -1
- package/dist/src/instrumentation/state.d.ts +6 -1
- package/dist/src/instrumentation/state.js +1 -1
- package/dist/src/internal/application/package.js +1 -1
- package/dist/src/internal/attachments/limited-read.d.ts +16 -0
- package/dist/src/internal/attachments/limited-read.js +1 -0
- package/dist/src/internal/authored-definition/schema-backed.d.ts +2 -1
- package/dist/src/internal/authored-definition/schema-backed.js +1 -1
- package/dist/src/internal/mcp/published-tools.d.ts +13 -0
- package/dist/src/internal/mcp/published-tools.js +1 -0
- package/dist/src/internal/mcp/request-trace-context.d.ts +12 -0
- package/dist/src/internal/mcp/request-trace-context.js +1 -0
- package/dist/src/internal/mcp/streamable-http-server.d.ts +7 -3
- package/dist/src/internal/mcp/streamable-http-server.js +1 -1
- package/dist/src/internal/model-auth/transport.d.ts +3 -3
- package/dist/src/internal/model-auth/transport.js +1 -1
- package/dist/src/models/auto.d.ts +5 -5
- package/dist/src/models/auto.js +2 -2
- package/dist/src/protocol/message.d.ts +7 -0
- package/dist/src/protocol/message.js +1 -1
- package/dist/src/protocol/session-projection.d.ts +159 -0
- package/dist/src/protocol/session-projection.js +1 -0
- package/dist/src/public/ai/index.d.ts +1 -1
- package/dist/src/public/ai/index.js +1 -1
- package/dist/src/public/channels/chat-sdk/attachment-refs.d.ts +15 -0
- package/dist/src/public/channels/chat-sdk/attachment-refs.js +1 -0
- package/dist/src/public/channels/chat-sdk/chatSdkChannel.js +2 -2
- package/dist/src/public/channels/chat-sdk/messageToUserContent.d.ts +11 -5
- package/dist/src/public/channels/chat-sdk/messageToUserContent.js +1 -1
- package/dist/src/public/channels/discord/attachments.d.ts +15 -0
- package/dist/src/public/channels/discord/attachments.js +1 -0
- package/dist/src/public/channels/discord/discordChannel.js +1 -1
- package/dist/src/public/channels/discord/inbound.d.ts +13 -0
- package/dist/src/public/channels/discord/inbound.js +1 -1
- package/dist/src/public/channels/linear/inbound-uploads.d.ts +21 -0
- package/dist/src/public/channels/linear/inbound-uploads.js +1 -0
- package/dist/src/public/channels/linear/linearChannel.js +1 -1
- package/dist/src/public/channels/linq/inboundContent.d.ts +7 -1
- package/dist/src/public/channels/linq/inboundContent.js +1 -1
- package/dist/src/public/channels/linq/linqChannel.js +1 -1
- package/dist/src/public/channels/mcp.d.ts +17 -2
- package/dist/src/public/channels/mcp.js +1 -1
- package/dist/src/public/channels/photon/attachments.d.ts +21 -0
- package/dist/src/public/channels/photon/attachments.js +1 -0
- package/dist/src/public/channels/photon/photonIMessageChannel.js +1 -1
- package/dist/src/public/channels/slack/attachments.d.ts +3 -1
- package/dist/src/public/channels/slack/attachments.js +1 -1
- package/dist/src/public/channels/slack/slackChannel.js +1 -1
- package/dist/src/public/channels/teams/attachments.d.ts +4 -4
- package/dist/src/public/channels/teams/attachments.js +1 -1
- package/dist/src/public/channels/telegram/attachments.js +1 -1
- package/dist/src/public/channels/twilio/api.d.ts +7 -0
- package/dist/src/public/channels/twilio/api.js +1 -1
- package/dist/src/public/channels/twilio/inbound.d.ts +7 -0
- package/dist/src/public/channels/twilio/inbound.js +2 -2
- package/dist/src/public/channels/twilio/twilioChannel.js +1 -1
- package/dist/src/public/definitions/connections/mcp.d.ts +1 -1
- package/dist/src/public/definitions/connections/openapi.d.ts +1 -1
- package/dist/src/react/index.d.ts +1 -0
- package/dist/src/react/index.js +1 -1
- package/dist/src/runtime/agent/mock-model-adapter.js +2 -2
- package/dist/src/runtime/connections/mcp-client.d.ts +18 -1
- package/dist/src/runtime/connections/mcp-client.js +1 -1
- package/dist/src/runtime/resolve-tool.js +1 -1
- package/dist/src/setup/boxes/resolve-provisioning.d.ts +2 -0
- package/dist/src/setup/boxes/resolve-provisioning.js +1 -1
- package/dist/src/setup/cli/prompt-ui.d.ts +3 -0
- package/dist/src/setup/cli/prompt-ui.js +3 -1
- package/dist/src/setup/cli/select-component.d.ts +1 -0
- package/dist/src/setup/cli/select-component.js +1 -1
- package/dist/src/setup/flows/ensure-vercel-project.d.ts +2 -0
- package/dist/src/setup/flows/ensure-vercel-project.js +1 -1
- package/dist/src/setup/integrations/discord/api.js +1 -1
- package/dist/src/setup/integrations/web/auth-options.d.ts +4 -0
- package/dist/src/setup/integrations/web/auth-options.js +1 -0
- package/dist/src/setup/integrations/web/auth-scaffold.d.ts +7 -0
- package/dist/src/setup/integrations/web/auth-scaffold.js +1 -0
- package/dist/src/setup/integrations/web/provision-auth.d.ts +3 -0
- package/dist/src/setup/integrations/web/provision-auth.js +1 -0
- package/dist/src/setup/integrations/web/setup.d.ts +11 -6
- package/dist/src/setup/integrations/web/setup.js +3 -3
- package/dist/src/setup/prompter.d.ts +2 -0
- package/dist/src/setup/prompter.js +1 -1
- package/dist/src/setup/scaffold/create/add-to-project.js +1 -1
- package/dist/src/setup/scaffold/create/project.js +2 -2
- package/dist/src/setup/scaffold/create/web-template.d.ts +1 -1
- package/dist/src/setup/scaffold/create/web-template.js +1 -0
- package/dist/src/setup/scaffold/update/web-options.js +1 -1
- package/dist/src/setup/vercel-project-api.d.ts +9 -0
- package/dist/src/setup/vercel-project-api.js +1 -1
- package/dist/src/setup/vercel-project.d.ts +4 -2
- package/dist/src/setup/vercel-project.js +0 -0
- package/dist/src/shared/web-search.d.ts +1 -1
- package/dist/src/subagents/adapter.js +1 -1
- package/dist/src/subagents/event-proxy-step.js +1 -1
- package/dist/src/subagents/hitl-proxy.d.ts +4 -28
- package/dist/src/subagents/hitl-proxy.js +1 -1
- package/dist/src/svelte/index.d.ts +2 -0
- package/dist/src/svelte/index.js +39 -2
- package/dist/src/svelte/use-eve-agent.js +1 -1
- package/dist/src/tools/approval/policies.d.ts +5 -5
- package/dist/src/tools/approval/policies.js +1 -1
- package/dist/src/tools/framework/agent.js +1 -1
- package/dist/src/tools/provided/agent-router.js +1 -1
- package/dist/src/tools/provided/ask-question-workflow.js +1 -1
- package/dist/src/tools/provided/ask-question.js +2 -2
- package/dist/src/tools/provided/bash.js +1 -1
- package/dist/src/tools/provided/framework-tool.d.ts +3 -0
- package/dist/src/tools/provided/framework-tool.js +1 -0
- package/dist/src/tools/provided/glob.js +2 -2
- package/dist/src/tools/provided/grep.js +2 -2
- package/dist/src/tools/provided/load-skill.js +1 -1
- package/dist/src/tools/provided/no-reply.js +1 -1
- package/dist/src/tools/provided/read-file.js +2 -2
- package/dist/src/tools/provided/sleep-workflow.js +1 -1
- package/dist/src/tools/provided/sleep.js +1 -1
- package/dist/src/tools/provided/task-cancel.js +1 -1
- package/dist/src/tools/provided/task-wait.js +1 -1
- package/dist/src/tools/provided/web-fetch.js +2 -2
- package/dist/src/tools/provided/web-search.js +1 -1
- package/dist/src/tools/provided/workflow.js +1 -1
- package/dist/src/tools/provided/write-file.js +2 -2
- package/dist/src/tracing/agent-approval-instrumentation.d.ts +2 -2
- package/dist/src/tracing/agent-approval-instrumentation.js +1 -1
- package/dist/src/tracing/agent-otel-provider.js +2 -2
- package/dist/src/tracing/agent-otel-usage.d.ts +1 -1
- package/dist/src/tracing/agent-otel-usage.js +1 -1
- package/dist/src/tracing/agent-span-contract.d.ts +0 -1
- package/dist/src/tracing/agent-span-contract.js +1 -1
- package/dist/src/tracing/agent-tool-instrumentation.d.ts +16 -6
- package/dist/src/tracing/agent-tool-instrumentation.js +1 -1
- package/dist/src/tracing/agent-trace-context-codec.js +1 -1
- package/dist/src/tracing/agent-trace-state.d.ts +7 -3
- package/dist/src/tracing/invoke-tool-span.js +1 -1
- package/dist/src/tracing/local-trace-analysis.js +1 -1
- package/dist/src/vue/index.d.ts +2 -0
- package/dist/src/vue/index.js +39 -2
- package/dist/src/vue/use-eve-agent.js +1 -1
- package/docs/agent-config.md +2 -2
- package/docs/channels/chat-sdk.mdx +3 -1
- package/docs/channels/custom.mdx +2 -0
- package/docs/channels/discord.mdx +5 -4
- package/docs/channels/eve.mdx +65 -0
- package/docs/channels/linear.mdx +1 -1
- package/docs/channels/linq.mdx +2 -0
- package/docs/channels/mcp.mdx +39 -1
- package/docs/channels/photon.mdx +5 -0
- package/docs/channels/teams.mdx +4 -7
- package/docs/channels/twilio.mdx +1 -1
- package/docs/concepts/built-in-tools.md +14 -2
- package/docs/concepts/sessions-runs-and-streaming.md +3 -3
- package/docs/connections/mcp.mdx +11 -0
- package/docs/connections/overview.mdx +2 -2
- package/docs/dynamic-configuration/meta.json +1 -1
- package/docs/evals/judge.mdx +16 -16
- package/docs/evals/overview.mdx +1 -1
- package/docs/guides/{evaluate.md → decide.md} +35 -35
- package/docs/guides/dynamic-capabilities.md +1 -1
- package/docs/guides/frontend/overview.mdx +3 -1
- package/docs/guides/hooks.md +1 -1
- package/docs/guides/instrumentation/instrumentation.mdx +23 -22
- package/docs/guides/instrumentation/otel.mdx +13 -2
- package/docs/reference/cli.md +3 -3
- package/docs/reference/typescript-api.md +4 -2
- package/docs/tools/human-in-the-loop.md +6 -4
- package/docs/tools/workflows.mdx +4 -4
- package/package.json +11 -11
- package/dist/src/ai/evaluate.d.ts +0 -7
- package/dist/src/ai/evaluate.js +0 -1
- package/dist/src/client/message-task-parts.d.ts +0 -8
- package/dist/src/client/message-task-parts.js +0 -1
- package/dist/src/compiled/_chunks/workflow/dist-C3ZE62yI.js +0 -25
- package/dist/src/compiled/_chunks/workflow/experimental-evaluation-Dm32egEG.js +0 -5
- package/dist/src/harness/active-turn-id.d.ts +0 -11
- package/dist/src/harness/active-turn-id.js +0 -1
- package/dist/src/harness/approval-candidates.js +0 -1
- package/dist/src/harness/approval-delivery-coordinator.js +0 -1
- package/dist/src/harness/approved-call-callers.d.ts +0 -11
- package/dist/src/harness/approved-call-callers.js +0 -1
- package/dist/src/harness/cancelled-turn-emission.d.ts +0 -12
- package/dist/src/harness/cancelled-turn-emission.js +0 -1
- package/dist/src/harness/emission-state.d.ts +0 -36
- package/dist/src/harness/emission-state.js +0 -1
- package/dist/src/harness/held-requests.d.ts +0 -18
- package/dist/src/harness/held-requests.js +0 -1
- package/dist/src/harness/hitl/approval-input-requests.d.ts +0 -27
- package/dist/src/harness/hitl/approval-input-requests.js +0 -1
- package/dist/src/harness/hitl/pending-input-resolution.d.ts +0 -54
- package/dist/src/harness/hitl/pending-input-resolution.js +0 -1
- package/dist/src/harness/hitl/session-limit-input-requests.d.ts +0 -12
- package/dist/src/harness/hitl/session-limit-input-requests.js +0 -1
- package/dist/src/harness/input-requests.d.ts +0 -37
- package/dist/src/harness/input-requests.js +0 -1
- package/dist/src/harness/pending-input-batches.d.ts +0 -81
- package/dist/src/harness/pending-input-batches.js +0 -1
- package/dist/src/harness/session-limit-enforcement.d.ts +0 -45
- package/dist/src/harness/session-limit-enforcement.js +0 -1
- package/dist/src/harness/stale-input-responses.js +0 -3
- package/dist/src/harness/workflow-dispatch.d.ts +0 -30
- package/dist/src/harness/workflow-dispatch.js +0 -1
- package/dist/src/public/channels/linear/inbound-images.d.ts +0 -19
- package/dist/src/public/channels/linear/inbound-images.js +0 -1
- package/dist/src/tracing/agent-action-instrumentation.d.ts +0 -26
- package/dist/src/tracing/agent-action-instrumentation.js +0 -1
- /package/dist/src/harness/{compaction.d.ts → compaction/engine.d.ts} +0 -0
- /package/dist/src/harness/{compaction-prompt.d.ts → compaction/prompt.d.ts} +0 -0
- /package/dist/src/harness/{compaction-prompt.js → compaction/prompt.js} +0 -0
- /package/dist/src/harness/{session-limit-continuation.d.ts → hitl/budget-request.d.ts} +0 -0
- /package/dist/src/harness/{session-limit-continuation.js → hitl/budget-request.js} +0 -0
- /package/dist/src/harness/{approval-delivery-coordinator.d.ts → hitl/coordinator.d.ts} +0 -0
- /package/dist/src/harness/{model-call-error.js → model-call/errors.js} +0 -0
|
@@ -208,6 +208,18 @@ import { webSearch } from "eve/tools/web_search";
|
|
|
208
208
|
export default webSearch({ provider: "parallel" });
|
|
209
209
|
```
|
|
210
210
|
|
|
211
|
+
Select Browserbase Search in the same slot:
|
|
212
|
+
|
|
213
|
+
```ts title="agent/tools/web_search.ts"
|
|
214
|
+
import { webSearch } from "eve/tools/web_search";
|
|
215
|
+
|
|
216
|
+
export default webSearch({ provider: "browserbase" });
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
Use a [Gateway model ID](../agent-config#set-the-model) to route searches through Browserbase. AI Gateway executes the search using `AI_GATEWAY_API_KEY` or Vercel project OIDC credentials; no `BROWSERBASE_API_KEY` is needed. See [Browserbase Search on AI Gateway](https://vercel.com/docs/ai-gateway/models-and-providers/web-search#using-browserbase-search).
|
|
220
|
+
|
|
221
|
+
The `provider` setting applies only to AI Gateway models. Unsupported direct providers omit `web_search`.
|
|
222
|
+
|
|
211
223
|
Replace provider-managed search with an authored implementation:
|
|
212
224
|
|
|
213
225
|
```ts title="agent/tools/web_search.ts"
|
|
@@ -286,9 +298,9 @@ export default disableTool();
|
|
|
286
298
|
|
|
287
299
|
`connection_search` and `connection_execute` give the model every tool from the agent's [connections](../connections) without adding each tool to the model's tool list. eve adds both when the agent has a static connection or a dynamic connection resolver, even when `defaultTools` is `false`, so there is no add command.
|
|
288
300
|
|
|
289
|
-
- `connection_search({ query?, connection?, signIn?, limit?, offset? })` returns matching tools with their connection, name, description, and a TypeScript signature rendered from the tool's schemas. Omit `query` to list every tool, or pair it with `connection` to list one connection's tools. A plain search never asks the user to sign in. For a connection
|
|
301
|
+
- `connection_search({ query?, connection?, signIn?, limit?, offset? })` returns matching tools with their connection, name, description, and a TypeScript signature rendered from the tool's schemas. Omit `query` to list every tool, or pair it with `connection` to list one connection's tools. A plain search never asks the user to sign in. For a connection whose server will not list its tools until the user signs in, the result tells the model that sign-in is needed: the connection appears under `unavailable` with `requiresSignIn: true` and an error that points to `signIn: true`. Tools from the other connections are still returned.
|
|
290
302
|
- `connection_search({ connection, signIn: true, query? })` asks the user to sign in to that one connection when they have not yet, then returns its matching tools. Without `connection` it fails, so the user is asked about one service at a time.
|
|
291
|
-
- `connection_execute({ connection, tool, input })` checks `input` against the tool's input schema, calls the tool, and returns its result. When the
|
|
303
|
+
- `connection_execute({ connection, tool, input })` checks `input` against the tool's input schema, calls the tool, and returns its result. When the server asks for the user's authorization, it also asks the user to sign in and the call parks until sign-in completes. The stream reports the call as a nested action named `<connection>__<tool>`, such as `linear__list_issues`, whose `parentCallId` is the `connection_execute` call id.
|
|
292
304
|
|
|
293
305
|
The definitions of both tools never change during a session. eve lists connection names and descriptions in append-only context messages rather than in the system prompt, so finding a tool, signing in, or resolving a dynamic connection keeps the cached prompt prefix.
|
|
294
306
|
|
|
@@ -127,11 +127,11 @@ A question or sign-in from inside a running call does not end the turn. This cov
|
|
|
127
127
|
|
|
128
128
|
### Task events
|
|
129
129
|
|
|
130
|
-
A call to a tool that runs as a task, such as an agent tool, `agentRouter()`, the `workflow` tool, or a workflow tool that defines [`task(input, ctx)`](/docs/tools/workflows#run-calls-as-tasks-task) or [`serve(receive, ctx)`](/docs/tools/workflows#resumable-tasks-serve), returns a receipt as its `action.result` and keeps working. `task.started` reports the task's `taskId` for the call's `callId` and `turnId`, both when a call starts a task and when a call reaches a resumable task by its `taskId`, before any event the task's run publishes for that call. `task.settled` comes once per call, with the same `turnId`, `name`, and `kind` as its `task.started`, `status` `"completed"`, `"failed"`, or `"cancelled"`, and the call's `output` when it completed or `error.message` when it failed. Events recorded by earlier eve versions omit `name` and `kind` on `task.settled`, so match them to their `task.started` by `callId` when those fields are missing. Results reach the model as a message in its history, not as a stream event, so read outcomes from `task.settled`. A session the task's run opens with `ctx.agent` is announced with `agent.started` carrying the `taskId`. An `input.requested`, `authorization.required`, or `authorization.completed` event from a task's run carries its `taskId`. Each of these events names the call the task is serving when it happens: a resumable task's later call brings its own `callId` and `turnId`, which may belong to a later turn than the call that started the task.
|
|
130
|
+
A call to a tool that runs as a task, such as an agent tool, `agentRouter()`, the `workflow` tool, or a workflow tool that defines [`task(input, ctx)`](/docs/tools/workflows#run-calls-as-tasks-task) or [`serve(receive, ctx)`](/docs/tools/workflows#resumable-tasks-serve), returns a receipt as its `action.result` and keeps working. `task.started` reports the task's `taskId` for the call's `callId` and `turnId`, both when a call starts a task and when a call reaches a resumable task by its `taskId`, before any event the task's run publishes for that call. `task.settled` comes once per call, with the same `turnId`, `name`, and `kind` as its `task.started`, `status` `"completed"`, `"failed"`, or `"cancelled"`, and the call's `output` when it completed or `error.message` when it failed. Events recorded by earlier eve versions omit `name` and `kind` on `task.settled`, so match them to their `task.started` by `callId` when those fields are missing. Results reach the model as a message in its history, not as a stream event, so read outcomes from `task.settled`. A session the task's run opens with `ctx.agent` is announced with `agent.started` carrying the `taskId`. An `input.requested`, `authorization.required`, or `authorization.completed` event from a task's run carries its `taskId`. Each of these events names the call the task is serving when it happens: a resumable task's later call brings its own `callId` and `turnId`, which may belong to a later turn than the call that started the task. An `input.requested` relayed from a subagent or a workflow tool run also carries `callId`, the call in this session that the request serves. The request keeps the coordinates it was raised with, and a relayed approval's `request.action.callId` still names the asking run's call, not one of this session's.
|
|
131
131
|
|
|
132
132
|
A turn doesn't end while tasks are working. When the model ends its text early, eve holds the turn and the stream emits `turn.waiting` for it; the turn continues once a result arrives or its caller writes. In a root session, that text completes as an ordinary `message.completed`. In a child session or a schedule's session, it reports `finishReason: "tool-calls"`, so channels post only the final reply. When the model calls `task_wait` and no result is ready yet, the stream also emits `turn.waiting`. Either way, the next `step.started` for the same `turnId` means the turn resumed, and `session.waiting` comes only after the turn ends. See [Tasks](/docs/tools/tasks#turns-wait-for-their-tasks).
|
|
133
133
|
|
|
134
|
-
`step.failed` and `turn.failed` carry `{ code, message, details? }` for the failed fragment or turn, and `session.failed` is the terminal session-level variant. `turn.cancelled` is not a failure: the cancelled turn ends without any failure event, `session.waiting` follows, and the session accepts the next message normally. Whatever the turn streamed before cancellation stays on the stream. Durable history keeps the accepted user input and previously settled work, and discards incomplete assistant output. Tool calls the cancellation stopped stay in history, each answered as cancelled, so the model sees that the work started and stopped instead of a request left unanswered. When a turn requested an output schema, the finalized payload lands on `result.completed` as `data.result` before the turn boundary. `authorization.required` carries the sign-in challenge (`data.authorization` may include `url`, `userCode`, `expiresAt`, `instructions`), and `authorization.completed` carries `data.outcome` (`"authorized" | "declined" | "failed" | "timed-out"`). Both carry `data.principalId` when a session principal started the sign-in. It holds the same value as `responderPrincipalId` on approval events, so a channel can deliver the challenge privately to that person even after someone else starts a later turn.
|
|
134
|
+
`step.failed` and `turn.failed` carry `{ code, message, details? }` for the failed fragment or turn, and `session.failed` is the terminal session-level variant. `turn.cancelled` is not a failure: the cancelled turn ends without any failure event, `session.waiting` follows, and the session accepts the next message normally. Whatever the turn streamed before cancellation stays on the stream. Durable history keeps the accepted user input and previously settled work, and discards incomplete assistant output. Tool calls the cancellation stopped stay in history, each answered as cancelled, so the model sees that the work started and stopped instead of a request left unanswered. Before `turn.cancelled`, the stream reports requests the cancel strands as `cancelled` in `input.resolved`. These include the turn's own questions, the session-limit prompt, and requests relayed from the work it stops. When a turn requested an output schema, the finalized payload lands on `result.completed` as `data.result` before the turn boundary. `authorization.required` carries the sign-in challenge (`data.authorization` may include `url`, `userCode`, `expiresAt`, `instructions`), and `authorization.completed` carries `data.outcome` (`"authorized" | "declined" | "failed" | "timed-out"`). Both carry `data.principalId` when a session principal started the sign-in. It holds the same value as `responderPrincipalId` on approval events, so a channel can deliver the challenge privately to that person even after someone else starts a later turn.
|
|
135
135
|
|
|
136
136
|
A provider response ending with `content-filter` fails with `MODEL_CALL_FAILED`,
|
|
137
137
|
`details.semanticErrorId: "model-response-content-filtered"`, and
|
|
@@ -261,7 +261,7 @@ curl -X POST http://127.0.0.1:2000/eve/v1/session/<sessionId>/reset \
|
|
|
261
261
|
|
|
262
262
|
Compaction summarizes context without adding a user message. User-role instructions are ordinary history and may be represented by the summary; system-role instructions remain outside it. Attributed [memory](../memory) records are excluded from the summary, canonicalized, and recalled again after the checkpoint. If a turn is active, eve queues the request until that turn settles. A successful compaction emits `compaction.requested` and `compaction.completed`, followed by `session.waiting`; if summarization fails before a checkpoint, the session returns to waiting with its previous history.
|
|
263
263
|
|
|
264
|
-
Clear removes model-message history in place, including static and dynamic user-role instructions and recalled memory records, while preserving the session identity, system-role instructions, tools, skills, application-defined durable state, limits, and sandbox. It clears framework memory locks and replay bookkeeping but does not delete data from a provider's external store. It does not rerun instruction definitions or resolvers. It emits `context.cleared` followed by `session.waiting`.
|
|
264
|
+
Clear removes model-message history in place, including static and dynamic user-role instructions and recalled memory records, while preserving the session identity, system-role instructions, tools, skills, application-defined durable state, limits, and sandbox. It clears framework memory locks and replay bookkeeping but does not delete data from a provider's external store. It does not rerun instruction definitions or resolvers. Approvals, the session-limit prompt, and sign-ins the cleared history asked for are withdrawn: each resolves as `cancelled` in `input.resolved` or `declined` in `authorization.completed`. Requests relayed from tasks that keep running stay answerable. It then emits `context.cleared` followed by `session.waiting`.
|
|
265
265
|
|
|
266
266
|
Reset terminally retires the exact session ID. A reset ID never becomes a new session; create another session explicitly for a fresh conversation. Compact, clear, and reset return `"no_active_session"` when the target is already inactive.
|
|
267
267
|
|
package/docs/connections/mcp.mdx
CHANGED
|
@@ -44,6 +44,17 @@ For OAuth-backed MCP servers, use the shared [Vercel Connect flow](../connection
|
|
|
44
44
|
|
|
45
45
|
`connect("...")` is user-scoped by default and requires an authenticated user on the active eve session. Use `connect({ connector, principalType: "app" })` when the server should act as the agent instead. The connections overview covers connector setup, session requirements, app and user scope, callback behavior, and troubleshooting.
|
|
46
46
|
|
|
47
|
+
## When eve asks for sign-in
|
|
48
|
+
|
|
49
|
+
eve asks the user to sign in only when the MCP server asks for it. When a connection has `auth` but no token yet, because its `getToken` throws `ConnectionAuthorizationRequiredError`, eve connects without an `Authorization` header and lists the server's tools. Any other error from `getToken` still fails the connection.
|
|
50
|
+
|
|
51
|
+
- **The server rejects every request without a token.** Most hosted OAuth servers, such as Linear, GitHub, and Notion, answer `initialize` or `tools/list` with HTTP `401`. A plain `connection_search` lists the connection under `unavailable` with `requiresSignIn: true`. Calling `connection_search({ connection, signIn: true })` or any of its tools with `connection_execute` asks the user to sign in.
|
|
52
|
+
- **The server lists tools without a token.** `connection_search` returns its tools with no prompt, and `connection_execute` runs tools the server answers anonymously. Sign-in starts only when a tool call comes back with HTTP `401`, or with an error result that carries a `WWW-Authenticate` challenge in `_meta["mcp/www_authenticate"]`.
|
|
53
|
+
|
|
54
|
+
After the user signs in, the call that asked for it runs again. eve reconnects with the token and reloads the tool list, which may now include tools the server hides from anonymous callers. A connection that already has a token sends it on every request, as before.
|
|
55
|
+
|
|
56
|
+
A server that leaves protected tools out of its anonymous `tools/list` never triggers sign-in, because the model cannot find those tools. Servers that support anonymous use should list every tool and refuse the protected calls.
|
|
57
|
+
|
|
47
58
|
## Static tokens and headers
|
|
48
59
|
|
|
49
60
|
MCP connections accept the shared connection `auth` and `headers` options. Use `auth.getToken` for a bearer token, `headers` for another scheme, and resolver functions when credentials or routing depend on the caller. See [Static-token auth](../connections#static-token-auth), [Headers](../connections#headers), and [Per-caller auth and headers](../connections#per-caller-auth-and-headers) for the canonical examples and token-lifecycle behavior.
|
|
@@ -194,7 +194,7 @@ export default defineMcpClientConnection({
|
|
|
194
194
|
});
|
|
195
195
|
```
|
|
196
196
|
|
|
197
|
-
`never()` lets every call through, `once()` asks for approval the first time in a session, `always()` asks every time, and `auto()` uses
|
|
197
|
+
`never()` lets every call through, `once()` asks for approval the first time in a session, `always()` asks every time, and `auto()` uses a decision model to allow routine calls and ask about dangerous or unclear effects. The pause and resume is the same human-in-the-loop flow covered in [Tools](/docs/tools).
|
|
198
198
|
|
|
199
199
|
For connection tools that can create, modify, delete, transmit, purchase, message, or access sensitive data, use approval, allow-lists, or other safeguards appropriate to the action.
|
|
200
200
|
|
|
@@ -227,7 +227,7 @@ export default defineMcpClientConnection({
|
|
|
227
227
|
|
|
228
228
|
Use the connector UID returned by `vercel connect create`; the display name passed to `--name` is not the UID. `connect("linear/myagent")` is shorthand for a user-scoped interactive OAuth connection: eve resolves a token for the active user before each tool call, emits `authorization.required` when that user has not authorized yet, and resumes the parked turn after the callback completes.
|
|
229
229
|
|
|
230
|
-
Discovering tools never posts a sign-in prompt for a service the answer does not need.
|
|
230
|
+
Discovering tools never posts a sign-in prompt for a service the answer does not need. Before the user signs in, eve connects to an MCP server without a token, so a server that lists its tools anonymously is searched like any other connection, and sign-in starts only when one of its tools asks for it (see [When eve asks for sign-in](/docs/connections/mcp#when-eve-asks-for-sign-in)). Most OAuth MCP servers reject even tool listing without the user's token. A plain `connection_search` lists those connections under `unavailable` with `requiresSignIn: true`, and the model asks for sign-in explicitly, one connection at a time, with `connection_search({ connection, signIn: true })`, which returns that connection's tools once the user signs in. Calling a tool with `connection_execute` on such a connection also asks for sign-in.
|
|
231
231
|
|
|
232
232
|
A sign-in holds the turn open until the user connects, the turn is cancelled, or the user sends another message. That message steers the turn: eve cancels the sign-in, reports `authorization.completed` with `outcome: "declined"`, and the model reads the message next. Messages from other people in a shared thread wait until the turn ends, and the completed sign-in resumes the turn as the person who started it.
|
|
233
233
|
|
package/docs/evals/judge.mdx
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: "Judge"
|
|
3
|
-
description: "Grade evals with
|
|
3
|
+
description: "Grade evals with decision models using criteria, typed questions, or batches, and set thresholds on each assertion."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
Use `t.judge(...)` when a deterministic [assertion](./assertions) cannot capture what a good response looks like. It calls [`
|
|
6
|
+
Use `t.judge(...)` when a deterministic [assertion](./assertions) cannot capture what a good response looks like. It calls [`decide` from `eve/ai`](../guides/decide#decide-inside-a-tool) and defaults to `typesafe-ai/jev`, TypeSafe AI's [Jev decision model](https://vercel.com/i/what-is-jev). The judge is separate from the agent under test and is used only for scoring.
|
|
7
7
|
|
|
8
8
|
```ts
|
|
9
9
|
import { defineEval } from "eve/evals";
|
|
@@ -17,7 +17,7 @@ export default defineEval({
|
|
|
17
17
|
});
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
-
No judge configuration is required. Configure credentials for the
|
|
20
|
+
No judge configuration is required. Configure credentials for the decider as described under [Configuring the judge model](#configuring-the-judge-model).
|
|
21
21
|
|
|
22
22
|
## The graders
|
|
23
23
|
|
|
@@ -41,11 +41,11 @@ t.judge({
|
|
|
41
41
|
.atLeast(0.75);
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
Choice questions require an `expected` option key. eve uses that expectation locally for scoring; it does not send it to the
|
|
44
|
+
Choice questions require an `expected` option key. eve uses that expectation locally for scoring; it does not send it to the decider. Instructions and rubric descriptions accept strings, JSON objects, or JSON arrays. Descriptions can also be `null`.
|
|
45
45
|
|
|
46
46
|
## Choose the value to grade
|
|
47
47
|
|
|
48
|
-
By default, the
|
|
48
|
+
By default, the decider receives `{ input, output }`: the latest prompt and assistant message from the session whose turn most recently settled. Approval responses and stream reads preserve that prompt. For multipart messages, including `session.sendFile()`, the prompt contains the text parts.
|
|
49
49
|
|
|
50
50
|
Pass `on` to replace the output. JSON values stay structured rather than being converted to strings:
|
|
51
51
|
|
|
@@ -66,9 +66,9 @@ t.judge("The assistant remembers the user's favorite word across turns.", {
|
|
|
66
66
|
|
|
67
67
|
The transcript contains user and assistant messages in turn order, excluding reasoning, tool calls, and tool results. See [Multi-turn evals](./cases#multi-turn-evals).
|
|
68
68
|
|
|
69
|
-
##
|
|
69
|
+
## Decide several questions together
|
|
70
70
|
|
|
71
|
-
A batch makes one
|
|
71
|
+
A batch makes one decision request against one shared state. Pass `state` to replace the default `{ input, output }` completely. Batch calls use `state` instead of `on`.
|
|
72
72
|
|
|
73
73
|
```ts
|
|
74
74
|
const judgments = t.judge({
|
|
@@ -120,9 +120,9 @@ Model settings resolve in this order:
|
|
|
120
120
|
1. Per-call: `t.judge("…", { model, modelOptions })`.
|
|
121
121
|
2. Per-eval: `defineEval({ judge: { model, modelOptions }, test })`.
|
|
122
122
|
3. Project default: `defineEvalConfig({ judge: { model, modelOptions } })`.
|
|
123
|
-
4. The default model used by `
|
|
123
|
+
4. The default model used by `decide`, currently `typesafe-ai/jev`.
|
|
124
124
|
|
|
125
|
-
A per-eval `judge` replaces the project's judge configuration. Per-call fields override the resolved configuration. `judge` and its `model` are optional; deterministic evals make no
|
|
125
|
+
A per-eval `judge` replaces the project's judge configuration. Per-call fields override the resolved configuration. `judge` and its `model` are optional; deterministic evals make no decision requests.
|
|
126
126
|
|
|
127
127
|
```ts title="evals/evals.config.ts"
|
|
128
128
|
import { defineEvalConfig } from "eve/evals";
|
|
@@ -132,30 +132,30 @@ export default defineEvalConfig({
|
|
|
132
132
|
});
|
|
133
133
|
```
|
|
134
134
|
|
|
135
|
-
String IDs resolve through the AI SDK default provider, or Vercel AI Gateway when none is configured. Gateway uses `AI_GATEWAY_API_KEY` or Vercel OIDC credentials; eve's existing local Gateway connection is also honored by `
|
|
135
|
+
String IDs resolve through the AI SDK default provider, or Vercel AI Gateway when none is configured. Gateway uses `AI_GATEWAY_API_KEY` or Vercel OIDC credentials; eve's existing local Gateway connection is also honored by `decide`. An explicit provider decision model uses that provider's credentials:
|
|
136
136
|
|
|
137
137
|
```ts
|
|
138
138
|
import { openai } from "@ai-sdk/openai";
|
|
139
139
|
|
|
140
140
|
t.judge("The response addresses the request.", {
|
|
141
|
-
model: openai.
|
|
141
|
+
model: openai.decisionModel("gpt-6-luna"),
|
|
142
142
|
modelOptions: { providerOptions: { openai: { reasoningEffort: "high" } } },
|
|
143
143
|
});
|
|
144
144
|
```
|
|
145
145
|
|
|
146
|
-
Use the provider's `
|
|
146
|
+
Use the provider's `decisionModel` factory, not a language model instance. AI SDK supplies decision adapters for supported language models. Provider support and the decision contract are experimental; see [AI SDK decision](https://ai-sdk.dev/docs/ai-sdk-core/decisions).
|
|
147
147
|
|
|
148
|
-
String model IDs resolve through the default provider's
|
|
148
|
+
String model IDs resolve through the default provider's decision API. Gateway accepts native decision models such as `typesafe-ai/jev`; passing a language model ID such as `openai/gpt-6-luna` does not automatically select the OpenAI adapter. Use a decision-model instance as shown above for language models.
|
|
149
149
|
|
|
150
|
-
Missing credentials, invalid answers, unsupported question types, and provider errors become failed gates even for tracked-only judgments. They do not silently skip the eval or select another model. Validation, authentication, and retries use the shared `
|
|
150
|
+
Missing credentials, invalid answers, unsupported question types, and provider errors become failed gates even for tracked-only judgments. They do not silently skip the eval or select another model. Validation, authentication, and retries use the shared `decide` implementation.
|
|
151
151
|
|
|
152
152
|
## Diagnostics and migration
|
|
153
153
|
|
|
154
|
-
Reports use `judge.boolean`, `judge.score`, and `judge.choice`; batch assertion names also include their question key. Diagnostics include the captured state, question, and raw answer. Assertion metadata retains the normalized score, model identity, token usage, warnings, and available provider and response metadata. Usage for a batch is marked as shared across its assertions, not a separate charge for each question.
|
|
154
|
+
Reports use `judge.boolean`, `judge.score`, and `judge.choice`; batch assertion names also include their question key. Diagnostics include the captured state, question, and raw answer. Assertion metadata retains the normalized score, model identity, token usage, warnings, and available provider and response metadata. Usage for a batch is marked as shared across its assertions, not a separate charge for each question. Decision models do not promise a prose rationale.
|
|
155
155
|
|
|
156
156
|
The autoevals graders have been removed. Replace `t.judge.autoevals.closedQA(criteria, options)` with `t.judge(criteria, options)`. Scores now represent a probability rather than a binary yes/no answer, so review thresholds against your examples. For factuality, summaries, or SQL equivalence, write an explicit question and include the reference in shared state. No old grader buckets are preserved.
|
|
157
157
|
|
|
158
|
-
Change configured language model instances to
|
|
158
|
+
Change configured language model instances to decision model instances. `modelOptions.providerOptions`, assertion labels, and threshold methods remain available. [Braintrust reporting](./reporters) continues to accept judge scores independently of the grader implementation.
|
|
159
159
|
|
|
160
160
|
## What to read next
|
|
161
161
|
|
package/docs/evals/overview.mdx
CHANGED
|
@@ -56,7 +56,7 @@ export default defineEvalConfig({
|
|
|
56
56
|
});
|
|
57
57
|
```
|
|
58
58
|
|
|
59
|
-
Everything is optional. `judge` sets the default model for [LLM-as-judge](./judge) assertions (`t.judge(...)`); omit it to use the default
|
|
59
|
+
Everything is optional. `judge` sets the default model for [LLM-as-judge](./judge) assertions (`t.judge(...)`); omit it to use the default decider, `typesafe-ai/jev`. Deterministic evals make no judge requests. `reporters`, `maxConcurrency`, and `timeoutMs` round out the defaults. Config `reporters` observe every eval in the run, so set one `Braintrust()` here instead of adding it to each eval. CLI flags (`--max-concurrency`, `--timeout`) and per-eval values take precedence over the config defaults.
|
|
60
60
|
|
|
61
61
|
### Run setup and teardown
|
|
62
62
|
|
|
@@ -1,21 +1,21 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Automatic Model Selection
|
|
3
|
-
description: "Choose agent models automatically or
|
|
3
|
+
description: "Choose agent models automatically or ask typed questions in your tools and application code."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
Use `auto` from `eve/models` to choose an agent model from an allowlist before
|
|
7
|
-
inference begins. It uses the [AI SDK
|
|
8
|
-
so the
|
|
9
|
-
an installed provider. Use `
|
|
7
|
+
inference begins. It uses the [AI SDK decision API](https://ai-sdk.dev/docs/ai-sdk-core/decisions),
|
|
8
|
+
so the decider can be a Vercel AI Gateway model ID or a decision model from
|
|
9
|
+
an installed provider. Use `decide` from `eve/ai` to ask typed questions in
|
|
10
10
|
your own tools or application code.
|
|
11
11
|
|
|
12
|
-
The AI SDK
|
|
12
|
+
The AI SDK decision model specification is experimental and can change in
|
|
13
13
|
patch releases.
|
|
14
14
|
|
|
15
15
|
## Choose from Gateway models
|
|
16
16
|
|
|
17
|
-
By default, `auto`
|
|
18
|
-
[Jev
|
|
17
|
+
By default, `auto` decides with `typesafe-ai/jev`, TypeSafe AI's
|
|
18
|
+
[Jev decision model](https://vercel.com/i/what-is-jev). Like other AI SDK
|
|
19
19
|
model strings, it uses Vercel AI Gateway unless the application has configured a
|
|
20
20
|
different global default provider.
|
|
21
21
|
|
|
@@ -35,13 +35,13 @@ export default defineAgent({
|
|
|
35
35
|
|
|
36
36
|
Configure Gateway authentication as you would for any other AI SDK model. eve
|
|
37
37
|
does not add a TypeSafe credential or transport layer. During `eve dev`, a
|
|
38
|
-
Gateway
|
|
38
|
+
Gateway decider uses the same connection selected through `/login` as Gateway
|
|
39
39
|
language models. A configured AI SDK default provider still owns string model
|
|
40
40
|
resolution during development. The TUI footer displays `dynamic model` when the
|
|
41
41
|
agent uses `auto`, then adds the resolved model for the current turn, such as
|
|
42
42
|
`dynamic model · openai/gpt-6-luna`.
|
|
43
43
|
|
|
44
|
-
Set `fallback` to keep the turn running when the
|
|
44
|
+
Set `fallback` to keep the turn running when the decider is unavailable or
|
|
45
45
|
returns an error. The fallback can be a Gateway model ID, an AI SDK language
|
|
46
46
|
model instance, or an object with `model` and a `reasoning` override:
|
|
47
47
|
|
|
@@ -60,15 +60,15 @@ export default defineAgent({
|
|
|
60
60
|
});
|
|
61
61
|
```
|
|
62
62
|
|
|
63
|
-
The fallback applies only to
|
|
64
|
-
|
|
63
|
+
The fallback applies only to decider failures. eve logs a warning with the
|
|
64
|
+
decider error and fallback identity when it switches models. Invalid prompts,
|
|
65
65
|
including requests without user text or with a latest message over the routing
|
|
66
|
-
limit, still fail before
|
|
66
|
+
limit, still fail before decider I/O. Cancelling the turn also aborts routing
|
|
67
67
|
instead of selecting the fallback.
|
|
68
68
|
|
|
69
69
|
## Use a provider directly
|
|
70
70
|
|
|
71
|
-
Install the provider package yourself and pass its
|
|
71
|
+
Install the provider package yourself and pass its decision model. The provider
|
|
72
72
|
owns its credentials and settings.
|
|
73
73
|
|
|
74
74
|
```sh
|
|
@@ -82,7 +82,7 @@ import { auto } from "eve/models";
|
|
|
82
82
|
|
|
83
83
|
export default defineAgent({
|
|
84
84
|
model: auto({
|
|
85
|
-
model: typeSafeAi.
|
|
85
|
+
model: typeSafeAi.decisionModel("jev-latest"),
|
|
86
86
|
options: {
|
|
87
87
|
"openai/gpt-6-sol": "Difficult reasoning and engineering tasks",
|
|
88
88
|
"openai/gpt-6-luna": "Routine tasks where fast completion matters",
|
|
@@ -91,12 +91,12 @@ export default defineAgent({
|
|
|
91
91
|
});
|
|
92
92
|
```
|
|
93
93
|
|
|
94
|
-
Any provider that implements the AI SDK `
|
|
94
|
+
Any provider that implements the AI SDK `Experimental_DecisionModel` contract
|
|
95
95
|
works here.
|
|
96
96
|
|
|
97
97
|
## Route to provider models and set reasoning
|
|
98
98
|
|
|
99
|
-
An option's key is the value shown to the
|
|
99
|
+
An option's key is the value shown to the decider. A string value describes a
|
|
100
100
|
Gateway model whose ID is the key. Use the object form when the selected model is
|
|
101
101
|
a provider instance, an alias, or needs a reasoning override.
|
|
102
102
|
|
|
@@ -120,7 +120,7 @@ export default defineAgent({
|
|
|
120
120
|
});
|
|
121
121
|
```
|
|
122
122
|
|
|
123
|
-
The
|
|
123
|
+
The decider sees option keys, descriptions, and recent text messages. It never
|
|
124
124
|
receives provider credentials or serialized language model instances. When it
|
|
125
125
|
selects `my_secret_model`, eve resolves the key back to the authored Anthropic
|
|
126
126
|
model.
|
|
@@ -129,16 +129,16 @@ Supported reasoning values are `"provider-default"`, `"none"`, `"minimal"`,
|
|
|
129
129
|
`"low"`, `"medium"`, `"high"`, and `"xhigh"`. An omitted value inherits the
|
|
130
130
|
agent's reasoning setting.
|
|
131
131
|
|
|
132
|
-
##
|
|
132
|
+
## Decide inside a tool
|
|
133
133
|
|
|
134
|
-
Use `
|
|
134
|
+
Use `decide` to ask choice, score, or boolean questions about the state you pass to it.
|
|
135
135
|
It defaults to `typesafe-ai/jev` and uses the same authentication as `auto`,
|
|
136
136
|
including the Gateway connection selected through `/login` during `eve dev`.
|
|
137
|
-
Pass `model` to use another
|
|
137
|
+
Pass `model` to use another decision model ID or a provider instance. A configured
|
|
138
138
|
AI SDK default provider takes precedence over the local Gateway connection.
|
|
139
139
|
|
|
140
140
|
```ts title="agent/tools/classify-request.ts"
|
|
141
|
-
import {
|
|
141
|
+
import { decide } from "eve/ai";
|
|
142
142
|
import { defineTool } from "eve/tools";
|
|
143
143
|
import { z } from "zod";
|
|
144
144
|
|
|
@@ -146,7 +146,7 @@ export default defineTool({
|
|
|
146
146
|
description: "Choose the team that can help with a customer request.",
|
|
147
147
|
inputSchema: z.object({ request: z.string().min(1).max(8000) }),
|
|
148
148
|
async execute({ request }, ctx) {
|
|
149
|
-
const result = await
|
|
149
|
+
const result = await decide({
|
|
150
150
|
state: { request },
|
|
151
151
|
questions: {
|
|
152
152
|
team: {
|
|
@@ -169,23 +169,23 @@ The choice above is typed as `"billing" | "support"`. Each question appears unde
|
|
|
169
169
|
its authored key in `result.answers`. Results also include token usage, warnings,
|
|
170
170
|
provider metadata, and response metadata. To use that choice to delegate while keeping specialist subagents out of the parent model's tools, see [Route to a hidden subagent with Jev](/docs/tools/workflows#route-to-a-hidden-subagent-with-jev).
|
|
171
171
|
|
|
172
|
-
`
|
|
172
|
+
`decide` accepts AI SDK decision options, including `maxRetries`, `headers`,
|
|
173
173
|
and `providerOptions`. Pass an `abortSignal` to cancel the request. Input and
|
|
174
174
|
answer validation, retries, and provider errors follow AI SDK semantics.
|
|
175
175
|
|
|
176
|
-
You can also call `
|
|
177
|
-
session. Each call performs its own
|
|
176
|
+
You can also call `decide` outside a tool; it does not require an active eve
|
|
177
|
+
session. Each call performs its own decision. `auto` uses this function
|
|
178
178
|
and adds the per-turn routing behavior described below.
|
|
179
179
|
|
|
180
180
|
## Judge eval results
|
|
181
181
|
|
|
182
|
-
Eval authors can use `t.judge(...)` to turn
|
|
182
|
+
Eval authors can use `t.judge(...)` to turn decision answers into scored assertions, including batches of questions sharing one state. It uses this same `decide` implementation and default model. See [Judge](../evals/judge) for criteria, rubrics, and thresholds.
|
|
183
183
|
|
|
184
|
-
##
|
|
184
|
+
## Decide tool approvals
|
|
185
185
|
|
|
186
|
-
Use `auto({ model? })` when
|
|
186
|
+
Use `auto({ model? })` when a decision model should decide whether a
|
|
187
187
|
tool call can run automatically or needs human approval. It accepts the same
|
|
188
|
-
AI SDK
|
|
188
|
+
AI SDK decision model strings and provider instances described above and
|
|
189
189
|
defaults to `typesafe-ai/jev`:
|
|
190
190
|
|
|
191
191
|
```ts title="agent/tools/deploy.ts"
|
|
@@ -201,7 +201,7 @@ export default defineTool({
|
|
|
201
201
|
});
|
|
202
202
|
```
|
|
203
203
|
|
|
204
|
-
The
|
|
204
|
+
The decision model reviews the tool name and input for dangerous effects. A
|
|
205
205
|
caution, failed review, or incomplete input requires human approval. See
|
|
206
206
|
[Human-in-the-loop approvals](/docs/human-in-the-loop#approvals) for classifier
|
|
207
207
|
options and data handling, and the
|
|
@@ -210,17 +210,17 @@ guide for an end-to-end walkthrough.
|
|
|
210
210
|
|
|
211
211
|
## Runtime behavior
|
|
212
212
|
|
|
213
|
-
`auto`
|
|
213
|
+
`auto` decides at the first `step.started` event, after the incoming prompt
|
|
214
214
|
is available and before the selected language model runs. It reuses that choice
|
|
215
215
|
for later tool-loop steps in the same turn. A new turn makes a new choice, and
|
|
216
216
|
child sessions route from their own prompts.
|
|
217
217
|
|
|
218
|
-
The
|
|
218
|
+
The decider receives up to eight recent user and assistant text messages,
|
|
219
219
|
capped at 16,000 characters. Requests without user text and latest messages over
|
|
220
220
|
the limit fail before provider I/O.
|
|
221
221
|
|
|
222
|
-
|
|
223
|
-
SDK semantics. Without `fallback`,
|
|
222
|
+
Decision validation, retries, provider errors, and model resolution follow AI
|
|
223
|
+
SDK semantics. Without `fallback`, a decider error fails the turn. With
|
|
224
224
|
`fallback`, eve uses and retains that model for the rest of the turn. Cancelling
|
|
225
|
-
the active turn aborts
|
|
225
|
+
the active turn aborts decision and prevents a routed or fallback choice from
|
|
226
226
|
being retained.
|
|
@@ -362,7 +362,7 @@ A dynamic connection, tool, or skill whose name matches an **authored** one **ov
|
|
|
362
362
|
|
|
363
363
|
At `turn.started`, model, tool, skill, and subagent resolvers receive the visible conversation history and incoming message in `ctx.messages`, oldest first. Request context is included, and history projection still applies. Read these messages from the handler's second argument; the event itself contains turn metadata. Instruction resolvers use the separate snapshot described under [Dynamic instructions](#dynamic-instructions).
|
|
364
364
|
|
|
365
|
-
A message that steers a held turn
|
|
365
|
+
A message that steers a held turn joins it without another `turn.started`, as does a completed sign-in, so `turn.started` resolvers do not see it; `step.started` resolvers do. A message sent while a session-limit prompt waits starts its own turn, so `turn.started` resolvers see it, and that turn waits for the prompt's answer before the model runs. When memory recall runs, its results appear in the projected snapshot before incoming input.
|
|
366
366
|
|
|
367
367
|
### Execution order
|
|
368
368
|
|
|
@@ -174,7 +174,9 @@ supplied `session` remains externally owned and is never replaced by prewarming
|
|
|
174
174
|
|
|
175
175
|
Common text, reasoning, file, and dynamic-tool parts follow the [AI SDK `UIMessage`](https://ai-sdk.dev/docs/reference/ai-sdk-core/ui-message) rendering convention, but the types are not interchangeable. eve also exposes authorization and HITL metadata, and a file part's URL can be absent. Adapt those parts before passing messages to an API typed as `UIMessage[]`.
|
|
176
176
|
|
|
177
|
-
|
|
177
|
+
A tool part's `state` comes from the shared lifecycle interpretation of the event stream. Its events supply input, output, and error text. A call waiting on an approval or a question reads `approval-requested` until the stream settles the request. An answer you submit doesn't change the part until then. `toolCallState(conversation, callId)` reads tool-call status, including task cancellation and calls interrupted without a result. The full event projection remains internal; `ConversationState` retains its messages, turns, inputs, tasks, and agent sessions.
|
|
178
|
+
|
|
179
|
+
When the root agent delegates, its stream emits `task.started` for the call, `agent.started` with the child's `sessionId`, and `task.settled` with the call's result, failure, or cancellation. The default reducer keeps the subagent's tool part running until `task.settled` arrives and then settles it with the call's outcome; the start receipt never settles it. `data.tasks[taskId].calls[callId]` records each call's `status`, `output`, and `error`, and `data.agents[sessionId]` records each session a call opened, with its `taskId` and opening `callId`. These records come from the root stream alone.
|
|
178
180
|
|
|
179
181
|
Detailed child progress lives on the child session's stream, not in the root `data.messages`. Pass `followSubagents: true` to follow the session each subagent call reaches while the component is mounted; following is off by default. Narrow `data.agents[sessionId].observation.status` before reading its `conversation`: `"not-followed"` has no child detail, `"following"` has live detail, and `"idle"` has everything the calls so far produced and resumes when another call reaches the task. `"unavailable"` means the stream failed, not that the child failed. Calls that continue a task with its `taskId` reach the same session, so one session's `conversation.messages`, `inputs`, and `turns` span all of them. Sessions that a custom task opens with `ctx.agent` are recorded but not followed.
|
|
180
182
|
|
package/docs/guides/hooks.md
CHANGED
|
@@ -227,7 +227,7 @@ The remaining subscribers for the event still run. Then eve cancels the turn the
|
|
|
227
227
|
|
|
228
228
|
Subagents may carry their own `agent/hooks/` directory. Subagent hooks fire only inside the subagent scope. Parent-agent hooks do not fire for subagent turns, and subagent hooks see only the subagent's own context.
|
|
229
229
|
|
|
230
|
-
Interactive events such as `input.requested` and `authorization.required` are also published on the parent stream. Parent hooks observe these events after the parent channel handler and stream write, with the parent's session, agent, and channel context. The event retains the child's turn coordinates, so `event.data.turnId` can differ from `ctx.session.turn.id`. The parent follows a proxied `input.requested` or `authorization.required` with `turn.waiting` for its own open turn, which stays open until the running call finishes. A proxied `authorization.completed` is not followed by a parent turn event. These parent events also invoke parent hooks; they do not resolve pending input requests.
|
|
230
|
+
Interactive events such as `input.requested` and `authorization.required` are also published on the parent stream. Parent hooks observe these events after the parent channel handler and stream write, with the parent's session, agent, and channel context. The event retains the child's turn coordinates, so `event.data.turnId` can differ from `ctx.session.turn.id`. A proxied `input.requested` also carries `event.data.callId`, the parent call that waits on it. The parent follows a proxied `input.requested` or `authorization.required` with `turn.waiting` for its own open turn, which stays open until the running call finishes. A proxied `authorization.completed` is not followed by a parent turn event. These parent events also invoke parent hooks; they do not resolve pending input requests.
|
|
231
231
|
|
|
232
232
|
## Hook vs tool vs provider
|
|
233
233
|
|
|
@@ -23,7 +23,7 @@ agent/instrumentation/
|
|
|
23
23
|
audit.ts lifecycle instrumentation
|
|
24
24
|
```
|
|
25
25
|
|
|
26
|
-
This instrumentation records
|
|
26
|
+
This instrumentation records tool timing and identity without receiving tool
|
|
27
27
|
arguments or results:
|
|
28
28
|
|
|
29
29
|
```ts title="agent/instrumentation/audit.ts"
|
|
@@ -31,15 +31,15 @@ import { defineInstrumentation } from "eve/instrumentation";
|
|
|
31
31
|
|
|
32
32
|
export default defineInstrumentation({
|
|
33
33
|
events: {
|
|
34
|
-
"
|
|
35
|
-
ctx.state.set({ name: event.
|
|
34
|
+
"tool.call.started": (event, ctx) => {
|
|
35
|
+
ctx.state.set({ name: event.toolName, startedAt: Date.now() });
|
|
36
36
|
},
|
|
37
|
-
"
|
|
37
|
+
"tool.call.completed": (event, ctx) => {
|
|
38
38
|
const started = ctx.state.get() as { name: string; startedAt: number } | undefined;
|
|
39
39
|
if (started === undefined) return;
|
|
40
40
|
|
|
41
41
|
console.log({
|
|
42
|
-
|
|
42
|
+
tool: started.name,
|
|
43
43
|
durationMs: Date.now() - started.startedAt,
|
|
44
44
|
outcome: event.outcome,
|
|
45
45
|
});
|
|
@@ -123,12 +123,12 @@ export default defineInstrumentation({
|
|
|
123
123
|
recordOutputs: false,
|
|
124
124
|
}),
|
|
125
125
|
events: {
|
|
126
|
-
"
|
|
126
|
+
"tool.call.started": async (event) => {
|
|
127
127
|
await sendAuditRecord({
|
|
128
128
|
id: event.idempotencyKey,
|
|
129
129
|
input: redactApiKey(event.input),
|
|
130
130
|
kind: event.kind,
|
|
131
|
-
name: event.
|
|
131
|
+
name: event.toolName,
|
|
132
132
|
});
|
|
133
133
|
},
|
|
134
134
|
},
|
|
@@ -155,21 +155,22 @@ Prefer `recordInputs: false` or `recordOutputs: false` when the destination does
|
|
|
155
155
|
start and its terminal event share an `idempotencyKey`; use it as a stable
|
|
156
156
|
operation key when upserting records across replays.
|
|
157
157
|
|
|
158
|
-
| Lifecycle | Events | Meaning
|
|
159
|
-
| ---------------- | ----------------------------------------------------------------------------------------------------------------- |
|
|
160
|
-
| Session | `session.started`, `session.waiting`, `session.completed`, `session.failed` | Track a session from creation through suspension or termination. `session.waiting` is resumable; `session.completed` and `session.failed` are terminal.
|
|
161
|
-
| Turn | `turn.started`, `turn.completed`, `turn.cancelled`, `turn.failed` | Track one turn. Cancellation is not a failure and commonly leaves the session waiting for another turn.
|
|
162
|
-
| Channel delivery | `channel.delivery.started`, `channel.delivery.completed`, `channel.delivery.cancelled`, `channel.delivery.failed` | Track an inbound channel delivery while eve processes its turn. These events are emitted when the channel provides delivery metadata.
|
|
163
|
-
| Step attempt | `step.attempt.started`, `step.attempt.completed`, `step.attempt.failed`, `step.attempt.metadata` | Track each model-step attempt; retries have separate attempts. `step.attempt.metadata` is emitted after a step only when the AI SDK supplies provider metadata.
|
|
164
|
-
| Model call | `model.call.started`, `model.call.completed`, `model.call.failed` | Track an individual call to the model provider. Completion includes usage and finish reason; content fields follow the trace policy.
|
|
165
|
-
|
|
|
166
|
-
|
|
|
167
|
-
|
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
158
|
+
| Lifecycle | Events | Meaning |
|
|
159
|
+
| ---------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
160
|
+
| Session | `session.started`, `session.waiting`, `session.completed`, `session.failed` | Track a session from creation through suspension or termination. `session.waiting` is resumable; `session.completed` and `session.failed` are terminal. |
|
|
161
|
+
| Turn | `turn.started`, `turn.completed`, `turn.cancelled`, `turn.failed` | Track one turn. Cancellation is not a failure and commonly leaves the session waiting for another turn. |
|
|
162
|
+
| Channel delivery | `channel.delivery.started`, `channel.delivery.completed`, `channel.delivery.cancelled`, `channel.delivery.failed` | Track an inbound channel delivery while eve processes its turn. These events are emitted when the channel provides delivery metadata. |
|
|
163
|
+
| Step attempt | `step.attempt.started`, `step.attempt.completed`, `step.attempt.failed`, `step.attempt.metadata` | Track each model-step attempt; retries have separate attempts. `step.attempt.metadata` is emitted after a step only when the AI SDK supplies provider metadata. |
|
|
164
|
+
| Model call | `model.call.started`, `model.call.completed`, `model.call.failed` | Track an individual call to the model provider. Completion includes usage and finish reason; content fields follow the trace policy. |
|
|
165
|
+
| Tool call | `tool.call.started`, `tool.call.completed`, `tool.call.failed` | Track a tool, skill load, local subagent, or remote-agent call through durable settlement. A failed event's `outcome` distinguishes failure, rejection, cancellation, and abandonment. |
|
|
166
|
+
| User input | `input.requested`, `input.resolved` | Track a question, tool approval, or session-limit request and its resolution. The resolution includes an outcome such as `answered`, `approved`, `denied`, or `cancelled`. |
|
|
167
|
+
| Memory operation | `memory.operation.started`, `memory.operation.completed`, `memory.operation.failed` | Track a memory-store search or write. Events include the operation name, phase, and memory slot; completion may include a record count and records. |
|
|
168
|
+
|
|
169
|
+
Each call emits one `tool.call.*` lifecycle to authored providers. SDK execution callbacks do not emit a second lifecycle. Tool calls can remain open across approvals, parked turns, and worker replacement.
|
|
170
|
+
|
|
171
|
+
For local and remote subagents, `tool.call.completed` and `tool.call.failed` include the child turn's `usage` directly. Token counts are always present when the child reports usage; `usage.costUsd` is present only when the model provider reports a price.
|
|
172
|
+
|
|
173
|
+
When upgrading, replace `action.started`, `action.completed`, and `action.failed` handlers with their `tool.call.*` equivalents. Read `event.toolName` instead of `event.name` on start events. The OpenTelemetry trace schema remains version 4.
|
|
173
174
|
|
|
174
175
|
Handlers from different files run concurrently and are failure-isolated. Do
|
|
175
176
|
not depend on execution order. Use `flush` to drain buffered records and
|
|
@@ -126,6 +126,10 @@ The resolver receives the channel, session, final model input, step, and turn.
|
|
|
126
126
|
Keys beginning with `eve.` are reserved. Content excluded by
|
|
127
127
|
`otel({ tracePolicy })` is not available to the resolver.
|
|
128
128
|
|
|
129
|
+
Authored values appear under `ai.settings.context.*`. Framework channel, session,
|
|
130
|
+
step, turn, and version identifiers use structural span attributes instead of
|
|
131
|
+
runtime-context copies. `ai.settings.context.eve.environment` remains available.
|
|
132
|
+
|
|
129
133
|
## Trace topology
|
|
130
134
|
|
|
131
135
|
OpenTelemetry destinations receive the following trace topology for an ordinary
|
|
@@ -135,8 +139,7 @@ agent turn:
|
|
|
135
139
|
invoke_agent <agent>
|
|
136
140
|
└── agent.step
|
|
137
141
|
├── chat <model>
|
|
138
|
-
└──
|
|
139
|
-
└── execute_tool <tool>
|
|
142
|
+
└── execute_tool <tool>
|
|
140
143
|
```
|
|
141
144
|
|
|
142
145
|
Each root agent turn starts a new trace. If eve receives a local caller's trace
|
|
@@ -146,6 +149,14 @@ context, the first subagent turn starts a new trace. Later turns in the child
|
|
|
146
149
|
session start new traces. They do not use the original dispatch span as their
|
|
147
150
|
parent.
|
|
148
151
|
|
|
152
|
+
Use `gen_ai.tool.call.id` to correlate tool and approval spans. Tool failures use
|
|
153
|
+
`error.type`. These attributes replace the duplicate `agent.action.call_id` and
|
|
154
|
+
`agent.action.error.code` fields without changing trace schema version 4.
|
|
155
|
+
|
|
156
|
+
Model and activation spans record token counts under `gen_ai.usage.*`, without
|
|
157
|
+
duplicate `agent.usage.*` token fields. Step and tool spans retain `agent.usage.*`.
|
|
158
|
+
Cost values remain unchanged.
|
|
159
|
+
|
|
149
160
|
A remote child starts a separate trace with a root `invoke_agent` span. Its first
|
|
150
161
|
turn links to the caller with `eve.link.type=agent.dispatch`. The receiving
|
|
151
162
|
deployment applies its sampler. Queries for local dispatches must use
|