eve 0.6.0-beta.8 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +278 -0
- package/README.md +9 -6
- package/dist/docs/public/README.md +1 -1
- package/dist/docs/public/advanced/auth-and-route-protection.md +3 -1
- package/dist/docs/public/advanced/default-harness.md +2 -2
- package/dist/docs/public/advanced/dev-tui.md +9 -11
- package/dist/docs/public/advanced/instrumentation.md +1 -1
- package/dist/docs/public/advanced/meta.json +0 -1
- package/dist/docs/public/advanced/security-model.md +1 -1
- package/dist/docs/public/advanced/session-context.md +1 -1
- package/dist/docs/public/agent-config.md +6 -3
- package/dist/docs/public/channels/eve.mdx +1 -1
- package/dist/docs/public/channels/slack.mdx +14 -2
- package/dist/docs/public/connections.mdx +3 -1
- package/dist/docs/public/evals/assertions.mdx +108 -0
- package/dist/docs/public/evals/cases.mdx +143 -0
- package/dist/docs/public/evals/judge.mdx +94 -0
- package/dist/docs/public/evals/meta.json +4 -0
- package/dist/docs/public/evals/overview.mdx +118 -0
- package/dist/docs/public/evals/reporters.mdx +62 -0
- package/dist/docs/public/evals/running.mdx +63 -0
- package/dist/docs/public/evals/targets.mdx +54 -0
- package/dist/docs/public/frontend/nextjs.mdx +13 -4
- package/dist/docs/public/frontend/nuxt.mdx +4 -4
- package/dist/docs/public/frontend/sveltekit.mdx +2 -2
- package/dist/docs/public/getting-started.mdx +25 -13
- package/dist/docs/public/meta.json +1 -0
- package/dist/docs/public/reference/cli.md +31 -9
- package/dist/docs/public/reference/typescript-api.md +6 -2
- package/dist/docs/public/sandbox.mdx +3 -1
- package/dist/docs/public/tools.mdx +0 -4
- package/dist/docs/public/tutorial/connect-a-warehouse.mdx +1 -1
- package/dist/docs/public/tutorial/first-agent.mdx +5 -2
- package/dist/src/chunks/{use-eve-agent-DCZbkLG7.js → use-eve-agent-DErQj5hs.js} +125 -37
- package/dist/src/chunks/{use-eve-agent-DoheC4_o.js → use-eve-agent-DoR8C4i6.js} +125 -37
- package/dist/src/cli/banner.d.ts +7 -0
- package/dist/src/cli/banner.js +1 -0
- package/dist/src/cli/commands/channel-add-conflicts.d.ts +1 -1
- package/dist/src/cli/commands/channels.d.ts +9 -6
- package/dist/src/cli/commands/channels.js +1 -1
- package/dist/src/cli/commands/deploy.d.ts +21 -0
- package/dist/src/cli/commands/deploy.js +1 -0
- package/dist/src/cli/commands/init-git.d.ts +15 -0
- package/dist/src/cli/commands/init-git.js +1 -0
- package/dist/src/cli/commands/init.d.ts +29 -0
- package/dist/src/cli/commands/init.js +1 -0
- package/dist/src/cli/commands/link.d.ts +21 -0
- package/dist/src/cli/commands/link.js +1 -0
- package/dist/src/cli/commands/preconditions.d.ts +7 -0
- package/dist/src/cli/commands/preconditions.js +1 -0
- package/dist/src/cli/commands/register-project-commands.d.ts +12 -0
- package/dist/src/cli/commands/register-project-commands.js +1 -0
- package/dist/src/cli/dev/tui/agent-header.d.ts +15 -9
- package/dist/src/cli/dev/tui/agent-header.js +1 -1
- package/dist/src/cli/dev/tui/blocks.d.ts +1 -1
- package/dist/src/cli/dev/tui/blocks.js +3 -2
- package/dist/src/cli/dev/tui/command-typeahead.d.ts +47 -0
- package/dist/src/cli/dev/tui/command-typeahead.js +1 -0
- package/dist/src/cli/dev/tui/dev-rebuild-status.d.ts +21 -0
- package/dist/src/cli/dev/tui/dev-rebuild-status.js +1 -0
- package/dist/src/cli/dev/tui/errors.d.ts +18 -0
- package/dist/src/cli/dev/tui/errors.js +1 -1
- package/dist/src/cli/dev/tui/prompt-command-handler.d.ts +14 -0
- package/dist/src/cli/dev/tui/prompt-command-handler.js +1 -0
- package/dist/src/cli/dev/tui/prompt-commands.d.ts +54 -0
- package/dist/src/cli/dev/tui/prompt-commands.js +2 -0
- package/dist/src/cli/dev/tui/runner.d.ts +62 -5
- package/dist/src/cli/dev/tui/runner.js +1 -1
- package/dist/src/cli/dev/tui/setup-commands.d.ts +48 -0
- package/dist/src/cli/dev/tui/setup-commands.js +2 -0
- package/dist/src/cli/dev/tui/setup-flow.d.ts +35 -0
- package/dist/src/cli/dev/tui/setup-issues.d.ts +40 -0
- package/dist/src/cli/dev/tui/setup-issues.js +1 -0
- package/dist/src/cli/dev/tui/setup-panel.d.ts +103 -0
- package/dist/src/cli/dev/tui/setup-panel.js +1 -0
- package/dist/src/cli/dev/tui/status-line.d.ts +25 -0
- package/dist/src/cli/dev/tui/status-line.js +1 -0
- package/dist/src/cli/dev/tui/stream-format.d.ts +16 -1
- package/dist/src/cli/dev/tui/stream-format.js +1 -1
- package/dist/src/cli/dev/tui/terminal-renderer.d.ts +32 -3
- package/dist/src/cli/dev/tui/terminal-renderer.js +5 -2
- package/dist/src/cli/dev/tui/test/index.d.ts +3 -1
- package/dist/src/cli/dev/tui/test/index.js +1 -1
- package/dist/src/cli/dev/tui/test/mock-terminal.d.ts +1 -0
- package/dist/src/cli/dev/tui/test/mock-terminal.js +1 -1
- package/dist/src/cli/dev/tui/theme.d.ts +10 -0
- package/dist/src/cli/dev/tui/theme.js +1 -1
- package/dist/src/cli/dev/tui/tui-prompter.d.ts +20 -0
- package/dist/src/cli/dev/tui/tui-prompter.js +1 -0
- package/dist/src/cli/dev/tui/tui.d.ts +6 -8
- package/dist/src/cli/dev/tui/tui.js +1 -1
- package/dist/src/cli/dev/tui/types.d.ts +4 -3
- package/dist/src/cli/dev/tui/vercel-status.d.ts +47 -0
- package/dist/src/cli/dev/tui/vercel-status.js +1 -0
- package/dist/src/cli/run.d.ts +12 -19
- package/dist/src/cli/run.js +2 -2
- package/dist/src/client/client.d.ts +8 -0
- package/dist/src/client/client.js +1 -1
- package/dist/src/client/file-parts.d.ts +18 -0
- package/dist/src/client/file-parts.js +1 -0
- package/dist/src/client/index.d.ts +3 -2
- package/dist/src/client/index.js +1 -1
- package/dist/src/client/message-response.js +1 -1
- package/dist/src/client/open-stream.d.ts +6 -0
- package/dist/src/client/open-stream.js +1 -1
- package/dist/src/client/session-utils.d.ts +5 -0
- package/dist/src/client/session-utils.js +1 -1
- package/dist/src/client/session.js +1 -1
- package/dist/src/client/types.d.ts +9 -2
- package/dist/src/compiled/.vendor-stamp.json +8 -8
- package/dist/src/compiled/@ai-sdk/anthropic/index.d.ts +56 -31
- package/dist/src/compiled/@ai-sdk/anthropic/index.js +2 -2
- package/dist/src/compiled/@ai-sdk/google/index.js +1 -1
- package/dist/src/compiled/@ai-sdk/mcp/index.js +1 -1
- package/dist/src/compiled/@ai-sdk/openai/index.d.ts +16 -9
- package/dist/src/compiled/@ai-sdk/openai/index.js +2 -2
- package/dist/src/compiled/@ai-sdk/otel/index.js +2 -2
- package/dist/src/compiled/@vercel/sandbox/index.js +1 -1
- package/dist/src/compiled/@workflow/core/index.js +2 -2
- package/dist/src/compiled/@workflow/core/runtime/helpers.d.ts +4 -2
- package/dist/src/compiled/@workflow/core/runtime/wait-until.d.ts +18 -0
- package/dist/src/compiled/@workflow/core/runtime.d.ts +3 -1
- package/dist/src/compiled/@workflow/core/runtime.js +28 -28
- package/dist/src/compiled/@workflow/core/serialization.d.ts +41 -0
- package/dist/src/compiled/@workflow/core/util.d.ts +0 -5
- package/dist/src/compiled/@workflow/core/version.d.ts +1 -1
- package/dist/src/compiled/@workflow/core/workflow.js +1 -1
- package/dist/src/compiled/@workflow/world/index.d.ts +2 -2
- package/dist/src/compiled/@workflow/world/queue.d.ts +28 -2
- package/dist/src/compiled/_chunks/workflow/{dist-gEXVSMPU.js → dist-CkMRLaRV.js} +1 -1
- package/dist/src/compiled/_chunks/workflow/functions-DuPjIvMH.js +1 -0
- package/dist/src/compiled/_chunks/workflow/resume-hook-CEAS3opc.js +12 -0
- package/dist/src/compiled/_chunks/workflow/sleep-Cup6vPoA.js +1 -0
- package/dist/src/compiled/_chunks/workflow/{symbols-BWCAoPHE.js → symbols-BUTtwS7j.js} +1 -1
- package/dist/src/compiled/just-bash/index.d.ts +27 -4
- package/dist/src/compiler/artifacts.js +1 -1
- package/dist/src/compiler/manifest.d.ts +8 -8
- package/dist/src/compiler/normalize-agent-config.d.ts +0 -10
- package/dist/src/compiler/normalize-agent-config.js +1 -1
- package/dist/src/compiler/normalize-channel.d.ts +2 -1
- package/dist/src/compiler/normalize-channel.js +1 -1
- package/dist/src/compiler/normalize-connection.d.ts +2 -1
- package/dist/src/compiler/normalize-connection.js +1 -1
- package/dist/src/compiler/normalize-helpers.d.ts +5 -0
- package/dist/src/compiler/normalize-helpers.js +1 -1
- package/dist/src/compiler/normalize-instructions.d.ts +3 -2
- package/dist/src/compiler/normalize-instructions.js +1 -1
- package/dist/src/compiler/normalize-manifest.js +2 -2
- package/dist/src/compiler/normalize-sandbox.d.ts +2 -1
- package/dist/src/compiler/normalize-sandbox.js +1 -1
- package/dist/src/compiler/normalize-schedule.d.ts +2 -1
- package/dist/src/compiler/normalize-schedule.js +1 -1
- package/dist/src/compiler/normalize-skill.d.ts +2 -1
- package/dist/src/compiler/normalize-skill.js +1 -1
- package/dist/src/compiler/normalize-subagent.d.ts +4 -1
- package/dist/src/compiler/normalize-subagent.js +1 -1
- package/dist/src/compiler/normalize-tool.d.ts +2 -1
- package/dist/src/compiler/normalize-tool.js +1 -1
- package/dist/src/compiler/workspace-resources.js +1 -1
- package/dist/src/context/node.d.ts +1 -1
- package/dist/src/evals/assertions/collector.d.ts +43 -0
- package/dist/src/evals/assertions/collector.js +1 -0
- package/dist/src/evals/assertions/run.d.ts +72 -0
- package/dist/src/evals/assertions/run.js +2 -0
- package/dist/src/evals/autoevals-client.js +2 -0
- package/dist/src/evals/cli/eval-client.d.ts +22 -0
- package/dist/src/evals/cli/eval-client.js +1 -0
- package/dist/src/evals/cli/eval.d.ts +12 -3
- package/dist/src/evals/cli/eval.js +1 -1
- package/dist/src/evals/context.d.ts +19 -0
- package/dist/src/evals/context.js +1 -0
- package/dist/src/evals/define-eval-config.d.ts +16 -0
- package/dist/src/evals/define-eval-config.js +1 -0
- package/dist/src/evals/define-eval.d.ts +20 -0
- package/dist/src/evals/define-eval.js +1 -0
- package/dist/src/evals/expect/index.d.ts +25 -0
- package/dist/src/evals/expect/index.js +1 -0
- package/dist/src/evals/index.d.ts +7 -2
- package/dist/src/evals/index.js +1 -1
- package/dist/src/evals/judge.d.ts +20 -0
- package/dist/src/evals/judge.js +1 -0
- package/dist/src/evals/match.d.ts +67 -0
- package/dist/src/evals/match.js +1 -0
- package/dist/src/evals/reporters/index.d.ts +1 -0
- package/dist/src/evals/reporters/index.js +1 -1
- package/dist/src/evals/requirements.d.ts +3 -0
- package/dist/src/evals/requirements.js +1 -0
- package/dist/src/evals/runner/artifacts.d.ts +7 -6
- package/dist/src/evals/runner/artifacts.js +3 -3
- package/dist/src/evals/runner/derive-run-facts.d.ts +17 -3
- package/dist/src/evals/runner/derive-run-facts.js +1 -1
- package/dist/src/evals/runner/discover.d.ts +31 -10
- package/dist/src/evals/runner/discover.js +1 -1
- package/dist/src/evals/runner/execute-eval.d.ts +25 -0
- package/dist/src/evals/runner/execute-eval.js +1 -0
- package/dist/src/evals/runner/execute-task.d.ts +31 -0
- package/dist/src/evals/runner/execute-task.js +1 -0
- package/dist/src/evals/runner/reporters/braintrust.d.ts +7 -5
- package/dist/src/evals/runner/reporters/braintrust.js +2 -2
- package/dist/src/evals/runner/reporters/console.d.ts +4 -4
- package/dist/src/evals/runner/reporters/console.js +1 -1
- package/dist/src/evals/runner/reporters/junit.d.ts +10 -0
- package/dist/src/evals/runner/reporters/junit.js +4 -0
- package/dist/src/evals/runner/reporters/types.d.ts +14 -8
- package/dist/src/evals/runner/run-evals.d.ts +38 -0
- package/dist/src/evals/runner/run-evals.js +1 -0
- package/dist/src/evals/runner/verdict.d.ts +13 -0
- package/dist/src/evals/runner/verdict.js +1 -0
- package/dist/src/evals/session.d.ts +52 -0
- package/dist/src/evals/session.js +1 -0
- package/dist/src/evals/target.d.ts +23 -0
- package/dist/src/evals/target.js +1 -0
- package/dist/src/evals/types.d.ts +353 -165
- package/dist/src/execution/compaction.d.ts +14 -0
- package/dist/src/execution/compaction.js +1 -0
- package/dist/src/execution/delegated-parent-notification.js +1 -1
- package/dist/src/execution/dispatch-runtime-actions-step.js +1 -1
- package/dist/src/execution/node-step.js +1 -1
- package/dist/src/execution/sandbox/bash-tool.d.ts +6 -6
- package/dist/src/execution/sandbox/bash-tool.js +1 -1
- package/dist/src/execution/sandbox/bindings/local.d.ts +11 -0
- package/dist/src/execution/sandbox/bindings/local.js +1 -1
- package/dist/src/execution/sandbox/glob-tool.js +3 -3
- package/dist/src/execution/sandbox/grep-tool.js +3 -3
- package/dist/src/execution/sandbox/read-file-tool.js +1 -1
- package/dist/src/execution/subagent-adapter.js +1 -1
- package/dist/src/execution/tool-auth.js +1 -1
- package/dist/src/execution/turn-workflow.js +1 -1
- package/dist/src/execution/workflow-runtime.d.ts +2 -2
- package/dist/src/execution/workflow-runtime.js +1 -1
- package/dist/src/execution/workflow-steps.js +1 -1
- package/dist/src/harness/action-result-helpers.js +1 -1
- package/dist/src/harness/authorization.d.ts +26 -0
- package/dist/src/harness/authorization.js +1 -1
- package/dist/src/harness/code-mode-lifecycle.js +1 -1
- package/dist/src/harness/emission.d.ts +12 -5
- package/dist/src/harness/emission.js +1 -1
- package/dist/src/harness/model-call-error.d.ts +35 -6
- package/dist/src/harness/model-call-error.js +1 -1
- package/dist/src/harness/step-hooks.d.ts +10 -4
- package/dist/src/harness/step-hooks.js +1 -1
- package/dist/src/harness/tool-loop.js +1 -1
- package/dist/src/harness/tools.d.ts +4 -6
- package/dist/src/harness/tools.js +1 -1
- package/dist/src/harness/turn-tag-state.d.ts +4 -0
- package/dist/src/harness/turn-tag-state.js +1 -1
- package/dist/src/harness/types.d.ts +4 -15
- package/dist/src/internal/application/cache-metadata.js +1 -1
- package/dist/src/internal/application/compiled-artifacts.js +1 -1
- package/dist/src/internal/application/package.js +1 -1
- package/dist/src/internal/application/paths.js +1 -1
- package/dist/src/internal/authored-definition/schema-backed.js +1 -1
- package/dist/src/internal/authored-module-loader.d.ts +4 -1
- package/dist/src/internal/authored-module-loader.js +2 -2
- package/dist/src/internal/authored-module-map-loader.js +1 -1
- package/dist/src/internal/nitro/dev-runtime-artifacts.d.ts +11 -0
- package/dist/src/internal/nitro/dev-runtime-artifacts.js +1 -1
- package/dist/src/internal/nitro/host/build-application.js +1 -1
- package/dist/src/internal/nitro/host/build-vercel-agent-summary.js +1 -1
- package/dist/src/internal/nitro/host/configure-nitro-routes.js +3 -3
- package/dist/src/internal/nitro/host/create-application-nitro.js +1 -1
- package/dist/src/internal/nitro/host/dev-authored-source-watcher.js +1 -1
- package/dist/src/internal/nitro/host/dev-watcher-log.d.ts +37 -0
- package/dist/src/internal/nitro/host/dev-watcher-log.js +1 -0
- package/dist/src/internal/nitro/host/ports.d.ts +8 -0
- package/dist/src/internal/nitro/host/ports.js +1 -0
- package/dist/src/internal/nitro/host/prepare-application-host.js +1 -1
- package/dist/src/internal/nitro/host/server-external-packages.d.ts +1 -1
- package/dist/src/internal/nitro/host/server-external-packages.js +1 -1
- package/dist/src/internal/nitro/host/start-development-server.js +1 -1
- package/dist/src/internal/nitro/host/start-production-server.js +1 -1
- package/dist/src/internal/nitro/routes/agent-info/build-agent-info-response-from-manifest.d.ts +5 -0
- package/dist/src/internal/nitro/routes/agent-info/build-agent-info-response-from-manifest.js +1 -0
- package/dist/src/internal/nitro/routes/agent-info/build-agent-info-response.d.ts +31 -2
- package/dist/src/internal/nitro/routes/agent-info/build-agent-info-response.js +1 -1
- package/dist/src/internal/nitro/routes/agent-info/load-agent-info-data.d.ts +13 -0
- package/dist/src/internal/nitro/routes/agent-info/load-agent-info-data.js +1 -1
- package/dist/src/internal/nitro/routes/info.d.ts +2 -2
- package/dist/src/internal/nitro/routes/info.js +1 -1
- package/dist/src/internal/workflow/queue-namespace.d.ts +5 -0
- package/dist/src/internal/workflow/queue-namespace.js +1 -0
- package/dist/src/internal/workflow-bundle/builder-support.js +2 -2
- package/dist/src/internal/workflow-bundle/builder.js +3 -5
- package/dist/src/internal/workflow-bundle/vercel-workflow-output.js +1 -1
- package/dist/src/internal/workflow-bundle/workflow-builders.d.ts +1 -1
- package/dist/src/internal/workflow-bundle/workflow-builders.js +1 -1
- package/dist/src/node_modules/.pnpm/@clack_core@1.3.1/node_modules/@clack/core/dist/index.js +4 -4
- package/dist/src/protocol/message.d.ts +15 -0
- package/dist/src/protocol/message.js +2 -2
- package/dist/src/public/channels/slack/api.d.ts +8 -0
- package/dist/src/public/channels/slack/api.js +1 -1
- package/dist/src/public/channels/slack/connections.d.ts +26 -18
- package/dist/src/public/channels/slack/connections.js +1 -1
- package/dist/src/public/channels/slack/defaults.d.ts +5 -2
- package/dist/src/public/channels/slack/defaults.js +1 -1
- package/dist/src/public/channels/slack/index.d.ts +1 -1
- package/dist/src/public/channels/slack/slackChannel.d.ts +65 -5
- package/dist/src/public/channels/slack/slackChannel.js +1 -1
- package/dist/src/public/channels/teams/defaults.js +1 -1
- package/dist/src/public/connections/errors.d.ts +8 -0
- package/dist/src/public/definitions/tool.d.ts +0 -33
- package/dist/src/public/next/index.d.ts +7 -1
- package/dist/src/public/next/index.js +1 -1
- package/dist/src/public/next/server.d.ts +1 -0
- package/dist/src/public/next/server.js +1 -1
- package/dist/src/public/nuxt/dev-server.js +1 -1
- package/dist/src/public/sandbox/backends/local.d.ts +2 -3
- package/dist/src/public/sveltekit/dev-server.js +1 -1
- package/dist/src/public/sveltekit/index.d.ts +1 -1
- package/dist/src/public/tools/defaults.d.ts +2 -4
- package/dist/src/public/tools/defaults.js +1 -1
- package/dist/src/public/tools/define-bash-tool.d.ts +3 -3
- package/dist/src/public/tools/define-bash-tool.js +1 -1
- package/dist/src/public/tools/define-read-file-tool.d.ts +0 -6
- package/dist/src/public/tools/define-read-file-tool.js +1 -1
- package/dist/src/public/tools/index.d.ts +2 -2
- package/dist/src/public/tools/index.js +1 -1
- package/dist/src/public/tools/internal.js +1 -1
- package/dist/src/runtime/actions/types.d.ts +11 -11
- package/dist/src/runtime/agent/mock-model-adapter.js +1 -1
- package/dist/src/runtime/agent/mock-model-fixtures.js +3 -2
- package/dist/src/runtime/agent/mock-model-skill-selection.js +3 -4
- package/dist/src/runtime/connections/callback-route.js +1 -1
- package/dist/src/runtime/connections/mcp-client.js +1 -1
- package/dist/src/runtime/connections/scoped-authorization.d.ts +21 -5
- package/dist/src/runtime/connections/scoped-authorization.js +1 -1
- package/dist/src/runtime/connections/types.d.ts +33 -0
- package/dist/src/runtime/connections/validate-authorization.js +1 -1
- package/dist/src/runtime/framework-tools/bash.d.ts +3 -3
- package/dist/src/runtime/framework-tools/bash.js +1 -1
- package/dist/src/runtime/framework-tools/connection-search-dynamic.d.ts +1 -1
- package/dist/src/runtime/framework-tools/connection-search-dynamic.js +1 -1
- package/dist/src/runtime/framework-tools/file-state.d.ts +3 -3
- package/dist/src/runtime/framework-tools/index.js +1 -1
- package/dist/src/runtime/framework-tools/read-file.js +2 -2
- package/dist/src/runtime/framework-tools/todo.d.ts +7 -0
- package/dist/src/runtime/framework-tools/todo.js +2 -2
- package/dist/src/runtime/governance/auth/http-basic.js +1 -1
- package/dist/src/runtime/input/types.d.ts +1 -1
- package/dist/src/runtime/resolve-tool.d.ts +2 -2
- package/dist/src/runtime/resolve-tool.js +1 -1
- package/dist/src/runtime/sandbox/keys.js +1 -1
- package/dist/src/runtime/session-callback-route.js +1 -1
- package/dist/src/runtime/types.d.ts +1 -7
- package/dist/src/services/dev-client/client-options.d.ts +8 -0
- package/dist/src/services/dev-client/client-options.js +1 -0
- package/dist/src/services/dev-client/runtime-artifacts.d.ts +13 -0
- package/dist/src/services/dev-client/runtime-artifacts.js +1 -0
- package/dist/src/services/dev-client.d.ts +13 -46
- package/dist/src/services/dev-client.js +1 -1
- package/dist/src/services/inspect-application.js +1 -1
- package/dist/src/setup/ask.d.ts +205 -0
- package/dist/src/setup/ask.js +1 -0
- package/dist/src/setup/boxes/add-channels.d.ts +100 -16
- package/dist/src/setup/boxes/add-channels.js +2 -1
- package/dist/src/setup/boxes/add-connections.d.ts +13 -23
- package/dist/src/setup/boxes/add-connections.js +1 -1
- package/dist/src/setup/boxes/apply-ai-gateway-credential.d.ts +2 -2
- package/dist/src/setup/boxes/apply-ai-gateway-credential.js +1 -1
- package/dist/src/setup/boxes/deploy-project.d.ts +46 -14
- package/dist/src/setup/boxes/deploy-project.js +1 -1
- package/dist/src/setup/boxes/detect-ai-gateway.d.ts +10 -3
- package/dist/src/setup/boxes/detect-ai-gateway.js +1 -1
- package/dist/src/setup/boxes/link-project.d.ts +3 -3
- package/dist/src/setup/boxes/link-project.js +1 -1
- package/dist/src/setup/boxes/one-shot-next-steps.d.ts +18 -0
- package/dist/src/setup/boxes/one-shot-next-steps.js +2 -0
- package/dist/src/setup/boxes/preflight.d.ts +14 -6
- package/dist/src/setup/boxes/preflight.js +1 -1
- package/dist/src/setup/boxes/resolve-provisioning.d.ts +36 -8
- package/dist/src/setup/boxes/resolve-provisioning.js +1 -1
- package/dist/src/setup/boxes/resolve-target.d.ts +25 -8
- package/dist/src/setup/boxes/resolve-target.js +1 -1
- package/dist/src/setup/boxes/scaffold.d.ts +12 -6
- package/dist/src/setup/boxes/scaffold.js +1 -1
- package/dist/src/setup/boxes/select-channels.d.ts +38 -9
- package/dist/src/setup/boxes/select-channels.js +1 -1
- package/dist/src/setup/boxes/select-chat.d.ts +15 -11
- package/dist/src/setup/boxes/select-chat.js +1 -1
- package/dist/src/setup/boxes/select-connections.d.ts +30 -0
- package/dist/src/setup/boxes/select-connections.js +1 -0
- package/dist/src/setup/boxes/select-model.d.ts +18 -14
- package/dist/src/setup/boxes/select-model.js +1 -1
- package/dist/src/setup/boxes/select-setup-mode.d.ts +32 -0
- package/dist/src/setup/boxes/select-setup-mode.js +1 -0
- package/dist/src/setup/channel-add-conflicts.d.ts +28 -0
- package/dist/src/setup/channel-add-conflicts.js +1 -0
- package/dist/src/setup/cli/channel-setup-prompter.d.ts +23 -0
- package/dist/src/setup/cli/channel-setup-prompter.js +1 -0
- package/dist/src/setup/cli/connection-add-prompter.d.ts +8 -0
- package/dist/src/setup/cli/connection-add-prompter.js +1 -0
- package/dist/src/setup/{scaffold/cli → cli}/index.d.ts +4 -3
- package/dist/src/setup/cli/index.js +1 -0
- package/dist/src/setup/{scaffold/cli → cli}/prompt-ui.d.ts +39 -15
- package/dist/src/setup/cli/prompt-ui.js +5 -0
- package/dist/src/setup/{scaffold/cli → cli}/rail-log.d.ts +2 -0
- package/dist/src/setup/{scaffold/cli → cli}/rail-log.js +2 -2
- package/dist/src/setup/{scaffold/cli → cli}/select-component.d.ts +18 -3
- package/dist/src/setup/cli/select-component.js +1 -0
- package/dist/src/setup/cli/select-option-codec.d.ts +12 -0
- package/dist/src/setup/cli/select-option-codec.js +1 -0
- package/dist/src/setup/{scaffold/cli → cli}/select-state.d.ts +13 -1
- package/dist/src/setup/cli/select-state.js +1 -0
- package/dist/src/setup/cli/whimsy.d.ts +16 -0
- package/dist/src/setup/cli/whimsy.js +1 -0
- package/dist/src/setup/{scaffold/steps/setup-connection.d.ts → connection-connector.d.ts} +3 -2
- package/dist/src/setup/connection-connector.js +1 -0
- package/dist/src/setup/flows/channels.d.ts +43 -0
- package/dist/src/setup/flows/channels.js +1 -0
- package/dist/src/setup/flows/deploy.d.ts +40 -0
- package/dist/src/setup/flows/deploy.js +1 -0
- package/dist/src/setup/flows/in-project.d.ts +16 -0
- package/dist/src/setup/flows/in-project.js +1 -0
- package/dist/src/setup/flows/link.d.ts +43 -0
- package/dist/src/setup/flows/link.js +1 -0
- package/dist/src/setup/flows/model.d.ts +112 -0
- package/dist/src/setup/flows/model.js +1 -0
- package/dist/src/setup/flows/vercel.d.ts +31 -0
- package/dist/src/setup/flows/vercel.js +2 -0
- package/dist/src/setup/gateway-models.js +1 -1
- package/dist/src/setup/headless.d.ts +1 -1
- package/dist/src/setup/index.d.ts +10 -4
- package/dist/src/setup/index.js +1 -1
- package/dist/src/setup/onboarding.d.ts +7 -4
- package/dist/src/setup/onboarding.js +1 -1
- package/dist/src/setup/package-manager.d.ts +27 -0
- package/dist/src/setup/package-manager.js +1 -0
- package/dist/src/setup/primitives/index.d.ts +3 -0
- package/dist/src/setup/primitives/index.js +1 -0
- package/dist/src/setup/primitives/pm/bun.d.ts +10 -0
- package/dist/src/setup/primitives/pm/bun.js +1 -0
- package/dist/src/setup/primitives/pm/index.d.ts +11 -0
- package/dist/src/setup/primitives/pm/index.js +1 -0
- package/dist/src/setup/primitives/pm/npm.d.ts +10 -0
- package/dist/src/setup/primitives/pm/npm.js +1 -0
- package/dist/src/setup/primitives/pm/pnpm.d.ts +27 -0
- package/dist/src/setup/primitives/pm/pnpm.js +8 -0
- package/dist/src/setup/primitives/pm/run.d.ts +23 -0
- package/dist/src/setup/primitives/pm/run.js +1 -0
- package/dist/src/setup/primitives/pm/shared.d.ts +8 -0
- package/dist/src/setup/primitives/pm/shared.js +1 -0
- package/dist/src/setup/primitives/pm/types.d.ts +37 -0
- package/dist/src/setup/primitives/pm/types.js +1 -0
- package/dist/src/setup/primitives/pm/yarn.d.ts +10 -0
- package/dist/src/setup/primitives/pm/yarn.js +1 -0
- package/dist/src/setup/primitives/run-pnpm.d.ts +1 -0
- package/dist/src/setup/primitives/run-pnpm.js +1 -0
- package/dist/src/setup/{scaffold/primitives → primitives}/run-vercel.d.ts +7 -0
- package/dist/src/setup/primitives/run-vercel.js +1 -0
- package/dist/src/setup/project-name.d.ts +4 -0
- package/dist/src/setup/project-name.js +1 -0
- package/dist/src/setup/project-resolution.d.ts +54 -0
- package/dist/src/setup/project-resolution.js +1 -0
- package/dist/src/setup/prompter.d.ts +52 -4
- package/dist/src/setup/prompter.js +1 -1
- package/dist/src/setup/quit-guard.d.ts +1 -1
- package/dist/src/setup/run-vercel-link.d.ts +1 -1
- package/dist/src/setup/run-vercel-link.js +1 -1
- package/dist/src/setup/runner.d.ts +5 -4
- package/dist/src/setup/runner.js +1 -1
- package/dist/src/setup/scaffold/channels-catalog.d.ts +3 -3
- package/dist/src/setup/scaffold/channels-catalog.js +1 -1
- package/dist/src/setup/scaffold/create/add-to-project.d.ts +26 -0
- package/dist/src/setup/scaffold/create/add-to-project.js +1 -0
- package/dist/src/setup/scaffold/create/project.d.ts +54 -0
- package/dist/src/setup/scaffold/create/project.js +80 -0
- package/dist/src/setup/scaffold/index.d.ts +4 -4
- package/dist/src/setup/scaffold/index.js +1 -1
- package/dist/src/setup/scaffold/{channels.d.ts → update/channels.d.ts} +11 -0
- package/dist/src/setup/scaffold/update/channels.js +7 -0
- package/dist/src/setup/scaffold/{connections.d.ts → update/connections.d.ts} +1 -1
- package/dist/src/setup/scaffold/update/connections.js +21 -0
- package/dist/src/setup/scaffold/version-tokens.d.ts +11 -0
- package/dist/src/setup/scaffold/version-tokens.js +1 -0
- package/dist/src/setup/{scaffold/steps/setup-slackbot.d.ts → slackbot.d.ts} +24 -20
- package/dist/src/setup/slackbot.js +1 -0
- package/dist/src/setup/state.d.ts +62 -15
- package/dist/src/setup/state.js +1 -1
- package/dist/src/setup/step.d.ts +9 -18
- package/dist/src/setup/vercel-project.d.ts +15 -8
- package/dist/src/setup/vercel-project.js +1 -1
- package/dist/src/shared/agent-definition.d.ts +9 -3
- package/dist/src/shared/default-agent-model.d.ts +5 -0
- package/dist/src/shared/default-agent-model.js +1 -0
- package/dist/src/shared/sandbox-backend.d.ts +11 -11
- package/dist/src/source-change/apply-model-name.d.ts +25 -0
- package/dist/src/source-change/apply-model-name.js +2 -0
- package/dist/src/source-change/static-source-change.d.ts +36 -0
- package/dist/src/source-change/static-source-change.js +1 -0
- package/dist/src/svelte/index.js +1 -1
- package/dist/src/svelte/use-eve-agent.js +1 -1
- package/dist/src/vue/index.js +1 -1
- package/dist/src/vue/use-eve-agent.js +1 -1
- package/package.json +23 -38
- package/dist/docs/public/advanced/evals.md +0 -121
- package/dist/src/cli/commands/setup.d.ts +0 -55
- package/dist/src/cli/commands/setup.js +0 -1
- package/dist/src/cli/dev/repl/input-requests.d.ts +0 -38
- package/dist/src/cli/dev/repl/input-requests.js +0 -1
- package/dist/src/cli/dev/repl/input.d.ts +0 -19
- package/dist/src/cli/dev/repl/input.js +0 -1
- package/dist/src/cli/dev/repl/repl.d.ts +0 -62
- package/dist/src/cli/dev/repl/repl.js +0 -2
- package/dist/src/cli/dev/repl/terminal.d.ts +0 -21
- package/dist/src/cli/dev/repl/terminal.js +0 -5
- package/dist/src/compiled/_chunks/workflow/resume-hook-0Zk0zSvq.js +0 -12
- package/dist/src/compiled/_chunks/workflow/sleep-DXZr2BgM.js +0 -1
- package/dist/src/evals/define-eval-suite.d.ts +0 -16
- package/dist/src/evals/define-eval-suite.js +0 -1
- package/dist/src/evals/runner/execute-case.d.ts +0 -23
- package/dist/src/evals/runner/execute-case.js +0 -1
- package/dist/src/evals/runner/execute-suite.d.ts +0 -24
- package/dist/src/evals/runner/execute-suite.js +0 -1
- package/dist/src/evals/scorers/autoevals-client.js +0 -2
- package/dist/src/evals/scorers/autoevals.d.ts +0 -58
- package/dist/src/evals/scorers/autoevals.js +0 -1
- package/dist/src/evals/scorers/json.d.ts +0 -10
- package/dist/src/evals/scorers/json.js +0 -1
- package/dist/src/evals/scorers/run.d.ts +0 -18
- package/dist/src/evals/scorers/run.js +0 -1
- package/dist/src/evals/scorers/sql.d.ts +0 -9
- package/dist/src/evals/scorers/sql.js +0 -1
- package/dist/src/evals/scorers/text.d.ts +0 -18
- package/dist/src/evals/scorers/text.js +0 -1
- package/dist/src/evals/scores/index.d.ts +0 -72
- package/dist/src/evals/scores/index.js +0 -1
- package/dist/src/execution/tool-compaction.d.ts +0 -9
- package/dist/src/execution/tool-compaction.js +0 -1
- package/dist/src/services/dev-client/stream.d.ts +0 -5
- package/dist/src/services/dev-client/stream.js +0 -1
- package/dist/src/services/dev-client/url.d.ts +0 -11
- package/dist/src/services/dev-client/url.js +0 -1
- package/dist/src/setup/channel-setup-prompter.d.ts +0 -8
- package/dist/src/setup/channel-setup-prompter.js +0 -1
- package/dist/src/setup/scaffold/channels.js +0 -7
- package/dist/src/setup/scaffold/cli/channel-add-prompter.d.ts +0 -12
- package/dist/src/setup/scaffold/cli/channel-add-prompter.js +0 -1
- package/dist/src/setup/scaffold/cli/channel-setup-prompter.d.ts +0 -56
- package/dist/src/setup/scaffold/cli/connection-add-prompter.d.ts +0 -44
- package/dist/src/setup/scaffold/cli/connection-add-prompter.js +0 -1
- package/dist/src/setup/scaffold/cli/index.js +0 -1
- package/dist/src/setup/scaffold/cli/prompt-ui.js +0 -5
- package/dist/src/setup/scaffold/cli/select-component.js +0 -1
- package/dist/src/setup/scaffold/cli/select-state.js +0 -1
- package/dist/src/setup/scaffold/connections.js +0 -21
- package/dist/src/setup/scaffold/pnpm-workspace.d.ts +0 -3
- package/dist/src/setup/scaffold/pnpm-workspace.js +0 -11
- package/dist/src/setup/scaffold/primitives/detect-deployment.d.ts +0 -13
- package/dist/src/setup/scaffold/primitives/detect-deployment.js +0 -1
- package/dist/src/setup/scaffold/primitives/index.d.ts +0 -3
- package/dist/src/setup/scaffold/primitives/index.js +0 -1
- package/dist/src/setup/scaffold/primitives/pnpm-invocation.d.ts +0 -12
- package/dist/src/setup/scaffold/primitives/pnpm-invocation.js +0 -1
- package/dist/src/setup/scaffold/primitives/run-pnpm.d.ts +0 -17
- package/dist/src/setup/scaffold/primitives/run-pnpm.js +0 -1
- package/dist/src/setup/scaffold/primitives/run-vercel.js +0 -1
- package/dist/src/setup/scaffold/project.d.ts +0 -21
- package/dist/src/setup/scaffold/project.js +0 -80
- package/dist/src/setup/scaffold/steps/deploy-to-vercel.d.ts +0 -17
- package/dist/src/setup/scaffold/steps/deploy-to-vercel.js +0 -1
- package/dist/src/setup/scaffold/steps/index.d.ts +0 -4
- package/dist/src/setup/scaffold/steps/index.js +0 -1
- package/dist/src/setup/scaffold/steps/project-resolution.d.ts +0 -19
- package/dist/src/setup/scaffold/steps/project-resolution.js +0 -1
- package/dist/src/setup/scaffold/steps/run-add-connection.d.ts +0 -40
- package/dist/src/setup/scaffold/steps/run-add-connection.js +0 -1
- package/dist/src/setup/scaffold/steps/run-add-to-agent.d.ts +0 -81
- package/dist/src/setup/scaffold/steps/run-add-to-agent.js +0 -2
- package/dist/src/setup/scaffold/steps/setup-connection.js +0 -1
- package/dist/src/setup/scaffold/steps/setup-slackbot.js +0 -1
- /package/dist/src/{setup/scaffold/cli/channel-setup-prompter.js → cli/dev/tui/setup-flow.js} +0 -0
- /package/dist/src/evals/{scorers/autoevals-client.d.ts → autoevals-client.d.ts} +0 -0
- /package/dist/src/setup/{scaffold/cli → cli}/command-output.d.ts +0 -0
- /package/dist/src/setup/{scaffold/cli → cli}/command-output.js +0 -0
- /package/dist/src/setup/{scaffold/human-action.d.ts → human-action.d.ts} +0 -0
- /package/dist/src/setup/{scaffold/human-action.js → human-action.js} +0 -0
- /package/dist/src/setup/{scaffold/primitives → primitives}/process-output.d.ts +0 -0
- /package/dist/src/setup/{scaffold/primitives → primitives}/process-output.js +0 -0
- /package/dist/src/setup/scaffold/{web-template.d.ts → create/web-template.d.ts} +0 -0
- /package/dist/src/setup/scaffold/{web-template.js → create/web-template.js} +0 -0
- /package/dist/src/setup/scaffold/{module-files.d.ts → update/module-files.d.ts} +0 -0
- /package/dist/src/setup/scaffold/{module-files.js → update/module-files.js} +0 -0
- /package/dist/src/setup/scaffold/{package-json.d.ts → update/package-json.d.ts} +0 -0
- /package/dist/src/setup/scaffold/{package-json.js → update/package-json.js} +0 -0
- /package/dist/src/setup/scaffold/{primitives → update}/update-connection-connector.d.ts +0 -0
- /package/dist/src/setup/scaffold/{primitives → update}/update-connection-connector.js +0 -0
- /package/dist/src/setup/scaffold/{primitives → update}/update-slack-channel.d.ts +0 -0
- /package/dist/src/setup/scaffold/{primitives → update}/update-slack-channel.js +0 -0
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Targets and requirements"
|
|
3
|
+
description: "Point evals at a local dev server or a deployment, and gate evals on target capabilities with requires."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
An eval target is always an HTTP URL. `eve eval` starts a local dev server, while `eve eval --url <url>` runs against an existing server or deployment — the same eval files work for both, which is what makes evals usable as end-to-end tests in CI.
|
|
7
|
+
|
|
8
|
+
The runner polls `/eve/v1/health`, verifies `/eve/v1/info`, and exposes the live target as `t.target` inside the `test` function.
|
|
9
|
+
|
|
10
|
+
## Target helpers
|
|
11
|
+
|
|
12
|
+
```ts title="evals/heartbeat.eval.ts"
|
|
13
|
+
import { defineEval } from "eve/evals";
|
|
14
|
+
|
|
15
|
+
export default defineEval({
|
|
16
|
+
requires: ["mockModels", "devRoutes"],
|
|
17
|
+
async test(t) {
|
|
18
|
+
const { sessionIds } = await t.target.dispatchSchedule("heartbeat");
|
|
19
|
+
await t.target.attachSession(sessionIds[0]!);
|
|
20
|
+
t.completed();
|
|
21
|
+
t.calledTool("send_report");
|
|
22
|
+
},
|
|
23
|
+
});
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
- `t.target.fetch(path, init)` performs an authenticated fetch against the target — useful for channel and webhook ingress.
|
|
27
|
+
- `t.target.dispatchSchedule(id)` triggers a [schedule](../schedules) through the dev-only schedule route and returns the session ids it created. It requires the `devRoutes` capability.
|
|
28
|
+
- `t.target.attachSession(sessionId, { startIndex? })` consumes one turn from a session created outside the eval — by a channel or a schedule — so its events feed the run-level assertions.
|
|
29
|
+
|
|
30
|
+
Sessions attached this way are full `EveEvalSession`s: you can keep driving them with `send` and read their event streams. The run-level assertions on `t` (`t.completed()`, `t.calledTool(...)`) read the whole run, attached sessions included.
|
|
31
|
+
|
|
32
|
+
## Requirements
|
|
33
|
+
|
|
34
|
+
Use `requires` to declare assumptions the runner verifies before executing an eval:
|
|
35
|
+
|
|
36
|
+
| Requirement | Means |
|
|
37
|
+
| -------------- | --------------------------------------------------------------------- |
|
|
38
|
+
| `"mockModels"` | `/eve/v1/info` reports the deterministic mock model adapter is active |
|
|
39
|
+
| `"devRoutes"` | `/eve/v1/info` reports dev-only routes are mounted |
|
|
40
|
+
| `"env:NAME"` | The eval process has environment variable `NAME` set |
|
|
41
|
+
|
|
42
|
+
Unmet requirements produce a visible `skipped` verdict and do not affect the exit code. Pass `--no-skips` when a CI leg must prove full coverage.
|
|
43
|
+
|
|
44
|
+
## Mock models
|
|
45
|
+
|
|
46
|
+
Deterministic evals — the kind you want in CI — should not depend on a live model. `eve eval --mock-models` starts the local dev server with deterministic authored models, and `requires: ["mockModels"]` makes the dependency explicit so the eval skips instead of flaking anywhere else.
|
|
47
|
+
|
|
48
|
+
`--mock-models` is invalid with `--url` because remote target capabilities are discovered, not set by the runner. For evals that genuinely need a real model — judging nuanced behavior, exercising provider-side tools — gate them on credentials instead (`requires: ["env:AI_GATEWAY_API_KEY"]`) and keep them in their own eval files so a tag filter can select or exclude them.
|
|
49
|
+
|
|
50
|
+
## What to read next
|
|
51
|
+
|
|
52
|
+
- [Running evals](./running): `--url`, `--mock-models`, and `--no-skips` in practice
|
|
53
|
+
- [Schedules](../schedules): the surface `dispatchSchedule` drives
|
|
54
|
+
- [Channels](../channels/overview): ingress you can exercise with `target.fetch`
|
|
@@ -34,6 +34,15 @@ All fields are optional.
|
|
|
34
34
|
| `eveBuildCommand` | `string` | `"eve build"` | Build command for the generated Eve Vercel service. Use it when the Eve service needs project-specific prework, without changing the Next.js build. |
|
|
35
35
|
| `configureVercelOutput` | `boolean` | `true` | Set to `false` to skip creating or updating `.vercel/output/config.json`. By default `withEve` writes the `experimentalServices` entries for both apps. |
|
|
36
36
|
| `servicePrefix` | `string` | `"/_eve_internal/eve"` | Private Vercel route namespace for the Eve service. Must match the Eve service's mount in your Vercel Build Output config when you set it manually. |
|
|
37
|
+
| `devServerTimeoutMs` | `number` | `180000` | Maximum time to wait for the Eve development server to become available. |
|
|
38
|
+
|
|
39
|
+
For unusually slow cold starts, increase the development timeout:
|
|
40
|
+
|
|
41
|
+
```ts
|
|
42
|
+
export default withEve(nextConfig, {
|
|
43
|
+
devServerTimeoutMs: 300_000,
|
|
44
|
+
});
|
|
45
|
+
```
|
|
37
46
|
|
|
38
47
|
## Call the hook
|
|
39
48
|
|
|
@@ -51,25 +60,25 @@ Before deploying a user-facing app, add `agent/channels/eve.ts` so production re
|
|
|
51
60
|
|
|
52
61
|
## Dev vs deploy topology
|
|
53
62
|
|
|
54
|
-
- **Local dev.** `
|
|
63
|
+
- **Local dev.** `npm run dev` boots the Eve dev server next to `next dev` and rewrites the Eve routes over to it. The browser only ever talks to the Next.js origin.
|
|
55
64
|
- **Vercel.** The web app and the Eve runtime deploy as a single project. The web app stays public; the Eve runtime sits behind it on the same site origin. When the agent needs its own build step, set `eveBuildCommand`:
|
|
56
65
|
|
|
57
66
|
```ts
|
|
58
67
|
export default withEve(nextConfig, {
|
|
59
|
-
eveBuildCommand: "
|
|
68
|
+
eveBuildCommand: "npm run build:eve",
|
|
60
69
|
});
|
|
61
70
|
```
|
|
62
71
|
|
|
63
72
|
- **Local production build.** `next build && next start` proxies Eve routes to `http://127.0.0.1:4274`. Change the port with `EVE_NEXT_PRODUCTION_PORT`:
|
|
64
73
|
|
|
65
74
|
```bash
|
|
66
|
-
EVE_NEXT_PRODUCTION_PORT=5000
|
|
75
|
+
EVE_NEXT_PRODUCTION_PORT=5000 npm run build && npm start
|
|
67
76
|
```
|
|
68
77
|
|
|
69
78
|
- **Non-Vercel hosts.** If the Eve service lives on a separate origin, tell Next.js where to find it with `EVE_NEXT_PRODUCTION_ORIGIN`:
|
|
70
79
|
|
|
71
80
|
```bash
|
|
72
|
-
EVE_NEXT_PRODUCTION_ORIGIN=https://agent.example.com
|
|
81
|
+
EVE_NEXT_PRODUCTION_ORIGIN=https://agent.example.com npm run build
|
|
73
82
|
```
|
|
74
83
|
|
|
75
84
|
## What to read next
|
|
@@ -56,14 +56,14 @@ Out of the box the module gives Eve a framework default channel. When you want t
|
|
|
56
56
|
|
|
57
57
|
## Dev vs deploy topology
|
|
58
58
|
|
|
59
|
-
- **Local dev.** `
|
|
59
|
+
- **Local dev.** `npm run dev` starts the Eve dev server next to `nuxt dev` and proxies the Eve routes through it. As far as the browser knows, everything is the Nuxt origin.
|
|
60
60
|
- **Vercel.** A single Vercel project carries both the Nuxt app and the Eve runtime. The web app stays public; the runtime sits behind it on the same origin. Set `eveBuildCommand` when the agent needs its own build step:
|
|
61
61
|
|
|
62
62
|
```ts
|
|
63
63
|
export default defineNuxtConfig({
|
|
64
64
|
modules: ["eve/nuxt"],
|
|
65
65
|
eve: {
|
|
66
|
-
eveBuildCommand: "
|
|
66
|
+
eveBuildCommand: "npm run build:eve",
|
|
67
67
|
},
|
|
68
68
|
});
|
|
69
69
|
```
|
|
@@ -71,8 +71,8 @@ Out of the box the module gives Eve a framework default channel. When you want t
|
|
|
71
71
|
- **Non-Vercel hosts.** Point Nuxt at a separate Eve origin with `EVE_NUXT_PRODUCTION_ORIGIN`. To override the local port (it defaults to `4274`), use `EVE_NUXT_PRODUCTION_PORT`:
|
|
72
72
|
|
|
73
73
|
```bash
|
|
74
|
-
EVE_NUXT_PRODUCTION_ORIGIN=https://agent.example.com
|
|
75
|
-
EVE_NUXT_PRODUCTION_PORT=5000
|
|
74
|
+
EVE_NUXT_PRODUCTION_ORIGIN=https://agent.example.com npm run build
|
|
75
|
+
EVE_NUXT_PRODUCTION_PORT=5000 npm run build && npm run preview
|
|
76
76
|
```
|
|
77
77
|
|
|
78
78
|
## What to read next
|
|
@@ -65,14 +65,14 @@ Out of the box the plugin uses a framework default channel. To set the route aut
|
|
|
65
65
|
|
|
66
66
|
## Dev vs deploy topology
|
|
67
67
|
|
|
68
|
-
- **Local dev.** `
|
|
68
|
+
- **Local dev.** `npm run dev` boots the Eve dev server next to SvelteKit and proxies the Eve routes to it, so the browser only ever hits the SvelteKit origin. `npm run build && npm run preview` behaves the same: the preview server gets its own Eve route proxy and either reuses the shared Eve server or starts one.
|
|
69
69
|
- **Vercel.** The SvelteKit app and the Eve runtime deploy as a single project. The web app is public; the Eve runtime sits behind it on the same origin. Use `eveBuildCommand` for a project-specific agent build:
|
|
70
70
|
|
|
71
71
|
```ts
|
|
72
72
|
export default defineConfig({
|
|
73
73
|
plugins: [
|
|
74
74
|
eveSvelteKit({
|
|
75
|
-
eveBuildCommand: "
|
|
75
|
+
eveBuildCommand: "npm run build:eve",
|
|
76
76
|
}),
|
|
77
77
|
sveltekit(),
|
|
78
78
|
],
|
|
@@ -8,24 +8,35 @@ Eve is a filesystem-first framework for durable agents: you write capabilities u
|
|
|
8
8
|
## Prerequisites
|
|
9
9
|
|
|
10
10
|
- Node `24.x`
|
|
11
|
-
-
|
|
11
|
+
- npm (bundled with Node)
|
|
12
12
|
|
|
13
13
|
You also need a model credential. Set the provider or gateway key your model string requires (gateway ids like `anthropic/claude-opus-4.8` route through the Vercel AI Gateway), or link a Vercel project that supplies one.
|
|
14
14
|
|
|
15
15
|
## Create Your Agent
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
Run `eve init` with `npx` before Eve is installed locally:
|
|
18
18
|
|
|
19
19
|
```bash
|
|
20
|
-
|
|
20
|
+
npx eve@latest init my-agent
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
-
The
|
|
23
|
+
The command creates an npm-managed child directory, uses Eve's default model,
|
|
24
|
+
installs dependencies, initializes Git, and starts the development server. Pass
|
|
25
|
+
`--channel-web-nextjs` to add the Web Chat application. Stop the server before
|
|
26
|
+
editing the generated agent. The command does not create a Vercel project or
|
|
27
|
+
deploy.
|
|
24
28
|
|
|
25
|
-
|
|
29
|
+
The target can also be an existing project directory (`eve init .`): the
|
|
30
|
+
project must have a `package.json`, the `agent/` files must not exist yet, and
|
|
31
|
+
the missing `eve`, `ai`, and `zod` dependencies are added without touching
|
|
32
|
+
anything else the project owns. Either way the final handoff runs the `eve dev`
|
|
33
|
+
binary through the project's package manager, never the project's own `dev`
|
|
34
|
+
script.
|
|
35
|
+
|
|
36
|
+
To add only the dependency to an existing app instead:
|
|
26
37
|
|
|
27
38
|
```bash
|
|
28
|
-
|
|
39
|
+
npm install eve@latest
|
|
29
40
|
```
|
|
30
41
|
|
|
31
42
|
## What's In Your Project
|
|
@@ -38,13 +49,13 @@ The scaffold writes two files; you add tools as you need them.
|
|
|
38
49
|
You are a concise assistant. Use tools when they are available.
|
|
39
50
|
```
|
|
40
51
|
|
|
41
|
-
`agent/agent.ts`
|
|
52
|
+
`agent/agent.ts` holds runtime config:
|
|
42
53
|
|
|
43
54
|
```ts
|
|
44
55
|
import { defineAgent } from "eve";
|
|
45
56
|
|
|
46
57
|
export default defineAgent({
|
|
47
|
-
model: "anthropic/claude-
|
|
58
|
+
model: "anthropic/claude-sonnet-4.6",
|
|
48
59
|
});
|
|
49
60
|
```
|
|
50
61
|
|
|
@@ -76,7 +87,7 @@ Tools run in your app runtime with full `process.env`, not inside the [sandbox](
|
|
|
76
87
|
From the app root:
|
|
77
88
|
|
|
78
89
|
```bash
|
|
79
|
-
|
|
90
|
+
npm run dev
|
|
80
91
|
```
|
|
81
92
|
|
|
82
93
|
Useful commands:
|
|
@@ -153,10 +164,10 @@ See [Sessions, runs & streaming](./advanced/sessions-runs-and-streaming) for the
|
|
|
153
164
|
|
|
154
165
|
If a coding agent (Claude Code, Cursor, and the like) is doing the setup, hand it this prompt:
|
|
155
166
|
|
|
156
|
-
<CopyPrompt text="Set up an Eve agent for the user. Eve is a filesystem-first TypeScript framework for durable agents, published as the npm package eve. Read its docs: once eve is installed they are bundled in the package at node_modules/eve/dist/docs/public; before eve is installed, read the published Introduction and Getting Started pages. If the project has no Eve app, scaffold one with `
|
|
167
|
+
<CopyPrompt text="Set up an Eve agent for the user. Eve is a filesystem-first TypeScript framework for durable agents, published as the npm package eve. Read its docs: once eve is installed they are bundled in the package at node_modules/eve/dist/docs/public; before eve is installed, read the published Introduction and Getting Started pages. If the project has no Eve app, scaffold one with `npx eve@latest init <name>`; add `--channel-web-nextjs` only when the user wants Web Chat. The init command installs dependencies, initializes Git, and starts the dev server, so run it in a controllable process and stop it before editing. To add Eve to an existing app, run `npm install eve@latest`. Make sure agent/agent.ts and agent/instructions.md exist, then add a first typed tool at agent/tools/get_weather.ts using defineTool from eve/tools with a Zod inputSchema and an inline execute. Start the dev server again, then exercise the HTTP API: create a session with POST /eve/v1/session, attach to GET /eve/v1/session/:id/stream, and send a follow-up with the returned continuationToken. Verify with the project's typecheck, adapt model and provider choices to the project, and do not commit unless the user asks.">
|
|
157
168
|
Set up an Eve agent: read the Eve docs (bundled at node_modules/eve/dist/docs/public once eve is
|
|
158
|
-
installed), scaffold with `
|
|
159
|
-
a typed tool at agent/tools/get_weather.ts, run it with `
|
|
169
|
+
installed), scaffold with `npx eve@latest init <name>` (or `npm install eve@latest` in an existing app), add
|
|
170
|
+
a typed tool at agent/tools/get_weather.ts, run it with `npm run dev`, then create a session, stream
|
|
160
171
|
it, and send a follow-up.
|
|
161
172
|
</CopyPrompt>
|
|
162
173
|
|
|
@@ -164,7 +175,8 @@ Once `eve` is a dependency, the full docs are bundled in the package, so the age
|
|
|
164
175
|
|
|
165
176
|
- Docs: `node_modules/eve/dist/docs/public/`
|
|
166
177
|
|
|
167
|
-
|
|
178
|
+
`eve init <name>` creates the base agent; `eve init .` adds one to an existing project. Add `--channel-web-nextjs` for Web Chat, or run
|
|
179
|
+
`eve channels add slack` later from an interactive terminal.
|
|
168
180
|
|
|
169
181
|
## What to read next
|
|
170
182
|
|
|
@@ -14,7 +14,9 @@ The `eve` binary (`bin: eve`) runs from your app root, and every command loads `
|
|
|
14
14
|
| `eve start` | Serve the built `.output/` app; prints the listening URL |
|
|
15
15
|
| `eve dev` | Start the local dev server and open the terminal UI |
|
|
16
16
|
| `eve dev <url>` | Connect the UI to an existing server URL (e.g. a remote deployment) instead of booting a local server |
|
|
17
|
-
| `eve
|
|
17
|
+
| `eve link` | Link the directory to a Vercel project and pull AI Gateway credentials |
|
|
18
|
+
| `eve deploy` | Deploy the agent to Vercel production (links first if needed) |
|
|
19
|
+
| `eve eval` | Run evals against the local app or a remote target |
|
|
18
20
|
| `eve channels add [kind]` | Scaffold a channel interactively, or by kind (`slack` \| `web`) |
|
|
19
21
|
| `eve channels list` | List user-authored channels |
|
|
20
22
|
|
|
@@ -77,7 +79,6 @@ Pass a bare URL as the only argument and the UI connects to that server instead
|
|
|
77
79
|
| `--host <host>` | Host interface to bind |
|
|
78
80
|
| `--port <port>` | Port to listen on (defaults to `$PORT`, then `3000`) |
|
|
79
81
|
| `-u, --url <url>` | Connect to an existing server URL instead of starting one |
|
|
80
|
-
| `--repl` | Use the classic line-based REPL instead of the terminal UI |
|
|
81
82
|
| `--no-ui` | Start the server without an interactive UI |
|
|
82
83
|
| `--name <name>` | Title shown in the terminal UI (defaults to the app folder name) |
|
|
83
84
|
| `--tools <mode>` | Tool-call rendering: `full` \| `collapsed` \| `auto-collapsed` \| `hidden` |
|
|
@@ -88,23 +89,44 @@ Pass a bare URL as the only argument and the UI connects to that server instead
|
|
|
88
89
|
| `--context-size <tokens>` | Model context window size, shown as a usage percentage |
|
|
89
90
|
| `--logs <mode>` | Server/agent logs to show: `all` \| `stderr` \| `none` |
|
|
90
91
|
|
|
92
|
+
Local dev keeps immutable runtime source snapshots under `.eve/dev-runtime/snapshots/` so in-flight sessions keep a consistent code revision while new prompts pick up rebuilds. `eve dev` prunes stale runtime snapshots and old local sandbox templates in the background on startup; stopping `eve dev` and deleting `.eve/dev-runtime/snapshots/` or `.eve/sandbox-cache/local/templates/` is safe when you want a manual cleanup.
|
|
93
|
+
|
|
94
|
+
## `eve link`
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
eve link
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Links the current directory to a Vercel project through interactive team/project pickers, then pulls the project's environment so an AI Gateway credential (`VERCEL_OIDC_TOKEN` or `AI_GATEWAY_API_KEY`) lands in `.env.local`, and verifies one actually did. Running it again re-links: the pickers always run, and the new choice wins. Interactive only — in CI, use `vercel link --project <name> --yes` instead. A running `eve dev` reloads env files automatically, so no restart is needed after the pull.
|
|
101
|
+
|
|
102
|
+
## `eve deploy`
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
eve deploy
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Deploys the agent to Vercel production (`vercel deploy --prod`), installing dependencies first and pulling environment variables after. An already-linked project deploys with or without a TTY (non-interactive runs pass the non-interactive `vercel` flags); an unlinked directory walks the `eve link` pickers when a terminal is present, and exits with guidance otherwise.
|
|
109
|
+
|
|
91
110
|
## `eve eval`
|
|
92
111
|
|
|
93
112
|
```bash
|
|
94
|
-
eve eval [
|
|
113
|
+
eve eval [evalId...] [--url <url>] [options]
|
|
95
114
|
```
|
|
96
115
|
|
|
116
|
+
Runs all discovered evals when no eval ids are given; ids match exactly or by directory prefix (`eve eval weather` runs everything under `evals/weather/`). Exits `0` when every eval passed its checks, `1` when any eval failed (a failed check, an execution error, or a `--strict` threshold miss), `2` on configuration errors.
|
|
117
|
+
|
|
97
118
|
| Flag | Effect |
|
|
98
119
|
| ----------------------- | ---------------------------------------------- |
|
|
99
|
-
| `--suite <id...>` | Suite IDs to run (repeatable) |
|
|
100
|
-
| `--all` | Run all discovered suites |
|
|
101
120
|
| `--url <url>` | Remote agent URL (skip local host startup) |
|
|
102
|
-
| `--
|
|
103
|
-
| `--
|
|
121
|
+
| `--tag <tag...>` | Run only evals carrying a tag |
|
|
122
|
+
| `--strict` | Below-threshold scores also fail the exit code |
|
|
123
|
+
| `--list` | Print discovered evals without running them |
|
|
124
|
+
| `--timeout <ms>` | Per-eval timeout in milliseconds |
|
|
125
|
+
| `--max-concurrency <n>` | Max concurrent eval executions (default 8) |
|
|
104
126
|
| `--json` | Output results as JSON |
|
|
105
|
-
| `--skip-report` | Skip
|
|
127
|
+
| `--skip-report` | Skip eval-defined reporters (e.g. Braintrust) |
|
|
106
128
|
|
|
107
|
-
See [Evals](../
|
|
129
|
+
See [Evals](../evals/overview) for authoring evals.
|
|
108
130
|
|
|
109
131
|
## Recommended loop
|
|
110
132
|
|
|
@@ -46,7 +46,8 @@ export default defineTool({
|
|
|
46
46
|
| `defineSandbox` | `eve/sandbox` | `agent/sandbox.ts` | [Sandbox](../sandbox) |
|
|
47
47
|
| `defineInstrumentation` | `eve/instrumentation` | `agent/instrumentation.ts` | [instrumentation.ts](../advanced/instrumentation) |
|
|
48
48
|
| `defineRemoteAgent` | `eve` | `agent/subagents/<id>/agent.ts` | [Remote agents](../advanced/remote-agents) |
|
|
49
|
-
| `
|
|
49
|
+
| `defineEval` | `eve/evals` | `evals/*.eval.ts` | [Evals](../evals/overview) |
|
|
50
|
+
| `defineEvalConfig` | `eve/evals` | `evals/evals.config.ts` | [Evals](../evals/overview) |
|
|
50
51
|
| `useEveAgent` | `eve/react`, `eve/vue`, `eve/svelte` | frontend | [Frontend](../frontend/overview) |
|
|
51
52
|
|
|
52
53
|
A few non-`define*` helpers round out the set: `disableTool` and `ExperimentalWorkflow` from `eve/tools` (see [Default harness](../advanced/default-harness)), the route verbs `GET`/`POST`/`PUT`/`PATCH`/`DELETE`/`WS` from `eve/channels`, the approval predicates `always`/`once`/`never` from `eve/tools/approval`, and the channel auth helpers `localDev`/`vercelOidc`/`placeholderAuth` from `eve/channels/auth`. To wrap a built-in tool, import its default value from `eve/tools/defaults` (`bash`, `readFile`, `writeFile`, `glob`, `grep`, `webFetch`, `webSearch`, `todo`, `loadSkill`).
|
|
@@ -83,7 +84,10 @@ A few non-`define*` helpers round out the set: `disableTool` and `ExperimentalWo
|
|
|
83
84
|
| `eve/context` | `defineState`, session and state types |
|
|
84
85
|
| `eve/sandbox` | `defineSandbox`, backends |
|
|
85
86
|
| `eve/instrumentation` | `defineInstrumentation`, `isChannel` |
|
|
86
|
-
| `eve/evals` | `
|
|
87
|
+
| `eve/evals` | `defineEval`, `defineEvalConfig`, eval types |
|
|
88
|
+
| `eve/evals/expect` | `includes`, `equals`, `matches`, `similarity` |
|
|
89
|
+
| `eve/evals/reporters` | `Braintrust`, `JUnit`, `EvalReporter` |
|
|
90
|
+
| `eve/evals/loaders` | `loadJson`, `loadYaml` |
|
|
87
91
|
| `eve/react`, `eve/vue`, `eve/svelte` | `useEveAgent` |
|
|
88
92
|
| `eve/next`, `eve/nuxt`, `eve/sveltekit` | framework bundler plugins |
|
|
89
93
|
| [`eve/client`](../client/overview) | `Client`, `ClientSession` |
|
|
@@ -130,13 +130,15 @@ export default defineSandbox({
|
|
|
130
130
|
});
|
|
131
131
|
```
|
|
132
132
|
|
|
133
|
+
In local development, `localBackend()` stores template and session files as normal directories under `.eve/sandbox-cache/local/`. Template directories are reused across sessions when the sandbox source, seed files, and `revalidationKey` still match; `eve dev` prunes stale local templates in the background. Session directories persist `/workspace` changes across turns for the same durable session.
|
|
134
|
+
|
|
133
135
|
You can write your own backend too: a `SandboxBackend` is just an object with a `name`, a `create`, and an optional `prewarm`. See the `SandboxBackend*` types on `eve/sandbox`.
|
|
134
136
|
|
|
135
137
|
## Lifecycle
|
|
136
138
|
|
|
137
139
|
There are two hooks, scoped differently:
|
|
138
140
|
|
|
139
|
-
- **`bootstrap({ use })`** is template-scoped and runs once when the template is built. Put reusable setup here, the kind every later session inherits: clone a baseline repo, install dependencies, seed files. Call `use()` to get a `SandboxSession`. Only filesystem state
|
|
141
|
+
- **`bootstrap({ use })`** is template-scoped and runs once when the template is built. Put reusable setup here, the kind every later session inherits: clone a baseline repo, install dependencies, seed files. Call `use()` to get a `SandboxSession`. Only template filesystem state and supported backend metadata carry into later sessions; config like network policy does not. If external inputs affect what bootstrap produces, set `revalidationKey: () => string` so Eve knows when to rebuild the template (authored sandbox source and seed contents are already tracked for you).
|
|
140
142
|
- **`onSession({ use, ctx })`** is durable-session-scoped and runs once per session. This is where per-session setup goes: network policy, resources, timeout, per-user credentials, one-time markers. Because it runs inside the active runtime context, it can read `ctx.session` and derive the current principal without baking credentials into the template. Call `use(opts?)` to get a `SandboxSession`; `opts` flow to the backend's update path after create.
|
|
141
143
|
|
|
142
144
|
```ts
|
|
@@ -99,10 +99,6 @@ toModelOutput(output) {
|
|
|
99
99
|
|
|
100
100
|
It receives the full, typed `execute` return and only affects the model. Channel event handlers and hooks still get the full output on `action.result`, so a channel can render rich platform output (Slack Block Kit, say) the model never sees. Return `{ type: "text", value }` for a summary, or `{ type: "json", value }` for a smaller object.
|
|
101
101
|
|
|
102
|
-
## Survive compaction with `onCompact`
|
|
103
|
-
|
|
104
|
-
When the harness [compacts history](./advanced/default-harness#compaction), it calls each tool's `onCompact(input, ctx)` in registration order and applies the merged result to the compacted session. Return nothing to leave things alone, `messages` to append to the compacted history, or a `sessionPatch` to shallow-merge into session state. It works on the projected output (what the model sees), and a throw fails the compaction step, so don't swallow errors.
|
|
105
|
-
|
|
106
102
|
## What to read next
|
|
107
103
|
|
|
108
104
|
- [Skills](./skills): on-demand procedures the model loads when relevant
|
|
@@ -24,7 +24,7 @@ export default defineMcpClientConnection({
|
|
|
24
24
|
|
|
25
25
|
`"warehouse"` is the UID you chose when registering the Connect client. By default this OAuth is user-scoped. Each end-user authorizes in their own browser, and Eve resolves that user's token before every tool call.
|
|
26
26
|
|
|
27
|
-
Connect is in private beta. Once enabled: `
|
|
27
|
+
Connect is in private beta. Once enabled: `npm install @vercel/connect`, create the Connect client (`vercel connect create <type> --name warehouse`), link it to your project, then `vercel link` and `vercel env pull` so `VERCEL_OIDC_TOKEN` is available locally. Full setup: [Connections](../connections).
|
|
28
28
|
|
|
29
29
|
## What the user sees
|
|
30
30
|
|
|
@@ -10,10 +10,13 @@ Step 1 gets it talking. The scaffold bundles a small sample dataset, so your fir
|
|
|
10
10
|
## Scaffold
|
|
11
11
|
|
|
12
12
|
```bash
|
|
13
|
-
|
|
13
|
+
npx eve@latest init analytics-assistant
|
|
14
14
|
```
|
|
15
15
|
|
|
16
|
-
The
|
|
16
|
+
The command writes the starter agent with Eve's default model and built-in HTTP API
|
|
17
|
+
channel (`agent/channels/eve.ts`), installs dependencies, initializes Git, and
|
|
18
|
+
starts the development server. Stop the server before continuing with the edits
|
|
19
|
+
below. It does not create a Vercel project or deploy.
|
|
17
20
|
|
|
18
21
|
## Set the model
|
|
19
22
|
|