@arnilo/prism 0.0.4 → 0.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/README.md +34 -10
- package/dist/agents.js +146 -19
- package/dist/cli-init.d.ts +41 -0
- package/dist/cli-init.js +390 -0
- package/dist/cli-runner.d.ts +7 -1
- package/dist/cli-runner.js +13 -1
- package/dist/content.d.ts +19 -0
- package/dist/content.js +197 -69
- package/dist/contracts.d.ts +94 -9
- package/dist/contracts.js +8 -0
- package/dist/feedback.d.ts +48 -0
- package/dist/feedback.js +230 -0
- package/dist/index.d.ts +6 -4
- package/dist/index.js +4 -3
- package/dist/providers/media.d.ts +3 -1
- package/dist/providers/media.js +11 -1
- package/dist/testing/feedback.d.ts +6 -0
- package/dist/testing/feedback.js +37 -0
- package/dist/testing/persistence-schema.d.ts +3 -3
- package/dist/testing/persistence-schema.js +32 -2
- package/dist/testing/run-ledger-conformance.js +7 -1
- package/docs/a2a.md +73 -0
- package/docs/agent-events.md +4 -6
- package/docs/agent-loops.md +1 -1
- package/docs/agent-session-runtime.md +14 -16
- package/docs/cli-rpc.md +35 -7
- package/docs/coding-agent-tools.md +2 -2
- package/docs/coding-security.md +7 -3
- package/docs/compaction-observational-memory.md +2 -0
- package/docs/context-and-skills.md +1 -0
- package/docs/credentials-and-redaction.md +2 -2
- package/docs/database-persistence.md +9 -6
- package/docs/evaluations.md +122 -0
- package/docs/extensions.md +2 -2
- package/docs/host-security.md +20 -3
- package/docs/index.md +29 -17
- package/docs/mcp-tools.md +49 -4
- package/docs/migration.md +33 -3
- package/docs/multimodal-content.md +14 -6
- package/docs/observability.md +14 -6
- package/docs/performance.md +209 -0
- package/docs/postgres-persistence.md +6 -4
- package/docs/provider-conformance.md +1 -0
- package/docs/provider-packages.md +2 -0
- package/docs/providers/ai-sdk.md +113 -0
- package/docs/public-contracts.md +6 -5
- package/docs/rag.md +113 -0
- package/docs/release-and-install.md +100 -77
- package/docs/review-coverage-2026-07-15.md +193 -0
- package/docs/runs-and-usage.md +41 -4
- package/docs/server.md +139 -0
- package/docs/settings-auth-trust-security.md +5 -5
- package/docs/sqlite-persistence.md +4 -3
- package/docs/supervisors.md +71 -0
- package/docs/workflow-orchestration-primitives.md +19 -3
- package/docs/workflows.md +97 -23
- package/docs/working-and-semantic-memory.md +169 -0
- package/package.json +12 -2
- package/templates/init/README.md.tmpl +28 -0
- package/templates/init/env.example.tmpl +1 -0
- package/templates/init/gitignore.tmpl +11 -0
- package/templates/init/optional/evals-example.ts.tmpl +17 -0
- package/templates/init/optional/workflows-example.ts.tmpl +27 -0
- package/templates/init/package.json.tmpl +22 -0
- package/templates/init/providers.json +76 -0
- package/templates/init/src/agent.ts.tmpl +10 -0
- package/templates/init/src/index.ts.tmpl +12 -0
- package/templates/init/src/tests/agent.test.ts.tmpl +24 -0
- package/templates/init/tsconfig.json.tmpl +15 -0
package/docs/index.md
CHANGED
|
@@ -6,12 +6,13 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
6
6
|
- [Public contracts](public-contracts.md): type shapes for messages, agents, tools, stores, generic `CheckpointStore`, atomic `LeaseStore`, bounded `EventMultiplexer`, resources, credentials, and events.
|
|
7
7
|
|
|
8
8
|
## Agent/session runtime
|
|
9
|
-
- [Agent/session runtime](agent-session-runtime.md): create agents and sessions,
|
|
9
|
+
- [Agent/session runtime](agent-session-runtime.md): create agents and sessions, get direct `AgentRunResult` values from `run`/`prompt`, use integrated `stream()`, and subscribe to normalized events. Covers tool-call loop transcript shape and prior-reasoning preservation across turns.
|
|
10
10
|
- [Agent definitions](agent-definitions.md): resolve declarative `AgentDefinition` values via `resolveAgentDefinition`, and turn app-config `<configRoot>/agents/<name>/AGENT.md` bundles into runnable agents via `discoverAgentBundles` / `resolveAgentBundle` (explicit tool/skill activation by name, fail-closed omitted capabilities, migration-only `activateAllCapabilities`, strict duplicate scope checks, configurable prompt layers, no auto-discovery).
|
|
11
11
|
- [Agent loops](agent-loops.md): replaceable per-run control loops — `singleShotLoop` default and `generate-validate-revise` with host-supplied `validator`/`parser`/`repairer` callbacks.
|
|
12
12
|
- [Agent events](agent-events.md): the `AgentEvent` stream — agent/turn/message (including live `tool_call_delta` fragments), provider turn timing, tool execution, queue/subscriber overflow, compaction/retry, artifact validation/refinement, and error variants, redacted via `redactAgentEvent`.
|
|
13
|
-
- [Observability](observability.md): metadata-only
|
|
14
|
-
- [
|
|
13
|
+
- [Observability](observability.md): metadata-only provider/tool and run-feedback/evaluation projection, terminal span cleanup, low-cardinality metrics, and optional `@arnilo/prism-observability-opentelemetry` adapter.
|
|
14
|
+
- [Evaluations](evaluations.md): optional deterministic scorers/datasets/experiments plus ID-only linkage from evaluation records to immutable owned run feedback.
|
|
15
|
+
- [Runs and usage ledger](runs-and-usage.md): durable run/event/tool/usage persistence plus bounded immutable run/trace feedback, evaluation links, ownership, redaction, query, and deletion semantics.
|
|
15
16
|
- [Performance limits](performance.md): bounded live subscriber queues, branch-read pagination expectations, JSONL/dev-store limits, and production sizing assumptions.
|
|
16
17
|
- [Structured output](structured-output.md): the `Artifact*` seam plus provider-native `StructuredOutputOptions` / `structuredOutputMode` for capable models.
|
|
17
18
|
|
|
@@ -19,12 +20,13 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
19
20
|
- [Compaction and retry policies](compaction-and-retry.md): summarize branch history and retry transient provider failures with host-replaceable policies.
|
|
20
21
|
- [LLM compaction package](compaction-llm.md): optional provider-backed compaction strategy package with max-output budgets mapped through `model.parameters.maxTokens` to provider wire fields.
|
|
21
22
|
- [Observational memory compaction package](compaction-observational-memory.md): optional source-backed memory, owned runtime append callback, provider-valid worker transcripts, fast compaction, recall tool, and status/view command package.
|
|
23
|
+
- [Working and semantic memory](working-and-semantic-memory.md): optional `@arnilo/prism-memory` working-memory store, semantic recall, Embedder/VectorStore contracts, in-memory adapters, and PostgreSQL/pgvector path.
|
|
22
24
|
- [Session stores](session-stores.md): `SessionStore` contract, `SessionAppendOptions`, `SessionAppendConflictError`, branch handles, `readBranchPath`, and dev-vs-production branch reads — start here for session persistence.
|
|
23
25
|
- [Session stores and branching](session-stores-and-branching.md): detailed branch semantics and helper reference (kept for compatibility; links back to the canonical atomic append / branch-handle sections).
|
|
24
26
|
- [Database persistence](database-persistence.md): production persistence contracts, shared schema/migration primitives (`@arnilo/prism/testing/persistence-schema`), conditional append transaction pattern, idempotency indexes, `readBranchPath`, reference relational schema, retention, migrations, and NoSQL mapping.
|
|
25
|
-
- [SQLite persistence](sqlite-persistence.md): optional
|
|
26
|
-
- [PostgreSQL persistence](postgres-persistence.md): optional pooled
|
|
27
|
-
- [Migration guide](migration.md): 0.0.3 compatibility and
|
|
27
|
+
- [SQLite persistence](sqlite-persistence.md): optional adapter with session/run storage, generic checkpoints/leases, and migration-003 owned run feedback over `better-sqlite3`.
|
|
28
|
+
- [PostgreSQL persistence](postgres-persistence.md): optional pooled session/run/checkpoint/lease/feedback adapter over `pg`, with advisory-lock migrations and opt-in live conformance.
|
|
29
|
+
- [Migration guide](migration.md): 0.0.3 compatibility and 0.0.5 adoption — first-party/custom database persistence plus explicit fail-closed tool/skill activation.
|
|
28
30
|
- [Node JSONL session store](node-jsonl-session-store.md): development-only JSONL file adapter for single-process Node hosts; no cross-process safety.
|
|
29
31
|
- [Persistence, credentials, and multimodality primitives](persistence-credentials-multimodality-primitives.md): Plan 056 inventory — session/run-ledger/persistence contracts, credential/OAuth seams, content/resource/model capabilities, package dependency matrix, conformance matrix, and threat model for production adapters.
|
|
30
32
|
|
|
@@ -36,23 +38,25 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
36
38
|
- [Provider request policies](provider-request-policies.md): chain `ProviderRequestPolicy` hooks, use `createSessionCachePolicy`, and merge legacy/structured cache options safely.
|
|
37
39
|
- [Provider packages](provider-packages.md): define explicit provider packages, model metadata, auth descriptors, request/cache policies, and provider-owned header precedence without package discovery or provider-specific core behavior; includes a first-party cache behavior summary.
|
|
38
40
|
- Phase 12 package workspaces: [`@arnilo/prism-provider-openai`](providers/openai.md), [`@arnilo/prism-provider-opencode-go`](providers/opencode-go.md), [`@arnilo/prism-provider-openrouter`](providers/openrouter.md), [`@arnilo/prism-provider-zai`](providers/zai.md), [`@arnilo/prism-provider-kimi`](providers/kimi.md), and [`@arnilo/prism-provider-neuralwatt`](providers/neuralwatt.md) with implicit vLLM prefix caching, reasoning controls (`reasoning_effort`/`thinking_token_budget`/`enable_thinking`/`preserve_thinking`/`clear_thinking`), reasoning preservation, OpenAI-style tool-call loop, quota, telemetry, and retry classification helpers.
|
|
41
|
+
- Optional AI SDK adapter: [`@arnilo/prism-provider-ai-sdk`](providers/ai-sdk.md) maps host-owned `LanguageModelV4` models onto Prism `AIProvider` streams (specification v4; available directly or through provider/all umbrellas).
|
|
39
42
|
- [OpenAI-compatible provider](providers/openai-compatible.md): optional provider subpath using native or injected `fetch` for Chat Completions streaming.
|
|
40
43
|
|
|
41
44
|
## Input, prompt, and context assembly
|
|
42
45
|
- [SDK customization guide](customization.md): map provider resolution, middleware, context, builders, injectors, loops, compaction, retry, stores, and skills to explicit host-wired APIs.
|
|
43
46
|
- [Input and prompt assembly](input-and-prompt-assembly.md): render tiny prompt templates and turn common host input, history, attachments, explicit resources, summaries, and tool results into messages with replaceable builders, provider-input assembly, legacy default order, and opt-in cache-aware ordering. Audio/file/document `ContentBlock` types and capability checks are documented there.
|
|
44
|
-
- [Multimodal content](multimodal-content.md):
|
|
47
|
+
- [Multimodal content](multimodal-content.md): complete-request media resolution and aggregate bounds, DNS-classified/address-pinned URLs, SSRF/MIME policy, and `ModelCapabilities.input` tags.
|
|
45
48
|
- [System prompts](system-prompts.md): compose explicit user/package/app/run system prompt layers, auto-load the standard `AGENTS.md` (workspace) / `SYSTEM.md` prompt files via the Node `loadSystemPromptFiles` loader (trust-gated for `AGENTS.md`), and append `SYSTEM.md` → per-agent `AGENT.md` body → repo `AGENTS.md` layers from a discovered agent bundle via `resolveAgentBundle`.
|
|
46
49
|
- [Instruction injection](instruction-injection.md): register package injectors that layer redacted instructions/context blocks without granting tools, permissions, or resource escapes.
|
|
47
50
|
- [Context and skills](context-and-skills.md): resolve ordered context providers and keep context/skill selection host-owned; omitted declarative skills stay inactive by default, `toolNames` fail closed before provider turns, and strict skill registries prevent silent shadowing.
|
|
51
|
+
- [Retrieval-augmented generation](rag.md): optional bounded text/Markdown chunking, Phase 7 vector indexing/retrieval, stable citations, and explicit inert context injection.
|
|
48
52
|
|
|
49
53
|
## Tools
|
|
50
54
|
- [Tools](tools.md): register host-owned active tools with replace-or-error duplicate policy, apply exact allow/deny filtering, and dispatch tool calls.
|
|
51
55
|
- [Tool execution primitives](tool-execution-primitives.md): JSON Schema validation, exclusive-aware bounded parallel dispatch, MCP bridge mapping, coding execution policy, and image-read bounds.
|
|
52
56
|
- [Tool validator JSON Schema package](../packages/tool-validator-json-schema/README.md): optional `@arnilo/prism-tool-validator-json-schema` adapter for `tool.parameters`.
|
|
53
|
-
- [MCP client bridge](mcp-tools.md): optional `@arnilo/prism-mcp` package mapping remote
|
|
57
|
+
- [MCP client bridge and server exposure](mcp-tools.md): optional `@arnilo/prism-mcp` package mapping remote tools into Prism and explicitly selected Prism tools/commands onto SDK `McpServer`.
|
|
54
58
|
- [Coding agent tools](coding-agent-tools.md): optional first-party package `@arnilo/prism-coding-agent` providing `shell`, `read`, `write`, and `edit` tools (ported from pi) as `ToolDefinition`s a host registers; pluggable operation backends, per-path mutation serialization, optional `ExecutionPolicy`, bounded image reads (`maxImageBytes`, `transformImage`), and read-only/coding aggregators. Host shell/filesystem access — gate with permission/trust policies and `@arnilo/prism-coding-security` approval.
|
|
55
|
-
- [Coding execution approval and sandboxing](coding-security.md):
|
|
59
|
+
- [Coding execution approval and sandboxing](coding-security.md): path/command approval, identity-scoped caching, shell-turn exclusivity, and abort-aware streaming sandbox adapters for coding tools.
|
|
56
60
|
|
|
57
61
|
## Extensions/plugins
|
|
58
62
|
- [Contribution discovery (workspace)](contribution-discovery.md): opt-in, realpath-contained directory scanner turning `SKILL.md`/`manifest.json` into inert `DiscoveredContribution` envelopes the host registers — no `import()`, no auto-activate, no provider scanning. Per-agent bundles remain app-controlled and are documented under Agent/session runtime.
|
|
@@ -66,29 +70,37 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
66
70
|
- [Node filesystem config loader](node-filesystem-config.md): explicitly read caller-named JSON config files in Node hosts.
|
|
67
71
|
- [Resource loading](resource-loading.md): decode text, JSON, binary, and manifest resources through caller-provided loaders with bounded byte limits.
|
|
68
72
|
|
|
73
|
+
## Server/API
|
|
74
|
+
- [Web-standard server handler](server.md): optional framework-free authorized direct/SSE agent and durable workflow run/status/cancel/resume routes with explicit bounds and zero default exposure.
|
|
75
|
+
|
|
76
|
+
## Multi-agent and interoperability
|
|
77
|
+
- [Supervisor delegation](supervisors.md): optional explicit child allow-list, derived memory scopes, narrowing-only permissions, lifecycle hooks, nested delegation, cancellation, and finite budgets.
|
|
78
|
+
- [A2A interoperability](a2a.md): optional A2A 1.0 cards, ES256 signatures, authorized JSON-RPC/SSE handler, and exact-origin remote client.
|
|
79
|
+
|
|
69
80
|
## CLI/RPC
|
|
70
|
-
- [CLI/RPC](cli-rpc.md): Run print/json modes and LF-delimited RPC over the public AgentSession runtime, including branch-handle results, fixed `forkSession`, and `checkout`.
|
|
71
|
-
- [Workflows](workflows.md): optional `@arnilo/prism-workflows` typed bounded DAG orchestration —
|
|
81
|
+
- [CLI/RPC](cli-rpc.md): Run print/json modes and LF-delimited RPC over the public AgentSession runtime, including branch-handle results, fixed `forkSession`, and `checkout`. `prism init` scaffolds a tiny TypeScript project with one selected provider and an offline mock test.
|
|
82
|
+
- [Workflows](workflows.md): optional `@arnilo/prism-workflows` typed bounded DAG orchestration — durable human suspend/resume, schedules/background execution, nested workflows, bounded validated state, immutable-lineage replay, multi-process coordination, events, and optional RPC/Web bindings. Interactive TUI (C-012) deferred.
|
|
72
83
|
- [Workflow orchestration primitives](workflow-orchestration-primitives.md): architecture inventory — workflow adapters consume core `CheckpointStore`, `LeaseStore`, and bounded `EventMultiplexer`; run control and optional RPC commands stay package-local.
|
|
73
|
-
- [Workflow/TUI scope](workflow-tui-primitives.md): records why 0.0.
|
|
84
|
+
- [Workflow/TUI scope](workflow-tui-primitives.md): records why 0.0.5 ships workflow APIs/RPC control but no interactive terminal UI.
|
|
74
85
|
|
|
75
86
|
## Security and credentials
|
|
76
|
-
- [Host security guide](host-security.md): fail-closed checklist for credentials, settings, redaction, trust roots, permission policies, persistence, extension loading, and tool validation.
|
|
77
|
-
- [Security/auth/trust](settings-auth-trust-security.md): settings providers, credential helpers, trust/permission policies, redaction controls, host-owned `AgentConfig
|
|
78
|
-
- [Credentials and redaction](credentials-and-redaction.md): compose explicit credential resolver order, use caller-supplied env objects/OAuth refresh helpers,
|
|
87
|
+
- [Host security guide](host-security.md): fail-closed checklist for credentials, settings, redaction, trust roots, remote media, permission/approval policies, persistence, extension loading, and tool validation.
|
|
88
|
+
- [Security/auth/trust](settings-auth-trust-security.md): settings providers, credential helpers, trust/permission policies, redaction controls, host-owned settings/credentials wiring outside `AgentConfig`, and security-boundary hardening summary.
|
|
89
|
+
- [Credentials and redaction](credentials-and-redaction.md): compose explicit credential resolver order, use caller-supplied env objects/OAuth refresh helpers, resolve credentials only at the provider edge, and redact known secret values.
|
|
79
90
|
- [Credential storage](credential-storage.md): optional `@arnilo/prism-credentials-node` encrypted-file and system-keychain adapters for durable host-owned credentials.
|
|
80
91
|
|
|
81
92
|
## Testing and examples
|
|
82
93
|
- Provider test doubles: `createMockProvider()` and provider event helpers are documented on the canonical Provider layer page above.
|
|
83
94
|
- [Provider conformance](provider-conformance.md): run network-free provider adapter assertions (stream order, abort, tool-call reconstruction, cache usage, content coverage, protected header ownership, secret leak) from `@arnilo/prism/testing/provider-conformance`.
|
|
84
95
|
- [Session store conformance](session-store-conformance.md): assert any `SessionStore` adapter satisfies append/idempotency/conflict/branch invariants from `@arnilo/prism/testing/session-store-conformance`.
|
|
85
|
-
- [Run ledger conformance](run-ledger-conformance.md): assert
|
|
96
|
+
- [Run ledger conformance](run-ledger-conformance.md): assert durable run/event/tool/usage writes and reopen survival. Run-feedback stores use `@arnilo/prism/testing/feedback` for append/query/delete/ownership linkage conformance.
|
|
86
97
|
- [Compaction conformance](compaction-conformance.md): assert any `CompactionStrategy` returns a non-empty redacted summary and observes abort from `@arnilo/prism/testing/compaction-conformance`.
|
|
87
98
|
- [Tool conformance](tool-conformance.md): assert the tool-dispatch blocked-reason matrix (unknown/denied/invalid/permission/validator) and success path from `@arnilo/prism/testing/tool-conformance`.
|
|
88
99
|
- [Extension conformance](extension-conformance.md): assert an `Extension` setup runs, contributions stay inert, and setup errors are redacted or rethrown from `@arnilo/prism/testing/extension-conformance`.
|
|
89
100
|
- `examples/`: compile-checked typed examples and runnable mock demos (SDK basics, provider registration, auth, tools, cache-aware prompt assembly, NeuralWatt agent run, stores/branching, compaction, observational-memory recall, structured-output/artifact-loop, CLI, RPC, workflow orchestration).
|
|
90
101
|
|
|
91
102
|
## Release and install
|
|
92
|
-
- [Release and install](release-and-install.md):
|
|
103
|
+
- [Release and install](release-and-install.md): 30-package graph and profiles, install/tarball rules, deterministic resumable provenance publication, and offline test budget.
|
|
104
|
+
- [Review coverage (2026-07-15)](review-coverage-2026-07-15.md): frozen 0.0.5 finding/feature ownership, existing-primitive inventory, package decisions, threat boundaries, exclusions, and measured Phase 0 baseline.
|
|
93
105
|
- [Review coverage (2026-07-14)](review-coverage-2026-07-14.md): traceability matrix linking review findings and bug-report fixes to plan tasks, tests, and documentation for release 0.0.4.
|
|
94
106
|
|
package/docs/mcp-tools.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
# MCP client bridge
|
|
1
|
+
# MCP client bridge and server exposure
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-mcp` connects
|
|
5
|
+
`@arnilo/prism-mcp` has two explicit directions. Its client bridge connects hosts to remote [Model Context Protocol](https://modelcontextprotocol.io) servers and maps discovered tools to ordinary `ToolDefinition`s. Its server API registers selected Prism `ToolDefinition` and `CommandDefinition` values on the official SDK `McpServer`, with required authorization and a bounded optional Web-standard Streamable HTTP handler. The package wraps `@modelcontextprotocol/sdk` v1.29+ and adds no MCP branch to core Prism.
|
|
6
6
|
|
|
7
7
|
Primary API:
|
|
8
8
|
|
|
@@ -21,11 +21,37 @@ await bridge.close(); // close client + transport
|
|
|
21
21
|
|
|
22
22
|
Advanced hosts that manage their own `Client` + `Transport` can call `attachMcpToolBridge(client, transport, options)` after `client.connect(transport)`.
|
|
23
23
|
|
|
24
|
+
Server direction:
|
|
25
|
+
|
|
26
|
+
```ts
|
|
27
|
+
import { createPrismMcpServer, createPrismMcpWebHandler } from "@arnilo/prism-mcp";
|
|
28
|
+
|
|
29
|
+
const server = createPrismMcpServer({
|
|
30
|
+
tools: [approvedTool],
|
|
31
|
+
commands: [approvedWorkflowStatusCommand],
|
|
32
|
+
authorize: async ({ authInfo, kind, name }) => hostPolicy(authInfo, kind, name)
|
|
33
|
+
? { allowed: true, ownership: { tenantId: "tenant-1" } }
|
|
34
|
+
: false,
|
|
35
|
+
validate,
|
|
36
|
+
permission,
|
|
37
|
+
redactor,
|
|
38
|
+
});
|
|
39
|
+
|
|
40
|
+
const handleMcp = await createPrismMcpWebHandler(server, {
|
|
41
|
+
resolveAuthInfo: authenticateRequest,
|
|
42
|
+
allowedHosts: ["api.example.test"],
|
|
43
|
+
allowedOrigins: ["https://app.example.test"],
|
|
44
|
+
});
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
`McpServer.connect(transport)` remains available for SDK stdio or in-memory transports. The helper uses SDK `WebStandardStreamableHTTPServerTransport` in bounded stateless JSON-response mode; it does not start a listener.
|
|
48
|
+
|
|
24
49
|
## When to use it
|
|
25
50
|
|
|
26
51
|
- **Integrate external MCP tool servers** (filesystem, databases, SaaS adapters) without reimplementing JSON-RPC transports in your app.
|
|
27
52
|
- **Keep core dispatch gates** — register returned tools and let `dispatchToolCall` enforce permission, JSON Schema validation (`ToolValidator`), middleware, abort, and parallel execution (Plan 055 Tasks 1–2).
|
|
28
53
|
- **Explicit lifecycle** — connect, refresh on `notifications/tools/list_changed`, and `close()` when the session ends.
|
|
54
|
+
- **Expose selected capabilities** — register a reviewed tool/command allow-list for MCP clients without a custom JSON-RPC server.
|
|
29
55
|
|
|
30
56
|
Do **not** use this package as a sandbox, permission engine, or auto-discovery loader. Hosts must trust configured commands/URLs and gate registration.
|
|
31
57
|
|
|
@@ -37,6 +63,8 @@ Do **not** use this package as a sandbox, permission engine, or auto-discovery l
|
|
|
37
63
|
|
|
38
64
|
The resolved `McpToolBridge` exposes `tools`, `refresh()`, and `close()`. Each discovered MCP tool becomes a normal Prism `ToolDefinition`; calls return `ToolResult`, with remote `isError` mapped to `ToolResult.error`. List-change notifications invalidate the cache but register nothing automatically.
|
|
39
65
|
|
|
66
|
+
`createPrismMcpServer()` returns the SDK `McpServer`. It lists only passed tools/commands; JSON Schema parameters are converted through installed Zod v4 for SDK validation, then Prism tool calls still pass through `dispatchToolCall` permission/validator/redactor gates. Command definitions support explicitly selected direct/background/replay workflow operations and optional ownership-scoped schedule operations from `createWorkflowCommands()`; none are registered unless the host passes those command definitions. Calls return bounded MCP text content and `isError` on denial/failure. `createPrismMcpWebHandler()` returns `(Request) => Promise<Response>`.
|
|
67
|
+
|
|
40
68
|
## Request/response example
|
|
41
69
|
|
|
42
70
|
```json
|
|
@@ -114,6 +142,19 @@ The host explicitly chooses the executable, arguments, environment, and working
|
|
|
114
142
|
|
|
115
143
|
Only `http:` and `https:` URLs are accepted. Authentication (Bearer tokens, cookies) is supplied through `requestInit.headers` by the host.
|
|
116
144
|
|
|
145
|
+
### MCP server options
|
|
146
|
+
|
|
147
|
+
| Option | Default | Purpose |
|
|
148
|
+
| --- | --- | --- |
|
|
149
|
+
| `tools` / `commands` | empty | Explicit allow-list; zero default exposure |
|
|
150
|
+
| `authorize` | required | Per-call host authz using SDK auth/session metadata |
|
|
151
|
+
| `permission` / `validate` / `redactor` | none | Core tool-dispatch gates and known-secret redaction |
|
|
152
|
+
| `maxResultBytes` | 1 MiB (8 MiB hard) | Bound mapped MCP call output |
|
|
153
|
+
| `maxConcurrentCalls` | 16 (256 hard) | Bound active tool/command execution |
|
|
154
|
+
| `callTimeoutMs` | 60 s (30 min hard) | Abort and return timed-out calls |
|
|
155
|
+
|
|
156
|
+
Web handler defaults: 1 MiB request (8 MiB hard), 2 MiB response (16 MiB hard), 32 concurrent requests (512 hard), and 60 s timeout (30 min hard). It parses bounded JSON before passing `parsedBody` to the SDK transport. `allowedHosts`/`allowedOrigins` activate SDK DNS-rebinding checks only when explicitly configured. Authentication data comes only from host `resolveAuthInfo()`.
|
|
157
|
+
|
|
117
158
|
## Security and performance notes
|
|
118
159
|
|
|
119
160
|
| Risk | Mitigation |
|
|
@@ -123,15 +164,19 @@ Only `http:` and `https:` URLs are accepted. Authentication (Bearer tokens, cook
|
|
|
123
164
|
| Tool-name shadowing | Prefixed names + `createToolRegistry({ duplicate: "error" })` |
|
|
124
165
|
| Oversized server output | `maxResultBytes` on content mapping |
|
|
125
166
|
| Unvalidated arguments | Register tools with `createJsonSchemaToolArgumentValidator()` at dispatch |
|
|
126
|
-
| Missing permission gate | `PermissionPolicy` on `tool:mcp:<serverId>:<name>:execute`
|
|
167
|
+
| Missing permission gate | Client direction: `PermissionPolicy` on `tool:mcp:<serverId>:<name>:execute`; server direction: required MCP `authorize` plus optional core `PermissionPolicy` |
|
|
168
|
+
| Accidental server exposure | Empty default arrays, duplicate-name rejection, explicit tools/commands only |
|
|
169
|
+
| Unbounded MCP HTTP | Bounded pre-parsed JSON, response bytes, concurrent requests, call timeout, SDK web-standard transport |
|
|
170
|
+
| Cross-tenant operation | Authorizer derives ownership from validated auth and passes it to tool dispatch/selected workflow commands; never trust arguments as identity |
|
|
127
171
|
|
|
128
|
-
MCP output is untrusted. Apply `SecretRedactor` and host logging policy to `ToolResult` before persisting or displaying.
|
|
172
|
+
MCP output is untrusted. Apply `SecretRedactor` and host logging policy to `ToolResult` before persisting or displaying. MCP server authorization does not replace tool `PermissionPolicy`, argument validation, coding `ExecutionPolicy`, workflow ownership checks, TLS, rate limiting, or sandboxing. A timed-out tool must cooperate with `AbortSignal` to stop side effects; the response is bounded even when untrusted code ignores abort.
|
|
129
173
|
|
|
130
174
|
## Related APIs
|
|
131
175
|
|
|
132
176
|
- [Tools](tools.md): registry, dispatch, validation
|
|
133
177
|
- [Tool execution primitives](tool-execution-primitives.md): Plan 055 design and conformance matrix
|
|
134
178
|
- [Host security guide](host-security.md): permission, trust, validation checklist
|
|
179
|
+
- [Web-standard server handler](server.md): agent/workflow HTTP routes and shared remote-boundary rules
|
|
135
180
|
- Package README: [`@arnilo/prism-mcp`](../packages/mcp/README.md)
|
|
136
181
|
|
|
137
182
|
## Testing
|
package/docs/migration.md
CHANGED
|
@@ -2,7 +2,28 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
Prism 0.0.
|
|
5
|
+
Prism 0.0.5 preserves documented 0.0.3 agent construction except for two intentional Phase 3 public-API cleanups:
|
|
6
|
+
|
|
7
|
+
1. **`session.run()` / `session.prompt()` return `AgentRunResult`** and `session.stream()` starts one owned run after subscribing. Callers that ignored the previous `Promise<void>` keep working; failed/aborted runs reject with `AgentRunError` (`.result` attached).
|
|
8
|
+
2. **`AgentConfig.extensions` / `settings` / `credentials` are removed.** Wire extensions through `createExtensionKernel()`, read settings in the host, and pass credential resolvers to the provider edge.
|
|
9
|
+
|
|
10
|
+
Phase 4 adds optional `@arnilo/prism-evals` for deterministic scorers/datasets/experiments. It is not a core dependency; install it directly or through `@arnilo/prism-all`.
|
|
11
|
+
|
|
12
|
+
Phase 5 adds `prism init <dir>` to the existing CLI. It scaffolds a tiny TypeScript project with one selected provider and an offline mock test. Optional `--with-workflows` / `--with-evals` flags add only those packages; storage and telemetry stay opt-in elsewhere.
|
|
13
|
+
|
|
14
|
+
Phase 6 adds optional `@arnilo/prism-provider-ai-sdk` for AI SDK `LanguageModelV4` interoperability. Install it with `@ai-sdk/provider@^4`, through `@arnilo/prism-providers`, or through `@arnilo/prism-all`; it is not a core dependency.
|
|
15
|
+
|
|
16
|
+
Phase 7 adds optional `@arnilo/prism-memory` for schema/template-backed working memory and embedding-based semantic recall. Install it directly or through `@arnilo/prism-all`; in-memory adapters are default, and PostgreSQL/pgvector is opt-in. It is not a core dependency.
|
|
17
|
+
|
|
18
|
+
Phase 8 extends `@arnilo/prism-workflows` compatibly. Nodes may return `suspend()`, and opted-in tool nodes may declare `approval`. Resuming a suspended run requires `{ decision, input?, expectedVersion }`; ordinary failed/aborted recovery resume remains unchanged. `WorkflowRunStatus` adds `suspended` and terminal `denied`. Suspension/resume records remain bounded checkpoint JSON, so SQLite/PostgreSQL require no migration.
|
|
19
|
+
|
|
20
|
+
Phase 9 adds optional `@arnilo/prism-rag` for bounded plain-text/Markdown chunking, Phase 7 vector indexing/retrieval, stable citations, and explicit context injection. Existing agents and memory stores are unchanged; install and attach its context provider explicitly. No database migration is required.
|
|
21
|
+
|
|
22
|
+
Phase 10 adds optional `@arnilo/prism-server` and extends `@arnilo/prism-mcp` with server-direction APIs. Existing agent/workflow/MCP client behavior is unchanged. Install the server package explicitly, pass selected capability maps plus required host authorization, and adapt its Web handler in the deployment host. MCP servers likewise register only passed tools/commands and require authorization. No listener, route, credential source, profile package, or database migration activates automatically.
|
|
23
|
+
|
|
24
|
+
Phase 11 compatibly extends `@arnilo/prism-workflows` with `workflowNode`, shared state fields/context updates, replay lineage, explicit background enqueue, and ownership-scoped schedules. Existing workflow definitions and direct runs remain valid; `WorkflowRunResult` now always includes `state`. State schemas require a host `validateState` callback. Schedules reuse generic checkpoint/lease stores, so SQLite/PostgreSQL need no migration and no scheduler starts automatically.
|
|
25
|
+
|
|
26
|
+
This page also covers two optional adoption paths:
|
|
6
27
|
|
|
7
28
|
1. **In-memory / JSONL → database-backed persistence** — replace the single-process development `SessionStore` with `@arnilo/prism-session-store-sqlite`, `@arnilo/prism-session-store-postgres`, or a host implementation, and optionally attach its durable `RunLedger`.
|
|
8
29
|
2. **Legacy permissive capability configuration → explicit activation** — name tools/skills and keep omitted capabilities fail-closed.
|
|
@@ -15,7 +36,7 @@ Read this page when:
|
|
|
15
36
|
|
|
16
37
|
- you are taking an app from the `createMemorySessionStore()` / `createJsonlSessionStore()` path to a multi-process, multi-tenant, or durable database backend;
|
|
17
38
|
- you are hardening an agent that previously relied on "every scoped tool/skill is active" and need to name capabilities explicitly;
|
|
18
|
-
- you are adopting 0.0.
|
|
39
|
+
- you are adopting 0.0.5 persistence, checkpoints/leases, workflows, structured output, multimodality, or explicit tool safety for the first time.
|
|
19
40
|
|
|
20
41
|
If you are new to Prism, start at [Session stores](session-stores.md) and [Agent/session runtime](agent-session-runtime.md) instead.
|
|
21
42
|
|
|
@@ -125,6 +146,8 @@ What you leave behind and why:
|
|
|
125
146
|
- `createMemorySessionStore()` — process-local maps; lost on restart, no cross-process locking. Keep for tests.
|
|
126
147
|
- `createJsonlSessionStore()` — single-process file adapter; reads are linear in file size, no cross-process lock, no durable idempotency table, two writers to the same file can race. Keep for local/dev only.
|
|
127
148
|
|
|
149
|
+
Prism 0.0.5 persistence adapters automatically apply additive schema step `002_usage_scope`, then `003_run_feedback`. Migration 003 creates immutable owned run/trace feedback with a run FK, cascade deletion, JSON tag/scorer/evaluation ID lists, and owner/run/trace cursor indexes. Existing rows are unchanged. Custom adapters may omit optional `ProductionPersistenceStore.feedback`; adopters implement `RunFeedbackStore` append/query/delete semantics and must verify exact linked-run ownership before insert.
|
|
150
|
+
|
|
128
151
|
See [Database persistence](database-persistence.md) for the full reference schema, indexes, conditional-append transaction pattern, retention, and NoSQL mapping; [Session stores](session-stores.md) for the `SessionStore` contract and branch helpers; [Session stores and branching](session-stores-and-branching.md) for branch semantics; [Runs and usage ledger](runs-and-usage.md) for the `RunLedger` record shapes and ordering rules.
|
|
129
152
|
|
|
130
153
|
### Migration 2 — permissive capability defaults → explicit capability activation
|
|
@@ -165,7 +188,7 @@ Runtime skill activation remains explicit: `RunOptions.activeSkills` narrows per
|
|
|
165
188
|
|
|
166
189
|
## Extension and configuration notes
|
|
167
190
|
|
|
168
|
-
- **Persistence remains host-configured.** Optional SQLite/PostgreSQL packages ship adapters and versioned setup, but hosts choose connection paths/pools, TLS, credentials, retention, tenant policy, and lifecycle. Core only consumes `SessionStore`, `RunLedger`, checkpoint, and lease contracts.
|
|
191
|
+
- **Persistence remains host-configured.** Optional SQLite/PostgreSQL packages ship adapters and versioned setup, but hosts choose connection paths/pools, TLS, credentials, retention, tenant policy, and lifecycle. Core only consumes `SessionStore`, `RunLedger`, feedback, checkpoint, and lease contracts.
|
|
169
192
|
- **`RunLedger` is not a `SessionStore` replacement.** Messages, branches, and session entries still flow through `SessionStore.append()`; the ledger records run/event/tool/usage facts. See [Runs and usage ledger](runs-and-usage.md).
|
|
170
193
|
- **Capability activation is config over code.** Every seam lives on `AgentDefinition` / `AgentDefinitionResolutionContext` / `RunOptions`; no auto-activation, no privilege grant. A declaration cannot grant permissions or bypass `toolNames`.
|
|
171
194
|
- **Migration order is decoupled.** You can adopt database persistence without changing capability activation, and vice versa. Both migrations are independent config swaps.
|
|
@@ -181,6 +204,13 @@ Runtime skill activation remains explicit: `RunOptions.activeSkills` narrows per
|
|
|
181
204
|
|
|
182
205
|
## Related APIs
|
|
183
206
|
|
|
207
|
+
- [Evaluations](evaluations.md): optional `@arnilo/prism-evals` scorers/datasets/experiments over `AgentRunResult`.
|
|
208
|
+
- [AI SDK provider adapter](providers/ai-sdk.md): optional `@arnilo/prism-provider-ai-sdk` `LanguageModelV4` bridge.
|
|
209
|
+
- [Working and semantic memory](working-and-semantic-memory.md): optional `@arnilo/prism-memory` working/semantic recall primitives.
|
|
210
|
+
- [Retrieval-augmented generation](rag.md): optional text/Markdown chunk, index, retrieval, and citation helpers.
|
|
211
|
+
- [Web-standard server handler](server.md): optional authorized agent/workflow HTTP routes.
|
|
212
|
+
- [Supervisor delegation](supervisors.md) and [A2A interoperability](a2a.md): optional install only; core agent/workflow behavior is unchanged. Child factories now receive package-derived memory IDs and narrowing permission, while remote endpoints require exact HTTPS origin allow-lists.
|
|
213
|
+
- [MCP client/server exposure](mcp-tools.md): selected MCP tools/commands and bounded Web transport.
|
|
184
214
|
- [Database persistence](database-persistence.md): production contracts, reference schema, indexes, conditional append, retention, migrations, and custom adapters.
|
|
185
215
|
- [SQLite persistence](sqlite-persistence.md): local durable first-party adapter and writer ceiling.
|
|
186
216
|
- [PostgreSQL persistence](postgres-persistence.md): pooled multi-process adapter, TLS/pool ownership, and live gate.
|
|
@@ -19,6 +19,7 @@ Do not embed provider upload IDs, tenant-scoped remote file IDs, or API-specific
|
|
|
19
19
|
```ts
|
|
20
20
|
import {
|
|
21
21
|
resolveMediaContentBlock,
|
|
22
|
+
resolveMediaContentBlocks,
|
|
22
23
|
assertSsrfAllowedUrl,
|
|
23
24
|
type AudioContent,
|
|
24
25
|
type FileContent,
|
|
@@ -42,6 +43,8 @@ const audio: AudioContent = {
|
|
|
42
43
|
await resolveMediaContentBlock(file, { bounds: { maxItemBytes: 10_000_000 } });
|
|
43
44
|
```
|
|
44
45
|
|
|
46
|
+
`ResolveMediaContentOptions` also exposes `ssrf`, `fetch`, `resolveHostname`, `requestUrl`, `loader`, `loadContext`, and `signal`. `MediaHostnameResolver`, `MediaHostAddress`, `MediaUrlRequester`, and `MediaUrlRequest` are exported for typed custom/test transports; ordinary callers need none of them.
|
|
47
|
+
|
|
45
48
|
Known `ModelCapabilities.input` tags are exported as `MODEL_INPUT_CAPABILITIES`:
|
|
46
49
|
|
|
47
50
|
| Tag | Block type | First-party mapping (declared capability required) |
|
|
@@ -54,10 +57,12 @@ Known `ModelCapabilities.input` tags are exported as `MODEL_INPUT_CAPABILITIES`:
|
|
|
54
57
|
|
|
55
58
|
## Outputs / response / events
|
|
56
59
|
|
|
57
|
-
- `resolveMediaContentBlock()` returns `{ mediaType, bytes, name?, durationMs?, transcript?, metadata? }`.
|
|
60
|
+
- `resolveMediaContentBlock()` resolves one item and returns `{ mediaType, bytes, name?, durationMs?, transcript?, metadata? }`.
|
|
61
|
+
- `resolveMediaContentBlocks()` is the complete-request path: it rejects item count/inline estimates before I/O, resolves each item within its bound, then enforces exact aggregate decoded bytes.
|
|
58
62
|
- `assertModelSupportsContentBlocks()` / `assertMessagesSupportModelCapabilities()` throw `UnsupportedModalityError` when a declared capability list omits the block modality.
|
|
59
63
|
- `assertMediaBlocksWithinBounds()` enforces per-item bytes, total request bytes, item count, and audio duration ceilings.
|
|
60
|
-
- `assertSsrfAllowedUrl()` rejects private
|
|
64
|
+
- `assertSsrfAllowedUrl()` rejects loopback, private, link-local, unspecified, multicast, IPv4-mapped private, and metadata literals/hosts unless explicitly allow-listed.
|
|
65
|
+
- Default URL resolution performs one bounded `dns.lookup(..., { all: true })`, rejects the whole hostname if any answer is non-public, then pins the selected public address into `http.request()`/`https.request()` so a second DNS lookup cannot rebind the connection.
|
|
61
66
|
- `sniffMediaMimeType()` / `assertDeclaredMediaTypeMatches()` compare declared MIME types to magic bytes.
|
|
62
67
|
- No events are emitted and no provider calls occur in these helpers.
|
|
63
68
|
|
|
@@ -125,17 +130,20 @@ try {
|
|
|
125
130
|
|
|
126
131
|
## Extension and configuration notes
|
|
127
132
|
|
|
128
|
-
- URL fetches use
|
|
133
|
+
- URL fetches use the DNS-classifying, address-pinned Node transport by default. `resolveHostname` and `requestUrl` are paired test/custom seams; `requestUrl` must connect to the supplied validated address while preserving the original URL hostname for HTTP Host/TLS verification.
|
|
134
|
+
- Supplying `fetch` is a trusted compatibility/custom-transport escape hatch: Prism still checks URL literals and host allow-lists, but the host-provided fetch owns DNS resolution, rebinding protection, redirects, proxies, TLS, auth, and logging.
|
|
129
135
|
- `resourceUri` resolution requires a caller-provided `ResourceLoader` and optional `ResourceLoadContext.permission` check.
|
|
130
136
|
- Local filesystem paths should use trust policies such as `createPathTrustPolicy()` before exposing URIs to loaders.
|
|
131
137
|
- Provider upload/create/delete lifecycles are provider-package-local. `@arnilo/prism-provider-openai` inlines files under 4 MiB as `data:<mediaType>;base64,...` `file_data`, otherwise uses a bounded per-run upload cache and best-effort `DELETE /v1/files` cleanup after each stream.
|
|
132
|
-
- Shared wire helpers live in `@arnilo/prism/providers/media` (`serializeOpenAIResponsesInputFile`, `serializePdfDocumentWireBlock`, `createBoundedUploadCache`).
|
|
138
|
+
- Shared wire helpers live in `@arnilo/prism/providers/media` (`resolveProviderMediaMessages`, `serializeOpenAIResponsesInputFile`, `serializePdfDocumentWireBlock`, `createBoundedUploadCache`). OpenAI Responses, Kimi, and OpenCode Go Anthropic routes resolve their complete media collection once before serialization or upload.
|
|
133
139
|
|
|
134
140
|
## Security and performance notes
|
|
135
141
|
|
|
136
|
-
- SSRF deny-by-default blocks loopback,
|
|
142
|
+
- SSRF deny-by-default blocks IPv4/IPv6 loopback, private/unique-local, link-local, unspecified, multicast, IPv4-mapped private, and cloud metadata targets. DNS answers are all classified before one public address is pinned; mixed public/private answers fail closed.
|
|
143
|
+
- `allowedHostnames` is an explicit trust override and may permit a private destination. `denyPrivateHosts: false` is broader and should be reserved for hosts that intentionally own private-network access.
|
|
144
|
+
- DNS lookup, connection, and body streaming share `fetchTimeoutMs` and caller abort; more than 32 resolved addresses, redirects, and oversized response bodies are rejected.
|
|
137
145
|
- MIME validation rejects common magic-byte spoofing; extensions alone are never trusted.
|
|
138
|
-
- Byte budgets use base64 size estimates before decode and re-check decoded
|
|
146
|
+
- Byte budgets use base64 size estimates before decode and re-check decoded bytes after every read/fetch. Complete-request resolution keeps at most the configured request budget plus one bounded item in memory and performs no provider upload/request until validation succeeds.
|
|
139
147
|
- Media errors omit raw bytes/base64 payloads from messages.
|
|
140
148
|
- Fetch readers are cancelled promptly after bound violations or abort signals.
|
|
141
149
|
|
package/docs/observability.md
CHANGED
|
@@ -11,6 +11,7 @@ APIs:
|
|
|
11
11
|
- `ProviderTurnMetadata`, `ToolExecutionMetadata` on `AgentEvent`
|
|
12
12
|
- `createProviderTurnMetadata()`, `readProviderHttpStatus()` in `@arnilo/prism`
|
|
13
13
|
- `createOpenTelemetryInstrumentation()`, `wrapOpenTelemetryApi()`, `createInMemoryTelemetry()` in `@arnilo/prism-observability-opentelemetry`
|
|
14
|
+
- `handleRunFeedback()` / `handleEvaluation()` for explicit safe post-run projection
|
|
14
15
|
|
|
15
16
|
## When to use it
|
|
16
17
|
|
|
@@ -58,7 +59,7 @@ const detach = telemetry.attachSession(session);
|
|
|
58
59
|
// or: for await (const event of session.subscribe()) telemetry.handleAgentEvent(event);
|
|
59
60
|
```
|
|
60
61
|
|
|
61
|
-
Set `enabled: false` or omit `tracer`/`meter` for a no-op adapter.
|
|
62
|
+
Set `enabled: false` or omit `tracer`/`meter` for a no-op adapter. Feedback handlers accept only `runId`, rating/score, booleans, bounded counts, and fixed status — never comment, tag values, scorer/evaluation IDs, or arbitrary metadata.
|
|
62
63
|
|
|
63
64
|
## Outputs / response / events
|
|
64
65
|
|
|
@@ -78,10 +79,13 @@ OpenTelemetry mapping (when enabled):
|
|
|
78
79
|
|
|
79
80
|
| Agent event | Span | Metric labels |
|
|
80
81
|
| --- | --- | --- |
|
|
81
|
-
| `agent_started` / `agent_finished` | `prism.agent.run` |
|
|
82
|
+
| `agent_started` / `agent_finished` / run `error` | `prism.agent.run` | `prism.run.tokens` on successful aggregate usage |
|
|
82
83
|
| `provider_turn_*` | `prism.provider.turn` | `provider_id`, `outcome` on duration histogram |
|
|
83
84
|
| `tool_execution_*` (terminal) | `prism.tool.execute` when started | `status` on duration histogram |
|
|
84
|
-
| `provider_turn_finished`
|
|
85
|
+
| `provider_turn_finished` usage | span attributes | `prism.provider.tokens` (`kind`: input/output/cache_*`) |
|
|
86
|
+
| `agent_finished` aggregate usage | — | `prism.run.tokens` (`kind`: input/output) |
|
|
87
|
+
| `handleRunFeedback` | active-run `prism.run.feedback` event or ended-run span | `prism.run.feedback` (`rating`, `linked_evaluation`) |
|
|
88
|
+
| `handleEvaluation` | active-run `prism.run.evaluation` event or ended-run span | `prism.run.evaluation` (`status`) |
|
|
85
89
|
|
|
86
90
|
High-cardinality identifiers (`sessionId`, `runId`, `requestId`, `toolCallId`) are **span attributes only**, never metric labels.
|
|
87
91
|
|
|
@@ -130,8 +134,10 @@ const session = createAgent({
|
|
|
130
134
|
}).createSession();
|
|
131
135
|
|
|
132
136
|
const detach = telemetry.attachSession(session);
|
|
133
|
-
await session.run("hello");
|
|
137
|
+
const result = await session.run("hello");
|
|
134
138
|
detach();
|
|
139
|
+
telemetry.handleRunFeedback({ runId: result.runId, rating: 1, hasComment: true, tagCount: 1, scorerCount: 1, evaluationCount: 1 });
|
|
140
|
+
telemetry.handleEvaluation({ runId: result.runId, status: "scored", score: 0.9, hasReason: true });
|
|
135
141
|
|
|
136
142
|
console.log(memory.spans.map((span) => span.name));
|
|
137
143
|
```
|
|
@@ -142,18 +148,20 @@ console.log(memory.spans.map((span) => span.name));
|
|
|
142
148
|
- `retry_scheduled` still signals backoff; each retry attempt emits its own `provider_turn_*` pair with `metadata.attempt`.
|
|
143
149
|
- NeuralWatt `neuralwatt:telemetry` provider events remain package-local; hosts may forward numeric cost/energy into custom metrics.
|
|
144
150
|
- `@arnilo/prism-observability-opentelemetry` is optional and included through `@arnilo/prism-sdk` and `@arnilo/prism-all`; instrumentation remains disabled until a host configures it.
|
|
145
|
-
- Exporter failures are isolated: instrumentation catches tracer/meter errors and invokes `onExporterError` without affecting the run.
|
|
151
|
+
- Exporter failures are isolated: instrumentation catches tracer/meter errors and invokes `onExporterError` without affecting the run, feedback persistence, or evaluation scoring.
|
|
152
|
+
- Run `error` events close every outstanding span attributable to that run. Detaching a session closes any remaining session spans; repeated terminal events are idempotent and cannot end a span twice.
|
|
146
153
|
- Disabled instrumentation performs no per-delta span work (`enabled: false` or missing tracer/meter).
|
|
147
154
|
|
|
148
155
|
## Security and performance notes
|
|
149
156
|
|
|
150
157
|
- Default events are metadata-only — no prompts, streamed deltas, tool arguments, or credentials.
|
|
151
158
|
- Opt-in content in other event types (`message_delta`, tool `result`) is still subject to `redactAgentEvent`.
|
|
152
|
-
- Metric labels stay low-cardinality (`provider_id`, `outcome`, `status`, token `kind
|
|
159
|
+
- Metric labels stay low-cardinality (`provider_id`, `outcome`, `status`, token `kind`, feedback rating bucket/link presence); never use `sessionId`/`runId`, comments, tag values, scorer/evaluation IDs, or arbitrary metadata as labels. Provider-turn and run-total tokens use distinct instruments, so one counter cannot double count both scopes.
|
|
153
160
|
- Target overhead when enabled is under 5% excluding exporter I/O; disabled hooks allocate no spans.
|
|
154
161
|
- Provider transport limits and redaction order are documented in [Provider primitives](provider-primitives.md).
|
|
155
162
|
|
|
156
163
|
## Related APIs
|
|
164
|
+
- [Evaluations](evaluations.md): optional scorers can link scores to run/session/trace IDs from agent events.
|
|
157
165
|
|
|
158
166
|
- [Agent events](agent-events.md): full `AgentEvent` union and subscriber semantics.
|
|
159
167
|
- [Runs and usage ledger](runs-and-usage.md): durable `AgentEventRecord` persistence.
|