@arnilo/prism 0.4.0 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +41 -1
- package/README.md +23 -20
- package/dist/agent-run-state.d.ts +1 -2
- package/dist/agent-run-state.js +0 -3
- package/dist/agent-session/session/assemble.d.ts +6 -0
- package/dist/agent-session/session/assemble.js +391 -0
- package/dist/agent-session/session/persist.d.ts +28 -0
- package/dist/agent-session/session/persist.js +166 -0
- package/dist/agent-session/session/provider-round.d.ts +6 -0
- package/dist/agent-session/session/provider-round.js +231 -0
- package/dist/agent-session/session/tool-round.d.ts +31 -0
- package/dist/agent-session/session/tool-round.js +473 -0
- package/dist/agent-session/session/types.d.ts +115 -0
- package/dist/agent-session/session/types.js +5 -0
- package/dist/agent-session/session.d.ts +49 -43
- package/dist/agent-session/session.js +24 -1180
- package/dist/capture.d.ts +63 -0
- package/dist/capture.js +67 -0
- package/dist/cli-init.d.ts +18 -2
- package/dist/cli-init.js +2 -7
- package/dist/cli-runner.d.ts +2 -2
- package/dist/cli-runner.js +45 -9
- package/dist/content.d.ts +3 -3
- package/dist/content.js +3 -1
- package/dist/contracts-core/agent.d.ts +4 -0
- package/dist/contracts-core/batch.d.ts +97 -0
- package/dist/contracts-core/batch.js +65 -0
- package/dist/contracts-core/content.d.ts +72 -1
- package/dist/contracts-core/embeddings.d.ts +30 -0
- package/dist/contracts-core/embeddings.js +17 -0
- package/dist/contracts-core/images.d.ts +60 -0
- package/dist/contracts-core/images.js +17 -0
- package/dist/contracts-core/moderation.d.ts +46 -0
- package/dist/contracts-core/moderation.js +34 -0
- package/dist/contracts-core/speech.d.ts +39 -0
- package/dist/contracts-core/speech.js +17 -0
- package/dist/contracts-core/transcription.d.ts +48 -0
- package/dist/contracts-core/transcription.js +17 -0
- package/dist/contracts-core/video.d.ts +61 -0
- package/dist/contracts-core/video.js +17 -0
- package/dist/contracts-core.d.ts +7 -0
- package/dist/contracts-core.js +7 -0
- package/dist/contracts-protocol.d.ts +2 -0
- package/dist/index.d.ts +7 -5
- package/dist/index.js +5 -4
- package/dist/input.js +3 -2
- package/dist/node/agent-definitions.d.ts +1 -8
- package/dist/node/agent-definitions.js +0 -34
- package/dist/node/settings.d.ts +0 -1
- package/dist/node/settings.js +0 -5
- package/dist/pinned-fetch.js +29 -3
- package/dist/provider-events.js +3 -4
- package/dist/provider-request-policy.d.ts +15 -0
- package/dist/provider-request-policy.js +52 -0
- package/dist/providers/media.d.ts +1 -2
- package/dist/providers/media.js +1 -4
- package/dist/rpc.d.ts +1 -1
- package/dist/rpc.js +4 -4
- package/dist/testing/provider-conformance.d.ts +114 -5
- package/dist/testing/provider-conformance.js +342 -0
- package/dist/testing/tool-effect-store-conformance.d.ts +0 -1
- package/dist/testing/tool-effect-store-conformance.js +0 -3
- package/dist/thinking.d.ts +48 -9
- package/dist/thinking.js +134 -8
- package/docs/0.1.0-readiness.md +3 -3
- package/docs/a2a.md +2 -2
- package/docs/acp.md +3 -3
- package/docs/ag-ui-adoption.md +1 -1
- package/docs/ag-ui.md +1 -2
- package/docs/agent-definitions.md +1 -1
- package/docs/agent-events.md +5 -5
- package/docs/agent-identity.md +13 -2
- package/docs/agent-session-runtime.md +2 -1
- package/docs/audit-export.md +3 -3
- package/docs/batch-jobs.md +120 -0
- package/docs/cli-rpc.md +20 -9
- package/docs/coding-agent-tools.md +19 -19
- package/docs/coding-review-and-diagnostics.md +2 -2
- package/docs/coding-security.md +4 -4
- package/docs/coding-workspaces.md +2 -2
- package/docs/compaction-llm.md +2 -0
- package/docs/compaction-observational-memory.md +3 -0
- package/docs/computer-use-linux.md +13 -2
- package/docs/context-and-skills.md +1 -1
- package/docs/conversations.md +4 -4
- package/docs/credential-storage.md +11 -7
- package/docs/credentials-and-redaction.md +1 -1
- package/docs/data-classification.md +1 -1
- package/docs/database-persistence.md +4 -4
- package/docs/dev-inspector.md +6 -6
- package/docs/device-adapters.md +2 -2
- package/docs/diagrams.md +1 -1
- package/docs/document-reader.md +6 -6
- package/docs/documents.md +5 -4
- package/docs/embeddings.md +112 -0
- package/docs/enterprise-postgres-state.md +7 -7
- package/docs/evaluations.md +8 -8
- package/docs/extensions.md +3 -3
- package/docs/forge-integration.md +3 -3
- package/docs/graft.md +2 -2
- package/docs/guardrails.md +1 -1
- package/docs/host-security.md +15 -15
- package/docs/image-generation.md +129 -0
- package/docs/impeccable.md +5 -3
- package/docs/index.md +64 -36
- package/docs/indexed-code-search.md +2 -2
- package/docs/input-and-prompt-assembly.md +1 -1
- package/docs/language-intelligence.md +4 -4
- package/docs/live-testing.md +126 -0
- package/docs/mcp-tools.md +43 -12
- package/docs/middleware-hooks.md +1 -1
- package/docs/migrate-to-0.4.md +3 -3
- package/docs/migrate-to-0.5.md +144 -0
- package/docs/migration.md +33 -1
- package/docs/model-registry.md +38 -0
- package/docs/model-routing.md +5 -5
- package/docs/moderation.md +117 -0
- package/docs/multi-agent-patterns.md +4 -4
- package/docs/multimodal-content.md +26 -2
- package/docs/obscura.md +2 -2
- package/docs/observability.md +32 -7
- package/docs/openapi-tools.md +13 -3
- package/docs/operations.md +11 -0
- package/docs/performance.md +7 -7
- package/docs/persistence-credentials-multimodality-primitives.md +6 -6
- package/docs/policy-and-audit.md +17 -7
- package/docs/ponytail.md +1 -1
- package/docs/postgres-persistence.md +5 -5
- package/docs/process-sessions.md +2 -2
- package/docs/prompt-registry.md +7 -7
- package/docs/provider-caching.md +8 -2
- package/docs/provider-conformance.md +23 -1
- package/docs/provider-packages.md +49 -17
- package/docs/provider-primitives.md +1 -1
- package/docs/provider-request-policies.md +19 -6
- package/docs/providers/ai-sdk.md +27 -3
- package/docs/providers/alibaba.md +17 -1
- package/docs/providers/anthropic.md +16 -0
- package/docs/providers/azure.md +29 -1
- package/docs/providers/bedrock.md +27 -0
- package/docs/providers/clinepass.md +16 -0
- package/docs/providers/commandcode.md +265 -0
- package/docs/providers/deepseek.md +16 -0
- package/docs/providers/google.md +16 -0
- package/docs/providers/hyper.md +296 -0
- package/docs/providers/kimi.md +16 -0
- package/docs/providers/neuralwatt.md +16 -0
- package/docs/providers/ollama.md +27 -0
- package/docs/providers/openai-compatible.md +16 -0
- package/docs/providers/openai.md +16 -0
- package/docs/providers/opencode-go.md +16 -0
- package/docs/providers/openrouter.md +17 -1
- package/docs/providers/vertex.md +28 -0
- package/docs/providers/xai.md +16 -0
- package/docs/providers/zai.md +16 -0
- package/docs/public-contracts.md +1 -1
- package/docs/rag.md +26 -4
- package/docs/release-and-install.md +103 -46
- package/docs/resource-loading.md +1 -1
- package/docs/runs-and-usage.md +14 -2
- package/docs/server.md +5 -5
- package/docs/settings-auth-trust-security.md +7 -5
- package/docs/sheets.md +2 -2
- package/docs/speech.md +126 -0
- package/docs/sqlite-persistence.md +4 -4
- package/docs/supervisors.md +3 -3
- package/docs/thinking-and-reasoning.md +99 -61
- package/docs/tool-conformance.md +1 -1
- package/docs/tool-execution-primitives.md +8 -8
- package/docs/tools.md +4 -4
- package/docs/use-case-model-selection.md +1 -1
- package/docs/web-tools.md +1 -1
- package/docs/wiki.md +1 -1
- package/docs/work-artifacts-and-review.md +17 -6
- package/docs/work-connectors.md +4 -4
- package/docs/work-tools.md +5 -5
- package/docs/workflow-orchestration-primitives.md +11 -11
- package/docs/workflows.md +5 -5
- package/package.json +11 -8
- package/templates/init/providers.json +24 -8
- package/docs/antigravity-agent.md +0 -207
package/docs/compaction-llm.md
CHANGED
|
@@ -32,6 +32,7 @@ Key exports:
|
|
|
32
32
|
| `providerRequestPolicies` | Optional Prism provider request policies applied before the summary call. |
|
|
33
33
|
| `customInstructions` | Additional summary focus appended to prompts. |
|
|
34
34
|
| `thinkingLevel` | Mapped into `ProviderRequest.options.compat` via `applyThinkingLevel` / `thinkingFamilyForModel` (not inert `extra.thinkingLevel`). See [Thinking and reasoning](thinking-and-reasoning.md). |
|
|
35
|
+
| session correlation | Summary `provider.generate` uses the **agent** `context.sessionId` (same cache key as the chat session). Kernel stamps `sessionId`/`cacheKey` even with no `providerRequestPolicies`. |
|
|
35
36
|
| `reserveTokens` | Output budget basis; defaults to `16384`, hard cap `131072`. |
|
|
36
37
|
| `keepRecentTokens` | Approximate recent-token budget; defaults to `20000`. |
|
|
37
38
|
| `maxSummaryTokens` / `maxOutputTokens` | Summary retention/request ceiling; default `16384`, hard cap `131072`. `maxSummaryTokens` wins over the compatibility alias. The finite value is written to `model.parameters.maxTokens`; first-party providers map it to their wire field. |
|
|
@@ -132,6 +133,7 @@ The strategy makes only the needed provider call(s): one history summary plus on
|
|
|
132
133
|
|
|
133
134
|
- [Use-case model selection](use-case-model-selection.md): `summaryModel` vs session `model` fallback.
|
|
134
135
|
- [Thinking and reasoning](thinking-and-reasoning.md): `thinkingLevel` → `compat`.
|
|
136
|
+
- [Provider request policies](provider-request-policies.md): kernel stamps agent `sessionId` on summary requests.
|
|
135
137
|
- [Compaction and retry policies](compaction-and-retry.md): replaceable compaction strategy boundary and core compaction strategy surface.
|
|
136
138
|
- [Observational memory compaction package](compaction-observational-memory.md): source-backed memory workers with the same use-case binding pattern.
|
|
137
139
|
- [Agent/session runtime](agent-session-runtime.md): `AgentSession.compact()` and opt-in auto-compaction.
|
|
@@ -156,6 +156,8 @@ Observer/reflector/dropper may use separate providers, models, instructions, thi
|
|
|
156
156
|
|
|
157
157
|
Token counting uses `estimateEntryTokens()` / `estimateMessageTokens()`.
|
|
158
158
|
|
|
159
|
+
Worker `provider.generate` calls use a **derived** correlation id `om:{session.id}` (shared by observer/reflector/dropper of that attach; adapters sanitize via `sanitizeCacheKey`). This is fully separate from the agent session id so OM cache does not collide with chat. Host `providerOptions.sessionId` still wins. Workers may use a different model than the session.
|
|
160
|
+
|
|
159
161
|
The runtime requires host-supplied `session`, an `appendEntry` callback bound to that session's owning store/branch, and at least one worker provider (`observation.provider` / `reflection.provider` / `dropper.provider`). Model selection uses [use-case model selection](use-case-model-selection.md): pass per-worker `model` (or settings `observation.model` / `reflection.model` / `dropper.model`) to override, and `sessionModel: agent.config.model` so workers fall back to the session model when no worker model is configured. `requireExplicitModel: true` restores the historical `missing_model` skip when no explicit worker model is set. It no longer accepts a separate `store` option because mismatched session/store pairs can append memory entries outside the active branch. After each memory append, the runtime checks the appended entry is visible at the session leaf and fails closed/restores the previous checkout if the callback points elsewhere. Optional credential resolution is explicit; missing requested credentials skip worker execution. Default credential requests use the **resolved** model's provider id.
|
|
160
162
|
|
|
161
163
|
`createObservationalMemoryCompactionStrategy()` keeps recent message entries like the default compaction strategy, renders existing observations/reflections as the summary, and returns a standard Prism compaction entry. Its `data` includes `throughEntryId`, `keepEntryIds`, `strategy`, `trigger`, and `memory: { type: "om.folded", version: 1, fullFold, observations, reflections, droppedObservationIds }`. When active observations exceed `context.observationsPoolMaxTokens`, it performs a full fold and synchronously trims lowest-relevance observations until the folded payload fits hard byte/token caps (or throws a typed error).
|
|
@@ -230,6 +232,7 @@ Ownership: funnel only within the `OwnershipScope` already on the parent agent/s
|
|
|
230
232
|
|
|
231
233
|
- [Use-case model selection](use-case-model-selection.md): session vs worker model binding and `resolveUseCaseModel`.
|
|
232
234
|
- [Thinking and reasoning](thinking-and-reasoning.md): `thinkingLevel` → provider `compat`.
|
|
235
|
+
- [Provider request policies](provider-request-policies.md): derived `om:{session.id}` on worker generate.
|
|
233
236
|
- [Compaction and retry policies](compaction-and-retry.md): replaceable compaction strategy boundary.
|
|
234
237
|
- [LLM compaction package](compaction-llm.md): existing optional compaction-package pattern.
|
|
235
238
|
- [Session stores and branching](session-stores-and-branching.md): branch entries that observational memory reads and appends to.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-computer-use-linux` wraps the host-owned [`computer-use-linux`](https://github.com/agent-sh/computer-use-linux) MCP binary as Prism `ToolDefinition`s. It connects over stdio only when `createComputerUseLinuxTools()` is called, keeps upstream tool names, filters unknown tools, and composes desktop admission, execution approval, result bounds, serialization, redaction, and trust labeling over Prism's existing seams.
|
|
5
|
+
`@arnilo/prism-coding-tools/computer-use-linux` wraps the host-owned [`computer-use-linux`](https://github.com/agent-sh/computer-use-linux) MCP binary as Prism `ToolDefinition`s. It connects over stdio only when `createComputerUseLinuxTools()` is called, keeps upstream tool names, filters unknown tools, and composes desktop admission, execution approval, result bounds, serialization, redaction, and trust labeling over Prism's existing seams.
|
|
6
6
|
|
|
7
7
|
The package also exports `loadComputerUseLinuxSkill()`, which loads the short Prism-authored desktop procedure bundled at `skills/computer-use-linux/SKILL.md`. It does not resolve or vendor an upstream skill tree.
|
|
8
8
|
|
|
@@ -75,7 +75,7 @@ import { createToolRegistry, type Skill } from "@arnilo/prism";
|
|
|
75
75
|
import {
|
|
76
76
|
createComputerUseLinuxTools,
|
|
77
77
|
loadComputerUseLinuxSkill,
|
|
78
|
-
} from "@arnilo/prism-computer-use-linux";
|
|
78
|
+
} from "@arnilo/prism-coding-tools/computer-use-linux";
|
|
79
79
|
|
|
80
80
|
async function installDesktop(hostSkills: { register(skill: Skill): void }, hostApproved: boolean) {
|
|
81
81
|
const desktop = await createComputerUseLinuxTools({
|
|
@@ -112,6 +112,17 @@ async function installDesktop(hostSkills: { register(skill: Skill): void }, host
|
|
|
112
112
|
- Imports are inert. The default setup surface is off, the skill file is capped at 64 KiB, and no full upstream skill tree is shipped.
|
|
113
113
|
- The host must keep credentials, desktop session state, binary paths, sandbox identity, and approval state outside model-controlled arguments.
|
|
114
114
|
|
|
115
|
+
## Live probe (plans/064 Task 7)
|
|
116
|
+
|
|
117
|
+
A live leg drives the host's real `computer-use-linux` MCP binary over stdio — real tool inventory, one bounded read-only screenshot, clean close (≤30 s):
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
PRISM_TEST_COMPUTER_USE=1 PRISM_COMPUTER_USE_BIN="$(command -v computer-use-linux)" \
|
|
121
|
+
node --test packages/prism-coding-tools/dist/computer-use-linux/__tests__/live.test.js
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Skips (never fails) when the flag, binary path, or a desktop session is unavailable. The suite performs no network I/O, so screenshot bytes cannot leave the process. Registered in `scripts/live-matrix.json` as `coding-tools/computer-use-live`.
|
|
125
|
+
|
|
115
126
|
## Related APIs
|
|
116
127
|
|
|
117
128
|
- [Device adapters](device-adapters.md): generic admission, shared limits, chunk bounds, and telemetry redaction contract.
|
|
@@ -200,7 +200,7 @@ await agent.createSession().run("…", { activeSkills: ["ponytail"] });
|
|
|
200
200
|
|
|
201
201
|
### Third-party behavior packages (Caveman, Ponytail, Impeccable)
|
|
202
202
|
|
|
203
|
-
`@arnilo/prism-caveman` and `@arnilo/prism-ponytail` register upstream skills into the extension kernel skill registry. Hosts should:
|
|
203
|
+
`@arnilo/prism-coding-tools/caveman` and `@arnilo/prism-coding-tools/ponytail` register upstream skills into the extension kernel skill registry. Hosts should:
|
|
204
204
|
|
|
205
205
|
1. `kernel.load([createCavemanExtension(...), createPonytailExtension(...)])` with session `appendEntry` / `getEntries` callbacks.
|
|
206
206
|
2. Build `createSkillRegistry(kernel.registries.skills.list())` and pass `activeSkills` / `resolveActiveSkills` names.
|
package/docs/conversations.md
CHANGED
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-server` ships a durable, user-scoped conversation service: create/list/get/continue/branch/archive/export/delete conversation threads on top of the existing session and event-ledger seams. A thread **is** an ownership-scoped session branch plus a `prismConversation` marker in `SessionRecord.metadata`; content stays in session entries and the redacted event ledger. Reconnectable replay pages durable redacted events without ever rerunning a provider or tool.
|
|
5
|
+
`@arnilo/prism-core/runtime/server` ships a durable, user-scoped conversation service: create/list/get/continue/branch/archive/export/delete conversation threads on top of the existing session and event-ledger seams. A thread **is** an ownership-scoped session branch plus a `prismConversation` marker in `SessionRecord.metadata`; content stays in session entries and the redacted event ledger. Reconnectable replay pages durable redacted events without ever rerunning a provider or tool.
|
|
6
6
|
|
|
7
|
-
Core (`@arnilo/prism`) exports only conversation **types and pure helpers** (`ConversationThread`, `ConversationError`, `CONVERSATION_METADATA_KEY`, thread-bound replay cursor codec, `conversationThreadFromRecord`, `conversationMarkerMetadata`). The service and optional HTTP handler live in `@arnilo/prism-server`.
|
|
7
|
+
Core (`@arnilo/prism`) exports only conversation **types and pure helpers** (`ConversationThread`, `ConversationError`, `CONVERSATION_METADATA_KEY`, thread-bound replay cursor codec, `conversationThreadFromRecord`, `conversationMarkerMetadata`). The service and optional HTTP handler live in `@arnilo/prism-core/runtime/server`.
|
|
8
8
|
|
|
9
9
|
## When to use it
|
|
10
10
|
|
|
@@ -15,8 +15,8 @@ Do not use it as a chat UI, a push/always-on daemon, or a file store. Slack/Team
|
|
|
15
15
|
## Inputs / request
|
|
16
16
|
|
|
17
17
|
```ts
|
|
18
|
-
import { createConversationService, createConversationHandler } from "@arnilo/prism-server";
|
|
19
|
-
import { createSqlitePersistence } from "@arnilo/prism-
|
|
18
|
+
import { createConversationService, createConversationHandler } from "@arnilo/prism-core/runtime/server";
|
|
19
|
+
import { createSqlitePersistence } from "@arnilo/prism-core/sessions/sqlite";
|
|
20
20
|
|
|
21
21
|
const persistence = createSqlitePersistence({ filename }); // implements ConversationServiceStore
|
|
22
22
|
const service = createConversationService(persistence, {
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
The optional `@arnilo/prism-credentials
|
|
5
|
+
The optional `@arnilo/prism-core/credentials/node` package ships host-owned credential persistence for Node.js CLI and desktop apps:
|
|
6
6
|
|
|
7
7
|
- **Encrypted file store** — AES-256-GCM envelope with scrypt KDF, atomic rename writes, versioned on-disk format
|
|
8
8
|
- **System keychain store** — cross-platform secret service via `@napi-rs/keyring@^1.3.0`
|
|
@@ -17,7 +17,7 @@ Factories:
|
|
|
17
17
|
- `createOAuthCredentialStoreAdapter(store)`
|
|
18
18
|
- `rotateEncryptedCredentialStorePassphrase(options)`
|
|
19
19
|
|
|
20
|
-
The `@arnilo/prism-credentials
|
|
20
|
+
The `@arnilo/prism-core/credentials/node/oidc` subpath adds the optional OIDC/JWKS identity verifier (`createOidcIdentityVerifier`) — pinned issuer/audience/JWKS verification over native `fetch` + WebCrypto (see [Agent identity](agent-identity.md)).
|
|
21
21
|
|
|
22
22
|
Core `@arnilo/prism` remains storage-free. Hosts choose a backend explicitly at startup; there is no global credential singleton and no silent fallback from keychain to plaintext file storage.
|
|
23
23
|
|
|
@@ -37,7 +37,7 @@ Do **not** use it when credentials should live in a remote vault, HSM, or cloud
|
|
|
37
37
|
import {
|
|
38
38
|
openEncryptedCredentialStore,
|
|
39
39
|
createKeychainCredentialStore,
|
|
40
|
-
} from "@arnilo/prism-credentials
|
|
40
|
+
} from "@arnilo/prism-core/credentials/node";
|
|
41
41
|
```
|
|
42
42
|
|
|
43
43
|
### Encrypted file
|
|
@@ -79,6 +79,10 @@ Encrypted file stores also expose:
|
|
|
79
79
|
|
|
80
80
|
`encryptBytes()` and `decryptBytes()` are Promise-based because they use asynchronous `node:crypto.scrypt`.
|
|
81
81
|
|
|
82
|
+
### MCP OAuth records (plan 063)
|
|
83
|
+
|
|
84
|
+
When backing `McpClientAuthState` from `@arnilo/prism-mcp` with one of these stores, key OAuth rows by the validated authorization-server `issuer` the SDK stamps onto every `StoredOAuthTokens`/`StoredOAuthClientInformation` record (the credential methods receive it). Credentials must never cross issuers: a store that cannot partition by issuer stays single-slot-safe (the MCP provider re-validates the stamp), but partitioning is the preferred shape for hosts talking to more than one MCP server. Refresh tokens belong only in the encrypted file or keychain backends — never the plaintext memory store in production.
|
|
85
|
+
|
|
82
86
|
Errors are explicit and fail closed:
|
|
83
87
|
|
|
84
88
|
| Error | Code | When |
|
|
@@ -126,7 +130,7 @@ import {
|
|
|
126
130
|
createOAuthCredentialStoreAdapter,
|
|
127
131
|
createStoredCredentialResolver,
|
|
128
132
|
openEncryptedCredentialStore,
|
|
129
|
-
} from "@arnilo/prism-credentials
|
|
133
|
+
} from "@arnilo/prism-core/credentials/node";
|
|
130
134
|
|
|
131
135
|
const store = await openEncryptedCredentialStore({
|
|
132
136
|
path: "./credentials.vault",
|
|
@@ -157,7 +161,7 @@ import {
|
|
|
157
161
|
createGoogleWorkspaceOAuthProvider,
|
|
158
162
|
createOAuthWorkTokenProvider,
|
|
159
163
|
createOAuthCredentialStoreAdapter,
|
|
160
|
-
} from "@arnilo/prism-credentials
|
|
164
|
+
} from "@arnilo/prism-core/credentials/node";
|
|
161
165
|
|
|
162
166
|
// Read-only mail/calendar (no mutation scopes requested).
|
|
163
167
|
const m365 = createMicrosoft365OAuthProvider({ clientId: "<app-id>", capabilities: ["mail", "calendar"], access: "read" });
|
|
@@ -178,7 +182,7 @@ await revokeOAuthCredential({ provider: m365, credentials: creds, store: createO
|
|
|
178
182
|
Passphrase rotation:
|
|
179
183
|
|
|
180
184
|
```ts
|
|
181
|
-
import { rotateEncryptedCredentialStorePassphrase } from "@arnilo/prism-credentials
|
|
185
|
+
import { rotateEncryptedCredentialStorePassphrase } from "@arnilo/prism-core/credentials/node";
|
|
182
186
|
|
|
183
187
|
await rotateEncryptedCredentialStorePassphrase({
|
|
184
188
|
path: "./credentials.vault",
|
|
@@ -198,7 +202,7 @@ import {
|
|
|
198
202
|
import {
|
|
199
203
|
createKeychainCredentialStore,
|
|
200
204
|
createStoredCredentialResolver,
|
|
201
|
-
} from "@arnilo/prism-credentials
|
|
205
|
+
} from "@arnilo/prism-core/credentials/node";
|
|
202
206
|
import { createOpenAIProviderPackage } from "@arnilo/prism-providers/openai";
|
|
203
207
|
|
|
204
208
|
const keychain = createKeychainCredentialStore({
|
|
@@ -133,4 +133,4 @@ A future provider-local OAuth adapter needs published permission for third-party
|
|
|
133
133
|
- [LLM compaction package](compaction-llm.md): resolves optional summary-provider credentials per compaction call and redacts exact known values.
|
|
134
134
|
- [OpenAI-compatible provider](providers/openai-compatible.md): resolves API keys per request and redacts known values from adapter errors.
|
|
135
135
|
|
|
136
|
-
Phase 10 added `createMemoryCredentialStore()`, `createChainedCredentialResolver()`, and `createSecretRedactor()` for opt-in in-memory auth and runtime redaction. By default the memory store serves a providerless record for a provider-scoped request of the same name — that record is then shared across every provider; pass `{ allowProviderFallback: false }` for exact-match-only resolution (strict provider scoping). Phase 11 adds OAuth/API-key contracts plus explicit resolver order helpers. Core still has no persistent secret store and does not read environment variables or files for credentials. For durable storage, use [`@arnilo/prism-credentials
|
|
136
|
+
Phase 10 added `createMemoryCredentialStore()`, `createChainedCredentialResolver()`, and `createSecretRedactor()` for opt-in in-memory auth and runtime redaction. By default the memory store serves a providerless record for a provider-scoped request of the same name — that record is then shared across every provider; pass `{ allowProviderFallback: false }` for exact-match-only resolution (strict provider scoping). Phase 11 adds OAuth/API-key contracts plus explicit resolver order helpers. Core still has no persistent secret store and does not read environment variables or files for credentials. For durable storage, use [`@arnilo/prism-core/credentials/node`](credential-storage.md) encrypted-file or keychain backends. See [Security/auth/trust](settings-auth-trust-security.md).
|
|
@@ -77,6 +77,6 @@ const redactor = createAuditFieldRedactor(fieldPolicy, { labelFor });
|
|
|
77
77
|
|
|
78
78
|
- `redactMessage` / `redactProviderRequest` / `redactAgentEvent` / `redactSessionEntry` / `redactRunLedgerRecord` — the egress seams that take the optional policy (secret redaction first, then classification).
|
|
79
79
|
- `createAuditFieldRedactor` → the audit-export `redact` hook; see [Signed, hash-chained audit export](audit-export.md).
|
|
80
|
-
- `createOpenTelemetryInstrumentation` in `@arnilo/prism-observability
|
|
80
|
+
- `createOpenTelemetryInstrumentation` in `@arnilo/prism-core/governance/observability` — the telemetry `fieldPolicy` option.
|
|
81
81
|
- `createProtectedFieldPolicy`, `ALLOW_FIELD_POLICY`, `FieldPolicyError`, `FIELD_POLICY_LIMITS` — the protected default and limits.
|
|
82
82
|
- The ERP-T9 threat matrix (`src/__tests__/field-policy.test.ts`) and the boundary-drill scripts cover the enforcement evidence.
|
|
@@ -6,11 +6,11 @@ The production persistence contracts describe database-neutral types for durable
|
|
|
6
6
|
|
|
7
7
|
Prism itself does not ship a production database adapter. The built-in `SessionStore` contract (`append` / `list` / optional `get`) remains the runtime seam; `ProductionPersistenceStore` is the optional adapter-facing contract for hosts that need paginated reads, tenant isolation, audit tables, retention, and optional generic `CheckpointStore` / `LeaseStore` capabilities.
|
|
8
8
|
|
|
9
|
-
Plan 056 Task 1 adds dialect-neutral shared primitives under `@arnilo/prism/testing/persistence-schema`, `@arnilo/prism/testing/session-store-conformance`, and `@arnilo/prism/testing/run-ledger-conformance`. Task 2 ships `@arnilo/prism-
|
|
9
|
+
Plan 056 Task 1 adds dialect-neutral shared primitives under `@arnilo/prism/testing/persistence-schema`, `@arnilo/prism/testing/session-store-conformance`, and `@arnilo/prism/testing/run-ledger-conformance`. Task 2 ships `@arnilo/prism-core/sessions/sqlite` (see [SQLite persistence](sqlite-persistence.md)); Task 3 ships `@arnilo/prism-core/sessions/postgres` (see [PostgreSQL persistence](postgres-persistence.md)). Both implement dialect-local SQL against the shared model; Prism core still ships no ORM, driver, or migration runner.
|
|
10
10
|
|
|
11
|
-
Release 0.0.23 additionally ships [`@arnilo/prism-enterprise
|
|
11
|
+
Release 0.0.23 additionally ships [`@arnilo/prism-core/enterprise/postgres`](enterprise-postgres-state.md), a separate PostgreSQL composition for policy decisions, evaluations, work-mutation claims, and model-router state. It is not a `ProductionPersistenceStore` replacement and does not store sessions/runs. Its fixed `prism_enterprise_migrations` history is independent of `prism_migrations`; hosts may use both compositions against the same validated schema.
|
|
12
12
|
|
|
13
|
-
The optional [`@arnilo/prism-prompts`](prompt-registry.md) package is another independent persistence surface: its SQLite/PostgreSQL adapters own `prism_prompts`, `prism_prompt_labels`, and `prism_prompt_migrations`, apply a checked `001_init` history, and filter every read/write by exact prompt ownership. It does not extend the shared session schema or place prompt bodies in run/session metadata.
|
|
13
|
+
The optional [`@arnilo/prism-core/governance/prompts`](prompt-registry.md) package is another independent persistence surface: its SQLite/PostgreSQL adapters own `prism_prompts`, `prism_prompt_labels`, and `prism_prompt_migrations`, apply a checked `001_init` history, and filter every read/write by exact prompt ownership. It does not extend the shared session schema or place prompt bodies in run/session metadata.
|
|
14
14
|
|
|
15
15
|
## When to use it
|
|
16
16
|
|
|
@@ -457,7 +457,7 @@ const dbStore: ProductionPersistenceStore = {
|
|
|
457
457
|
- `SessionStore` (`append`/`list`/`get`/optional `readBranchPath`) can be implemented on top of `ProductionPersistenceStore` or kept separate.
|
|
458
458
|
- **State-concurrency conformance (0.2.2):** durable adapters must pass `assertStateConcurrencyConforms` from `@arnilo/prism/testing/state-concurrency-conformance` against both the memory stores and their own implementation (approval determinism, checkpoint CAS, replay-cursor resume, idempotency retry, router reservation, conversation metadata CAS, unknown-outcome recovery). The harness uses deterministic barriers only — no timing-only sleeps — and runs the memory leg in the default `npm test` and the durable legs in `test:postgres`/`test:nats`; `scripts/phase22-conformance.test.mjs` asserts every store leg executed (missing protected environment records a named BLOCKED GATE, never a green skip).
|
|
459
459
|
- Cursor values and idempotency keys are host-defined and opaque to Prism.
|
|
460
|
-
- First-party SQLite/PostgreSQL adapters expose `persistence.checkpoints` and `persistence.leases`, backed by package-owned `prism_checkpoints` / `prism_leases` tables. `@arnilo/prism-workflows` consumes them for durable resume, human suspension, multi-process coordination, Phase 11 schedule records/fire leases, shared state, and replay lineage; workflow code owns no SQL table. `suspended`/`denied`, schedules, state history, and replay lineage remain namespaces/categories plus bounded checkpoint JSON values, so Phases 8 and 11 need no database migration.
|
|
460
|
+
- First-party SQLite/PostgreSQL adapters expose `persistence.checkpoints` and `persistence.leases`, backed by package-owned `prism_checkpoints` / `prism_leases` tables. `@arnilo/prism-core/runtime/workflows` consumes them for durable resume, human suspension, multi-process coordination, Phase 11 schedule records/fire leases, shared state, and replay lineage; workflow code owns no SQL table. `suspended`/`denied`, schedules, state history, and replay lineage remain namespaces/categories plus bounded checkpoint JSON values, so Phases 8 and 11 need no database migration.
|
|
461
461
|
|
|
462
462
|
Schema version **7** adds the exact-owner durable event retention index (`prism_agent_events_owner_timestamp_sequence_idx`). Distributed subscribe/LISTEN remains PostgreSQL-only via `persistence.events`.
|
|
463
463
|
|
package/docs/dev-inspector.md
CHANGED
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-dev` is a **loopback-only local dev inspector** over a host's already-configured Prism agent — the `prism dev` playground server (plan 040). It is a **composition-only consumer**: it adds zero core primitives and imports no core internals. Every capability is an existing public seam consumed verbatim:
|
|
5
|
+
`@arnilo/prism-coding-tools/dev` is a **loopback-only local dev inspector** over a host's already-configured Prism agent — the `prism dev` playground server (plan 040). It is a **composition-only consumer**: it adds zero core primitives and imports no core internals. Every capability is an existing public seam consumed verbatim:
|
|
6
6
|
|
|
7
|
-
- `@arnilo/prism-server` `createPrismHandler` — direct `POST /prism/agents/:id/runs` and SSE `POST /prism/agents/:id/stream` agent routes, authorization, ownership propagation, and the durable `Last-Event-ID` event route when an exposure carries `events` + `resolveRun`.
|
|
7
|
+
- `@arnilo/prism-core/runtime/server` `createPrismHandler` — direct `POST /prism/agents/:id/runs` and SSE `POST /prism/agents/:id/stream` agent routes, authorization, ownership propagation, and the durable `Last-Event-ID` event route when an exposure carries `events` + `resolveRun`.
|
|
8
8
|
- Core durable `AgentEventSource` contract (`page`/`subscribe`) — replay and reconnect without re-execution.
|
|
9
9
|
- `@arnilo/prism-ag-ui/renderer` — event projection for the served UI page (plan 040 Task 3).
|
|
10
10
|
- Run-ledger records (`RunRecord`/`AgentEventRecord`/`ToolCallRecord`/`UsageRecord`) surfaced only through the seams above — the package never touches a ledger.
|
|
@@ -12,16 +12,16 @@
|
|
|
12
12
|
|
|
13
13
|
## When to use it
|
|
14
14
|
|
|
15
|
-
Use it when iterating on prompts in a local Prism host and you want a inspectable timeline (events, tool calls, usage, HITL decisions, run replay) instead of building your own trace viewer. Do not deploy it: it is a developer-time surface,
|
|
15
|
+
Use it when iterating on prompts in a local Prism host and you want a inspectable timeline (events, tool calls, usage, HITL decisions, run replay) instead of building your own trace viewer. Do not deploy it: it is a developer-time surface, deliberately excluded from production dependency use, and it must never be the production API boundary — that stays `@arnilo/prism-core/runtime/server` under host authorization.
|
|
16
16
|
|
|
17
17
|
### Quickstart — `prism dev` (plan 040 Task 4)
|
|
18
18
|
|
|
19
19
|
```bash
|
|
20
|
-
npm install --save-dev @arnilo/prism-dev
|
|
20
|
+
npm install --save-dev @arnilo/prism-coding-tools/dev
|
|
21
21
|
cd my-agent && npm run dev # → prism dev → http://127.0.0.1:4311
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
-
`prism dev` (and the standalone `prism-dev` bin, plus the programmatic `runDevCli` from `@arnilo/prism-dev/cli`) boots the inspector over the current `prism init` scaffold: it imports `dist/agent.js` and calls its `createAppAgent()` export — the scaffold's own agent, with its own credentials. It defaults to `127.0.0.1:4311`, prints the loopback URL once listening (start-to-listen under 1s excluding provider network), and `Ctrl+C` drains and closes. A non-loopback `--host` is refused before binding (`ERR_PRISM_DEV_REMOTE_BIND`); the CLI never reads environment secrets itself. See `docs/cli-rpc.md` for the flag table.
|
|
24
|
+
`prism dev` (and the standalone `prism-dev` bin, plus the programmatic `runDevCli` from `@arnilo/prism-coding-tools/dev/cli`) boots the inspector over the current `prism init` scaffold: it imports `dist/agent.js` and calls its `createAppAgent()` export — the scaffold's own agent, with its own credentials. It defaults to `127.0.0.1:4311`, prints the loopback URL once listening (start-to-listen under 1s excluding provider network), and `Ctrl+C` drains and closes. A non-loopback `--host` is refused before binding (`ERR_PRISM_DEV_REMOTE_BIND`); the CLI never reads environment secrets itself. See `docs/cli-rpc.md` for the flag table.
|
|
25
25
|
|
|
26
26
|
## Inputs / request
|
|
27
27
|
|
|
@@ -89,7 +89,7 @@ POST /runs/<runId>/decisions/<approvalId>
|
|
|
89
89
|
## Implementation example
|
|
90
90
|
|
|
91
91
|
```ts
|
|
92
|
-
import { createPrismDevInspector } from "@arnilo/prism-dev";
|
|
92
|
+
import { createPrismDevInspector } from "@arnilo/prism-coding-tools/dev";
|
|
93
93
|
|
|
94
94
|
const inspector = createPrismDevInspector({
|
|
95
95
|
agent, // host-built agent (mock or provider-backed)
|
package/docs/device-adapters.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
Optional realtime voice and desktop OS / computer-control surface for Prism agents, shipped in 0.0.14 as a **contract + deny-by-default policy** in `@arnilo/prism` (`src/devices.ts`). The first vendor adapter, `@arnilo/prism-computer-use-linux`, wraps the host-owned `computer-use-linux` MCP binary without changing this generic contract. The contract composes over the existing `PermissionPolicy`, `RunLimits`, approval (`tool_approval`), and redactor seams; it adds no second approval runtime and no device framework.
|
|
5
|
+
Optional realtime voice and desktop OS / computer-control surface for Prism agents, shipped in 0.0.14 as a **contract + deny-by-default policy** in `@arnilo/prism` (`src/devices.ts`). The first vendor adapter, `@arnilo/prism-coding-tools/computer-use-linux`, wraps the host-owned `computer-use-linux` MCP binary without changing this generic contract. The contract composes over the existing `PermissionPolicy`, `RunLimits`, approval (`tool_approval`), and redactor seams; it adds no second approval runtime and no device framework.
|
|
6
6
|
|
|
7
7
|
## When to use it
|
|
8
8
|
|
|
@@ -79,7 +79,7 @@ if (chunk.accepted) emit(redactDeviceTelemetry(createSecretRedactor([token]), fr
|
|
|
79
79
|
|
|
80
80
|
- Frozen caps: audio/screenshot/stream chunk **1 MiB / 8 MiB**; concurrent device sessions per identity **1 / 4**. Device wall time / turns / tool calls consume the shared `RunLimits` (admission fails closed without run accounting).
|
|
81
81
|
- `enabled` resolves to `true` only on an explicit `true`; any other value is disabled. `requireApproval` stays `true` unless the host explicitly sets `false` (it should not).
|
|
82
|
-
- `@arnilo/prism-computer-use-linux` is the first vendor package. It remains optional, Linux-only, host-binary-owned, and outside umbrella profiles; this page stays generic so future voice or desktop vendors can satisfy the same contract via `runDevicePolicyConformance`.
|
|
82
|
+
- `@arnilo/prism-coding-tools/computer-use-linux` is the first vendor package. It remains optional, Linux-only, host-binary-owned, and outside umbrella profiles; this page stays generic so future voice or desktop vendors can satisfy the same contract via `runDevicePolicyConformance`.
|
|
83
83
|
|
|
84
84
|
## Security and performance notes
|
|
85
85
|
|
package/docs/diagrams.md
CHANGED
|
@@ -244,4 +244,4 @@ const summary = validateDrawioXml(xml, {
|
|
|
244
244
|
- [`@arnilo/prism-office/sheets`](./sheets.md): Spreadsheet and CSV parsing engine with strict financial decimal safety guarantees.
|
|
245
245
|
- [`@arnilo/prism-web-tools/browser`](./browser-automation.md): Browser automation tools and quarantine lifecycle.
|
|
246
246
|
- [`@arnilo/prism-ag-ui`](./ag-ui.md): Agent-User Interface projection and timeline components.
|
|
247
|
-
- [`@arnilo/prism-observability
|
|
247
|
+
- [`@arnilo/prism-core/governance/observability`](./observability.md): OpenTelemetry instrumentation and trace adapters.
|
package/docs/document-reader.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Document reader (`@arnilo/prism-document-reader`)
|
|
1
|
+
# Document reader (`@arnilo/prism-coding-tools/document-reader`)
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
@@ -6,7 +6,7 @@ Optional bounded literal-text extraction for PDF and DOCX files, consumed by the
|
|
|
6
6
|
|
|
7
7
|
## When to use it
|
|
8
8
|
|
|
9
|
-
Use when coding agents must read PDF/Office files (specs, requirements docs, reports) as literal text. Do **not** use it when embedded content execution, macro evaluation, or external resource fetching is required — this adapter never does any of those by construction, and the optional peer parsers (`pdf-parse`, `mammoth`) are the only parsing code involved. Docker-less hosts that need document reads pair this with the network-free native sandbox backend (`@arnilo/prism-coding-security` `createNativeSandbox`) for the surrounding tool execution.
|
|
9
|
+
Use when coding agents must read PDF/Office files (specs, requirements docs, reports) as literal text. Do **not** use it when embedded content execution, macro evaluation, or external resource fetching is required — this adapter never does any of those by construction, and the optional peer parsers (`pdf-parse`, `mammoth`) are the only parsing code involved. Docker-less hosts that need document reads pair this with the network-free native sandbox backend (`@arnilo/prism-coding-tools/security` `createNativeSandbox`) for the surrounding tool execution.
|
|
10
10
|
|
|
11
11
|
Activation is explicit: no file-extension sniffing anywhere enables parsing. Absent `documentReader` option = exactly the 0.1.5 read behavior.
|
|
12
12
|
|
|
@@ -33,8 +33,8 @@ Errors: `DocumentReaderError` with code `ERR_PRISM_DOCUMENT_READER` for missing
|
|
|
33
33
|
## Request/response example
|
|
34
34
|
|
|
35
35
|
```ts
|
|
36
|
-
import { createReadTool } from "@arnilo/prism-coding-agent";
|
|
37
|
-
import { createDocumentReader } from "@arnilo/prism-document-reader";
|
|
36
|
+
import { createReadTool } from "@arnilo/prism-coding-tools/agent";
|
|
37
|
+
import { createDocumentReader } from "@arnilo/prism-coding-tools/document-reader";
|
|
38
38
|
|
|
39
39
|
const documentReader = await createDocumentReader({
|
|
40
40
|
maxBytes: 32 * 1024 * 1024,
|
|
@@ -49,7 +49,7 @@ A `read` of `spec.pdf` yields text content extracted from the PDF (up to 2 MiB o
|
|
|
49
49
|
## Implementation example
|
|
50
50
|
|
|
51
51
|
```ts
|
|
52
|
-
import { createDocumentReader, createPdfParser, type DocumentParser } from "@arnilo/prism-document-reader";
|
|
52
|
+
import { createDocumentReader, createPdfParser, type DocumentParser } from "@arnilo/prism-coding-tools/document-reader";
|
|
53
53
|
|
|
54
54
|
// Host-selected parser wiring: swap in a different PDF backend without touching bounds.
|
|
55
55
|
const myPdfParser: DocumentParser = {
|
|
@@ -79,7 +79,7 @@ const reader = await createDocumentReader({ parsers: [myPdfParser, await createP
|
|
|
79
79
|
|
|
80
80
|
## Related APIs
|
|
81
81
|
|
|
82
|
-
- `createReadTool` / `DocumentReader` / `DocumentReaderResult` (`@arnilo/prism-coding-agent`)
|
|
82
|
+
- `createReadTool` / `DocumentReader` / `DocumentReaderResult` (`@arnilo/prism-coding-tools/agent`)
|
|
83
83
|
- `SecretRedactor` (`@arnilo/prism` redaction)
|
|
84
84
|
- `docs/_evidence/phase18-primitive-review.md` (doc-reader threat model D1–D8)
|
|
85
85
|
- `docs/coding-security.md` (native sandbox backend for surrounding execution containment)
|
package/docs/documents.md
CHANGED
|
@@ -66,6 +66,7 @@ All package operations throw typed exceptions derived from `DocumentsError`:
|
|
|
66
66
|
| `DocumentsCapExceededError` | `ERR_PRISM_DOCUMENTS_CAP_EXCEEDED` | Document exceeds byte size, block count, cell count, or slide count caps. |
|
|
67
67
|
| `DocumentsParseError` | `ERR_PRISM_DOCUMENTS_PARSE_FAILED` | Input buffer lacks PK zip signature, is corrupted, or fails OOXML part parsing. |
|
|
68
68
|
| `DocumentsFormatError` | `ERR_PRISM_DOCUMENTS_UNSUPPORTED_FORMAT` | Format mismatch (e.g. attempting to generate PPTX from a `DocModel`). |
|
|
69
|
+
| `DocumentsPatchError` | `ERR_PRISM_DOCUMENTS_UNSAFE_PATH` | `__proto__`, `prototype`, or `constructor` appears as a `metadata` target or inside a block/slide/sheet `patch` segment (rejected before mutation to prevent `Object.prototype` pollution). |
|
|
69
70
|
| `DocumentsPatchError` | `ERR_PRISM_DOCUMENTS_PATCH_FAILED` | Out-of-bounds index target, unknown patch operation, or invalid patch structure. |
|
|
70
71
|
|
|
71
72
|
## Request/response example
|
|
@@ -207,7 +208,7 @@ Financial worksheets often require exact decimal representations that JavaScript
|
|
|
207
208
|
|
|
208
209
|
## Related APIs
|
|
209
210
|
|
|
210
|
-
- [`@arnilo/prism-document-reader`](./document-reader.md): Bounded literal text extraction from PDF and DOCX documents for coding agent tools.
|
|
211
|
-
- [`@arnilo/prism-work
|
|
212
|
-
- [`@arnilo/prism-coding-agent`](./coding-agent-tools.md): Coding tools and file operations.
|
|
213
|
-
- [`@arnilo/prism-observability
|
|
211
|
+
- [`@arnilo/prism-coding-tools/document-reader`](./document-reader.md): Bounded literal text extraction from PDF and DOCX documents for coding agent tools.
|
|
212
|
+
- [`@arnilo/prism-core/integrations/work`](./work-tools.md): Microsoft 365 and Google Workspace identity-scoped connectors.
|
|
213
|
+
- [`@arnilo/prism-coding-tools/agent`](./coding-agent-tools.md): Coding tools and file operations.
|
|
214
|
+
- [`@arnilo/prism-core/governance/observability`](./observability.md): OpenTelemetry instrumentation and trace adapters.
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
# Embeddings
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
`EmbeddingsProvider` is the provider-neutral embeddings contract: one-shot batch
|
|
6
|
+
text→vector generation with usage accounting, per-item error mapping, and
|
|
7
|
+
capability-gated models. First-party adapters ship in
|
|
8
|
+
[`@arnilo/prism-providers/openai`](providers/openai.md) (OpenAI-compatible
|
|
9
|
+
`POST {base}/embeddings`) and [`@arnilo/prism-providers/alibaba`](providers/alibaba.md)
|
|
10
|
+
(DashScope `compatible-mode/v1/embeddings`); offline conformance runs via
|
|
11
|
+
`runEmbeddingsConformance` from `@arnilo/prism/testing/provider-conformance`.
|
|
12
|
+
|
|
13
|
+
## When to use it
|
|
14
|
+
|
|
15
|
+
Use it when a host owns its embedding pipeline (RAG ingestion, memory indexing,
|
|
16
|
+
semantic search) and wants a portable contract instead of per-provider HTTP code.
|
|
17
|
+
Do not use it as a live integration runner: the contract is caller-gated, adapters
|
|
18
|
+
perform no network on construction, and batch caps are enforced with typed errors
|
|
19
|
+
rather than silent auto-chunking — chunk callers themselves (for example,
|
|
20
|
+
`@arnilo/prism-memory`'s `embedBatched`).
|
|
21
|
+
|
|
22
|
+
## Inputs / request
|
|
23
|
+
|
|
24
|
+
| Field | Type | Meaning |
|
|
25
|
+
| --- | --- | --- |
|
|
26
|
+
| `model` | `string` | Embedding model id, e.g. `text-embedding-3-small` or `text-embedding-v4`. |
|
|
27
|
+
| `inputs` | `readonly string[]` | Texts to embed; order is preserved on `result.vectors`. Non-empty and within the provider batch cap. |
|
|
28
|
+
| `dimensions` | `number?` | Output-dimensions override; only for models that support reduced dimensions (OpenAI `dimensions` param, DashScope 64–2048). |
|
|
29
|
+
| `signal` | `AbortSignal?` | Cancellation; observed by the adapter transport. |
|
|
30
|
+
|
|
31
|
+
Adapter options: `apiKey` (`CredentialValueSource` — the existing credential seam,
|
|
32
|
+
resolved per call and redacted from errors), `baseUrl`/`preset` (Alibaba),
|
|
33
|
+
`fetch` (inject a fake transport for offline tests), `headers`, and default
|
|
34
|
+
`dimensions`/`encodingFormat` where the provider supports them.
|
|
35
|
+
|
|
36
|
+
## Outputs / response / events
|
|
37
|
+
|
|
38
|
+
| Field | Type | Meaning |
|
|
39
|
+
| --- | --- | --- |
|
|
40
|
+
| `vectors` | `readonly (readonly number[])[]` | Vectors in input order; `vectors[i]` corresponds to `inputs[i]`. |
|
|
41
|
+
| `usage` | `Usage` | Provider-reported token usage (`inputTokens`, `totalTokens`); empty object when the provider omits it. |
|
|
42
|
+
| `dimensions` | `number` | Actual vector dimensionality reported by the response. |
|
|
43
|
+
|
|
44
|
+
Failures throw `EmbeddingsError` with a stable `code`:
|
|
45
|
+
`empty_input` (no inputs), `batch_too_large` (over the provider cap — OpenAI 2048,
|
|
46
|
+
DashScope 10), `request_failed` (non-2xx, secret-redacted message),
|
|
47
|
+
`response_malformed` (missing index, wrong dimensionality), `unsupported_model`
|
|
48
|
+
(via `assertEmbeddingsSupported` when the host checks
|
|
49
|
+
`ModelCapabilities.embeddings`).
|
|
50
|
+
|
|
51
|
+
## Request/response example
|
|
52
|
+
|
|
53
|
+
```json
|
|
54
|
+
{ "model": "text-embedding-3-small", "input": ["hello", "world"], "dimensions": 256 }
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Implementation example
|
|
58
|
+
|
|
59
|
+
```ts
|
|
60
|
+
import { createOpenAIEmbeddingsProvider } from "@arnilo/prism-providers/openai";
|
|
61
|
+
import { runEmbeddingsConformance } from "@arnilo/prism/testing/provider-conformance";
|
|
62
|
+
|
|
63
|
+
const embeddings = createOpenAIEmbeddingsProvider({ apiKey: process.env.OPENAI_API_KEY });
|
|
64
|
+
const result = await embeddings.embedMany({
|
|
65
|
+
model: "text-embedding-3-small",
|
|
66
|
+
inputs: ["hello", "world"],
|
|
67
|
+
});
|
|
68
|
+
// result.vectors.length === 2; result.usage.inputTokens reported
|
|
69
|
+
|
|
70
|
+
// Offline conformance (fake transport, no network):
|
|
71
|
+
await runEmbeddingsConformance({
|
|
72
|
+
provider: createOpenAIEmbeddingsProvider({ apiKey: "sk-test", fetch: fakeFetch }),
|
|
73
|
+
model: "text-embedding-3-small",
|
|
74
|
+
maxBatchSize: 2048,
|
|
75
|
+
sample: { inputs: ["a", "b"], dimensions: 2 },
|
|
76
|
+
});
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Extension and configuration notes
|
|
80
|
+
|
|
81
|
+
- Implement `EmbeddingsProvider` (`{ id, embedMany }`) for other vendors; the
|
|
82
|
+
contract is structural — no base class, no registry.
|
|
83
|
+
- Models declare support with `capabilities.embeddings`; hosts gate with
|
|
84
|
+
`modelSupportsEmbeddings(capabilities)` / `assertEmbeddingsSupported(model)`,
|
|
85
|
+
mirroring the structured-output guard pattern.
|
|
86
|
+
- `@arnilo/prism-memory` keeps its dependency-free `Embedder` host seam; adapters
|
|
87
|
+
bridge structurally (`createAlibabaEmbedder` remains assignable to `Embedder`
|
|
88
|
+
without importing it). The contract is a superset: it adds usage and per-item
|
|
89
|
+
error mapping.
|
|
90
|
+
- Adapters never auto-chunk: a batch over the provider cap rejects with
|
|
91
|
+
`batch_too_large`, so `embedBatched`-style callers own batching and preserve
|
|
92
|
+
per-item error attribution.
|
|
93
|
+
|
|
94
|
+
## Security and performance notes
|
|
95
|
+
|
|
96
|
+
- API keys resolve through the existing `CredentialValueSource` seam and are
|
|
97
|
+
redacted from every thrown error (`redactSecrets`); no new secret paths.
|
|
98
|
+
- Responses are read through the bounded transport (`readBoundedResponseJson` /
|
|
99
|
+
`readBoundedResponseText`) — response bodies cannot exhaust memory.
|
|
100
|
+
- Input text is never logged; error messages carry status and redacted body only.
|
|
101
|
+
- One HTTP request per `embedMany` call; response mapping allocates one vector
|
|
102
|
+
copy per input and nothing else. Per-item token caps are server-enforced;
|
|
103
|
+
the local cap is batch count.
|
|
104
|
+
|
|
105
|
+
## Related APIs
|
|
106
|
+
|
|
107
|
+
- [`@arnilo/prism-memory`](working-and-semantic-memory.md): `Embedder` host seam and `embedBatched` —
|
|
108
|
+
consumption side of the structural bridge.
|
|
109
|
+
- [Provider conformance](provider-conformance.md): `runEmbeddingsConformance` and
|
|
110
|
+
the offline conformance matrix this contract joins.
|
|
111
|
+
- [Provider packages](provider-packages.md): subpath import rules for
|
|
112
|
+
`@arnilo/prism-providers/openai` and `@arnilo/prism-providers/alibaba`.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-enterprise
|
|
5
|
+
`@arnilo/prism-core/enterprise/postgres` is one optional PostgreSQL composition for existing enterprise state seams:
|
|
6
6
|
|
|
7
7
|
| State | Composition property | Durable behavior |
|
|
8
8
|
| --- | --- | --- |
|
|
@@ -14,7 +14,7 @@
|
|
|
14
14
|
| ERP messaging | `erpMessaging` | Transactional outbox/inbox markers plus bounded, tenant-scoped at-least-once dispatch (migration 004). |
|
|
15
15
|
| Multi-party approvals | `createPostgresApprovalStore({ pool, schema, authority })` | Immutable approval requests, role/quorum decisions, revocation, bounded delegation, and atomic grant consumption (migration 005). |
|
|
16
16
|
|
|
17
|
-
`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`, `004_erp_messaging`, `005_erp_approvals`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-
|
|
17
|
+
`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`, `004_erp_messaging`, `005_erp_approvals`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-core/sessions/postgres`](postgres-persistence.md).
|
|
18
18
|
|
|
19
19
|
## When to use it
|
|
20
20
|
|
|
@@ -25,7 +25,7 @@ Use memory/file stores only for tests, demos, or a deliberately single-process h
|
|
|
25
25
|
## Inputs / request
|
|
26
26
|
|
|
27
27
|
```ts
|
|
28
|
-
import { createPostgresEnterpriseState } from "@arnilo/prism-enterprise
|
|
28
|
+
import { createPostgresEnterpriseState } from "@arnilo/prism-core/enterprise/postgres";
|
|
29
29
|
import { Pool } from "pg";
|
|
30
30
|
|
|
31
31
|
const pool = new Pool({
|
|
@@ -95,7 +95,7 @@ A migration creates `prism_policy_decisions`, `prism_evaluations`, `prism_work_i
|
|
|
95
95
|
## Implementation example
|
|
96
96
|
|
|
97
97
|
```ts
|
|
98
|
-
import { createPostgresErpMessaging } from "@arnilo/prism-enterprise
|
|
98
|
+
import { createPostgresErpMessaging } from "@arnilo/prism-core/enterprise/postgres";
|
|
99
99
|
|
|
100
100
|
const messaging = createPostgresErpMessaging({ pool, schema: "prism" });
|
|
101
101
|
const client = await pool.connect();
|
|
@@ -141,7 +141,7 @@ await messaging.dispatcher.replay({
|
|
|
141
141
|
|
|
142
142
|
```ts
|
|
143
143
|
import type { AgentIdentity } from "@arnilo/prism";
|
|
144
|
-
import { createPostgresEnterpriseState, type PostgresEnterpriseState } from "@arnilo/prism-enterprise
|
|
144
|
+
import { createPostgresEnterpriseState, type PostgresEnterpriseState } from "@arnilo/prism-core/enterprise/postgres";
|
|
145
145
|
|
|
146
146
|
const identity: AgentIdentity = {
|
|
147
147
|
tenantId: "tenant-1",
|
|
@@ -202,13 +202,13 @@ export async function recordEnterpriseState(state: PostgresEnterpriseState) {
|
|
|
202
202
|
|
|
203
203
|
## Extension and configuration notes
|
|
204
204
|
|
|
205
|
-
- `createModelRouter({ resolver, stateStore: state.modelRouter })` keeps allow-list, residency, fallback, and diagnostics behavior in `@arnilo/prism-model-router`; this package only supplies durable state. Router admission reservations (`reserveBudget`/`commitBudget`/`releaseBudget` on `state.modelRouter`) live in the `reservations` JSONB column of `prism_model_router_budgets`: one atomic UPSERT per admission, fencing-token-guarded commit/release in a SERIALIZABLE transaction, and TTL reconciliation as unknown usage; see [Model routing](model-routing.md).
|
|
205
|
+
- `createModelRouter({ resolver, stateStore: state.modelRouter })` keeps allow-list, residency, fallback, and diagnostics behavior in `@arnilo/prism-core/governance/model-router`; this package only supplies durable state. Router admission reservations (`reserveBudget`/`commitBudget`/`releaseBudget` on `state.modelRouter`) live in the `reservations` JSONB column of `prism_model_router_budgets`: one atomic UPSERT per admission, fencing-token-guarded commit/release in a SERIALIZABLE transaction, and TTL reconciliation as unknown usage; see [Model routing](model-routing.md).
|
|
206
206
|
- `createPostgresErpMessaging({ pool, schema? })` is the direct messaging composition. `outbox.append` and `inbox.record` accept a caller-owned `PoolClient`; the host must put them in the same transaction as its local mutation. The dispatcher owns only short claim/transition transactions and never invokes business callbacks or stores executable handlers.
|
|
207
207
|
- `createPostgresApprovalStore({ pool, schema?, authority })` is the direct approval composition (migration 005). `authority.resolveRoles(actor, request)` and `policyRevision` are host-owned; Prism persists only accepted role grants and delegation chains. `decide`/`revoke` lock the request row and revision-check the terminal transition in one transaction. `consume` accepts an optional caller-owned `client`; grant consumption and the protected action commit (or roll back) together.
|
|
208
208
|
- Rate/budget/circuit tables are capped like the memory store: `consumeRate`/`readBudget`/`addUsage`/`reserveBudget` accept `maxRateKeys`/`maxBudgetKeys` (the router passes its resolved limits) and evict the least-recently-used row on new-key insert — never the row just inserted, never a budget row holding an active reservation — else fail closed with `ERR_PRISM_MODEL_ROUTER_STATE`. Cleanup prunes expired reservations within its bounded batch.
|
|
209
209
|
- Policy/evaluation/query public contracts stay in their owning packages. This package exports `createPostgresEnterpriseState`, `createPostgresApprovalStore`, `createPostgresErpMessaging`, their options/result/types, and `EnterprisePostgresError`; it has no SQL, DDL, codec, queryable, or migration subpath.
|
|
210
210
|
- The fixed schema has no generic key/value table and no background cleanup scheduler. Schedule `state.cleanup()` from an authorized host job, size its bounded batch for the deployment, and monitor unknown work rows for reconciliation. Run protected integration checks with `PRISM_TEST_POSTGRES_URL="$DATABASE_URL" npm run test:postgres`; the command rejects an absent URL instead of silently skipping database coverage.
|
|
211
|
-
- The OPA adapter (`@arnilo/prism-policy/opa`, 0.0.28) records decisions into the same `state.policy` store unchanged via `evaluateAndAppend` — see [Policy and audit](policy-and-audit.md#opa-external-policy-adapter-arniloprism-policyopa-008).
|
|
211
|
+
- The OPA adapter (`@arnilo/prism-core/governance/policy/opa`, 0.0.28) records decisions into the same `state.policy` store unchanged via `evaluateAndAppend` — see [Policy and audit](policy-and-audit.md#opa-external-policy-adapter-arniloprism-policyopa-008).
|
|
212
212
|
- Request-path state SQL uses `SELECT`, `INSERT`, `UPDATE`, and `DELETE` on the eight state tables. The open/migration lifecycle additionally needs schema/catalog/advisory-lock and DDL permissions. Use a deployment migration principal for that lifecycle and a least-privilege request role for request traffic; this release intentionally does not ship a migration CLI or worker.
|
|
213
213
|
|
|
214
214
|
## Security and performance notes
|
package/docs/evaluations.md
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-evals` adds optional deterministic scorers, immutable datasets, bounded persistence-trace grading, explicit host model judges, pairwise comparisons, CI thresholds, live post-run scoring, and batch experiments over `AgentRunResult`. Scores are finite numbers in `[0, 1]` with optional reason/metadata and linkage to run/session/trace/experiment IDs.
|
|
5
|
+
`@arnilo/prism-core/governance/evals` adds optional deterministic scorers, immutable datasets, bounded persistence-trace grading, explicit host model judges, pairwise comparisons, CI thresholds, live post-run scoring, and batch experiments over `AgentRunResult`. Scores are finite numbers in `[0, 1]` with optional reason/metadata and linkage to run/session/trace/experiment IDs.
|
|
6
6
|
|
|
7
7
|
## When to use it
|
|
8
8
|
|
|
9
|
-
Use this package when a host needs offline quality checks or sampled live scoring without coupling scorers into core agent execution. Install it directly or through `@arnilo/prism-
|
|
9
|
+
Use this package when a host needs offline quality checks or sampled live scoring without coupling scorers into core agent execution. Install it directly or through the `@arnilo/prism-core` family package; installation does not attach scorers to runs.
|
|
10
10
|
|
|
11
11
|
## Inputs / request
|
|
12
12
|
|
|
@@ -60,7 +60,7 @@ import {
|
|
|
60
60
|
defineScorer,
|
|
61
61
|
runExperiment,
|
|
62
62
|
scoreRunLive,
|
|
63
|
-
} from "@arnilo/prism-evals";
|
|
63
|
+
} from "@arnilo/prism-core/governance/evals";
|
|
64
64
|
|
|
65
65
|
const scorer = defineScorer({
|
|
66
66
|
id: "contains-citation",
|
|
@@ -145,7 +145,7 @@ assertEvaluationThreshold(report, { minimumMean: 0.9, maximumFailures: 0 });
|
|
|
145
145
|
Plan 043 adds `datasetFromRuns`: it turns recorded runs (production incident transcripts included) into dataset items in one call — resolve through `createPersistenceTraceResolver`, redact, map through an optional host `toItem`, and append as a **new immutable dataset version**. The prior version object is never mutated.
|
|
146
146
|
|
|
147
147
|
```ts
|
|
148
|
-
import { datasetFromRuns, defineDataset } from "@arnilo/prism-evals";
|
|
148
|
+
import { datasetFromRuns, defineDataset } from "@arnilo/prism-core/governance/evals";
|
|
149
149
|
|
|
150
150
|
const result = await datasetFromRuns({
|
|
151
151
|
runIds: ["run_9f2", "run_a71"],
|
|
@@ -169,16 +169,16 @@ Omit `toItem` and the default mapping is used: `input` = first user message, `ex
|
|
|
169
169
|
- Feedback costs one bounded, owner-scoped query per curation batch (`store.feedback`); records are read id-only per the feedback linkage contract — scorer payloads are never copied. A feedback query failure (e.g. scope contract mismatch) omits `expected` instead of aborting the batch.
|
|
170
170
|
- Every item field passes the host `redactor` after host mapping — fail closed: a redactor failure or an item over the frozen 4 MiB cap (`ERR_PRISM_EVAL_CURATE`) skips the run or aborts the append before anything is persisted. Cross-tenant runs are never readable (resolver ownership check → `ownership mismatch` skip).
|
|
171
171
|
|
|
172
|
-
Prompt versions can ride the same primitives: [`assertPromptPromotion`](prompt-registry.md#eval-gated-promotion) in `@arnilo/prism-prompts` resolves two prompt versions, runs them through `runComparison`, and returns a `promote`/`hold` verdict with per-scorer aggregates and a bounded report — never applying the change itself.
|
|
172
|
+
Prompt versions can ride the same primitives: [`assertPromptPromotion`](prompt-registry.md#eval-gated-promotion) in `@arnilo/prism-core/governance/prompts` resolves two prompt versions, runs them through `runComparison`, and returns a `promote`/`hold` verdict with per-scorer aggregates and a bounded report — never applying the change itself.
|
|
173
173
|
|
|
174
174
|
## Coding and browser adversarial evaluations (0.0.9)
|
|
175
175
|
|
|
176
176
|
Release 0.0.9 ships curated network-free adversarial fixtures in package tests:
|
|
177
177
|
|
|
178
|
-
- `@arnilo/prism-coding-agent` `eval-fixtures.test.ts`: safe native list vs shell, Git path/ref injection, dirty-tree rollback, unknown named-check failure, PR-handoff artifact completeness, and prompt-injection file content under read-only tools.
|
|
178
|
+
- `@arnilo/prism-coding-tools/agent` `eval-fixtures.test.ts`: safe native list vs shell, Git path/ref injection, dirty-tree rollback, unknown named-check failure, PR-handoff artifact completeness, and prompt-injection file content under read-only tools.
|
|
179
179
|
- `browser` `eval-fixtures.test.ts`: stale snapshot refs, side-effect approval, private/loopback/file deny, upload/download/screenshot policy, CSS/evaluate target rejection, and hostile accessible-name text.
|
|
180
180
|
|
|
181
|
-
Fixtures reuse `@arnilo/prism-evals` (`defineDataset` / `defineScorer` / `scoreRun` / `assertEvaluationThreshold` / `serializeEvaluationReport`). Optional SWE-bench-compatible or live-browser harnesses remain host adapters — they are not default dependencies or quality claims. Protected real Docker/Playwright gates stay env-gated (`PRISM_TEST_DOCKER_SANDBOX`, `PRISM_LIVE_PLAYWRIGHT`) and never enter `sdk:ready`.
|
|
181
|
+
Fixtures reuse `@arnilo/prism-core/governance/evals` (`defineDataset` / `defineScorer` / `scoreRun` / `assertEvaluationThreshold` / `serializeEvaluationReport`). Optional SWE-bench-compatible or live-browser harnesses remain host adapters — they are not default dependencies or quality claims. Protected real Docker/Playwright gates stay env-gated (`PRISM_TEST_DOCKER_SANDBOX`, `PRISM_LIVE_PLAYWRIGHT`) and never enter `sdk:ready`.
|
|
182
182
|
|
|
183
183
|
## PostgreSQL enterprise state (0.0.23)
|
|
184
184
|
|
|
@@ -214,7 +214,7 @@ The protected runner (`scripts/phase27-erp-journey.test.mjs`) carries the journe
|
|
|
214
214
|
### Hard-gate usage
|
|
215
215
|
|
|
216
216
|
```ts
|
|
217
|
-
import { createErpInvariantScorers, erpInvariantDataset, scoreRun } from "@arnilo/prism-evals";
|
|
217
|
+
import { createErpInvariantScorers, erpInvariantDataset, scoreRun } from "@arnilo/prism-core/governance/evals";
|
|
218
218
|
|
|
219
219
|
const scorers = createErpInvariantScorers();
|
|
220
220
|
const records = await scoreRun({
|