@arnilo/prism 0.2.5 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -1
- package/README.md +1 -0
- package/dist/field-policy.d.ts +119 -0
- package/dist/field-policy.js +418 -0
- package/dist/index.d.ts +3 -1
- package/dist/index.js +2 -1
- package/dist/redaction.d.ts +6 -5
- package/dist/redaction.js +18 -10
- package/docs/0.1.0-readiness.md +10 -10
- package/docs/acp.md +2 -0
- package/docs/audit-export.md +151 -0
- package/docs/browser-automation.md +1 -1
- package/docs/coding-agent-tools.md +5 -4
- package/docs/coding-review-and-diagnostics.md +76 -0
- package/docs/coding-security.md +2 -0
- package/docs/coding-workspaces.md +69 -0
- package/docs/data-classification.md +82 -0
- package/docs/disaster-recovery.md +71 -0
- package/docs/enterprise-postgres-state.md +60 -7
- package/docs/evaluations.md +45 -0
- package/docs/forge-integration.md +6 -0
- package/docs/host-security.md +4 -0
- package/docs/index.md +12 -7
- package/docs/indexed-code-search.md +82 -0
- package/docs/language-intelligence.md +15 -0
- package/docs/migration.md +29 -0
- package/docs/operations.md +104 -0
- package/docs/policy-and-audit.md +34 -0
- package/docs/process-sessions.md +58 -3
- package/docs/release-0.2.7-evidence.md +514 -0
- package/docs/release-and-install.md +74 -3
- package/docs/work-artifacts-and-review.md +4 -0
- package/docs/workflows.md +48 -3
- package/package.json +2 -2
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Disaster recovery and backup operations
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
This page is the operator runbook for backup, restore, migration rollback, point-in-time recovery (PITR), and disaster recovery (DR), proven by plan 027 Task 7 with a protected drill that uses only standard PostgreSQL tools (`pg_dump` custom format, `pg_restore`, `pg_basebackup`, `psql`) plus the existing migration runner. The drill seeds representative multi-tenant 0.2.7 state (sessions, workflow/saga/ACP/conversation checkpoints, leases, legal holds, tenant quotas, policy decisions, evaluations, work idempotency, tool effects, model-router budgets, ERP outbox/inbox, approvals) through the real store APIs, backs it up, restores it into an explicitly confirmed disposable database, verifies per-table row counts and content digests equal the source, rehearses the 0.2.6 → 0.2.7 migration forward with old rows preserved, rehearses rollback by restoring the pre-upgrade backup, and runs PITR against a WAL-archived cluster to a point between two known writes. The recorded run lives in `docs/_evidence/phase27-dr-evidence.json`.
|
|
6
|
+
|
|
7
|
+
## When to use it
|
|
8
|
+
|
|
9
|
+
- Before running any migration or release in a production environment: decide the rollback path first (roll-forward repair preferred; backup-restore is the last resort and only in a disposable environment).
|
|
10
|
+
- When an operator must restore a database: read the guarded-command requirements below — this project has no "force restore" shortcut, and destructive commands always require explicit positive confirmation.
|
|
11
|
+
- When sizing backup windows or reviewing RPO/RTO: read the measured numbers in the evidence file and the ownership table (managed backup, encryption, retention scheduling, and cross-region replication are operator-owned and not claimed by Prism).
|
|
12
|
+
|
|
13
|
+
## Inputs / request
|
|
14
|
+
|
|
15
|
+
- A source instance URL (`PRISM_TEST_POSTGRES_URL` in the drill; any supported PostgreSQL ≥ 14 in practice).
|
|
16
|
+
- An explicitly named, disposable target database on a loopback non-production instance, supplied as `--target` plus the confirmation token `--confirm-target prism_dr_restore`. The target must not already exist; the drill refuses dirty state rather than clobbering it.
|
|
17
|
+
- A separate WAL-archived cluster for PITR (`PRISM_PITR_URL`) with `wal_level=replica`, `archive_mode=on`, and `archive_command='cp %p /wal_archive/%f'`; source and PITR containers must mount a shared host dir at `/dr` for artifact exchange.
|
|
18
|
+
- Sufficient free space (the drill asserts ≥ 512 MB headroom on the artifact dir before starting).
|
|
19
|
+
|
|
20
|
+
## Outputs / response / events
|
|
21
|
+
|
|
22
|
+
- A custom-format backup artifact (`.dump`) with its SHA-256 digest, byte size, duration, and table list count.
|
|
23
|
+
- A restore report: per-table count and content-digest equality against the source (application-level verification, not just exit codes), and duration.
|
|
24
|
+
- A migration report: the 0.2.6-era schema (migrations 001–003) with legacy rows, the upgraded 0.2.7 schema (all five migrations) with old rows preserved and new tables initialized empty, and the rollback rehearsal showing the pre-upgrade backup restores exactly and excludes 0.2.7 tables.
|
|
25
|
+
- A PITR report: recovery target time between two known writes, the earlier write present and the later write absent, recovery duration, and measured RPO/RTO.
|
|
26
|
+
|
|
27
|
+
## Request/response example
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
# Protected drill (standard tools only, orchestrated by the script):
|
|
31
|
+
PRISM_PITR_URL=postgresql://user:***@localhost:55436/postgres \
|
|
32
|
+
node scripts/phase27-dr.test.mjs \
|
|
33
|
+
--source "$PRISM_TEST_POSTGRES_URL" \
|
|
34
|
+
--target postgresql://user:***@localhost:55432/prism_dr_target \
|
|
35
|
+
--confirm-target prism_dr_restore
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
```ts
|
|
39
|
+
// Verifying a restore at the application level (from the drill):
|
|
40
|
+
const before = await digest(pool, "prism_erp_outbox"); // md5 of ordered rows
|
|
41
|
+
const after = await digest(restorePool, "prism_erp_outbox");
|
|
42
|
+
assert.equal(after, before); // content equality, not exit code
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Implementation example
|
|
46
|
+
|
|
47
|
+
The drill's legs: (1) seed multi-tenant state through `createPostgresPersistence` and `createPostgresEnterpriseState` plus `createPostgresApprovalStore` with a host authority; (2) `pg_dump -F c` into the shared `/dr` mount; (3) `createdb` the confirmed target and `pg_restore --no-owner --no-privileges`, then verify counts and digests per table; (4) build the 0.2.6-era schema from the raw DDL builders with the recorded migration registry rows, seed legacy rows through the real stores, take a backup, run `applyEnterpriseMigrations` (004/005 apply, old rows preserved, new tables empty), then restore the pre-upgrade backup into a fresh database and confirm 0.2.7 tables are absent; (5) on the WAL-archived cluster, take a `pg_basebackup`, insert two marker transactions a known distance apart, switch WAL and wait for archiving, then start a recovered instance with `recovery.signal`, a bounded `restore_command`, and `recovery_target_time` set between the two writes with `recovery_target_action=pause`, and verify the earlier marker exists and the later one does not.
|
|
48
|
+
|
|
49
|
+
## Extension and configuration notes
|
|
50
|
+
|
|
51
|
+
- Storage, encryption at rest, retention scheduling, and cross-region replication are operator-owned: the drill proves the command and verification path, not a managed backup service.
|
|
52
|
+
- Rollback decision tree: prefer roll-forward repair after a production migration. Restore-from-backup is the last resort, permitted only in a disposable evidence environment; the recorded loss window is "writes between the pre-upgrade backup and the rollback restore".
|
|
53
|
+
- Recovery parameters: `recovery_target_time` in the recovered instance needs `recovery.signal` (or `standby.signal`) present, a `restore_command`, and a full timestamp including the sub-second fraction and offset — truncating to whole seconds can land the recovery point before the intended write.
|
|
54
|
+
- Guards that must stay in place: source and target databases must differ; the target host must be loopback and the database name must not match production patterns; the target must not pre-exist; the confirmation token is mandatory; secret canaries seeded into the data must never appear in the manifest or console output.
|
|
55
|
+
- Quarterly re-run the drill and refresh `docs/_evidence/phase27-dr-evidence.json` as the schema evolves; treat any change to table counts/digests as needing a new recorded run.
|
|
56
|
+
|
|
57
|
+
## Security and performance notes
|
|
58
|
+
|
|
59
|
+
- Credentials are explicit and redacted: the manifest stores URLs with passwords masked, and the drill asserts the source/target/PITR passwords and the seeded secret canary never appear in the evidence, logs, or console. Connection strings are passed to the tools via the container environment, never printed.
|
|
60
|
+
- Destructive commands (dropping schemas/databases, restoring over an existing target) require explicit operator action; the drill fails closed on dirty state and never deletes anything itself.
|
|
61
|
+
- Legal-hold data is verified: legal-hold records and their referenced rows survive backup and restore with content digests intact; enforcement of holds stays host-owned (the stores preserve the records; the leases/quota/outbox lifecycle logic stays unchanged).
|
|
62
|
+
- Performance: measured in the disposable environment — backup 108,291 bytes in 122 ms, restore 382 ms, PITR recovery 1.2 s with the two markers a sub-second apart (RPO ≈ 0 s, RTO ≈ 1 s). These are environment-local measurements for sizing, not guarantees; the drill records sizes, timings, and digests in the evidence file and fails on breached frozen budgets.
|
|
63
|
+
- No exactly-once guarantee is claimed anywhere in the backup/restore path; the drill verifies observable equality (counts and digests) instead.
|
|
64
|
+
|
|
65
|
+
## Related APIs
|
|
66
|
+
|
|
67
|
+
- `createPostgresPersistence` / `createPostgresEnterpriseState` / `createPostgresApprovalStore` — the stores whose state the drill seeds and verifies.
|
|
68
|
+
- `applyEnterpriseMigrations` — the migration runner used for the forward upgrade rehearsal.
|
|
69
|
+
- `scripts/phase27-dr.test.mjs` — the protected drill; `docs/_evidence/phase27-dr-evidence.json` — the recorded run.
|
|
70
|
+
- [Operations runbook: high availability, failover, and fencing](operations.md) — the lease/fence model and failover ceiling the same state relies on.
|
|
71
|
+
- [Signed, hash-chained audit export](audit-export.md) — the append-only audit ledger preserved through backup/restore.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-enterprise-postgres` is one optional PostgreSQL composition for
|
|
5
|
+
`@arnilo/prism-enterprise-postgres` is one optional PostgreSQL composition for existing enterprise state seams:
|
|
6
6
|
|
|
7
7
|
| State | Composition property | Durable behavior |
|
|
8
8
|
| --- | --- | --- |
|
|
@@ -11,9 +11,10 @@
|
|
|
11
11
|
| Work mutations | `workIdempotency` | Atomic claim/CAS lifecycle for connector effects. |
|
|
12
12
|
| Model routing | `modelRouter` | Shared rate, budget, and circuit state for router replicas. |
|
|
13
13
|
| Tool effects | `toolEffects` | Durable `ToolEffectStore` claim/CAS for recoverable tool side effects (migration 002). |
|
|
14
|
-
|
|
|
14
|
+
| ERP messaging | `erpMessaging` | Transactional outbox/inbox markers plus bounded, tenant-scoped at-least-once dispatch (migration 004). |
|
|
15
|
+
| Multi-party approvals | `createPostgresApprovalStore({ pool, schema, authority })` | Immutable approval requests, role/quorum decisions, revocation, bounded delegation, and atomic grant consumption (migration 005). |
|
|
15
16
|
|
|
16
|
-
`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-session-store-postgres`](postgres-persistence.md).
|
|
17
|
+
`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`, `004_erp_messaging`, `005_erp_approvals`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-session-store-postgres`](postgres-persistence.md).
|
|
17
18
|
|
|
18
19
|
## When to use it
|
|
19
20
|
|
|
@@ -58,7 +59,7 @@ interface PostgresEnterpriseState {
|
|
|
58
59
|
readonly workIdempotency: IdempotencyStore;
|
|
59
60
|
readonly modelRouter: ModelRouterStateStore;
|
|
60
61
|
readonly toolEffects: ToolEffectStore;
|
|
61
|
-
readonly
|
|
62
|
+
readonly erpMessaging: PostgresErpMessaging;
|
|
62
63
|
cleanup(input: EnterpriseStateCleanupInput): Promise<EnterpriseStateCleanupResult>;
|
|
63
64
|
close(): Promise<void>;
|
|
64
65
|
}
|
|
@@ -68,6 +69,10 @@ interface PostgresEnterpriseState {
|
|
|
68
69
|
|
|
69
70
|
Work mutations expose six observable states: **absent**, `in_progress`, `completed`, `failed_retryable`, `failed_terminal`, and `unknown`. `begin()` atomically returns `acquired` or the existing record; `complete`/`fail`/`markUnknown` use claim-token plus version compare-and-swap. `unknown` requires an operator/connector-specific `resolveUnknown` decision. It is not automatically replayed and it does **not** claim exactly-once external effects.
|
|
70
71
|
|
|
72
|
+
ERP messaging exposes outbox states `pending`, `dispatched`, `retryable`, `completed`, `unknown`, and `dead_letter`. The caller owns the `pg.PoolClient` transaction: business mutation plus `erpMessaging.outbox.append(client, input)` commit or roll back together. Consumers call `erpMessaging.inbox.record(client, input)` before their local mutation in the same transaction; duplicate delivery returns `false`. `dispatcher.claim()` uses bounded `FOR UPDATE SKIP LOCKED` pages and leases. Acknowledgement, retry, and unknown transitions use claim-token plus version CAS. Expired leases become `unknown`; replay/dead-letter requires a host-verified actor and non-empty audit reference. Delivery remains at-least-once, never exactly-once.
|
|
73
|
+
|
|
74
|
+
Approvals expose request states `pending`, `approved`, `rejected`, `revoked`, and `consumed`. Immutable request data (action digest, requester, role/quorum requirements, separation flag, expiry, delegation depth) plus a monotonic `revision` and the accepted decision array live in one row. `decide`/`revoke` lock the row `FOR UPDATE` and revision-check the terminal transition in one transaction; a rejection is a terminal veto. `consume` verifies tenant, action digest, expiry, policy revision, and revision before flipping `approved` → `consumed` inside the caller-owned transaction, so grant consumption and the protected action commit (or roll back) together. Roles come from the host `ApprovalAuthority`; Prism never treats model/tool/subagent claims as principals.
|
|
75
|
+
|
|
71
76
|
Model-router state is asynchronous and owner/principal/provider/model scoped. Supplying it to `createModelRouter({ stateStore })` requires awaited `resolve`, `recordUsage`, and `recordOutcome` calls with verified identity. The legacy synchronous `providerSource` facade throws `ERR_PRISM_MODEL_ROUTER_ASYNC_STATE` when durable state is configured.
|
|
72
77
|
|
|
73
78
|
## Request/response example
|
|
@@ -85,10 +90,55 @@ Model-router state is asynchronous and owner/principal/provider/model scoped. Su
|
|
|
85
90
|
}
|
|
86
91
|
```
|
|
87
92
|
|
|
88
|
-
A migration creates `prism_policy_decisions`, `prism_evaluations`, `prism_work_idempotency`, three `prism_model_router_*` tables, and its separate `prism_enterprise_migrations` history. Migration `003_router_reservations` adds the nullable-by-default `reservations` JSONB column to `prism_model_router_budgets` (atomic reservation slots for router admission; 0.2.1 readers ignore it). Startup serializes per-schema setup with an advisory transaction lock and rejects checksum or catalog drift rather than silently repairing it.
|
|
93
|
+
A migration creates `prism_policy_decisions`, `prism_evaluations`, `prism_work_idempotency`, three `prism_model_router_*` tables, `prism_erp_outbox`, `prism_erp_inbox`, `prism_erp_approvals`, and its separate `prism_enterprise_migrations` history. Migration `003_router_reservations` adds the nullable-by-default `reservations` JSONB column to `prism_model_router_budgets` (atomic reservation slots for router admission; 0.2.1 readers ignore it). Migration `004_erp_messaging` adds tenant/message and tenant/consumer/message primary keys plus claim, lease, and inbox indexes. Migration `005_erp_approvals` adds the one-row-per-request approval table (PK `tenant_id + id`, status check, decisions JSONB, status/created indexes). Startup serializes per-schema setup with an advisory transaction lock and rejects checksum or catalog drift rather than silently repairing it.
|
|
89
94
|
|
|
90
95
|
## Implementation example
|
|
91
96
|
|
|
97
|
+
```ts
|
|
98
|
+
import { createPostgresErpMessaging } from "@arnilo/prism-enterprise-postgres";
|
|
99
|
+
|
|
100
|
+
const messaging = createPostgresErpMessaging({ pool, schema: "prism" });
|
|
101
|
+
const client = await pool.connect();
|
|
102
|
+
try {
|
|
103
|
+
await client.query("BEGIN");
|
|
104
|
+
await client.query("UPDATE invoices SET status = $1 WHERE tenant_id = $2 AND id = $3", ["posted", tenantId, invoiceId]);
|
|
105
|
+
await messaging.outbox.append(client, {
|
|
106
|
+
tenantId,
|
|
107
|
+
messageId: `invoice:${invoiceId}:posted`,
|
|
108
|
+
topic: "invoice.posted",
|
|
109
|
+
payload: { invoiceId },
|
|
110
|
+
});
|
|
111
|
+
await client.query("COMMIT");
|
|
112
|
+
} catch (error) {
|
|
113
|
+
await client.query("ROLLBACK");
|
|
114
|
+
throw error;
|
|
115
|
+
} finally {
|
|
116
|
+
client.release();
|
|
117
|
+
}
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Consumer transaction uses same client:
|
|
121
|
+
|
|
122
|
+
```ts
|
|
123
|
+
await client.query("BEGIN");
|
|
124
|
+
if (await messaging.inbox.record(client, { tenantId, consumer: "ledger", messageId })) {
|
|
125
|
+
await client.query("UPDATE ledger SET posted = TRUE WHERE tenant_id = $1 AND message_id = $2", [tenantId, messageId]);
|
|
126
|
+
}
|
|
127
|
+
await client.query("COMMIT");
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Dead-letter/replay is host-authorized and auditable:
|
|
131
|
+
|
|
132
|
+
```ts
|
|
133
|
+
await messaging.dispatcher.replay({
|
|
134
|
+
tenantId,
|
|
135
|
+
messageId,
|
|
136
|
+
expectedVersion,
|
|
137
|
+
auditRef: "audit:erp-replay:2026-08-17T00:00:00Z",
|
|
138
|
+
authorizedBy: verifiedOperatorIdentity,
|
|
139
|
+
});
|
|
140
|
+
```
|
|
141
|
+
|
|
92
142
|
```ts
|
|
93
143
|
import type { AgentIdentity } from "@arnilo/prism";
|
|
94
144
|
import { createPostgresEnterpriseState, type PostgresEnterpriseState } from "@arnilo/prism-enterprise-postgres";
|
|
@@ -153,11 +203,13 @@ export async function recordEnterpriseState(state: PostgresEnterpriseState) {
|
|
|
153
203
|
## Extension and configuration notes
|
|
154
204
|
|
|
155
205
|
- `createModelRouter({ resolver, stateStore: state.modelRouter })` keeps allow-list, residency, fallback, and diagnostics behavior in `@arnilo/prism-model-router`; this package only supplies durable state. Router admission reservations (`reserveBudget`/`commitBudget`/`releaseBudget` on `state.modelRouter`) live in the `reservations` JSONB column of `prism_model_router_budgets`: one atomic UPSERT per admission, fencing-token-guarded commit/release in a SERIALIZABLE transaction, and TTL reconciliation as unknown usage; see [Model routing](model-routing.md).
|
|
206
|
+
- `createPostgresErpMessaging({ pool, schema? })` is the direct messaging composition. `outbox.append` and `inbox.record` accept a caller-owned `PoolClient`; the host must put them in the same transaction as its local mutation. The dispatcher owns only short claim/transition transactions and never invokes business callbacks or stores executable handlers.
|
|
207
|
+
- `createPostgresApprovalStore({ pool, schema?, authority })` is the direct approval composition (migration 005). `authority.resolveRoles(actor, request)` and `policyRevision` are host-owned; Prism persists only accepted role grants and delegation chains. `decide`/`revoke` lock the request row and revision-check the terminal transition in one transaction. `consume` accepts an optional caller-owned `client`; grant consumption and the protected action commit (or roll back) together.
|
|
156
208
|
- Rate/budget/circuit tables are capped like the memory store: `consumeRate`/`readBudget`/`addUsage`/`reserveBudget` accept `maxRateKeys`/`maxBudgetKeys` (the router passes its resolved limits) and evict the least-recently-used row on new-key insert — never the row just inserted, never a budget row holding an active reservation — else fail closed with `ERR_PRISM_MODEL_ROUTER_STATE`. Cleanup prunes expired reservations within its bounded batch.
|
|
157
|
-
- Policy/evaluation/query public contracts stay in their owning packages. This package exports
|
|
209
|
+
- Policy/evaluation/query public contracts stay in their owning packages. This package exports `createPostgresEnterpriseState`, `createPostgresApprovalStore`, `createPostgresErpMessaging`, their options/result/types, and `EnterprisePostgresError`; it has no SQL, DDL, codec, queryable, or migration subpath.
|
|
158
210
|
- The fixed schema has no generic key/value table and no background cleanup scheduler. Schedule `state.cleanup()` from an authorized host job, size its bounded batch for the deployment, and monitor unknown work rows for reconciliation. Run protected integration checks with `PRISM_TEST_POSTGRES_URL="$DATABASE_URL" npm run test:postgres`; the command rejects an absent URL instead of silently skipping database coverage.
|
|
159
211
|
- The OPA adapter (`@arnilo/prism-policy/opa`, 0.0.28) records decisions into the same `state.policy` store unchanged via `evaluateAndAppend` — see [Policy and audit](policy-and-audit.md#opa-external-policy-adapter-arniloprism-policyopa-008).
|
|
160
|
-
- Request-path state SQL uses `SELECT`, `INSERT`, `UPDATE`, and `DELETE` on the
|
|
212
|
+
- Request-path state SQL uses `SELECT`, `INSERT`, `UPDATE`, and `DELETE` on the eight state tables. The open/migration lifecycle additionally needs schema/catalog/advisory-lock and DDL permissions. Use a deployment migration principal for that lifecycle and a least-privilege request role for request traffic; this release intentionally does not ship a migration CLI or worker.
|
|
161
213
|
|
|
162
214
|
## Security and performance notes
|
|
163
215
|
|
|
@@ -165,6 +217,7 @@ export async function recordEnterpriseState(state: PostgresEnterpriseState) {
|
|
|
165
217
|
- Every value is a bound parameter. Schema identifiers are validated; table/index names are fixed. Cursors embed and recheck ownership, so a foreign tenant cannot reuse a page cursor.
|
|
166
218
|
- Policy records cap at 64 KiB; evaluations at 64 KiB; work rows at 8 KiB; router material at 512 bytes. JSON rejects prototype-pollution keys, non-finite values, excess depth/properties, and over-size material.
|
|
167
219
|
- PostgreSQL transaction SQLSTATE `40001`/`40P01` retries whole safe transactions up to three times. Connector effects remain outside those transactions and ambiguous errors become `unknown` rather than being retried.
|
|
220
|
+
- ERP claim pages are tenant-scoped and bounded at 1,000 rows; default batch is 100, lease TTL is 30 seconds with a 5-minute hard cap, and retry attempts are capped at 10. Protected PostgreSQL evidence measured 1,000 queued rows per tenant across 10 tenants at p50 5.999 ms / p95 7.827 ms / p99 8.066 ms for 100-row claims; the representative plan used `prism_erp_outbox_claim_idx` with no sequential scan. This is comparison evidence, not a hardware-independent guarantee.
|
|
168
221
|
- Recorded `postgres:16-alpine` evidence (Node 24.18.0/Linux x64, 10 tenants × 10 principals × 1,000 policy/evaluation rows, 10,000 router keys, 16 clients) stayed below 50 ms p95 for point operations and 100 ms for cursor/cleanup pages. The highest recorded p95 was router-circuit contention at 28.410 ms. Fourteen representative `EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)` shapes used named indexes with no sequential scans. These are recorded comparison evidence, not hardware-independent guarantees.
|
|
169
222
|
|
|
170
223
|
## Related APIs
|
package/docs/evaluations.md
CHANGED
|
@@ -158,6 +158,51 @@ const page = await state.evaluations.query({ tenantId: "t1", userId: "u1", statu
|
|
|
158
158
|
|
|
159
159
|
Memory evaluation storage remains suitable for development and deterministic tests; it is not cross-replica production storage. PostgreSQL bounds each evaluation row to 64 KiB (reason/error 8 KiB each and metadata 32 KiB).
|
|
160
160
|
|
|
161
|
+
## ERP invariant evals (0.2.7)
|
|
162
|
+
|
|
163
|
+
Plan 027 adds two frozen exports to this package for the deterministic ERP release journey: `erpInvariantDataset` and `createErpInvariantScorers`. Scorers consume **structured journey facts only** — never model prose, credentials, or classified payloads. Each of the eight invariants is a hard 0/1 gate; no weighted average can hide an atomicity or security failure.
|
|
164
|
+
|
|
165
|
+
### Fact schema
|
|
166
|
+
|
|
167
|
+
The protected runner (`scripts/phase27-erp-journey.test.mjs`) carries the journey facts as JSON in `result.text`. Every scorer isolates its facts block and returns 1 only when every required fact is present and truthy:
|
|
168
|
+
|
|
169
|
+
| Invariant (scorer id) | Facts block | Required facts |
|
|
170
|
+
|---|---|---|
|
|
171
|
+
| `atomic-intent` | `atomic` | `committedAtomically` |
|
|
172
|
+
| `single-local-effect` | `delivery` | `singleLocalEffect`, `duplicateDelivered`, `businessMutationCount` |
|
|
173
|
+
| `compensation-terminal` | `compensation` | `compensated`, `reconciled`, `terminalStatus` |
|
|
174
|
+
| `quorum-provenance` | `quorum` | `distinctApprovers`, `requesterDenied`, `subagentDenied`, `revokedDenied`, `provenance` |
|
|
175
|
+
| `chain-verification` | `chain` | `verified`, `tamperedDetected`, `nextDigest` |
|
|
176
|
+
| `no-leak` | `noLeak` | `classifiedDenied`, `crossTenantDenied`, `secretRedacted` |
|
|
177
|
+
| `fenced-failover` | `fencedFailover` | `resumedByPeer`, `staleWriteRejected`, `cursorPreserved`, `failoverMs` |
|
|
178
|
+
| `restore-equality` | `restore` | `factsMatch`, `digestsMatch`, `drEvidenceFresh`, `restoreMs` |
|
|
179
|
+
|
|
180
|
+
### Hard-gate usage
|
|
181
|
+
|
|
182
|
+
```ts
|
|
183
|
+
import { createErpInvariantScorers, erpInvariantDataset, scoreRun } from "@arnilo/prism-evals";
|
|
184
|
+
|
|
185
|
+
const scorers = createErpInvariantScorers();
|
|
186
|
+
const records = await scoreRun({
|
|
187
|
+
result, // AgentRunResult whose .text is the JSON journey facts
|
|
188
|
+
scorers,
|
|
189
|
+
datasetId: erpInvariantDataset.id,
|
|
190
|
+
});
|
|
191
|
+
if (records.some((record) => record.status !== "scored" || record.score !== 1)) {
|
|
192
|
+
process.exitCode = 1; // a single failing invariant fails the whole gate
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
### Execution command and substitutes
|
|
197
|
+
|
|
198
|
+
```sh
|
|
199
|
+
# Protected run (requires a disposable PostgreSQL instance):
|
|
200
|
+
PRISM_TEST_POSTGRES_URL=postgresql://... node --test scripts/phase27-erp-journey.test.mjs
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
The journey reuses the two-replica failover worker (`scripts/phase27-ha-worker.mjs`) and asserts the comprehensive DR drill evidence (`docs/_evidence/phase27-dr-evidence.json`) is present and not stale. Local substitutes are labelled in the journey evidence and never converted into production claims: an in-memory WORM/SIEM sink (host owns the immutable store in production), in-memory saga checkpoint/lease stores (saga durability is proven in its own suite), and a logical pg-client backup/restore of the ERP tables (comprehensive PITR is in the DR drill evidence). Passing this protected journey **does not** satisfy the 0.3.0 live-service matrix.
|
|
204
|
+
|
|
205
|
+
|
|
161
206
|
## Related APIs
|
|
162
207
|
|
|
163
208
|
- [Agent/session runtime](agent-session-runtime.md): `AgentRunResult` and `session.run()`
|
|
@@ -98,12 +98,18 @@ if (!report.alreadyMerged && report.pushed) {
|
|
|
98
98
|
|
|
99
99
|
## Extension and configuration notes
|
|
100
100
|
|
|
101
|
+
Forge breadth is demand-gated (plan 026 Task 4): GitLab and Bitbucket adapters stay deferred while no named consumer is recorded in the phase26 freeze manifest's demand registry (`scripts/phase26-freeze-manifest.json`). A deferred adapter has no source file, docs page, or export; activation requires recording a named host/consumer/date/use case and shipping at most one adapter (GitLab or Bitbucket) against the existing `ForgeOperations` contract. Unsupported provider operations fail with a stable typed error — no fake capability, no catalog/factory.
|
|
102
|
+
|
|
101
103
|
Credentials resolve per call through the host resolver; GitHub App installation tokens and PATs are both supported (same `Bearer` REST header and `x-access-token` git header). Least-privilege guidance: App installation tokens with `contents: write` + `pull_requests: write` + `issues: read` cover the six operations; PATs should be fine-grained to the single repository and read/write scope needed. Policy denials propagate as the core `ExecutionDeniedError` (`ERR_PRISM_EXECUTION_DENIED`) — no forge request is attempted — so hosts can distinguish refusal from forge failure. Pagination is sequential (per-request `pagesPerOperation` cap); `requestConcurrency` is a validated ceiling, not a target. The adapter performs no DNS/egress control itself — sandboxed hosts route forge traffic through the Phase 9 egress policy (Task 6).
|
|
102
104
|
|
|
103
105
|
## Security and performance notes
|
|
104
106
|
|
|
105
107
|
Tokens never appear in argv, git config files, logs, model context, or stored events: REST uses the `Authorization` header on a bounded `fetch`, and git uses `GIT_CONFIG_*` environment variables scoped to the single push process. Request bodies and responses are bounded by `payloadBytes` (streamed, content-length pre-checked); timeouts and rate-limit backoff respect `requestTimeoutMs` and `Retry-After`; page fetches stop at `pagesPerOperation`. Repository binding is fixed at construction; tenant binding is checked per mutation; ownership mismatch fails closed. Rate-limit responses map to `ERR_PRISM_FORGE_RATE_LIMIT`, 404 to `ERR_PRISM_FORGE_API`, 422 to `ERR_PRISM_FORGE_STALE`, 401/403 to `ERR_PRISM_FORGE_AUTH`, and cap violations to `ERR_PRISM_FORGE_LIMIT`.
|
|
106
108
|
|
|
109
|
+
## Protected journey cross-link (0.2.6, plan 026 Task 7)
|
|
110
|
+
|
|
111
|
+
The protected coding journey runs the real forge leg end to end: the packed consumer clones `PRISM_CODING_FORGE_REPOSITORY`, pushes the run-suffixed branch, creates the PR with lookup-before-create idempotency (the ToolEffectStore dedupes replays), reads check runs, reconciles the handoff, and cleans up by closing the PR (`PATCH state=closed`) and deleting the branch — credentials late-bound via the resolver and `GIT_CONFIG_*` env, never argv or logs. See [Release and install](release-and-install.md).
|
|
112
|
+
|
|
107
113
|
## Related APIs
|
|
108
114
|
|
|
109
115
|
- [Tool effects](tool-effects.md): `ToolEffectStore` idempotency and unknown-outcome recovery used by every forge mutation
|
package/docs/host-security.md
CHANGED
|
@@ -131,6 +131,9 @@ Wire those values where they matter: provider adapters receive the resolved cred
|
|
|
131
131
|
|
|
132
132
|
## Security and performance notes
|
|
133
133
|
|
|
134
|
+
- 0.2.7 (plan 027 Task 8) adds field-level classification and fail-closed redaction at data boundaries: `applyFieldPolicy` walks JSON-like values with explicit `allow`/`redact`/`tokenize`/`deny` decisions, the protected default denies unknown fields on outbound/persisted boundaries, and labels come from per-boundary `labelFor` hints — never auto-discovered. Policy errors, cycles, unsupported types, and budget breaches fail closed without echoing values; the audit seam transforms before canonical hashing and retains only `{path, reason}` provenance; the telemetry seam drops attributes on policy error. See [Data classification and field-level redaction](data-classification.md) for the full contract, boundary matrix, limits, and the recorded overhead vs the pre-existing redaction walk.
|
|
135
|
+
|
|
136
|
+
|
|
134
137
|
- Fail closed: unknown providers, unknown tools, denied tools, invalid tool arguments, missing skill tool dependencies, trust failures, permission failures, append conflicts, and validator failures should stop the unsafe action.
|
|
135
138
|
- Prism does not sandbox host tools, extensions, provider adapters, credential resolvers, or custom middleware. Use OS/container/process isolation when code is untrusted.
|
|
136
139
|
- Redaction is exact known-secret replacement only. It is not arbitrary secret detection, entropy scanning, or DLP.
|
|
@@ -234,3 +237,4 @@ Every durable `AgentEventSource` page/subscribe and tool-effect claim rechecks e
|
|
|
234
237
|
- [Database persistence](database-persistence.md): production schema, ownership, indexes, retention, and adapter readiness checklist.
|
|
235
238
|
- [Enterprise PostgreSQL state](enterprise-postgres-state.md): durable policy/evaluation/work/router state and database-role boundary.
|
|
236
239
|
- [Provider caching](provider-caching.md): cache keys and provider-owned header safety rules.
|
|
240
|
+
- [Data classification and field-level redaction](data-classification.md): the field policy contract, protected default, and fail-closed boundary matrix (plan 027 Task 8).
|
package/docs/index.md
CHANGED
|
@@ -7,7 +7,8 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
7
7
|
|
|
8
8
|
## Identity and governance
|
|
9
9
|
- [Agent identity](agent-identity.md): host-verified `Principal` / `AgentIdentity`, delegation narrowing, ownership projection, and redacted telemetry refs for enterprise runs/tools/MCP/A2A/workflows; optional OIDC/JWKS verifier adapter (`@arnilo/prism-credentials-node/oidc` — pinned issuer/audience/JWKS, bounded claims, fail closed).
|
|
10
|
-
- [Policy and audit](policy-and-audit.md): optional `@arnilo/prism-policy` decision ledger (allow/deny/modify/approval), evidence refs only, cursor-paginated WORM export,
|
|
10
|
+
- [Policy and audit](policy-and-audit.md): optional `@arnilo/prism-policy` decision ledger (allow/deny/modify/approval), evidence refs only, cursor-paginated WORM export, durable PostgreSQL composition, and multi-party approvals — immutable requests with role/quorum requirements, requester/approver separation, expiry, revocation, bounded delegation, rejection, policy-revision pins, and atomic grant consumption via a host `ApprovalAuthority` (NIST AC-5 applied as control guidance, not certification); 0.0.28 adds the OPA REST evaluator (`@arnilo/prism-policy/opa` — pinned SSRF-checked endpoint, redacted input, fail-closed deny, optional bundle-revision pin); 0.2.1 makes the OPA decision fetch DNS-pinned (core `pinnedFetch`).
|
|
11
|
+
- [Signed, hash-chained audit export](audit-export.md): `@arnilo/prism-policy` exports tenant-scoped audit records as signed, hash-chained batches — canonical RFC 8785 envelopes, SHA-256 record chain, host-signed manifests, WORM acknowledgement gating cursor advance, SIEM mirroring with replayable pending status, redaction/legal-hold provenance, and independent `verifyAuditBatch`/CLI verification with no key storage or vendor SDKs in Prism.
|
|
11
12
|
- [Model routing](model-routing.md): optional `@arnilo/prism-model-router` allow-list/residency/budget/rate/circuit/fallback governance with redacted diagnostics; budget admission reserves per-request caps atomically (commit/release/TTL reconciliation); durable state requires awaited identity-scoped calls; host-configurable selection policies (reference cost/latency policy ranks by `ModelCost` then in-memory latency EMA fed from `recordOutcome`).
|
|
12
13
|
|
|
13
14
|
## Agent/session runtime
|
|
@@ -17,6 +18,9 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
17
18
|
- [Guardrails](guardrails.md): typed fail-closed input/output/tool checks with buffered provider output and redacted decision records.
|
|
18
19
|
- [Agent events](agent-events.md): live `session.subscribe` plus durable `AgentEventSource` page/subscribe/resume for cross-replica reconnect; message/progress deltas never create spans. Durable sources: PostgreSQL `LISTEN`/`NOTIFY` (reference, `@arnilo/prism-session-store-postgres` root export) and NATS JetStream (`@arnilo/prism-session-store-nats`, FR-5) with restart-stable durable consumer identity (`prism_<hmac16>`) for cursor resume across crash/restart.
|
|
19
20
|
- [Observability](observability.md): OTel GenAI agent/provider/tool hierarchy, host context parenting, bounded trace linkage, safe evaluation events, controlled metrics, and exporter isolation.
|
|
21
|
+
- [Operations runbook](operations.md): the high-availability/failover runbook for plan 027 Task 6 — LeaseStore/CheckpointStore fencing model, local-registry limitations, uncertain-commit replay rules, the recorded two-replica drill (`scripts/phase27-ha.test.mjs`, evidence in `docs/_evidence/phase27-ha-evidence.json`), failover ceiling, and the prohibition on manual lease unlocks.
|
|
22
|
+
- [Disaster recovery and backup operations](disaster-recovery.md): the plan 027 Task 7 runbook — standard-tool backup/restore/migration-rollback/PITR/DR drill (`scripts/phase27-dr.test.mjs`), guarded commands, app-level verification, the rollback decision tree, and measured RPO/RTO in `docs/_evidence/phase27-dr-evidence.json`.
|
|
23
|
+
- [Data classification and field-level redaction](data-classification.md): the plan 027 Task 8 contract — `applyFieldPolicy` walking JSON-like values with allow/redact/tokenize/deny decisions, the fail-closed protected default, per-boundary `labelFor` hints (no auto-discovery), sparse-copy overhead, and the ERP-T9 leak matrix incl. egress/audit/telemetry seams.
|
|
20
24
|
- [Evaluations](evaluations.md): deterministic and bounded trace/model-judge/pairwise scoring, CI thresholds, OTel trace-reference linkage, coding/browser adversarial fixtures, ID-only linkage to immutable owned run feedback, and optional durable PostgreSQL records.
|
|
21
25
|
- [Runs and usage ledger](runs-and-usage.md): durable run/event/tool/usage persistence, optional bounded FIFO durability policies, session snapshot caching, and immutable run/trace feedback.
|
|
22
26
|
- [Performance limits](performance.md): **0.1.0 capacity envelopes** (frozen performance contract, 24 network-free + 16 protected p95 rows, startup/pack rows), 0.0.26 coding-intelligence/process/forge/egress network-free evidence, 0.0.25 durable-loop/HITL/A2UI network-free evidence, 0.0.24 distributed event/effect PostgreSQL evidence, 0.0.23 enterprise state evidence, 0.0.15 network-free provider/RAG/memory benchmark evidence and frozen caps, bounded evaluation traces/judges/reports, and production sizing assumptions.
|
|
@@ -34,7 +38,7 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
34
38
|
- [Database persistence](database-persistence.md): production persistence contracts, shared checksummed migration/full-shape catalog primitives (`@arnilo/prism/testing/persistence-schema`), conditional append, indexes, `readBranchPath`, reference relational schema, retention/legal-hold/quota lifecycle (`lifecycle`), and NoSQL mapping. `appendSession` gains version/CAS (migration `008_session_version`); durable adapters must pass `assertStateConcurrencyConforms` (`@arnilo/prism/testing/state-concurrency-conformance`: approval/checkpoint-CAS/cursor/idempotency/reservation/conversation-metadata/unknown-outcome probes; memory leg in `npm test`, durable legs in `test:postgres`/`test:nats`, `scripts/phase22-conformance.test.mjs` gate accounting).
|
|
35
39
|
- [SQLite persistence](sqlite-persistence.md): optional `better-sqlite3` adapter with session/run storage, checkpoints/leases, feedback, FTS `searchSessions` (migration-v4), and transactionally verified/backfilled migration metadata.
|
|
36
40
|
- [PostgreSQL persistence](postgres-persistence.md): optional pooled `pg` adapter with session/run/checkpoint/lease/feedback storage, FTS `searchSessions` (migration-v4), advisory-locked checksummed/full-shape migrations, and opt-in live conformance.
|
|
37
|
-
- [Enterprise PostgreSQL state](enterprise-postgres-state.md): optional `@arnilo/prism-enterprise-postgres` composition for durable policy/evaluation/work-idempotency/model-router/`toolEffects` state, exact ownership, checksummed migrations (001-
|
|
41
|
+
- [Enterprise PostgreSQL state](enterprise-postgres-state.md): optional `@arnilo/prism-enterprise-postgres` composition for durable policy/evaluation/work-idempotency/model-router/`toolEffects` state, transactional ERP outbox/inbox messaging, and multi-party approval records (`createPostgresApprovalStore`, migration 005 — one locked row per request, revision-checked terminal transitions, caller-transaction grant consumption), exact tenant ownership, checksummed migrations (001-005; 003 adds router budget reservation slots; 004 adds bounded at-least-once dispatch), and explicit cleanup.
|
|
38
42
|
- [Migration guide](migration.md): **0.1.4 → 0.1.5** documented breaking cut — deprecated-option removal (the inert provider request knobs, `maxToolRounds` alias, observational-memory flat keys/worker aliases, `autoResizeImages`, `INIT_PROVIDERS`) with exact replacement table, before/after examples, and fail-closed refusal behavior; **0.0.28 → 0.1.0** release-candidate hardening (no migration); **0.0.17 → 0.1.0 upgrade matrix** (store compatibility per release line: compatible / tested migration / tested refusal, plus breaking-default callouts); **0.0.27** ACP coding-host interop (capability advertise-when, session modes/config, MCP select, lifecycle events, elicitation); **0.0.26** coding intelligence, managed processes, forge, and safe egress; **0.0.25** durable custom loops and shared human-in-the-loop decisions; **0.0.24** distributed events and recoverable tool effects; **0.0.23** enterprise PostgreSQL state adapters (async router and work reconciliation); **0.0.22** third-party behavior integrations (Caveman, Ponytail); **0.0.21** coding-tool capability gaps (`outputMode`, glob, read-before-write, delete/move, aggregator 9/4); **0.0.20** progressive skill disclosure, empty registry default, `load_skill`, priority budget demotion, optional tool-result fold; **0.0.19** observational memory lifecycle; **0.0.15** OpenAI hosted tools/continuation/Realtime, exact AI SDK v4 matrix, RAG lifecycle/reranking/trust/status, and memory export/rebuild; **0.0.14** conversations, memory consent/lifecycle, artifact co-work review, AG-UI co-work events, scoped M365/GWS OAuth connectors, browser checkpoints, device contracts, and Alibaba/Ollama providers; plus prior release migrations.
|
|
39
43
|
- [Node JSONL session store](node-jsonl-session-store.md): development-only JSONL file adapter for single-process Node hosts; no cross-process safety; `searchSessions` throws `SessionSearchUnsupportedError`.
|
|
40
44
|
- [Persistence, credentials, and multimodality primitives](persistence-credentials-multimodality-primitives.md): Plan 056 inventory — session/run-ledger/persistence contracts, credential/OAuth seams, content/resource/model capabilities, package dependency matrix, conformance matrix, and threat model for production adapters.
|
|
@@ -74,9 +78,9 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
74
78
|
- [Work connectors](work-connectors.md): connector principles, capability gates, scoped OAuth establishment (0.0.14), and out-of-scope boundaries (Slack/Teams channels not shipped) for Microsoft 365 / Google Workspace.
|
|
75
79
|
- [Browser automation](browser-automation.md): optional `@arnilo/prism-browser` with host-supplied Playwright contexts, AI-mode snapshots/refs, ordered `browser_open`/`browser_snapshot`/`browser_act`/`browser_close` plus (0.1.4) `browser_evaluate`/`browser_observe` and CDP `block_urls`/`unblock_urls`/`throttle`/`emulate` act actions on Chromium hosts, egress/side-effect/upload/download/screenshot policy, finite page/action/snapshot/network/artifact caps, and 0.0.14 verified-state checkpoints with reload/verify-before-side-effect.
|
|
76
80
|
- [Device adapters](device-adapters.md): deny-by-default realtime voice / desktop-control contract + conformance (0.0.14); no vendor package — admission fails closed without explicit consent+sandbox+approval, stream bounds, shared `RunLimits`, redacted telemetry.
|
|
77
|
-
- [Coding agent tools](coding-agent-tools.md): optional `shell`, `read`, `write`, `edit`, `repo_list`, `repo_search`, `glob`, `delete`, and `move` definitions plus opt-in `createGitTools()` / `coding_check`, opt-in `createAskUserDecisionTool` (single/multi/free-text + durable suspend glue), and `runCodingGoalVerify`; durable plan/todo Markdown helpers with workflow `state.coding` checkpoint metadata; streamed text pages, `repo_search` `outputMode`, bounded glob, optional read-before-write, optional Git-aware (`createGitAwareRepositoryOperations`) ignore-aware enumeration with native fallback, finite Git/check/plan/ask caps, bounded image/edit reads and write/edit payloads, finite shell wall/total-output limits, secure host-owned spill cleanup, pluggable bounded operation contracts, per-path mutation serialization, and optional `ExecutionPolicy`. 0.1.6 adds the optional [document reader](document-reader.md) slot (`@arnilo/prism-document-reader`, plan 018 closeout `doc-reader`): bounded PDF/DOCX literal-text extraction behind `createReadTool({ documentReader })` with magic-byte format gating, input/page/text caps, fail-closed optional peer parsers, and no embedded-content execution or external fetching. 0.1.6 also adds opt-in recursive `delete` (`recursive: true`, bounded fan-out, symlink children never followed) and bounded `{a,b}` glob expansion (`braceExpansion`, max 128 alternatives / 4096 bytes, fail-closed) behind plan 018 closeout `delete-glob`. No PDF/trash/PTY in the 0.0.21 baseline (0.1.6's document reader is the demand-gated optional exception); Phase 9 adds optional language intelligence (separate page). Limits do not sandbox host access—gate with permission/trust policy and `@arnilo/prism-coding-security`.
|
|
81
|
+
- [Coding agent tools](coding-agent-tools.md): optional `shell`, `read`, `write`, `edit`, `repo_list`, `repo_search`, `glob`, `delete`, and `move` definitions plus opt-in `createGitTools()` / `coding_check`, opt-in `createAskUserDecisionTool` (single/multi/free-text + durable suspend glue), and `runCodingGoalVerify`; durable plan/todo Markdown helpers with workflow `state.coding` checkpoint metadata; streamed text pages, `repo_search` `outputMode`, bounded glob, optional read-before-write, optional Git-aware (`createGitAwareRepositoryOperations`) ignore-aware enumeration with native fallback, finite Git/check/plan/ask caps, bounded image/edit reads and write/edit payloads, finite shell wall/total-output limits, secure host-owned spill cleanup, pluggable bounded operation contracts, per-path mutation serialization, and optional `ExecutionPolicy`. 0.1.6 adds the optional [document reader](document-reader.md) slot (`@arnilo/prism-document-reader`, plan 018 closeout `doc-reader`): bounded PDF/DOCX literal-text extraction behind `createReadTool({ documentReader })` with magic-byte format gating, input/page/text caps, fail-closed optional peer parsers, and no embedded-content execution or external fetching. 0.1.6 also adds opt-in recursive `delete` (`recursive: true`, bounded fan-out, symlink children never followed) and bounded `{a,b}` glob expansion (`braceExpansion`, max 128 alternatives / 4096 bytes, fail-closed) behind plan 018 closeout `delete-glob`. No PDF/trash/PTY in the 0.0.21 baseline (0.1.6's document reader is the demand-gated optional exception); Phase 9 adds optional language intelligence (separate page). 0.2.6 adds the optional [Indexed code search](indexed-code-search.md) seam: host-owned incremental index (`update/remove/search/status/dispose`) with explicit `indexed_literal`/`semantic` modes behind `createIndexedRepositoryOperations`, literal remains the default, stale/failed/unsupported indexes fail closed with `ERR_PRISM_INDEX_*` and results are labeled `untrusted_index`. 0.2.6 also adds [Coding workspaces](coding-workspaces.md) (plan 026 Task 3): `createCodingWorkspaceLifecycle` registers host repositories and creates/lists/locks/removes linked worktrees with CheckpointStore CAS records, LeaseStore fencing, credential-free remote fingerprints, and a cleanup policy that refuses dirty/locked/unowned/mismatched trees unless the host allows it. 0.2.6 adds [Coding review and diagnostics](coding-review-and-diagnostics.md) (plan 026 Task 6): bounded patch-review manifests (`createCodingPatchReviewManifest` + `assertCodingPatchAccepted`, pending/accepted/rejected/superseded bound to patch digest + artifact revision + repository/worktree/base/head identity, composed over the server ArtifactService, never applying/committing automatically), normalized LSP/check diagnostics with deterministic added/removed/unchanged deltas, and opt-in LSP document synchronization (`syncDocument`, pull diagnostics with resultId reuse, stale-version guards). Limits do not sandbox host access—gate with permission/trust policy and `@arnilo/prism-coding-security`.
|
|
78
82
|
- [Language intelligence](language-intelligence.md): optional host-activated `createLanguageIntelligence` — bounded in-package LSP 3.17 JSON-RPC client (Content-Length framing), host-selected server command/args per language, workspace symbols/definitions/references/diagnostics/hover/rename; lazy spawn; URI root confinement; rename gated by `ExecutionPolicy` + atomic write/mutation queue; frozen message/diagnostic/pending/result/timeout/server caps. No `vscode-languageserver-protocol` dependency.
|
|
79
|
-
- [Process sessions](process-sessions.md): optional host-activated `createProcessSessions` — long-running process registry (start/cursor-paged output/input/wait/signal/kill/release), native or sandbox `startProcess` backend (fail closed when absent), ownership/identity + expiry sweep on access, `reconcile` / sandbox-loss → `unknown` (never fabricates exitCode), durable command fingerprint metadata, `CodingProcessEvent` host sink, `ExecutionPolicy` before spawn and on mutate, frozen session/input/lifetime/output caps; PTY fails closed as unsupported.
|
|
83
|
+
- [Process sessions](process-sessions.md): optional host-activated `createProcessSessions` — long-running process registry (start/cursor-paged output/input/wait/signal/kill/release), native or sandbox `startProcess` backend (fail closed when absent), ownership/identity + expiry sweep on access, `reconcile` / sandbox-loss → `unknown` (never fabricates exitCode), durable command fingerprint metadata, `CodingProcessEvent` host sink, `ExecutionPolicy` before spawn and on mutate, frozen session/input/lifetime/output caps; host-selected PTY (`pty: true` delegates only to the host `ptyBackend`, fails closed as unsupported when absent, bounded resize/TERM/attach caps). Durable process recovery (plan 026 Task 5): with `checkpoints`+`leases`+`ownerId`, intent is persisted before spawn and transitions are CAS/fence-written; `recover()` is attach-if-attested via a host `recoveryBackend`, otherwise starting/running records atomically become `unknown` (no fabricated exit, no PID probing), fenced so two replicas cannot both own a process.
|
|
80
84
|
- [Forge integration](forge-integration.md): optional host-activated `createGitHubForge` — reference GitHub adapter (issue context, authenticated push via `BoundGitRunner` + `GIT_CONFIG_*` credential injection, PR create/update, review comments, check/status retrieval, bounded `reconcileHandoff`), every mutation gated by `ExecutionPolicy` and recorded in `ToolEffectStore` (retry never duplicates PRs/comments), typed `ForgeError` codes (auth/API/stale/rate-limit/limit/ownership), frozen page/payload/comment/concurrency/timeout caps, no octokit dependency, tokens never in argv/logs/events.
|
|
81
85
|
- [Coding execution approval and sandboxing](coding-security.md): path/command approval, identity-scoped caching, shell-turn exclusivity, required `workspaceMode` (`host`/`sandbox`) with fail-closed mixed wiring, `createSandboxCodingComposition()` sandbox capability metadata — 0.2.0 plan 020 Task 4 ships explicit `SandboxCapabilities` (`workspaceCoherent`/`filesystemIsolated`/`networkIsolated`/`processIsolated`/`privilegeIsolated`/`egressRestricted`) with omission resolving false, truthful Docker/native metadata, and `containmentClaim` retained only as a deprecated conservative projection — disposable Docker/OCI sandbox reference with bounded workspace import/export (0.1.6 adds the Linux-only network-free `createNativeSandbox` backend — fresh netns per command via `unshare`, `ulimit` hard caps, cwd containment, fails closed where egress denial is impossible), optional `DisposableSandbox.startProcess` / `SandboxProcessHandle` for process-session backends, and allow-list egress (`createEgressPolicy` deny-all exact rules + frozen presets, `createAllowListEgressProxy` HTTP/CONNECT proxy with pinned-DNS rebinding defense, private/metadata IP denial, redirect re-validation + hop cap, byte/time caps, per-decision audit, `composeEgressSandboxNetwork` attestation recorded as `prism.egress.*` labels; TLS pass-through, no interception).
|
|
82
86
|
|
|
@@ -99,12 +103,12 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
99
103
|
- [Supervisor delegation](supervisors.md): optional explicit child allow-list, derived memory scopes, narrowing-only permissions, lifecycle hooks, nested delegation, cancellation, finite budgets, host-projected delegation telemetry, and separate A2A durable adapter boundary.
|
|
100
104
|
- [A2A interoperability](a2a.md): A2A 1.0 JSON-RPC/HTTPS cards plus host-owned durable task get/list/cancel/subscribe, shared `AgentEventSource` task adapter, bounded rich parts/replay, principal-scoped push configs, exact-origin verified client, rich stream seam for explicit AG-UI fronting, and server-side `createAgUiA2AServer` exposure of a local AG-UI agent (0.0.26).
|
|
101
105
|
- [Frontend interoperability (AG-UI and ACP)](ag-ui.md): optional `@arnilo/prism-ag-ui` full AG-UI 0.0.57 input/event/capability mapper, authorized Web handler/distributed source follow, opt-in A2UI painting middleware, explicit hardened MCP/MCP Apps/remote A2A adapters, a framework-free reference renderer subpath (`@arnilo/prism-ag-ui/renderer`, 0.0.26), and stable ACP sibling over shared redacted event and durable-approval seams; 0.0.14 adds reconnectable co-work events.
|
|
102
|
-
- [ACP coding-host interop](acp.md): stable ACP v1 `createPrismAcpAgent()`/`createAcpEventMapper()` over `@agentclientprotocol/sdk@1.3.0` — capability advertisement is a pure function of host seams (sessions load/list/delete/resume/dirs, close always, prompt media/embedded, MCP http/sse), client fs/terminal adapters, modes and config options as host overlays, `CodingLifecycleEvent` mapping, four-outcome approvals with elicitation, and frozen caps (0.0.27); 0.1.1 adds ownership-scoped persistence guidance for host-persisted modes/config (plan 013 Task 5 — the agent never persists them); 0.1.6 adds the optional host-owned `AcpSessionStore` durability seam — live registry (modes/config/cwd/ownership) survives agent restart, restore is ownership-scoped and fail-closed (plan 018 Task 2).
|
|
106
|
+
- [ACP coding-host interop](acp.md): stable ACP v1 `createPrismAcpAgent()`/`createAcpEventMapper()` over `@agentclientprotocol/sdk@1.3.0` — capability advertisement is a pure function of host seams (sessions load/list/delete/resume/dirs, close always, prompt media/embedded, MCP http/sse), client fs/terminal adapters, modes and config options as host overlays, `CodingLifecycleEvent` mapping, four-outcome approvals with elicitation, and frozen caps (0.0.27); 0.1.1 adds ownership-scoped persistence guidance for host-persisted modes/config (plan 013 Task 5 — the agent never persists them); 0.1.6 adds the optional host-owned `AcpSessionStore` durability seam — live registry (modes/config/cwd/ownership) survives agent restart, restore is ownership-scoped and fail-closed (plan 018 Task 2). 0.2.6 adds durable run recovery (plan 026 Task 5): bounded `activeRun` refs on persisted sessions, restart re-resolution against `AgentRunLifecycle` (suspended → pending approval ids, terminal → terminal, unprovable in-flight → unknown, never a restarted prompt), and durable ownership/version/fence-checked cancellation that never replays tools; docs/migration.md records the additive-field decision and the 0.2.5 → 0.2.6 downgrade rules.
|
|
103
107
|
- [AG-UI adoption evaluation](ag-ui-adoption.md): official 0.0.57 input/event/capability matrix and shipped hardened MCP/MCP Apps/A2A handshake boundaries.
|
|
104
108
|
|
|
105
109
|
## CLI/RPC
|
|
106
110
|
- [CLI/RPC](cli-rpc.md): Run print/json modes and LF-delimited RPC over the public AgentSession runtime, including mid-run `steer`, branch-handle results, fixed `forkSession`, and `checkout`. `prism init` scaffolds a tiny TypeScript project with one selected provider and an offline mock test; `prism providers add <name>` scaffolds an OpenAI-compatible provider package (manifest, provider, models, cache helpers, conformance test, docs stub).
|
|
107
|
-
- [Workflows](workflows.md): optional `@arnilo/prism-workflows` typed bounded DAG orchestration — explicit recursive definition revisions, exact-owner cancellation/active identity, finite hard limits, durable human suspend/resume, schedules/background execution, revocable proactive schedule capability tokens, nested workflows, replay, coordination, events, and optional RPC/Web bindings. Compose coding plans/checkpoints via workspace Markdown + `state.coding` without a second runtime. Active-run registry is non-durable, in-process only, with bounded sweep/cap cleanup. Interactive TUI (C-012) deferred.
|
|
111
|
+
- [Workflows](workflows.md): optional `@arnilo/prism-workflows` typed bounded DAG orchestration plus linear durable sagas — explicit recursive definition revisions, exact-owner cancellation/active identity, finite hard limits, durable human suspend/resume, schedules/background execution, revocable proactive schedule capability tokens, nested workflows, replay, coordination, events, saga compensation/reconciliation, and optional RPC/Web bindings. Compose coding plans/checkpoints via workspace Markdown + `state.coding` without a second runtime. Active-run registry is non-durable, in-process only, with bounded sweep/cap cleanup. Interactive TUI (C-012) deferred.
|
|
108
112
|
- [Workflow orchestration primitives](workflow-orchestration-primitives.md): architecture inventory — workflow adapters consume core `CheckpointStore`, `LeaseStore`, and bounded `EventMultiplexer`; run control and optional RPC commands stay package-local.
|
|
109
113
|
- [Workflow/TUI scope](workflow-tui-primitives.md): records why 0.0.5 ships workflow APIs/RPC control but no interactive terminal UI.
|
|
110
114
|
|
|
@@ -129,7 +133,8 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
|
|
|
129
133
|
- [Ponytail behavior integration](ponytail.md): optional `@arnilo/prism-ponytail` — upstream Ponytail skills/commands, `ponytail-mode` injector, session `ponytail-mode` persistence; resolves peer `@dietrichgebert/ponytail` or `upstreamPath`; opt-in (not in code/sdk profiles).
|
|
130
134
|
|
|
131
135
|
## Release and install
|
|
132
|
-
- [Release and install](release-and-install.md): current **0.2.5** 50-package graph (root + 49 workspace packages) — plan 025 the maintainability-and-bounded-performance cut: **god-module splits** (the six remaining implementation monoliths — `src/contracts-core.ts` 1,719 L, `src/agent-session.ts` 2,049 L, `workflows/src/run.ts` 1,227 L, `server/src/handler.ts` 1,005 L, `coding-agent/src/repository.ts` 974 L, `ag-ui/src/acp/agent.ts` 836 L — split into cohesive family files behind preserved barrels, compat-preserving with zero breaking deltas, no `exports`-map subpath, `RuntimeAgentSession` kept as one class with a recorded reason), **persistence-mechanics dedup** (21 pure ownership/cursor/checkpoint/lifecycle/search helpers moved into the dependency-free `session-store-codecs`; postgres/sqlite adapters shrank 273 lines; SQL dialect stays per-adapter; no schema/shape change; cross-store conformance green), **bounded accumulation removed** (per-push `Buffer.concat` in language framing + tar parsing → chunk-array readers; framing ~100–200× faster at 4,000 chunks, tar linear at 8 MiB, caps fail-closed byte-identical; CLI `collectOutput` audited already linear), **dead-code cleanup internal-only** (62 candidates triaged: 2 internal removals + 60 allow-listed in `docs/_evidence/phase25-dead-exports-triage.md`), and **coverage close** (76 behavior-backed regressions; core 90.53/84.20/90.54 → 91.43/84.80/91.60); additive-only compat (105 helper exports), no migration; then plan 024 the package-documentation-and-compatibility-truth cut: **umbrella wording matches manifests** (`@arnilo/prism-providers` installs 11 of 14 provider adapters — Azure/Bedrock/Vertex are added separately by `prism-all`; `prism-all` installs 20 direct / 43 transitive packages and omits document-reader, OpenAPI tools, NATS, Caveman, Ponytail; membership unchanged in 0.2.x), **manifest-derived package truth** (`scripts/package-truth.mjs` → `scripts/package-truth.json` is the single source for counts, provider membership, and closures; docs literals regenerate from it and drift fails the gates), **peer-version policy Decision A** (exact `@arnilo/prism: 0.2.4` pins, atomic-upgrade rule, ERESOLVE refusal for partial upgrades, `^1.0.0` widening at 1.x), and **current-line truth** (`docs/0.1.0-readiness.md` at the 0.2.x line with 0.1.7 as the terminal 0.1.x baseline); no runtime contract delta (compat gate at 0.2.4: version literal only), no migration; then plan 023 the build-coverage-and-release-evidence-integrity cut: **build serialization** (dependency-free `scripts/with-build-lock.mjs` — one O_EXCL lockfile at `node_modules/.prism-build.lock` serializing every emit/test leaf so concurrent compilers can never expose a partial live `dist/`, stale-PID reclaim, env-overridable `PRISM_BUILD_LOCK_TIMEOUT_MS`, fail-closed; documented direct-`tsc` caveat), **corrected workspace coverage denominators** (package-local `--test-coverage-include=dist/**` so imported core `dist` no longer pollutes workspace rows — `mcp` 45.47→90.25, `rag` 19.70→94.82; evidence-based per-package thresholds in `scripts/coverage-thresholds.json` with `protectedException` for durable-leg packages shown separately, machine-readable `scripts/coverage-summary.json`), **machine-auditable release skip manifest** (`scripts/release-skip-manifest.mjs` → `scripts/release-evidence.json`: every surface recorded `pass`/`skip`/`blocked`/`protected` with reason and required env; the 33 protected/live skips named; a required surface without evidence records `blocked` and fails the release gate fail-closed — missing credentials/services can never convert into a green release), and **stabilized quality gates** (Biome 2.x `preset` config migration with zero lint diagnostics, the racy 150ms MCP bridge timing assert replaced by a deterministic barrier, load-sensitive guards carry documented `ponytail:` ceilings, machine-readable `lint-report.sarif` + `unused-report.json` retained by CI); no runtime contract delta (compat gate at 0.2.3: version literal only), no migration; then plan 022 the concurrent-state-and-durability-integrity cut: atomic model-budget reservation (`ModelRouterStateStore.reserveBudget`/`commitBudget`/`releaseBudget` with fencing tokens, `reservationTtlMs` expiry and unknown-usage reconciliation, rate/budget key-map caps with LRU eviction that never drops a held reservation), atomic conversation metadata (`SessionRecord.version` + `appendSession` `expectedVersion` CAS across Postgres/SQLite — create-only `0`, exact-version `N>0`, legacy last-write-wins when omitted; `SessionMetadataConflictError` `metadata_conflict` with versions only, HTTP 409; concurrent create/branch/archive single-statement with branch caps inside the CAS, archive wins, deleted rows never resurrect), single-consumer `EventMultiplexer` (`EventMultiplexerError` `ERR_PRISM_EVENT_MULTIPLEXER_SINGLE_CONSUMER` instead of silent queue sharing), restart-stable NATS durable consumer identity (`prism_<hmac16>` with no random suffix — crash-resumed subscribe continues from the last ack, orphaned 0.2.1 consumers reclaimed on clean stop), and bounded non-durable active-run registries (sweep + fail-closed 512 cap `ERR_PRISM_WORKFLOW_RUN_REGISTRY_OVERFLOW`); new regression surface `scripts/phase22-security.test.mjs` (4 blockers + gate accounting over built public entrypoints) + packed plain-JS `security22.mjs` consumer + the `@arnilo/prism/testing/state-concurrency-conformance` harness (7 probes across memory/Postgres/SQLite/NATS legs, no timing-only sleeps) + the `scripts/phase22-conformance.test.mjs` gate; additive-only compat (new exports only, no removals); forward-only migrations 008 (`prism_sessions.version`) and 003 (`prism_model_router_budgets.reservations`); migration `0.2.1 → 0.2.2`; then plan 021 the provider-completion-and-outbound-trust-boundaries cut: strict stream completion is the shared OpenAI-compatible default (truncated streams fail `incomplete_delta`, explicit `strictCompletion: false` opt-out), bounded success bodies via `readBoundedResponseJson` on all discovery/quota/embeddings/upload/OAuth JSON endpoints (65,536-byte ceiling, depth/property/shape caps), DNS-pinned OIDC JWKS/OPA/content fetches through the core `pinnedFetch` primitive with 3xx redirects rejected outright (private/metadata answers fail closed `ssrf_denied`), shared bounded OAuth device/token polling (`pollDeviceCodeToken`) across provider-openai and credentials-node, and the four edge fixes (Azure/Vertex credential-once, Bedrock duplicate-case/repeated-query SigV4 canonicalization, OpenAI upload failed-DELETE retention, cache `__overflow__` tokens-only); public-entrypoint threat-suite `scripts/phase21-security.test.mjs` + packed plain-JS consumer; additive-only compat (MCP transport helpers re-exported from core, no removals); migration `0.2.0 → 0.2.1`; then plan 020 the fail-closed runtime-and-sandbox-security cut on the 0.2.x review-remediation line: durable-resume decision validation in core (`assertValidAgentRunResume` — unknown decisions/malformed batches fail closed with `ERR_PRISM_DECISION_*` before any state claim, checkpoint write, or tool execution; server parser remains defense in depth), isolated work-tool subprocess environments (`@arnilo/prism-work-tools` — fixed base allow-list + explicit env + forced HOME/telemetry + late-bound per-identity tokens, 64-name/64-KiB caps, absolute binary/configDir, linear output capture), and explicit sandbox capabilities (`@arnilo/prism-coding-security` — `SandboxAdapter.capabilities` with omission-is-false fail-closed resolution, `SandboxCodingComposition.capabilities` from verified wiring, `containmentClaim` deprecated as the conservative projection; Docker reports only verified controls, native reports filesystem/process/privilege `false`); public-entrypoint security conformance (`scripts/phase20-security.test.mjs`, wired into `security:threat-suites`), packed plain-JS consumer regressions, and the sandbox-browser workflow's fail-loud Docker/native capability evidence gate — 0.2.0 never ships while a blocker is skipped; migration and rollback notes in `docs/migration.md` `0.1.7 → 0.2.0`, store-compatible with 0.1.7 in both directions; 0.1.7 was the performance-and-DX patch — dependency-free `createCacheTelemetry()` per-provider/model cache hit/miss aggregator (bounded cardinality with `__overflow__`, token counters/rates only, host-activated), host-configurable `ModelRouterSelectionPolicy` on `createModelRouter` with the reference `createCostLatencySelection` (ModelCost rank then in-memory latency EMA, default ordered behavior byte-identical), `prism providers add <name>` OpenAI-compatible provider scaffold (manifest/provider/models/cache/conformance test/docs stub, npm-name + traversal + symlink-escape validation, placeholders only), and the async `AgUiProjection` verification closeout (plan 009 Task 15 evidence recorded, no new code); plan 017 the documented breaking cut — deprecated-option removal with `docs/migration.md` `0.1.4 → 0.1.5` section and reviewed compat-baseline regeneration via `--allow-break` then `--update-baseline`: the inert provider request knobs, `RunOptions.maxToolRounds`, observational-memory flat settings keys + top-level worker aliases, `ReadToolOptions.autoResizeImages`, `INIT_PROVIDERS`; all removals fail closed naming their replacement; plan 016 internal god-module split — `agents.ts`/`contracts.ts` reorganized behind barrel re-exports with a byte-identical public entry surface, measured tree-shaking improvement in `scripts/phase16-baseline.json`, and additive `@arnilo/prism-browser` Chrome DevTools Protocol capabilities — `browser_evaluate`/`browser_observe` and `block_urls`/`unblock_urls`/`throttle`/`emulate` act actions; plan 015 dead-code and deprecation hygiene on the frozen 0.1.x line — parameterized benchmark runner `scripts/benchmark.mjs` absorbing the per-version runners, archived review-coverage evidence in `docs/_evidence/`, non-blocking unused-code sweep `npm run sweep:unused`, opt-in checkpoint persistence for loaded-skill names and read-path sets; plan 014 Alibaba provider enrichment — embeddings, video input, verified compatible-mode surface decision table; plan 013 post-release hardening — build single-flight, MCP SSE relay test, combined coverage summary, canonical manifest-count narrative, ACP modes/config persistence guidance; Phase 12 release-candidate hardening; plan 012 — freeze manifest, compatibility matrix, upgrade matrix, packed-install e2e journeys, restart-recovery evidence, capacity envelopes, security policy), exact-peer/install/tarball rules, deterministic resumable publication and publish dry-run, frozen 0.1.x compatibility and support matrix (Node/PostgreSQL/platform/provider/protocol pins and unsupported combinations, machine-checked against `scripts/phase12-freeze-manifest.json`), protected PostgreSQL gate, pinned supply-chain gates, offline tests, the 0.0.15 provider/AI-SDK/RAG/memory protected live-canary matrix, and sandbox-browser Docker/Playwright gates.
|
|
136
|
+
- [0.2.7 Task 0 scope evidence](release-0.2.7-evidence.md): frozen ERP primitives, demand decisions, threat mappings, budgets, protected-gate policy, and API ownership; not a production-readiness claim.
|
|
137
|
+
- [Release and install](release-and-install.md): current **0.2.7** 50-package graph (root + 49 workspace packages) — plan 026 the fully-featured coding-agent-readiness cut: **host-selected PTY** (`pty: true` delegates only to the host `ptyBackend`, fails closed as unsupported when absent, bounded resize/TERM/attach caps), **indexed code search** (host-owned incremental index seam with explicit `indexed_literal`/`semantic` modes, literal remains the default, stale/failed/untrusted indexes fail closed `ERR_PRISM_INDEX_*`, results labeled `untrusted_index`), **coding workspaces** (`createCodingWorkspaceLifecycle`: durable CheckpointStore CAS records + LeaseStore fencing, locked worktrees, credential-free fingerprints, cleanup refusal matrix), **durable recovery** (process intent/ACP `activeRun` refs over Postgres/SQLite stores with attach-if-attested `recover()` and durable fence-checked cancellation, never fabricated exits), **patch review and diagnostics** (`createCodingPatchReviewManifest` + `assertCodingPatchAccepted` with pending/accepted/rejected/superseded bound to digest + revision + identity, opt-in LSP `syncDocument`/`diagnosticDelta`), and the **protected real coding journey** (packed consumer through real provider/Docker/Postgres/GitHub/Playwright/PTY services with retained evidence report; forge breadth GitLab/Bitbucket stays demand-gated); then plan 025 the maintainability-and-bounded-performance cut: **god-module splits** (the six remaining implementation monoliths — `src/contracts-core.ts` 1,719 L, `src/agent-session.ts` 2,049 L, `workflows/src/run.ts` 1,227 L, `server/src/handler.ts` 1,005 L, `coding-agent/src/repository.ts` 974 L, `ag-ui/src/acp/agent.ts` 836 L — split into cohesive family files behind preserved barrels, compat-preserving with zero breaking deltas, no `exports`-map subpath, `RuntimeAgentSession` kept as one class with a recorded reason), **persistence-mechanics dedup** (21 pure ownership/cursor/checkpoint/lifecycle/search helpers moved into the dependency-free `session-store-codecs`; postgres/sqlite adapters shrank 273 lines; SQL dialect stays per-adapter; no schema/shape change; cross-store conformance green), **bounded accumulation removed** (per-push `Buffer.concat` in language framing + tar parsing → chunk-array readers; framing ~100–200× faster at 4,000 chunks, tar linear at 8 MiB, caps fail-closed byte-identical; CLI `collectOutput` audited already linear), **dead-code cleanup internal-only** (62 candidates triaged: 2 internal removals + 60 allow-listed in `docs/_evidence/phase25-dead-exports-triage.md`), and **coverage close** (76 behavior-backed regressions; core 90.53/84.20/90.54 → 91.43/84.80/91.60); additive-only compat (105 helper exports), no migration; then plan 024 the package-documentation-and-compatibility-truth cut: **umbrella wording matches manifests** (`@arnilo/prism-providers` installs 11 of 14 provider adapters — Azure/Bedrock/Vertex are added separately by `prism-all`; `prism-all` installs 20 direct / 43 transitive packages and omits document-reader, OpenAPI tools, NATS, Caveman, Ponytail; membership unchanged in 0.2.x), **manifest-derived package truth** (`scripts/package-truth.mjs` → `scripts/package-truth.json` is the single source for counts, provider membership, and closures; docs literals regenerate from it and drift fails the gates), **peer-version policy Decision A** (exact `@arnilo/prism: 0.2.4` pins, atomic-upgrade rule, ERESOLVE refusal for partial upgrades, `^1.0.0` widening at 1.x), and **current-line truth** (`docs/0.1.0-readiness.md` at the 0.2.x line with 0.1.7 as the terminal 0.1.x baseline); no runtime contract delta (compat gate at 0.2.4: version literal only), no migration; then plan 023 the build-coverage-and-release-evidence-integrity cut: **build serialization** (dependency-free `scripts/with-build-lock.mjs` — one O_EXCL lockfile at `node_modules/.prism-build.lock` serializing every emit/test leaf so concurrent compilers can never expose a partial live `dist/`, stale-PID reclaim, env-overridable `PRISM_BUILD_LOCK_TIMEOUT_MS`, fail-closed; documented direct-`tsc` caveat), **corrected workspace coverage denominators** (package-local `--test-coverage-include=dist/**` so imported core `dist` no longer pollutes workspace rows — `mcp` 45.47→90.25, `rag` 19.70→94.82; evidence-based per-package thresholds in `scripts/coverage-thresholds.json` with `protectedException` for durable-leg packages shown separately, machine-readable `scripts/coverage-summary.json`), **machine-auditable release skip manifest** (`scripts/release-skip-manifest.mjs` → `scripts/release-evidence.json`: every surface recorded `pass`/`skip`/`blocked`/`protected` with reason and required env; the 33 protected/live skips named; a required surface without evidence records `blocked` and fails the release gate fail-closed — missing credentials/services can never convert into a green release), and **stabilized quality gates** (Biome 2.x `preset` config migration with zero lint diagnostics, the racy 150ms MCP bridge timing assert replaced by a deterministic barrier, load-sensitive guards carry documented `ponytail:` ceilings, machine-readable `lint-report.sarif` + `unused-report.json` retained by CI); no runtime contract delta (compat gate at 0.2.3: version literal only), no migration; then plan 022 the concurrent-state-and-durability-integrity cut: atomic model-budget reservation (`ModelRouterStateStore.reserveBudget`/`commitBudget`/`releaseBudget` with fencing tokens, `reservationTtlMs` expiry and unknown-usage reconciliation, rate/budget key-map caps with LRU eviction that never drops a held reservation), atomic conversation metadata (`SessionRecord.version` + `appendSession` `expectedVersion` CAS across Postgres/SQLite — create-only `0`, exact-version `N>0`, legacy last-write-wins when omitted; `SessionMetadataConflictError` `metadata_conflict` with versions only, HTTP 409; concurrent create/branch/archive single-statement with branch caps inside the CAS, archive wins, deleted rows never resurrect), single-consumer `EventMultiplexer` (`EventMultiplexerError` `ERR_PRISM_EVENT_MULTIPLEXER_SINGLE_CONSUMER` instead of silent queue sharing), restart-stable NATS durable consumer identity (`prism_<hmac16>` with no random suffix — crash-resumed subscribe continues from the last ack, orphaned 0.2.1 consumers reclaimed on clean stop), and bounded non-durable active-run registries (sweep + fail-closed 512 cap `ERR_PRISM_WORKFLOW_RUN_REGISTRY_OVERFLOW`); new regression surface `scripts/phase22-security.test.mjs` (4 blockers + gate accounting over built public entrypoints) + packed plain-JS `security22.mjs` consumer + the `@arnilo/prism/testing/state-concurrency-conformance` harness (7 probes across memory/Postgres/SQLite/NATS legs, no timing-only sleeps) + the `scripts/phase22-conformance.test.mjs` gate; additive-only compat (new exports only, no removals); forward-only migrations 008 (`prism_sessions.version`) and 003 (`prism_model_router_budgets.reservations`); migration `0.2.1 → 0.2.2`; then plan 021 the provider-completion-and-outbound-trust-boundaries cut: strict stream completion is the shared OpenAI-compatible default (truncated streams fail `incomplete_delta`, explicit `strictCompletion: false` opt-out), bounded success bodies via `readBoundedResponseJson` on all discovery/quota/embeddings/upload/OAuth JSON endpoints (65,536-byte ceiling, depth/property/shape caps), DNS-pinned OIDC JWKS/OPA/content fetches through the core `pinnedFetch` primitive with 3xx redirects rejected outright (private/metadata answers fail closed `ssrf_denied`), shared bounded OAuth device/token polling (`pollDeviceCodeToken`) across provider-openai and credentials-node, and the four edge fixes (Azure/Vertex credential-once, Bedrock duplicate-case/repeated-query SigV4 canonicalization, OpenAI upload failed-DELETE retention, cache `__overflow__` tokens-only); public-entrypoint threat-suite `scripts/phase21-security.test.mjs` + packed plain-JS consumer; additive-only compat (MCP transport helpers re-exported from core, no removals); migration `0.2.0 → 0.2.1`; then plan 020 the fail-closed runtime-and-sandbox-security cut on the 0.2.x review-remediation line: durable-resume decision validation in core (`assertValidAgentRunResume` — unknown decisions/malformed batches fail closed with `ERR_PRISM_DECISION_*` before any state claim, checkpoint write, or tool execution; server parser remains defense in depth), isolated work-tool subprocess environments (`@arnilo/prism-work-tools` — fixed base allow-list + explicit env + forced HOME/telemetry + late-bound per-identity tokens, 64-name/64-KiB caps, absolute binary/configDir, linear output capture), and explicit sandbox capabilities (`@arnilo/prism-coding-security` — `SandboxAdapter.capabilities` with omission-is-false fail-closed resolution, `SandboxCodingComposition.capabilities` from verified wiring, `containmentClaim` deprecated as the conservative projection; Docker reports only verified controls, native reports filesystem/process/privilege `false`); public-entrypoint security conformance (`scripts/phase20-security.test.mjs`, wired into `security:threat-suites`), packed plain-JS consumer regressions, and the sandbox-browser workflow's fail-loud Docker/native capability evidence gate — 0.2.0 never ships while a blocker is skipped; migration and rollback notes in `docs/migration.md` `0.1.7 → 0.2.0`, store-compatible with 0.1.7 in both directions; 0.1.7 was the performance-and-DX patch — dependency-free `createCacheTelemetry()` per-provider/model cache hit/miss aggregator (bounded cardinality with `__overflow__`, token counters/rates only, host-activated), host-configurable `ModelRouterSelectionPolicy` on `createModelRouter` with the reference `createCostLatencySelection` (ModelCost rank then in-memory latency EMA, default ordered behavior byte-identical), `prism providers add <name>` OpenAI-compatible provider scaffold (manifest/provider/models/cache/conformance test/docs stub, npm-name + traversal + symlink-escape validation, placeholders only), and the async `AgUiProjection` verification closeout (plan 009 Task 15 evidence recorded, no new code); plan 017 the documented breaking cut — deprecated-option removal with `docs/migration.md` `0.1.4 → 0.1.5` section and reviewed compat-baseline regeneration via `--allow-break` then `--update-baseline`: the inert provider request knobs, `RunOptions.maxToolRounds`, observational-memory flat settings keys + top-level worker aliases, `ReadToolOptions.autoResizeImages`, `INIT_PROVIDERS`; all removals fail closed naming their replacement; plan 016 internal god-module split — `agents.ts`/`contracts.ts` reorganized behind barrel re-exports with a byte-identical public entry surface, measured tree-shaking improvement in `scripts/phase16-baseline.json`, and additive `@arnilo/prism-browser` Chrome DevTools Protocol capabilities — `browser_evaluate`/`browser_observe` and `block_urls`/`unblock_urls`/`throttle`/`emulate` act actions; plan 015 dead-code and deprecation hygiene on the frozen 0.1.x line — parameterized benchmark runner `scripts/benchmark.mjs` absorbing the per-version runners, archived review-coverage evidence in `docs/_evidence/`, non-blocking unused-code sweep `npm run sweep:unused`, opt-in checkpoint persistence for loaded-skill names and read-path sets; plan 014 Alibaba provider enrichment — embeddings, video input, verified compatible-mode surface decision table; plan 013 post-release hardening — build single-flight, MCP SSE relay test, combined coverage summary, canonical manifest-count narrative, ACP modes/config persistence guidance; Phase 12 release-candidate hardening; plan 012 — freeze manifest, compatibility matrix, upgrade matrix, packed-install e2e journeys, restart-recovery evidence, capacity envelopes, security policy), exact-peer/install/tarball rules, deterministic resumable publication and publish dry-run, frozen 0.1.x compatibility and support matrix (Node/PostgreSQL/platform/provider/protocol pins and unsupported combinations, machine-checked against `scripts/phase12-freeze-manifest.json`), protected PostgreSQL gate, pinned supply-chain gates, offline tests, the 0.0.15 provider/AI-SDK/RAG/memory protected live-canary matrix, and sandbox-browser Docker/Playwright gates. 0.2.6 (plan 026 Task 7) adds the protected coding journey: `scripts/phase26-coding-journey.test.mjs` runs a packed consumer through real provider calls, a digest-pinned Docker sandbox, the durable Postgres worktree lifecycle, provider-driven ACP edits with policy approval, named checks with `diagnosticDelta`, patch review over the server ArtifactService, cross-replica process recovery, durable cancellation, real GitHub PR push/reconcile/cleanup, host Playwright inspection, and the host PTY adapter (frozen profile) — the retained `scripts/phase26-coding-journey-report.json` gates release evidence (pass/blocked/protected, never a passing skip).
|
|
133
138
|
- [0.1.0 / 1.0 readiness gates](0.1.0-readiness.md): command-per-gate 1.0 readiness table — frozen API surface + compat gate, migration/docs tripwires, budget table, live-suite matrix, security matrix, current-line status (**0.2.5** current line; 0.1.7 terminal 0.1.x baseline), signed-publication/live-canary prerequisites for 1.0, and Phase 12 demand-evidence entry criteria.
|
|
134
139
|
- [Review coverage archive](_evidence/): per-phase evidence freezes (plans 067–079, releases 0.0.4–0.0.16) — traceability matrices, provider validation, capability/primitive/limit matrices, benchmark budgets, and artifact-diet findings; tarball-excluded, kept in-repo for audit.
|
|
135
140
|
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Indexed code search
|
|
2
|
+
|
|
3
|
+
Optional host-owned incremental search index for `repo_search` (plan 026 Task 2, `@arnilo/prism-coding-agent`). The index is a seam: Prism defines the contract, validates requests and results, and scopes identity and freshness — the host owns persistence, build, watch, and any embedding/ranking engine. There is no bundled index engine, vector store, watcher daemon, or embedding SDK.
|
|
4
|
+
|
|
5
|
+
## Activation
|
|
6
|
+
|
|
7
|
+
Nothing starts on import or construction. A host builds and updates the index explicitly through the facade:
|
|
8
|
+
|
|
9
|
+
```ts
|
|
10
|
+
import { createIndexedRepositoryOperations, createGitAwareRepositoryOperations } from "@arnilo/prism-coding-agent";
|
|
11
|
+
|
|
12
|
+
const operations = createIndexedRepositoryOperations(cwd, {
|
|
13
|
+
index: hostIndex, // RepositoryIndexBackend
|
|
14
|
+
fallback: createGitAwareRepositoryOperations(cwd),
|
|
15
|
+
allowedModes: ["literal", "indexed_literal", "semantic"],
|
|
16
|
+
stale: { maxAgeMs: 60_000, requireSourceRevision: true },
|
|
17
|
+
});
|
|
18
|
+
|
|
19
|
+
await operations.index.update({ repositoryId, worktreeId, sourceRevision, changes });
|
|
20
|
+
await operations.index.remove({ paths });
|
|
21
|
+
await operations.index.status();
|
|
22
|
+
await operations.index.dispose();
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
The returned object is a `RepositoryOperations` with an added `index` facade. `mode: "literal"` routes to the fallback unchanged; `indexed_literal` and `semantic` are served only by the host backend.
|
|
26
|
+
|
|
27
|
+
## Backend contract
|
|
28
|
+
|
|
29
|
+
`RepositoryIndexBackend`:
|
|
30
|
+
|
|
31
|
+
- `capabilities.semantic` — explicit declaration; semantic mode is never duck-typed.
|
|
32
|
+
- `update(request)` — add/edit changes with `repositoryId`/`worktreeId`/`sourceRevision` (bounded, never credential-bearing).
|
|
33
|
+
- `remove(request)` — drop paths.
|
|
34
|
+
- `search(request)` — bounded query with `mode`, optional repo-relative `path` scope, `maxResults`, `signal`, `deadlineMs`.
|
|
35
|
+
- `status()` — state `empty | building | ready | stale | failed`, `sourceRevision`, `updatedAt`.
|
|
36
|
+
- `dispose()`.
|
|
37
|
+
|
|
38
|
+
Index state machine is frozen: `empty | building | ready | stale | failed`.
|
|
39
|
+
|
|
40
|
+
## Mode semantics
|
|
41
|
+
|
|
42
|
+
- `literal` (default): native bounded substring search; byte-identical behavior to a host without an index.
|
|
43
|
+
- `indexed_literal`: host index result, relevance scores.
|
|
44
|
+
- `semantic`: host semantic search; requires `capabilities.semantic === true`.
|
|
45
|
+
|
|
46
|
+
Missing capability, mode not in `allowedModes`, stale revision, or failed index returns a stable `ERR_PRISM_INDEX_*` error. There is **no silent fallback** that changes query meaning — an index result is never replaced by a literal scan behind the caller's back.
|
|
47
|
+
|
|
48
|
+
`createRepoSearchTool(cwd, { operations, modes })` exposes exactly the modes listed in `modes` (default `["literal"]`); the JSON schema `enum` matches.
|
|
49
|
+
|
|
50
|
+
## Stale-index and freshness
|
|
51
|
+
|
|
52
|
+
`stale: { maxAgeMs, requireSourceRevision }`:
|
|
53
|
+
|
|
54
|
+
- `maxAgeMs` (default 60 000, hard 300 000): queries fail with `ERR_PRISM_INDEX_STALE` when `updatedAt` is absent or older than the window.
|
|
55
|
+
- `requireSourceRevision: true`: queries fail when the index does not attest a `sourceRevision`.
|
|
56
|
+
|
|
57
|
+
State `empty`/`building` also fail stale; state `failed` fails with `ERR_PRISM_INDEX_FAILED`; unknown states fail closed. The stale-index contract refuses to serve rather than silently degrade. Hosts choose the window; the default is deliberately conservative.
|
|
58
|
+
|
|
59
|
+
## Trust
|
|
60
|
+
|
|
61
|
+
Index output is untrusted:
|
|
62
|
+
|
|
63
|
+
- every hit path is containment-checked under the repository root and the requested path scope (absolute paths, `..` escapes, backslashes, and scope escapes fail with `ERR_PRISM_INDEX_UNTRUSTED`);
|
|
64
|
+
- scores must be finite and in `[0, 1]`, otherwise fail closed;
|
|
65
|
+
- duplicate paths are deduped (first wins);
|
|
66
|
+
- snippets are truncated to the byte cap (default 4096, hard 16384);
|
|
67
|
+
- result count is capped (default 1000, hard 10000);
|
|
68
|
+
- results carry `indexed` provenance (mode/state/sourceRevision/updatedAt) and `untrusted_index: true` — consumers must not treat index text as fact; mutations still require a fresh read/policy.
|
|
69
|
+
|
|
70
|
+
Backend throws are mapped to generic `ERR_PRISM_INDEX_FAILED` without embedded backend error text; query deadline (default 30 s, hard 120 s) and abort map to `ERR_PRISM_INDEX_TIMEOUT`.
|
|
71
|
+
|
|
72
|
+
## Update caps
|
|
73
|
+
|
|
74
|
+
Per update: 1000 changes default / 10000 hard, 16 MiB / 64 MiB total request bytes (paths, old paths, revision, ids). `rename` is routed as remove of the old path plus add of the new; `delete` routes to `remove`. Over-limit updates fail with `ERR_PRISM_INDEX_LIMIT` before any backend call.
|
|
75
|
+
|
|
76
|
+
## Errors
|
|
77
|
+
|
|
78
|
+
`ERR_PRISM_INDEX_UNSUPPORTED`, `ERR_PRISM_INDEX_STALE`, `ERR_PRISM_INDEX_FAILED`, `ERR_PRISM_INDEX_LIMIT`, `ERR_PRISM_INDEX_TIMEOUT`, `ERR_PRISM_INDEX_UNTRUSTED` (`IndexError`).
|
|
79
|
+
|
|
80
|
+
## Scale evidence
|
|
81
|
+
|
|
82
|
+
`scripts/phase26-index-benchmark.test.mjs` runs in the root test chain: a 100000-file metadata fixture, indexed query p95 ≤ 250 ms, 1000-file batch update ≤ 1 s, peak heap ≤ +64 MiB, semantic query bounded, and the literal baseline unchanged.
|
|
@@ -40,6 +40,21 @@ await lang.rename({ file: "src/a.ts", line: 10, character: 4, newName: "renamed"
|
|
|
40
40
|
await lang.dispose();
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
+
## Document synchronization and diagnostics (0.2.6, plan 026)
|
|
44
|
+
|
|
45
|
+
LSP stays opt-in: `createLanguageIntelligence` is a standalone host-activated factory — nothing spawns on construction and no agent/tool assembly instantiates it.
|
|
46
|
+
|
|
47
|
+
- `syncDocument(file)` — re-syncs a file after an external edit via full-content `textDocument/didChange` (protocol-valid LSP 3.17; no diff engine). Versions are monotonic per document: didOpen stamps 1, each didChange increments.
|
|
48
|
+
- `diagnosticDelta({ files, previous })` — bounded diagnostic refresh for changed files only (never a whole-workspace pull). When the server advertises `diagnosticProvider`, the client uses pull diagnostics (`textDocument/diagnostic` with `previousResultId` reuse — `kind: full|unchanged`; the cached set is reused on `unchanged`); otherwise it reads the push cache (`textDocument/publishDiagnostics` always replaces the full set, `[]` clears). Results are normalized (`NormalizedDiagnostic`), generation-stamped with the document version, and diffed with `diagnosticDelta` into deterministic `added` / `removed` / `unchanged`. Stale views (a previous generation newer than the refresh) are dropped per file — a stale-version response never overwrites newer results. Refresh honors the standard LSP caps (message bytes, diagnostics/file, results/query, timeout, files per request).
|
|
49
|
+
|
|
50
|
+
```ts
|
|
51
|
+
await lang.syncDocument("src/app.ts");
|
|
52
|
+
const delta = await lang.diagnosticDelta({
|
|
53
|
+
files: ["src/app.ts"],
|
|
54
|
+
previous: { "src/app.ts": { generation: 3, diagnostics: priorDiags } },
|
|
55
|
+
});
|
|
56
|
+
```
|
|
57
|
+
|
|
43
58
|
## Inputs / request
|
|
44
59
|
|
|
45
60
|
`createLanguageIntelligence` options:
|