@arnilo/prism 0.2.6 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +4 -1
- package/README.md +1 -0
- package/dist/field-policy.d.ts +119 -0
- package/dist/field-policy.js +418 -0
- package/dist/index.d.ts +3 -1
- package/dist/index.js +2 -1
- package/dist/redaction.d.ts +6 -5
- package/dist/redaction.js +18 -10
- package/docs/0.1.0-readiness.md +6 -6
- package/docs/audit-export.md +151 -0
- package/docs/data-classification.md +82 -0
- package/docs/disaster-recovery.md +71 -0
- package/docs/enterprise-postgres-state.md +60 -7
- package/docs/evaluations.md +45 -0
- package/docs/host-security.md +4 -0
- package/docs/index.md +9 -4
- package/docs/migration.md +13 -0
- package/docs/operations.md +104 -0
- package/docs/policy-and-audit.md +34 -0
- package/docs/release-0.2.7-evidence.md +514 -0
- package/docs/release-and-install.md +29 -3
- package/docs/workflows.md +48 -3
- package/package.json +2 -2
package/docs/0.1.0-readiness.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# 0.1.0 / 1.0 Readiness Gates
|
|
2
2
|
|
|
3
|
-
Status: **0.2.
|
|
3
|
+
Status: **0.2.7** is the current release line (the 0.2.x review-remediation line: fail-closed runtime/sandbox security, provider completion and outbound trust boundaries, concurrent-state/durability integrity, build/coverage/release-evidence integrity, package/documentation/compatibility truth, maintainability and bounded performance, fully featured coding-agent readiness); **0.1.7** was the terminal 0.1.x baseline; **1.0** readiness remains operator-gated, not automatic.
|
|
4
4
|
|
|
5
5
|
This page distills runnable readiness gates into one command-per-gate table.
|
|
6
6
|
The **Last evidence** column records the 0.1.0-tree snapshot (plan 012 Tasks
|
|
@@ -20,15 +20,15 @@ Historical release lines (0.0.16 floor → 0.0.27 Phase 10 ACP interop → 0.1.0
|
|
|
20
20
|
keep their per-phase evidence in the pages above; this page records the 0.2.6
|
|
21
21
|
snapshot (plan 026) with the 0.1.x tables below as the historical record.
|
|
22
22
|
|
|
23
|
-
## Current line (0.2.
|
|
23
|
+
## Current line (0.2.7)
|
|
24
24
|
|
|
25
25
|
| Item | Status |
|
|
26
26
|
|---|---|
|
|
27
|
-
| Published graph | **50** publishable manifests at exact **0.2.
|
|
27
|
+
| Published graph | **50** publishable manifests at exact **0.2.7** (root + 49 workspace packages: 14 provider adapters + 9 `prism-*` family/profile + 26 capability; generated by `node scripts/package-truth.mjs` → `scripts/package-truth.json`) |
|
|
28
28
|
| Current-line cut | The 0.2.x review-remediation line, additive-only vs the frozen 0.1.x contract: 0.2.0 fail-closed runtime/sandbox security (durable-resume decision validation, work-tool env isolation, explicit sandbox capabilities), 0.2.1 provider completion + outbound trust boundaries (strict stream completion, bounded success bodies, DNS-pinned OIDC/OPA fetches), 0.2.2 concurrent-state/durability integrity (model-budget reservation, conversation-metadata CAS, single-consumer EventMultiplexer, NATS durable identity), 0.2.3 build/coverage/release-evidence integrity (build single-flight, corrected coverage denominators, release skip manifest, stabilized quality gates), 0.2.4 package/documentation/compatibility truth (umbrella wording matches manifests, manifest-derived package truth, peer-version policy Decision A, current-line truth), 0.2.5 maintainability and bounded performance (god-module splits into cohesive family files behind preserved barrels, persistence-mechanics dedup into `session-store-codecs`, quadratic `Buffer.concat` removed from framing/tar, dead-code cleanup internal-only, 76 behavior-backed coverage regressions), 0.2.6 fully featured coding-agent readiness (host-selected PTY backend, scalable indexed code-search seam, multi-worktree/repository lifecycle, forge breadth demand-gated, durable ACP/process recovery, patch review + incremental diagnostics, protected real coding journey) |
|
|
29
|
-
| Upgrade path | `docs/migration.md` `0.2.
|
|
30
|
-
| Compat promise | Additive-only vs the frozen 0.1.x contract; `scripts/compat-baseline` regenerated at 0.2.
|
|
31
|
-
| Security policy | `npm audit --audit-level=moderate` 0 at 0.2.
|
|
29
|
+
| Upgrade path | `docs/migration.md` `0.2.6 → 0.2.7` (additive; two new forward-only ERP migrations 004/005 — outbox/inbox + approvals tables; rollback = stop 0.2.7 workers, drop the ERP tables, restore 0.2.6 manifests/tag); store-compatible throughout 0.2.x |
|
|
30
|
+
| Compat promise | Additive-only vs the frozen 0.1.x contract; `scripts/compat-baseline` regenerated at 0.2.7 (version literal + additive PTY/index/workspace/recovery/review exports), zero breaking deltas |
|
|
31
|
+
| Security policy | `npm audit --audit-level=moderate` 0 at 0.2.7; threat-suites legs (phase8–11 + phase20–26) green; protected Postgres recovery/workspace conformance, PTY, real coding journey, NATS/live-canary legs operator-gated |
|
|
32
32
|
| Docs freeze | tripwires green including the canonical manifest-count tripwire (50/49/14/9/26), the plan 024 package-truth tests (generator reproducibility + artifact equality + closure asserts + derived docs truth), the plan 025 bounded-accumulation near-limit probe, and the plan 026 freeze tripwires (per-task markers, threat T1–T8 test mapping, exit gate green) |
|
|
33
33
|
| 0.1.x line | **0.1.7** (plan 019) is the terminal 0.1.x baseline; the 0.1.1 table below keeps the plan 013 snapshot; the 0.1.0 table keeps the plan 012 snapshot; the **0.0.16** values remain the historical network-free floor |
|
|
34
34
|
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
# Signed, hash-chained audit export
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
`@arnilo/prism-policy` exports tenant-scoped audit records as signed, hash-chained
|
|
6
|
+
batches: each record envelope is canonicalized (RFC 8785 semantics), hashed with
|
|
7
|
+
SHA-256 including the prior digest, so records form a tamper-evident chain. A
|
|
8
|
+
batch of chained records is wrapped in a manifest that a host-provided
|
|
9
|
+
`AuditSigner` signs; the signed artifact is written to an immutable WORM sink
|
|
10
|
+
which must acknowledge the exact artifact digest before the export cursor
|
|
11
|
+
advances, and is mirrored to an optional SIEM sink with replayable status. An
|
|
12
|
+
independent verifier (`verifyAuditBatch` or `scripts/verify-audit-export.mjs`)
|
|
13
|
+
re-derives everything from the artifact bytes and a public key — no ledger,
|
|
14
|
+
sink, or private key needed.
|
|
15
|
+
|
|
16
|
+
## When to use it
|
|
17
|
+
|
|
18
|
+
- You keep an append-only policy/audit ledger and must prove to auditors that
|
|
19
|
+
exported records were not reordered, edited, inserted, or truncated after the
|
|
20
|
+
fact.
|
|
21
|
+
- You need a durable WORM copy with an integrity receipt and a replayable SIEM
|
|
22
|
+
mirror, without embedding cloud SDKs or key storage in Prism.
|
|
23
|
+
- You must export only the redacted bytes an external verifier will actually
|
|
24
|
+
see, with redaction provenance and legal-hold provenance preserved.
|
|
25
|
+
|
|
26
|
+
Do not use it for exactly-once delivery claims: like the messaging outbox,
|
|
27
|
+
exports are at-least-once, and the exporter never proves that a record exists —
|
|
28
|
+
it proves that what was exported is exactly what the verifier can reproduce.
|
|
29
|
+
|
|
30
|
+
## Inputs / request
|
|
31
|
+
|
|
32
|
+
- `createAuditExporter({ source, cursorStore, signer, wormSink, siemSink?, redact? })`
|
|
33
|
+
- `source` — tenant-scoped, stable-order `AuditPageSource`; page cursors are
|
|
34
|
+
one-shot tokens (re-reading a cursor that already served its final page
|
|
35
|
+
yields an empty page).
|
|
36
|
+
- `cursorStore` — CAS-versioned `AuditCursorStore`; the exporter advances the
|
|
37
|
+
cursor only on a matched version after the WORM acknowledgement.
|
|
38
|
+
- `signer` — host `AuditSigner` (`sign(bytes) -> Uint8Array`, optional
|
|
39
|
+
`keyId`/`algorithm`); Prism never accepts raw private keys.
|
|
40
|
+
- `wormSink` — required immutable sink returning `{ batchId, digest }`; the
|
|
41
|
+
exporter refuses to advance unless both match.
|
|
42
|
+
- `siemSink` — optional replayable mirror.
|
|
43
|
+
- `redact` — optional `AuditRedactionPolicy` applied before hashing.
|
|
44
|
+
- `exportNext({ tenantId, maxRecords?, maxBytes?, signal? })` — processes one
|
|
45
|
+
page; returns `{ batchId, firstSequence, lastSequence, recordCount,
|
|
46
|
+
wormAcked, siemStatus: "disabled" | "sent" | "pending", nextDigest,
|
|
47
|
+
artifactBytes }`.
|
|
48
|
+
- `verifyAuditBatch({ artifactBytes, publicKey, expectedTenantId,
|
|
49
|
+
previousDigest?, expectedFirstSequence?, expectedLastSequence? })`.
|
|
50
|
+
|
|
51
|
+
## Outputs / response / events
|
|
52
|
+
|
|
53
|
+
A batch artifact written to the WORM sink contains:
|
|
54
|
+
|
|
55
|
+
```json
|
|
56
|
+
{
|
|
57
|
+
"schemaVersion": 1,
|
|
58
|
+
"document": "{ ...canonical signed manifest as a single JSON string... }",
|
|
59
|
+
"signature": { "algorithm": "sha256", "keyId": "k1", "value": "<base64>" }
|
|
60
|
+
}
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
The embedded document is the manifest: `tenantId`, `batchId`, `algorithm`,
|
|
64
|
+
`firstSequence`, `lastSequence`, `previousDigest`, `nextDigest`, and
|
|
65
|
+
`records` — each record carrying `sequence`, `priorDigest`, `digest`,
|
|
66
|
+
optional `legalHold`, optional `redactions` (`{ path, reason }[]`), and the
|
|
67
|
+
canonical `record` payload. `verifyAuditBatch` returns `{ ok, errors, batch }`.
|
|
68
|
+
|
|
69
|
+
## Request/response example
|
|
70
|
+
|
|
71
|
+
```ts
|
|
72
|
+
import { createAuditExporter, createMemoryAuditCursorStore } from "@arnilo/prism-policy";
|
|
73
|
+
|
|
74
|
+
const exporter = createAuditExporter({
|
|
75
|
+
source, // host: tenant-scoped record pages
|
|
76
|
+
cursorStore: createMemoryAuditCursorStore(), // host durable store in production
|
|
77
|
+
signer, // host: HSM/KMS-backed signer
|
|
78
|
+
wormSink, // host: S3 object-lock / WORM bucket
|
|
79
|
+
siemSink, // host: SIEM or event stream
|
|
80
|
+
});
|
|
81
|
+
const result = await exporter.exportNext({ tenantId: "acme", maxRecords: 1000 });
|
|
82
|
+
// result.artifactBytes -> store on WORM; result.nextDigest -> pass as
|
|
83
|
+
// previousDigest when verifying the next batch.
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
## Implementation example
|
|
87
|
+
|
|
88
|
+
```ts
|
|
89
|
+
import { readFileSync } from "node:fs";
|
|
90
|
+
import { verifyAuditBatch } from "@arnilo/prism-policy";
|
|
91
|
+
|
|
92
|
+
const artifactBytes = new Uint8Array(readFileSync("./acme-000001.json"));
|
|
93
|
+
const verified = verifyAuditBatch({
|
|
94
|
+
artifactBytes,
|
|
95
|
+
publicKey: readFileSync("./audit-verification.pem", "utf8"),
|
|
96
|
+
expectedTenantId: "acme",
|
|
97
|
+
previousDigest: "0000000000000000000000000000000000000000000000000000000000000000",
|
|
98
|
+
});
|
|
99
|
+
if (!verified.ok) throw new Error(verified.errors.join("; "));
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
## Extension and configuration notes
|
|
103
|
+
|
|
104
|
+
- Key rotation: the artifact records the signer's `keyId`; verification fails
|
|
105
|
+
explicitly under a rotated key, so consumers select the key named by the
|
|
106
|
+
artifact's `signature.keyId`. Hosts own key lifecycle and the public-key
|
|
107
|
+
distribution path.
|
|
108
|
+
- Failed batches retain their one-page payload in exporter memory and replay
|
|
109
|
+
the same batch id on the next `exportNext`; WORM-failed or signer-failed
|
|
110
|
+
batches are never re-read from the source. A cursor CAS race after WORM
|
|
111
|
+
acknowledgment is surfaced as an explicit error and is not retried by this
|
|
112
|
+
exporter (the batch is already durable).
|
|
113
|
+
- SIEM failures do not fail the export: the batch reaches WORM, the cursor
|
|
114
|
+
advances, and a bounded `siemPending` list records what is not mirrored.
|
|
115
|
+
`retryPendingSiem` replays a pending batch when the host supplies its
|
|
116
|
+
artifact bytes (e.g. fetched from WORM) and verifies the digest matches.
|
|
117
|
+
- Legal holds: the exporter preserves a `legalHold` flag on envelope records
|
|
118
|
+
and never broadens tenant access; enforcing a hold is the host source's job.
|
|
119
|
+
Redaction (`redact`) strips values before hashing so the verifier sees
|
|
120
|
+
exactly the exported bytes; only `{ path, reason }` provenance survives.
|
|
121
|
+
- Caps are frozen: 1,000 records or 10 MiB per batch, at most 8 un-mirrored
|
|
122
|
+
SIEM batches retained.
|
|
123
|
+
|
|
124
|
+
## Security and performance notes
|
|
125
|
+
|
|
126
|
+
- Bytes are canonical JSON (RFC 8785 semantics: sorted keys, ECMAScript
|
|
127
|
+
shortest number round-trip, `-0` collapsed, lowercase control escapes);
|
|
128
|
+
non-finite numbers, BigInt, undefined, functions, symbols, and cyclic
|
|
129
|
+
values are rejected rather than coerced.
|
|
130
|
+
- Each record digest covers `schemaVersion`, `tenantId`, `sequence`,
|
|
131
|
+
`priorDigest`, `legalHold`, `redactions`, and the canonical record — a
|
|
132
|
+
cross-tenant record can never enter another tenant's chain, and the verifier
|
|
133
|
+
replays every envelope from the artifact's own bytes.
|
|
134
|
+
- The WORM acknowledgement must name the batch and match the artifact digest;
|
|
135
|
+
a lying or partial acknowledgement cannot falsely advance the cursor.
|
|
136
|
+
- Signer keys and raw values never enter logs, records, or artifacts.
|
|
137
|
+
- Performance: hashing and signing are linear in record bytes; batches are
|
|
138
|
+
bounded pages built without loading full history. Verification is also
|
|
139
|
+
linear and stateless. Prism does not certify compliance with NIST or any
|
|
140
|
+
SIEM/WORM vendor program; audit-export is the transport, hosts own the
|
|
141
|
+
custody chain and compliance posture.
|
|
142
|
+
|
|
143
|
+
## Related APIs
|
|
144
|
+
|
|
145
|
+
- `createPolicyDecisionStore` / `exportPolicyDecisions` — the ledger that
|
|
146
|
+
commonly backfills an `AuditPageSource`.
|
|
147
|
+
- `PersistenceLifecycleStore` — legal-hold and retention for lifecycle
|
|
148
|
+
records the source may consult.
|
|
149
|
+
- `createMemoryAuditCursorStore` — reference cursor store (replace with a
|
|
150
|
+
durable CAS store in production).
|
|
151
|
+
- `scripts/verify-audit-export.mjs` — standalone verifier CLI.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Data classification and field-level redaction
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
Field-level classification and fail-closed redaction at data boundaries (plan 027 Task 8). An explicit host policy classifies every field of a JSON-like value that crosses a boundary — provider prompt egress, tool dispatch/result persistence, artifact write/read/export, audit hashing, telemetry attributes/events, and export — and returns one of four decisions per field: `allow`, `redact`, `tokenize`, or `deny`. Unknown fields fail closed under the protected default; tenant and legal-hold context is carried through the walk. The policy walk is bounded (depth/keys/string budget, cycle detection, wall-clock budget when configured), rejects unsupported values instead of stringifying guesses, and preserves shape when redaction is required (only touched paths are allocated — untouched subtrees share the input reference).
|
|
6
|
+
|
|
7
|
+
Classification is explicit: labels come from a `labelFor` hint function supplied by the boundary owner. There is no automatic sensitive-data discovery, no global registry, no decorator framework, and no second policy language. Existing hardcoded secret redaction (`createSecretRedactor`) remains in place as defense in depth and runs before the policy pass at egress seams.
|
|
8
|
+
|
|
9
|
+
## When to use it
|
|
10
|
+
|
|
11
|
+
- When a boundary must guarantee that classified fields (secrets, financial data, personal data) never reach a sink unchanged, and unknown fields must be blocked rather than guessed at.
|
|
12
|
+
- When different destinations need different handling of the same payload — e.g. allow a field in the prompt but redact it in telemetry — the destination is part of every decision input.
|
|
13
|
+
- When the protected profile must run fail-closed: the provided `createProtectedFieldPolicy()` denies unknown labels on outbound/persisted boundaries by default.
|
|
14
|
+
- When deep-copy cost at a boundary matters: the sparse-copy walker avoids duplicate serialization and shares pristine subtrees.
|
|
15
|
+
|
|
16
|
+
Compatibility note: existing callers that do not supply a policy are untouched (identity fast path); the protected profile is what enables fail-closed behavior, and protected deployments should supply it at every boundary.
|
|
17
|
+
|
|
18
|
+
## Inputs / request
|
|
19
|
+
|
|
20
|
+
- `applyFieldPolicy(value, policy, options)` — `value` is any JSON-like structure (plain objects, arrays, primitives, `Date`/`RegExp`/buffers pass through; `Map`/`Set` normalize to object/array shapes; functions, bigints, symbols, class instances, and cycles are rejected).
|
|
21
|
+
- `policy` — `(input: FieldPolicyInput) => FieldPolicyDecision`, where the input carries `{path, destination, label?, kind, tenantId?, direction, purpose?}`.
|
|
22
|
+
- `options` — `destination` (required), `direction` (default `"outbound"`), `tenantId`, `purpose`, `labelFor` (explicit key→label hints; no auto-discovery), `onRedact` (provenance hook used by the audit adapter), `maxDepth` (32), `maxKeys` (10,000), `maxChars` (1,000,000), `maxPolicyMs` (only when set), `tokenPrefix` (`tok_`).
|
|
23
|
+
- The protected default is `createProtectedFieldPolicy({ publicLabels, deniedLabels, redactedLabels, tokenizedLabels })`: `public`/structural labels pass, `secret` and `financial` deny, `personal` redacts, `token` tokenizes, and anything unlabeled denies on outbound destinations (`prompt`, `tool`, `artifact`, `audit`, `telemetry`, `export`, `persistence`) while passing inbound.
|
|
24
|
+
|
|
25
|
+
## Outputs / response / events
|
|
26
|
+
|
|
27
|
+
- A tree of the same shape with decisions applied: `deny` replaces the value with `[DENIED]`, `redact` replaces string leaves with `[REDACTED]` while preserving containers, `tokenize` replaces string leaves with a deterministic `tok_<hash>` (stable across runs for the same path+value, safe for audit chains), `allow` keeps the value. Untouched branches share the input reference (sparse copy); the input is never mutated.
|
|
28
|
+
- `onRedact` fires once per transformed field with `{path, reason}`; values are never included.
|
|
29
|
+
- `FieldPolicyError` (code `ERR_PRISM_FIELD_POLICY`) on: policy throw, invalid decision, cyclic reference, unsupported value type, or depth/key/byte/time budget breach. Error messages contain the path and the policy error class — never the value.
|
|
30
|
+
|
|
31
|
+
## Request/response example
|
|
32
|
+
|
|
33
|
+
```ts
|
|
34
|
+
import { applyFieldPolicy, createProtectedFieldPolicy } from "@arnilo/prism";
|
|
35
|
+
|
|
36
|
+
const fieldPolicy = createProtectedFieldPolicy();
|
|
37
|
+
const labelFor = (key: string) =>
|
|
38
|
+
key === "apiKey" ? "secret" : key === "email" ? "personal" : key === "score" ? "public" : undefined;
|
|
39
|
+
|
|
40
|
+
const out = applyFieldPolicy(
|
|
41
|
+
{ score: 1, apiKey: "demo-secret-value", email: "ops@example.test", extra: "unknown" },
|
|
42
|
+
fieldPolicy,
|
|
43
|
+
{ destination: "prompt", direction: "outbound", labelFor },
|
|
44
|
+
);
|
|
45
|
+
// { score: 1, apiKey: "[DENIED]", email: "[REDACTED]", extra: "[DENIED]" }
|
|
46
|
+
|
|
47
|
+
// Audit seam: transformation precedes canonical hashing; only {path, reason} survives.
|
|
48
|
+
const redactor = createAuditFieldRedactor(fieldPolicy, { labelFor });
|
|
49
|
+
// pass redactor as the exporter's `redact` option: createAuditExporter({ redact: redactor, ... })
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## Implementation example
|
|
53
|
+
|
|
54
|
+
- Root ownership: the contract lives in `src/field-policy.ts` of `@arnilo/prism` and is exported from the package index (`applyFieldPolicy`, `createProtectedFieldPolicy`, `hookFieldPolicy`-style adapter `createAuditFieldRedactor`, `ALLOW_FIELD_POLICY`, `FieldPolicyError`, `FIELD_POLICY_LIMITS`).
|
|
55
|
+
- Egress seams: `redactMessage`, `redactProviderRequest`, `redactAgentEvent`, `redactSessionEntry`, and `redactRunLedgerRecord` (from `./redaction.js`) take an optional `(fieldPolicy, destination, labelFor)` — secret redaction runs first, then the policy pass. Without a policy the functions are unchanged (identity).
|
|
56
|
+
- Audit export: `createAuditFieldRedactor(fieldPolicy, { tenantId, labelFor, purpose })` produces the structural `AuditRedactionPolicy` the exporter already applies before canonical hashing; the record's denied/redacted/tokenized bytes are exactly what gets hashed and verified.
|
|
57
|
+
- Telemetry: `createOpenTelemetryInstrumentation({ fieldPolicy })` filters or masks exported span attributes and events — `allow` keeps, `redact` masks with `[REDACTED]`, `deny` drops, `tokenize` hashes — and policy errors drop the attribute without ever echoing the value.
|
|
58
|
+
- Postgres persistence: stores persist exactly what the caller-appointed policy authorizes; the protected profile is a caller-supplied boundary control, not a store default (outbox payloads keep their canonical-collision semantics unchanged).
|
|
59
|
+
|
|
60
|
+
## Extension and configuration notes
|
|
61
|
+
|
|
62
|
+
- Label vocabulary is bounded by the boundary owner: `public`, `personal`, `secret`, `financial`, `token` in the protected default; custom profiles override `deniedLabels`/`redactedLabels`/`tokenizedLabels` maps with their own reasons.
|
|
63
|
+
- Legal hold does not broaden view/export permissions: holds are store-level (deletion prevention), and export-boundary policy denies classified/unknown fields regardless of hold flags.
|
|
64
|
+
- The `labelFor` hint function is the only discovery mechanism; it must be supplied per boundary by the owner who knows the shape. The walker never guesses labels from key names.
|
|
65
|
+
- Tokens are deterministic per (path, value) for a given run; they are not reversible by design and are not a pseudonymization system (no k-anonymity or re-identification risk model).
|
|
66
|
+
- Migration guidance: for existing callers, add the policy at the outermost seam (the redaction functions or the audit/telemetry options) and verify on canaries first; there is no global config key that enables classification, so adoption is per-boundary and explicit.
|
|
67
|
+
|
|
68
|
+
## Security and performance notes
|
|
69
|
+
|
|
70
|
+
- Fail-closed guarantees: unknown fields never cross outbound/persisted boundaries under the protected default; policy exceptions, invalid decisions, and budget breaches throw without echoing values; cycles and unsupported types throw instead of stringifying guesses; tenant mismatch can be enforced inside host policy via `tenantId` on every decision input.
|
|
71
|
+
- Secret canaries: the ERP-T9 matrix proves secret/personal/financial canaries never reach prompt, tool, artifact, audit, telemetry, persistence, or export sinks (denied = `[DENIED]`, redacted = `[REDACTED]`, tokenized = `tok_…`, and the canary string appears nowhere in transformed output or provenance lists).
|
|
72
|
+
- Bounds (frozen): depth 32, keys 10,000, string budget 1,000,000 chars, optional wall-clock budget; the walk is recursive with an active-path set — cycles terminate, diamond references are re-walked per branch.
|
|
73
|
+
- Overhead (frozen cap `classificationMaxOverheadPercent = 10`): measured against the pre-existing boundary walk (the secret-redaction walk boundaries already ran before classification existed) on the frozen representative payload sizes, interleaved A/B — prompt 4,164 B → 99.0%, toolArgs 2,114 B → 95.8%, toolResult 9,095 B → 97.1%, artifactMetadata 3,692 B → 99.0%, auditRecord 4,243 B → 99.8%, telemetry 1,760 B → 97.7%, exportPage 10,726 B → 98.0% of the redactor-walk baseline (peak 99.8%, all ≤ 110%). The raw ratio vs native `JSON.stringify` is recorded in the Task 8 evidence (≈1.0–1.3× across fixtures); a JS policy gateway cannot beat a native serializer, so the frozen cap is defined against the walk work the boundary already performed, and this stays the benchmark contract.
|
|
74
|
+
- The telemetry seam drops attributes on policy error rather than failing the whole span; the audit seam maintains redaction provenance `{path, reason}` only — values never enter the hash chain.
|
|
75
|
+
|
|
76
|
+
## Related APIs
|
|
77
|
+
|
|
78
|
+
- `redactMessage` / `redactProviderRequest` / `redactAgentEvent` / `redactSessionEntry` / `redactRunLedgerRecord` — the egress seams that take the optional policy (secret redaction first, then classification).
|
|
79
|
+
- `createAuditFieldRedactor` → the audit-export `redact` hook; see [Signed, hash-chained audit export](audit-export.md).
|
|
80
|
+
- `createOpenTelemetryInstrumentation` in `@arnilo/prism-observability-opentelemetry` — the telemetry `fieldPolicy` option.
|
|
81
|
+
- `createProtectedFieldPolicy`, `ALLOW_FIELD_POLICY`, `FieldPolicyError`, `FIELD_POLICY_LIMITS` — the protected default and limits.
|
|
82
|
+
- The ERP-T9 threat matrix (`src/__tests__/field-policy.test.ts`) and the boundary-drill scripts cover the enforcement evidence.
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Disaster recovery and backup operations
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
This page is the operator runbook for backup, restore, migration rollback, point-in-time recovery (PITR), and disaster recovery (DR), proven by plan 027 Task 7 with a protected drill that uses only standard PostgreSQL tools (`pg_dump` custom format, `pg_restore`, `pg_basebackup`, `psql`) plus the existing migration runner. The drill seeds representative multi-tenant 0.2.7 state (sessions, workflow/saga/ACP/conversation checkpoints, leases, legal holds, tenant quotas, policy decisions, evaluations, work idempotency, tool effects, model-router budgets, ERP outbox/inbox, approvals) through the real store APIs, backs it up, restores it into an explicitly confirmed disposable database, verifies per-table row counts and content digests equal the source, rehearses the 0.2.6 → 0.2.7 migration forward with old rows preserved, rehearses rollback by restoring the pre-upgrade backup, and runs PITR against a WAL-archived cluster to a point between two known writes. The recorded run lives in `docs/_evidence/phase27-dr-evidence.json`.
|
|
6
|
+
|
|
7
|
+
## When to use it
|
|
8
|
+
|
|
9
|
+
- Before running any migration or release in a production environment: decide the rollback path first (roll-forward repair preferred; backup-restore is the last resort and only in a disposable environment).
|
|
10
|
+
- When an operator must restore a database: read the guarded-command requirements below — this project has no "force restore" shortcut, and destructive commands always require explicit positive confirmation.
|
|
11
|
+
- When sizing backup windows or reviewing RPO/RTO: read the measured numbers in the evidence file and the ownership table (managed backup, encryption, retention scheduling, and cross-region replication are operator-owned and not claimed by Prism).
|
|
12
|
+
|
|
13
|
+
## Inputs / request
|
|
14
|
+
|
|
15
|
+
- A source instance URL (`PRISM_TEST_POSTGRES_URL` in the drill; any supported PostgreSQL ≥ 14 in practice).
|
|
16
|
+
- An explicitly named, disposable target database on a loopback non-production instance, supplied as `--target` plus the confirmation token `--confirm-target prism_dr_restore`. The target must not already exist; the drill refuses dirty state rather than clobbering it.
|
|
17
|
+
- A separate WAL-archived cluster for PITR (`PRISM_PITR_URL`) with `wal_level=replica`, `archive_mode=on`, and `archive_command='cp %p /wal_archive/%f'`; source and PITR containers must mount a shared host dir at `/dr` for artifact exchange.
|
|
18
|
+
- Sufficient free space (the drill asserts ≥ 512 MB headroom on the artifact dir before starting).
|
|
19
|
+
|
|
20
|
+
## Outputs / response / events
|
|
21
|
+
|
|
22
|
+
- A custom-format backup artifact (`.dump`) with its SHA-256 digest, byte size, duration, and table list count.
|
|
23
|
+
- A restore report: per-table count and content-digest equality against the source (application-level verification, not just exit codes), and duration.
|
|
24
|
+
- A migration report: the 0.2.6-era schema (migrations 001–003) with legacy rows, the upgraded 0.2.7 schema (all five migrations) with old rows preserved and new tables initialized empty, and the rollback rehearsal showing the pre-upgrade backup restores exactly and excludes 0.2.7 tables.
|
|
25
|
+
- A PITR report: recovery target time between two known writes, the earlier write present and the later write absent, recovery duration, and measured RPO/RTO.
|
|
26
|
+
|
|
27
|
+
## Request/response example
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
# Protected drill (standard tools only, orchestrated by the script):
|
|
31
|
+
PRISM_PITR_URL=postgresql://user:***@localhost:55436/postgres \
|
|
32
|
+
node scripts/phase27-dr.test.mjs \
|
|
33
|
+
--source "$PRISM_TEST_POSTGRES_URL" \
|
|
34
|
+
--target postgresql://user:***@localhost:55432/prism_dr_target \
|
|
35
|
+
--confirm-target prism_dr_restore
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
```ts
|
|
39
|
+
// Verifying a restore at the application level (from the drill):
|
|
40
|
+
const before = await digest(pool, "prism_erp_outbox"); // md5 of ordered rows
|
|
41
|
+
const after = await digest(restorePool, "prism_erp_outbox");
|
|
42
|
+
assert.equal(after, before); // content equality, not exit code
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Implementation example
|
|
46
|
+
|
|
47
|
+
The drill's legs: (1) seed multi-tenant state through `createPostgresPersistence` and `createPostgresEnterpriseState` plus `createPostgresApprovalStore` with a host authority; (2) `pg_dump -F c` into the shared `/dr` mount; (3) `createdb` the confirmed target and `pg_restore --no-owner --no-privileges`, then verify counts and digests per table; (4) build the 0.2.6-era schema from the raw DDL builders with the recorded migration registry rows, seed legacy rows through the real stores, take a backup, run `applyEnterpriseMigrations` (004/005 apply, old rows preserved, new tables empty), then restore the pre-upgrade backup into a fresh database and confirm 0.2.7 tables are absent; (5) on the WAL-archived cluster, take a `pg_basebackup`, insert two marker transactions a known distance apart, switch WAL and wait for archiving, then start a recovered instance with `recovery.signal`, a bounded `restore_command`, and `recovery_target_time` set between the two writes with `recovery_target_action=pause`, and verify the earlier marker exists and the later one does not.
|
|
48
|
+
|
|
49
|
+
## Extension and configuration notes
|
|
50
|
+
|
|
51
|
+
- Storage, encryption at rest, retention scheduling, and cross-region replication are operator-owned: the drill proves the command and verification path, not a managed backup service.
|
|
52
|
+
- Rollback decision tree: prefer roll-forward repair after a production migration. Restore-from-backup is the last resort, permitted only in a disposable evidence environment; the recorded loss window is "writes between the pre-upgrade backup and the rollback restore".
|
|
53
|
+
- Recovery parameters: `recovery_target_time` in the recovered instance needs `recovery.signal` (or `standby.signal`) present, a `restore_command`, and a full timestamp including the sub-second fraction and offset — truncating to whole seconds can land the recovery point before the intended write.
|
|
54
|
+
- Guards that must stay in place: source and target databases must differ; the target host must be loopback and the database name must not match production patterns; the target must not pre-exist; the confirmation token is mandatory; secret canaries seeded into the data must never appear in the manifest or console output.
|
|
55
|
+
- Quarterly re-run the drill and refresh `docs/_evidence/phase27-dr-evidence.json` as the schema evolves; treat any change to table counts/digests as needing a new recorded run.
|
|
56
|
+
|
|
57
|
+
## Security and performance notes
|
|
58
|
+
|
|
59
|
+
- Credentials are explicit and redacted: the manifest stores URLs with passwords masked, and the drill asserts the source/target/PITR passwords and the seeded secret canary never appear in the evidence, logs, or console. Connection strings are passed to the tools via the container environment, never printed.
|
|
60
|
+
- Destructive commands (dropping schemas/databases, restoring over an existing target) require explicit operator action; the drill fails closed on dirty state and never deletes anything itself.
|
|
61
|
+
- Legal-hold data is verified: legal-hold records and their referenced rows survive backup and restore with content digests intact; enforcement of holds stays host-owned (the stores preserve the records; the leases/quota/outbox lifecycle logic stays unchanged).
|
|
62
|
+
- Performance: measured in the disposable environment — backup 108,291 bytes in 122 ms, restore 382 ms, PITR recovery 1.2 s with the two markers a sub-second apart (RPO ≈ 0 s, RTO ≈ 1 s). These are environment-local measurements for sizing, not guarantees; the drill records sizes, timings, and digests in the evidence file and fails on breached frozen budgets.
|
|
63
|
+
- No exactly-once guarantee is claimed anywhere in the backup/restore path; the drill verifies observable equality (counts and digests) instead.
|
|
64
|
+
|
|
65
|
+
## Related APIs
|
|
66
|
+
|
|
67
|
+
- `createPostgresPersistence` / `createPostgresEnterpriseState` / `createPostgresApprovalStore` — the stores whose state the drill seeds and verifies.
|
|
68
|
+
- `applyEnterpriseMigrations` — the migration runner used for the forward upgrade rehearsal.
|
|
69
|
+
- `scripts/phase27-dr.test.mjs` — the protected drill; `docs/_evidence/phase27-dr-evidence.json` — the recorded run.
|
|
70
|
+
- [Operations runbook: high availability, failover, and fencing](operations.md) — the lease/fence model and failover ceiling the same state relies on.
|
|
71
|
+
- [Signed, hash-chained audit export](audit-export.md) — the append-only audit ledger preserved through backup/restore.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## What it does
|
|
4
4
|
|
|
5
|
-
`@arnilo/prism-enterprise-postgres` is one optional PostgreSQL composition for
|
|
5
|
+
`@arnilo/prism-enterprise-postgres` is one optional PostgreSQL composition for existing enterprise state seams:
|
|
6
6
|
|
|
7
7
|
| State | Composition property | Durable behavior |
|
|
8
8
|
| --- | --- | --- |
|
|
@@ -11,9 +11,10 @@
|
|
|
11
11
|
| Work mutations | `workIdempotency` | Atomic claim/CAS lifecycle for connector effects. |
|
|
12
12
|
| Model routing | `modelRouter` | Shared rate, budget, and circuit state for router replicas. |
|
|
13
13
|
| Tool effects | `toolEffects` | Durable `ToolEffectStore` claim/CAS for recoverable tool side effects (migration 002). |
|
|
14
|
-
|
|
|
14
|
+
| ERP messaging | `erpMessaging` | Transactional outbox/inbox markers plus bounded, tenant-scoped at-least-once dispatch (migration 004). |
|
|
15
|
+
| Multi-party approvals | `createPostgresApprovalStore({ pool, schema, authority })` | Immutable approval requests, role/quorum decisions, revocation, bounded delegation, and atomic grant consumption (migration 005). |
|
|
15
16
|
|
|
16
|
-
`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-session-store-postgres`](postgres-persistence.md).
|
|
17
|
+
`createPostgresEnterpriseState()` opens a host-supplied or adapter-owned `pg` pool, verifies/applies checksum-protected enterprise migrations (`001_enterprise_state`, `002_tool_effects`, `003_router_reservations`, `004_erp_messaging`, `005_erp_approvals`), and returns those stores plus explicit cleanup and close operations. Importing it performs no I/O. It is separate from session/run persistence in [`@arnilo/prism-session-store-postgres`](postgres-persistence.md).
|
|
17
18
|
|
|
18
19
|
## When to use it
|
|
19
20
|
|
|
@@ -58,7 +59,7 @@ interface PostgresEnterpriseState {
|
|
|
58
59
|
readonly workIdempotency: IdempotencyStore;
|
|
59
60
|
readonly modelRouter: ModelRouterStateStore;
|
|
60
61
|
readonly toolEffects: ToolEffectStore;
|
|
61
|
-
readonly
|
|
62
|
+
readonly erpMessaging: PostgresErpMessaging;
|
|
62
63
|
cleanup(input: EnterpriseStateCleanupInput): Promise<EnterpriseStateCleanupResult>;
|
|
63
64
|
close(): Promise<void>;
|
|
64
65
|
}
|
|
@@ -68,6 +69,10 @@ interface PostgresEnterpriseState {
|
|
|
68
69
|
|
|
69
70
|
Work mutations expose six observable states: **absent**, `in_progress`, `completed`, `failed_retryable`, `failed_terminal`, and `unknown`. `begin()` atomically returns `acquired` or the existing record; `complete`/`fail`/`markUnknown` use claim-token plus version compare-and-swap. `unknown` requires an operator/connector-specific `resolveUnknown` decision. It is not automatically replayed and it does **not** claim exactly-once external effects.
|
|
70
71
|
|
|
72
|
+
ERP messaging exposes outbox states `pending`, `dispatched`, `retryable`, `completed`, `unknown`, and `dead_letter`. The caller owns the `pg.PoolClient` transaction: business mutation plus `erpMessaging.outbox.append(client, input)` commit or roll back together. Consumers call `erpMessaging.inbox.record(client, input)` before their local mutation in the same transaction; duplicate delivery returns `false`. `dispatcher.claim()` uses bounded `FOR UPDATE SKIP LOCKED` pages and leases. Acknowledgement, retry, and unknown transitions use claim-token plus version CAS. Expired leases become `unknown`; replay/dead-letter requires a host-verified actor and non-empty audit reference. Delivery remains at-least-once, never exactly-once.
|
|
73
|
+
|
|
74
|
+
Approvals expose request states `pending`, `approved`, `rejected`, `revoked`, and `consumed`. Immutable request data (action digest, requester, role/quorum requirements, separation flag, expiry, delegation depth) plus a monotonic `revision` and the accepted decision array live in one row. `decide`/`revoke` lock the row `FOR UPDATE` and revision-check the terminal transition in one transaction; a rejection is a terminal veto. `consume` verifies tenant, action digest, expiry, policy revision, and revision before flipping `approved` → `consumed` inside the caller-owned transaction, so grant consumption and the protected action commit (or roll back) together. Roles come from the host `ApprovalAuthority`; Prism never treats model/tool/subagent claims as principals.
|
|
75
|
+
|
|
71
76
|
Model-router state is asynchronous and owner/principal/provider/model scoped. Supplying it to `createModelRouter({ stateStore })` requires awaited `resolve`, `recordUsage`, and `recordOutcome` calls with verified identity. The legacy synchronous `providerSource` facade throws `ERR_PRISM_MODEL_ROUTER_ASYNC_STATE` when durable state is configured.
|
|
72
77
|
|
|
73
78
|
## Request/response example
|
|
@@ -85,10 +90,55 @@ Model-router state is asynchronous and owner/principal/provider/model scoped. Su
|
|
|
85
90
|
}
|
|
86
91
|
```
|
|
87
92
|
|
|
88
|
-
A migration creates `prism_policy_decisions`, `prism_evaluations`, `prism_work_idempotency`, three `prism_model_router_*` tables, and its separate `prism_enterprise_migrations` history. Migration `003_router_reservations` adds the nullable-by-default `reservations` JSONB column to `prism_model_router_budgets` (atomic reservation slots for router admission; 0.2.1 readers ignore it). Startup serializes per-schema setup with an advisory transaction lock and rejects checksum or catalog drift rather than silently repairing it.
|
|
93
|
+
A migration creates `prism_policy_decisions`, `prism_evaluations`, `prism_work_idempotency`, three `prism_model_router_*` tables, `prism_erp_outbox`, `prism_erp_inbox`, `prism_erp_approvals`, and its separate `prism_enterprise_migrations` history. Migration `003_router_reservations` adds the nullable-by-default `reservations` JSONB column to `prism_model_router_budgets` (atomic reservation slots for router admission; 0.2.1 readers ignore it). Migration `004_erp_messaging` adds tenant/message and tenant/consumer/message primary keys plus claim, lease, and inbox indexes. Migration `005_erp_approvals` adds the one-row-per-request approval table (PK `tenant_id + id`, status check, decisions JSONB, status/created indexes). Startup serializes per-schema setup with an advisory transaction lock and rejects checksum or catalog drift rather than silently repairing it.
|
|
89
94
|
|
|
90
95
|
## Implementation example
|
|
91
96
|
|
|
97
|
+
```ts
|
|
98
|
+
import { createPostgresErpMessaging } from "@arnilo/prism-enterprise-postgres";
|
|
99
|
+
|
|
100
|
+
const messaging = createPostgresErpMessaging({ pool, schema: "prism" });
|
|
101
|
+
const client = await pool.connect();
|
|
102
|
+
try {
|
|
103
|
+
await client.query("BEGIN");
|
|
104
|
+
await client.query("UPDATE invoices SET status = $1 WHERE tenant_id = $2 AND id = $3", ["posted", tenantId, invoiceId]);
|
|
105
|
+
await messaging.outbox.append(client, {
|
|
106
|
+
tenantId,
|
|
107
|
+
messageId: `invoice:${invoiceId}:posted`,
|
|
108
|
+
topic: "invoice.posted",
|
|
109
|
+
payload: { invoiceId },
|
|
110
|
+
});
|
|
111
|
+
await client.query("COMMIT");
|
|
112
|
+
} catch (error) {
|
|
113
|
+
await client.query("ROLLBACK");
|
|
114
|
+
throw error;
|
|
115
|
+
} finally {
|
|
116
|
+
client.release();
|
|
117
|
+
}
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Consumer transaction uses same client:
|
|
121
|
+
|
|
122
|
+
```ts
|
|
123
|
+
await client.query("BEGIN");
|
|
124
|
+
if (await messaging.inbox.record(client, { tenantId, consumer: "ledger", messageId })) {
|
|
125
|
+
await client.query("UPDATE ledger SET posted = TRUE WHERE tenant_id = $1 AND message_id = $2", [tenantId, messageId]);
|
|
126
|
+
}
|
|
127
|
+
await client.query("COMMIT");
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Dead-letter/replay is host-authorized and auditable:
|
|
131
|
+
|
|
132
|
+
```ts
|
|
133
|
+
await messaging.dispatcher.replay({
|
|
134
|
+
tenantId,
|
|
135
|
+
messageId,
|
|
136
|
+
expectedVersion,
|
|
137
|
+
auditRef: "audit:erp-replay:2026-08-17T00:00:00Z",
|
|
138
|
+
authorizedBy: verifiedOperatorIdentity,
|
|
139
|
+
});
|
|
140
|
+
```
|
|
141
|
+
|
|
92
142
|
```ts
|
|
93
143
|
import type { AgentIdentity } from "@arnilo/prism";
|
|
94
144
|
import { createPostgresEnterpriseState, type PostgresEnterpriseState } from "@arnilo/prism-enterprise-postgres";
|
|
@@ -153,11 +203,13 @@ export async function recordEnterpriseState(state: PostgresEnterpriseState) {
|
|
|
153
203
|
## Extension and configuration notes
|
|
154
204
|
|
|
155
205
|
- `createModelRouter({ resolver, stateStore: state.modelRouter })` keeps allow-list, residency, fallback, and diagnostics behavior in `@arnilo/prism-model-router`; this package only supplies durable state. Router admission reservations (`reserveBudget`/`commitBudget`/`releaseBudget` on `state.modelRouter`) live in the `reservations` JSONB column of `prism_model_router_budgets`: one atomic UPSERT per admission, fencing-token-guarded commit/release in a SERIALIZABLE transaction, and TTL reconciliation as unknown usage; see [Model routing](model-routing.md).
|
|
206
|
+
- `createPostgresErpMessaging({ pool, schema? })` is the direct messaging composition. `outbox.append` and `inbox.record` accept a caller-owned `PoolClient`; the host must put them in the same transaction as its local mutation. The dispatcher owns only short claim/transition transactions and never invokes business callbacks or stores executable handlers.
|
|
207
|
+
- `createPostgresApprovalStore({ pool, schema?, authority })` is the direct approval composition (migration 005). `authority.resolveRoles(actor, request)` and `policyRevision` are host-owned; Prism persists only accepted role grants and delegation chains. `decide`/`revoke` lock the request row and revision-check the terminal transition in one transaction. `consume` accepts an optional caller-owned `client`; grant consumption and the protected action commit (or roll back) together.
|
|
156
208
|
- Rate/budget/circuit tables are capped like the memory store: `consumeRate`/`readBudget`/`addUsage`/`reserveBudget` accept `maxRateKeys`/`maxBudgetKeys` (the router passes its resolved limits) and evict the least-recently-used row on new-key insert — never the row just inserted, never a budget row holding an active reservation — else fail closed with `ERR_PRISM_MODEL_ROUTER_STATE`. Cleanup prunes expired reservations within its bounded batch.
|
|
157
|
-
- Policy/evaluation/query public contracts stay in their owning packages. This package exports
|
|
209
|
+
- Policy/evaluation/query public contracts stay in their owning packages. This package exports `createPostgresEnterpriseState`, `createPostgresApprovalStore`, `createPostgresErpMessaging`, their options/result/types, and `EnterprisePostgresError`; it has no SQL, DDL, codec, queryable, or migration subpath.
|
|
158
210
|
- The fixed schema has no generic key/value table and no background cleanup scheduler. Schedule `state.cleanup()` from an authorized host job, size its bounded batch for the deployment, and monitor unknown work rows for reconciliation. Run protected integration checks with `PRISM_TEST_POSTGRES_URL="$DATABASE_URL" npm run test:postgres`; the command rejects an absent URL instead of silently skipping database coverage.
|
|
159
211
|
- The OPA adapter (`@arnilo/prism-policy/opa`, 0.0.28) records decisions into the same `state.policy` store unchanged via `evaluateAndAppend` — see [Policy and audit](policy-and-audit.md#opa-external-policy-adapter-arniloprism-policyopa-008).
|
|
160
|
-
- Request-path state SQL uses `SELECT`, `INSERT`, `UPDATE`, and `DELETE` on the
|
|
212
|
+
- Request-path state SQL uses `SELECT`, `INSERT`, `UPDATE`, and `DELETE` on the eight state tables. The open/migration lifecycle additionally needs schema/catalog/advisory-lock and DDL permissions. Use a deployment migration principal for that lifecycle and a least-privilege request role for request traffic; this release intentionally does not ship a migration CLI or worker.
|
|
161
213
|
|
|
162
214
|
## Security and performance notes
|
|
163
215
|
|
|
@@ -165,6 +217,7 @@ export async function recordEnterpriseState(state: PostgresEnterpriseState) {
|
|
|
165
217
|
- Every value is a bound parameter. Schema identifiers are validated; table/index names are fixed. Cursors embed and recheck ownership, so a foreign tenant cannot reuse a page cursor.
|
|
166
218
|
- Policy records cap at 64 KiB; evaluations at 64 KiB; work rows at 8 KiB; router material at 512 bytes. JSON rejects prototype-pollution keys, non-finite values, excess depth/properties, and over-size material.
|
|
167
219
|
- PostgreSQL transaction SQLSTATE `40001`/`40P01` retries whole safe transactions up to three times. Connector effects remain outside those transactions and ambiguous errors become `unknown` rather than being retried.
|
|
220
|
+
- ERP claim pages are tenant-scoped and bounded at 1,000 rows; default batch is 100, lease TTL is 30 seconds with a 5-minute hard cap, and retry attempts are capped at 10. Protected PostgreSQL evidence measured 1,000 queued rows per tenant across 10 tenants at p50 5.999 ms / p95 7.827 ms / p99 8.066 ms for 100-row claims; the representative plan used `prism_erp_outbox_claim_idx` with no sequential scan. This is comparison evidence, not a hardware-independent guarantee.
|
|
168
221
|
- Recorded `postgres:16-alpine` evidence (Node 24.18.0/Linux x64, 10 tenants × 10 principals × 1,000 policy/evaluation rows, 10,000 router keys, 16 clients) stayed below 50 ms p95 for point operations and 100 ms for cursor/cleanup pages. The highest recorded p95 was router-circuit contention at 28.410 ms. Fourteen representative `EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON)` shapes used named indexes with no sequential scans. These are recorded comparison evidence, not hardware-independent guarantees.
|
|
169
222
|
|
|
170
223
|
## Related APIs
|
package/docs/evaluations.md
CHANGED
|
@@ -158,6 +158,51 @@ const page = await state.evaluations.query({ tenantId: "t1", userId: "u1", statu
|
|
|
158
158
|
|
|
159
159
|
Memory evaluation storage remains suitable for development and deterministic tests; it is not cross-replica production storage. PostgreSQL bounds each evaluation row to 64 KiB (reason/error 8 KiB each and metadata 32 KiB).
|
|
160
160
|
|
|
161
|
+
## ERP invariant evals (0.2.7)
|
|
162
|
+
|
|
163
|
+
Plan 027 adds two frozen exports to this package for the deterministic ERP release journey: `erpInvariantDataset` and `createErpInvariantScorers`. Scorers consume **structured journey facts only** — never model prose, credentials, or classified payloads. Each of the eight invariants is a hard 0/1 gate; no weighted average can hide an atomicity or security failure.
|
|
164
|
+
|
|
165
|
+
### Fact schema
|
|
166
|
+
|
|
167
|
+
The protected runner (`scripts/phase27-erp-journey.test.mjs`) carries the journey facts as JSON in `result.text`. Every scorer isolates its facts block and returns 1 only when every required fact is present and truthy:
|
|
168
|
+
|
|
169
|
+
| Invariant (scorer id) | Facts block | Required facts |
|
|
170
|
+
|---|---|---|
|
|
171
|
+
| `atomic-intent` | `atomic` | `committedAtomically` |
|
|
172
|
+
| `single-local-effect` | `delivery` | `singleLocalEffect`, `duplicateDelivered`, `businessMutationCount` |
|
|
173
|
+
| `compensation-terminal` | `compensation` | `compensated`, `reconciled`, `terminalStatus` |
|
|
174
|
+
| `quorum-provenance` | `quorum` | `distinctApprovers`, `requesterDenied`, `subagentDenied`, `revokedDenied`, `provenance` |
|
|
175
|
+
| `chain-verification` | `chain` | `verified`, `tamperedDetected`, `nextDigest` |
|
|
176
|
+
| `no-leak` | `noLeak` | `classifiedDenied`, `crossTenantDenied`, `secretRedacted` |
|
|
177
|
+
| `fenced-failover` | `fencedFailover` | `resumedByPeer`, `staleWriteRejected`, `cursorPreserved`, `failoverMs` |
|
|
178
|
+
| `restore-equality` | `restore` | `factsMatch`, `digestsMatch`, `drEvidenceFresh`, `restoreMs` |
|
|
179
|
+
|
|
180
|
+
### Hard-gate usage
|
|
181
|
+
|
|
182
|
+
```ts
|
|
183
|
+
import { createErpInvariantScorers, erpInvariantDataset, scoreRun } from "@arnilo/prism-evals";
|
|
184
|
+
|
|
185
|
+
const scorers = createErpInvariantScorers();
|
|
186
|
+
const records = await scoreRun({
|
|
187
|
+
result, // AgentRunResult whose .text is the JSON journey facts
|
|
188
|
+
scorers,
|
|
189
|
+
datasetId: erpInvariantDataset.id,
|
|
190
|
+
});
|
|
191
|
+
if (records.some((record) => record.status !== "scored" || record.score !== 1)) {
|
|
192
|
+
process.exitCode = 1; // a single failing invariant fails the whole gate
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
### Execution command and substitutes
|
|
197
|
+
|
|
198
|
+
```sh
|
|
199
|
+
# Protected run (requires a disposable PostgreSQL instance):
|
|
200
|
+
PRISM_TEST_POSTGRES_URL=postgresql://... node --test scripts/phase27-erp-journey.test.mjs
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
The journey reuses the two-replica failover worker (`scripts/phase27-ha-worker.mjs`) and asserts the comprehensive DR drill evidence (`docs/_evidence/phase27-dr-evidence.json`) is present and not stale. Local substitutes are labelled in the journey evidence and never converted into production claims: an in-memory WORM/SIEM sink (host owns the immutable store in production), in-memory saga checkpoint/lease stores (saga durability is proven in its own suite), and a logical pg-client backup/restore of the ERP tables (comprehensive PITR is in the DR drill evidence). Passing this protected journey **does not** satisfy the 0.3.0 live-service matrix.
|
|
204
|
+
|
|
205
|
+
|
|
161
206
|
## Related APIs
|
|
162
207
|
|
|
163
208
|
- [Agent/session runtime](agent-session-runtime.md): `AgentRunResult` and `session.run()`
|
package/docs/host-security.md
CHANGED
|
@@ -131,6 +131,9 @@ Wire those values where they matter: provider adapters receive the resolved cred
|
|
|
131
131
|
|
|
132
132
|
## Security and performance notes
|
|
133
133
|
|
|
134
|
+
- 0.2.7 (plan 027 Task 8) adds field-level classification and fail-closed redaction at data boundaries: `applyFieldPolicy` walks JSON-like values with explicit `allow`/`redact`/`tokenize`/`deny` decisions, the protected default denies unknown fields on outbound/persisted boundaries, and labels come from per-boundary `labelFor` hints — never auto-discovered. Policy errors, cycles, unsupported types, and budget breaches fail closed without echoing values; the audit seam transforms before canonical hashing and retains only `{path, reason}` provenance; the telemetry seam drops attributes on policy error. See [Data classification and field-level redaction](data-classification.md) for the full contract, boundary matrix, limits, and the recorded overhead vs the pre-existing redaction walk.
|
|
135
|
+
|
|
136
|
+
|
|
134
137
|
- Fail closed: unknown providers, unknown tools, denied tools, invalid tool arguments, missing skill tool dependencies, trust failures, permission failures, append conflicts, and validator failures should stop the unsafe action.
|
|
135
138
|
- Prism does not sandbox host tools, extensions, provider adapters, credential resolvers, or custom middleware. Use OS/container/process isolation when code is untrusted.
|
|
136
139
|
- Redaction is exact known-secret replacement only. It is not arbitrary secret detection, entropy scanning, or DLP.
|
|
@@ -234,3 +237,4 @@ Every durable `AgentEventSource` page/subscribe and tool-effect claim rechecks e
|
|
|
234
237
|
- [Database persistence](database-persistence.md): production schema, ownership, indexes, retention, and adapter readiness checklist.
|
|
235
238
|
- [Enterprise PostgreSQL state](enterprise-postgres-state.md): durable policy/evaluation/work/router state and database-role boundary.
|
|
236
239
|
- [Provider caching](provider-caching.md): cache keys and provider-owned header safety rules.
|
|
240
|
+
- [Data classification and field-level redaction](data-classification.md): the field policy contract, protected default, and fail-closed boundary matrix (plan 027 Task 8).
|