@agent-custody/state 0.1.7 → 0.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -7
- package/dist/cli.js +60 -2
- package/dist/evals-file.d.ts +53 -0
- package/dist/evals-file.js +43 -0
- package/dist/index.d.ts +4 -0
- package/dist/index.js +2 -0
- package/dist/ledger.d.ts +12 -3
- package/dist/ledger.js +24 -21
- package/dist/storage.d.ts +32 -0
- package/dist/storage.js +67 -0
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -138,9 +138,23 @@ import { Ledger, ledgerUnderTest, runAll, SCENARIOS, formatReport } from "@agent
|
|
|
138
138
|
console.log(formatReport(await runAll(ledgerUnderTest(new Ledger("./ledger.jsonl")), SCENARIOS)));
|
|
139
139
|
```
|
|
140
140
|
|
|
141
|
+
From the shell, on a schedule:
|
|
142
|
+
|
|
143
|
+
```bash
|
|
144
|
+
agent-custody-memory eval --baseline # built-in scenarios, ledger and a naive store side by side
|
|
145
|
+
agent-custody-memory eval --scenarios incidents.json --sign keys/eval.key --out report.json # your own incidents, signed report
|
|
146
|
+
agent-custody-memory eval --verify report.json --key keys/eval.pub # what a reviewer runs
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
A scenario file is `{ "version": "0.1", "scenarios": [{ "name", "ops": [...] }] }` with the same write, read, and retract ops as the built-ins; every read carries `expect`, null meaning nothing. The file is validated and a bad one names the offending op. The signed report is an in-toto statement over the scores, bound to a digest of the scenarios it ran, in the same envelope format as receipts; `eval --verify` checks the signature and the digest. `eval` exits non-zero when the ledger scores below perfect on the scenarios it was given, so a cron job that runs it fails loudly when a change regresses memory behaviour.
|
|
150
|
+
|
|
141
151
|
## The ledger
|
|
142
152
|
|
|
143
|
-
`src/ledger.ts`
|
|
153
|
+
`src/ledger.ts` keeps an append-only log of events, in memory for its queries and in one of two stores for durability. **JSONL** is the default: one event per line, readable by anyone, the file you copy for an audit. **SQLite** is for durability and shared use, chosen by giving the ledger a path ending in `.sqlite` or `.db`: transactional writes with write-ahead logging, an in-place forget that overwrites the erased value (secure delete, then a truncating checkpoint so nothing lingers in the write-ahead log), indexes by fact and by time, and a file more than one process can open. Node ships the SQLite module, so there is no native dependency; on Node 22 it prints an experimental warning once. `agent-custody-memory export --ledger ledger.sqlite --out ledger.jsonl` writes the auditable JSONL from any store, so the copy-the-file audit path survives the choice. The whole ledger test suite runs against both stores.
|
|
154
|
+
|
|
155
|
+
Choose JSONL for one server process and for anything an auditor should be able to read with `cat`. Choose SQLite when the ledger is shared over HTTP by several gateways, when a crash between two writes must not cost you an event, or when a deletion demand must leave no trace of the value in the file. Query pushdown to SQLite's indexes is not done yet: both stores load every event into memory, which is fine to tens of thousands of facts. [Issue #8](https://github.com/ch4r10t33r/agent-custody/issues/8) tracks indexed queries for larger ledgers.
|
|
156
|
+
|
|
157
|
+
The log holds these kinds of event.
|
|
144
158
|
|
|
145
159
|
- **assert** creates a fact: subject, predicate, value, the space it lives in (a person, a team, an org), the actor that wrote it, the source receipt id if the write went through a receipts producer, and the valid-time interval. An assert may **supersede** an earlier fact, which ends that fact's validity where the new one begins.
|
|
146
160
|
- **retract** says a fact should never have been believed. This is the undo. The record stays, so queries about earlier moments still see the fact, and anything the retracted fact had superseded is believed again.
|
|
@@ -159,14 +173,16 @@ The ledger refuses to supersede a fact that is unknown, already superseded, or r
|
|
|
159
173
|
## Layout
|
|
160
174
|
|
|
161
175
|
```
|
|
162
|
-
src/ledger.ts the fact record, the
|
|
176
|
+
src/ledger.ts the fact record, the event kinds, as-of queries, supersession, retraction, forget, holds, sweeps
|
|
177
|
+
src/storage.ts the event stores: JSONL (default, auditable) and SQLite (durable, shared), chosen by file extension
|
|
163
178
|
src/server.ts the ledger as MCP tools; source and actor taken from the gateway's _meta
|
|
164
179
|
src/http.ts the memory server over Streamable HTTP with bearer auth, for a shared ledger
|
|
165
|
-
src/cli.ts agent-custody-memory serve (stdio or --http, with --retention and --forget-key-env), sweep (ledger-only or --via a gateway), blast
|
|
180
|
+
src/cli.ts agent-custody-memory serve (stdio or --http, with --retention and --forget-key-env), sweep (ledger-only or --via a gateway), eval, export, blast
|
|
166
181
|
src/blast.ts blast radius: from receipts' consumed facts and the ledger's source receipts, forward
|
|
167
182
|
src/stores.ts write-through adapters: Mem0 and Zep, and the Store interface for others
|
|
168
183
|
src/evals.ts the memory-mutation harness: scenarios, scoring, report
|
|
169
184
|
src/evals-ledger.ts the ledger and a naive overwrite store behind the harness interface
|
|
185
|
+
src/evals-file.ts scenario files, validated; signed eval reports and their verification
|
|
170
186
|
src/index.ts public surface
|
|
171
187
|
examples/ runnable walkthroughs, each ends with OK and is run by the test suite
|
|
172
188
|
test/ one test per question a platform owner asks after a memory incident
|
|
@@ -177,7 +193,7 @@ tsconfig.build.json emits dist/ for consumers; the repo itself runs the .ts dir
|
|
|
177
193
|
|
|
178
194
|
**Done**
|
|
179
195
|
|
|
180
|
-
- Bitemporal fact ledger with supersession, retraction, as-of and history queries, persisted as JSONL.
|
|
196
|
+
- Bitemporal fact ledger with supersession, retraction, as-of and history queries, persisted as JSONL or SQLite behind one store interface, with export back to JSONL.
|
|
181
197
|
- The memory server: the ledger as MCP tools behind the receipts gateway, with the source receipt id and the attested actor supplied by the gateway, policy over spaces, and a denial receipt for every refused write.
|
|
182
198
|
- Forget digests are keyed under a server-held secret, or absent on request, so an erased value cannot be guessed back from the file.
|
|
183
199
|
- Retention windows per space in the server, sweeps that default to them, and a `sweep --via` trigger that runs retention through a gateway as a named principal, on any timer.
|
|
@@ -189,13 +205,13 @@ tsconfig.build.json emits dist/ for consumers; the repo itself runs the .ts dir
|
|
|
189
205
|
- Policy over provenance: the gateway looks up the fact a write supersedes or a retraction targets, so policy decides on its space, actor, and provenance; a claimed fact can be displaced, an attested org fact cannot.
|
|
190
206
|
- Consumed facts and blast radius: the memory server declares the facts it serves, the gateway records them on every later receipt, and `blast` walks from a fact to every downstream call and derived belief, transitively, with its retraction status.
|
|
191
207
|
- Write-through adapters for Mem0 and Zep: every write lands in the store with custody metadata, the store id is recorded on the fact, retractions reach the store, and failures are ordered so nothing is half-recorded.
|
|
208
|
+
- The eval CLI: built-in or custom scenario files, the naive baseline beside the ledger, a signed report a reviewer verifies, and a non-zero exit on regression, for cron.
|
|
192
209
|
- The memory-mutation eval harness: stale reads, contradictions, blast radius, and correct reads over scripted incidents, scored the same way for the ledger and for anything behind the same interface.
|
|
193
210
|
- Quarantine: facts carry `attested` or `claimed` provenance; claimed facts are hidden from reads by default and a gateway-only `memory.confirm` lifts them, as a recorded event.
|
|
194
211
|
|
|
195
212
|
**Next, in the order it pays off**
|
|
196
213
|
|
|
197
|
-
1.
|
|
214
|
+
1. Indexed queries on the SQLite store, so a shared ledger with millions of events answers reads without loading them all. [Issue #8](https://github.com/ch4r10t33r/agent-custody/issues/8).
|
|
198
215
|
2. Write-through adapters for Letta, LangMem, and Cognee, one per user who asks. [Issue #4](https://github.com/ch4r10t33r/agent-custody/issues/4).
|
|
199
|
-
3. The
|
|
200
|
-
4. The hosted plane, behind early access: tenanted log, then memory, then reports and a control plane. [Issue #6](https://github.com/ch4r10t33r/agent-custody/issues/6).
|
|
216
|
+
3. The hosted plane, behind early access: tenanted log, then memory, then reports and a control plane. [Issue #6](https://github.com/ch4r10t33r/agent-custody/issues/6).
|
|
201
217
|
|
package/dist/cli.js
CHANGED
|
@@ -4,7 +4,13 @@ import { Ledger } from "./ledger.js";
|
|
|
4
4
|
import { createMemoryServer, serveStdio } from "./server.js";
|
|
5
5
|
import { blastRadius, formatBlastRadius, loadReceipts } from "./blast.js";
|
|
6
6
|
import { serveMemoryHttp } from "./http.js";
|
|
7
|
-
import { loadPrivateKey } from "@agent-custody/receipts";
|
|
7
|
+
import { loadPrivateKey, loadPublicKey } from "@agent-custody/receipts";
|
|
8
|
+
import { mkdtempSync, readFileSync, writeFileSync } from "node:fs";
|
|
9
|
+
import { tmpdir } from "node:os";
|
|
10
|
+
import { join } from "node:path";
|
|
11
|
+
import { formatReport, runAll, SCENARIOS } from "./evals.js";
|
|
12
|
+
import { ledgerUnderTest, overwriteStoreUnderTest } from "./evals-ledger.js";
|
|
13
|
+
import { loadScenarios, signReport, verifyReport } from "./evals-file.js";
|
|
8
14
|
import { fileURLToPath } from "node:url";
|
|
9
15
|
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
|
10
16
|
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
|
|
@@ -30,7 +36,7 @@ function parseRetention(spec) {
|
|
|
30
36
|
}
|
|
31
37
|
const USAGE = `agent-custody-memory <command>
|
|
32
38
|
|
|
33
|
-
serve --ledger <ledger.jsonl> [--allow-direct] [--key <memory.key>] [--forget-key-env NAME] [--retention 'org=P365D,team:*=P90D']
|
|
39
|
+
serve --ledger <ledger.jsonl|ledger.sqlite> [--allow-direct] [--key <memory.key>] [--forget-key-env NAME] [--retention 'org=P365D,team:*=P90D']
|
|
34
40
|
--forget-key-env names a secret kept outside the ledger; forgotten values then leave an HMAC, not a guessable hash.
|
|
35
41
|
the memory server over stdio; run it as the receipts gateway's upstream.
|
|
36
42
|
--key signs every result for its receipt, so executions verify as attested by this server.
|
|
@@ -42,6 +48,11 @@ const USAGE = `agent-custody-memory <command>
|
|
|
42
48
|
Without --before, the memory server's --retention windows decide. Put this on a timer.
|
|
43
49
|
sweep --ledger <ledger.jsonl> --before <ISO instant> [--space <space>] --reason <text> [--actor <id>] [--forget-key-env NAME] [--no-digest]
|
|
44
50
|
retention on the ledger file alone, for ledgers with no gateway or stores in front of them.
|
|
51
|
+
eval [--scenarios <file.json>] [--baseline] [--json] [--sign <key> --out <report.json>]
|
|
52
|
+
runs the memory-mutation scenarios on a fresh ledger; --baseline also scores a naive
|
|
53
|
+
overwrite store; --sign writes a signed report. Exits 1 if the ledger regresses.
|
|
54
|
+
eval --verify <report.json> --key <pub> checks a signed report and prints its scores
|
|
55
|
+
export --ledger <ledger.sqlite> --out <ledger.jsonl> the auditable JSONL of any ledger, one event per line
|
|
45
56
|
blast --ledger <ledger.jsonl> --receipts <dir> --fact <factId> [--json]
|
|
46
57
|
everything that relied on a fact: later calls, derived beliefs, and whether it was retracted
|
|
47
58
|
`;
|
|
@@ -98,6 +109,53 @@ async function main(argv) {
|
|
|
98
109
|
console.log(` ${f.factId} ${f.digestKind}${f.valueDigest ? ` ${f.valueDigest.slice(0, 12)}` : ""}`);
|
|
99
110
|
return 0;
|
|
100
111
|
}
|
|
112
|
+
case "eval": {
|
|
113
|
+
const { values } = parseArgs({ args: rest, options: { scenarios: { type: "string" }, baseline: { type: "boolean", default: false }, json: { type: "boolean", default: false }, sign: { type: "string" }, out: { type: "string" }, verify: { type: "string" }, key: { type: "string", multiple: true } } });
|
|
114
|
+
if (values.verify) {
|
|
115
|
+
if (!values.key?.length)
|
|
116
|
+
throw new Error("eval --verify needs --key <pub>");
|
|
117
|
+
const r = verifyReport(JSON.parse(readFileSync(values.verify, "utf8")), values.key.map(loadPublicKey));
|
|
118
|
+
if (!r.ok) {
|
|
119
|
+
console.log(`NOT VERIFIED: ${r.error}`);
|
|
120
|
+
return 1;
|
|
121
|
+
}
|
|
122
|
+
console.log(`VERIFIED signed by ${r.keyid.slice(0, 12)} system ${r.predicate.system} ran ${r.predicate.ranAt} scenarios ${r.predicate.scenarioNames.length} (digest ${r.predicate.scenariosDigest.slice(0, 12)})`);
|
|
123
|
+
console.log(formatReport(r.predicate.report));
|
|
124
|
+
return 0;
|
|
125
|
+
}
|
|
126
|
+
const scenarios = values.scenarios ? loadScenarios(values.scenarios) : SCENARIOS;
|
|
127
|
+
const ledger = new Ledger(join(mkdtempSync(join(tmpdir(), "agent-custody-eval-")), "ledger.jsonl"));
|
|
128
|
+
const report = await runAll(ledgerUnderTest(ledger), scenarios);
|
|
129
|
+
const baseline = values.baseline ? await runAll(overwriteStoreUnderTest(), scenarios) : null;
|
|
130
|
+
const regressed = report.totals.correctReads < report.totals.reads || report.scenarios.some((s) => s.failures.length > 0);
|
|
131
|
+
if (values.json)
|
|
132
|
+
console.log(JSON.stringify({ ledger: report, baseline }, null, 2));
|
|
133
|
+
else {
|
|
134
|
+
console.log("The ledger:");
|
|
135
|
+
console.log(formatReport(report));
|
|
136
|
+
if (baseline) {
|
|
137
|
+
console.log("\nA naive overwrite store:");
|
|
138
|
+
console.log(formatReport(baseline));
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
if (values.sign) {
|
|
142
|
+
if (!values.out)
|
|
143
|
+
throw new Error("eval --sign needs --out <report.json>");
|
|
144
|
+
writeFileSync(values.out, JSON.stringify(signReport("agent-custody-ledger", scenarios, report, loadPrivateKey(values.sign)), null, 2));
|
|
145
|
+
console.error(`signed report written to ${values.out}`);
|
|
146
|
+
}
|
|
147
|
+
return regressed ? 1 : 0;
|
|
148
|
+
}
|
|
149
|
+
case "export": {
|
|
150
|
+
const { values } = parseArgs({ args: rest, options: { ledger: { type: "string" }, out: { type: "string" } } });
|
|
151
|
+
if (!values.ledger || !values.out)
|
|
152
|
+
throw new Error("export needs --ledger and --out");
|
|
153
|
+
const l = new Ledger(values.ledger);
|
|
154
|
+
writeFileSync(values.out, l.export().map((e) => JSON.stringify(e)).join("\n") + "\n");
|
|
155
|
+
console.log(`exported ${l.size} event(s) from ${l.location} to ${values.out}`);
|
|
156
|
+
l.close();
|
|
157
|
+
return 0;
|
|
158
|
+
}
|
|
101
159
|
case "blast": {
|
|
102
160
|
const { values } = parseArgs({ args: rest, options: { ledger: { type: "string" }, receipts: { type: "string" }, fact: { type: "string" }, json: { type: "boolean", default: false } } });
|
|
103
161
|
if (!values.ledger || !values.receipts || !values.fact)
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
import { z } from "zod";
|
|
2
|
+
import { type Envelope, type KeyPair, type PublicKeyRef } from "@agent-custody/receipts";
|
|
3
|
+
import type { Report, Scenario } from "./evals.ts";
|
|
4
|
+
export declare const EVAL_REPORT_TYPE = "https://agent-custody.dev/eval-report/v0.1";
|
|
5
|
+
export declare const ScenarioFileSchema: z.ZodObject<{
|
|
6
|
+
version: z.ZodLiteral<"0.1">;
|
|
7
|
+
scenarios: z.ZodArray<z.ZodObject<{
|
|
8
|
+
name: z.ZodString;
|
|
9
|
+
ops: z.ZodArray<z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
10
|
+
op: z.ZodLiteral<"write">;
|
|
11
|
+
key: z.ZodString;
|
|
12
|
+
subject: z.ZodString;
|
|
13
|
+
predicate: z.ZodString;
|
|
14
|
+
value: z.ZodUnknown;
|
|
15
|
+
space: z.ZodOptional<z.ZodString>;
|
|
16
|
+
actor: z.ZodOptional<z.ZodString>;
|
|
17
|
+
supersedes: z.ZodOptional<z.ZodString>;
|
|
18
|
+
bad: z.ZodOptional<z.ZodBoolean>;
|
|
19
|
+
}, z.core.$strip>, z.ZodObject<{
|
|
20
|
+
op: z.ZodLiteral<"read">;
|
|
21
|
+
subject: z.ZodString;
|
|
22
|
+
predicate: z.ZodOptional<z.ZodString>;
|
|
23
|
+
space: z.ZodOptional<z.ZodString>;
|
|
24
|
+
expect: z.ZodUnknown;
|
|
25
|
+
}, z.core.$strip>, z.ZodObject<{
|
|
26
|
+
op: z.ZodLiteral<"retract">;
|
|
27
|
+
key: z.ZodString;
|
|
28
|
+
actor: z.ZodOptional<z.ZodString>;
|
|
29
|
+
reason: z.ZodOptional<z.ZodString>;
|
|
30
|
+
}, z.core.$strip>], "op">>;
|
|
31
|
+
}, z.core.$strip>>;
|
|
32
|
+
}, z.core.$strip>;
|
|
33
|
+
export type ScenarioFile = z.infer<typeof ScenarioFileSchema>;
|
|
34
|
+
/** Loads and validates a scenario file. Every read must carry `expect`, null meaning "nothing". */
|
|
35
|
+
export declare function loadScenarios(path: string): Scenario[];
|
|
36
|
+
export interface EvalReportPredicate {
|
|
37
|
+
system: string;
|
|
38
|
+
scenariosDigest: string;
|
|
39
|
+
scenarioNames: string[];
|
|
40
|
+
report: Report;
|
|
41
|
+
ranAt: string;
|
|
42
|
+
}
|
|
43
|
+
/** An in-toto statement over the report, signed with the given key: the artefact a reviewer verifies. */
|
|
44
|
+
export declare function signReport(system: string, scenarios: Scenario[], report: Report, key: KeyPair): Envelope;
|
|
45
|
+
export type ReportCheck = {
|
|
46
|
+
ok: true;
|
|
47
|
+
predicate: EvalReportPredicate;
|
|
48
|
+
keyid: string;
|
|
49
|
+
} | {
|
|
50
|
+
ok: false;
|
|
51
|
+
error: string;
|
|
52
|
+
};
|
|
53
|
+
export declare function verifyReport(envelope: Envelope, keys: PublicKeyRef[]): ReportCheck;
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
// Scenario files and signed eval reports. A team encodes its own memory incidents as scenarios, runs them on a
|
|
2
|
+
// schedule, and hands a reviewer a report signed with a key, verifiable with the same envelope format as receipts.
|
|
3
|
+
import { readFileSync } from "node:fs";
|
|
4
|
+
import { z } from "zod";
|
|
5
|
+
import { digestOf, dsseSign, dsseVerify } from "@agent-custody/receipts";
|
|
6
|
+
export const EVAL_REPORT_TYPE = "https://agent-custody.dev/eval-report/v0.1";
|
|
7
|
+
const OpSchema = z.discriminatedUnion("op", [
|
|
8
|
+
z.object({ op: z.literal("write"), key: z.string().min(1), subject: z.string().min(1), predicate: z.string().min(1), value: z.unknown(), space: z.string().min(1).optional(), actor: z.string().min(1).optional(), supersedes: z.string().min(1).optional(), bad: z.boolean().optional() }),
|
|
9
|
+
z.object({ op: z.literal("read"), subject: z.string().min(1), predicate: z.string().min(1).optional(), space: z.string().min(1).optional(), expect: z.unknown() }),
|
|
10
|
+
z.object({ op: z.literal("retract"), key: z.string().min(1), actor: z.string().min(1).optional(), reason: z.string().min(1).optional() }),
|
|
11
|
+
]);
|
|
12
|
+
export const ScenarioFileSchema = z.object({
|
|
13
|
+
version: z.literal("0.1"),
|
|
14
|
+
scenarios: z.array(z.object({ name: z.string().min(1), ops: z.array(OpSchema).min(1) })).min(1),
|
|
15
|
+
});
|
|
16
|
+
/** Loads and validates a scenario file. Every read must carry `expect`, null meaning "nothing". */
|
|
17
|
+
export function loadScenarios(path) {
|
|
18
|
+
const parsed = ScenarioFileSchema.safeParse(JSON.parse(readFileSync(path, "utf8")));
|
|
19
|
+
if (!parsed.success)
|
|
20
|
+
throw new Error(`scenario file ${path}: ${parsed.error.issues.map((i) => `${i.path.join(".") || "(root)"}: ${i.message}`).join("; ")}`);
|
|
21
|
+
for (const [si, s] of parsed.data.scenarios.entries())
|
|
22
|
+
for (const [oi, o] of s.ops.entries())
|
|
23
|
+
if (o.op === "read" && !("expect" in o))
|
|
24
|
+
throw new Error(`scenario file ${path}: scenarios.${si}.ops.${oi}: a read needs expect (null for nothing)`);
|
|
25
|
+
return parsed.data.scenarios;
|
|
26
|
+
}
|
|
27
|
+
/** An in-toto statement over the report, signed with the given key: the artefact a reviewer verifies. */
|
|
28
|
+
export function signReport(system, scenarios, report, key) {
|
|
29
|
+
const predicate = { system, scenariosDigest: digestOf(scenarios), scenarioNames: scenarios.map((s) => s.name), report, ranAt: new Date().toISOString() };
|
|
30
|
+
const statement = { _type: "https://in-toto.io/Statement/v1", subject: [{ name: `eval:${system}`, digest: { sha256: digestOf(report) } }], predicateType: EVAL_REPORT_TYPE, predicate };
|
|
31
|
+
return dsseSign("application/vnd.in-toto+json", statement, key);
|
|
32
|
+
}
|
|
33
|
+
export function verifyReport(envelope, keys) {
|
|
34
|
+
const v = dsseVerify(envelope, keys);
|
|
35
|
+
if (!v.ok)
|
|
36
|
+
return { ok: false, error: v.error };
|
|
37
|
+
const st = v.payload;
|
|
38
|
+
if (st.predicateType !== EVAL_REPORT_TYPE || !st.predicate)
|
|
39
|
+
return { ok: false, error: `not an eval report: ${String(st.predicateType)}` };
|
|
40
|
+
if (st.subject?.[0]?.digest?.sha256 !== digestOf(st.predicate.report))
|
|
41
|
+
return { ok: false, error: "report digest does not match the subject" };
|
|
42
|
+
return { ok: true, predicate: st.predicate, keyid: v.keyid };
|
|
43
|
+
}
|
package/dist/index.d.ts
CHANGED
|
@@ -1,10 +1,14 @@
|
|
|
1
1
|
export { Ledger } from "./ledger.ts";
|
|
2
|
+
export { JsonlStore, SqliteStore, openStore } from "./storage.ts";
|
|
3
|
+
export type { EventStore } from "./storage.ts";
|
|
2
4
|
export type { AsOf, AssertEvent, AssertInput, ConfirmEvent, ConfirmInput, Fact, FactProvenance, DigestKind, ForgetEvent, ForgetInput, HoldEvent, HoldInput, LedgerEvent, SweepInput, RetractEvent, RetractInput, Source } from "./ledger.ts";
|
|
3
5
|
export { AGENT_META_KEY, FACTS_META_KEY, OBSERVED_META_KEY, RECEIPT_META_KEY, SERVER_VERSION, TOOLS, createMemoryServer, durationMs, retentionCutoff, serveStdio } from "./server.ts";
|
|
4
6
|
export type { MemoryServerOptions } from "./server.ts";
|
|
5
7
|
export { SCENARIOS, formatReport, runAll, runScenario } from "./evals.ts";
|
|
6
8
|
export type { MemoryUnderTest, Op, Report, Scenario, ScenarioScore } from "./evals.ts";
|
|
7
9
|
export { ledgerUnderTest, overwriteStoreUnderTest } from "./evals-ledger.ts";
|
|
10
|
+
export { EVAL_REPORT_TYPE, ScenarioFileSchema, loadScenarios, signReport, verifyReport } from "./evals-file.ts";
|
|
11
|
+
export type { EvalReportPredicate, ReportCheck, ScenarioFile } from "./evals-file.ts";
|
|
8
12
|
export { factMetadata, factText, mem0Store, zepStore } from "./stores.ts";
|
|
9
13
|
export { blastRadius, formatBlastRadius, loadReceipts } from "./blast.ts";
|
|
10
14
|
export { memoryHttpHandler, serveMemoryHttp } from "./http.ts";
|
package/dist/index.js
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
// Public surface of @agent-custody/state.
|
|
2
2
|
export { Ledger } from "./ledger.js";
|
|
3
|
+
export { JsonlStore, SqliteStore, openStore } from "./storage.js";
|
|
3
4
|
export { AGENT_META_KEY, FACTS_META_KEY, OBSERVED_META_KEY, RECEIPT_META_KEY, SERVER_VERSION, TOOLS, createMemoryServer, durationMs, retentionCutoff, serveStdio } from "./server.js";
|
|
4
5
|
export { SCENARIOS, formatReport, runAll, runScenario } from "./evals.js";
|
|
5
6
|
export { ledgerUnderTest, overwriteStoreUnderTest } from "./evals-ledger.js";
|
|
7
|
+
export { EVAL_REPORT_TYPE, ScenarioFileSchema, loadScenarios, signReport, verifyReport } from "./evals-file.js";
|
|
6
8
|
export { factMetadata, factText, mem0Store, zepStore } from "./stores.js";
|
|
7
9
|
export { blastRadius, formatBlastRadius, loadReceipts } from "./blast.js";
|
|
8
10
|
export { memoryHttpHandler, serveMemoryHttp } from "./http.js";
|
package/dist/ledger.d.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { type EventStore } from "./storage.ts";
|
|
1
2
|
/** Where a write came from. A receipt id means the write went through a receipts producer and can be verified there. */
|
|
2
3
|
export interface Source {
|
|
3
4
|
receiptId: string | null;
|
|
@@ -155,14 +156,22 @@ export interface AsOf {
|
|
|
155
156
|
}
|
|
156
157
|
export declare class Ledger {
|
|
157
158
|
private readonly events;
|
|
158
|
-
private readonly
|
|
159
|
+
private readonly store;
|
|
159
160
|
private readonly now;
|
|
160
161
|
private readonly forgetKey;
|
|
161
|
-
/**
|
|
162
|
-
|
|
162
|
+
/**
|
|
163
|
+
* `location` is a path: JSONL by default, SQLite when it ends in .sqlite or .db; or pass a store.
|
|
164
|
+
* forgetKey: a secret kept outside the store; with it, forgotten values leave an HMAC rather than a plain hash.
|
|
165
|
+
*/
|
|
166
|
+
constructor(location: string | EventStore, opts?: {
|
|
163
167
|
now?: () => Date;
|
|
164
168
|
forgetKey?: string | Buffer;
|
|
165
169
|
});
|
|
170
|
+
/** Where the events live, for reports. */
|
|
171
|
+
get location(): string;
|
|
172
|
+
/** Every event in order, for export. */
|
|
173
|
+
export(): LedgerEvent[];
|
|
174
|
+
close(): void;
|
|
166
175
|
get size(): number;
|
|
167
176
|
/** The checks assert makes, without appending. For callers that must do something irreversible before the append. */
|
|
168
177
|
validateAssert(input: AssertInput): void;
|
package/dist/ledger.js
CHANGED
|
@@ -3,28 +3,33 @@
|
|
|
3
3
|
// Nothing is ever edited in place. Correcting a belief is a new event, so "what did the agent believe at T" is always answerable.
|
|
4
4
|
import { randomUUID } from "node:crypto";
|
|
5
5
|
import { createHash, createHmac } from "node:crypto";
|
|
6
|
-
import {
|
|
7
|
-
import { dirname } from "node:path";
|
|
6
|
+
import { openStore } from "./storage.js";
|
|
8
7
|
const RANK = { claimed: 0, attested: 1, verified: 2 };
|
|
9
8
|
export class Ledger {
|
|
10
|
-
events
|
|
11
|
-
|
|
9
|
+
events;
|
|
10
|
+
store;
|
|
12
11
|
now;
|
|
13
12
|
forgetKey;
|
|
14
|
-
/**
|
|
15
|
-
|
|
16
|
-
|
|
13
|
+
/**
|
|
14
|
+
* `location` is a path: JSONL by default, SQLite when it ends in .sqlite or .db; or pass a store.
|
|
15
|
+
* forgetKey: a secret kept outside the store; with it, forgotten values leave an HMAC rather than a plain hash.
|
|
16
|
+
*/
|
|
17
|
+
constructor(location, opts = {}) {
|
|
18
|
+
this.store = typeof location === "string" ? openStore(location) : location;
|
|
17
19
|
this.now = opts.now ?? (() => new Date());
|
|
18
20
|
this.forgetKey = opts.forgetKey ? Buffer.from(opts.forgetKey) : null;
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
21
|
+
this.events = this.store.load();
|
|
22
|
+
}
|
|
23
|
+
/** Where the events live, for reports. */
|
|
24
|
+
get location() {
|
|
25
|
+
return this.store.location;
|
|
26
|
+
}
|
|
27
|
+
/** Every event in order, for export. */
|
|
28
|
+
export() {
|
|
29
|
+
return [...this.events];
|
|
30
|
+
}
|
|
31
|
+
close() {
|
|
32
|
+
this.store.close();
|
|
28
33
|
}
|
|
29
34
|
get size() {
|
|
30
35
|
return this.events.length;
|
|
@@ -162,13 +167,11 @@ export class Ledger {
|
|
|
162
167
|
if (e.kind === "assert" && e.fact.factId === input.factId) {
|
|
163
168
|
e.fact.value = null;
|
|
164
169
|
e.fact.forgotten = { valueDigest, digestKind, at: txTime };
|
|
170
|
+
this.store.replaceAssert(e);
|
|
165
171
|
}
|
|
166
172
|
}
|
|
167
173
|
const event = { eventId: randomUUID(), kind: "forget", txTime, factId: input.factId, actor: input.actor, reason: input.reason, source: input.source ?? { receiptId: null }, valueDigest, digestKind };
|
|
168
|
-
this.
|
|
169
|
-
const tmp = `${this.file}.tmp`;
|
|
170
|
-
writeFileSync(tmp, this.events.map((e) => JSON.stringify(e)).join("\n") + "\n");
|
|
171
|
-
renameSync(tmp, this.file);
|
|
174
|
+
this.append(event);
|
|
172
175
|
return event;
|
|
173
176
|
}
|
|
174
177
|
/** The facts believed at a moment. Valid time answers "was it true then"; transaction time answers "did the ledger know it then". */
|
|
@@ -228,7 +231,7 @@ export class Ledger {
|
|
|
228
231
|
return this.events.some((e) => (e.kind === "retract" || e.kind === "forget") && e.factId === factId);
|
|
229
232
|
}
|
|
230
233
|
append(event) {
|
|
231
|
-
|
|
234
|
+
this.store.append(event);
|
|
232
235
|
this.events.push(event);
|
|
233
236
|
}
|
|
234
237
|
}
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
import type { AssertEvent, LedgerEvent } from "./ledger.ts";
|
|
2
|
+
export interface EventStore {
|
|
3
|
+
readonly kind: "jsonl" | "sqlite";
|
|
4
|
+
readonly location: string;
|
|
5
|
+
/** every event, in order */
|
|
6
|
+
load(): LedgerEvent[];
|
|
7
|
+
append(event: LedgerEvent): void;
|
|
8
|
+
/** replaces one assert event in place, for forget: the value is gone from the store, not merely superseded */
|
|
9
|
+
replaceAssert(event: AssertEvent): void;
|
|
10
|
+
close(): void;
|
|
11
|
+
}
|
|
12
|
+
export declare class JsonlStore implements EventStore {
|
|
13
|
+
readonly kind: "jsonl";
|
|
14
|
+
readonly location: string;
|
|
15
|
+
constructor(file: string);
|
|
16
|
+
load(): LedgerEvent[];
|
|
17
|
+
append(event: LedgerEvent): void;
|
|
18
|
+
replaceAssert(event: AssertEvent): void;
|
|
19
|
+
close(): void;
|
|
20
|
+
}
|
|
21
|
+
export declare class SqliteStore implements EventStore {
|
|
22
|
+
readonly kind: "sqlite";
|
|
23
|
+
readonly location: string;
|
|
24
|
+
private readonly db;
|
|
25
|
+
constructor(file: string);
|
|
26
|
+
load(): LedgerEvent[];
|
|
27
|
+
append(event: LedgerEvent): void;
|
|
28
|
+
replaceAssert(event: AssertEvent): void;
|
|
29
|
+
close(): void;
|
|
30
|
+
}
|
|
31
|
+
/** JSONL unless the path ends in .sqlite or .db. */
|
|
32
|
+
export declare function openStore(location: string): EventStore;
|
package/dist/storage.js
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
// Where the ledger's events live. The ledger keeps every event in memory for its queries; a store makes them durable.
|
|
2
|
+
// JSONL is the default and the auditable artefact: one event per line, readable by anyone, copied for an audit.
|
|
3
|
+
// SQLite is for durability and shared use: transactional writes, write-ahead logging, an in-place forget that leaves
|
|
4
|
+
// no copy of the value behind, and a file more than one process can open. Node ships the SQLite module; no native
|
|
5
|
+
// dependency.
|
|
6
|
+
import { appendFileSync, existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
|
7
|
+
import { createRequire } from "node:module";
|
|
8
|
+
import { dirname } from "node:path";
|
|
9
|
+
export class JsonlStore {
|
|
10
|
+
kind = "jsonl";
|
|
11
|
+
location;
|
|
12
|
+
constructor(file) {
|
|
13
|
+
this.location = file;
|
|
14
|
+
if (!existsSync(file))
|
|
15
|
+
mkdirSync(dirname(file), { recursive: true });
|
|
16
|
+
}
|
|
17
|
+
load() {
|
|
18
|
+
if (!existsSync(this.location))
|
|
19
|
+
return [];
|
|
20
|
+
return readFileSync(this.location, "utf8").split("\n").filter((l) => l.trim()).map((l) => JSON.parse(l));
|
|
21
|
+
}
|
|
22
|
+
append(event) {
|
|
23
|
+
appendFileSync(this.location, JSON.stringify(event) + "\n");
|
|
24
|
+
}
|
|
25
|
+
replaceAssert(event) {
|
|
26
|
+
const all = this.load().map((e) => (e.kind === "assert" && e.eventId === event.eventId ? event : e));
|
|
27
|
+
const tmp = `${this.location}.tmp`;
|
|
28
|
+
writeFileSync(tmp, all.map((e) => JSON.stringify(e)).join("\n") + "\n");
|
|
29
|
+
renameSync(tmp, this.location);
|
|
30
|
+
}
|
|
31
|
+
close() { }
|
|
32
|
+
}
|
|
33
|
+
export class SqliteStore {
|
|
34
|
+
kind = "sqlite";
|
|
35
|
+
location;
|
|
36
|
+
db;
|
|
37
|
+
constructor(file) {
|
|
38
|
+
this.location = file;
|
|
39
|
+
mkdirSync(dirname(file), { recursive: true });
|
|
40
|
+
// Loaded on demand so a JSONL ledger never touches the SQLite module, which warns on Node 22 that it is experimental.
|
|
41
|
+
const { DatabaseSync } = createRequire(import.meta.url)("node:sqlite");
|
|
42
|
+
this.db = new DatabaseSync(file);
|
|
43
|
+
// secure_delete overwrites removed content with zeros, so a forgotten value does not linger in freed page space.
|
|
44
|
+
this.db.exec("PRAGMA journal_mode = WAL; PRAGMA synchronous = FULL; PRAGMA secure_delete = ON;");
|
|
45
|
+
this.db.exec("CREATE TABLE IF NOT EXISTS events (seq INTEGER PRIMARY KEY AUTOINCREMENT, event_id TEXT NOT NULL UNIQUE, kind TEXT NOT NULL, tx_time TEXT NOT NULL, fact_id TEXT NOT NULL, json TEXT NOT NULL)");
|
|
46
|
+
this.db.exec("CREATE INDEX IF NOT EXISTS events_fact ON events(fact_id); CREATE INDEX IF NOT EXISTS events_tx ON events(tx_time)");
|
|
47
|
+
}
|
|
48
|
+
load() {
|
|
49
|
+
return this.db.prepare("SELECT json FROM events ORDER BY seq").all().map((r) => JSON.parse(r.json));
|
|
50
|
+
}
|
|
51
|
+
append(event) {
|
|
52
|
+
const factId = event.kind === "assert" ? event.fact.factId : event.factId;
|
|
53
|
+
this.db.prepare("INSERT INTO events (event_id, kind, tx_time, fact_id, json) VALUES (?, ?, ?, ?, ?)").run(event.eventId, event.kind, event.txTime, factId, JSON.stringify(event));
|
|
54
|
+
}
|
|
55
|
+
replaceAssert(event) {
|
|
56
|
+
this.db.prepare("UPDATE events SET json = ? WHERE event_id = ?").run(JSON.stringify(event), event.eventId);
|
|
57
|
+
// The old row image would otherwise linger in the write-ahead log; a truncating checkpoint removes it.
|
|
58
|
+
this.db.exec("PRAGMA wal_checkpoint(TRUNCATE)");
|
|
59
|
+
}
|
|
60
|
+
close() {
|
|
61
|
+
this.db.close();
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
/** JSONL unless the path ends in .sqlite or .db. */
|
|
65
|
+
export function openStore(location) {
|
|
66
|
+
return /\.(sqlite|db)$/i.test(location) ? new SqliteStore(location) : new JsonlStore(location);
|
|
67
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agent-custody/state",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.8",
|
|
4
4
|
"description": "Governed memory for AI agents: a fact ledger with provenance, valid time, rollback, and lineage, built on @agent-custody/receipts",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"repository": {
|
|
@@ -33,7 +33,7 @@
|
|
|
33
33
|
"node": ">=22"
|
|
34
34
|
},
|
|
35
35
|
"dependencies": {
|
|
36
|
-
"@agent-custody/receipts": "0.1.
|
|
36
|
+
"@agent-custody/receipts": "0.1.8",
|
|
37
37
|
"@modelcontextprotocol/sdk": "^1.30.0",
|
|
38
38
|
"zod": "^4.5.4"
|
|
39
39
|
},
|