audrey 1.0.3 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +30 -0
- package/README.md +301 -435
- package/SECURITY.md +3 -3
- package/benchmarks/output/adapter-self-test/guardbench-adapter-self-test.json +2 -2
- package/benchmarks/output/external/guardbench-external-dry-run.json +1 -1
- package/benchmarks/output/external/guardbench-external-evidence.json +1 -1
- package/benchmarks/output/guardbench-conformance-card.json +12 -12
- package/benchmarks/output/guardbench-raw.json +106 -106
- package/benchmarks/output/guardbench-summary.json +168 -168
- package/benchmarks/output/leaderboard/guardbench-leaderboard.json +5 -5
- package/benchmarks/output/leaderboard/guardbench-leaderboard.md +2 -2
- package/benchmarks/output/submission-bundle/guardbench-conformance-card.json +12 -12
- package/benchmarks/output/submission-bundle/guardbench-raw.json +106 -106
- package/benchmarks/output/submission-bundle/guardbench-summary.json +168 -168
- package/benchmarks/output/submission-bundle/submission-manifest.json +11 -11
- package/benchmarks/output/submission-bundle/validation-report.json +1 -1
- package/benchmarks/output/summary.json +57 -57
- package/benchmarks/public-paths.mjs +14 -1
- package/dist/mcp-server/config.d.ts +4 -1
- package/dist/mcp-server/config.d.ts.map +1 -1
- package/dist/mcp-server/config.js +47 -7
- package/dist/mcp-server/config.js.map +1 -1
- package/dist/mcp-server/hooks.d.ts +41 -0
- package/dist/mcp-server/hooks.d.ts.map +1 -0
- package/dist/mcp-server/hooks.js +313 -0
- package/dist/mcp-server/hooks.js.map +1 -0
- package/dist/mcp-server/index.d.ts +5 -10
- package/dist/mcp-server/index.d.ts.map +1 -1
- package/dist/mcp-server/index.js +588 -286
- package/dist/mcp-server/index.js.map +1 -1
- package/dist/src/action-key.d.ts +1 -0
- package/dist/src/action-key.d.ts.map +1 -1
- package/dist/src/action-key.js +11 -13
- package/dist/src/action-key.js.map +1 -1
- package/dist/src/affect.d.ts +1 -0
- package/dist/src/affect.d.ts.map +1 -1
- package/dist/src/affect.js +9 -4
- package/dist/src/affect.js.map +1 -1
- package/dist/src/audrey.d.ts +11 -3
- package/dist/src/audrey.d.ts.map +1 -1
- package/dist/src/audrey.js +59 -25
- package/dist/src/audrey.js.map +1 -1
- package/dist/src/autopilot.d.ts +32 -0
- package/dist/src/autopilot.d.ts.map +1 -0
- package/dist/src/autopilot.js +997 -0
- package/dist/src/autopilot.js.map +1 -0
- package/dist/src/capsule.d.ts +2 -0
- package/dist/src/capsule.d.ts.map +1 -1
- package/dist/src/capsule.js +27 -12
- package/dist/src/capsule.js.map +1 -1
- package/dist/src/consolidate.d.ts +1 -1
- package/dist/src/consolidate.d.ts.map +1 -1
- package/dist/src/consolidate.js +14 -8
- package/dist/src/consolidate.js.map +1 -1
- package/dist/src/controller.d.ts +2 -0
- package/dist/src/controller.d.ts.map +1 -1
- package/dist/src/controller.js +18 -11
- package/dist/src/controller.js.map +1 -1
- package/dist/src/db.d.ts.map +1 -1
- package/dist/src/db.js +121 -33
- package/dist/src/db.js.map +1 -1
- package/dist/src/decay.d.ts +2 -1
- package/dist/src/decay.d.ts.map +1 -1
- package/dist/src/decay.js +14 -10
- package/dist/src/decay.js.map +1 -1
- package/dist/src/embedding.d.ts.map +1 -1
- package/dist/src/embedding.js.map +1 -1
- package/dist/src/encode.d.ts.map +1 -1
- package/dist/src/encode.js +4 -2
- package/dist/src/encode.js.map +1 -1
- package/dist/src/events.d.ts +2 -0
- package/dist/src/events.d.ts.map +1 -1
- package/dist/src/events.js +15 -1
- package/dist/src/events.js.map +1 -1
- package/dist/src/feedback.d.ts +1 -0
- package/dist/src/feedback.d.ts.map +1 -1
- package/dist/src/feedback.js +13 -9
- package/dist/src/feedback.js.map +1 -1
- package/dist/src/forget.d.ts.map +1 -1
- package/dist/src/forget.js.map +1 -1
- package/dist/src/impact.d.ts +1 -1
- package/dist/src/impact.d.ts.map +1 -1
- package/dist/src/impact.js +40 -30
- package/dist/src/impact.js.map +1 -1
- package/dist/src/import.d.ts.map +1 -1
- package/dist/src/import.js +13 -9
- package/dist/src/import.js.map +1 -1
- package/dist/src/index.d.ts +2 -0
- package/dist/src/index.d.ts.map +1 -1
- package/dist/src/index.js +1 -0
- package/dist/src/index.js.map +1 -1
- package/dist/src/interference.d.ts +2 -1
- package/dist/src/interference.d.ts.map +1 -1
- package/dist/src/interference.js +44 -17
- package/dist/src/interference.js.map +1 -1
- package/dist/src/llm.d.ts.map +1 -1
- package/dist/src/llm.js +12 -1
- package/dist/src/llm.js.map +1 -1
- package/dist/src/migrate.d.ts.map +1 -1
- package/dist/src/migrate.js +9 -9
- package/dist/src/migrate.js.map +1 -1
- package/dist/src/preflight.d.ts +2 -0
- package/dist/src/preflight.d.ts.map +1 -1
- package/dist/src/preflight.js +29 -12
- package/dist/src/preflight.js.map +1 -1
- package/dist/src/promote.d.ts +1 -0
- package/dist/src/promote.d.ts.map +1 -1
- package/dist/src/promote.js +12 -8
- package/dist/src/promote.js.map +1 -1
- package/dist/src/recall.d.ts.map +1 -1
- package/dist/src/recall.js +152 -100
- package/dist/src/recall.js.map +1 -1
- package/dist/src/reflexes.d.ts.map +1 -1
- package/dist/src/reflexes.js.map +1 -1
- package/dist/src/rollback.d.ts.map +1 -1
- package/dist/src/rollback.js.map +1 -1
- package/dist/src/routes.d.ts +1 -0
- package/dist/src/routes.d.ts.map +1 -1
- package/dist/src/routes.js +73 -20
- package/dist/src/routes.js.map +1 -1
- package/dist/src/server.d.ts +1 -0
- package/dist/src/server.d.ts.map +1 -1
- package/dist/src/server.js +2 -2
- package/dist/src/server.js.map +1 -1
- package/dist/src/types.d.ts +1 -0
- package/dist/src/types.d.ts.map +1 -1
- package/dist/src/utils.d.ts +2 -0
- package/dist/src/utils.d.ts.map +1 -1
- package/dist/src/utils.js +18 -0
- package/dist/src/utils.js.map +1 -1
- package/dist/src/validate.d.ts +1 -0
- package/dist/src/validate.d.ts.map +1 -1
- package/dist/src/validate.js +19 -10
- package/dist/src/validate.js.map +1 -1
- package/docs/AUDREY_PAPER_OUTLINE.md +6 -3
- package/docs/PRODUCTION_BACKLOG.md +47 -30
- package/docs/paper/06-implementation.md +5 -3
- package/docs/paper/07-evaluation.md +5 -5
- package/docs/paper/08-discussion-limitations.md +1 -1
- package/docs/paper/audrey-paper-v1.md +11 -9
- package/docs/paper/evidence-ledger.md +8 -8
- package/docs/paper/output/arxiv/arxiv-manifest.json +4 -4
- package/docs/paper/output/arxiv/main.tex +11 -9
- package/docs/paper/output/arxiv-compile-report.json +3 -3
- package/docs/paper/output/submission-bundle/README.md +301 -435
- package/docs/paper/output/submission-bundle/benchmarks/output/adapter-self-test/guardbench-adapter-self-test.json +2 -2
- package/docs/paper/output/submission-bundle/benchmarks/output/external/guardbench-external-dry-run.json +1 -1
- package/docs/paper/output/submission-bundle/benchmarks/output/external/guardbench-external-evidence.json +1 -1
- package/docs/paper/output/submission-bundle/benchmarks/output/guardbench-conformance-card.json +12 -12
- package/docs/paper/output/submission-bundle/benchmarks/output/guardbench-raw.json +106 -106
- package/docs/paper/output/submission-bundle/benchmarks/output/guardbench-summary.json +168 -168
- package/docs/paper/output/submission-bundle/benchmarks/output/leaderboard/guardbench-leaderboard.json +5 -5
- package/docs/paper/output/submission-bundle/benchmarks/output/leaderboard/guardbench-leaderboard.md +2 -2
- package/docs/paper/output/submission-bundle/benchmarks/output/submission-bundle/submission-manifest.json +11 -11
- package/docs/paper/output/submission-bundle/benchmarks/output/submission-bundle/validation-report.json +1 -1
- package/docs/paper/output/submission-bundle/benchmarks/output/summary.json +52 -52
- package/docs/paper/output/submission-bundle/docs/AUDREY_PAPER_OUTLINE.md +6 -3
- package/docs/paper/output/submission-bundle/docs/paper/06-implementation.md +5 -3
- package/docs/paper/output/submission-bundle/docs/paper/07-evaluation.md +5 -5
- package/docs/paper/output/submission-bundle/docs/paper/08-discussion-limitations.md +1 -1
- package/docs/paper/output/submission-bundle/docs/paper/audrey-paper-v1.md +11 -9
- package/docs/paper/output/submission-bundle/docs/paper/evidence-ledger.md +8 -8
- package/docs/paper/output/submission-bundle/docs/paper/output/arxiv/arxiv-manifest.json +4 -4
- package/docs/paper/output/submission-bundle/docs/paper/output/arxiv/main.tex +11 -9
- package/docs/paper/output/submission-bundle/docs/paper/output/arxiv-compile-report.json +3 -3
- package/docs/paper/output/submission-bundle/package.json +5 -2
- package/docs/paper/output/submission-bundle/paper-submission-manifest.json +41 -41
- package/package.json +5 -2
- package/scripts/audit-release-completion.mjs +8 -8
- package/scripts/finalize-release.mjs +1 -1
- package/scripts/prepare-release-cut.mjs +1 -1
- package/scripts/publish-release-bundle.mjs +8 -4
- package/scripts/publish-release-github-api.mjs +1 -1
- package/scripts/sync-paper-artifacts.mjs +4 -5
- package/scripts/verify-release-readiness.mjs +1 -1
package/README.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
<div align="center">
|
|
2
|
-
<img src="docs/assets/audrey-wordmark.png" alt="Audrey
|
|
2
|
+
<img src="docs/assets/audrey-wordmark.png" alt="Audrey" width="720">
|
|
3
3
|
|
|
4
|
-
<p><strong>
|
|
4
|
+
<p><strong>Memory that shows up before your coding agent makes the same mistake twice.</strong></p>
|
|
5
5
|
|
|
6
6
|
<p>
|
|
7
|
-
|
|
8
|
-
|
|
7
|
+
Audrey gives Codex and Claude Code one local, evidence-backed memory loop:
|
|
8
|
+
remember what mattered, recall it automatically, check before acting, and learn from what happened next.
|
|
9
9
|
</p>
|
|
10
10
|
|
|
11
11
|
<p>
|
|
@@ -15,551 +15,417 @@
|
|
|
15
15
|
</p>
|
|
16
16
|
</div>
|
|
17
17
|
|
|
18
|
-
##
|
|
18
|
+
## Your agent should remember the work, not just the chat
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
You fix the deploy command on Monday. On Thursday, a fresh session tries the broken version again.
|
|
21
21
|
|
|
22
|
-
|
|
22
|
+
You explain that this repository never commits generated files. The next agent helpfully commits them.
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
You discover a subtle migration rule, write it down somewhere, and still have to remember to paste it into every new conversation.
|
|
25
25
|
|
|
26
|
-
|
|
26
|
+
That is the gap Audrey closes.
|
|
27
27
|
|
|
28
|
-
|
|
28
|
+
Audrey sits beside the agent and participates in the work automatically. At the start of a session it brings back a small, relevant memory packet. When you submit a prompt, it recalls project facts, preferences, procedures, and recent risks. Before a side-effectful tool runs, Audrey checks the proposed action against prior evidence. Afterward, it links the outcome back to the exact check that preceded it.
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
The model does not have to remember that a memory tool exists. That is the point.
|
|
31
31
|
|
|
32
|
-
|
|
32
|
+
## Meet Audrey Autopilot
|
|
33
33
|
|
|
34
|
-
|
|
35
|
-
- `memory_recall` finds durable context by semantic similarity.
|
|
36
|
-
- `memory_preflight` checks prior failures, risks, rules, and relevant procedures before an action.
|
|
37
|
-
- `memory_reflexes` converts remembered evidence into trigger-response guidance agents can follow.
|
|
38
|
-
- `memory_validate` closes the loop after the action: `helpful`, `used`, or `wrong` outcomes feed salience and can bind back to the exact preflight event, evidence ids, and Guard action fingerprint.
|
|
39
|
-
- `memory_dream` consolidates episodes into principles and applies decay.
|
|
40
|
-
- `audrey impact` and `audrey doctor` tell a human or CI system whether the runtime is doing real work and is actually ready.
|
|
34
|
+
Install Audrey once, review the hooks once, and then use Codex or Claude Code normally.
|
|
41
35
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
</div>
|
|
47
|
-
|
|
48
|
-
## Quick Start
|
|
36
|
+
```bash
|
|
37
|
+
npm install -g audrey
|
|
38
|
+
audrey install --host auto
|
|
39
|
+
```
|
|
49
40
|
|
|
50
|
-
|
|
41
|
+
`auto` configures whichever supported CLIs are installed. You can choose one explicitly:
|
|
51
42
|
|
|
52
43
|
```bash
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
npx audrey guard --tool Bash "npm run deploy"
|
|
44
|
+
audrey install --host codex
|
|
45
|
+
audrey install --host claude-code
|
|
56
46
|
```
|
|
57
47
|
|
|
58
|
-
|
|
48
|
+
Restart the host after installation. Codex asks you to trust non-managed hooks once through `/hooks`; Claude Code may also ask you to approve project or plugin components. Audrey is automatic after that explicit install-and-trust step—never secretly installed.
|
|
49
|
+
|
|
50
|
+
Autopilot then closes the loop:
|
|
51
|
+
|
|
52
|
+
| Moment | What Audrey does |
|
|
53
|
+
|---|---|
|
|
54
|
+
| Session starts | Injects a compact, agent-scoped memory briefing |
|
|
55
|
+
| You send a prompt | Recalls relevant evidence; explicitly durable phrases such as “remember that…” or “I prefer…” can become memories |
|
|
56
|
+
| Bash/edit/write is proposed | Checks exact prior failures, trusted rules, procedures, contradictions, and memory health |
|
|
57
|
+
| The tool finishes | Correlates `tool_use_id` to the Guard receipt and records the redacted outcome |
|
|
58
|
+
| A tool failure is reported | Forms a durable, sanitized failure memory for the next attempt |
|
|
59
|
+
| The turn stops or context compacts | Runs lightweight, due-only consolidation without holding the conversation open |
|
|
60
|
+
|
|
61
|
+
Infrastructure failures are fail-open by default: a broken memory service must not strand a developer. Teams that need enforcement can set `AUDREY_HOOK_FAIL_CLOSED=1`.
|
|
62
|
+
|
|
63
|
+
## A small story about a failed deploy
|
|
59
64
|
|
|
60
|
-
|
|
65
|
+
The first attempt fails:
|
|
61
66
|
|
|
62
67
|
```text
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
Verdict: ready
|
|
68
|
+
$ npm run deploy
|
|
69
|
+
Error: deployment target is missing
|
|
66
70
|
```
|
|
67
71
|
|
|
68
|
-
|
|
72
|
+
Audrey keeps a redacted trace and the exact action fingerprint. If another session proposes the same action before the problem is fixed, Guard returns a denial with evidence. Change the command or fix the target and Audrey lets the work continue. Once that exact action succeeds, the old failure no longer blocks it.
|
|
69
73
|
|
|
70
|
-
|
|
74
|
+
This is more useful than “the vector search found a vaguely similar error.” Audrey creates a receipt before the action, records what happened after it, and preserves the lineage between the two.
|
|
71
75
|
|
|
72
|
-
|
|
76
|
+
Try the complete loop without an API key or network call:
|
|
73
77
|
|
|
74
78
|
```bash
|
|
75
|
-
npx audrey
|
|
76
|
-
npx audrey install --host claude-code --dry-run
|
|
77
|
-
npx audrey install --host generic --dry-run
|
|
79
|
+
npx audrey demo --scenario repeated-failure
|
|
78
80
|
```
|
|
79
81
|
|
|
80
|
-
|
|
82
|
+
## What Audrey remembers
|
|
81
83
|
|
|
82
|
-
|
|
83
|
-
npx audrey mcp-config codex
|
|
84
|
-
npx audrey mcp-config generic
|
|
85
|
-
npx audrey mcp-config vscode
|
|
86
|
-
npx audrey hook-config claude-code
|
|
87
|
-
```
|
|
84
|
+
Audrey treats memory as more than a pile of text chunks.
|
|
88
85
|
|
|
89
|
-
|
|
86
|
+
- Episodes are things that happened: a user decision, a tool result, a project fact, a preference.
|
|
87
|
+
- Semantic memories are principles supported by accumulated evidence.
|
|
88
|
+
- Procedural memories are ways of acting: how to retry, verify, avoid, or recover.
|
|
89
|
+
- Contradictions stay visible instead of being silently overwritten.
|
|
90
|
+
- Confidence changes with source quality, evidence, age, retrieval, interference, context, and feedback.
|
|
91
|
+
- Low-value memories decay; repeated evidence can consolidate into longer-lived knowledge.
|
|
90
92
|
|
|
91
|
-
|
|
92
|
-
npx audrey install
|
|
93
|
-
claude mcp list
|
|
94
|
-
```
|
|
93
|
+
Every context packet includes memory IDs, confidence, provenance where available, and a reason for inclusion. Uncertain or disputed memories are labeled as such. Retrieved content is wrapped with a simple rule: memory is evidence, not authority; current system and user instructions always win.
|
|
95
94
|
|
|
96
|
-
|
|
97
|
-
claude-code`, then apply with `npx audrey hook-config claude-code --apply
|
|
98
|
-
--scope project` for `.claude/settings.local.json` or `--scope user` for
|
|
99
|
-
`~/.claude/settings.json`. Audrey merges the hook block into existing settings
|
|
100
|
-
and writes a timestamped backup before changing a non-empty file. The generated
|
|
101
|
-
`PreToolUse` hook runs `audrey guard --hook --fail-on-warn`; the `PostToolUse`
|
|
102
|
-
and `PostToolUseFailure` hooks record redacted tool traces. Verify the active
|
|
103
|
-
hook set inside Claude Code with `/hooks`.
|
|
95
|
+
## What Audrey deliberately does not do
|
|
104
96
|
|
|
105
|
-
|
|
97
|
+
Audrey does not upload your memory to a hosted service by default. It does not treat every sentence as permanent truth. It does not promote instructions from arbitrary tool output into trusted policy. It does not claim that a small local benchmark proves state-of-the-art memory quality.
|
|
106
98
|
|
|
107
|
-
|
|
99
|
+
Raw prompt events and tool bodies are not retained by default. Audrey stores hashes, bounded summaries, fingerprints, and redaction metadata. Explicit user-memory language is persisted intentionally; tool failure memories are sanitized first. Admin export/import/forget surfaces are disabled unless `AUDREY_ENABLE_ADMIN_TOOLS=1`.
|
|
108
100
|
|
|
109
|
-
|
|
101
|
+
At-rest encryption, identity-bound tenant authorization, rate limiting, and regulated retention remain deployment responsibilities today. They are not hidden behind a “production ready” badge.
|
|
110
102
|
|
|
111
|
-
|
|
103
|
+
## Why a team might actually want this
|
|
112
104
|
|
|
113
|
-
|
|
114
|
-
AUDREY_AGENT=ollama-local-agent npx audrey serve
|
|
115
|
-
curl http://localhost:7437/health
|
|
116
|
-
curl http://localhost:7437/v1/status
|
|
117
|
-
```
|
|
105
|
+
### Fewer repeated mistakes
|
|
118
106
|
|
|
119
|
-
|
|
107
|
+
Guard checks memory at the point where it can change an action, not after the damage is done. Exact failure fingerprints avoid the noisy “one Bash command failed, so all Bash commands are suspicious” behavior.
|
|
120
108
|
|
|
121
|
-
|
|
122
|
-
AUDREY_AGENT=ollama-local-agent npx audrey serve
|
|
123
|
-
OLLAMA_MODEL=qwen3 node examples/ollama-memory-agent.js "What should you remember about Audrey?"
|
|
124
|
-
```
|
|
109
|
+
### Continuity across agent sessions
|
|
125
110
|
|
|
126
|
-
|
|
111
|
+
Audrey is not tied to one model vendor. Codex and Claude Code use the same memory runtime and the same evidence contract. MCP, REST, JavaScript, and Python clients make the core usable in custom agents too.
|
|
127
112
|
|
|
128
|
-
|
|
129
|
-
|---|---|
|
|
130
|
-
| Check memory before acting | `POST /v1/preflight` |
|
|
131
|
-
| Get reflex rules for an action | `POST /v1/reflexes` |
|
|
132
|
-
| Store a useful observation | `POST /v1/encode` |
|
|
133
|
-
| Recall relevant context | `POST /v1/recall` |
|
|
134
|
-
| Get a turn-sized memory packet | `POST /v1/capsule` |
|
|
135
|
-
| Check health | `GET /v1/status` |
|
|
113
|
+
### Evidence a human can inspect
|
|
136
114
|
|
|
137
|
-
|
|
115
|
+
Allow, warn, and block decisions carry receipts and evidence IDs. Outcome records connect back to those receipts. Teams can ask not only “what did the agent remember?” but “which memory changed this action, and was that useful?”
|
|
138
116
|
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
| CLI | `doctor`, `demo`, `guard`, `install`, `mcp-config`, `hook-config`, `status`, `dream`, `reembed`, `observe-tool`, `promote`, `impact` |
|
|
143
|
-
| REST API | Hono server with `/health` and `/v1/*` routes |
|
|
144
|
-
| JavaScript SDK | Direct TypeScript/Node import from `audrey` |
|
|
145
|
-
| Python client | `pip install audrey-memory`, calls the REST sidecar |
|
|
146
|
-
| Storage | Local SQLite plus `sqlite-vec`, no hosted database required |
|
|
147
|
-
| Deployment | npm package, Docker, Compose, host-specific MCP config generation |
|
|
148
|
-
| Safety loop | preflight warnings, reflexes, redacted tool traces, contradiction handling |
|
|
117
|
+
### Local control
|
|
118
|
+
|
|
119
|
+
The default store is SQLite, FTS5, and `sqlite-vec`. Local embeddings are the default. Cloud embedding or LLM providers require explicit configuration.
|
|
149
120
|
|
|
150
|
-
|
|
121
|
+
### A safer shared store
|
|
151
122
|
|
|
152
|
-
|
|
123
|
+
Agent-scoped recall now continues through validation, contradiction detection, interference, affect, failure lookup, capsules, greetings, Guard, and REST request routing. Hidden retrieval candidates do not reinforce themselves; only memories actually surfaced to the caller receive retrieval bookkeeping.
|
|
153
124
|
|
|
154
|
-
|
|
155
|
-
- Semantic memory: consolidated principles extracted from repeated evidence.
|
|
156
|
-
- Procedural memory: remembered ways to act, avoid, retry, or verify.
|
|
157
|
-
- Affect and salience: emotional weight and importance influence recall.
|
|
158
|
-
- Interference and decay: stale, conflicting, or low-confidence memories lose authority over time.
|
|
159
|
-
- Contradiction handling: competing claims are tracked instead of silently overwritten.
|
|
160
|
-
- Tool-trace learning: failed commands and risky actions become future preflight warnings.
|
|
125
|
+
Vector candidates are partitioned by agent before nearest-neighbor ranking, so one busy agent cannot crowd another out of a bounded search. For hard tenant boundaries, still use a distinct `AUDREY_DATA_DIR` per tenant or security domain.
|
|
161
126
|
|
|
162
|
-
|
|
127
|
+
## See it before installing anything
|
|
163
128
|
|
|
164
|
-
|
|
129
|
+
```bash
|
|
130
|
+
npx audrey doctor
|
|
131
|
+
npx audrey demo
|
|
132
|
+
npx audrey install --host auto --dry-run
|
|
133
|
+
```
|
|
165
134
|
|
|
166
|
-
|
|
135
|
+
The dry run prints the MCP and lifecycle-hook configuration for both hosts without changing files.
|
|
167
136
|
|
|
168
|
-
|
|
169
|
-
|
|
137
|
+
<div align="center">
|
|
138
|
+
<img src="docs/assets/audrey-feature-grid.jpg" alt="Audrey memory continuity, recall, evidence, local storage, and memory-before-action" width="760">
|
|
139
|
+
</div>
|
|
170
140
|
|
|
171
|
-
|
|
172
|
-
dataDir: './audrey-data',
|
|
173
|
-
agent: 'support-agent',
|
|
174
|
-
embedding: { provider: 'local', dimensions: 384 },
|
|
175
|
-
});
|
|
141
|
+
## Where we want to take it
|
|
176
142
|
|
|
177
|
-
|
|
178
|
-
content: 'Stripe returns HTTP 429 above 100 req/s',
|
|
179
|
-
source: 'direct-observation',
|
|
180
|
-
tags: ['stripe', 'rate-limit'],
|
|
181
|
-
});
|
|
143
|
+
The ambition is a temporal evidence graph for agents: immutable observations, explicit validity windows, source trust, evolving claims, scoped procedures, and outcome-calibrated policy. The defensible part is not storing more text. It is knowing what was believed, why, in which context, for how long, and whether acting on it helped.
|
|
182
144
|
|
|
183
|
-
|
|
145
|
+
Near-term work includes durable background cognition jobs, tenant namespaces bound to credentials, memory quarantine and taint propagation, public long-horizon evaluations, encrypted backup options, and a persistent local daemon that removes per-hook model startup entirely.
|
|
184
146
|
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
147
|
+
If that is the kind of agent infrastructure you want to build, open an issue or start with the demo. Audrey is MIT licensed, and the product boundary is intentionally inspectable.
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Technical reference
|
|
188
152
|
|
|
189
|
-
|
|
153
|
+
Everything below is the machinery. The short version above is the product.
|
|
154
|
+
|
|
155
|
+
### Requirements and packages
|
|
156
|
+
|
|
157
|
+
- Node.js 20+
|
|
158
|
+
- npm package: `audrey`
|
|
159
|
+
- Python client: `audrey-memory`
|
|
160
|
+
- Default storage: local SQLite + FTS5 + `sqlite-vec`
|
|
161
|
+
- Default embeddings: local 384-dimensional model
|
|
190
162
|
|
|
191
163
|
```bash
|
|
164
|
+
npm install audrey
|
|
192
165
|
pip install audrey-memory
|
|
193
166
|
```
|
|
194
167
|
|
|
195
|
-
|
|
196
|
-
|
|
168
|
+
For Autopilot, prefer a global or otherwise stable installation. Hook and MCP configuration pins the actual Node executable and Audrey entrypoint; an ephemeral `npx` cache is not a durable production runtime.
|
|
169
|
+
|
|
170
|
+
### Host configuration
|
|
171
|
+
|
|
172
|
+
Preview or apply lifecycle hooks independently:
|
|
197
173
|
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
174
|
+
```bash
|
|
175
|
+
audrey hook-config claude-code
|
|
176
|
+
audrey hook-config claude-code --apply --scope local
|
|
177
|
+
audrey hook-config claude-code --apply --scope project
|
|
178
|
+
audrey hook-config claude-code --apply --scope user
|
|
179
|
+
|
|
180
|
+
audrey hook-config codex
|
|
181
|
+
audrey hook-config codex --apply --scope project
|
|
182
|
+
audrey hook-config codex --apply --scope user
|
|
202
183
|
```
|
|
203
184
|
|
|
204
|
-
|
|
185
|
+
Claude Code scope mapping follows the host’s terminology:
|
|
186
|
+
|
|
187
|
+
- `local` → `.claude/settings.local.json`
|
|
188
|
+
- `project` → `.claude/settings.json`
|
|
189
|
+
- `user` → `~/.claude/settings.json`
|
|
190
|
+
|
|
191
|
+
Codex supports project `.codex/hooks.json` and user `~/.codex/hooks.json`; it has no local hook scope. Audrey preserves unrelated hooks, replaces older Audrey-owned handlers, writes a private timestamped backup, and is idempotent on repeat installation. Project-adjacent backup names match `*.audrey-*.bak`; keep that pattern ignored because a host config can contain unrelated credentials.
|
|
205
192
|
|
|
206
|
-
Audrey
|
|
193
|
+
Audrey respects `CLAUDE_CONFIG_DIR` and `CODEX_HOME`. Generated hooks pin the stable Node executable, Audrey entrypoint, data directory, agent identity, and non-secret provider choices used at install time. With local embeddings, an Autopilot install performs one warmup so the first real hook is not also the first model load; set `AUDREY_DISABLE_WARMUP=1` to skip it.
|
|
207
194
|
|
|
208
|
-
|
|
195
|
+
Generate MCP configuration without applying it:
|
|
209
196
|
|
|
210
197
|
```bash
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
npm run bench:guard:validate
|
|
215
|
-
npx audrey doctor
|
|
216
|
-
npx audrey demo
|
|
198
|
+
audrey mcp-config codex
|
|
199
|
+
audrey mcp-config generic
|
|
200
|
+
audrey mcp-config vscode
|
|
217
201
|
```
|
|
218
202
|
|
|
219
|
-
|
|
203
|
+
Remove Audrey-owned MCP registrations and hooks with the same host and scope you installed:
|
|
220
204
|
|
|
221
205
|
```bash
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
206
|
+
audrey uninstall --host auto --scope user
|
|
207
|
+
audrey uninstall --host claude-code --scope local
|
|
208
|
+
audrey uninstall --host codex --scope project
|
|
225
209
|
```
|
|
226
210
|
|
|
227
|
-
|
|
211
|
+
Add `--dry-run` to preview uninstall without changing either host. Add `--mcp-only` only when you intentionally want to preserve Audrey hooks.
|
|
228
212
|
|
|
229
|
-
|
|
230
|
-
- Pin `AUDREY_EMBEDDING_PROVIDER` and `AUDREY_LLM_PROVIDER` explicitly.
|
|
231
|
-
- Back up the SQLite data directory before provider or dimension changes.
|
|
232
|
-
- Keep API keys and raw credentials out of encoded memory content.
|
|
233
|
-
- Use `AUDREY_API_KEY` if the REST sidecar is reachable beyond the local process boundary.
|
|
234
|
-
- Run `npx audrey dream` on a schedule so consolidation and decay stay current.
|
|
235
|
-
- Add application-level encryption, retention, access control, and audit logging for regulated environments.
|
|
213
|
+
### Autopilot safety contract
|
|
236
214
|
|
|
237
|
-
|
|
215
|
+
The shared hook adapter normalizes current Codex and Claude Code payloads.
|
|
238
216
|
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
| `AUDREY_PORT` | `7437` | REST sidecar port. |
|
|
247
|
-
| `AUDREY_HOST` | `127.0.0.1` | REST sidecar bind address. Set to `0.0.0.0` only with `AUDREY_API_KEY`. |
|
|
248
|
-
| `AUDREY_API_KEY` | unset | Bearer token required for non-loopback REST traffic. |
|
|
249
|
-
| `AUDREY_ALLOW_NO_AUTH` | `0` | Set to `1` to allow non-loopback bind without an API key. Don't. |
|
|
250
|
-
| `AUDREY_ENABLE_ADMIN_TOOLS` | `0` | Set to `1` to enable export, import, and forget routes/tools. Disabled by default. |
|
|
251
|
-
| `AUDREY_PROMOTE_ROOTS` | unset | Colon/semicolon-separated extra roots for `audrey promote --yes` writes. By default writes are restricted to `process.cwd()`. |
|
|
252
|
-
| `AUDREY_DEBUG` | `0` | Set to `1` to print MCP info logs (server started, warmup completed). Errors always log. |
|
|
253
|
-
| `AUDREY_PROFILE` | `0` | Set to `1` to emit per-stage timings via MCP `_meta.diagnostics`. |
|
|
254
|
-
| `AUDREY_DISABLE_WARMUP` | `0` | Set to `1` to skip background embedding warmup at MCP boot. |
|
|
255
|
-
| `AUDREY_ONNX_VERBOSE` | `0` | Set to `1` to restore ONNX runtime EP-assignment warnings (suppressed by default). |
|
|
256
|
-
| `AUDREY_PRAGMA_DEFAULTS` | `1` | Set to `0` to revert SQLite PRAGMA tuning to better-sqlite3 defaults. |
|
|
257
|
-
| `AUDREY_CONTEXT_BUDGET_CHARS` | `4000` | Default Memory Capsule character budget. |
|
|
258
|
-
|
|
259
|
-
## Benchmarks
|
|
260
|
-
|
|
261
|
-
Audrey ships three benchmark families.
|
|
262
|
-
|
|
263
|
-
### Performance snapshot
|
|
264
|
-
|
|
265
|
-
`npm run bench:perf-snapshot` measures encode and hybrid recall latency at multiple corpus sizes against the in-process mock provider. It reports p50/p95/p99 plus machine provenance so the numbers are reproducible and honest about what they cover.
|
|
217
|
+
- Context injection is bounded by `AUDREY_CONTEXT_BUDGET_CHARS` (default 4000; Autopilot uses a conservative 3200-character packet unless overridden).
|
|
218
|
+
- Prompt and tool retrieval queries are bounded before embedding. Large edits carry hashes and lengths instead of file bodies; exact Guard identity uses a full redacted digest rather than a truncated prefix.
|
|
219
|
+
- Only `Bash`, `Edit`, `Write`, `NotebookEdit`, and `apply_patch` are guarded and observed by the generated default hooks.
|
|
220
|
+
- Pre/post correlation uses `session_id + tool_use_id`, so parallel tool calls do not attach to the wrong receipt.
|
|
221
|
+
- Claude `PostToolUseFailure` and Codex responses that explicitly expose a non-zero exit normalize to the same failure path. Current Codex hooks can omit Bash exit status; Audrey records an opaque result as `unknown`, never as invented success.
|
|
222
|
+
- Context and Guard failures emit `{}` and log to stderr unless fail-closed mode is explicitly enabled.
|
|
223
|
+
- Stop hooks always emit valid JSON and never continue or block a completed turn.
|
|
266
224
|
|
|
267
|
-
|
|
268
|
-
npm run build
|
|
269
|
-
npm run bench:perf-snapshot # default sizes 100, 1000, 5000
|
|
270
|
-
node benchmarks/perf-snapshot.js --sizes 1000,10000 --json # custom shape
|
|
271
|
-
```
|
|
225
|
+
Codex hook interception is a guardrail, not a complete shell-policy boundary. The current host contract does not intercept every richer `unified_exec` path and may omit the exit status of silent Bash failures. See the [Codex hooks documentation](https://learn.chatgpt.com/docs/hooks). Use the Guard receipt as evidence, and keep sandboxing, approvals, CI, and deployment controls in place.
|
|
272
226
|
|
|
273
|
-
|
|
227
|
+
### JavaScript API
|
|
274
228
|
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
| 100 | 0.33 | 0.59 | 0.54 | 1.82 | 2.71 |
|
|
278
|
-
| 1,000 | 0.31 | 2.15 | 1.57 | 2.36 | 21.18 |
|
|
279
|
-
| 5,000 | 0.31 | 1.84 | 2.09 | 3.42 | 16.58 |
|
|
229
|
+
```js
|
|
230
|
+
import { Audrey, MemoryController } from 'audrey';
|
|
280
231
|
|
|
281
|
-
|
|
232
|
+
const memory = new Audrey({
|
|
233
|
+
dataDir: './audrey-data',
|
|
234
|
+
agent: 'payments-agent',
|
|
235
|
+
embedding: { provider: 'local', dimensions: 384 },
|
|
236
|
+
});
|
|
282
237
|
|
|
283
|
-
|
|
238
|
+
await memory.encode({
|
|
239
|
+
content: 'Stripe returns HTTP 429 above 100 requests per second.',
|
|
240
|
+
source: 'direct-observation',
|
|
241
|
+
tags: ['stripe', 'rate-limit'],
|
|
242
|
+
context: { service: 'billing' },
|
|
243
|
+
});
|
|
244
|
+
|
|
245
|
+
const capsule = await memory.capsule('increase Stripe throughput', {
|
|
246
|
+
scope: 'agent',
|
|
247
|
+
budgetChars: 3000,
|
|
248
|
+
});
|
|
284
249
|
|
|
285
|
-
|
|
250
|
+
const guard = new MemoryController(memory);
|
|
251
|
+
const before = await guard.beforeAction({
|
|
252
|
+
action: 'deploy the billing worker',
|
|
253
|
+
tool: 'Bash',
|
|
254
|
+
command: 'npm run deploy:billing',
|
|
255
|
+
cwd: process.cwd(),
|
|
256
|
+
});
|
|
286
257
|
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
npm run bench:memory:check # release gate, exits non-zero on regression
|
|
258
|
+
console.log(before.decision, before.evidenceIds);
|
|
259
|
+
await memory.closeAsync();
|
|
290
260
|
```
|
|
291
261
|
|
|
292
|
-
###
|
|
293
|
-
|
|
294
|
-
`npm run bench:guard:check` runs Audrey's local GuardBench comparative suite:
|
|
295
|
-
ten pre-action scenarios across Audrey Guard, no-memory, recent-window,
|
|
296
|
-
vector-only, and FTS-only adapters. The scenarios cover exact repeated
|
|
297
|
-
failures, required procedures, changed file scopes, changed commands,
|
|
298
|
-
recovered failures, recall degradation, redaction safety, conflicting
|
|
299
|
-
instructions, and noisy stores. It writes
|
|
300
|
-
`benchmarks/output/guardbench-summary.json`,
|
|
301
|
-
`benchmarks/output/guardbench-manifest.json`, and
|
|
302
|
-
`benchmarks/output/guardbench-raw.json`. The emitted manifest, summary, and raw
|
|
303
|
-
output shapes are validated by JSON schemas under `benchmarks/schemas/`.
|
|
304
|
-
|
|
305
|
-
Latest local result in this checkout: 10/10 scenarios passed, 100% prevention
|
|
306
|
-
rate, 0% false-block rate, 0 raw secret leaks, 0 published artifact leaks in
|
|
307
|
-
the raw-secret sweep, and 3.09ms / 28.181ms
|
|
308
|
-
p50/p95 guard latency under the mock-provider methodology.
|
|
309
|
-
|
|
310
|
-
**Methodology caveats, on purpose.** All numbers above are produced against
|
|
311
|
-
the in-process mock 64-dim embedding provider documented in the run's
|
|
312
|
-
`provenance` block. They characterize Audrey's controller and SQLite path,
|
|
313
|
-
not real-provider end-to-end latency or production false-positive rates. The
|
|
314
|
-
100% prevention rate is over the 5 GuardBench scenarios that expect a
|
|
315
|
-
`block` decision (the suite is 10 scenarios total, mixed across allow / warn
|
|
316
|
-
/ block). Local baseline decision accuracy was: no-memory 10%, recent-window
|
|
317
|
-
60%, vector-only 40%, and FTS-only 10%; none of the local baselines passed
|
|
318
|
-
the GuardBench decision-plus-evidence contract, which since v1.0.1 requires
|
|
319
|
-
the correct decision plus at least one returned evidence id for `block` /
|
|
320
|
-
`warn` scenarios (no longer Audrey-specific lineage phrasing — see
|
|
321
|
-
`CHANGELOG.md#101---2026-05-15`). External-system numbers for Mem0 and Zep
|
|
322
|
-
are explicitly out of scope for this Stage-A artifact; live credentialed
|
|
323
|
-
runs land in a v2 paper after raw evidence bundles publish.
|
|
262
|
+
### REST sidecar
|
|
324
263
|
|
|
325
264
|
```bash
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
npm run bench:guard:manifest
|
|
329
|
-
npm run bench:guard:validate
|
|
330
|
-
npm run bench:guard:card
|
|
331
|
-
npm run bench:guard:bundle
|
|
332
|
-
npm run bench:guard:bundle:verify
|
|
333
|
-
npm run bench:guard:leaderboard
|
|
334
|
-
npm run bench:guard:adapter-registry:validate
|
|
335
|
-
npm run bench:guard:adapter-module:validate
|
|
336
|
-
npm run bench:guard:adapter-self-test
|
|
337
|
-
npm run bench:guard:adapter-self-test:validate
|
|
338
|
-
npm run bench:guard:publication:verify
|
|
339
|
-
npm run bench:guard:adapter-smoke
|
|
340
|
-
npm run bench:guard:adapter-conformance
|
|
341
|
-
npm run bench:guard:external:dry-run
|
|
342
|
-
npm run bench:guard:mem0 -- --dry-run
|
|
343
|
-
npm run bench:guard:zep -- --dry-run
|
|
344
|
-
node benchmarks/adapter-self-test.mjs --adapter ./path/to/adapter.mjs
|
|
345
|
-
node benchmarks/guardbench.js --adapter ./path/to/adapter.mjs --check
|
|
265
|
+
AUDREY_AGENT=payments-agent audrey serve
|
|
266
|
+
curl http://127.0.0.1:7437/health
|
|
346
267
|
```
|
|
347
268
|
|
|
348
|
-
|
|
349
|
-
`adapter`, or `createGuardBenchAdapter()`. The adapter receives scenario seed
|
|
350
|
-
data and the proposed action, but the harness withholds `expectedDecision` and
|
|
351
|
-
`requiredEvidence` until scoring. Start from
|
|
352
|
-
`benchmarks/adapters/example-allow.mjs` when wiring a new system. Adapter
|
|
353
|
-
authors can import `defineGuardBenchAdapter()` and `defineGuardBenchResult()`
|
|
354
|
-
from `benchmarks/adapter-kit.mjs` to validate module shape and decision output
|
|
355
|
-
while developing.
|
|
269
|
+
Core routes:
|
|
356
270
|
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
271
|
+
| Need | Route |
|
|
272
|
+
|---|---|
|
|
273
|
+
| Encode an episode | `POST /v1/encode` |
|
|
274
|
+
| Recall memory | `POST /v1/recall` |
|
|
275
|
+
| Build a context packet | `POST /v1/capsule` |
|
|
276
|
+
| Check before an action | `POST /v1/preflight` |
|
|
277
|
+
| Create a Guard receipt | `POST /v1/guard/before` |
|
|
278
|
+
| Close a Guard receipt | `POST /v1/guard/after` |
|
|
279
|
+
| Consolidate and decay | `POST /v1/dream` |
|
|
280
|
+
| Health and index state | `GET /v1/status` |
|
|
360
281
|
|
|
361
|
-
|
|
282
|
+
Use `AUDREY_API_KEY` for any non-loopback deployment. `X-Audrey-Agent` scopes encode, recall, capsules, preflight, Guard, consolidation, and greetings inside a trusted deployment; it is a routing header, not an authentication boundary. Bind agent/tenant identity at your gateway rather than trusting an arbitrary public header.
|
|
362
283
|
|
|
363
|
-
|
|
364
|
-
npm run bench:guard:adapter-module:validate -- --adapter ./path/to/adapter.mjs
|
|
365
|
-
```
|
|
284
|
+
### Python client
|
|
366
285
|
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
contract and row conformance while explicitly allowing low benchmark scores, so
|
|
370
|
-
authors can separate "valid submission shape" from "competitive GuardBench
|
|
371
|
-
performance." The generated self-test report is validated against
|
|
372
|
-
`benchmarks/schemas/guardbench-adapter-self-test.schema.json`. Reviewers can
|
|
373
|
-
validate a submitted report without rerunning an adapter through `npm run
|
|
374
|
-
bench:guard:adapter-self-test:validate -- --report ./guardbench-adapter-self-test.json`.
|
|
286
|
+
```python
|
|
287
|
+
from audrey_memory import Audrey
|
|
375
288
|
|
|
376
|
-
|
|
377
|
-
|
|
289
|
+
memory = Audrey(base_url="http://127.0.0.1:7437", agent="payments-agent")
|
|
290
|
+
memory_id = memory.encode(
|
|
291
|
+
"Stripe returns HTTP 429 above 100 requests per second.",
|
|
292
|
+
source="direct-observation",
|
|
293
|
+
)
|
|
294
|
+
results = memory.recall("Stripe rate limit", limit=5)
|
|
295
|
+
memory.close()
|
|
296
|
+
```
|
|
378
297
|
|
|
379
|
-
|
|
380
|
-
set MEM0_API_KEY=...
|
|
381
|
-
npm run bench:guard:mem0
|
|
298
|
+
The Python package is a client for the REST sidecar; the memory runtime remains in the Node process.
|
|
382
299
|
|
|
383
|
-
|
|
384
|
-
|
|
300
|
+
### Memory and retrieval pipeline
|
|
301
|
+
|
|
302
|
+
```text
|
|
303
|
+
episode
|
|
304
|
+
├─ transactional SQLite + vector + FTS write
|
|
305
|
+
├─ agent-scoped interference / resonance / validation
|
|
306
|
+
├─ reinforcement or contradiction evidence
|
|
307
|
+
└─ sleep-time consolidation into semantic or procedural memory
|
|
308
|
+
|
|
309
|
+
query
|
|
310
|
+
├─ bounded vector candidates
|
|
311
|
+
├─ FTS5 lexical candidates
|
|
312
|
+
├─ reciprocal-rank fusion and confidence scoring
|
|
313
|
+
├─ context / affect / recency / interference modifiers
|
|
314
|
+
└─ final-only retrieval bookkeeping
|
|
385
315
|
```
|
|
386
316
|
|
|
387
|
-
|
|
388
|
-
`graph.search`, and benchmark-user cleanup. If Zep graph ingestion needs more
|
|
389
|
-
time in a live account, set `ZEP_GUARDBENCH_INGEST_DELAY_MS` before the run.
|
|
390
|
-
|
|
391
|
-
Run `npm run bench:guard:external:dry-run` before coordinating credentialed
|
|
392
|
-
runs. It walks the runtime-env adapter registry, writes non-secret
|
|
393
|
-
`external-run-metadata.json` files for each adapter, and reports which runtime
|
|
394
|
-
environment variables are still missing. The external dry-run matrix report is schema-bound by
|
|
395
|
-
`benchmarks/schemas/guardbench-external-dry-run.schema.json` and written to
|
|
396
|
-
`benchmarks/output/external/guardbench-external-dry-run.json`.
|
|
397
|
-
|
|
398
|
-
Run `npm run bench:guard:external:evidence` after dry-runs or live runs to
|
|
399
|
-
write `benchmarks/output/external/guardbench-external-evidence.json`. This
|
|
400
|
-
external evidence verification report is schema-bound by
|
|
401
|
-
`benchmarks/schemas/guardbench-external-evidence.schema.json`, treats dry-run
|
|
402
|
-
or missing-key rows as pending in normal release gates, and checks that saved
|
|
403
|
-
metadata does not contain runtime credential values. Use
|
|
404
|
-
`npm run bench:guard:external:evidence:strict` when Mem0/Zep keys have been
|
|
405
|
-
provided; strict mode fails until every runtime-env adapter has a passed live
|
|
406
|
-
bundle.
|
|
407
|
-
|
|
408
|
-
External runs write `external-run-metadata.json` alongside the GuardBench
|
|
409
|
-
summary, manifest, and raw output bundle under
|
|
410
|
-
`benchmarks/output/external/<adapter>/`. The external runner validates the
|
|
411
|
-
emitted bundle with `benchmarks/validate-guardbench-artifacts.mjs` before
|
|
412
|
-
marking the run passed, and separately records adapter conformance so a valid
|
|
413
|
-
low-scoring adapter is distinguished from a malformed adapter. When
|
|
414
|
-
`external-run-metadata.json` is present, the validator also checks it against
|
|
415
|
-
`benchmarks/schemas/guardbench-external-run.schema.json` and verifies any
|
|
416
|
-
recorded SHA-256 artifact hashes against the bundle on disk.
|
|
417
|
-
|
|
418
|
-
For a shareable submission artifact, run `npm run bench:guard:card -- --dir
|
|
419
|
-
<output-dir>`. This writes `guardbench-conformance-card.json` with the subject
|
|
420
|
-
name, run status, score, conformance result, artifact hashes, optional
|
|
421
|
-
external-run metadata hash, and machine provenance. The standalone validator
|
|
422
|
-
checks the card when it is present.
|
|
423
|
-
|
|
424
|
-
For a portable submission directory, run `npm run bench:guard:bundle -- --dir
|
|
425
|
-
<output-dir>`. This creates `submission-bundle/` with the raw GuardBench
|
|
426
|
-
artifacts, conformance card, JSON schemas, validation report, and
|
|
427
|
-
`submission-manifest.json` with SHA-256 hashes for every bundled file.
|
|
428
|
-
Reviewers can run `npm run bench:guard:bundle:verify -- --dir
|
|
429
|
-
<submission-bundle>` to check manifest hashes, bundled schemas, and artifact
|
|
430
|
-
validation from the bundle alone.
|
|
431
|
-
|
|
432
|
-
For benchmark aggregation, run `npm run bench:guard:leaderboard -- --bundle
|
|
433
|
-
<submission-bundle>`. The leaderboard builder verifies each bundle before
|
|
434
|
-
ranking and writes JSON plus Markdown reports under `benchmarks/output/leaderboard/`.
|
|
435
|
-
|
|
436
|
-
Before publishing benchmark artifacts, run `npm run
|
|
437
|
-
bench:guard:publication:verify`. This single benchmark-focused verifier checks
|
|
438
|
-
the adapter registry, default adapter module, adapter self-test report,
|
|
439
|
-
GuardBench manifest/summary/raw artifacts, submission bundle, external dry-run
|
|
440
|
-
matrix, external evidence verification report, leaderboard, and a local
|
|
441
|
-
absolute-path sweep over the public artifact set.
|
|
442
|
-
The verifier validates its own machine-readable report against
|
|
443
|
-
`benchmarks/schemas/guardbench-publication-verification.schema.json` before it
|
|
444
|
-
exits.
|
|
445
|
-
|
|
446
|
-
Before turning the paper into public posts or submissions, run `npm run
|
|
447
|
-
paper:claims`. It validates `docs/paper/claim-register.json` against the
|
|
448
|
-
current paper, README, GuardBench artifacts, publication verifier, and external
|
|
449
|
-
evidence status so pending Mem0/Zep live-score claims cannot slip into public
|
|
450
|
-
copy.
|
|
451
|
-
Run `npm run paper:publication-pack` to verify the ready-to-use arXiv, Hacker
|
|
452
|
-
News, Reddit, X, and LinkedIn drafts in `docs/paper/publication-pack.json`
|
|
453
|
-
before browser-based submission. The X URL reserve is explicit: the first X
|
|
454
|
-
post carries `reservedUrlChars: 24`, and submitted artifact-url targets in
|
|
455
|
-
`browser-launch-results.json` must record the final `artifactUrl`.
|
|
456
|
-
Run `npm run paper:arxiv` to generate a deterministic TeX source package under
|
|
457
|
-
`docs/paper/output/arxiv/`, and `npm run paper:arxiv:verify` to check hashes,
|
|
458
|
-
citation conversion, bibliography coverage, seeded-secret redaction, and local
|
|
459
|
-
absolute-path leakage before arXiv upload.
|
|
460
|
-
Run `npm run paper:arxiv:compile` to record a schema-bound compile report at
|
|
461
|
-
`docs/paper/output/arxiv-compile-report.json`. It attempts `tectonic`,
|
|
462
|
-
`latexmk`, `pdflatex`/`bibtex`, or `uvx tecto` with a local bundle proxy when
|
|
463
|
-
available; `npm run paper:arxiv:compile:strict` stays blocked on hosts without
|
|
464
|
-
supported TeX tooling.
|
|
465
|
-
Run `npm run paper:launch-plan` to verify
|
|
466
|
-
`docs/paper/browser-launch-plan.json`, which maps those drafts to manual
|
|
467
|
-
browser targets, login/captcha expectations, platform-rule checks, source
|
|
468
|
-
URLs, and post-submit URL capture.
|
|
469
|
-
Run `npm run paper:launch-results` to validate
|
|
470
|
-
`docs/paper/browser-launch-results.json`, the post-submit ledger for arXiv,
|
|
471
|
-
Hacker News, Reddit, X, and LinkedIn targets. The normal verifier allows
|
|
472
|
-
pending rows with explicit blockers; `npm run paper:launch-results:strict`
|
|
473
|
-
fails until every target has a submitted, operator-verified public URL.
|
|
474
|
-
Run `npm run paper:bundle` to generate
|
|
475
|
-
`docs/paper/output/submission-bundle/`, a hash-manifested package containing
|
|
476
|
-
paper sources, claim and publication registers, GuardBench outputs, schemas,
|
|
477
|
-
and package metadata. `npm run paper:bundle:verify` checks the manifest and
|
|
478
|
-
file hashes before browser upload.
|
|
479
|
-
Run `npm run release:readiness` for the pending-aware Audrey 1.0 checklist.
|
|
480
|
-
It keeps code/paper readiness separate from publish blockers; `npm run
|
|
481
|
-
release:readiness:strict` fails until the 1.0 version surfaces,
|
|
482
|
-
source-control state, live remote-head verification, Python artifacts, npm
|
|
483
|
-
registry/auth readiness, PyPI publish readiness, arXiv compile proof, browser
|
|
484
|
-
publication URLs, and live Mem0/Zep evidence are complete.
|
|
485
|
-
Run `npm run release:cut:plan` to preview the exact 1.0 version/changelog
|
|
486
|
-
edits across npm, lockfile, MCP, and Python surfaces. `npm run
|
|
487
|
-
release:cut:apply -- --target-version 1.0.0` writes those edits only when the
|
|
488
|
-
final cut is intentional. The generated changelog section is release-note copy,
|
|
489
|
-
not a TODO scaffold; `release:readiness:strict` rejects placeholder changelog
|
|
490
|
-
markers before publication.
|
|
491
|
-
Run `npm run security:audit` before packaging or publishing; the release gates
|
|
492
|
-
call it after artifact verification so production dependency advisories cannot
|
|
493
|
-
slip past the final package check.
|
|
494
|
-
|
|
495
|
-
## Command Reference
|
|
317
|
+
Agent-scoped vector search uses a native `sqlite-vec` partition key before nearest-neighbor ranking, not post-filtered whole-store candidates. If fusion underfills, Audrey makes one bounded partition-local retry. Semantic and procedural retrieval counts update only as final results are yielded; deduplicated, over-limit, and unconsumed stream candidates receive no authority boost.
|
|
496
318
|
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
319
|
+
### MCP surface
|
|
320
|
+
|
|
321
|
+
Audrey exposes 22 MCP tools plus status, recent-memory, and principle resources and briefing/recall/reflection prompts. The main groups are:
|
|
322
|
+
|
|
323
|
+
- capture: `memory_encode`, `memory_reflect`, `memory_observe_tool`
|
|
324
|
+
- retrieval: `memory_recall`, `memory_capsule`, `memory_greeting`
|
|
325
|
+
- action safety: `memory_preflight`, `memory_guard_before`, `memory_guard_after`, `memory_reflexes`
|
|
326
|
+
- lifecycle: `memory_consolidate`, `memory_dream`, `memory_decay`, `memory_resolve_truth`
|
|
327
|
+
- governance: `memory_validate`, `memory_promote`, `memory_forget`, `memory_export`, `memory_import`, `memory_status`, `memory_introspect`
|
|
328
|
+
|
|
329
|
+
The server also sends host instructions explaining the Guard receipt loop when lifecycle hooks are unavailable.
|
|
501
330
|
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
331
|
+
### Environment variables
|
|
332
|
+
|
|
333
|
+
| Variable | Default | Purpose |
|
|
334
|
+
|---|---|---|
|
|
335
|
+
| `AUDREY_DATA_DIR` | `~/.audrey/data` | SQLite store; use a distinct directory per tenant/security boundary |
|
|
336
|
+
| `AUDREY_AGENT` | host-specific | Logical memory owner used for scoped operations |
|
|
337
|
+
| `AUDREY_EMBEDDING_PROVIDER` | `local` | `local`, `gemini`, `openai`, or `mock` |
|
|
338
|
+
| `AUDREY_LLM_PROVIDER` | auto | `anthropic`, `openai`, or `mock` for reflection/consolidation |
|
|
339
|
+
| `AUDREY_LLM_MODEL` | provider default | Explicit LLM model override |
|
|
340
|
+
| `AUDREY_DEVICE` | `gpu` | Local embedding device; falls back to CPU |
|
|
341
|
+
| `AUDREY_CONTEXT_BUDGET_CHARS` | `4000` | Maximum default capsule size |
|
|
342
|
+
| `AUDREY_AUTOPILOT_SCOPE` | `agent` | `agent` or explicit cross-agent `shared` recall for hooks |
|
|
343
|
+
| `AUDREY_HOOK_FAIL_CLOSED` | `0` | Deny guarded actions when Audrey itself fails |
|
|
344
|
+
| `AUDREY_API_KEY` | unset | Bearer token for REST access |
|
|
345
|
+
| `AUDREY_HOST` | `127.0.0.1` | REST bind address |
|
|
346
|
+
| `AUDREY_PORT` | `7437` | REST port |
|
|
347
|
+
| `AUDREY_ENABLE_ADMIN_TOOLS` | `0` | Enable export, import, and forget operations |
|
|
348
|
+
| `AUDREY_ENABLE_SHARED_SCOPE` | `0` | Allow explicit cross-agent REST recall; admin tools also enable it |
|
|
349
|
+
| `AUDREY_PROFILE` | `0` | Include stage timing diagnostics |
|
|
350
|
+
| `AUDREY_DISABLE_WARMUP` | `0` | Disable MCP embedding warmup |
|
|
351
|
+
| `AUDREY_PRAGMA_DEFAULTS` | `1` | Set `0` to use better-sqlite3 PRAGMA defaults |
|
|
352
|
+
|
|
353
|
+
Provider secrets are never embedded in generated hook commands. `--include-secrets` applies only to MCP registration; prefer host environment injection or a secret manager.
|
|
354
|
+
|
|
355
|
+
### Production checklist
|
|
356
|
+
|
|
357
|
+
- Give every tenant or hard isolation domain its own `AUDREY_DATA_DIR`.
|
|
358
|
+
- Pin embedding and LLM providers explicitly.
|
|
359
|
+
- Back up the store before provider, dimension, or version migrations.
|
|
360
|
+
- Put the REST sidecar behind authentication and rate limits; do not expose an agent-selection header as identity.
|
|
361
|
+
- Leave REST shared scope disabled unless cross-agent retrieval is intentional and authorized by your own identity layer.
|
|
362
|
+
- Keep credentials and regulated raw content out of encoded memories.
|
|
363
|
+
- Decide retention, deletion, encryption, and audit policy before regulated use.
|
|
364
|
+
- Monitor `audrey status --json --fail-on-unhealthy`.
|
|
365
|
+
- Keep the hook runtime on a stable installed path.
|
|
366
|
+
- Load-test concurrent writers for your topology; SQLite WAL is not a distributed coordination layer.
|
|
367
|
+
|
|
368
|
+
### Benchmarks and evidence
|
|
369
|
+
|
|
370
|
+
Run the release gates locally:
|
|
371
|
+
|
|
372
|
+
```bash
|
|
373
|
+
npm test
|
|
374
|
+
npm run bench:memory:check
|
|
375
|
+
npm run bench:guard:check
|
|
376
|
+
npm run bench:guard:publication:verify
|
|
377
|
+
npm run smoke:cli
|
|
378
|
+
npm run pack:check
|
|
529
379
|
```
|
|
530
380
|
|
|
531
|
-
|
|
381
|
+
GuardBench currently contains ten local, deterministic pre-action scenarios covering repeated failures, procedures, scope changes, recovery, redaction, conflicting instructions, and noisy stores. The checked-in v1 methodology uses a mock 64-dimensional embedding provider and exists to catch regressions. A perfect local pass is not a claim about real-provider latency or production false-positive rates.
|
|
532
382
|
|
|
533
|
-
|
|
383
|
+
<!-- guardbench-summary:start -->
|
|
384
|
+
Latest local result in this checkout: 10/10 scenarios passed, 100% prevention rate, 0% false-block rate, 0 raw secret leaks, 0 published artifact leaks, and 3.501ms / 20.658ms p50/p95 Guard latency under the mock-provider methodology.
|
|
385
|
+
<!-- guardbench-summary:end -->
|
|
534
386
|
|
|
535
|
-
-
|
|
536
|
-
- [Audrey paper outline](docs/AUDREY_PAPER_OUTLINE.md)
|
|
537
|
-
- Public setup, runtime, benchmark, and command guidance is maintained in this README.
|
|
387
|
+
`benchmarks/perf-snapshot.js` measures encode and hybrid-recall p50/p95/p99 at configurable corpus sizes with machine and provider provenance. Run it on the hardware and embedding provider you plan to operate; hosted-provider latency is dominated by its network round trip.
|
|
538
388
|
|
|
539
|
-
|
|
389
|
+
The longer-term public evaluation target includes [LongMemEval](https://arxiv.org/abs/2410.10813), [MemoryAgentBench](https://arxiv.org/abs/2507.05257), and adversarial memory-poisoning cases. Relevant design directions include bitemporal knowledge graphs in [Zep/Graphiti](https://arxiv.org/abs/2501.13956), evolving memory organization in [A-MEM](https://arxiv.org/abs/2502.12110), and sleep-time agent compute in [Sleep-time Compute](https://arxiv.org/abs/2504.13171).
|
|
540
390
|
|
|
541
|
-
|
|
391
|
+
### Development
|
|
542
392
|
|
|
543
393
|
```bash
|
|
394
|
+
git clone https://github.com/Evilander/Audrey.git
|
|
395
|
+
cd Audrey
|
|
544
396
|
npm ci
|
|
545
397
|
npm run build
|
|
546
|
-
npm run lint
|
|
547
|
-
npm run format
|
|
398
|
+
npm run lint
|
|
399
|
+
npm run format:check
|
|
548
400
|
npm test
|
|
549
401
|
```
|
|
550
402
|
|
|
551
|
-
|
|
403
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md), [SECURITY.md](SECURITY.md), and [docs/MEMORY_BENCHMARKING.md](docs/MEMORY_BENCHMARKING.md).
|
|
404
|
+
|
|
405
|
+
### Maintainer release gates
|
|
406
|
+
|
|
407
|
+
These commands are intentionally documented because the paper and release evidence ledger verifies them against the public source tree:
|
|
552
408
|
|
|
553
409
|
```bash
|
|
554
|
-
npm run
|
|
555
|
-
|
|
410
|
+
npm run bench:guard:zep
|
|
411
|
+
npm run bench:guard:external:dry-run
|
|
412
|
+
npm run bench:guard:external:evidence
|
|
413
|
+
npm run bench:guard:external:evidence:strict
|
|
414
|
+
|
|
415
|
+
npm run paper:arxiv:compile
|
|
416
|
+
npm run paper:arxiv:compile:strict
|
|
417
|
+
npm run paper:launch-results
|
|
418
|
+
npm run paper:launch-results:strict
|
|
419
|
+
|
|
420
|
+
npm run release:cut:plan
|
|
421
|
+
npm run release:cut:apply
|
|
422
|
+
npm run release:readiness
|
|
423
|
+
npm run release:readiness:strict
|
|
556
424
|
npm run python:release:check
|
|
557
425
|
```
|
|
558
426
|
|
|
559
|
-
`
|
|
560
|
-
directories do not block test startup. `npm run release:gate:sandbox` remains
|
|
561
|
-
available for hosts that block child-process spawning entirely.
|
|
427
|
+
Live Zep runs require `ZEP_API_KEY`; `ZEP_GUARDBENCH_INGEST_DELAY_MS` tunes ingestion settling time. The external dry-run matrix proves adapter shape without credentials, while external evidence verification distinguishes pending runs from verified live evidence.
|
|
562
428
|
|
|
563
|
-
|
|
429
|
+
Publication packaging performs an absolute-path sweep, reserves an X URL reserve in social copy, and checks submitted artifact-url targets. Release readiness separately reports source-control state, live remote-head verification, npm registry/auth readiness, and PyPI publish readiness.
|
|
564
430
|
|
|
565
|
-
MIT.
|
|
431
|
+
MIT licensed. Built for agents that should get better at the work without becoming less accountable.
|