wendkeep 0.86.0 → 0.88.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.githooks/commit-msg +16 -0
- package/.githooks/prepare-commit-msg +16 -0
- package/CHANGELOG.md +32 -0
- package/README.en.md +3 -1
- package/README.md +3 -1
- package/docs/en/commands/commit.md +159 -0
- package/docs/en/commands/observer-security.md +154 -0
- package/docs/en/commands/observer.md +30 -12
- package/docs/pt-BR/commands/commit.md +159 -0
- package/docs/pt-BR/commands/observer-security.md +154 -0
- package/docs/pt-BR/commands/observer.md +30 -12
- package/hooks/observer-publish.mjs +3 -1
- package/package.json +6 -2
- package/packages/cli/src/index.mjs +11 -1
- package/packages/commit/package.json +6 -0
- package/packages/commit/src/cli.mjs +89 -0
- package/packages/commit/src/commit-input.mjs +181 -0
- package/packages/commit/src/commit-message.mjs +51 -0
- package/packages/commit/src/commit-policy.mjs +144 -0
- package/packages/commit/src/git-runtime.mjs +428 -0
- package/packages/commit/src/index.mjs +28 -0
- package/packages/commit/src/proof-validation.mjs +443 -0
- package/packages/mcp/src/executor.mjs +35 -2
- package/packages/observer/package.json +16 -0
- package/packages/observer/src/audit.mjs +1 -0
- package/packages/observer/src/authz.mjs +38 -0
- package/packages/observer/src/encryption.mjs +75 -0
- package/packages/observer/src/index.mjs +7 -0
- package/packages/observer/src/policy.mjs +305 -0
- package/packages/observer/src/purge.mjs +100 -0
- package/packages/observer/src/redaction.mjs +54 -0
- package/packages/observer/src/retention.mjs +39 -0
- package/packages/observer/src/token-registry.mjs +122 -0
- package/schema/commit-message-v1.schema.json +75 -0
- package/schema/observer/006-observer-security.sql +64 -0
- package/schema/observer-policy-v1.schema.json +63 -0
- package/schema/sync-event-v1.schema.json +10 -0
- package/scripts/validate-commit-range.mjs +244 -0
- package/src/doctor.mjs +7 -0
- package/src/git-commit-hooks.mjs +112 -0
- package/src/init.mjs +13 -0
- package/src/observer-auth.mjs +8 -0
- package/src/observer-privacy.mjs +7 -3
- package/src/observer-publish.mjs +31 -0
- package/src/observer-server.mjs +179 -20
- package/src/observer-sql-migrate.mjs +5 -2
- package/src/observer-sql-publish.mjs +114 -39
- package/src/observer-sql-store.mjs +299 -45
- package/src/observer-transcript-store.mjs +23 -8
- package/src/observer.mjs +145 -12
- package/src/skills-seed.mjs +79 -0
- package/src/sync-protocol.mjs +20 -0
- package/web/observer/app.mjs +107 -31
- package/web/observer/index.html +7 -0
- package/web/observer/styles.css +5 -0
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
#!/bin/sh
|
|
2
|
+
set -eu
|
|
3
|
+
|
|
4
|
+
CLI=${WENDKEEP_COMMIT_CLI:-}
|
|
5
|
+
if [ -z "$CLI" ]; then
|
|
6
|
+
if [ -f "./node_modules/wendkeep/bin/wendkeep.mjs" ]; then
|
|
7
|
+
CLI=./node_modules/wendkeep/bin/wendkeep.mjs
|
|
8
|
+
elif [ -f "./bin/wendkeep.mjs" ]; then
|
|
9
|
+
CLI=./bin/wendkeep.mjs
|
|
10
|
+
else
|
|
11
|
+
echo "WENDKEEP_COMMIT_CLI_MISSING: install wendkeep or run wendkeep doctor" >&2
|
|
12
|
+
exit 2
|
|
13
|
+
fi
|
|
14
|
+
fi
|
|
15
|
+
|
|
16
|
+
exec node "$CLI" commit validate --message-file "$1" --consume-context
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
#!/bin/sh
|
|
2
|
+
set -eu
|
|
3
|
+
|
|
4
|
+
CLI=${WENDKEEP_COMMIT_CLI:-}
|
|
5
|
+
if [ -z "$CLI" ]; then
|
|
6
|
+
if [ -f "./node_modules/wendkeep/bin/wendkeep.mjs" ]; then
|
|
7
|
+
CLI=./node_modules/wendkeep/bin/wendkeep.mjs
|
|
8
|
+
elif [ -f "./bin/wendkeep.mjs" ]; then
|
|
9
|
+
CLI=./bin/wendkeep.mjs
|
|
10
|
+
else
|
|
11
|
+
echo "WENDKEEP_COMMIT_CLI_MISSING: install wendkeep or run wendkeep doctor" >&2
|
|
12
|
+
exit 2
|
|
13
|
+
fi
|
|
14
|
+
fi
|
|
15
|
+
|
|
16
|
+
exec node "$CLI" commit prepare --message-file "$1" --source "${2:-}" --commit "${3:-}"
|
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,38 @@ All notable changes to **wendkeep** are documented here. Format based on
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project follows
|
|
5
5
|
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.88.0] — 2026-08-29
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **Políticas oficiais e RBAC do Observer por projeto.** Captura de transcripts, uso, memória e
|
|
12
|
+
sync passa por policy única, redaction por classe/campo e matriz explícita de papéis e scopes;
|
|
13
|
+
tokens são armazenados somente por hash, escopados, expirados, revogáveis e rotacionáveis.
|
|
14
|
+
- **Ciclo de vida auditável.** Retenção e purge idempotentes ficam disponíveis por CLI e endpoint,
|
|
15
|
+
com receipts retriáveis, tratamento de chegadas tardias e reconciliação segura do índice FTS.
|
|
16
|
+
|
|
17
|
+
### Security
|
|
18
|
+
|
|
19
|
+
- **Criptografia fail-closed no Observer.** Quando exigida, ingestão, outbox e publicação recusam
|
|
20
|
+
plaintext; o backfill protege dados legados antes de leituras, e backups de migração v5 usam
|
|
21
|
+
AES-256-GCM, manifest de integridade, permissões restritas e restore autenticado.
|
|
22
|
+
- **Bootstrap remoto sem credencial irrestrita.** Server, MCP, CLI e Compose usam o registry
|
|
23
|
+
hash-only com projeto, papel, scopes e expiração explícitos, incluindo revogação e rotação.
|
|
24
|
+
|
|
25
|
+
## [0.87.0] — 2026-08-29
|
|
26
|
+
|
|
27
|
+
### Added
|
|
28
|
+
|
|
29
|
+
- **Política universal de commits baseada em evidências.** O novo pacote `wendkeep/commit`, a CLI,
|
|
30
|
+
os hooks Git e o gate de intervalo produzem e validam mensagens determinísticas a partir de
|
|
31
|
+
autoridade, tarefas, spec, escopo e sensores rederiváveis no checkout exato do commit.
|
|
32
|
+
|
|
33
|
+
### Security
|
|
34
|
+
|
|
35
|
+
- **Prova remota limitada ao que o CI consegue rederivar.** Evidências causais locais permanecem
|
|
36
|
+
não publicadas; alegações remotas indisponíveis, provas circulares, sensores divergentes,
|
|
37
|
+
adulterações de escopo e caminhos privados falham de modo fechado antes da persistência.
|
|
38
|
+
|
|
7
39
|
## [0.86.0] — 2026-08-28
|
|
8
40
|
|
|
9
41
|
### Added
|
package/README.en.md
CHANGED
|
@@ -151,6 +151,7 @@ npx wendkeep init --no-companions --no-mcp --yes # zero companions,
|
|
|
151
151
|
| `--no-mcp` | Skip **wendkeep's own** vault MCP (`wendkeep-vault`). Companion MCPs still follow `--companions`. |
|
|
152
152
|
| `--no-colors` | Skip the Obsidian color system (`.obsidian` snippet + graph groups). |
|
|
153
153
|
| `--vscode-worktree-tasks` | Create a local, Git-excluded `.vscode/tasks.json` for creating/listing/opening/finishing worktrees; never overwrites an existing file. Also accepted by `sync`. |
|
|
154
|
+
| `--git-commit-hooks` | Opt in to copying commit hooks into `.githooks/` and setting local `core.hooksPath`; custom hooks are preserved without `--force`. |
|
|
154
155
|
| `--yes`, `-y` | Non-interactive; accept defaults (skips the language / vault / companion prompts). |
|
|
155
156
|
| `--force` | Overwrite existing wendkeep config blocks. |
|
|
156
157
|
|
|
@@ -259,7 +260,8 @@ The README is the map; the guides provide syntax, options, exit codes, examples,
|
|
|
259
260
|
| **Notes and knowledge** | BUG/APR/ADR, repairs, renumbering, lessons, and dashboard | [Notes and knowledge](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/notes-and-knowledge.md) |
|
|
260
261
|
| **Costs and observability** | safe dry-run, tri-state, aggregation, trends, and historical rebuild | [Costs and observability](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/costs-and-observability.md) |
|
|
261
262
|
| **Maintenance and diagnostics** | doctor, frontier/manifest freshness, drift, version, and help | [Maintenance and diagnostics](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/maintenance-and-diagnostics.md) |
|
|
262
|
-
| **
|
|
263
|
+
| **Evidence-based commits** | `wendkeep commit`, tasks derived from contracts, tests only from canonical execution bound to the exact SHA, opt-in Git hooks, privacy, and remote range gate | [Commits](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/commit.md) |
|
|
264
|
+
| **Local Observer** | `observer serve`, scoped/expiring hash-only bootstrap, incremental publishing with canonical timestamps, `reconcile`, policy, stable structural identities, content-bound hashes, deletion-safe capture, required encryption, retention, purge, and multi-project index | [Local Observer](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/observer.md) · [Security](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/observer-security.md) |
|
|
263
265
|
|
|
264
266
|
Operations that deserve step-by-step guidance: [verify and exits 0/1/2](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/verify.md),
|
|
265
267
|
[causal TDD attestation](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/tdd.md),
|
package/README.md
CHANGED
|
@@ -151,6 +151,7 @@ npx wendkeep init --no-companions --no-mcp --yes # zero companions,
|
|
|
151
151
|
| `--no-mcp` | Skip **wendkeep's own** vault MCP (`wendkeep-vault`). Companion MCPs still follow `--companions`. |
|
|
152
152
|
| `--no-colors` | Skip the Obsidian color system (`.obsidian` snippet + graph groups). |
|
|
153
153
|
| `--vscode-worktree-tasks` | Create a local, Git-excluded `.vscode/tasks.json` for creating/listing/opening/finishing worktrees; never overwrites an existing file. Also accepted by `sync`. |
|
|
154
|
+
| `--git-commit-hooks` | Opt in to copying commit hooks into `.githooks/` and setting local `core.hooksPath`; custom hooks are preserved without `--force`. |
|
|
154
155
|
| `--yes`, `-y` | Non-interactive; accept defaults (skips the language / vault / companion prompts). |
|
|
155
156
|
| `--force` | Overwrite existing wendkeep config blocks. |
|
|
156
157
|
|
|
@@ -259,7 +260,8 @@ The README is the map; the guides provide syntax, options, exit codes, examples,
|
|
|
259
260
|
| **Notes and knowledge** | BUG/APR/ADR, repairs, renumbering, lessons, and dashboard | [Notes and knowledge](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/notes-and-knowledge.md) |
|
|
260
261
|
| **Costs and observability** | safe dry-run, tri-state, aggregation, trends, and historical rebuild | [Costs and observability](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/costs-and-observability.md) |
|
|
261
262
|
| **Maintenance and diagnostics** | doctor, frontier/manifest freshness, drift, version, and help | [Maintenance and diagnostics](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/maintenance-and-diagnostics.md) |
|
|
262
|
-
| **
|
|
263
|
+
| **Evidence-based commits** | `wendkeep commit`, tasks derived from contracts, tests only from canonical execution bound to the exact SHA, opt-in Git hooks, privacy, and remote range gate | [Commits](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/commit.md) |
|
|
264
|
+
| **Local Observer** | `observer serve`, scoped/expiring hash-only bootstrap, incremental publishing with canonical timestamps, `reconcile`, policy, stable structural identities, content-bound hashes, deletion-safe capture, required encryption, retention, purge, and multi-project index | [Local Observer](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/observer.md) · [Security](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/observer-security.md) |
|
|
263
265
|
|
|
264
266
|
Operations that deserve step-by-step guidance: [verify and exits 0/1/2](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/verify.md),
|
|
265
267
|
[causal TDD attestation](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/tdd.md),
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# Evidence-based commits
|
|
2
|
+
|
|
3
|
+
**English** · [Português](../../pt-BR/commands/commit.md)
|
|
4
|
+
|
|
5
|
+
## Purpose
|
|
6
|
+
|
|
7
|
+
Produce the same auditable message from Codex, Claude Code, or another Git client using typed input,
|
|
8
|
+
public references, and a staged-index summary only. The deterministic kernel does not read the
|
|
9
|
+
Vault, `.brain`, session registries, or the network.
|
|
10
|
+
|
|
11
|
+
## When to use
|
|
12
|
+
|
|
13
|
+
Use before `feat`, `fix`, `refactor`, or `perf` implementation commits that must record causal
|
|
14
|
+
authority, tasks, tests, scope, and verifiable evidence consistently across harnesses.
|
|
15
|
+
|
|
16
|
+
## When not to use
|
|
17
|
+
|
|
18
|
+
Do not use it to invent proof, publish private content, rewrite history, or automate pushes.
|
|
19
|
+
`docs`, `test`, and `chore` commits need no context only when every changed file is objectively
|
|
20
|
+
documentation/test material. Product changes require the governed body regardless of type.
|
|
21
|
+
|
|
22
|
+
## Prerequisites
|
|
23
|
+
|
|
24
|
+
Run inside a Git repository with WendKeep installed locally and the selected product files already
|
|
25
|
+
present in the staged index.
|
|
26
|
+
|
|
27
|
+
## Syntax
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
npx --no-install wendkeep commit context --input <json|-> [--json]
|
|
31
|
+
npx --no-install wendkeep commit context --clear [--json]
|
|
32
|
+
npx --no-install wendkeep commit render --input <json|->
|
|
33
|
+
npx --no-install wendkeep commit prepare --message-file <path> [--source <source>]
|
|
34
|
+
npx --no-install wendkeep commit validate --message-file <path> [--json]
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Options and exit codes
|
|
38
|
+
|
|
39
|
+
- Exit `0`: context written/cleared or message valid.
|
|
40
|
+
- Exit `1`: invalid governed message.
|
|
41
|
+
- Exit `2`: invalid argument, JSON, Git state, privacy boundary, or stale context.
|
|
42
|
+
- `--consume-context` is reserved for the `commit-msg` wrapper and removes context after validation.
|
|
43
|
+
|
|
44
|
+
## Opt-in installation
|
|
45
|
+
|
|
46
|
+
The default `init` does not enable Git hooks. To copy the portable wrappers and set
|
|
47
|
+
`core.hooksPath=.githooks` for this repository only:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
npx --no-install wendkeep init --git-commit-hooks --yes
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
Custom hooks are never overwritten silently. When `init` finds a conflict it preserves the file.
|
|
54
|
+
Review it and rerun with `--force` only when replacement is intended; the previous file is retained
|
|
55
|
+
as `.bak`.
|
|
56
|
+
A custom `core.hooksPath` is also a conflict and remains untouched without `--force`.
|
|
57
|
+
|
|
58
|
+
## Examples
|
|
59
|
+
|
|
60
|
+
### Prepare a commit
|
|
61
|
+
|
|
62
|
+
Create JSON matching `schema/commit-message-v1.schema.json`. Declare authority and evidence
|
|
63
|
+
references, but do not provide `tasks`, `tests`, `fresh`, or `verified`. Runtime derives tasks from
|
|
64
|
+
completed canonical Task Contracts. Tests come only from `[sensor:<id>]` sensors executed by the
|
|
65
|
+
collector; `[phase:verify]` alone is never a result. Sensors
|
|
66
|
+
declared in an Envelope must exactly match the canonical reexecution in IDs, configuration,
|
|
67
|
+
command, severity, and result; only that reexecution emits a `Tests` line. The remote gate
|
|
68
|
+
re-executes each sensor in the checkout for that exact commit SHA. Every
|
|
69
|
+
published reference gets a re-derived SHA-256 digest. ADR/design validate artifact ID and path;
|
|
70
|
+
tasks containing `[req:]` require a versioned, sanitized `spec` reference that defines every
|
|
71
|
+
requirement. Evidence Envelope, Verdict, receipt, and TDD attestation may participate in local
|
|
72
|
+
validation, but are omitted from remote Evidence: worktree/session/branch IDs are not published,
|
|
73
|
+
and self-contained consistency is not promoted to proof. A message claiming them as
|
|
74
|
+
`fresh`/`verified` is rejected with `WENDKEEP_COMMIT_REMOTE_PROOF_UNAVAILABLE`. The fixed trailers
|
|
75
|
+
`Remote-Proof-Scope: git,authority,tasks,spec,sensors` and `Local-Causal-Proof: unpublished` make
|
|
76
|
+
that boundary explicit. Range re-derives authority/artifacts, task/spec, Git `Scope`, and
|
|
77
|
+
config/sensors from the SHA, and only canonical reexecution emits `Tests`. `Co-Authored-By` is
|
|
78
|
+
omitted until a trusted identity registry can resolve it.
|
|
79
|
+
|
|
80
|
+
The normal authority is the causal ADR:
|
|
81
|
+
|
|
82
|
+
```json
|
|
83
|
+
{ "authority": { "kind": "adr", "adr": "ADR-1234", "ref": "docs/ADR-1234.md", "issue": "#123" } }
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Only when no causal change/ADR exists may the native harness declare the fallback below. `issue`
|
|
87
|
+
must be `#NNN` and `design` must be versioned in the same commit under
|
|
88
|
+
`docs/superpowers/specs/` or `plans/`:
|
|
89
|
+
|
|
90
|
+
```json
|
|
91
|
+
{
|
|
92
|
+
"authority": {
|
|
93
|
+
"kind": "native",
|
|
94
|
+
"issue": "#40",
|
|
95
|
+
"design": "docs/superpowers/specs/approved-design.md"
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Runtime observes effective profile `OFF`, no causal context/change/lease or ADR, and the issue in the
|
|
101
|
+
design. This mode emits unique `Authority: native-no-causal-change`, `Issue`, and `Design` trailers. Free text, unversioned
|
|
102
|
+
design, stale/unverified proof, or a missing body/tests fail closed.
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
git add <product-files>
|
|
106
|
+
npx --no-install wendkeep commit context --input commit-input.json
|
|
107
|
+
git commit -m "feat(scope): draft"
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
`commit context` calculates the staged diff SHA-256 and stores the sanitized context at
|
|
111
|
+
`.git/wendkeep-commit-input.json`, outside the working tree. `prepare-commit-msg` replaces the draft
|
|
112
|
+
with the canonical message; `commit-msg` validates it and consumes the context. If the index changes,
|
|
113
|
+
the context becomes stale and must be recreated.
|
|
114
|
+
`commit-msg` rereads context, compares the complete message and staged hash/files, and consumes
|
|
115
|
+
context only after success. Merge, squash, and amend clear incompatible context so it cannot leak
|
|
116
|
+
to the next commit. `--message-file` stays inside the repository or Git directory.
|
|
117
|
+
|
|
118
|
+
Other commands:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
npx --no-install wendkeep commit render --input commit-input.json
|
|
122
|
+
npx --no-install wendkeep commit validate --message-file .git/COMMIT_EDITMSG
|
|
123
|
+
npx --no-install wendkeep commit context --clear
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Objectively trivial commits remain unchanged. Implementation
|
|
127
|
+
commits (`feat`, `fix`, `refactor`, and `perf`) require a Conventional Commit subject with an ADR or
|
|
128
|
+
the restricted native fallback above, Capability, Evidence, Tasks, Tests, and Scope sections, the
|
|
129
|
+
staged hash, and a `WendKeep-Commit: v1` trailer. Amend, merge, and squash never receive duplicate or
|
|
130
|
+
invented proof.
|
|
131
|
+
|
|
132
|
+
## Privacy and fail-closed behavior
|
|
133
|
+
|
|
134
|
+
- Embedded absolute paths, any configured/default Vault, `.brain`, session registries, PII, and secrets are rejected
|
|
135
|
+
before persistence.
|
|
136
|
+
- `reported`, `legacy-unbound`, `stale`, and `unproven` evidence cannot be presented as proof.
|
|
137
|
+
- Context holds sanitized references and diff metadata, never private Vault content.
|
|
138
|
+
- `--no-verify` is not accepted: CI validates every new commit, including merges and novel resolutions.
|
|
139
|
+
|
|
140
|
+
## Expected result
|
|
141
|
+
|
|
142
|
+
A deterministic, self-contained message with no private material, the hash of the committed index,
|
|
143
|
+
and coherent causal trailers.
|
|
144
|
+
|
|
145
|
+
## Common errors and diagnosis
|
|
146
|
+
|
|
147
|
+
`wendkeep doctor` reports `[commit-hooks] healthy`, `disabled`, `missing`, or `drift` and remains
|
|
148
|
+
read-only. After review, recover missing or divergent files with:
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
npx --no-install wendkeep init --git-commit-hooks --force --yes
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
When abandoning a commit, remove only its transient context with
|
|
155
|
+
`wendkeep commit context --clear`. No command rewrites history or pushes automatically.
|
|
156
|
+
|
|
157
|
+
## Next steps
|
|
158
|
+
|
|
159
|
+
Review the generated message, commit it, and let the PR range gate detect any local bypass.
|
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
# Observer security
|
|
2
|
+
|
|
3
|
+
**English** · [Português](../../pt-BR/commands/observer-security.md)
|
|
4
|
+
|
|
5
|
+
## Purpose
|
|
6
|
+
|
|
7
|
+
The Observer is a local/team read model, never a new authority over the Vault, specs, memory, or
|
|
8
|
+
sync. Its threat model assumes a compromised remote host, stolen token, curious operator, copied
|
|
9
|
+
database/outbox, adversarial payload, and interrupted purge. Host/Origin validation remains active;
|
|
10
|
+
mutations and sensitive reads fail closed, including on loopback.
|
|
11
|
+
|
|
12
|
+
| Class | Default | Primary risk |
|
|
13
|
+
|---|---|---|
|
|
14
|
+
| document | `metadata` | full memory/decisions |
|
|
15
|
+
| transcript | `metadata` | conversation and tools |
|
|
16
|
+
| prompt/response | `redacted` | PII and secrets |
|
|
17
|
+
| usage | `aggregate` | cost and operational identity |
|
|
18
|
+
| audit/receipt | minimal metadata | deleting the proof itself |
|
|
19
|
+
|
|
20
|
+
Policy v1 restricts by class, `project_id`, path glob, and `entity_type`. Later rules win only in
|
|
21
|
+
the matching project. Redaction covers Bearer values, URL/connection-string credentials, access
|
|
22
|
+
keys, email, phone, and safe configurable regular expressions. Its schema is
|
|
23
|
+
`schema/observer-policy-v1.schema.json`.
|
|
24
|
+
With `transcript_capture: messages`, arrays, JSONL, and the canonical `{messages:[...]}` envelope
|
|
25
|
+
retain only `user|assistant|system` messages with string `role`/`content` after redaction; extra
|
|
26
|
+
fields, tool records, and malformed entries are dropped or fail closed.
|
|
27
|
+
Explicit policy is the publisher's sole capture authority; `WENDKEEP_OBSERVER_CAPTURE_LEVEL` is
|
|
28
|
+
only translated into policy for legacy callers that supplied no policy file, and can never elevate
|
|
29
|
+
or suppress explicit `none|metadata|messages|full` or `selected` documents.
|
|
30
|
+
For document/transcript upserts, `content_hash` always binds the final content after capture and
|
|
31
|
+
redaction; metadata/selected capture uses the SHA-256 of empty content. Document deletions remain
|
|
32
|
+
effective even when content capture is `none`, preserve their path/revision/operation metadata,
|
|
33
|
+
and never carry stale content or content hashes.
|
|
34
|
+
Redaction never rewrites validated structural identity fields such as project/event/entity IDs,
|
|
35
|
+
logical paths, revisions, or operations. Path privacy is enforced fail-closed by project/path
|
|
36
|
+
capture rules; it is not implemented by renaming a storage key through a content-redaction rule.
|
|
37
|
+
The per-event structural contract also preserves accepted snake/camel aliases, document/session/
|
|
38
|
+
agent/call/transcript/rollup keys, timestamps, roles, status, coverage, model/pricing dimensions,
|
|
39
|
+
workflow, and source provenance. `title`, `summary`, `agent_name`, content, prompt/response, and
|
|
40
|
+
metadata remain redactable display/content fields.
|
|
41
|
+
During incremental publishing, missing or blank turn timestamps inherit the batch's canonical
|
|
42
|
+
instant, numeric epoch milliseconds are normalized to ISO 8601, and invalid non-empty values fail
|
|
43
|
+
closed before policy/store; the event and payload use the same instant.
|
|
44
|
+
|
|
45
|
+
## When to use
|
|
46
|
+
|
|
47
|
+
Use it before enabling Observer for real data, registering or revoking credentials, restricting
|
|
48
|
+
capture, protecting SQLite/outbox, defining retention, or deleting data with verifiable proof.
|
|
49
|
+
|
|
50
|
+
## When not to use
|
|
51
|
+
|
|
52
|
+
Do not use it as a corporate KMS/secret manager, to publish Vault/runtime data, to replace local
|
|
53
|
+
authority, or to delete tables and indexes manually. `full` capture remains opt-in and subject to
|
|
54
|
+
policy/redaction.
|
|
55
|
+
|
|
56
|
+
## Prerequisites
|
|
57
|
+
|
|
58
|
+
Use Node.js 22.13+, keep the bind on loopback, and inject tokens/keys only through environment
|
|
59
|
+
variables. The bootstrap token is registered hash-only with explicit projects and a finite expiry;
|
|
60
|
+
it is not a registry-bypassing wildcard admin. For Docker, also set
|
|
61
|
+
`WENDKEEP_OBSERVER_BOOTSTRAP_PROJECTS`, `WENDKEEP_OBSERVER_BOOTSTRAP_EXPIRES_AT`, and a 32-byte
|
|
62
|
+
hex/base64 `WENDKEEP_OBSERVER_ENCRYPTION_KEY`. The operator owns the key and external receipts.
|
|
63
|
+
|
|
64
|
+
## Syntax
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
npx wendkeep observer serve --token <token> --bootstrap-projects <p1,p2> --bootstrap-expires-at <ISO> [--bootstrap-token-id <id>] [--require-loopback-auth] [--require-encryption]
|
|
68
|
+
npx wendkeep observer security token create --project-id <project> --role <role> --scopes <scopes> --token-env <env> --expires-at <ISO>
|
|
69
|
+
npx wendkeep observer security token rotate --project-id <project> --token-id <id> --token-env <env> --expires-at <ISO> [--new-token-id <id>]
|
|
70
|
+
npx wendkeep observer security token revoke --project-id <project> --token-id <id>
|
|
71
|
+
npx wendkeep observer security policy set --project-id <project> --file <policy.json>
|
|
72
|
+
npx wendkeep observer security policy show --project-id <project>
|
|
73
|
+
npx wendkeep observer security purge --project-id <project> --before <ISO> --classes <classes> [--dry-run]
|
|
74
|
+
npx wendkeep observer security retention run --project-id <project> [--dry-run] [--operation-id <id>]
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Options and exit codes
|
|
78
|
+
|
|
79
|
+
- `viewer` reads metadata/aggregates; `auditor` may receive sensitive scopes; `publisher` ingests;
|
|
80
|
+
`admin` manages policy, purge, and recovery. Role, scope, and project must authorize together.
|
|
81
|
+
- Tokens persist only as SHA-256; expiry, rotation, and revocation take effect without restart.
|
|
82
|
+
- After rotating bootstrap credentials, update both token and token ID in the environment; restart
|
|
83
|
+
never reactivates an expired or revoked predecessor.
|
|
84
|
+
- `--token-env` names the variable holding the secret; the command never prints its value.
|
|
85
|
+
- `--require-loopback-auth` protects the whole API; sensitive reads require a token without it too.
|
|
86
|
+
- `--require-encryption` fails when external key material is missing or invalid.
|
|
87
|
+
- `WENDKEEP_OBSERVER_REQUIRE_ENCRYPTION=1` applies the same fail-closed behavior to `status`,
|
|
88
|
+
`security`, `register`, `publish`, and `reconcile`; with a configured key, every first v5 upgrade
|
|
89
|
+
creates only `.bak.enc` plus its manifest before any read/backfill.
|
|
90
|
+
- Exit `0` means the operation completed; exit `1` means invalid configuration, authorization,
|
|
91
|
+
policy, key, or operation. The hook retains fail-open exit `0` for the local workflow, but aborts
|
|
92
|
+
before persisting unsafe content.
|
|
93
|
+
|
|
94
|
+
Audit stores capability, outcome, route, and time, never a Bearer value, prompt, response, or payload.
|
|
95
|
+
|
|
96
|
+
## Examples
|
|
97
|
+
|
|
98
|
+
Explicit audited offline recovery:
|
|
99
|
+
|
|
100
|
+
```powershell
|
|
101
|
+
$env:OBSERVER_RECOVERY_TOKEN = '<strong-temporary-secret>'
|
|
102
|
+
npx wendkeep observer security token create --data-dir C:\WendKeepObserver `
|
|
103
|
+
--project-id project-a --role admin --scopes '*' --token-env OBSERVER_RECOVERY_TOKEN `
|
|
104
|
+
--expires-at 2026-09-29T12:00:00Z --reason 'offline recovery' --json
|
|
105
|
+
npx wendkeep observer security token revoke --data-dir C:\WendKeepObserver `
|
|
106
|
+
--project-id project-a --token-id <id> --reason 'recovery complete' --json
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Always dry-run purge first. The retention runner is explicit and idempotent (CLI or
|
|
110
|
+
`POST /v1/projects/:id/security/retention`), with no hidden timer:
|
|
111
|
+
|
|
112
|
+
```powershell
|
|
113
|
+
npx wendkeep observer security purge --data-dir C:\WendKeepObserver `
|
|
114
|
+
--project-id project-a --before 2026-08-01T00:00:00Z `
|
|
115
|
+
--classes documents,calls,transcripts --dry-run --json
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
```powershell
|
|
119
|
+
npx wendkeep observer security retention run --data-dir C:\WendKeepObserver `
|
|
120
|
+
--project-id project-a --operation-id scheduled-2026-08-29 --dry-run --json
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
## Expected result
|
|
124
|
+
|
|
125
|
+
TTL is independent for documents, calls, and transcripts. Counts, projection/FTS removal, events,
|
|
126
|
+
and receipt share one transaction; retry is idempotent and late old data creates new proof.
|
|
127
|
+
|
|
128
|
+
AES-256-GCM uses project/class/record/field AAD and an external `keyProvider`. The v6 backfill
|
|
129
|
+
removes plaintext and derived indexes before reads; a wrong key fails without disclosing content.
|
|
130
|
+
Structural migration `006-observer-security.sql` creates a backup, checks checksum, rolls back, and
|
|
131
|
+
supports retry. In required at-rest mode the backup is `.bak.enc`, carries a manifest/key ID and
|
|
132
|
+
restrictive permissions, fails restore with a wrong key, and leaves no plaintext `.bak` behind.
|
|
133
|
+
|
|
134
|
+
The hook applies metadata/redacted policy by default; `WENDKEEP_OBSERVER_POLICY_FILE` selects an
|
|
135
|
+
explicit policy. `WENDKEEP_OBSERVER_OUTBOX_KEY_ENV` names the outbox key variable and
|
|
136
|
+
`WENDKEEP_OBSERVER_OUTBOX_KEY_ID` identifies the key. Compose requires authentication and
|
|
137
|
+
encryption. The dashboard keeps Bearer only in memory, exports a sanitized copy, and exposes
|
|
138
|
+
Security. MCP requires scopes for calls/full search. Sync carries only `policy_ref`, without
|
|
139
|
+
duplicating tokens or authority.
|
|
140
|
+
|
|
141
|
+
## Common errors and diagnosis
|
|
142
|
+
|
|
143
|
+
- `observer_token_missing|expired|revoked`: create/rotate a scoped token or use offline recovery.
|
|
144
|
+
- `observer_project_forbidden|role_forbidden|scope_forbidden`: check project/role/scope intersection.
|
|
145
|
+
- `observer_encryption_key_unavailable|observer_decryption_failed`: check key ID/material; never
|
|
146
|
+
weaken required mode.
|
|
147
|
+
- `observer_policy_invalid`: validate fields/captures and remove invalid or explosive regexes.
|
|
148
|
+
- v6 migration failure: preserve `.pre-006-*.bak.enc` and its manifest, correct the cause, and retry.
|
|
149
|
+
|
|
150
|
+
## Next steps
|
|
151
|
+
|
|
152
|
+
Read [Local Observer](observer.md), dry-run retention, validate revoked/expired tokens, and keep the
|
|
153
|
+
receipt outside the database when external proof is required. Never publish the database, backup,
|
|
154
|
+
outbox, key, token, or `/data`.
|
|
@@ -26,7 +26,9 @@ WendKeep hooks.
|
|
|
26
26
|
|
|
27
27
|
Use Node.js 22.13 or newer for the SQL Observer. Keep Core and the remaining commands continue to
|
|
28
28
|
support Node.js 18 or newer. Explicitly register each project and set `WENDKEEP_OBSERVER_TOKEN`;
|
|
29
|
-
|
|
29
|
+
every mutation and every sensitive-content read requires a Bearer token, including on loopback.
|
|
30
|
+
Metadata and aggregates may remain locally open when `--require-loopback-auth` is omitted. See
|
|
31
|
+
[Observer security](observer-security.md).
|
|
30
32
|
|
|
31
33
|
## Syntax
|
|
32
34
|
|
|
@@ -36,7 +38,7 @@ npx wendkeep observer register --project <project> --vault <vault> --data-dir <d
|
|
|
36
38
|
npx wendkeep observer publish --project <project> --vault <vault> --data-dir <directory>
|
|
37
39
|
npx wendkeep observer reconcile --project <project> --vault <vault> --data-dir <directory> [--url http://127.0.0.1:8787]
|
|
38
40
|
npx wendkeep observer memory import --project <project> --vault <vault> --url http://127.0.0.1:8787 --token <token> --json
|
|
39
|
-
npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <directory> --token <token>
|
|
41
|
+
npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <directory> --token <token> --bootstrap-projects <p1,p2> --bootstrap-expires-at <ISO> [--require-loopback-auth] [--require-encryption]
|
|
40
42
|
```
|
|
41
43
|
|
|
42
44
|
## Options and exit codes
|
|
@@ -46,7 +48,13 @@ npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <directory>
|
|
|
46
48
|
- `--project` and `--vault` identify a project for `register`, `publish`, `reconcile`, and `memory import`.
|
|
47
49
|
- `--host` accepts only `127.0.0.1`, `localhost`, or `::1`; other hosts are rejected before
|
|
48
50
|
listening.
|
|
49
|
-
- `--token` or `WENDKEEP_OBSERVER_TOKEN`
|
|
51
|
+
- `--token` or `WENDKEEP_OBSERVER_TOKEN` supplies hash-only bootstrap material; explicit projects
|
|
52
|
+
and finite expiry are required, and all mutations/sensitive reads go through the registry;
|
|
53
|
+
`--allow-non-loopback` fails without one.
|
|
54
|
+
- `--require-loopback-auth` also requires Bearer for local metadata and aggregates and enables the
|
|
55
|
+
project's secure ingestion policy.
|
|
56
|
+
- `--require-encryption` requires a 32-byte hex/base64 `WENDKEEP_OBSERVER_ENCRYPTION_KEY`; use
|
|
57
|
+
`WENDKEEP_OBSERVER_ENCRYPTION_KEY_ID` to identify the external key.
|
|
50
58
|
- `WENDKEEP_OBSERVER_CAPTURE_LEVEL` accepts `metadata` (default, no messages), `messages`, or
|
|
51
59
|
`full-transcript`. Absolute local paths are never published.
|
|
52
60
|
- Exit `0` means success; exit `1` means configuration or operation failure; the publisher hook
|
|
@@ -57,7 +65,10 @@ npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <directory>
|
|
|
57
65
|
```powershell
|
|
58
66
|
npx wendkeep observer register --project C:\GitHub\WendKeep --vault C:\GitHub\WendKeep\.WendKeep-vault --data-dir C:\WendKeepObserver
|
|
59
67
|
$env:WENDKEEP_OBSERVER_TOKEN = '<strong-local-token>'
|
|
60
|
-
|
|
68
|
+
$env:WENDKEEP_OBSERVER_BOOTSTRAP_PROJECTS = 'project-a'
|
|
69
|
+
$env:WENDKEEP_OBSERVER_BOOTSTRAP_EXPIRES_AT = '2026-09-29T12:00:00Z'
|
|
70
|
+
$env:WENDKEEP_OBSERVER_ENCRYPTION_KEY = '<32-bytes-in-hex-or-base64>'
|
|
71
|
+
npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir C:\WendKeepObserver --token $env:WENDKEEP_OBSERVER_TOKEN --require-loopback-auth
|
|
61
72
|
$env:WENDKEEP_OBSERVER_URL = 'http://127.0.0.1:8787'
|
|
62
73
|
```
|
|
63
74
|
|
|
@@ -67,15 +78,19 @@ For local Docker:
|
|
|
67
78
|
docker compose -f docker/wendkeep-observer/compose.yaml up -d --build
|
|
68
79
|
```
|
|
69
80
|
|
|
81
|
+
Compose requires token, bootstrap allowlist/expiry, and key material; it starts with full
|
|
82
|
+
authentication and required encryption. An `encryption_required` policy refuses plaintext ingest/outbox.
|
|
83
|
+
|
|
70
84
|
## Local web dashboard
|
|
71
85
|
|
|
72
86
|
With the server running, open [http://127.0.0.1:8787/](http://127.0.0.1:8787/) in a browser. The
|
|
73
|
-
dashboard is served by the same process
|
|
74
|
-
|
|
87
|
+
dashboard is served by the same process. Enter the token in the local form: it remains only in page
|
|
88
|
+
memory, is sent as Bearer for queries, and is discarded on reload. Keep the port bound to the
|
|
89
|
+
computer loopback; do not expose this address on a network interface.
|
|
75
90
|
|
|
76
91
|
The dashboard shows the multi-project list, version, health, latest session, active change, change
|
|
77
92
|
count, and last capture time. Opening a project exposes Overview, Consumption, Sessions, Memory,
|
|
78
|
-
Changes, and
|
|
93
|
+
Changes, Sync, and Security screens. Consumption shows total cost, token categories, primary agents,
|
|
79
94
|
subagents, providers, models, daily trend, historical coverage, and calls with prompt, response,
|
|
80
95
|
and transcript content according to the selected capture level. Loading, empty, unavailable-server, conflict, no-pricing, and stale-data
|
|
81
96
|
states are visible, with manual refresh and an automatic 15-second refresh.
|
|
@@ -160,18 +175,21 @@ complete, archive, repair, or promote state.
|
|
|
160
175
|
- `POST /v1/projects/:project_id/ingest` — idempotent batches of documents, sessions, agents,
|
|
161
176
|
rollups, calls, and transcripts.
|
|
162
177
|
- `GET /v1/projects/:project_id/memory/tree` — document tree and metadata.
|
|
163
|
-
- `GET /v1/projects/:project_id/memory/document?path=...` — complete Markdown content.
|
|
178
|
+
- `GET /v1/projects/:project_id/memory/document?path=...` — complete Markdown content; requires Bearer.
|
|
164
179
|
- `GET /v1/projects/:project_id/memory/search?q=...` — ranked chunk search with matching passage
|
|
165
|
-
and provenance; uses a lexical fallback when FTS5 is unavailable.
|
|
180
|
+
and provenance; uses a lexical fallback when FTS5 is unavailable and requires Bearer.
|
|
166
181
|
- `GET /v1/projects/:project_id/sync` — mode, counts, conflicts, and latest event.
|
|
167
182
|
- `PUT /v1/projects/:project_id/sync` — compatibility configuration; SQL remains authoritative.
|
|
168
|
-
- `GET /v1/projects/:project_id/memory/export` — read-only export
|
|
183
|
+
- `GET /v1/projects/:project_id/memory/export` — read-only export sanitized by default; requires Bearer.
|
|
169
184
|
- `POST /v1/projects/:project_id/memory/events` — idempotent batch ingestion.
|
|
170
185
|
- `GET /v1/projects/:project_id/usage/summary` — filterable totals by period, change, session,
|
|
171
186
|
agent, provider, model, and role.
|
|
172
187
|
- `GET /v1/projects/:project_id/usage/breakdown` — agent, subagent, and model hierarchy.
|
|
173
|
-
- `GET /v1/projects/:project_id/usage/calls` — individual calls with prompt and response.
|
|
174
|
-
- `GET /v1/projects/:project_id/transcripts/:transcript_id` — compressed transcript validated by hash.
|
|
188
|
+
- `GET /v1/projects/:project_id/usage/calls` — individual calls with prompt and response; requires Bearer.
|
|
189
|
+
- `GET /v1/projects/:project_id/transcripts/:transcript_id` — compressed transcript validated by hash; requires Bearer.
|
|
190
|
+
- `GET /v1/projects/:project_id/security` — policy, token counts, and sanitized audit; requires admin.
|
|
191
|
+
- `PUT /v1/projects/:project_id/security/policy` — updates the effective policy without restart; requires admin.
|
|
192
|
+
- `POST /v1/projects/:project_id/security/purge` — transactional dry-run/purge with receipt; requires admin.
|
|
175
193
|
|
|
176
194
|
The `/v1` routes reject transported or expanded bodies above their limits and validate project,
|
|
177
195
|
path, revision, hash, idempotency, and isolation before writing to SQLite. Use `memory/export` for
|