wendkeep 0.85.1 → 0.86.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/README.en.md +2 -1
- package/README.md +2 -1
- package/docs/en/commands/evidence-embeddings.md +243 -0
- package/docs/en/commands/mcp.md +67 -7
- package/docs/pt-BR/commands/evidence-embeddings.md +244 -0
- package/docs/pt-BR/commands/mcp.md +66 -7
- package/hooks/evidence-context.mjs +41 -7
- package/hooks/evidence-recall.mjs +10 -0
- package/package.json +1 -1
- package/packages/mcp/src/effects.mjs +3 -2
- package/packages/mcp/src/evidence-recall.mjs +130 -0
- package/packages/mcp/src/executor.mjs +4 -0
- package/packages/mcp/src/server.mjs +31 -1
- package/packages/vault/src/evidence-embedding-plugin.mjs +531 -0
- package/packages/vault/src/evidence-index-store.mjs +360 -0
- package/packages/vault/src/evidence-recall-page.mjs +381 -0
- package/packages/vault/src/evidence-search-index.mjs +917 -0
- package/packages/vault/src/index.mjs +12 -1
- package/packages/vault/src/memory-ledger-view-base.mjs +545 -0
- package/packages/vault/src/memory-ledger-view.mjs +41 -0
- package/packages/vault/src/memory-rotation-store.mjs +967 -0
- package/packages/vault/src/memory-segment-store.mjs +820 -0
- package/packages/vault/src/memory-snapshot-store.mjs +1105 -0
- package/packages/vault/src/memory-store-base.mjs +1161 -0
- package/packages/vault/src/memory-store-core.mjs +2 -0
- package/packages/vault/src/memory-store.mjs +46 -1161
- package/src/doctor.mjs +41 -5
- package/src/evidence-search-health.mjs +221 -0
- package/src/memory-scale-health.mjs +210 -0
- package/src/observer-snapshot.mjs +87 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,32 @@ All notable changes to **wendkeep** are documented here. Format based on
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project follows
|
|
5
5
|
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.86.0] — 2026-08-28
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **Memory ledger bounded para Vaults longos.** Snapshots determinísticos, replay apenas do tail,
|
|
12
|
+
segmentos imutáveis com manifest encadeado e rotação/compactação transacional preservam a
|
|
13
|
+
autoridade append-only, com dry-run, journal, receipt, recovery e benchmarks sintéticos de
|
|
14
|
+
100 mil eventos.
|
|
15
|
+
- **Recall incremental, paginado e indexado.** Estado por documento/chunk, cursor vinculado ao
|
|
16
|
+
índice/consulta/filtros/`as_of`, budgets UTF-8 exatos, sidecar lexical bounded e SQLite FTS5
|
|
17
|
+
opcional evitam scans integrais sem transformar artefatos derivados em autoridade; o MCP expõe
|
|
18
|
+
a mesma leitura paginada e bounded.
|
|
19
|
+
- **Contrato opcional de embeddings locais.** `wendkeep/vault` exporta manifest versionado,
|
|
20
|
+
integrity-bound e in-process para reranquear somente um prefixo de candidatos; fica desligado
|
|
21
|
+
por padrão, proíbe rede/retenção, limita batch/bytes, preserva proveniência e degrada para a
|
|
22
|
+
ordem lexical sem incluir modelo, provider ou vector database no Core.
|
|
23
|
+
|
|
24
|
+
### Changed
|
|
25
|
+
|
|
26
|
+
- **Saúde de escala no `doctor` e Observer.** Snapshots operacionais publicam somente contagens,
|
|
27
|
+
bytes, atualidade, backend e estados sanitizados de memória/recall, sem rebuild, leitura do
|
|
28
|
+
corpus ou vazamento de termos e paths.
|
|
29
|
+
- **Fechamento coordenado da issue #82.** A PR agregadora #124 reconcilia os slices #111–#123,
|
|
30
|
+
mantém documentação PT-BR/EN e tarball em paridade e alinha package/lock/CHANGELOG para a
|
|
31
|
+
entrega minor única.
|
|
32
|
+
|
|
7
33
|
## [0.85.1] — 2026-08-25
|
|
8
34
|
|
|
9
35
|
### Fixed
|
package/README.en.md
CHANGED
|
@@ -250,6 +250,7 @@ The README is the map; the guides provide syntax, options, exit codes, examples,
|
|
|
250
250
|
| **Portable state** | `portable status/export/import/diff`, authored/runtime boundary, redaction, and the `active-work` snapshot | [Portable state](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/portable.md) |
|
|
251
251
|
| **Local-first sync** | `sync status/push/pull/conflicts/resolve`, revision/CAS, outbox, leases, and explicit conflicts | [Sync protocol](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/sync-protocol.md) |
|
|
252
252
|
| **Native MCP** | semantic tools, effects/capabilities, stdio, schemas, pagination, budgets, audit, and client configuration | [Native MCP](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/mcp.md) |
|
|
253
|
+
| **Evidence recall and embeddings** | cursor/byte budgets, bounded lexical/FTS sidecars, and an optional local plugin contract that is off by default | [Optional embedding plugin](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/evidence-embeddings.md) |
|
|
253
254
|
| **Host capabilities** | versioned lifecycle/effect matrix, degraded mode, human waivers, and evidence/Observer coverage | [Host capabilities](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/capabilities.md) |
|
|
254
255
|
| **Operating profiles** | `profile`, `flow`, always-on Keep Core, and Wend Runtime governance | [Operating profiles](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/operating-profiles.md) |
|
|
255
256
|
| **Changes and verification** | `change`, specs, sensors, TDD, evidence, Task Contracts, and archive | [Changes and verification](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/changes-and-verification.md) |
|
|
@@ -665,7 +666,7 @@ The archive **gate** blocks unless: the change scaffold is filled (G0), no task
|
|
|
665
666
|
## Notes & roadmap
|
|
666
667
|
|
|
667
668
|
- **Vault folder names default to Portuguese** (`02-Sessões`, `04-Decisões`, …). Pass `wendkeep init --locale en` for an English vault (`02-Sessions`, `04-Decisions`, English scaffold/skills). The locale is a vault property, locked at init; parsers are bilingual so mixed content never breaks.
|
|
668
|
-
- **
|
|
669
|
+
- **Default recall remains local lexical/FTS.** Embeddings require an explicitly supplied trusted local plugin with forbidden network access, budgets, and lexical fallback; Core ships no model or provider.
|
|
669
670
|
- **Transcript formats are agent‑internal** and can change between agent versions; parsing is isolated but may need updates.
|
|
670
671
|
- Installer wires **both agents**: `.claude/settings.json` + `.mcp.json` for Claude Code, `.codex/hooks.json` for Codex. **Eight Codex hooks are compatible**; `change-guard` validates the project lease in `PreToolUse` and denies mutations outside it. The four hooks without an equivalent remain Claude-only: `change-warn`, `plan-capture`, `decision-capture`, and `task-log`.
|
|
671
672
|
- **Codex hooks start untrusted.** They are enumerated but not executed until you approve the “Hooks need review” prompt; `init` cannot pre‑approve them (`--dangerously-bypass-hook-trust` is per‑invocation and stores no trusted hash). Trust is keyed to the hook's identity, so hand‑written wendkeep Codex hooks predating `0.46.0` — which ran at the 600s default because they used `timeout` instead of `timeoutSec` — cost one re‑review after `init` corrects the key. Expected, not a regression. `import --source codex` still backfills past Codex sessions either way.
|
package/README.md
CHANGED
|
@@ -250,6 +250,7 @@ The README is the map; the guides provide syntax, options, exit codes, examples,
|
|
|
250
250
|
| **Portable state** | `portable status/export/import/diff`, authored/runtime boundary, redaction, and the `active-work` snapshot | [Portable state](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/portable.md) |
|
|
251
251
|
| **Local-first sync** | `sync status/push/pull/conflicts/resolve`, revision/CAS, outbox, leases, and explicit conflicts | [Sync protocol](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/sync-protocol.md) |
|
|
252
252
|
| **Native MCP** | semantic tools, effects/capabilities, stdio, schemas, pagination, budgets, audit, and client configuration | [Native MCP](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/mcp.md) |
|
|
253
|
+
| **Evidence recall and embeddings** | cursor/byte budgets, bounded lexical/FTS sidecars, and an optional local plugin contract that is off by default | [Optional embedding plugin](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/evidence-embeddings.md) |
|
|
253
254
|
| **Host capabilities** | versioned lifecycle/effect matrix, degraded mode, human waivers, and evidence/Observer coverage | [Host capabilities](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/capabilities.md) |
|
|
254
255
|
| **Operating profiles** | `profile`, `flow`, always-on Keep Core, and Wend Runtime governance | [Operating profiles](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/operating-profiles.md) |
|
|
255
256
|
| **Changes and verification** | `change`, specs, sensors, TDD, evidence, Task Contracts, and archive | [Changes and verification](https://github.com/rogersialves/wendkeep/blob/main/docs/en/commands/changes-and-verification.md) |
|
|
@@ -665,7 +666,7 @@ The archive **gate** blocks unless: the change scaffold is filled (G0), no task
|
|
|
665
666
|
## Notes & roadmap
|
|
666
667
|
|
|
667
668
|
- **Vault folder names default to Portuguese** (`02-Sessões`, `04-Decisões`, …). Pass `wendkeep init --locale en` for an English vault (`02-Sessions`, `04-Decisions`, English scaffold/skills). The locale is a vault property, locked at init; parsers are bilingual so mixed content never breaks.
|
|
668
|
-
- **
|
|
669
|
+
- **Default recall remains local lexical/FTS.** Embeddings require an explicitly supplied trusted local plugin with forbidden network access, budgets, and lexical fallback; Core ships no model or provider.
|
|
669
670
|
- **Transcript formats are agent‑internal** and can change between agent versions; parsing is isolated but may need updates.
|
|
670
671
|
- Installer wires **both agents**: `.claude/settings.json` + `.mcp.json` for Claude Code, `.codex/hooks.json` for Codex. **Eight Codex hooks are compatible**; `change-guard` validates the project lease in `PreToolUse` and denies mutations outside it. The four hooks without an equivalent remain Claude-only: `change-warn`, `plan-capture`, `decision-capture`, and `task-log`.
|
|
671
672
|
- **Codex hooks start untrusted.** They are enumerated but not executed until you approve the “Hooks need review” prompt; `init` cannot pre‑approve them (`--dangerously-bypass-hook-trust` is per‑invocation and stores no trusted hash). Trust is keyed to the hook's identity, so hand‑written wendkeep Codex hooks predating `0.46.0` — which ran at the 600s default because they used `timeout` instead of `timeoutSec` — cost one re‑review after `init` corrects the key. Expected, not a regression. `import --source codex` still backfills past Codex sessions either way.
|
|
@@ -0,0 +1,243 @@
|
|
|
1
|
+
# Optional evidence embedding plugin
|
|
2
|
+
|
|
3
|
+
[Português](../../pt-BR/commands/evidence-embeddings.md)
|
|
4
|
+
|
|
5
|
+
## Purpose
|
|
6
|
+
|
|
7
|
+
Define a programmatic boundary for local semantic reranking without adding a model, ML runtime,
|
|
8
|
+
vector database, HTTP client, or provider dependency to WendKeep Core.
|
|
9
|
+
|
|
10
|
+
This surface is **not a CLI command**. It is exported by `wendkeep/vault` for local plugins that are
|
|
11
|
+
explicitly supplied by the application's composition root.
|
|
12
|
+
|
|
13
|
+
## When to use
|
|
14
|
+
|
|
15
|
+
Use this API when a trusted composition root needs to rerank a small candidate set already filtered
|
|
16
|
+
by lexical/FTS recall while keeping the model and adapter outside Core.
|
|
17
|
+
|
|
18
|
+
Embeddings are disabled by default.
|
|
19
|
+
|
|
20
|
+
```js
|
|
21
|
+
const result = await rerankEvidenceCandidatesWithEmbedding(rows, query);
|
|
22
|
+
// result.metrics.status === 'disabled'
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Core does not:
|
|
26
|
+
|
|
27
|
+
- discover plugins in `node_modules`;
|
|
28
|
+
- execute `import()` from Vault configuration;
|
|
29
|
+
- download models;
|
|
30
|
+
- open network connections;
|
|
31
|
+
- persist vectors;
|
|
32
|
+
- hand the full corpus to a plugin.
|
|
33
|
+
|
|
34
|
+
A plugin receives data only when the caller supplies the plugin object and sets `enabled: true`.
|
|
35
|
+
|
|
36
|
+
## When not to use
|
|
37
|
+
|
|
38
|
+
Do not use the plugin as an authoritative index, a scope-expansion mechanism, a code autoloader, or
|
|
39
|
+
a replacement for lexical filters/recall. Do not use a remote provider either: the contract requires
|
|
40
|
+
local in-process execution with no network and no retention.
|
|
41
|
+
|
|
42
|
+
### Authority contract
|
|
43
|
+
|
|
44
|
+
Authority remains ordered as follows:
|
|
45
|
+
|
|
46
|
+
```text
|
|
47
|
+
Vault Markdown/JSONL → incremental index → lexical/FTS candidates → optional reranking
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
The plugin operates only on a bounded prefix of already-filtered candidates. It cannot:
|
|
51
|
+
|
|
52
|
+
- make its index or cache authoritative;
|
|
53
|
+
- silently broaden project, session, change, or logical-path scope;
|
|
54
|
+
- hide `authority`, `validity`, or provenance;
|
|
55
|
+
- remove unprocessed candidates—they remain at the end in their original order;
|
|
56
|
+
- mutate the source objects returned by Core.
|
|
57
|
+
|
|
58
|
+
## Prerequisites
|
|
59
|
+
|
|
60
|
+
- a reviewed local plugin explicitly loaded by the application;
|
|
61
|
+
- a model/configuration pinned by SHA-256 fingerprint;
|
|
62
|
+
- explicit batch and byte budgets;
|
|
63
|
+
- candidates already filtered by lexical/FTS recall.
|
|
64
|
+
|
|
65
|
+
### Versioned manifest
|
|
66
|
+
|
|
67
|
+
Use `buildEvidenceEmbeddingManifest()` to create the manifest and
|
|
68
|
+
`createEvidenceEmbeddingPlugin()` to bind it to the `embed` function.
|
|
69
|
+
|
|
70
|
+
Required fields:
|
|
71
|
+
|
|
72
|
+
| Field | Rule |
|
|
73
|
+
|---|---|
|
|
74
|
+
| `schema_version` | `1` |
|
|
75
|
+
| `protocol_version` | `1` |
|
|
76
|
+
| `plugin_id` | stable local identifier |
|
|
77
|
+
| `plugin_version` | adapter version |
|
|
78
|
+
| `model_id` | model identifier |
|
|
79
|
+
| `model_revision` | immutable revision used by the adapter |
|
|
80
|
+
| `model_fingerprint` | `sha256:<64 hex>` for the effective model/configuration |
|
|
81
|
+
| `dimensions` | 1 through 65536 |
|
|
82
|
+
| `locality` | exactly `local` |
|
|
83
|
+
| `transport` | exactly `in-process` |
|
|
84
|
+
| `network` | exactly `forbidden` |
|
|
85
|
+
| `retention` | exactly `none` |
|
|
86
|
+
| `max_batch_size` | 1 through 512 documents |
|
|
87
|
+
| `max_input_bytes` | 1 through 4 MiB |
|
|
88
|
+
| `integrity` | hash of the canonical manifest payload |
|
|
89
|
+
|
|
90
|
+
The manifest is declarative. Core cannot sandbox arbitrary JavaScript, so install only trusted local
|
|
91
|
+
plugins and review their code. WendKeep prevents automatic loading and validates the contract before
|
|
92
|
+
handing over any evidence, but it does not turn third-party code into trusted code.
|
|
93
|
+
|
|
94
|
+
## Syntax
|
|
95
|
+
|
|
96
|
+
```text
|
|
97
|
+
buildEvidenceEmbeddingManifest(options)
|
|
98
|
+
createEvidenceEmbeddingPlugin({ manifest, embed })
|
|
99
|
+
verifyEvidenceEmbeddingPlugin(plugin)
|
|
100
|
+
rerankEvidenceCandidatesWithEmbedding(rows, query, options)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## Examples
|
|
104
|
+
|
|
105
|
+
```js
|
|
106
|
+
import {
|
|
107
|
+
buildEvidenceEmbeddingManifest,
|
|
108
|
+
createEvidenceEmbeddingPlugin,
|
|
109
|
+
rerankEvidenceCandidatesWithEmbedding,
|
|
110
|
+
} from 'wendkeep/vault';
|
|
111
|
+
|
|
112
|
+
const manifest = buildEvidenceEmbeddingManifest({
|
|
113
|
+
plugin_id: 'local.my-embedding',
|
|
114
|
+
plugin_version: '1.0.0',
|
|
115
|
+
model_id: 'local.my-model',
|
|
116
|
+
model_revision: '2026.08.27',
|
|
117
|
+
model_fingerprint: 'sha256:<model-and-configuration-hash>',
|
|
118
|
+
dimensions: 384,
|
|
119
|
+
max_batch_size: 64,
|
|
120
|
+
max_input_bytes: 262144,
|
|
121
|
+
});
|
|
122
|
+
|
|
123
|
+
const plugin = createEvidenceEmbeddingPlugin({
|
|
124
|
+
manifest,
|
|
125
|
+
async embed(request, { signal }) {
|
|
126
|
+
// Local adapter: no network access and no text retention.
|
|
127
|
+
// Return exactly one query vector and one vector for every document.id.
|
|
128
|
+
return {
|
|
129
|
+
schema_version: 1,
|
|
130
|
+
model_fingerprint: manifest.model_fingerprint,
|
|
131
|
+
query_vector: await localModel.embed(request.query.text, { signal }),
|
|
132
|
+
document_vectors: await Promise.all(request.documents.map(async (document) => ({
|
|
133
|
+
id: document.id,
|
|
134
|
+
vector: await localModel.embed(document.text, { signal }),
|
|
135
|
+
}))),
|
|
136
|
+
};
|
|
137
|
+
},
|
|
138
|
+
});
|
|
139
|
+
|
|
140
|
+
const reranked = await rerankEvidenceCandidatesWithEmbedding(rows, query, {
|
|
141
|
+
enabled: true,
|
|
142
|
+
plugin,
|
|
143
|
+
maxCandidates: 64,
|
|
144
|
+
maxInputBytes: 262144,
|
|
145
|
+
});
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
The request contains only:
|
|
149
|
+
|
|
150
|
+
```json
|
|
151
|
+
{
|
|
152
|
+
"schema_version": 1,
|
|
153
|
+
"model_fingerprint": "sha256:...",
|
|
154
|
+
"query": { "text": "..." },
|
|
155
|
+
"documents": [
|
|
156
|
+
{ "id": "<chunk_id>", "text": "<title + heading + content>" }
|
|
157
|
+
]
|
|
158
|
+
}
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
`logical_path` is not sent to the plugin. Full provenance remains on the source objects and returns
|
|
162
|
+
with the reranked order.
|
|
163
|
+
|
|
164
|
+
## Expected result
|
|
165
|
+
|
|
166
|
+
The response is fail-closed and may contain only:
|
|
167
|
+
|
|
168
|
+
```json
|
|
169
|
+
{
|
|
170
|
+
"schema_version": 1,
|
|
171
|
+
"model_fingerprint": "sha256:...",
|
|
172
|
+
"query_vector": [0.1, 0.2],
|
|
173
|
+
"document_vectors": [
|
|
174
|
+
{ "id": "<chunk_id>", "vector": [0.3, 0.4] }
|
|
175
|
+
]
|
|
176
|
+
}
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
Core rejects:
|
|
180
|
+
|
|
181
|
+
- dimensions that differ from the manifest;
|
|
182
|
+
- `NaN`, infinity, or zero-norm vectors;
|
|
183
|
+
- a divergent model fingerprint;
|
|
184
|
+
- missing, duplicate, or unknown documents;
|
|
185
|
+
- extra envelope fields;
|
|
186
|
+
- a vector count that differs from the document count.
|
|
187
|
+
|
|
188
|
+
The canonical adapter uses cosine similarity. Ties preserve the original order.
|
|
189
|
+
|
|
190
|
+
## Options and exit codes
|
|
191
|
+
|
|
192
|
+
The effective limit is always the lower value between caller and plugin manifest.
|
|
193
|
+
|
|
194
|
+
- `maxCandidates`: default 128; maximum 512;
|
|
195
|
+
- `maxInputBytes`: default 256 KiB; maximum 4 MiB;
|
|
196
|
+
- documents that do not fit remain after the reranked prefix;
|
|
197
|
+
- Core never sends a partial document;
|
|
198
|
+
- a query or first document that cannot fit produces
|
|
199
|
+
`EVIDENCE_EMBEDDING_BUDGET_EXCEEDED`.
|
|
200
|
+
|
|
201
|
+
With `required: false`—the default—a contract, budget, response, or execution failure returns the
|
|
202
|
+
original lexical order and `metrics.status: "fallback"`. With `required: true`, the typed error is
|
|
203
|
+
propagated.
|
|
204
|
+
|
|
205
|
+
Main codes:
|
|
206
|
+
|
|
207
|
+
- `EVIDENCE_EMBEDDING_PLUGIN_INVALID`;
|
|
208
|
+
- `EVIDENCE_EMBEDDING_BUDGET_EXCEEDED`;
|
|
209
|
+
- `EVIDENCE_EMBEDDING_RESPONSE_INVALID`;
|
|
210
|
+
- `EVIDENCE_EMBEDDING_EXECUTION_FAILED`.
|
|
211
|
+
|
|
212
|
+
## Safe operation
|
|
213
|
+
|
|
214
|
+
1. Keep `enabled: false` until the plugin and model hash are verified.
|
|
215
|
+
2. Run `verifyEvidenceEmbeddingPlugin(plugin)` before registering the adapter.
|
|
216
|
+
3. Pin `plugin_version`, `model_revision`, and `model_fingerprint`; do not use mutable aliases such
|
|
217
|
+
as `latest`.
|
|
218
|
+
4. Start with small budgets and compare the result against lexical/FTS recall.
|
|
219
|
+
5. Record metrics only—IDs, counts, bytes, and timings—never query, text, or vectors.
|
|
220
|
+
6. Treat a fingerprint change as a new generation of any plugin-owned cache.
|
|
221
|
+
|
|
222
|
+
## Common errors and diagnosis
|
|
223
|
+
|
|
224
|
+
When `metrics.status` is `fallback`:
|
|
225
|
+
|
|
226
|
+
1. disable the plugin; lexical/FTS recall remains the safe route;
|
|
227
|
+
2. validate the manifest and inspect `metrics.reason`;
|
|
228
|
+
3. confirm dimensions, fingerprint, and vector count;
|
|
229
|
+
4. discard only plugin-owned caches and rebuild them from `EVIDENCE_INDEX.jsonl` authority;
|
|
230
|
+
5. never delete or edit `EVIDENCE_INDEX.jsonl`, Markdown, or Core sidecars to repair an embedding
|
|
231
|
+
provider;
|
|
232
|
+
6. re-enable with `required: false` and promote to `required: true` only in an environment that
|
|
233
|
+
genuinely requires the provider.
|
|
234
|
+
|
|
235
|
+
## Next steps
|
|
236
|
+
|
|
237
|
+
This contract does not install a model or automatically add embeddings to MCP, doctor, or Observer.
|
|
238
|
+
It defines the safe, testable boundary for a future sibling adapter. Core remains complete and
|
|
239
|
+
functional without any plugin.
|
|
240
|
+
|
|
241
|
+
Use the [Native MCP](mcp.md) guide for the existing paged/lexical surface and the
|
|
242
|
+
[maintenance and diagnostics](maintenance-and-diagnostics.md) guide to inspect derived-artifact
|
|
243
|
+
health without rebuilding it.
|
package/docs/en/commands/mcp.md
CHANGED
|
@@ -63,20 +63,71 @@ wendkeep mcp config --client cursor --vault <vault>
|
|
|
63
63
|
```
|
|
64
64
|
|
|
65
65
|
Reads: `wendkeep_project_status`, `wendkeep_context_status`, `wendkeep_memory_recall`,
|
|
66
|
-
`
|
|
67
|
-
`
|
|
68
|
-
`
|
|
69
|
-
`wendkeep_observer_query`.
|
|
66
|
+
`wendkeep_evidence_recall`, `wendkeep_memory_conflicts`, `wendkeep_change_list`,
|
|
67
|
+
`wendkeep_change_show`, `wendkeep_change_status`, `wendkeep_spec_effective`,
|
|
68
|
+
`wendkeep_task_show`, `wendkeep_task_evaluate`, `wendkeep_handoff_current`,
|
|
69
|
+
`wendkeep_evidence_latest`, and `wendkeep_observer_query`.
|
|
70
70
|
|
|
71
71
|
Writes: `wendkeep_memory_assert`, `wendkeep_checkpoint_create`, `wendkeep_context_select`,
|
|
72
72
|
`wendkeep_task_claim`, `wendkeep_task_complete`, and `wendkeep_handoff_publish`.
|
|
73
73
|
|
|
74
|
+
## Paged indexed evidence recall
|
|
75
|
+
|
|
76
|
+
`wendkeep_evidence_recall` is the bounded surface for retrieving Vault evidence. It selects
|
|
77
|
+
candidates through the persistent lexical sidecar or optional SQLite/FTS5, reranks them with the
|
|
78
|
+
canonical scorer, and returns a compact page. `wendkeep_memory_recall` remains available as the
|
|
79
|
+
legacy API and does not silently inherit the new contract.
|
|
80
|
+
|
|
81
|
+
Main input:
|
|
82
|
+
|
|
83
|
+
- `project_root` and `query` are required;
|
|
84
|
+
- `limit` accepts 1 through 100 results per page;
|
|
85
|
+
- `cursor` is opaque and is valid only for the same query, filters, and logical index;
|
|
86
|
+
- `max_bytes` accepts 2 through 524288 and exactly bounds the serialized JSON in `results`; the
|
|
87
|
+
default is 64 KiB;
|
|
88
|
+
- `candidate_limit` accepts 1 through 4096 candidates;
|
|
89
|
+
- `posting_budget` accepts 1 through 1048576 visited postings;
|
|
90
|
+
- `backend` accepts `auto`, `sqlite`, or `lexical`;
|
|
91
|
+
- `filters` supports exact matches for `authority`, `validity`, `entity_type`, `project_id`,
|
|
92
|
+
`change_slug`, `session_id`, `work_session_id`, and `logical_path`, plus
|
|
93
|
+
`logical_path_prefix`. Each filter may be a string or a string list.
|
|
94
|
+
|
|
95
|
+
Example call:
|
|
96
|
+
|
|
97
|
+
```json
|
|
98
|
+
{
|
|
99
|
+
"name": "wendkeep_evidence_recall",
|
|
100
|
+
"arguments": {
|
|
101
|
+
"project_root": "<project>",
|
|
102
|
+
"query": "authentication contract",
|
|
103
|
+
"limit": 5,
|
|
104
|
+
"max_bytes": 65536,
|
|
105
|
+
"candidate_limit": 512,
|
|
106
|
+
"posting_budget": 65536,
|
|
107
|
+
"backend": "auto",
|
|
108
|
+
"filters": {
|
|
109
|
+
"authority": "verified",
|
|
110
|
+
"validity": "active",
|
|
111
|
+
"logical_path_prefix": "04-Decisions/"
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
The response contains `results`, `next_cursor`, `has_more`, `as_of`, and page counts/bytes. Each
|
|
118
|
+
result omits `content`, reports `content_bytes`, retains a bounded `excerpt`, and replaces
|
|
119
|
+
`logical_path` with `logical_ref`, a Vault-relative reference—never an absolute path. The
|
|
120
|
+
`candidates` block exposes backend, count, postings, rebuild, and fallback metadata. When the
|
|
121
|
+
candidate budget did not cover every possible match, `complete_candidate_set` is `false`; this
|
|
122
|
+
prevents a consumer from treating a truncated selection as exhaustive.
|
|
123
|
+
|
|
74
124
|
## Expected result
|
|
75
125
|
|
|
76
126
|
The handshake and `tools/list` return valid JSON-RPC. Every tool declares a versioned
|
|
77
127
|
effect/capability and schemas. Known reads skip the mutation gate while retaining explicit
|
|
78
|
-
project/worktree binding, cursor pagination,
|
|
79
|
-
|
|
128
|
+
project/worktree binding, cursor pagination, budgets, redaction, timeout, and cancellation.
|
|
129
|
+
Observer is declared unavailable below Node 22.13 without blocking Core on Node 18. Indexed recall
|
|
130
|
+
also works on Node 18 through the lexical fallback; SQLite/FTS5 remains optional.
|
|
80
131
|
|
|
81
132
|
Writes require `project_root`, `session_id`, `active_context_id`, `actor`, `reason`, the exact
|
|
82
133
|
capability, and `lease.id`/`lease.expires_at`; the executor revalidates causal authorization and CLI
|
|
@@ -89,8 +140,17 @@ outcome, code, and duration—never arguments or payloads.
|
|
|
89
140
|
- `MCP_CAPABILITY_REQUIRED` / `MCP_SCOPE_AUTH_REQUIRED`: capability missing or unauthorized.
|
|
90
141
|
- `MCP_LEASE_EXPIRED`: obtain a new authorization/lease; do not hand-edit its timestamp.
|
|
91
142
|
- `MCP_PROJECT_SCOPE_MISMATCH`: `project_root` and `worktree_root` use different bindings.
|
|
92
|
-
- `MCP_REQUEST_TOO_LARGE` / `MCP_RESPONSE_TOO_LARGE`:
|
|
143
|
+
- `MCP_REQUEST_TOO_LARGE` / `MCP_RESPONSE_TOO_LARGE`: reduce budgets and continue with the cursor.
|
|
93
144
|
- `MCP_RUNTIME_UNSUPPORTED`: use Node 22.13+ for Observer; Core remains available.
|
|
145
|
+
- `MCP_EVIDENCE_QUERY_REQUIRED`: provide a non-empty query.
|
|
146
|
+
- `MCP_EVIDENCE_CURSOR_INVALID`: the cursor was altered, became stale, or was reused with a
|
|
147
|
+
different query/filter set.
|
|
148
|
+
- `MCP_EVIDENCE_BUDGET_TOO_SMALL`: even the next result's minimum metadata cannot fit
|
|
149
|
+
`max_bytes`.
|
|
150
|
+
- `MCP_EVIDENCE_BACKEND_UNAVAILABLE`: SQLite was required but FTS5 is unavailable; use `auto` or
|
|
151
|
+
`lexical`.
|
|
152
|
+
- `MCP_EVIDENCE_ARTIFACT_UNSAFE`: a derived artifact violated the Vault's physical boundary.
|
|
153
|
+
- `MCP_EVIDENCE_RECALL_INVALID`: a filter, backend, or limit is outside the contract.
|
|
94
154
|
|
|
95
155
|
## Next steps
|
|
96
156
|
|
|
@@ -0,0 +1,244 @@
|
|
|
1
|
+
# Plugin opcional de embeddings de evidências
|
|
2
|
+
|
|
3
|
+
[English](../../en/commands/evidence-embeddings.md)
|
|
4
|
+
|
|
5
|
+
## Objetivo
|
|
6
|
+
|
|
7
|
+
Definir uma fronteira programática para reranqueamento semântico local sem adicionar modelo,
|
|
8
|
+
runtime de ML, vector database, cliente HTTP ou dependência de provider ao Core do WendKeep.
|
|
9
|
+
|
|
10
|
+
Esta superfície **não é um comando CLI**. Ela é exportada por `wendkeep/vault` para plugins locais
|
|
11
|
+
carregados explicitamente pelo composition root da aplicação.
|
|
12
|
+
|
|
13
|
+
## Quando usar
|
|
14
|
+
|
|
15
|
+
Use esta API quando um composition root confiável precisar reranquear um conjunto pequeno de
|
|
16
|
+
candidatos que o recall lexical/FTS já filtrou, mantendo o modelo e o adapter fora do Core.
|
|
17
|
+
|
|
18
|
+
Embeddings ficam desligados por padrão.
|
|
19
|
+
|
|
20
|
+
```js
|
|
21
|
+
const result = await rerankEvidenceCandidatesWithEmbedding(rows, query);
|
|
22
|
+
// result.metrics.status === 'disabled'
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
O Core:
|
|
26
|
+
|
|
27
|
+
- não procura plugins em `node_modules`;
|
|
28
|
+
- não executa `import()` a partir de configuração do Vault;
|
|
29
|
+
- não baixa modelos;
|
|
30
|
+
- não abre conexão de rede;
|
|
31
|
+
- não persiste vetores;
|
|
32
|
+
- não entrega o corpus inteiro ao plugin.
|
|
33
|
+
|
|
34
|
+
Um plugin só recebe dados quando o chamador fornece o objeto do plugin e define `enabled: true`.
|
|
35
|
+
|
|
36
|
+
## Quando não usar
|
|
37
|
+
|
|
38
|
+
Não use o plugin como índice autoritativo, mecanismo de ampliação de escopo, autoload de código ou
|
|
39
|
+
substituto para filtros/recall lexical. Também não use provider remoto: o contrato exige execução
|
|
40
|
+
local, in-process, sem rede e sem retenção.
|
|
41
|
+
|
|
42
|
+
### Contrato de autoridade
|
|
43
|
+
|
|
44
|
+
A ordem de autoridade permanece:
|
|
45
|
+
|
|
46
|
+
```text
|
|
47
|
+
Markdown/JSONL do Vault → índice incremental → candidatos lexical/FTS → reranqueamento opcional
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
O plugin atua somente sobre um prefixo bounded de candidatos já filtrados. Ele não pode:
|
|
51
|
+
|
|
52
|
+
- tornar seu índice ou cache a autoridade;
|
|
53
|
+
- ampliar silenciosamente o escopo de projeto, sessão, change ou logical path;
|
|
54
|
+
- ocultar `authority`, `validity` ou proveniência;
|
|
55
|
+
- remover candidatos não processados — eles permanecem no final, na ordem original;
|
|
56
|
+
- alterar os objetos-fonte retornados pelo Core.
|
|
57
|
+
|
|
58
|
+
## Pré-requisitos
|
|
59
|
+
|
|
60
|
+
- plugin local revisado e carregado explicitamente pela aplicação;
|
|
61
|
+
- modelo/configuração fixados por fingerprint SHA-256;
|
|
62
|
+
- budgets de batch e bytes definidos;
|
|
63
|
+
- candidatos já filtrados pelo recall lexical/FTS.
|
|
64
|
+
|
|
65
|
+
### Manifest versionado
|
|
66
|
+
|
|
67
|
+
Use `buildEvidenceEmbeddingManifest()` para criar o manifest e
|
|
68
|
+
`createEvidenceEmbeddingPlugin()` para vinculá-lo à função `embed`.
|
|
69
|
+
|
|
70
|
+
Campos obrigatórios:
|
|
71
|
+
|
|
72
|
+
| Campo | Regra |
|
|
73
|
+
|---|---|
|
|
74
|
+
| `schema_version` | `1` |
|
|
75
|
+
| `protocol_version` | `1` |
|
|
76
|
+
| `plugin_id` | identificador estável e local |
|
|
77
|
+
| `plugin_version` | versão do adapter |
|
|
78
|
+
| `model_id` | identificador do modelo |
|
|
79
|
+
| `model_revision` | revisão imutável usada pelo adapter |
|
|
80
|
+
| `model_fingerprint` | `sha256:<64 hex>` do modelo/configuração efetiva |
|
|
81
|
+
| `dimensions` | 1 a 65536 |
|
|
82
|
+
| `locality` | exatamente `local` |
|
|
83
|
+
| `transport` | exatamente `in-process` |
|
|
84
|
+
| `network` | exatamente `forbidden` |
|
|
85
|
+
| `retention` | exatamente `none` |
|
|
86
|
+
| `max_batch_size` | 1 a 512 documentos |
|
|
87
|
+
| `max_input_bytes` | 1 a 4 MiB |
|
|
88
|
+
| `integrity` | hash do payload canônico do manifest |
|
|
89
|
+
|
|
90
|
+
O manifest é declarativo. JavaScript arbitrário não pode ser sandboxado pelo Core; portanto, instale
|
|
91
|
+
somente plugins locais confiáveis e revise seu código. O WendKeep impede carregamento automático e
|
|
92
|
+
valida o contrato antes de entregar qualquer evidência, mas não transforma código de terceiros em
|
|
93
|
+
código confiável.
|
|
94
|
+
|
|
95
|
+
## Sintaxe
|
|
96
|
+
|
|
97
|
+
```text
|
|
98
|
+
buildEvidenceEmbeddingManifest(options)
|
|
99
|
+
createEvidenceEmbeddingPlugin({ manifest, embed })
|
|
100
|
+
verifyEvidenceEmbeddingPlugin(plugin)
|
|
101
|
+
rerankEvidenceCandidatesWithEmbedding(rows, query, options)
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## Exemplos
|
|
105
|
+
|
|
106
|
+
```js
|
|
107
|
+
import {
|
|
108
|
+
buildEvidenceEmbeddingManifest,
|
|
109
|
+
createEvidenceEmbeddingPlugin,
|
|
110
|
+
rerankEvidenceCandidatesWithEmbedding,
|
|
111
|
+
} from 'wendkeep/vault';
|
|
112
|
+
|
|
113
|
+
const manifest = buildEvidenceEmbeddingManifest({
|
|
114
|
+
plugin_id: 'local.minha-embedding',
|
|
115
|
+
plugin_version: '1.0.0',
|
|
116
|
+
model_id: 'local.meu-modelo',
|
|
117
|
+
model_revision: '2026.08.27',
|
|
118
|
+
model_fingerprint: 'sha256:<hash-do-modelo-e-configuracao>',
|
|
119
|
+
dimensions: 384,
|
|
120
|
+
max_batch_size: 64,
|
|
121
|
+
max_input_bytes: 262144,
|
|
122
|
+
});
|
|
123
|
+
|
|
124
|
+
const plugin = createEvidenceEmbeddingPlugin({
|
|
125
|
+
manifest,
|
|
126
|
+
async embed(request, { signal }) {
|
|
127
|
+
// Adapter local: nenhum acesso de rede e nenhuma retenção de texto.
|
|
128
|
+
// Deve devolver exatamente uma query vector e um vector para cada document.id.
|
|
129
|
+
return {
|
|
130
|
+
schema_version: 1,
|
|
131
|
+
model_fingerprint: manifest.model_fingerprint,
|
|
132
|
+
query_vector: await localModel.embed(request.query.text, { signal }),
|
|
133
|
+
document_vectors: await Promise.all(request.documents.map(async (document) => ({
|
|
134
|
+
id: document.id,
|
|
135
|
+
vector: await localModel.embed(document.text, { signal }),
|
|
136
|
+
}))),
|
|
137
|
+
};
|
|
138
|
+
},
|
|
139
|
+
});
|
|
140
|
+
|
|
141
|
+
const reranked = await rerankEvidenceCandidatesWithEmbedding(rows, query, {
|
|
142
|
+
enabled: true,
|
|
143
|
+
plugin,
|
|
144
|
+
maxCandidates: 64,
|
|
145
|
+
maxInputBytes: 262144,
|
|
146
|
+
});
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
O request contém somente:
|
|
150
|
+
|
|
151
|
+
```json
|
|
152
|
+
{
|
|
153
|
+
"schema_version": 1,
|
|
154
|
+
"model_fingerprint": "sha256:...",
|
|
155
|
+
"query": { "text": "..." },
|
|
156
|
+
"documents": [
|
|
157
|
+
{ "id": "<chunk_id>", "text": "<title + heading + content>" }
|
|
158
|
+
]
|
|
159
|
+
}
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
`logical_path` não é enviado ao plugin. A proveniência completa permanece nos objetos-fonte e volta
|
|
163
|
+
com a ordem reranqueada.
|
|
164
|
+
|
|
165
|
+
## Resultado esperado
|
|
166
|
+
|
|
167
|
+
A resposta é fail-closed e deve conter apenas:
|
|
168
|
+
|
|
169
|
+
```json
|
|
170
|
+
{
|
|
171
|
+
"schema_version": 1,
|
|
172
|
+
"model_fingerprint": "sha256:...",
|
|
173
|
+
"query_vector": [0.1, 0.2],
|
|
174
|
+
"document_vectors": [
|
|
175
|
+
{ "id": "<chunk_id>", "vector": [0.3, 0.4] }
|
|
176
|
+
]
|
|
177
|
+
}
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
O Core rejeita:
|
|
181
|
+
|
|
182
|
+
- dimensão diferente do manifest;
|
|
183
|
+
- `NaN`, infinito ou vetor de norma zero;
|
|
184
|
+
- fingerprint de modelo divergente;
|
|
185
|
+
- documento ausente, duplicado ou desconhecido;
|
|
186
|
+
- campos adicionais no envelope;
|
|
187
|
+
- quantidade de vetores diferente da quantidade de documentos.
|
|
188
|
+
|
|
189
|
+
A similaridade usada pelo adapter canônico é cosseno. Empates preservam a ordem original.
|
|
190
|
+
|
|
191
|
+
## Opções e códigos de saída
|
|
192
|
+
|
|
193
|
+
O limite efetivo é sempre o menor entre o chamador e o manifest do plugin.
|
|
194
|
+
|
|
195
|
+
- `maxCandidates`: padrão 128; máximo 512;
|
|
196
|
+
- `maxInputBytes`: padrão 256 KiB; máximo 4 MiB;
|
|
197
|
+
- documentos que não couberem permanecem depois do prefixo reranqueado;
|
|
198
|
+
- o Core nunca envia parcialmente um documento;
|
|
199
|
+
- query ou primeiro documento que não cabem produzem
|
|
200
|
+
`EVIDENCE_EMBEDDING_BUDGET_EXCEEDED`.
|
|
201
|
+
|
|
202
|
+
Com `required: false` — padrão — erro de contrato, budget, resposta ou execução devolve a ordem
|
|
203
|
+
lexical original e `metrics.status: "fallback"`. Com `required: true`, o erro tipado é propagado.
|
|
204
|
+
|
|
205
|
+
Códigos principais:
|
|
206
|
+
|
|
207
|
+
- `EVIDENCE_EMBEDDING_PLUGIN_INVALID`;
|
|
208
|
+
- `EVIDENCE_EMBEDDING_BUDGET_EXCEEDED`;
|
|
209
|
+
- `EVIDENCE_EMBEDDING_RESPONSE_INVALID`;
|
|
210
|
+
- `EVIDENCE_EMBEDDING_EXECUTION_FAILED`.
|
|
211
|
+
|
|
212
|
+
## Operação segura
|
|
213
|
+
|
|
214
|
+
1. Mantenha `enabled: false` até validar o plugin e o hash do modelo.
|
|
215
|
+
2. Execute `verifyEvidenceEmbeddingPlugin(plugin)` antes de registrar o adapter.
|
|
216
|
+
3. Fixe `plugin_version`, `model_revision` e `model_fingerprint`; não use alias mutável como
|
|
217
|
+
`latest`.
|
|
218
|
+
4. Comece com budgets pequenos e compare a ordem com o recall lexical/FTS.
|
|
219
|
+
5. Registre apenas métricas — IDs, contagens, bytes e tempos — nunca query, texto ou vetores.
|
|
220
|
+
6. Trate mudança de fingerprint como uma geração nova de qualquer cache pertencente ao plugin.
|
|
221
|
+
|
|
222
|
+
## Erros comuns e diagnóstico
|
|
223
|
+
|
|
224
|
+
Quando `metrics.status` for `fallback`:
|
|
225
|
+
|
|
226
|
+
1. desative o plugin; o recall lexical/FTS continua sendo a rota segura;
|
|
227
|
+
2. valide o manifest e confira `metrics.reason`;
|
|
228
|
+
3. confirme dimensão, fingerprint e quantidade dos vetores;
|
|
229
|
+
4. descarte somente caches pertencentes ao plugin e reconstrua-os pela autoridade
|
|
230
|
+
`EVIDENCE_INDEX.jsonl`;
|
|
231
|
+
5. nunca apague ou edite `EVIDENCE_INDEX.jsonl`, Markdown ou sidecars do Core para reparar um
|
|
232
|
+
provider de embeddings;
|
|
233
|
+
6. reative com `required: false` e promova para `required: true` apenas em um ambiente que realmente
|
|
234
|
+
exige o provider.
|
|
235
|
+
|
|
236
|
+
## Próximos passos
|
|
237
|
+
|
|
238
|
+
Este contrato não instala um modelo nem integra embeddings automaticamente ao MCP, doctor ou
|
|
239
|
+
Observer. Ele define a fronteira segura e testável para um adapter irmão futuro. O Core continua
|
|
240
|
+
completo e funcional sem qualquer plugin.
|
|
241
|
+
|
|
242
|
+
Use o guia de [MCP nativo](mcp.md) para a superfície paginada/lexical já exposta e o guia de
|
|
243
|
+
[manutenção e diagnóstico](maintenance-and-diagnostics.md) para inspecionar a saúde dos artefatos
|
|
244
|
+
derivados sem reconstruí-los.
|