wendkeep 0.75.0 → 0.75.1

package/CHANGELOG.md

@@ -4,6 +4,18 @@ All notable changes to **wendkeep** are documented here. Format based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project follows
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.75.1] — 2026-08-20
+
+### Fixed
+
+- **Coalescência sem perda.** A outbox seleciona o identificador próprio de cada tipo de evento;
+  chamadas, agentes, rollups e transcripts distintos da mesma sessão não disputam mais a mesma chave.
+- **Reconciliação realmente integral.** `observer reconcile --url` e `observer memory import`
+  ignoram o cursor incremental local, consultam as revisões remotas e regeneram documentos,
+  consumo, chamadas e transcripts mesmo após restauração ou troca do Observer.
+- **Leases compatíveis com o transporte.** Locks de batch e publisher permanecem válidos além do
+  timeout máximo de request e só podem ser liberados pelo proprietário que os adquiriu.
+
 ## [0.75.0] — 2026-08-20
 
 ### Added

package/README.en.md

@@ -88,7 +88,7 @@ Decisions, dead ends, the reason you chose X over Y — gone next session. The p
 | **Cost** — what it all cost | Per‑model, cache‑aware token pricing per session — plus `cost --trend` with a run‑rate projection across the whole vault; research previews without a final rate remain unestimated. |
 | **Multi‑agent** — one vault, both agents | `init` wires the session hooks into `.claude/settings.json` *and* `.codex/hooks.json`, and every note is tagged with the agent that wrote it: Claude Code is detected from its environment, anything else is recorded as Codex. One shared graph, whichever agent you are in. |
 | **Local‑first** — no cloud, no account | Everything is plain Markdown on your disk. An optional MCP server (`@bitbonsai/mcpvault`) lets the agent read/write the vault. |
-| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, FTS5 chunks, sessions, agents, tokens, costs, calls, and transcripts in SQLite. Identities and foreign keys are project-scoped; each event is atomic. Hooks publish only what changed, while `observer reconcile` reserves full scans for explicit recovery. |
+| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, FTS5 chunks, sessions, agents, tokens, costs, calls, and transcripts in SQLite. Identities and foreign keys are project-scoped; each event is atomic. Hooks publish only what changed; `observer reconcile --url` ignores the incremental cursor to regenerate the complete projection while preserving local/remote revision baselines. |
 
 During historical migration, the Observer preserves differences between frontmatter totals and the
 ledger as explicit reconciliation rows, and disambiguates duplicate `session_id` values per file

package/README.md

@@ -88,7 +88,7 @@ Decisions, dead ends, the reason you chose X over Y — gone next session. The p
 | **Cost** — what it all cost | Per‑model, cache‑aware token pricing per session — plus `cost --trend` with a run‑rate projection across the whole vault; research previews without a final rate remain unestimated. |
 | **Multi‑agent** — one vault, both agents | `init` wires the session hooks into `.claude/settings.json` *and* `.codex/hooks.json`, and every note is tagged with the agent that wrote it: Claude Code is detected from its environment, anything else is recorded as Codex. One shared graph, whichever agent you are in. |
 | **Local‑first** — no cloud, no account | Everything is plain Markdown on your disk. An optional MCP server (`@bitbonsai/mcpvault`) lets the agent read/write the vault. |
-| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, FTS5 chunks, sessions, agents, tokens, costs, calls, and transcripts in SQLite. Identities and foreign keys are project-scoped; each event is atomic. Hooks publish only what changed, while `observer reconcile` reserves full scans for explicit recovery. |
+| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, FTS5 chunks, sessions, agents, tokens, costs, calls, and transcripts in SQLite. Identities and foreign keys are project-scoped; each event is atomic. Hooks publish only what changed; `observer reconcile --url` ignores the incremental cursor to regenerate the complete projection while preserving local/remote revision baselines. |
 
 During historical migration, the Observer preserves differences between frontmatter totals and the
 ledger as explicit reconciliation rows, and disambiguates duplicate `session_id` values per file

package/docs/en/commands/observer.md

@@ -92,8 +92,13 @@ agent sessions, cost rollups, and calls. Messages and transcripts are only sent
 that explicitly enable them. The container stores everything in
 `/data/observer.sqlite`; it does not mount `C:\GitHub` or any `.WendKeep-vault`. Markdown is only
 the text held in SQL and is recreated as files only by an explicit read-only export.
-`memory import` performs the initial load and returns file/hash parity. During migration, the
-cost/token total recorded in frontmatter is preserved through an explicit reconciliation row when
+`memory import` performs the initial load and returns file/hash parity. The remote recovery
+commands `observer reconcile --url` and `observer memory import` do not use the
+incremental cursor to decide what to omit: they regenerate documents, sessions, usage, calls, and
+transcripts. The cursor still supplies the local revision baseline and the remote tree, when
+available, supplies the highest known baseline; a transient read failure never lowers the locally
+persisted revision. During migration, the cost/token total recorded in frontmatter is preserved
+through an explicit reconciliation row when
 the detailed ledger does not add up; that row does not invent calls. Historical sessions sharing
 one `session_id` receive a canonical per-file identity so one rollup cannot overwrite the other.
 

package/docs/pt-BR/commands/observer.md

@@ -94,6 +94,11 @@ níveis de captura que os habilitam. O container grava tudo em
 `/data/observer.sqlite`; não monta `C:\GitHub` nem qualquer `.WendKeep-vault`. Markdown é aceito
 somente como conteúdo de uma coluna SQL e volta a existir como arquivo apenas pela exportação
 read-only sob demanda. `memory import` faz a carga inicial e retorna a paridade por arquivo e hash.
+Na recuperação remota, `observer reconcile --url` e `observer memory import` não usam o cursor
+incremental para decidir o que omitir: regeneram documentos, sessões, consumo, chamadas e
+transcripts. O cursor continua fornecendo o baseline local de revisão e a árvore remota, quando
+disponível, fornece o maior baseline conhecido; uma falha temporária nessa leitura nunca reduz a
+revisão persistida localmente.
 Na migração, o total de custo/token registrado no frontmatter é preservado por uma linha de
 reconciliação quando o ledger detalhado não fecha com ele; essa linha não inventa chamadas.
 Sessões históricas com o mesmo `session_id` recebem uma identidade canônica por arquivo para

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wendkeep",
-  "version": "0.75.0",
+  "version": "0.75.1",
   "description": "Vault-first persistent memory for AI coding agents, with an optional profile-aware governance runtime: OFF, FLOW, GUIDE, GOVERN, or ASSURE. Local-first and agent-agnostic (Claude Code, Codex, Cursor…).",
   "type": "module",
   "workspaces": [

package/src/observer-sql-publish.mjs

@@ -26,6 +26,7 @@ export const SQL_EVENT_BATCH_SIZE = 64;
 export const SQL_EVENT_BATCH_BYTES = 8 * 1024 * 1024;
 const REQUEST_TIMEOUT_MS = 15000;
 const MAX_REQUEST_TIMEOUT_MS = 120000;
+export const SQL_LEASE_STALE_MS = MAX_REQUEST_TIMEOUT_MS + 30000;
 const REQUEST_TIMEOUT_BYTES_STEP = 1024 * 1024;
 const CAPTURE_LEVELS = new Set(['metadata', 'messages', 'full-transcript']);
 
@@ -262,7 +263,7 @@ function dedupeEvents(events) {
   });
 }
 
-export function buildObserverSqlEventBatch({ vaultBase, projectId, input = {}, now = new Date(), state = readState(vaultBase), remoteDocuments = {}, captureLevel = process.env.WENDKEEP_OBSERVER_CAPTURE_LEVEL || 'metadata' } = {}) {
+export function buildObserverSqlEventBatch({ vaultBase, projectId, input = {}, now = new Date(), state = readState(vaultBase), remoteDocuments = {}, captureLevel = process.env.WENDKEEP_OBSERVER_CAPTURE_LEVEL || 'metadata', forceFull = false } = {}) {
   if (!vaultBase || !projectId) throw new Error('vaultBase e projectId são obrigatórios.');
   const occurredAt = isoNow(now);
   const resolvedCaptureLevel = normalizeObserverCaptureLevel(captureLevel);
@@ -289,7 +290,7 @@ export function buildObserverSqlEventBatch({ vaultBase, projectId, input = {}, n
     nextState.files[file.logicalPath] = { content_hash: contentHash, revision: revision || 1 };
     const fm = parseFrontmatter(content);
     if (fm.type === 'session') sessionContexts.push({ file, content, fm, contentHash, revision: revision || 1, sessionId: sessionIdentity.get(file.logicalPath) });
-    if (previous?.content_hash === contentHash) continue;
+    if (!forceFull && previous?.content_hash === contentHash) continue;
     changed += 1;
     events.push(documentEvent({ projectId, logicalPath: file.logicalPath, content, metadata: fm, revision: revision || 1, occurredAt }));
     if (fm.type === 'session') {
@@ -309,7 +310,7 @@ export function buildObserverSqlEventBatch({ vaultBase, projectId, input = {}, n
       const content = readFileSync(source.path, 'utf8');
       const fingerprint = hash(content);
       const previousTranscript = state.transcripts?.[source.transcriptId];
-      if (previousTranscript?.content_hash === fingerprint && previousTranscript?.coverage === resolvedCaptureLevel) continue;
+      if (!forceFull && previousTranscript?.content_hash === fingerprint && previousTranscript?.coverage === resolvedCaptureLevel) continue;
       const complete = completeTranscriptEvents({ projectId, sessionId, mainAgentId, provider, model, source, now: occurredAt, captureLevel: resolvedCaptureLevel });
       nextState.transcripts[source.transcriptId] = { content_hash: complete.fingerprint, coverage: resolvedCaptureLevel };
       const summaryId = complete.transcriptId;
@@ -467,11 +468,20 @@ function queueOutbox(vaultBase, batch) {
 
 function eventCoalesceKey(event) {
   const payload = event.payload || {};
+  const entityFields = {
+    'document.upsert': ['logical_path'],
+    'document.delete': ['logical_path'],
+    'session.upsert': ['session_id'],
+    'agent.upsert': ['agent_id'],
+    'usage.rollup': ['rollup_key'],
+    llm_call: ['call_id'],
+    'transcript.upsert': ['transcript_id'],
+  }[event.kind] || [];
+  const entityId = entityFields.map((field) => payload[field]).find(Boolean) || event.event_id;
   return [
     event.project_id,
     event.kind,
-    payload.logical_path || payload.session_id || payload.agent_id || payload.rollup_key
-      || payload.call_id || payload.transcript_id || event.event_id,
+    entityId,
   ].join('\u001f');
 }
 
@@ -480,15 +490,17 @@ function waitSync(milliseconds) { Atomics.wait(WAIT_ARRAY, 0, 0, milliseconds);
 
 function acquireBatchFileLease(path, waitMs = 0) {
   const lock = `${path}.lock`;
+  const token = `${process.pid}-${Date.now()}-${Math.random().toString(16).slice(2)}`;
   const deadline = Date.now() + Math.max(0, waitMs);
   do {
     try {
       mkdirSync(lock);
-      return lock;
+      atomicJson(join(lock, 'owner.json'), { token, pid: process.pid, acquired_at: new Date().toISOString() });
+      return { path: lock, token };
     } catch (error) {
       if (error?.code !== 'EEXIST') throw error;
       try {
-        if (Date.now() - statSync(lock).mtimeMs > 60_000) {
+        if (Date.now() - statSync(lock).mtimeMs > SQL_LEASE_STALE_MS) {
           const stale = `${lock}.stale-${process.pid}-${Date.now()}`;
           renameSync(lock, stale);
           rmSync(stale, { recursive: true, force: true });
@@ -502,8 +514,10 @@ function acquireBatchFileLease(path, waitMs = 0) {
   return null;
 }
 
-function releaseBatchFileLease(lock) {
-  if (lock) rmSync(lock, { recursive: true, force: true });
+function releaseBatchFileLease(lease) {
+  if (!lease) return;
+  const owner = readJson(join(lease.path, 'owner.json'), {});
+  if (owner.token === lease.token) rmSync(lease.path, { recursive: true, force: true });
 }
 
 /** Queue a precise writer batch and replace older pending state for the same logical scope. */
@@ -625,7 +639,7 @@ function acquirePublisherLease(vaultBase, currentTime = Date.now()) {
     if (error?.code !== 'EEXIST') throw error;
     let age = 0;
     try { age = currentTime - statSync(path).mtimeMs; } catch { return null; }
-    if (age <= 60_000) return null;
+    if (age <= SQL_LEASE_STALE_MS) return null;
     const stale = `${path}.stale-${token}`;
     try {
       renameSync(path, stale);
@@ -770,14 +784,14 @@ export async function publishObserverSqlIncremental({
   };
 }
 
-export async function publishObserverSql({ vaultBase, projectId, url = process.env.WENDKEEP_OBSERVER_URL || '', input = {}, now = new Date(), fetchImpl = globalThis.fetch, token = process.env.WENDKEEP_OBSERVER_TOKEN || '', captureLevel = process.env.WENDKEEP_OBSERVER_CAPTURE_LEVEL || 'metadata' } = {}) {
+export async function publishObserverSql({ vaultBase, projectId, url = process.env.WENDKEEP_OBSERVER_URL || '', input = {}, now = new Date(), fetchImpl = globalThis.fetch, token = process.env.WENDKEEP_OBSERVER_TOKEN || '', captureLevel = process.env.WENDKEEP_OBSERVER_CAPTURE_LEVEL || 'metadata', forceFull = false } = {}) {
   if (!vaultBase || !projectId) throw new Error('vaultBase e projectId são obrigatórios.');
   const replay = await retryObserverSqlOutbox({ vaultBase, projectId, url, fetchImpl, token });
   const state = readState(vaultBase);
-  const remoteDocuments = Object.keys(state.files || {}).length === 0
+  const remoteDocuments = forceFull || Object.keys(state.files || {}).length === 0
     ? await readRemoteDocuments({ url, projectId, fetchImpl, token })
     : {};
-  const batch = buildObserverSqlEventBatch({ vaultBase, projectId, input, now, state, remoteDocuments, captureLevel });
+  const batch = buildObserverSqlEventBatch({ vaultBase, projectId, input, now, state, remoteDocuments, captureLevel, forceFull });
   if (!batch.events.length) {
     atomicJson(statePath(vaultBase), batch.nextState);
     return { ok: true, queued: false, scanned: batch.scanned, changed: batch.changed, pending: listSqlOutbox(vaultBase).length, replay };

package/src/observer.mjs

@@ -147,6 +147,7 @@ export async function runObserver(argv = [], { write = (chunk) => process.stdout
       projectId: snapshot.project_id,
       url,
       token,
+      forceFull: true,
       captureLevel: optionValue(argv, '--capture-level') || process.env.WENDKEEP_OBSERVER_CAPTURE_LEVEL || 'metadata',
     });
     const snapshotResponse = await fetch(`${String(url).replace(/\/$/, '')}/v1/projects/${encodeURIComponent(snapshot.project_id)}/snapshot`, {
@@ -209,6 +210,7 @@ export async function runObserver(argv = [], { write = (chunk) => process.stdout
       projectId: snapshot.project_id,
       url,
       token,
+      forceFull: true,
       captureLevel: optionValue(argv, '--capture-level') || process.env.WENDKEEP_OBSERVER_CAPTURE_LEVEL || 'metadata',
     });
     const snapshotResponse = await fetch(`${String(url).replace(/\/$/, '')}/v1/projects/${encodeURIComponent(snapshot.project_id)}/snapshot`, {