wendkeep 0.70.0 → 0.72.0

package/CHANGELOG.md

@@ -4,6 +4,63 @@ All notable changes to **wendkeep** are documented here. Format based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project follows
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.72.0] — 2026-08-17
+
+### Added
+
+- **Observer SQL authority.** O volume Docker agora usa `/data/observer.sqlite` como
+  autoridade única para documentos, sessões, agentes, uso, chamadas LLM e transcripts
+  comprimidos; as migrações SQL são versionadas e idempotentes.
+- **Ingestão resiliente.** Hooks de sessão e subagentes enviam eventos idempotentes ao
+  Observer, preservam custos registrados e usam outbox local quando o container está
+  indisponível.
+- **Dashboard de Consumo.** Cada projeto ganhou resumo de tokens/custos, filtros,
+  hierarquia agente/subagente/modelo, tendência diária, chamadas e leitura de transcript.
+- **Migração sem perda.** Conteúdo legado do volume, `MEMORY_EVENTS.jsonl`, frontmatter de
+  custo e históricos sem transcript são importados sem apagar as fontes existentes;
+  históricos incompletos são marcados como `summary_only`; divergências entre frontmatter e
+  ledger ficam em linhas explícitas de reconciliação e `session_id` duplicado é desambiguado
+  por arquivo.
+
+### Changed
+
+- Markdown deixou de ser autoridade operacional no container. Ele permanece armazenado como
+  conteúdo documental no SQLite e só é materializado por exportação explícita.
+- O transporte divide lotes por quantidade e tamanho, reconhece retries do hash legado e
+  preserva transcripts grandes dentro do limite HTTP do Observer.
+- Lotes SQL agora usam gzip no transporte e são expandidos com limite controlado no Observer,
+  permitindo importar transcripts históricos que excedem 64 MB em JSON puro sem aumentar
+  indiscriminadamente o limite de requisição.
+
+## [0.71.1] — 2026-08-17
+
+### Added
+
+- **Observer memory authority.** O container local agora recebe a cópia integral das sessões,
+  decisões, bugs, aprendizados, specs, changes, CORE, DIGEST, SHARED_MEMORY e estado do brain.
+  A API oferece árvore, leitura completa, busca por corpo, sincronização, modos
+  `mirror`/`container-read`/`container-authority` e exportação read-only.
+- **Workspace de memória no navegador.** O painel local navega por projeto, sessões, memória,
+  changes, sincronização e leitor Markdown completo, sem formulário ou token no loopback.
+- **Importação do WendKeep.** `observer memory import` registra a carga inicial e informa paridade
+  por arquivo e hash; o volume Docker permanece preservado em rebuild e reinício.
+
+### Fixed
+
+- **Observer local abre diretamente.** O dashboard não exige mais token, formulário ou
+  `Authorization` para consultar a projeção no loopback; o compose também inicia sem variável de
+  token obrigatória.
+
+## [0.71.0] — 2026-08-17
+
+### Added
+
+- **Painel web local do Observer.** O servidor agora entrega uma interface read-only no navegador
+  para consultar projetos, saúde, sessões, changes e snapshots sanitizados da última captura. A interface é empacotada no
+  npm e na imagem Docker, exige o token local para consultar `/v1`, mantém o token somente no
+  `sessionStorage` da aba e
+  não amplia o contrato de dados sanitizados.
+
 ## [0.70.0] — 2026-08-17
 
 ### Added

package/README.en.md

@@ -78,7 +78,7 @@ npx wendkeep import                          # backfill past Claude + Codex sess
 
 ## The problem: the context dies when the window closes
 
-Decisions, dead ends, the reason you chose X over Y — gone next session. The pieces to fix that exist but are scattered (qmd‑sessions, memsearch, Nexus, hand‑written hooks). wendkeep ships them as one turnkey package that writes into a knowledge graph **inside the Obsidian vault you already use** — no manual setup, no snapshot to keep in sync.
+Decisions, dead ends, the reason you chose X over Y — gone next session. The pieces to fix that exist but are scattered (qmd‑sessions, memsearch, Nexus, hand‑written hooks). wendkeep ships local capture plus an optional Docker Observer that keeps complete memory browsable without depending on Obsidian.
 
 | | |
 |---|---|
@@ -88,7 +88,11 @@ Decisions, dead ends, the reason you chose X over Y — gone next session. The p
 | **Cost** — what it all cost | Per‑model, cache‑aware token pricing per session — plus `cost --trend` with a run‑rate projection across the whole vault; research previews without a final rate remain unestimated. |
 | **Multi‑agent** — one vault, both agents | `init` wires the session hooks into `.claude/settings.json` *and* `.codex/hooks.json`, and every note is tagged with the agent that wrote it: Claude Code is detected from its environment, anything else is recorded as Codex. One shared graph, whichever agent you are in. |
 | **Local‑first** — no cloud, no account | Everything is plain Markdown on your disk. An optional MCP server (`@bitbonsai/mcpvault`) lets the agent read/write the vault. |
-| **Local Observer** — many projects, one view | `wendkeep observer` keeps a local HTTP index of sanitized, idempotent snapshots; vaults remain local, the container does not mount them, and unavailable hooks use an outbox without blocking the session. |
+| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, sessions, agents, tokens, costs, calls, and transcripts in the Docker volume SQLite database; every project gets a **Consumption** tab in the read-only dashboard at `http://127.0.0.1:8787/`, loopback has no token, and unavailable hooks use a gzip outbox without blocking the session, including large transcripts. |
+
+During historical migration, the Observer preserves differences between frontmatter totals and the
+ledger as explicit reconciliation rows, and disambiguates duplicate `session_id` values per file
+without inventing calls.
 
 ## Requirements
 
@@ -111,7 +115,7 @@ npx wendkeep init
 
 1. Create the vault folder taxonomy and a templated `README.md` (default vault: `<project>/.<project-name>-vault`, e.g. `.MyApp-vault`; override with `--vault`).
 2. Write a provider-neutral **`.wendkeep.json`** binding at the project root and a matching `.brain/PROJECT.json` marker in the vault, then merge the session hooks into **`.claude/settings.json`**. The binding is provider-neutral by design: any agent resolves the same vault from its session `cwd`, with no machine-global environment variable. Older registrations already in `.claude/settings.json` are adopted automatically.
-3. Wire the Codex hooks in **`.codex/hooks.json`** — ten compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` only publishes a sanitized, fail-open projection; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
+3. Wire the Codex hooks in **`.codex/hooks.json`** — eleven compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` + `observer-publish` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` keeps the sanitized index projection and also sends the local SQL authority with documents, consumption, and transcripts; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
 4. Add the **`wendkeep-vault`** MCP server to `.mcp.json` so the agent can read/write the vault. Skip with `--no-mcp` — e.g. when the agent already has a vault MCP. (`--no-mcp` skips *only wendkeep's own* MCP; companion MCPs still follow `--companions`.)
 5. Offer to pin **companion** plugins/MCP (multi-choice; **none** pre-checked — wendkeep is a neutral harness and presumes no third-party plugin). Each is wired the most agent-agnostic way it supports:
    - **`context-mode`** — context optimizer + FTS5 memory, wired as a Claude Code plugin. It ships its own MCP server, so wendkeep deliberately adds no `.mcp.json` entry (registering both cold-started two servers at once). On non-Claude agents, add the MCP by hand: `npx -y context-mode`.

package/README.md

@@ -78,7 +78,7 @@ npx wendkeep import                          # backfill past Claude + Codex sess
 
 ## The problem: the context dies when the window closes
 
-Decisions, dead ends, the reason you chose X over Y — gone next session. The pieces to fix that exist but are scattered (qmd‑sessions, memsearch, Nexus, hand‑written hooks). wendkeep ships them as one turnkey package that writes into a knowledge graph **inside the Obsidian vault you already use** — no manual setup, no snapshot to keep in sync.
+Decisions, dead ends, the reason you chose X over Y — gone next session. The pieces to fix that exist but are scattered (qmd‑sessions, memsearch, Nexus, hand‑written hooks). wendkeep ships local capture plus an optional Docker Observer that keeps complete memory browsable without depending on Obsidian.
 
 | | |
 |---|---|
@@ -88,7 +88,11 @@ Decisions, dead ends, the reason you chose X over Y — gone next session. The p
 | **Cost** — what it all cost | Per‑model, cache‑aware token pricing per session — plus `cost --trend` with a run‑rate projection across the whole vault; research previews without a final rate remain unestimated. |
 | **Multi‑agent** — one vault, both agents | `init` wires the session hooks into `.claude/settings.json` *and* `.codex/hooks.json`, and every note is tagged with the agent that wrote it: Claude Code is detected from its environment, anything else is recorded as Codex. One shared graph, whichever agent you are in. |
 | **Local‑first** — no cloud, no account | Everything is plain Markdown on your disk. An optional MCP server (`@bitbonsai/mcpvault`) lets the agent read/write the vault. |
-| **Local Observer** — many projects, one view | `wendkeep observer` keeps a local HTTP index of sanitized, idempotent snapshots; vaults remain local, the container does not mount them, and unavailable hooks use an outbox without blocking the session. |
+| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, sessions, agents, tokens, costs, calls, and transcripts in the Docker volume SQLite database; every project gets a **Consumption** tab in the read-only dashboard at `http://127.0.0.1:8787/`, loopback has no token, and unavailable hooks use a gzip outbox without blocking the session, including large transcripts. |
+
+During historical migration, the Observer preserves differences between frontmatter totals and the
+ledger as explicit reconciliation rows, and disambiguates duplicate `session_id` values per file
+without inventing calls.
 
 ## Requirements
 
@@ -111,7 +115,7 @@ npx wendkeep init
 
 1. Create the vault folder taxonomy and a templated `README.md` (default vault: `<project>/.<project-name>-vault`, e.g. `.MyApp-vault`; override with `--vault`).
 2. Write a provider-neutral **`.wendkeep.json`** binding at the project root and a matching `.brain/PROJECT.json` marker in the vault, then merge the session hooks into **`.claude/settings.json`**. The binding is provider-neutral by design: any agent resolves the same vault from its session `cwd`, with no machine-global environment variable. Older registrations already in `.claude/settings.json` are adopted automatically.
-3. Wire the Codex hooks in **`.codex/hooks.json`** — ten compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` only publishes a sanitized, fail-open projection; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
+3. Wire the Codex hooks in **`.codex/hooks.json`** — eleven compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` + `observer-publish` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` keeps the sanitized index projection and also sends the local SQL authority with documents, consumption, and transcripts; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
 4. Add the **`wendkeep-vault`** MCP server to `.mcp.json` so the agent can read/write the vault. Skip with `--no-mcp` — e.g. when the agent already has a vault MCP. (`--no-mcp` skips *only wendkeep's own* MCP; companion MCPs still follow `--companions`.)
 5. Offer to pin **companion** plugins/MCP (multi-choice; **none** pre-checked — wendkeep is a neutral harness and presumes no third-party plugin). Each is wired the most agent-agnostic way it supports:
    - **`context-mode`** — context optimizer + FTS5 memory, wired as a Claude Code plugin. It ships its own MCP server, so wendkeep deliberately adds no `.mcp.json` entry (registering both cold-started two servers at once). On non-Claude agents, add the MCP by hand: `npx -y context-mode`.

package/docs/en/commands/observer.md

@@ -4,23 +4,28 @@
 
 ## Purpose
 
-The Observer consolidates observability for multiple WendKeep projects in a local service without
-copying or taking ownership of their vaults.
+The Observer consolidates observability for multiple WendKeep projects in a local service. The
+Docker volume keeps `/data/observer.sqlite` as the single authority for documents, sessions,
+agents, usage, calls, and complete transcripts. The content can be browsed and searched in the
+container without depending on Obsidian for queries.
 
 ## When to use
 
-Use it to query summarized changes, sessions, tasks, and health across projects through one local
-projection while each vault remains authoritative.
+Use it to query changes, sessions, decisions, bugs, learnings, specs, brain documents, per-agent
+and per-model consumption, and health across projects through one local memory. During the
+transition, the vault and legacy Markdown files remain a recovery copy; the Observer is
+authoritative for queries made through its container.
 
 ## When not to use
 
-Do not use the Observer to edit, complete, or archive changes, curate memory, store transcripts,
-or replace local hooks. Do not expose it to the network in this version.
+Do not use the Observer to edit, complete, or archive changes, curate memory, automatically export
+the authority back to Markdown, or expose the service to the network. Edits still go through local
+WendKeep hooks.
 
 ## Prerequisites
 
-Initialize projects with WendKeep, explicitly register each project, and set a local token before
-starting the HTTP server.
+Initialize projects with WendKeep and explicitly register each project before starting the HTTP
+server. The default local mode has no token to configure.
 
 ## Syntax
 
@@ -28,6 +33,7 @@ starting the HTTP server.
 npx wendkeep observer status --data-dir <directory> --json
 npx wendkeep observer register --project <project> --vault <vault> --data-dir <directory>
 npx wendkeep observer publish --project <project> --vault <vault> --data-dir <directory>
+npx wendkeep observer memory import --project <project> --vault <vault> --url http://127.0.0.1:8787 --json
 npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <directory>
 ```
 
@@ -35,17 +41,17 @@ npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <directory>
 
 - `--data-dir` selects the local event and index directory; the default is
   `WENDKEEP_OBSERVER_DATA_DIR` or `~/.wendkeep-observer`.
-- `--project` and `--vault` identify a project only for `register` and `publish`.
+- `--project` and `--vault` identify a project for `register`, `publish`, and `memory import`.
 - `--host` accepts only `127.0.0.1`, `localhost`, or `::1`; other hosts are rejected before
   listening.
-- `--token` or `WENDKEEP_OBSERVER_TOKEN` protects `/v1`; `GET /healthz` exposes no project data.
+- `/v1` is open in the default local mode; keep `--host 127.0.0.1` and do not publish the port on
+  a network address.
 - Exit `0` means success; exit `1` means configuration or operation failure; the publisher hook
   also returns `0` when the Observer is unavailable.
 
 ## Examples
 
 ```powershell
-$env:WENDKEEP_OBSERVER_TOKEN = '<local-token>'
 npx wendkeep observer register --project C:\GitHub\WendKeep --vault C:\GitHub\WendKeep\.WendKeep-vault --data-dir C:\WendKeepObserver
 npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir C:\WendKeepObserver
 $env:WENDKEEP_OBSERVER_URL = 'http://127.0.0.1:8787'
@@ -54,30 +60,53 @@ $env:WENDKEEP_OBSERVER_URL = 'http://127.0.0.1:8787'
 For local Docker:
 
 ```powershell
-$env:WENDKEEP_OBSERVER_TOKEN = '<local-token>'
 docker compose -f docker/wendkeep-observer/compose.yaml up -d --build
 ```
 
-## Expected result
+## Local web dashboard
+
+With the server running, open [http://127.0.0.1:8787/](http://127.0.0.1:8787/) in a browser. The
+dashboard is served by the same process and opens directly, without a form or token. Keep the port
+bound to the computer loopback; do not expose this address on a network interface.
+
+The dashboard shows the multi-project list, version, health, latest session, active change, change
+count, and last capture time. Opening a project exposes Overview, Consumption, Sessions, Memory,
+Changes, and Sync screens. Consumption shows total cost, token categories, primary agents,
+subagents, providers, models, daily trend, historical coverage, and calls with prompt, response,
+and complete transcript. Loading, empty, unavailable-server, conflict, no-pricing, and stale-data
+states are visible, with manual refresh and an automatic 15-second refresh.
 
-`register` stores only `project_id`, name, version, and registration time. `publish` reads the
-local vault, produces a sanitized snapshot, and sends an idempotent event. The container stores
-only `EVENTS.jsonl` and `INDEX.json` in the `observer-data` volume; it does not mount `C:\GitHub`
-or any `.WendKeep-vault`.
+If the browser shows the shell but the list fails, check the service health at
+`http://127.0.0.1:8787/healthz` and confirm that the container is running.
 
-`init` projects `observer-publish` into `SessionStart` and `Stop` after the primary hooks. Without
-`WENDKEEP_OBSERVER_URL`, the hook is a no-op. When the server is stopped, it writes to
-`.brain/observer-outbox/` and does not block the session; a later run retries pending events.
+## Expected result
+
+`register` stores `project_id`, name, version, and registration time. `publish` reads the local
+vault, produces the snapshot, and sends idempotent events to SQLite containing the complete content
+of sessions, decisions, bugs, learnings, specs, changes, CORE, DIGEST, SHARED_MEMORY, brain state,
+agent sessions, cost rollups, calls, and transcripts. The container stores everything in
+`/data/observer.sqlite`; it does not mount `C:\GitHub` or any `.WendKeep-vault`. Markdown is only
+the text held in SQL and is recreated as files only by an explicit read-only export.
+`memory import` performs the initial load and returns file/hash parity. During migration, the
+cost/token total recorded in frontmatter is preserved through an explicit reconciliation row when
+the detailed ledger does not add up; that row does not invent calls. Historical sessions sharing
+one `session_id` receive a canonical per-file identity so one rollup cannot overwrite the other.
+
+`init` projects `observer-publish` into `SessionStart`, `Stop`, and `SubagentStop` after the primary
+hooks. When the server is unavailable, it writes snapshots to `.brain/observer-outbox/` and SQL
+events to `.brain/observer-sql-outbox/` without blocking the session; a later run retries the
+batches. SQL batches use gzip so complete transcripts larger than 64 MB as plain JSON remain within
+the transport limit; the Observer decompresses and validates the body before ingesting it. The
+outbox is temporary transport, not authority.
 
 ## Common errors and diagnosis
 
 - `project_not_registered`: run `observer register` before publishing.
-- `unauthorized`: check `Authorization: Bearer <token>` and `WENDKEEP_OBSERVER_TOKEN`.
 - `host loopback`: replace `0.0.0.0` or a LAN address with `127.0.0.1`.
-- Pending outbox: the service was unavailable; preserve `.brain/observer-outbox/` and rerun the
-  publisher. Do not delete events manually.
-- The Observer does not read raw content, paths, transcripts, or memory; such rejections are
-  expected and should be investigated at the snapshot source.
+- Pending outbox: the service was unavailable; preserve `.brain/observer-outbox/` and
+  `.brain/observer-sql-outbox/`, then rerun the publisher. Do not delete events manually.
+- If memory or usage is incomplete, check the Sync screen, preserve the outbox, and run
+  `observer memory import` to rebuild the load from the vault.
 
 ## Next steps
 
@@ -87,18 +116,34 @@ projection.
 
 ## Data authority
 
-Each project vault remains authoritative for sessions, changes, tasks, memory, and evidence. The
-Observer is a read-only, rebuildable projection; its queries do not complete, archive, repair, or
-promote state in a vault.
+The container SQLite database is canonical for Observer queries and stores the complete published
+content. The vault and any legacy `/data/memory` remain preserved during migration as a transition
+and recovery copy; hooks do not update container Markdown after cutover. Observer screens do not
+complete, archive, repair, or promote state.
 
 ## Minimal API
 
-- `GET /healthz` — availability without project data.
-- `GET /v1/projects` — projects with an accepted snapshot.
+- `GET /healthz` — availability, SQLite migration version, and legacy migration state.
+- `GET /v1/projects` — projects registered in SQLite, with a snapshot when available.
 - `GET /v1/projects/:project_id` — the latest project snapshot.
 - `GET /v1/projects/:project_id/changes` — change summary from the snapshot.
-- `PUT /v1/projects/:project_id` — authenticated explicit registration.
-- `POST /v1/projects/:project_id/snapshot` — authenticated idempotent ingestion.
-
-The `/v1` routes reject oversized bodies and never accept vault paths, transcripts, secrets, or raw
-memory content.
+- `PUT /v1/projects/:project_id` — explicit local registration.
+- `POST /v1/projects/:project_id/snapshot` — idempotent local ingestion.
+- `POST /v1/projects/:project_id/ingest` — idempotent batches of documents, sessions, agents,
+  rollups, calls, and transcripts.
+- `GET /v1/projects/:project_id/memory/tree` — document tree and metadata.
+- `GET /v1/projects/:project_id/memory/document?path=...` — complete Markdown content.
+- `GET /v1/projects/:project_id/memory/search?q=...` — path and body search.
+- `GET /v1/projects/:project_id/sync` — mode, counts, conflicts, and latest event.
+- `PUT /v1/projects/:project_id/sync` — compatibility configuration; SQL remains authoritative.
+- `GET /v1/projects/:project_id/memory/export` — read-only export with complete content.
+- `POST /v1/projects/:project_id/memory/events` — idempotent batch ingestion.
+- `GET /v1/projects/:project_id/usage/summary` — filterable totals by period, change, session,
+  agent, provider, model, and role.
+- `GET /v1/projects/:project_id/usage/breakdown` — agent, subagent, and model hierarchy.
+- `GET /v1/projects/:project_id/usage/calls` — individual calls with prompt and response.
+- `GET /v1/projects/:project_id/transcripts/:transcript_id` — compressed transcript validated by hash.
+
+The `/v1` routes reject transported or expanded bodies above their limits and validate project,
+path, revision, hash, idempotency, and isolation before writing to SQLite. Use `memory/export` for
+a Markdown copy; it does not alter SQL authority.

package/docs/pt-BR/commands/observer.md

@@ -4,23 +4,28 @@
 
 ## Objetivo
 
-O Observer consolida a observabilidade de vários projetos WendKeep em um serviço local, sem
-copiar ou assumir a propriedade dos vaults.
+O Observer consolida a observabilidade de vários projetos WendKeep em um serviço local. O volume
+Docker mantém o SQLite `/data/observer.sqlite` como autoridade única para documentos, sessões,
+agentes, uso, chamadas e transcripts completos. O conteúdo fica disponível para navegação e busca
+no próprio container, sem depender do Obsidian para consulta.
 
 ## Quando usar
 
-Use para consultar changes, sessões, tarefas resumidas e saúde de vários projetos em uma única
-projeção local, mantendo cada vault como fonte oficial.
+Use para consultar changes, sessões, decisões, bugs, aprendizados, specs, documentos do brain,
+consumo por agente/modelo e saúde de vários projetos em uma única memória local. Durante a
+transição, o vault e os arquivos Markdown legados continuam preservados como cópia de origem para
+recuperação; o Observer é a autoridade de consulta do seu container.
 
 ## Quando não usar
 
-Não use o Observer para editar, concluir ou arquivar changes, curar memória, armazenar transcripts
-ou substituir os hooks locais. Não o exponha na rede nesta versão.
+Não use o Observer para editar, concluir ou arquivar changes, curar memória, exportar a autoridade
+de volta para Markdown automaticamente ou expor o serviço na rede. As edições continuam passando
+pelos hooks e pelo WendKeep local.
 
 ## Pré-requisitos
 
-Tenha os projetos inicializados com WendKeep, registre explicitamente cada projeto e defina um
-token local antes de iniciar o servidor HTTP.
+Tenha os projetos inicializados com WendKeep e registre explicitamente cada projeto antes de
+iniciar o servidor HTTP. No modo local padrão não há token para configurar.
 
 ## Sintaxe
 
@@ -28,6 +33,7 @@ token local antes de iniciar o servidor HTTP.
 npx wendkeep observer status --data-dir <diretório> --json
 npx wendkeep observer register --project <projeto> --vault <vault> --data-dir <diretório>
 npx wendkeep observer publish --project <projeto> --vault <vault> --data-dir <diretório>
+npx wendkeep observer memory import --project <projeto> --vault <vault> --url http://127.0.0.1:8787 --json
 npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <diretório>
 ```
 
@@ -35,18 +41,17 @@ npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir <diretório>
 
 - `--data-dir` escolhe o diretório local de eventos e índice; o padrão é
   `WENDKEEP_OBSERVER_DATA_DIR` ou `~/.wendkeep-observer`.
-- `--project` e `--vault` identificam o projeto somente nos comandos `register` e `publish`.
+- `--project` e `--vault` identificam o projeto nos comandos `register`, `publish` e `memory import`.
 - `--host` aceita somente `127.0.0.1`, `localhost` ou `::1`; outros hosts são recusados antes do
   listen.
-- `--token` ou `WENDKEEP_OBSERVER_TOKEN` protege as rotas `/v1`; `GET /healthz` permanece sem
-  dados de projeto.
+- as rotas `/v1` ficam abertas no modo local padrão; mantenha `--host 127.0.0.1` e não publique a
+  porta em um endereço de rede.
 - Exit `0` indica sucesso; exit `1` indica falha de configuração ou operação; o hook publisher
   também retorna `0` quando o Observer está indisponível.
 
 ## Exemplos
 
 ```powershell
-$env:WENDKEEP_OBSERVER_TOKEN = '<token-local>'
 npx wendkeep observer register --project C:\GitHub\WendKeep --vault C:\GitHub\WendKeep\.WendKeep-vault --data-dir C:\WendKeepObserver
 npx wendkeep observer serve --host 127.0.0.1 --port 8787 --data-dir C:\WendKeepObserver
 $env:WENDKEEP_OBSERVER_URL = 'http://127.0.0.1:8787'
@@ -55,31 +60,55 @@ $env:WENDKEEP_OBSERVER_URL = 'http://127.0.0.1:8787'
 Para Docker local:
 
 ```powershell
-$env:WENDKEEP_OBSERVER_TOKEN = '<token-local>'
 docker compose -f docker/wendkeep-observer/compose.yaml up -d --build
 ```
 
-## Resultado esperado
+## Painel web local
+
+Com o servidor em execução, abra [http://127.0.0.1:8787/](http://127.0.0.1:8787/) no navegador.
+O painel é servido pelo mesmo processo e abre diretamente, sem formulário ou token. A porta fica
+presa ao loopback do computador; não coloque o endereço em uma interface de rede.
+
+O painel mostra a lista multi-projeto, versão, saúde, sessão mais recente, change ativa, contagem
+de changes e data da última captura. Ao abrir um projeto, o workspace oferece Overview, Consumo,
+Sessões, Memória, Changes e Sincronização. A aba Consumo mostra custo total, tokens por categoria,
+agentes principais, subagentes, provedores, modelos, tendência diária, cobertura histórica e
+chamadas com prompt, resposta e transcript completo. Os estados de carregamento, vazio, servidor
+indisponível, conflito, modelo sem tarifa e dados desatualizados ficam visíveis, e a atualização
+pode ser manual ou automática a cada 15 segundos.
 
-`register` grava apenas `project_id`, nome, versão e data de registro. `publish` lê o vault local,
-produz um snapshot sanitizado e envia um evento idempotente. O container mantém somente
-`EVENTS.jsonl` e `INDEX.json` no volume `observer-data`; não monta `C:\GitHub` nem qualquer
-`.WendKeep-vault`.
+Se o navegador mostrar a tela mas a lista falhar, confirme a saúde em
+`http://127.0.0.1:8787/healthz` e verifique se o container está em execução.
 
-O `init` projeta `observer-publish` para `SessionStart` e `Stop` depois dos hooks principais. Sem
-`WENDKEEP_OBSERVER_URL`, o hook é no-op. Com o servidor parado, ele grava em
-`.brain/observer-outbox/` e não bloqueia a sessão; uma execução posterior tenta reenviar eventos
-pendentes.
+## Resultado esperado
+
+`register` grava `project_id`, nome, versão e data de registro. `publish` lê o vault local,
+produz o snapshot e envia eventos idempotentes para o SQLite com o conteúdo integral das sessões,
+decisões, bugs, aprendizados, specs, changes, CORE, DIGEST, SHARED_MEMORY, estado do brain,
+sessões de agentes, rollups de custo, chamadas e transcripts. O container grava tudo em
+`/data/observer.sqlite`; não monta `C:\GitHub` nem qualquer `.WendKeep-vault`. Markdown é aceito
+somente como conteúdo de uma coluna SQL e volta a existir como arquivo apenas pela exportação
+read-only sob demanda. `memory import` faz a carga inicial e retorna a paridade por arquivo e hash.
+Na migração, o total de custo/token registrado no frontmatter é preservado por uma linha de
+reconciliação quando o ledger detalhado não fecha com ele; essa linha não inventa chamadas.
+Sessões históricas com o mesmo `session_id` recebem uma identidade canônica por arquivo para
+evitar que um rollup sobrescreva o outro.
+
+O `init` projeta `observer-publish` para `SessionStart`, `Stop` e `SubagentStop` depois dos hooks
+principais. Sem servidor disponível, ele grava snapshots em `.brain/observer-outbox/` e eventos
+SQL em `.brain/observer-sql-outbox/`, sem bloquear a sessão; uma execução posterior tenta
+reenviar os lotes. Os lotes SQL são enviados com gzip para que transcripts completos maiores que
+64 MB em JSON puro continuem dentro do limite do transporte; o Observer descomprime e valida o
+corpo antes de ingerir. O outbox é transporte temporário, não autoridade.
 
 ## Erros comuns e diagnóstico
 
 - `project_not_registered`: rode `observer register` antes de publicar.
-- `unauthorized`: confira `Authorization: Bearer <token>` e `WENDKEEP_OBSERVER_TOKEN`.
 - `host loopback`: troque `0.0.0.0` ou endereço LAN por `127.0.0.1`.
-- Outbox pendente: o serviço estava indisponível; preserve `.brain/observer-outbox/` e repita o
-  publisher. Não apague eventos manualmente.
-- O Observer não lê conteúdo bruto, caminhos, transcripts ou memória; rejeições desse tipo são
-  esperadas e devem ser investigadas na origem do snapshot.
+- Outbox pendente: o serviço estava indisponível; preserve `.brain/observer-outbox/` e
+  `.brain/observer-sql-outbox/` e repita o publisher. Não apague eventos manualmente.
+- Se a memória ou o consumo ficarem incompletos, verifique a tela Sincronização, preserve o
+  outbox e rode `observer memory import` para reconstruir a carga a partir do vault.
 
 ## Próximos passos
 
@@ -88,18 +117,34 @@ removido com `docker compose down -v` durante a operação normal, pois isso apa
 
 ## Autoridade dos dados
 
-O vault de cada projeto continua sendo a fonte oficial de sessões, changes, tarefas, memória e
-evidências. O Observer é uma projeção read-only reconstruível; suas consultas não concluem,
-arquivam, reparam ou promovem estado no vault.
+O SQLite do container é a memória canônica para consultas do Observer e guarda o conteúdo integral
+publicado. O vault e qualquer `/data/memory` legado continuam preservados durante a migração como
+cópia de transição e fonte de recuperação; os hooks não atualizam Markdown no container depois do
+corte. As telas do Observer não concluem, arquivam, reparam ou promovem estado.
 
 ## API mínima
 
-- `GET /healthz` — disponibilidade sem dados de projeto.
-- `GET /v1/projects` — projetos com snapshot aceito.
+- `GET /healthz` — disponibilidade, versão das migrações SQLite e estado da migração legada.
+- `GET /v1/projects` — projetos registrados no SQLite, com snapshot quando houver.
 - `GET /v1/projects/:project_id` — último snapshot do projeto.
 - `GET /v1/projects/:project_id/changes` — resumo das changes do snapshot.
-- `PUT /v1/projects/:project_id` — registro explícito autenticado.
-- `POST /v1/projects/:project_id/snapshot` — ingestão autenticada e idempotente.
-
-As rotas `/v1` rejeitam corpo acima do limite e nunca aceitam caminho de vault, transcript, segredo
-ou conteúdo bruto de memória.
+- `PUT /v1/projects/:project_id` — registro explícito local.
+- `POST /v1/projects/:project_id/snapshot` — ingestão local idempotente.
+- `POST /v1/projects/:project_id/ingest` — lote idempotente de documentos, sessões, agentes, rollups,
+  chamadas e transcripts.
+- `GET /v1/projects/:project_id/memory/tree` — árvore e metadados dos documentos.
+- `GET /v1/projects/:project_id/memory/document?path=...` — conteúdo Markdown integral.
+- `GET /v1/projects/:project_id/memory/search?q=...` — busca no caminho e no corpo.
+- `GET /v1/projects/:project_id/sync` — modo, contagem, conflitos e último evento.
+- `PUT /v1/projects/:project_id/sync` — compatibilidade de configuração; a autoridade continua SQL.
+- `GET /v1/projects/:project_id/memory/export` — exportação read-only com conteúdo completo.
+- `POST /v1/projects/:project_id/memory/events` — ingestão idempotente em lote.
+- `GET /v1/projects/:project_id/usage/summary` — totais filtráveis por período, change, sessão,
+  agente, provedor, modelo e papel.
+- `GET /v1/projects/:project_id/usage/breakdown` — hierarquia de agentes, subagentes e modelos.
+- `GET /v1/projects/:project_id/usage/calls` — chamadas individuais com prompt e resposta.
+- `GET /v1/projects/:project_id/transcripts/:transcript_id` — transcript comprimido, validado por hash.
+
+As rotas `/v1` rejeitam corpo transportado ou expandido acima do limite e validam projeto, caminho,
+revisão, hash, idempotência e isolamento antes de gravar o conteúdo no SQLite. Para preservar uma
+cópia Markdown, use a rota `memory/export`; ela não altera a autoridade SQL.

package/hooks/observer-publish.mjs

@@ -9,6 +9,7 @@ async function main() {
   const result = await publishObserverSnapshot({
     vaultBase: resolved.base,
     projectRoot: resolved.projectRoot,
+    input,
   });
   if (!result.ok && result.error) debugLog('Observer publish fail-open:', result.error);
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wendkeep",
-  "version": "0.70.0",
+  "version": "0.72.0",
   "description": "Vault-first persistent memory for AI coding agents, with an optional profile-aware governance runtime: OFF, FLOW, GUIDE, GOVERN, or ASSURE. Local-first and agent-agnostic (Claude Code, Codex, Cursor…).",
   "type": "module",
   "workspaces": [
@@ -30,6 +30,7 @@
     "hooks",
     "packages",
     "schema",
+    "web/observer",
     "docs/pt-BR/commands/*.md",
     "docs/en/commands/*.md",
     "README.md",
@@ -40,7 +41,7 @@
     "node": ">=18"
   },
   "scripts": {
-    "check": "node --check scripts/release.mjs && node --check scripts/release-plan.mjs && node --check bin/wendkeep.mjs && node --check packages/cli/src/index.mjs && node --check src/init.mjs && node --check src/doctor.mjs && node --check src/project-vault.mjs && node --check src/observer-snapshot.mjs && node --check src/observer-store.mjs && node --check src/observer-server.mjs && node --check src/observer.mjs && node --check src/observer-publish.mjs && node --check src/operating-profile.mjs && node --check src/profile.mjs && node --check src/flow.mjs && node --check hooks/observer-publish.mjs && node --check hooks/operating-profile-runtime.mjs && node --check hooks/operating-profile-task-store.mjs && node --check hooks/flow-core.mjs && node --check hooks/flow-protected-policy.mjs && node --check hooks/git-snapshot.mjs && node --check hooks/vault-path-safety.mjs && node --check hooks/vault-runtime-store.mjs && node --check packages/harness/src/index.mjs && node --check packages/harness/src/flow-store.mjs && node --check packages/harness/src/operating-profile.mjs && node --check packages/harness/src/sensors-core.mjs && node --check packages/integrations/src/host-hooks.mjs && node --check packages/integrations/src/hook-envelope.mjs && node --check packages/integrations/src/prompt-content.mjs && node --check packages/integrations/src/transcript-usage.mjs && node --check packages/integrations/src/transcripts.mjs && node --check packages/integrations/src/session-identity.mjs && node --check packages/integrations/src/index.mjs && node --check packages/mcp/src/config.mjs && node --check packages/mcp/src/index.mjs && node --check packages/vault/src/index.mjs && node --check packages/vault/src/project-vault.mjs && node --check packages/vault/src/vault-path-safety.mjs && node --check packages/vault/src/locale.mjs && node --check packages/vault/src/memory-schema.mjs && node --check packages/vault/src/memory-mode.mjs && node --check packages/vault/src/memory-handoff.mjs && node --check packages/vault/src/memory-store.mjs && node --check packages/vault/src/validate-core.mjs && node --check packages/vault/src/validate-memory.mjs",
+    "check": "node --check scripts/release.mjs && node --check scripts/release-plan.mjs && node --check bin/wendkeep.mjs && node --check packages/cli/src/index.mjs && node --check src/init.mjs && node --check src/doctor.mjs && node --check src/project-vault.mjs && node --check src/observer-snapshot.mjs && node --check src/observer-store.mjs && node --check src/observer-memory.mjs && node --check src/observer-memory-publish.mjs && node --check src/observer-sql-store.mjs && node --check src/observer-sql-migrate.mjs && node --check src/observer-sql-publish.mjs && node --check src/observer-transcript-store.mjs && node --check src/observer-server.mjs && node --check src/observer.mjs && node --check src/observer-publish.mjs && node --check src/operating-profile.mjs && node --check src/profile.mjs && node --check src/flow.mjs && node --check web/observer/app.mjs && node --check hooks/observer-publish.mjs && node --check hooks/operating-profile-runtime.mjs && node --check hooks/operating-profile-task-store.mjs && node --check hooks/flow-core.mjs && node --check hooks/flow-protected-policy.mjs && node --check hooks/git-snapshot.mjs && node --check hooks/vault-path-safety.mjs && node --check hooks/vault-runtime-store.mjs && node --check packages/harness/src/index.mjs && node --check packages/harness/src/flow-store.mjs && node --check packages/harness/src/operating-profile.mjs && node --check packages/harness/src/sensors-core.mjs && node --check packages/integrations/src/host-hooks.mjs && node --check packages/integrations/src/hook-envelope.mjs && node --check packages/integrations/src/prompt-content.mjs && node --check packages/integrations/src/transcript-usage.mjs && node --check packages/integrations/src/transcripts.mjs && node --check packages/integrations/src/session-identity.mjs && node --check packages/integrations/src/index.mjs && node --check packages/mcp/src/config.mjs && node --check packages/mcp/src/index.mjs && node --check packages/vault/src/index.mjs && node --check packages/vault/src/project-vault.mjs && node --check packages/vault/src/vault-path-safety.mjs && node --check packages/vault/src/locale.mjs && node --check packages/vault/src/memory-schema.mjs && node --check packages/vault/src/memory-mode.mjs && node --check packages/vault/src/memory-handoff.mjs && node --check packages/vault/src/memory-store.mjs && node --check packages/vault/src/validate-core.mjs && node --check packages/vault/src/validate-memory.mjs",
     "test": "node --test --test-concurrency=2",
     "release": "node scripts/release.mjs",
     "release:dry": "node scripts/release.mjs --dry-run",

package/packages/integrations/src/host-hooks.mjs

@@ -24,6 +24,8 @@ export const SESSION_HOOKS = [
   { event: 'PostToolUse', matcher: 'AskUserQuestion', name: 'decision-capture', timeout: 15, statusMessage: 'wendkeep: recording decision' },
   // Refresh subagent/workflow telemetry as each subagent finishes (resilient to a missed Stop).
   { event: 'SubagentStop', matcher: null, name: 'subagent-stop', timeout: 20, codex: true, statusMessage: 'wendkeep: subagent telemetry' },
+  // Publish the SQL observer projection after the subagent telemetry is settled.
+  { event: 'SubagentStop', matcher: null, name: 'observer-publish', timeout: 5, order: 20, codex: true, statusMessage: 'wendkeep: publishing local observer usage' },
   // Log plan/task progress into the active session note when a task is marked complete.
   // codex: TaskCompleted is not in Codex's hook event enum.
   { event: 'TaskCompleted', matcher: null, name: 'task-log', timeout: 10, statusMessage: 'wendkeep: plan progress' },