wendkeep 0.71.1 → 0.72.0

package/CHANGELOG.md

@@ -4,6 +4,34 @@ All notable changes to **wendkeep** are documented here. Format based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this project follows
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.72.0] — 2026-08-17
+
+### Added
+
+- **Observer SQL authority.** O volume Docker agora usa `/data/observer.sqlite` como
+  autoridade única para documentos, sessões, agentes, uso, chamadas LLM e transcripts
+  comprimidos; as migrações SQL são versionadas e idempotentes.
+- **Ingestão resiliente.** Hooks de sessão e subagentes enviam eventos idempotentes ao
+  Observer, preservam custos registrados e usam outbox local quando o container está
+  indisponível.
+- **Dashboard de Consumo.** Cada projeto ganhou resumo de tokens/custos, filtros,
+  hierarquia agente/subagente/modelo, tendência diária, chamadas e leitura de transcript.
+- **Migração sem perda.** Conteúdo legado do volume, `MEMORY_EVENTS.jsonl`, frontmatter de
+  custo e históricos sem transcript são importados sem apagar as fontes existentes;
+  históricos incompletos são marcados como `summary_only`; divergências entre frontmatter e
+  ledger ficam em linhas explícitas de reconciliação e `session_id` duplicado é desambiguado
+  por arquivo.
+
+### Changed
+
+- Markdown deixou de ser autoridade operacional no container. Ele permanece armazenado como
+  conteúdo documental no SQLite e só é materializado por exportação explícita.
+- O transporte divide lotes por quantidade e tamanho, reconhece retries do hash legado e
+  preserva transcripts grandes dentro do limite HTTP do Observer.
+- Lotes SQL agora usam gzip no transporte e são expandidos com limite controlado no Observer,
+  permitindo importar transcripts históricos que excedem 64 MB em JSON puro sem aumentar
+  indiscriminadamente o limite de requisição.
+
 ## [0.71.1] — 2026-08-17
 
 ### Added

package/README.en.md

@@ -88,7 +88,11 @@ Decisions, dead ends, the reason you chose X over Y — gone next session. The p
 | **Cost** — what it all cost | Per‑model, cache‑aware token pricing per session — plus `cost --trend` with a run‑rate projection across the whole vault; research previews without a final rate remain unestimated. |
 | **Multi‑agent** — one vault, both agents | `init` wires the session hooks into `.claude/settings.json` *and* `.codex/hooks.json`, and every note is tagged with the agent that wrote it: Claude Code is detected from its environment, anything else is recorded as Codex. One shared graph, whichever agent you are in. |
 | **Local‑first** — no cloud, no account | Everything is plain Markdown on your disk. An optional MCP server (`@bitbonsai/mcpvault`) lets the agent read/write the vault. |
-| **Local Observer** — many projects, one view | `wendkeep observer` keeps snapshots plus a complete copy of sessions, decisions, bugs, learnings, specs, and changes in the Docker volume; the read-only dashboard opens directly at `http://127.0.0.1:8787/`, local mode has no token, and unavailable hooks use an outbox without blocking the session. |
+| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, sessions, agents, tokens, costs, calls, and transcripts in the Docker volume SQLite database; every project gets a **Consumption** tab in the read-only dashboard at `http://127.0.0.1:8787/`, loopback has no token, and unavailable hooks use a gzip outbox without blocking the session, including large transcripts. |
+
+During historical migration, the Observer preserves differences between frontmatter totals and the
+ledger as explicit reconciliation rows, and disambiguates duplicate `session_id` values per file
+without inventing calls.
 
 ## Requirements
 
@@ -111,7 +115,7 @@ npx wendkeep init
 
 1. Create the vault folder taxonomy and a templated `README.md` (default vault: `<project>/.<project-name>-vault`, e.g. `.MyApp-vault`; override with `--vault`).
 2. Write a provider-neutral **`.wendkeep.json`** binding at the project root and a matching `.brain/PROJECT.json` marker in the vault, then merge the session hooks into **`.claude/settings.json`**. The binding is provider-neutral by design: any agent resolves the same vault from its session `cwd`, with no machine-global environment variable. Older registrations already in `.claude/settings.json` are adopted automatically.
-3. Wire the Codex hooks in **`.codex/hooks.json`** — ten compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` only publishes a sanitized, fail-open projection; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
+3. Wire the Codex hooks in **`.codex/hooks.json`** — eleven compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` + `observer-publish` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` keeps the sanitized index projection and also sends the local SQL authority with documents, consumption, and transcripts; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
 4. Add the **`wendkeep-vault`** MCP server to `.mcp.json` so the agent can read/write the vault. Skip with `--no-mcp` — e.g. when the agent already has a vault MCP. (`--no-mcp` skips *only wendkeep's own* MCP; companion MCPs still follow `--companions`.)
 5. Offer to pin **companion** plugins/MCP (multi-choice; **none** pre-checked — wendkeep is a neutral harness and presumes no third-party plugin). Each is wired the most agent-agnostic way it supports:
    - **`context-mode`** — context optimizer + FTS5 memory, wired as a Claude Code plugin. It ships its own MCP server, so wendkeep deliberately adds no `.mcp.json` entry (registering both cold-started two servers at once). On non-Claude agents, add the MCP by hand: `npx -y context-mode`.

package/README.md

@@ -88,7 +88,11 @@ Decisions, dead ends, the reason you chose X over Y — gone next session. The p
 | **Cost** — what it all cost | Per‑model, cache‑aware token pricing per session — plus `cost --trend` with a run‑rate projection across the whole vault; research previews without a final rate remain unestimated. |
 | **Multi‑agent** — one vault, both agents | `init` wires the session hooks into `.claude/settings.json` *and* `.codex/hooks.json`, and every note is tagged with the agent that wrote it: Claude Code is detected from its environment, anything else is recorded as Codex. One shared graph, whichever agent you are in. |
 | **Local‑first** — no cloud, no account | Everything is plain Markdown on your disk. An optional MCP server (`@bitbonsai/mcpvault`) lets the agent read/write the vault. |
-| **Local Observer** — many projects, one view | `wendkeep observer` keeps snapshots plus a complete copy of sessions, decisions, bugs, learnings, specs, and changes in the Docker volume; the read-only dashboard opens directly at `http://127.0.0.1:8787/`, local mode has no token, and unavailable hooks use an outbox without blocking the session. |
+| **Local Observer** — many projects, one view | `wendkeep observer` keeps documents, sessions, agents, tokens, costs, calls, and transcripts in the Docker volume SQLite database; every project gets a **Consumption** tab in the read-only dashboard at `http://127.0.0.1:8787/`, loopback has no token, and unavailable hooks use a gzip outbox without blocking the session, including large transcripts. |
+
+During historical migration, the Observer preserves differences between frontmatter totals and the
+ledger as explicit reconciliation rows, and disambiguates duplicate `session_id` values per file
+without inventing calls.
 
 ## Requirements
 
@@ -111,7 +115,7 @@ npx wendkeep init
 
 1. Create the vault folder taxonomy and a templated `README.md` (default vault: `<project>/.<project-name>-vault`, e.g. `.MyApp-vault`; override with `--vault`).
 2. Write a provider-neutral **`.wendkeep.json`** binding at the project root and a matching `.brain/PROJECT.json` marker in the vault, then merge the session hooks into **`.claude/settings.json`**. The binding is provider-neutral by design: any agent resolves the same vault from its session `cwd`, with no machine-global environment variable. Older registrations already in `.claude/settings.json` are adopted automatically.
-3. Wire the Codex hooks in **`.codex/hooks.json`** — ten compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` only publishes a sanitized, fail-open projection; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
+3. Wire the Codex hooks in **`.codex/hooks.json`** — eleven compatible entries: `brain-inject` + `session-start` + `observer-publish` on `SessionStart`, `session-ensure` + `change-context` on `UserPromptSubmit`, `session-stop` + `observer-publish` + `change-nag` on `Stop`, `subagent-stop` + `observer-publish` on `SubagentStop`, and `change-guard` on `PreToolUse` for `Bash`, `exec_command`, `apply_patch`, and mutable MCP tools, always in the `npx wendkeep hook <name>` form. `observer-publish` keeps the sanitized index projection and also sends the local SQL authority with documents, consumption, and transcripts; it does not replace the local lifecycle. The guard accepts object, raw-string, and argv Codex payloads; before a mutation it compares the session with the project, Git root, remote, branch, and worktree, denying missing or divergent targets. The other four stay out because Codex offers no equivalent payload, tool, or event: `change-warn` (no reliable `tool_input.file_path`), `plan-capture` (no `ExitPlanMode`), `decision-capture` (`AskUserQuestion` is Claude-only), and `task-log` (`TaskCompleted` is not in Codex's event enum). Codex scope blocks use `permissionDecision: "deny"`; `ask` is never emitted in `PreToolUse`. The merge remains non-destructive, preserves third-party hooks, and migrates legacy `timeout` to `timeoutSec`. **Codex enumerates every hook as untrusted and runs none until you approve the “Hooks need review” prompt at startup — `init` cannot pre-approve them**.
 4. Add the **`wendkeep-vault`** MCP server to `.mcp.json` so the agent can read/write the vault. Skip with `--no-mcp` — e.g. when the agent already has a vault MCP. (`--no-mcp` skips *only wendkeep's own* MCP; companion MCPs still follow `--companions`.)
 5. Offer to pin **companion** plugins/MCP (multi-choice; **none** pre-checked — wendkeep is a neutral harness and presumes no third-party plugin). Each is wired the most agent-agnostic way it supports:
    - **`context-mode`** — context optimizer + FTS5 memory, wired as a Claude Code plugin. It ships its own MCP server, so wendkeep deliberately adds no `.mcp.json` entry (registering both cold-started two servers at once). On non-Claude agents, add the MCP by hand: `npx -y context-mode`.

package/docs/en/commands/observer.md

@@ -4,20 +4,23 @@
 
 ## Purpose
 
-The Observer consolidates observability for multiple WendKeep projects in a local service and
-stores a complete copy of the memory published by hooks in the Docker volume. The content can be
-browsed and searched in the container without depending on Obsidian for queries.
+The Observer consolidates observability for multiple WendKeep projects in a local service. The
+Docker volume keeps `/data/observer.sqlite` as the single authority for documents, sessions,
+agents, usage, calls, and complete transcripts. The content can be browsed and searched in the
+container without depending on Obsidian for queries.
 
 ## When to use
 
-Use it to query changes, sessions, decisions, bugs, learnings, specs, brain documents, and health
-across projects through one local memory. During the transition, the vault is preserved as a
-recovery copy; the Observer is authoritative for queries made through its container.
+Use it to query changes, sessions, decisions, bugs, learnings, specs, brain documents, per-agent
+and per-model consumption, and health across projects through one local memory. During the
+transition, the vault and legacy Markdown files remain a recovery copy; the Observer is
+authoritative for queries made through its container.
 
 ## When not to use
 
-Do not use the Observer to edit, complete, or archive changes, curate memory, store transcripts,
-or expose the service to the network. Edits still go through local WendKeep hooks.
+Do not use the Observer to edit, complete, or archive changes, curate memory, automatically export
+the authority back to Markdown, or expose the service to the network. Edits still go through local
+WendKeep hooks.
 
 ## Prerequisites
 
@@ -67,10 +70,11 @@ dashboard is served by the same process and opens directly, without a form or to
 bound to the computer loopback; do not expose this address on a network interface.
 
 The dashboard shows the multi-project list, version, health, latest session, active change, change
-count, and last capture time. Opening a project exposes Overview, Sessions, Memory, Changes, and
-Sync screens. Each list opens the complete Markdown document in a read-only reader, with local
-filtering, body search, and a source toggle. Loading, empty, unavailable-server, conflict, and
-stale-data states are visible, with manual refresh and an automatic 15-second refresh.
+count, and last capture time. Opening a project exposes Overview, Consumption, Sessions, Memory,
+Changes, and Sync screens. Consumption shows total cost, token categories, primary agents,
+subagents, providers, models, daily trend, historical coverage, and calls with prompt, response,
+and complete transcript. Loading, empty, unavailable-server, conflict, no-pricing, and stale-data
+states are visible, with manual refresh and an automatic 15-second refresh.
 
 If the browser shows the shell but the list fails, check the service health at
 `http://127.0.0.1:8787/healthz` and confirm that the container is running.
@@ -78,25 +82,31 @@ If the browser shows the shell but the list fails, check the service health at
 ## Expected result
 
 `register` stores `project_id`, name, version, and registration time. `publish` reads the local
-vault, produces the snapshot, and also sends idempotent events containing the complete content of
-sessions, decisions, bugs, learnings, specs, changes, CORE, DIGEST, SHARED_MEMORY, and brain
-state. The container stores Markdown under `/data/memory` plus `MEMORY_EVENTS.jsonl` and
-`MEMORY_INDEX.json` in the `observer-data` volume; it does not mount `C:\GitHub` or any
-`.WendKeep-vault`. `memory import` performs the initial load and returns file/hash parity.
-
-`init` projects `observer-publish` into `SessionStart` and `Stop` after the primary hooks. Without
-`WENDKEEP_OBSERVER_URL`, the hook is a no-op. When the server is stopped, it writes snapshots to
-`.brain/observer-outbox/` and complete memory to `.brain/observer-memory-outbox/` without blocking
-the session; a later run retries both event types.
+vault, produces the snapshot, and sends idempotent events to SQLite containing the complete content
+of sessions, decisions, bugs, learnings, specs, changes, CORE, DIGEST, SHARED_MEMORY, brain state,
+agent sessions, cost rollups, calls, and transcripts. The container stores everything in
+`/data/observer.sqlite`; it does not mount `C:\GitHub` or any `.WendKeep-vault`. Markdown is only
+the text held in SQL and is recreated as files only by an explicit read-only export.
+`memory import` performs the initial load and returns file/hash parity. During migration, the
+cost/token total recorded in frontmatter is preserved through an explicit reconciliation row when
+the detailed ledger does not add up; that row does not invent calls. Historical sessions sharing
+one `session_id` receive a canonical per-file identity so one rollup cannot overwrite the other.
+
+`init` projects `observer-publish` into `SessionStart`, `Stop`, and `SubagentStop` after the primary
+hooks. When the server is unavailable, it writes snapshots to `.brain/observer-outbox/` and SQL
+events to `.brain/observer-sql-outbox/` without blocking the session; a later run retries the
+batches. SQL batches use gzip so complete transcripts larger than 64 MB as plain JSON remain within
+the transport limit; the Observer decompresses and validates the body before ingesting it. The
+outbox is temporary transport, not authority.
 
 ## Common errors and diagnosis
 
 - `project_not_registered`: run `observer register` before publishing.
 - `host loopback`: replace `0.0.0.0` or a LAN address with `127.0.0.1`.
-- Pending outbox: the service was unavailable; preserve `.brain/observer-outbox/` and rerun the
-  publisher. Do not delete events manually.
-- If memory is incomplete, check the Sync screen, preserve the outbox, and run
-  `observer memory import` to rebuild the copy from the vault.
+- Pending outbox: the service was unavailable; preserve `.brain/observer-outbox/` and
+  `.brain/observer-sql-outbox/`, then rerun the publisher. Do not delete events manually.
+- If memory or usage is incomplete, check the Sync screen, preserve the outbox, and run
+  `observer memory import` to rebuild the load from the vault.
 
 ## Next steps
 
@@ -106,25 +116,34 @@ projection.
 
 ## Data authority
 
-The container is canonical for Observer queries and stores the complete published content. The
-vault remains preserved locally during migration as a transition and recovery copy; Observer
-screens do not complete, archive, repair, or promote state.
+The container SQLite database is canonical for Observer queries and stores the complete published
+content. The vault and any legacy `/data/memory` remain preserved during migration as a transition
+and recovery copy; hooks do not update container Markdown after cutover. Observer screens do not
+complete, archive, repair, or promote state.
 
 ## Minimal API
 
-- `GET /healthz` — availability without project data.
-- `GET /v1/projects` — projects with an accepted snapshot.
+- `GET /healthz` — availability, SQLite migration version, and legacy migration state.
+- `GET /v1/projects` — projects registered in SQLite, with a snapshot when available.
 - `GET /v1/projects/:project_id` — the latest project snapshot.
 - `GET /v1/projects/:project_id/changes` — change summary from the snapshot.
 - `PUT /v1/projects/:project_id` — explicit local registration.
 - `POST /v1/projects/:project_id/snapshot` — idempotent local ingestion.
+- `POST /v1/projects/:project_id/ingest` — idempotent batches of documents, sessions, agents,
+  rollups, calls, and transcripts.
 - `GET /v1/projects/:project_id/memory/tree` — document tree and metadata.
 - `GET /v1/projects/:project_id/memory/document?path=...` — complete Markdown content.
 - `GET /v1/projects/:project_id/memory/search?q=...` — path and body search.
 - `GET /v1/projects/:project_id/sync` — mode, counts, conflicts, and latest event.
-- `PUT /v1/projects/:project_id/sync` — explicitly changes the local mode.
+- `PUT /v1/projects/:project_id/sync` — compatibility configuration; SQL remains authoritative.
 - `GET /v1/projects/:project_id/memory/export` — read-only export with complete content.
 - `POST /v1/projects/:project_id/memory/events` — idempotent batch ingestion.
-
-The `/v1` routes reject oversized bodies and validate project, path, revision, hash, and
-isolation before writing content to the volume.
+- `GET /v1/projects/:project_id/usage/summary` — filterable totals by period, change, session,
+  agent, provider, model, and role.
+- `GET /v1/projects/:project_id/usage/breakdown` — agent, subagent, and model hierarchy.
+- `GET /v1/projects/:project_id/usage/calls` — individual calls with prompt and response.
+- `GET /v1/projects/:project_id/transcripts/:transcript_id` — compressed transcript validated by hash.
+
+The `/v1` routes reject transported or expanded bodies above their limits and validate project,
+path, revision, hash, idempotency, and isolation before writing to SQLite. Use `memory/export` for
+a Markdown copy; it does not alter SQL authority.

package/docs/pt-BR/commands/observer.md

@@ -4,21 +4,23 @@
 
 ## Objetivo
 
-O Observer consolida a observabilidade de vários projetos WendKeep em um serviço local e grava no
-volume Docker uma cópia integral da memória publicada pelos hooks. O conteúdo fica disponível
-para navegação e busca no próprio container, sem depender do Obsidian para consulta.
+O Observer consolida a observabilidade de vários projetos WendKeep em um serviço local. O volume
+Docker mantém o SQLite `/data/observer.sqlite` como autoridade única para documentos, sessões,
+agentes, uso, chamadas e transcripts completos. O conteúdo fica disponível para navegação e busca
+no próprio container, sem depender do Obsidian para consulta.
 
 ## Quando usar
 
-Use para consultar changes, sessões, decisões, bugs, aprendizados, specs, documentos do brain e
-saúde de vários projetos em uma única memória local. Durante a transição, o vault continua sendo
-preservado como cópia de origem para recuperação; o Observer é a autoridade de consulta do seu
-container.
+Use para consultar changes, sessões, decisões, bugs, aprendizados, specs, documentos do brain,
+consumo por agente/modelo e saúde de vários projetos em uma única memória local. Durante a
+transição, o vault e os arquivos Markdown legados continuam preservados como cópia de origem para
+recuperação; o Observer é a autoridade de consulta do seu container.
 
 ## Quando não usar
 
-Não use o Observer para editar, concluir ou arquivar changes, curar memória, armazenar transcripts
-ou expor o serviço na rede. As edições continuam passando pelos hooks e pelo WendKeep local.
+Não use o Observer para editar, concluir ou arquivar changes, curar memória, exportar a autoridade
+de volta para Markdown automaticamente ou expor o serviço na rede. As edições continuam passando
+pelos hooks e pelo WendKeep local.
 
 ## Pré-requisitos
 
@@ -68,11 +70,12 @@ O painel é servido pelo mesmo processo e abre diretamente, sem formulário ou t
 presa ao loopback do computador; não coloque o endereço em uma interface de rede.
 
 O painel mostra a lista multi-projeto, versão, saúde, sessão mais recente, change ativa, contagem
-de changes e data da última captura. Ao abrir um projeto, o workspace oferece as telas Overview,
-Sessões, Memória, Changes e Sincronização. Cada lista abre o documento Markdown completo em um
-leitor read-only, com filtro local, busca no corpo e alternância para a fonte. Os estados de
-carregamento, vazio, servidor indisponível, conflito e dados desatualizados ficam visíveis, e a
-atualização pode ser manual ou automática a cada 15 segundos.
+de changes e data da última captura. Ao abrir um projeto, o workspace oferece Overview, Consumo,
+Sessões, Memória, Changes e Sincronização. A aba Consumo mostra custo total, tokens por categoria,
+agentes principais, subagentes, provedores, modelos, tendência diária, cobertura histórica e
+chamadas com prompt, resposta e transcript completo. Os estados de carregamento, vazio, servidor
+indisponível, conflito, modelo sem tarifa e dados desatualizados ficam visíveis, e a atualização
+pode ser manual ou automática a cada 15 segundos.
 
 Se o navegador mostrar a tela mas a lista falhar, confirme a saúde em
 `http://127.0.0.1:8787/healthz` e verifique se o container está em execução.
@@ -80,25 +83,32 @@ Se o navegador mostrar a tela mas a lista falhar, confirme a saúde em
 ## Resultado esperado
 
 `register` grava `project_id`, nome, versão e data de registro. `publish` lê o vault local,
-produz o snapshot e também envia eventos idempotentes com o conteúdo integral das sessões,
-decisões, bugs, aprendizados, specs, changes, CORE, DIGEST, SHARED_MEMORY e estado do brain. O
-container grava os Markdown em `/data/memory` e mantém `MEMORY_EVENTS.jsonl` e
-`MEMORY_INDEX.json` no volume `observer-data`; não monta `C:\GitHub` nem qualquer
-`.WendKeep-vault`. `memory import` faz a carga inicial e retorna a paridade por arquivo e hash.
-
-O `init` projeta `observer-publish` para `SessionStart` e `Stop` depois dos hooks principais. Sem
-`WENDKEEP_OBSERVER_URL`, o hook é no-op. Com o servidor parado, ele grava os snapshots em
-`.brain/observer-outbox/` e a memória integral em `.brain/observer-memory-outbox/`, sem bloquear a
-sessão; uma execução posterior tenta reenviar os dois tipos de evento.
+produz o snapshot e envia eventos idempotentes para o SQLite com o conteúdo integral das sessões,
+decisões, bugs, aprendizados, specs, changes, CORE, DIGEST, SHARED_MEMORY, estado do brain,
+sessões de agentes, rollups de custo, chamadas e transcripts. O container grava tudo em
+`/data/observer.sqlite`; não monta `C:\GitHub` nem qualquer `.WendKeep-vault`. Markdown é aceito
+somente como conteúdo de uma coluna SQL e volta a existir como arquivo apenas pela exportação
+read-only sob demanda. `memory import` faz a carga inicial e retorna a paridade por arquivo e hash.
+Na migração, o total de custo/token registrado no frontmatter é preservado por uma linha de
+reconciliação quando o ledger detalhado não fecha com ele; essa linha não inventa chamadas.
+Sessões históricas com o mesmo `session_id` recebem uma identidade canônica por arquivo para
+evitar que um rollup sobrescreva o outro.
+
+O `init` projeta `observer-publish` para `SessionStart`, `Stop` e `SubagentStop` depois dos hooks
+principais. Sem servidor disponível, ele grava snapshots em `.brain/observer-outbox/` e eventos
+SQL em `.brain/observer-sql-outbox/`, sem bloquear a sessão; uma execução posterior tenta
+reenviar os lotes. Os lotes SQL são enviados com gzip para que transcripts completos maiores que
+64 MB em JSON puro continuem dentro do limite do transporte; o Observer descomprime e valida o
+corpo antes de ingerir. O outbox é transporte temporário, não autoridade.
 
 ## Erros comuns e diagnóstico
 
 - `project_not_registered`: rode `observer register` antes de publicar.
 - `host loopback`: troque `0.0.0.0` ou endereço LAN por `127.0.0.1`.
-- Outbox pendente: o serviço estava indisponível; preserve `.brain/observer-outbox/` e repita o
-  publisher. Não apague eventos manualmente.
-- Se a memória ficar incompleta, verifique a tela Sincronização, preserve o outbox e rode
-  `observer memory import` para reconstruir a cópia a partir do vault.
+- Outbox pendente: o serviço estava indisponível; preserve `.brain/observer-outbox/` e
+  `.brain/observer-sql-outbox/` e repita o publisher. Não apague eventos manualmente.
+- Se a memória ou o consumo ficarem incompletos, verifique a tela Sincronização, preserve o
+  outbox e rode `observer memory import` para reconstruir a carga a partir do vault.
 
 ## Próximos passos
 
@@ -107,25 +117,34 @@ removido com `docker compose down -v` durante a operação normal, pois isso apa
 
 ## Autoridade dos dados
 
-O container é a memória canônica para consultas do Observer e guarda o conteúdo integral
-publicado. O vault continua preservado localmente durante a migração como cópia de transição e
-fonte de recuperação; as telas do Observer não concluem, arquivam, reparam ou promovem estado.
+O SQLite do container é a memória canônica para consultas do Observer e guarda o conteúdo integral
+publicado. O vault e qualquer `/data/memory` legado continuam preservados durante a migração como
+cópia de transição e fonte de recuperação; os hooks não atualizam Markdown no container depois do
+corte. As telas do Observer não concluem, arquivam, reparam ou promovem estado.
 
 ## API mínima
 
-- `GET /healthz` — disponibilidade sem dados de projeto.
-- `GET /v1/projects` — projetos com snapshot aceito.
+- `GET /healthz` — disponibilidade, versão das migrações SQLite e estado da migração legada.
+- `GET /v1/projects` — projetos registrados no SQLite, com snapshot quando houver.
 - `GET /v1/projects/:project_id` — último snapshot do projeto.
 - `GET /v1/projects/:project_id/changes` — resumo das changes do snapshot.
 - `PUT /v1/projects/:project_id` — registro explícito local.
 - `POST /v1/projects/:project_id/snapshot` — ingestão local idempotente.
+- `POST /v1/projects/:project_id/ingest` — lote idempotente de documentos, sessões, agentes, rollups,
+  chamadas e transcripts.
 - `GET /v1/projects/:project_id/memory/tree` — árvore e metadados dos documentos.
 - `GET /v1/projects/:project_id/memory/document?path=...` — conteúdo Markdown integral.
 - `GET /v1/projects/:project_id/memory/search?q=...` — busca no caminho e no corpo.
 - `GET /v1/projects/:project_id/sync` — modo, contagem, conflitos e último evento.
-- `PUT /v1/projects/:project_id/sync` — altera explicitamente o modo local.
+- `PUT /v1/projects/:project_id/sync` — compatibilidade de configuração; a autoridade continua SQL.
 - `GET /v1/projects/:project_id/memory/export` — exportação read-only com conteúdo completo.
 - `POST /v1/projects/:project_id/memory/events` — ingestão idempotente em lote.
-
-As rotas `/v1` rejeitam corpo acima do limite e validam projeto, caminho, revisão, hash e
-isolamento antes de gravar o conteúdo no volume.
+- `GET /v1/projects/:project_id/usage/summary` — totais filtráveis por período, change, sessão,
+  agente, provedor, modelo e papel.
+- `GET /v1/projects/:project_id/usage/breakdown` — hierarquia de agentes, subagentes e modelos.
+- `GET /v1/projects/:project_id/usage/calls` — chamadas individuais com prompt e resposta.
+- `GET /v1/projects/:project_id/transcripts/:transcript_id` — transcript comprimido, validado por hash.
+
+As rotas `/v1` rejeitam corpo transportado ou expandido acima do limite e validam projeto, caminho,
+revisão, hash, idempotência e isolamento antes de gravar o conteúdo no SQLite. Para preservar uma
+cópia Markdown, use a rota `memory/export`; ela não altera a autoridade SQL.

package/hooks/observer-publish.mjs

@@ -9,6 +9,7 @@ async function main() {
   const result = await publishObserverSnapshot({
     vaultBase: resolved.base,
     projectRoot: resolved.projectRoot,
+    input,
   });
   if (!result.ok && result.error) debugLog('Observer publish fail-open:', result.error);
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wendkeep",
-  "version": "0.71.1",
+  "version": "0.72.0",
   "description": "Vault-first persistent memory for AI coding agents, with an optional profile-aware governance runtime: OFF, FLOW, GUIDE, GOVERN, or ASSURE. Local-first and agent-agnostic (Claude Code, Codex, Cursor…).",
   "type": "module",
   "workspaces": [
@@ -41,7 +41,7 @@
     "node": ">=18"
   },
   "scripts": {
-    "check": "node --check scripts/release.mjs && node --check scripts/release-plan.mjs && node --check bin/wendkeep.mjs && node --check packages/cli/src/index.mjs && node --check src/init.mjs && node --check src/doctor.mjs && node --check src/project-vault.mjs && node --check src/observer-snapshot.mjs && node --check src/observer-store.mjs && node --check src/observer-memory.mjs && node --check src/observer-memory-publish.mjs && node --check src/observer-server.mjs && node --check src/observer.mjs && node --check src/observer-publish.mjs && node --check src/operating-profile.mjs && node --check src/profile.mjs && node --check src/flow.mjs && node --check web/observer/app.mjs && node --check hooks/observer-publish.mjs && node --check hooks/operating-profile-runtime.mjs && node --check hooks/operating-profile-task-store.mjs && node --check hooks/flow-core.mjs && node --check hooks/flow-protected-policy.mjs && node --check hooks/git-snapshot.mjs && node --check hooks/vault-path-safety.mjs && node --check hooks/vault-runtime-store.mjs && node --check packages/harness/src/index.mjs && node --check packages/harness/src/flow-store.mjs && node --check packages/harness/src/operating-profile.mjs && node --check packages/harness/src/sensors-core.mjs && node --check packages/integrations/src/host-hooks.mjs && node --check packages/integrations/src/hook-envelope.mjs && node --check packages/integrations/src/prompt-content.mjs && node --check packages/integrations/src/transcript-usage.mjs && node --check packages/integrations/src/transcripts.mjs && node --check packages/integrations/src/session-identity.mjs && node --check packages/integrations/src/index.mjs && node --check packages/mcp/src/config.mjs && node --check packages/mcp/src/index.mjs && node --check packages/vault/src/index.mjs && node --check packages/vault/src/project-vault.mjs && node --check packages/vault/src/vault-path-safety.mjs && node --check packages/vault/src/locale.mjs && node --check packages/vault/src/memory-schema.mjs && node --check packages/vault/src/memory-mode.mjs && node --check packages/vault/src/memory-handoff.mjs && node --check packages/vault/src/memory-store.mjs && node --check packages/vault/src/validate-core.mjs && node --check packages/vault/src/validate-memory.mjs",
+    "check": "node --check scripts/release.mjs && node --check scripts/release-plan.mjs && node --check bin/wendkeep.mjs && node --check packages/cli/src/index.mjs && node --check src/init.mjs && node --check src/doctor.mjs && node --check src/project-vault.mjs && node --check src/observer-snapshot.mjs && node --check src/observer-store.mjs && node --check src/observer-memory.mjs && node --check src/observer-memory-publish.mjs && node --check src/observer-sql-store.mjs && node --check src/observer-sql-migrate.mjs && node --check src/observer-sql-publish.mjs && node --check src/observer-transcript-store.mjs && node --check src/observer-server.mjs && node --check src/observer.mjs && node --check src/observer-publish.mjs && node --check src/operating-profile.mjs && node --check src/profile.mjs && node --check src/flow.mjs && node --check web/observer/app.mjs && node --check hooks/observer-publish.mjs && node --check hooks/operating-profile-runtime.mjs && node --check hooks/operating-profile-task-store.mjs && node --check hooks/flow-core.mjs && node --check hooks/flow-protected-policy.mjs && node --check hooks/git-snapshot.mjs && node --check hooks/vault-path-safety.mjs && node --check hooks/vault-runtime-store.mjs && node --check packages/harness/src/index.mjs && node --check packages/harness/src/flow-store.mjs && node --check packages/harness/src/operating-profile.mjs && node --check packages/harness/src/sensors-core.mjs && node --check packages/integrations/src/host-hooks.mjs && node --check packages/integrations/src/hook-envelope.mjs && node --check packages/integrations/src/prompt-content.mjs && node --check packages/integrations/src/transcript-usage.mjs && node --check packages/integrations/src/transcripts.mjs && node --check packages/integrations/src/session-identity.mjs && node --check packages/integrations/src/index.mjs && node --check packages/mcp/src/config.mjs && node --check packages/mcp/src/index.mjs && node --check packages/vault/src/index.mjs && node --check packages/vault/src/project-vault.mjs && node --check packages/vault/src/vault-path-safety.mjs && node --check packages/vault/src/locale.mjs && node --check packages/vault/src/memory-schema.mjs && node --check packages/vault/src/memory-mode.mjs && node --check packages/vault/src/memory-handoff.mjs && node --check packages/vault/src/memory-store.mjs && node --check packages/vault/src/validate-core.mjs && node --check packages/vault/src/validate-memory.mjs",
     "test": "node --test --test-concurrency=2",
     "release": "node scripts/release.mjs",
     "release:dry": "node scripts/release.mjs --dry-run",

package/packages/integrations/src/host-hooks.mjs

@@ -24,6 +24,8 @@ export const SESSION_HOOKS = [
   { event: 'PostToolUse', matcher: 'AskUserQuestion', name: 'decision-capture', timeout: 15, statusMessage: 'wendkeep: recording decision' },
   // Refresh subagent/workflow telemetry as each subagent finishes (resilient to a missed Stop).
   { event: 'SubagentStop', matcher: null, name: 'subagent-stop', timeout: 20, codex: true, statusMessage: 'wendkeep: subagent telemetry' },
+  // Publish the SQL observer projection after the subagent telemetry is settled.
+  { event: 'SubagentStop', matcher: null, name: 'observer-publish', timeout: 5, order: 20, codex: true, statusMessage: 'wendkeep: publishing local observer usage' },
   // Log plan/task progress into the active session note when a task is marked complete.
   // codex: TaskCompleted is not in Codex's hook event enum.
   { event: 'TaskCompleted', matcher: null, name: 'task-log', timeout: 10, statusMessage: 'wendkeep: plan progress' },

package/schema/observer/001-authority.sql

@@ -0,0 +1,107 @@
+PRAGMA foreign_keys = ON;
+
+CREATE TABLE IF NOT EXISTS schema_migrations (
+  version INTEGER PRIMARY KEY,
+  name TEXT NOT NULL,
+  applied_at TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS projects (
+  project_id TEXT PRIMARY KEY,
+  project_name TEXT NOT NULL,
+  wendkeep_version TEXT NOT NULL DEFAULT '',
+  registered_at TEXT NOT NULL,
+  updated_at TEXT NOT NULL
+);
+
+CREATE TABLE IF NOT EXISTS ingest_events (
+  event_id TEXT PRIMARY KEY,
+  project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
+  kind TEXT NOT NULL,
+  payload_hash TEXT NOT NULL,
+  payload_json TEXT NOT NULL,
+  occurred_at TEXT NOT NULL,
+  ingested_at TEXT NOT NULL,
+  status TEXT NOT NULL DEFAULT 'accepted'
+);
+
+CREATE INDEX IF NOT EXISTS idx_ingest_events_project_time
+  ON ingest_events(project_id, occurred_at);
+
+CREATE TABLE IF NOT EXISTS memory_events (
+  event_id TEXT PRIMARY KEY REFERENCES ingest_events(event_id) ON DELETE CASCADE,
+  project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
+  entity_type TEXT NOT NULL,
+  logical_path TEXT NOT NULL,
+  operation TEXT NOT NULL,
+  revision INTEGER NOT NULL DEFAULT 1,
+  content_hash TEXT NOT NULL DEFAULT '',
+  source_session_id TEXT NOT NULL DEFAULT '',
+  source_turn_id TEXT NOT NULL DEFAULT '',
+  occurred_at TEXT NOT NULL,
+  payload_json TEXT NOT NULL,
+  UNIQUE(project_id, logical_path, revision)
+);
+
+CREATE INDEX IF NOT EXISTS idx_memory_events_project_path
+  ON memory_events(project_id, logical_path, revision);
+
+CREATE TABLE IF NOT EXISTS documents (
+  document_id TEXT PRIMARY KEY,
+  project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
+  logical_path TEXT NOT NULL,
+  entity_type TEXT NOT NULL,
+  title TEXT NOT NULL DEFAULT '',
+  content TEXT NOT NULL DEFAULT '',
+  metadata_json TEXT NOT NULL DEFAULT '{}',
+  content_hash TEXT NOT NULL DEFAULT '',
+  revision INTEGER NOT NULL DEFAULT 1,
+  source_session_id TEXT NOT NULL DEFAULT '',
+  source_turn_id TEXT NOT NULL DEFAULT '',
+  captured_at TEXT NOT NULL,
+  deleted_at TEXT,
+  UNIQUE(project_id, logical_path)
+);
+
+CREATE INDEX IF NOT EXISTS idx_documents_project_type
+  ON documents(project_id, entity_type, deleted_at);
+
+CREATE INDEX IF NOT EXISTS idx_documents_project_revision
+  ON documents(project_id, revision);
+
+CREATE TABLE IF NOT EXISTS sessions (
+  session_id TEXT PRIMARY KEY,
+  project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
+  provider TEXT NOT NULL DEFAULT '',
+  status TEXT NOT NULL DEFAULT 'unknown',
+  summary TEXT NOT NULL DEFAULT '',
+  change_slug TEXT NOT NULL DEFAULT '',
+  started_at TEXT,
+  ended_at TEXT,
+  updated_at TEXT NOT NULL,
+  metadata_json TEXT NOT NULL DEFAULT '{}'
+);
+
+CREATE INDEX IF NOT EXISTS idx_sessions_project_time
+  ON sessions(project_id, started_at, ended_at);
+
+CREATE TABLE IF NOT EXISTS agent_runs (
+  agent_id TEXT PRIMARY KEY,
+  project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
+  session_id TEXT NOT NULL REFERENCES sessions(session_id) ON DELETE CASCADE,
+  parent_agent_id TEXT,
+  role TEXT NOT NULL DEFAULT 'main',
+  agent_name TEXT NOT NULL DEFAULT '',
+  agent_type TEXT NOT NULL DEFAULT '',
+  workflow TEXT NOT NULL DEFAULT '',
+  status TEXT NOT NULL DEFAULT 'unknown',
+  model TEXT NOT NULL DEFAULT '',
+  effort TEXT NOT NULL DEFAULT '',
+  started_at TEXT,
+  ended_at TEXT,
+  metadata_json TEXT NOT NULL DEFAULT '{}',
+  FOREIGN KEY(parent_agent_id) REFERENCES agent_runs(agent_id) ON DELETE SET NULL
+);
+
+CREATE INDEX IF NOT EXISTS idx_agent_runs_project_session
+  ON agent_runs(project_id, session_id, role);

package/schema/observer/002-usage.sql

@@ -0,0 +1,72 @@
+CREATE TABLE IF NOT EXISTS usage_rollups (
+  rollup_key TEXT PRIMARY KEY,
+  project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
+  session_id TEXT NOT NULL REFERENCES sessions(session_id) ON DELETE CASCADE,
+  agent_id TEXT NOT NULL REFERENCES agent_runs(agent_id) ON DELETE CASCADE,
+  role TEXT NOT NULL DEFAULT 'main',
+  provider TEXT NOT NULL DEFAULT '',
+  model_provider TEXT NOT NULL DEFAULT '',
+  model TEXT NOT NULL DEFAULT '',
+  effort TEXT NOT NULL DEFAULT '',
+  calls INTEGER NOT NULL DEFAULT 0,
+  tokens_input INTEGER NOT NULL DEFAULT 0,
+  tokens_cache_write INTEGER NOT NULL DEFAULT 0,
+  tokens_cache_read INTEGER NOT NULL DEFAULT 0,
+  tokens_output INTEGER NOT NULL DEFAULT 0,
+  tokens_reasoning INTEGER NOT NULL DEFAULT 0,
+  tokens_total INTEGER NOT NULL DEFAULT 0,
+  cost_usd REAL NOT NULL DEFAULT 0,
+  cost_status TEXT NOT NULL DEFAULT 'unknown',
+  pricing_source TEXT NOT NULL DEFAULT '',
+  pricing_version TEXT NOT NULL DEFAULT '',
+  wasted_usd REAL NOT NULL DEFAULT 0,
+  revision INTEGER NOT NULL DEFAULT 1,
+  occurred_at TEXT NOT NULL,
+  source_event_id TEXT NOT NULL REFERENCES ingest_events(event_id) ON DELETE CASCADE,
+  metadata_json TEXT NOT NULL DEFAULT '{}'
+);
+
+CREATE INDEX IF NOT EXISTS idx_usage_rollups_project_time
+  ON usage_rollups(project_id, occurred_at);
+
+CREATE INDEX IF NOT EXISTS idx_usage_rollups_project_agent
+  ON usage_rollups(project_id, agent_id, role);
+
+CREATE INDEX IF NOT EXISTS idx_usage_rollups_project_model
+  ON usage_rollups(project_id, model_provider, model);
+
+CREATE TABLE IF NOT EXISTS llm_calls (
+  call_id TEXT PRIMARY KEY,
+  project_id TEXT NOT NULL REFERENCES projects(project_id) ON DELETE CASCADE,
+  session_id TEXT NOT NULL REFERENCES sessions(session_id) ON DELETE CASCADE,
+  agent_id TEXT NOT NULL REFERENCES agent_runs(agent_id) ON DELETE CASCADE,
+  role TEXT NOT NULL DEFAULT 'main',
+  provider TEXT NOT NULL DEFAULT '',
+  model_provider TEXT NOT NULL DEFAULT '',
+  model TEXT NOT NULL DEFAULT '',
+  effort TEXT NOT NULL DEFAULT '',
+  sequence INTEGER NOT NULL DEFAULT 0,
+  occurred_at TEXT NOT NULL,
+  tokens_input INTEGER NOT NULL DEFAULT 0,
+  tokens_cache_write INTEGER NOT NULL DEFAULT 0,
+  tokens_cache_read INTEGER NOT NULL DEFAULT 0,
+  tokens_output INTEGER NOT NULL DEFAULT 0,
+  tokens_reasoning INTEGER NOT NULL DEFAULT 0,
+  tokens_total INTEGER NOT NULL DEFAULT 0,
+  cost_usd REAL NOT NULL DEFAULT 0,
+  cost_status TEXT NOT NULL DEFAULT 'unknown',
+  transcript_id TEXT,
+  prompt_text TEXT NOT NULL DEFAULT '',
+  response_text TEXT NOT NULL DEFAULT '',
+  status TEXT NOT NULL DEFAULT 'complete',
+  metadata_json TEXT NOT NULL DEFAULT '{}'
+);
+
+CREATE INDEX IF NOT EXISTS idx_llm_calls_project_time
+  ON llm_calls(project_id, occurred_at);
+
+CREATE INDEX IF NOT EXISTS idx_llm_calls_project_agent
+  ON llm_calls(project_id, agent_id, sequence);
+
+CREATE INDEX IF NOT EXISTS idx_llm_calls_project_model
+  ON llm_calls(project_id, model_provider, model);