@superbased/observer 1.8.2 → 1.8.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -82,30 +82,33 @@ the [main repo](https://github.com/marmutapp/superbased-observer).
82
82
  ## Five-minute quickstart
83
83
 
84
84
  ```bash
85
- # 1) Install. `observer init` is OPTIONAL — only run it if you want
86
- # the MCP server registered with your AI clients (gives them
87
- # on-demand tools like check_file_freshness / get_cost_summary
88
- # at the cost of ~1,800 tokens of schema per turn).
85
+ # 1) Install.
89
86
  npm install -g @superbased/observer
90
- observer init # OPTIONAL — interactive: pick clients;
91
- # writes MCP + codex proxy-route into AI client configs.
92
- # Skip this step for an MCP-free install.
93
-
94
- # 2) Start the long-running services (proxy + watcher + dashboard).
95
- # Auto-registers HOOKS for every detected AI tool on first launch.
96
- observer start &
97
-
98
- # 3) Engage the proxy by pointing your AI client at the local URL.
99
- # See "Per-AI-client setup" for the matching env var.
100
- export ANTHROPIC_BASE_URL=http://127.0.0.1:8820 # Claude Code
101
- export OPENAI_BASE_URL=http://127.0.0.1:8820/v1 # Codex / OpenAI
102
-
103
- # 4) Open the dashboard.
104
- open http://127.0.0.1:8081/ # macOS
105
- xdg-open http://127.0.0.1:8081/ # Linux
106
- start http://127.0.0.1:8081/ # Windows
87
+
88
+ # 2) Start everything: proxy + watcher + dashboard in one process.
89
+ # Hooks auto-register for every detected AI tool, and the
90
+ # dashboard opens in your browser (suppress with --no-open).
91
+ observer start
107
92
  ```
108
93
 
94
+ From here the dashboard drives:
95
+
96
+ 3. **Route your AI client through the proxy** — on the Compression
97
+ tab's **Proxy** banner, click your tool's status pill, then
98
+ **Route through the observer proxy…**. The button previews the
99
+ exact file change and writes only on confirm. (Every other
100
+ routing mechanism — `observer init`, the `observer claude` /
101
+ `observer codex` wrappers, plain env vars — is listed in
102
+ [Per-AI-client setup](#per-ai-client-setup).)
103
+ 4. **Use your AI tool as normal.** The Overview tab's onboarding
104
+ checklist tracks your first captured session; cost, compression,
105
+ and cache numbers populate within minutes of real activity.
106
+
107
+ `observer init` is OPTIONAL — run it only if you want the MCP server
108
+ registered with your AI clients (gives them on-demand tools like
109
+ `check_file_freshness` / `get_cost_summary`, at the cost of ~1,800
110
+ tokens of schema per turn). Skip it for an MCP-free install.
111
+
109
112
  **What `start` does vs what `init` adds:**
110
113
 
111
114
  | Step | Hooks | Proxy listening | Watcher | Dashboard | MCP in AI clients | Codex proxy route |
@@ -117,16 +120,46 @@ start http://127.0.0.1:8081/ # Windows
117
120
  MCP and codex routing are explicit-only because both write per-client
118
121
  config files. Hooks self-heal on every `start`.
119
122
 
120
- After ten minutes of normal AI-coding usage, the dashboard will be
121
- populated with cost over time, per-tool activity, compression
122
- savings, and stale-reread waste signals.
123
-
124
123
 
125
124
  ## Per-AI-client setup
126
125
 
127
- Different clients send to different upstreams. The local proxy on
128
- `127.0.0.1:8820` handles all of them — routes by URL path. Set the
129
- env var that matches your client(s); both can coexist on one machine.
126
+ ### Routing through the proxy the canonical section
127
+
128
+ Accurate token counts, conversation compression, and cache tracking
129
+ all require your AI client to send its API traffic through the local
130
+ proxy on `127.0.0.1:8820`. There is one decision to make — *how* to
131
+ route — and four mechanisms that all end at the same proxy. Pick the
132
+ first one that fits:
133
+
134
+ | Mechanism | What it does | Use it when |
135
+ |---|---|---|
136
+ | **Dashboard button** *(recommended)* | On the Compression tab's **Proxy** banner, click your tool's status pill, then **Route through the observer proxy…**. It previews the exact file change and writes only on confirm: Claude Code gets `"env": { "ANTHROPIC_BASE_URL": … }` in `~/.claude/settings.json`; Codex gets an `observer` model provider in `~/.codex/config.toml`. Durable — every later session routes automatically, no wrapper, no env vars. Undo: remove the entry, or `observer uninstall --claude-code` / `--codex`. | You have a browser open. This is the default path. |
137
+ | **`observer init`** | Writes the same durable routing from the CLI, alongside hooks + MCP registration (each side defaults on; opt out with `--skip-hooks` / `--skip-mcp` / `--skip-proxy-route`). | Headless machines, scripted setups. |
138
+ | **Wrapper commands** — `observer claude` / `observer codex` | Launch the tool with routing injected for that session only; nothing is written to any config file. The claude wrapper also re-exports a fresh Pro/Max OAuth token so the SDK's OAuth path can't bypass the proxy (an expired stored token is left alone — Claude Code then refreshes natively). `observer claude --verify` runs the pre-flight checks without launching. | Trying the proxy out, or you don't want config writes. |
139
+ | **Env var export** | `export ANTHROPIC_BASE_URL=http://127.0.0.1:8820` (Anthropic clients) or `export OPENAI_BASE_URL=http://127.0.0.1:8820/v1` (OpenAI clients — note the `/v1`) in the shell that launches the tool. | One-off shells; clients without a button or wrapper. |
140
+
141
+ Routing is per-tool, not global: route Claude Code and Codex
142
+ independently, and both can coexist on one machine — the proxy
143
+ routes by URL path.
144
+
145
+ > **Claude Code + MCP servers**: set `ENABLE_TOOL_SEARCH=true` in the
146
+ > same environment. Claude Code's SDK disables deferred MCP schema
147
+ > loading whenever `ANTHROPIC_BASE_URL` is set and eager-inlines every
148
+ > registered MCP schema (~+21K tokens/turn) — enough to turn the proxy
149
+ > into a net cost. The override restores lazy loading; the proxy
150
+ > forwards `tool_reference` blocks byte-identically.
151
+
152
+ > **If the daemon is down while a tool is routed**, Claude Code does
153
+ > not silently bypass to the real API — it retries the dead port
154
+ > (verified live: it hangs rather than failing over). `observer
155
+ > doctor` and the dashboard's Health panel flag this state as a
156
+ > "proxy routing gap"; bring the daemon back with `observer start`.
157
+
158
+ ### Capture modes per client
159
+
160
+ The proxy covers the clients that speak the Anthropic / OpenAI wire
161
+ APIs. Everything else is captured passively from session logs. The
162
+ table below is the full per-client reference.
130
163
 
131
164
  | AI client | Env var | Capture mode |
132
165
  |---------------------------------|-------------------------------------------------------|--------------|
@@ -134,15 +167,17 @@ env var that matches your client(s); both can coexist on one machine.
134
167
  | **Cursor** (Anthropic mode) | `ANTHROPIC_BASE_URL=http://127.0.0.1:8820` | proxy + JSONL |
135
168
  | **Codex** | `OPENAI_BASE_URL=http://127.0.0.1:8820/v1` (note `/v1`) | proxy + JSONL with API-key auth; ChatGPT-plan login currently behaves as JSONL only |
136
169
  | **Cursor** (OpenAI mode) | `OPENAI_BASE_URL=http://127.0.0.1:8820/v1` | proxy + JSONL |
137
- | **Cline / Roo Code** | `ANTHROPIC_BASE_URL=...` or `OPENAI_BASE_URL=...` per provider | proxy + JSONL |
170
+ | **Cline (VS Code) / Roo Code** | `ANTHROPIC_BASE_URL=...` or `OPENAI_BASE_URL=...` per provider | proxy + JSONL |
171
+ | **Cline CLI** (npm `cline` 3.0.20+) | (no proxy route yet — the CLI ignores base-URL env vars; per-provider `baseUrl` overrides live in `~/.cline/data/settings/providers.json`, a credentials file only the operator should edit) | JSONL — SQLite session metadata at `~/.cline/data/db/sessions.db` (schema v1, 28 columns) plus per-session `<id>.messages.json` content blocks. Captures `user_prompt` / `assistant_text` / `tool_use` / paired `tool_result` rows + Tier 2 per-API-call token usage from `modelInfo` overrides. Subagent + team linkage via 5 new `ActionMetadata` fields (`ParentSessionID`, `ParentAgentID`, `AgentID`, `IsSubagent`, `TeamName`). 28-tool taxonomy (10 core + 18 `team_*` coordination primitives). WSL2 over `/mnt/c` uses the same `stageMirror` pattern as opencode/kilocode. Tagged `Source=jsonl, Reliability=approximate`. |
138
172
  | **GitHub Copilot** | (no proxy yet) | JSONL only |
139
173
  | **OpenCode** ([opencode.ai](https://opencode.ai/)) | (no proxy yet) | SQLite — actual install path is `~/.local/share/opencode/opencode.db` (XDG). Captures **token counts + model + cost** per assistant message from OpenCode's InfoData (`tokens.input/output/reasoning/cache.{read,write}` + `cost`); **subtask** parts → `spawn_subagent` actions; **todo** table → `todo_update` actions; tool-name coverage extended to webfetch/websearch/task/todowrite/todoread/multiedit. Tagged `Source=jsonl, Reliability=approximate`. |
140
174
  | **OpenClaw** ([openclaw.ai](https://openclaw.ai/)) | (no proxy yet) | JSONL + sqlite — `~/.openclaw/tasks/runs.sqlite` + `~/.openclaw/agents/<agent>/sessions/sessions.json` |
141
175
  | **Pi** ([pi.dev](https://pi.dev/)) | (no proxy yet) | JSONL — `~/.pi/agent/sessions/--<path>--/*.jsonl` (per upstream `docs/session-format.md` v3). Captures user / assistant / toolResult / `bashExecution` message roles; `usage.cost.total` → per-message USD; terminal `stopReason` (stop/length/error/aborted) → `task_complete` with `success=false` for failures (mid-turn `toolUse` is correctly skipped); `thinking` blocks surface as preceding reasoning. Tagged `Source=jsonl, Reliability=approximate`. |
142
176
  | **Google Antigravity** | (no proxy yet) | Encrypted protobuf — `~/.gemini/antigravity/conversations/*.pb` (Linux-native) and the matching Windows-side path on WSL2. Observer ships a per-OS Chromium-pattern `oscrypt` key fetcher (macOS Keychain / libsecret / DPAPI / WSL2-via-PowerShell helper) and a multi-cipher try-loop for local decryption. Sessions whose ciphers don't validate locally fall back to the language_server's `GetCascadeTrajectory` gRPC endpoint via a built-in helper (`antigravity-bridge.exe` on WSL2 / native gRPC elsewhere) — extracts model + per-turn token counts + Tier 0–6 ToolEvents (file views, artifact edits/writes, user prompts, assistant text, run_command terminal snapshots, structured plan steps, final summaries). State index + per-conversation title/workspace URI read from `state.vscdb` + `state.vscdb.backup`. Tagged `Source=jsonl, Reliability=approximate`. |
143
177
  | **Kilo Code IDE extension (legacy)** | (no proxy yet) | JSON — `<vsCodeGlobalStorage>/kilocode.kilo-code/tasks/<taskId>/api_conversation_history.json`. The legacy Kilo extension is a Cline + Roo Code fork and shares the Cline parser; emitted rows are re-tagged `Tool="kilo-code"` so dashboard rollups don't blur Kilo activity into Cline. Tagged `Source=jsonl, Reliability=approximate`. |
144
- | **Kilo Code CLI (current)** | Optional: `OPENAI_BASE_URL=http://127.0.0.1:8820/v1` (Kilo Gateway is OpenAI-compatible at the wire) | SQLite — `~/.local/share/kilo/kilo.db` on every OS (Kilo intentionally mirrors XDG; Windows does NOT use `%APPDATA%`). The new `@kilocode/cli` (npm) is a fork of sst/opencode and uses the same `message`/`part`/`todo` tables shape with Kilo additions (`project`, `workspace`, `event`, `session_message`, `account`, `permission`, `session_share`). Captures **token counts + model + cost** per assistant message from `message.data.tokens = {total, input, output, reasoning, cache: {read, write}}`. Tool name coverage inherits OpenCode's surface (`read` → `read_file`, `bash` → `run_command`, `websearch` → `web_search`, etc.). Tagged `Source=jsonl, Reliability=approximate`. |
178
+ | **Kilo Code CLI (current)** | (no proxy route yet — base-URL env vars are not honored; a project-scoped `kilo.json` provider `baseURL` override reaches the proxy, but the Gateway provider's model-catalog calls share that base URL and have no upstream there — see `docs/kilo-code-adapter.md`) | SQLite — `~/.local/share/kilo/kilo.db` on every OS (Kilo intentionally mirrors XDG; Windows does NOT use `%APPDATA%`). The new `@kilocode/cli` (npm) is a fork of sst/opencode and uses the same `message`/`part`/`todo` tables shape with Kilo additions (`project`, `workspace`, `event`, `session_message`, `account`, `permission`, `session_share`). Captures **token counts + model + cost** per assistant message from `message.data.tokens = {total, input, output, reasoning, cache: {read, write}}`. Tool name coverage inherits OpenCode's surface (`read` → `read_file`, `bash` → `run_command`, `websearch` → `web_search`, etc.). Tagged `Source=jsonl, Reliability=approximate`. |
145
179
  | **Gemini CLI** | (no proxy yet) | JSONL or single-object JSON — `~/.gemini/tmp/<hash>/chats/session-*.{json,jsonl}`. Dual-format dispatch: legacy single-object JSON (size-based cursor, cline-style) and proposed JSONL event records (byte-offset cursor, issue [#15292](https://github.com/google-gemini/gemini-cli/issues/15292)). Action mapping covers `read_file` / `write_file` / `edit_file` / `run_command` / `search_files` / `web_fetch` and arbitrary MCP tool calls. Project root falls back through tool-call `cwd` → `~/.gemini/history/<hash>/.git/config` worktree pointer → synthetic `[gemini-cli:<hash>]` key (promoted via ON CONFLICT DO UPDATE on `sessions.project_id` once a future scan supplies a real cwd). Tagged `Source=jsonl, Reliability=approximate`. |
180
+ | **Hermes Agent** ([Nous Research](https://github.com/NousResearch/hermes-agent)) | (provider-routed; uses your existing `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` when set) | Hooks + SQLite — Python plugin at `~/.hermes/plugins/superbased-observer/` registers `post_tool_call` / `post_api_request` / `on_session_start` / `on_session_end` / `subagent_stop` callbacks (fire-and-forget via `observer hook hermes`); the watcher walks `~/.hermes/state.db` (schema v14, `messages.active = 1` filter) and emits `ToolEvent` + `TokenEvent` rows via `modernc.org/sqlite`. 70+ Hermes tools fold into the normalized action set (`read_file` / `write_file` / `patch` / `terminal` / `search_files` / `web_search` / `web_extract` / `browser_*` / `delegate_task` / `todo` / `clarify` / `memory` + `mcp_call` fall-through). Token bundles lifted from `post_api_request.usage{input/output/cache/reasoning_tokens}`; OpenRouter `:suffix` tails (`:free`, `:beta`, `:fast`) preserved so the dashboard distinguishes paid vs free tiers. Install via `observer init --hermes`; backfill via `observer backfill --hermes-rescan`. Tagged `Source=jsonl, Reliability=approximate`. |
146
181
 
147
182
  **JSONL-only** clients are captured passively by the watcher whenever
148
183
  `observer start` is running. Hooks self-heal on every `start`, so a
@@ -151,8 +186,8 @@ for them on the Compression tab (those need the proxy), but every tool
151
186
  call shows up on Sessions / Actions / Discovery / Tools / Patterns and
152
187
  the JSONL-derived token counts feed the Cost tab. Reliability tagging
153
188
  is per-adapter: Claude Code emits `unreliable` (the JSONL stream uses
154
- streaming-time placeholder counts per spec §24); Codex / Cline / Pi /
155
- OpenCode / OpenClaw / Antigravity / Gemini CLI / Kilo Code / Kilo Code CLI emit `approximate`
189
+ streaming-time placeholder counts per spec §24); Codex / Cline / Cline CLI /
190
+ Pi / OpenCode / OpenClaw / Antigravity / Gemini CLI / Hermes / Kilo Code / Kilo Code CLI emit `approximate`
156
191
  (provider-reported usage that hasn't been reconciled against an
157
192
  upstream invoice).
158
193
 
@@ -161,32 +196,6 @@ For **Codex specifically**, Observer currently has two practical support modes:
161
196
  - `Proxy + JSONL`: Codex is routed through `OPENAI_BASE_URL=http://127.0.0.1:8820/v1` and Observer can link proxy turns to the session, so live compression metrics are available.
162
197
  - `JSONL only`: Observer can still recover sessions, actions, and approximate token counts from `~/.codex/sessions`, but live proxy compression is currently not available when Codex is logged in with a ChatGPT plan on the local machine.
163
198
 
164
- ### Persistent setups
165
-
166
- **Claude Code** (`~/.claude/settings.json`):
167
-
168
- ```json
169
- {
170
- "env": { "ANTHROPIC_BASE_URL": "http://127.0.0.1:8820" },
171
- "hooks": { /* `observer init` writes these */ }
172
- }
173
- ```
174
-
175
- **Codex** (`~/.codex/config.toml`):
176
-
177
- ```toml
178
- [env]
179
- OPENAI_BASE_URL = "http://127.0.0.1:8820/v1"
180
- ```
181
-
182
- **Shell rc** (`~/.bashrc` / `~/.zshrc`) — affects every program:
183
-
184
- ```bash
185
- export ANTHROPIC_BASE_URL=http://127.0.0.1:8820
186
- export OPENAI_BASE_URL=http://127.0.0.1:8820/v1
187
- ```
188
-
189
-
190
199
  ## Architecture in detail
191
200
 
192
201
  <p align="center">
@@ -252,8 +261,10 @@ Pure-Go via `modernc.org/sqlite`, no CGO. WAL mode by default.
252
261
 
253
262
  ### 4. Local dashboard (`:8081`)
254
263
 
255
- Eight tabs covering: Overview, Cost, Sessions, Actions, Tools,
256
- Compression, Discovery, Patterns. See [Dashboard tour](#dashboard-tour).
264
+ Fifteen tabs in four groups — Monitor (Overview, Live, Sessions,
265
+ Actions, Search), Analyze (Cost, Analysis, Tools), Optimize
266
+ (Compression, Cache, Suggestions, Discovery, Patterns), Configure
267
+ (Privacy, Settings). See [Dashboard tour](#dashboard-tour).
257
268
 
258
269
  Static HTML + Chart.js. No analytics, no external requests.
259
270
 
@@ -277,7 +288,24 @@ running `init`.
277
288
 
278
289
  ## Dashboard tour
279
290
 
280
- Open `http://127.0.0.1:8081/` after `observer start`. Ten tabs.
291
+ `observer start` opens the dashboard automatically on interactive
292
+ launches (suppress with `--no-open`; the URL is also printed —
293
+ `http://127.0.0.1:8081/` by default). Fifteen tabs in four nav groups
294
+ (Monitor / Analyze / Optimize / Configure) — the tour below covers the
295
+ core surfaces; Live (recent sessions with a real-time action feed),
296
+ Search (full-text over captured tool outputs), and Privacy (capture
297
+ map + scrub tester) are self-explanatory once you're in. On an empty
298
+ database the Overview tab leads with a three-step onboarding
299
+ checklist that takes you from zero to your first captured session
300
+ without leaving the browser.
301
+
302
+ **Just looking?** The empty-database Overview also offers **demo
303
+ mode**: one click seeds a temporary synthetic dataset so every chart
304
+ and table renders with realistic data — parsed through the real
305
+ ingest pipeline into a temp DB; your real `observer.db` is never read
306
+ or written. A persistent banner marks demo state, and one click
307
+ clears it again. The fastest way to evaluate the dashboard before
308
+ routing any real traffic.
281
309
 
282
310
  ### Overview tab
283
311
 
@@ -355,7 +383,7 @@ was tier-fallback rather than billing-grade ("accurate" reliability).
355
383
  Click a row to open the session-detail panel:
356
384
 
357
385
  <p align="center">
358
- <img src="https://github.com/marmutapp/superbased-observer/raw/main/docs/assets/screenshots/10-session-detail.png" alt="Session detail slide-over" width="900">
386
+ <img src="https://github.com/marmutapp/superbased-observer/raw/main/docs/assets/screenshots/11-session-detail.png" alt="Session detail slide-over" width="900">
359
387
  </p>
360
388
 
361
389
 
@@ -435,10 +463,119 @@ saved by trimming requests before forwarding upstream:
435
463
  score (for drops), and a **Source** column showing whether the
436
464
  event came from a main-thread or sub-agent runtime call
437
465
 
466
+ ### Cache tab
467
+
468
+ <p align="center">
469
+ <img src="https://github.com/marmutapp/superbased-observer/raw/main/docs/assets/screenshots/08-cache.png" alt="Cache tab" width="900">
470
+ </p>
471
+
472
+ Anthropic prompt-cache observation, attribution, and forecasting —
473
+ how well your sessions are reusing provider-cached prefixes.
474
+
475
+ - **KPI tiles**: Cache ratio (cache_read ÷ cache_write tokens; the
476
+ headline cache-payback signal), Cache read, Cache write, and
477
+ Avoidable spend / Event count. The avoidable-spend tile is the
478
+ dollar overhead of cache rewrites that wouldn't have happened on
479
+ a perfectly cache-friendly session — it surfaces in warn tone when
480
+ non-zero.
481
+ - **By model + By project tables** — for each rollup dimension: R%/W%
482
+ mix bars + absolute Read / Write / Events + cache Ratio + Avoidable
483
+ $. Identifies the model or repo that's pulling the most overhead.
484
+ - **Top causes histogram** — proportional bar list of `cache_events.cause`
485
+ values. Healthy baseline is `suffix_growth` + `hit` dominating
486
+ (info-toned warm prefix growth). Real invalidations (`system_changed`,
487
+ `expiry_rewrite`, `model_switch_rewrite`) render in warn tone. Causes
488
+ that legitimately fire on real operator toggles — currently
489
+ `tools_changed` on MCP server connect/disconnect — render with a
490
+ neutral "flagged" pill rather than alarm-red.
491
+ - **Worst sessions table** — ranked by rewrite count; click a row to
492
+ open the session's Cache panel with the full per-turn timeline.
493
+
494
+ #### How the cache data gets captured
495
+
496
+ Two capture paths feed the same engine; both write the same three
497
+ local-only tables (`cache_segments`, `cache_entries`, `cache_events`;
498
+ migrations 036 + 037 — never pushed to a Teams org server, pinned by
499
+ `tests/invariant/privacy_test.go::TestSelectUnpushedSinceExcludesCacheTables`):
500
+
501
+ 1. **Tier-1 (proxy)** — point your AI client at the local proxy
502
+ (`ANTHROPIC_BASE_URL=http://127.0.0.1:8820` for Claude Code;
503
+ `OPENAI_BASE_URL=http://127.0.0.1:8820/v1` for codex and
504
+ copilot-cli). The proxy reads the upstream usage envelope
505
+ (`cache_read_input_tokens` + `cache_creation_input_tokens`) and the
506
+ cachetrack engine attributes each turn live as it streams. Every
507
+ claude-code prompt-cache event surfaces here in real time.
508
+ 2. **Tier-2 (transcript watcher)** — the watcher tails claude-code
509
+ JSONL transcripts and feeds the same engine retroactively for
510
+ sessions that didn't route through the proxy. Run
511
+ `observer backfill --cache-rescan` to retrofit pre-cachetrack
512
+ history.
513
+
514
+ #### How to enable / disable
515
+
516
+ - **Default-on.** Per spec §11, the loader merges `[cachetrack].enabled = true`
517
+ if the section is absent. No action needed for a fresh install.
518
+ - To disable: set `[cachetrack].enabled = false` in
519
+ `~/.observer/config.toml` and restart `observer start`.
520
+ - Inspect engine health: `observer cache-health --json` — §10 grading
521
+ gate + read:write consistency + cause-concentration WARNs.
522
+ - 90-day retention is on by default
523
+ (`[cachetrack].retention_days = 90`); the maintenance tick prunes
524
+ stale `cache_*` rows alongside the existing retention pass.
525
+
526
+ Operator reference: [`docs/cache-tracking.md`](https://github.com/marmutapp/superbased-observer/blob/main/docs/cache-tracking.md).
527
+
528
+ ### Suggestions tab
529
+
530
+ The **advisor** — a prescriptive suggestions engine that turns the
531
+ window's captured activity into ranked, dollar- or minute-quantified
532
+ recommendations. **On by default** and entirely local: detectors are
533
+ pure reads over data the observer already stores, recomputed fresh on
534
+ every view — zero LLM calls, zero network.
535
+
536
+ Twenty detectors across four categories:
537
+
538
+ - **cost** — session balloons (one session's context growth dominating
539
+ spend), idle re-cache (cache rewrites after idle gaps), long-context
540
+ tier crossings, trivial sessions on expensive models, cache hit-rate,
541
+ cache-write waste, prefix thrash, read-heavy sessions on expensive
542
+ models, effort overprovisioning, cross-session stale reads,
543
+ web-search spend, spend spikes ("today is 3× your daily average" —
544
+ the while-it's-still-happening check, linking to the Live view),
545
+ routing evidence ready (the advise shadow met its promotion gate
546
+ with routing still not enforcing — linking to the Shadow card,
547
+ which owns the evidence and the consent-gated promote).
548
+ - **latency** — fast-tier premium (sessions that paid the 2× fast-tier
549
+ rate without needing it).
550
+ - **quality** — unrecovered failures, quality regressions.
551
+ - **hygiene** — MCP schema overhead vs. actual MCP calls, compression
552
+ off while proxied traffic flows, capture running without proxy
553
+ routing, guard observing idle (recurring high-severity verdicts
554
+ with zero approvals or tuning — linking to the Security page).
555
+
556
+ Each card carries the quantified claim ($ or minutes, with a
557
+ **show math** toggle exposing the arithmetic), a confidence score, a
558
+ scope chip (window / project / session — session scopes deep-link to
559
+ the session detail), and — when a dashboard control can fix the
560
+ finding — an action button that navigates to the right surface
561
+ (writes only ever happen behind that surface's own consent flow).
562
+ `snooze 7d` quiets a card temporarily; `dismiss` retires it, with a
563
+ 7-day cooldown so the same finding doesn't re-nag. The all-clear
564
+ state means exactly what it says: nothing worth nagging about in
565
+ this window.
566
+
567
+ Tune it under Settings → Advisor (`[advisor]` in config.toml):
568
+ evidence window (default 14 days), confidence floor (0.5), minimum
569
+ claimed savings ($1), and the opt-in **session digest** — when
570
+ `session_digest = true`, the Claude Code session-start hook injects a
571
+ ≤400-token digest of the top suggestions into new sessions (off by
572
+ default; the hook only reads a pre-computed snapshot, refreshed every
573
+ 30 minutes). CLI twin: `observer advise`.
574
+
438
575
  ### Discovery tab
439
576
 
440
577
  <p align="center">
441
- <img src="https://github.com/marmutapp/superbased-observer/raw/main/docs/assets/screenshots/08-discovery.png" alt="Discovery tab" width="900">
578
+ <img src="https://github.com/marmutapp/superbased-observer/raw/main/docs/assets/screenshots/09-discovery.png" alt="Discovery tab" width="900">
442
579
  </p>
443
580
 
444
581
  Wasted-effort signals:
@@ -469,21 +606,73 @@ observations push it higher) and an observation count.
469
606
  high-confidence ones into `CLAUDE.md` / `AGENTS.md` / `.cursorrules`
470
607
  so new sessions inherit your habits.
471
608
 
609
+ ### Security tab
610
+
611
+ The guard layer's operating surface — not just a verdict viewer.
612
+ Posture tiles (mode, verdict counts, audit-chain check) over a
613
+ filterable verdict timeline (rule / severity / decision / session,
614
+ with rule IDs resolved to their full definitions), then the routine
615
+ workflows end to end: a consent-gated **mode control** that shows the
616
+ simulate evidence before you flip enforce, the **enforce-readiness
617
+ card** (a replay of your real history under the enforce projection,
618
+ with per-rule blocking concentration), the **approvals register**
619
+ (scoped, expiring exceptions — DB writes, live immediately), a
620
+ **lint-gated policy editor** for your user rules (malformed files are
621
+ refused with the problems listed; `.bak` undo), **budget guardrails**
622
+ suggested from your own observed spend with a daily burn-down meter,
623
+ MCP pin approvals, and one-click **compliance evidence** downloads
624
+ (report / JSONL / CEF / audit verification).
625
+
626
+ ### Routing tab
627
+
628
+ Model routing's transparency + adoption surface. While routing is
629
+ off, a **Preview savings** card replays your last 30 days under any
630
+ policy template — read-only, nothing enabled. Once advise mode runs,
631
+ the page shows the decisions feed (every decision expandable to its
632
+ matched rule, reason codes, and cache economics), savings with CI95
633
+ error bars, the expanded policy rule table (never a black box,
634
+ demoted rules marked), the tier map with calibration overlays, the
635
+ model health board, and the **advise-shadow readiness ladder** with
636
+ its consent-gated promote control. The **Apply to tools** card turns
637
+ observed sub-agent evidence into per-tool native config — dry-run
638
+ preview, one consent per file write, backups + per-file revert, and
639
+ an append-only audit ledger of everything observer changed.
640
+
472
641
  ### Settings tab
473
642
 
474
643
  <p align="center">
475
- <img src="https://github.com/marmutapp/superbased-observer/raw/main/docs/assets/screenshots/09-settings.png" alt="Settings tab" width="900">
644
+ <img src="https://github.com/marmutapp/superbased-observer/raw/main/docs/assets/screenshots/10-settings.png" alt="Settings tab" width="900">
476
645
  </p>
477
646
 
478
- Fully editable visual editor for everything in `config.toml`.
479
- Pricing overrides hot-reload (no daemon restart `cost.Engine`
480
- swaps the pricing table atomically via `atomic.Pointer.Store`).
481
- The Backfill panel surfaces every `observer backfill` mode as
482
- click-to-run buttons that spawn the CLI as a child process and
483
- stream output back live. Watcher / Freshness / Retention / Hooks /
484
- Proxy / Compression / Intelligence sections are schema-driven forms
485
- with inline help; a "Restart daemon" banner appears whenever a
486
- section is saved that consumers bind at startup.
647
+ Fully editable visual editor for everything in `config.toml`, plus
648
+ the operational panels a config file can't express:
649
+
650
+ - **Config sections** schema-driven forms for Watcher, Freshness,
651
+ Retention, Hooks, Proxy, Compression, Intelligence, Advisor, Cache
652
+ tracking, Secrets scrubbing, MCP, Profiles, Org share, and OTel.
653
+ Each section is honest about reload semantics: pricing and profile
654
+ changes apply hot, MCP settings apply to the next AI session, and
655
+ restart-gated sections raise a persistent restart-pending banner
656
+ that names the exact command and clears itself only when the
657
+ daemon actually restarts.
658
+ - **Connected tools** — one row per detected AI client (detected /
659
+ capturing / hooks / MCP / proxied), a per-tool setup wizard that
660
+ previews every config write and applies each one only on its own
661
+ consent click, and a Launch button (with a copy-paste fallback)
662
+ for opening a routed session.
663
+ - **Health** — the full `observer doctor` check set with remediation
664
+ hints, plus recent command failures (recovered vs not).
665
+ - **Backfill** — every `observer backfill` mode as a click-to-run
666
+ job with live streamed output, plus a full-rescan card with an
667
+ adapter picker.
668
+ - **Storage** — what's actually taking the space: per-table database
669
+ size breakdown (index and FTS5 shadow-table bytes folded into the
670
+ table they serve), vacuum and online backup as click-to-run jobs,
671
+ and the documented restore procedure (restore is deliberately
672
+ manual: stop the daemon, swap the file, start). CLI twin:
673
+ `observer db stats|vacuum|backup`.
674
+ - **Config file card** — view the active file; one-click restore of
675
+ the automatic `.bak` taken before every save.
487
676
 
488
677
  ### Help drawer
489
678
 
@@ -560,7 +749,7 @@ a content-aware compressor. Six compressors:
560
749
  | Mechanism | What it does | When it fires |
561
750
  |-----------|--------------|---------------|
562
751
  | **json** | Replaces every scalar value with a type sentinel (`"<string>"`, `"<number>"`, …) preserving structure (keys, arrays, nesting). Arrays of length > 1 collapse to one element with `_len: N`. | API responses, structured logs, telemetry exports. JSON tool_results are usually the biggest savings target. |
563
- | **code** | Heuristic skeleton: keeps top-of-file imports + signature lines (function / method / class / struct / interface / type), drops bodies. | Source code files. **On by default** as of v1.7.23 for the `claude-code` recipe (V7-24 empirical winner). |
752
+ | **code** | Heuristic skeleton: keeps top-of-file imports + signature lines (function / method / class / struct / interface / type), drops bodies. | Source code files. **On by default** as of v1.7.23 in the `claude-code` profile (V7-24 empirical winner). |
564
753
  | **logs** | Two-pass: collapses adjacent identical lines to `<line> [×N]`, then head+tail-truncates to 200 lines if still long. | Log-shaped output — `go test ./...`, `npm run build`, polling/retry loops. Lossless on distinct lines; only the truncate pass is lossy. |
565
754
  | **text** | Catch-all for content not classified as code/json/logs/diff/html. Head+tail truncation: keeps 40 + 40 lines on inputs over 80 lines. | Markdown bodies, README excerpts, narrative descriptions. |
566
755
  | **diff** | Strips unified-diff context beyond ±1 line of each change. Keeps every header, every `+`/`-` line, drops the rest with elision markers. | `git diff`, patch tool outputs. Lossless on changes; lossy only on the cheap-to-rebuild context. |
@@ -606,14 +795,34 @@ tab's "Recent compression events" view surfaces these with
606
795
  mechanism, original / compressed / saved bytes, message slot, and
607
796
  importance score (for drops).
608
797
 
609
- ### Tuning
610
-
611
- `observer config` settings (in `~/.observer/config.toml`). These are the only
612
- keys the conversation compressor reads:
798
+ ### Tuning — profiles pick the parameters for you
799
+
800
+ Compression parameters are supplied by **profiles**: named parameter
801
+ sets resolved per traffic class at the proxy boundary. The moment you
802
+ enable compression, Anthropic-path traffic gets the `claude-code`
803
+ profile and OpenAI-path traffic gets `codex-safe` — each tool gets
804
+ its tuned parameters simultaneously; there is no recipe-picking step.
805
+ Inspect with `observer profile list` / `observer profile show
806
+ <name>`; reassign per class or per tool (`observer profile assign
807
+ openai codex-variant`, `observer profile assign tool:kilo-code-cli
808
+ codex-safe`); build your own (`observer profile create mine --from
809
+ claude-code`, or Settings → Profiles in the dashboard). Per-tool
810
+ assignment needs the proxy to identify the connecting tool: hook-fed
811
+ identity (Claude Code) is exact, and Codex, Kilo CLI, opencode, and
812
+ Claude Code are also recognized by their request headers; clients
813
+ that send no distinctive headers (e.g. Cline CLI) fall back to the
814
+ per-provider profile. Profile
815
+ edits apply to **new sessions without a daemon restart**. A repo can
816
+ also carry `<root>/.observer/config.toml` with `[profiles]` /
817
+ `[compression]` keys to override per project — project files can turn
818
+ compression *off* for their traffic, never on.
819
+
820
+ The master config (`~/.observer/config.toml`) keeps the one on/off
821
+ switch and the fallback parameters used by the `default` profile:
613
822
 
614
823
  ```toml
615
824
  [compression.conversation]
616
- enabled = true
825
+ enabled = true # THE switch — profiles never flip this
617
826
  mode = "cache_aware" # "token" | "cache" | "cache_aware" — default cache_aware; see matrix below
618
827
  target_ratio = 0.85
619
828
  preserve_last_n = 5 # never drop the most recent N messages
@@ -651,20 +860,22 @@ reference lives in `docs/compression-modes.md`.
651
860
 
652
861
  ### Measured savings (v1.7.23)
653
862
 
654
- We A/B every shipped recipe against an OFF baseline on a real refactor
863
+ We A/B every shipped profile against an OFF baseline on a real refactor
655
864
  workload (`lumen` TypeScript codebase, 408-line Zustand store →
656
865
  4 domain sub-stores) on the v1.7.22 binary tip. The numbers below are
657
866
  the **most recent statistically-meaningful measurements**.
658
867
 
659
- **Pick a recipe based on which model you're running:**
868
+ The first two profiles are what the per-provider auto-assignment
869
+ applies (Anthropic → `claude-code`, OpenAI → `codex-safe`);
870
+ `codex-variant` is the manual pick for `-codex`-family models:
660
871
 
661
- | Recipe | Use when your model is… | Workload | n | Δ vs OFF (mean cost) | What's compressed |
872
+ | Profile | Use when your model is… | Workload | n | Δ vs OFF (mean cost) | What's compressed |
662
873
  |---|---|---|---|---|---|
663
- | **`claude-code`** *(default)* | Any Anthropic Claude model — `claude-sonnet-4-6`, `claude-opus-4-7`, `claude-haiku-4-5`, … | Refactor, Claude Sonnet 4.6 via Claude Code 2.1.158 | n=8 B vs n=4 OFF | **−6.9%** (CV 7.6%; tighter than OFF's 7.5%) | json + logs + code bodies; cache-aware; stash disabled |
874
+ | **`claude-code`** *(default)* | Any Anthropic Claude model — `claude-sonnet-4-6`, `claude-opus-4-7`, `claude-haiku-4-5`, … | Refactor, Claude Sonnet 4.6 via Claude Code 2.1.158 | n=8 B vs n=4 OFF | **−6.9%** (CV 7.6%; tighter than OFF's 7.5%) | json + logs + code bodies + tool-defs trim (A2-adopted 2026-06-11: −12.5% vs the pre-A2 set, zero `tools_changed` cache events); cache-aware; stash disabled |
664
875
  | **`codex-variant`** | OpenAI's `-codex` reasoning fork — `gpt-5.3-codex`, `gpt-5.4-codex`, `gpt-5-codex-agent`, anything matching `*-codex*` | Refactor, gpt-5.3-codex | n=10 B vs n=10 OFF | **−10%** ($0.270 vs $0.300) | Tools-defs trim; cache-aware; no per-type compression |
665
- | **`codex-safe`** | Plain OpenAI GPT under the codex CLI — `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.5`, `gpt-4o`, any non-`-codex` | Refactor, gpt-5.4 + `apply_patch` | n=3 B vs n=4 OFF | not statistically distinguishable on this workload | logs only; cache-aware |
876
+ | **`codex-safe`** | Plain OpenAI GPT under the codex CLI — `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.5`, `gpt-4o`, any non-`-codex` | Refactor, gpt-5.4 + `apply_patch` | n=3 B vs n=4 OFF | not statistically distinguishable on this workload | logs + tools-defs trim (A6-adopted 2026-06-11: fired on 117/117 candidate requests, ~0.8KB/request of definition prose, task-completion parity + no cache degradation at n=9 vs n=7; effect scales with envelope bulk — MCP-registered installs save more); token mode |
666
877
 
667
- The word **"variant"** in `codex-variant` refers to the *model variant* (the `-codex` reasoning fork of GPT), NOT a variant of the codex CLI. Both codex recipes are for the codex CLI; they differ only in which model family they assume. `codex-safe` is so named because plain GPT models tolerate logs trimming safely — it's not "safer than codex-variant."
878
+ The word **"variant"** in `codex-variant` refers to the *model variant* (the `-codex` reasoning fork of GPT), NOT a variant of the codex CLI. Both codex profiles are for the codex CLI; they differ only in which model family they assume. `codex-safe` is so named because plain GPT models tolerate logs trimming safely — it's not "safer than codex-variant."
668
879
 
669
880
  **Honest caveats:**
670
881
 
@@ -749,6 +960,126 @@ counts one API call 2-4×. Two layers of dedup catch this:
749
960
  Migration 007 ran a one-time pass collapsing pre-fix duplicates.
750
961
 
751
962
 
963
+ ## Security & control layer (guard)
964
+
965
+ The guard evaluates every captured agent action against a
966
+ table-driven policy — built-in rules (destructive commands, project
967
+ boundaries, secrets egress, MCP pinning, taint/dataflow, budgets)
968
+ plus your own TOML rules — records each verdict in a hash-chained,
969
+ tamper-evident audit table, and on the channels that support it can
970
+ block before execution. It ships **observe-only by default**: a fresh
971
+ install records and alerts, and never blocks anything until you
972
+ explicitly flip enforce.
973
+
974
+ What makes it different from standalone command-guards:
975
+
976
+ - **One policy, every tool.** The same rules evaluate Claude Code,
977
+ Codex, Cursor, Cline and the rest through one engine — and
978
+ `observer init --guard` additionally *compiles* the policy into
979
+ each client's own native permission dialect, so the client itself
980
+ enforces even when the observer daemon is down.
981
+ - **Replay before you enforce.** `observer guard simulate --since 168h`
982
+ replays your real captured history against current policy and shows
983
+ what last week WOULD have flagged or blocked — a confidence check
984
+ only a tool that already has your history can run.
985
+ - **Proxy-level secrets egress scanning.** Clients routed through the
986
+ observer proxy get outbound API bodies scanned for credentials,
987
+ with mask/deny enforcement — an egress point no other local-first
988
+ tool sits on.
989
+ - **Auditable, honestly.** Verdicts are hash-chained
990
+ (tamper-EVIDENT, not tamper-proof — the docs say exactly what that
991
+ means), exportable as JSONL/CEF for your SIEM, and summarized by
992
+ `observer guard report` into a compliance evidence pack mapped to
993
+ SOC 2 / NIST 800-53.
994
+ - **Operable from the dashboard, not just the CLI.** The Security
995
+ page runs the routine workflows end to end: a consent-gated mode
996
+ control that shows the simulate evidence before you flip enforce,
997
+ an Enforce-readiness replay over your real history, the approvals
998
+ register (scoped, expiring exceptions — live immediately), a
999
+ lint-gated policy editor for your user rules (malformed files are
1000
+ refused before they touch disk, with `.bak` undo), budget
1001
+ guardrails suggested from your own observed spend (p95 + headroom)
1002
+ with a daily burn-down meter, MCP pin approvals, and one-click
1003
+ compliance evidence downloads.
1004
+
1005
+ Honest scope: hooks see declared commands, not side effects (use OS
1006
+ sandboxes for containment — the guard detects and recommends them);
1007
+ most adapters are watcher-channel and can only flag post-hoc; the
1008
+ proxy scan only covers proxy-routed clients. The full capability
1009
+ matrix is on the dashboard's Security page, and the no-network
1010
+ invariant holds — nothing leaves your machine unless you opt into
1011
+ Teams push, OTel export, or the cloud alerting tier, each
1012
+ individually.
1013
+
1014
+ Quick start:
1015
+
1016
+ ```
1017
+ observer guard status # mode, rules, recent verdicts, chain check
1018
+ observer guard test "rm -rf ./build" # would this block?
1019
+ observer guard simulate --since 168h # what would last week have flagged?
1020
+ observer guard enable --enforce # flip to enforce when ready
1021
+ ```
1022
+
1023
+ Full references:
1024
+ [guard.md](https://github.com/marmutapp/superbased-observer/blob/main/docs/guard.md)
1025
+ (operator guide + getting started),
1026
+ [guard-rules.md](https://github.com/marmutapp/superbased-observer/blob/main/docs/guard-rules.md)
1027
+ (rule catalog),
1028
+ [guard-policy-authoring.md](https://github.com/marmutapp/superbased-observer/blob/main/docs/guard-policy-authoring.md)
1029
+ (writing rules + cookbook),
1030
+ [guard-enforce-runbook.md](https://github.com/marmutapp/superbased-observer/blob/main/docs/guard-enforce-runbook.md)
1031
+ (the observe→enforce migration),
1032
+ [guard-compliance.md](https://github.com/marmutapp/superbased-observer/blob/main/docs/guard-compliance.md)
1033
+ (SOC 2 / NIST mapping).
1034
+
1035
+ ## Model routing
1036
+
1037
+ The routing layer selects the right model for each KIND of work —
1038
+ from your own observed evidence, never a brochure — through two
1039
+ channels: **A** writes each AI tool's own config (claude-code
1040
+ per-subagent `model:` frontmatter — dry-run first, backed up,
1041
+ revertable, audit-ledgered; other tools get exact paste-able
1042
+ snippets), and **B** rewrites the `model` field on requests already
1043
+ transiting the proxy. It is **opt-in, advise-first, and fail-open
1044
+ everywhere**: a routing failure can never break a turn that would
1045
+ have succeeded.
1046
+
1047
+ The adoption ladder is deliberate, and every step has a dashboard
1048
+ surface: **preview** what a policy would have saved over your last 30
1049
+ days while routing is still off (a read-only replay of recorded
1050
+ turns) → enable **advise** (decision rows accrue; requests untouched)
1051
+ → read the **shadow** (the §R22 readiness ladder: enough decisions,
1052
+ real net savings after cache forfeits, zero quality flags) →
1053
+ **promote** to enforce through a consent dialog that restates the
1054
+ evidence — and warns loudly when the gate is not met. Enforce
1055
+ rewrites within the same provider shape only, holds switches that
1056
+ would forfeit a warm prompt cache worth more than they save, and
1057
+ auto-demotes any rule whose downshifts grade as regressing
1058
+ (loudly, never silently). Custom `[[routing.rules]]` are editable in
1059
+ a lint-gated dashboard editor with a worked recipe gallery in the
1060
+ docs.
1061
+
1062
+ Privacy posture: decision rows and calibration are **node-local**
1063
+ (never pushed to an org server); org-distributed policy fragments are
1064
+ signed and structurally cannot flip your enforce switch — **no remote
1065
+ enforce toggle exists, by design**.
1066
+
1067
+ Quick start:
1068
+
1069
+ ```
1070
+ # Preview before enabling anything (CLI twin of the dashboard card):
1071
+ observer routing simulate --policy value --days 30
1072
+ # Then: dashboard → Routing → Preview savings → Enable advise mode.
1073
+ observer routing shadow # the promotion evidence, any time
1074
+ observer routing apply --tool claude-code # Channel A, dry-run default
1075
+ ```
1076
+
1077
+ Full reference:
1078
+ [model-routing.md](https://github.com/marmutapp/superbased-observer/blob/main/docs/model-routing.md)
1079
+ (operator guide + the first-30-days runbook + the rules recipe
1080
+ gallery).
1081
+
1082
+
752
1083
  ## Terminology and glossary
753
1084
 
754
1085
  Quick reference; the in-platform help drawer (press `?` on the
@@ -836,9 +1167,11 @@ Every command supports `--help` for the full surface.
836
1167
 
837
1168
  | Subcommand | Purpose |
838
1169
  |------------------------------------|---------|
839
- | `observer init` | Register hooks + MCP server with installed AI clients |
1170
+ | `observer init` | Register hooks + MCP server + durable proxy routes with installed AI clients (each side defaults on; opt out with `--skip-hooks` / `--skip-mcp` / `--skip-proxy-route`). With **zero flags on a terminal** it runs an interactive checklist instead: detects your tools, previews each pending config write, and asks consent one write at a time (MCP never pre-selected). Any flag or redirected stdio keeps the batch behaviour. |
840
1171
  | `observer uninstall` | Reverse `observer init` |
841
- | `observer start` | Run watcher + dashboard + proxy in one process (recommended). Flags: `--dashboard-addr ADDR` (default `127.0.0.1:8081`), `--no-dashboard` to skip the HTTP UI. |
1172
+ | `observer start` | Run watcher + dashboard + proxy in one process (recommended). Auto-opens the dashboard on interactive launches. Flags: `--dashboard-addr ADDR` (default `127.0.0.1:8081`), `--no-dashboard` to skip the HTTP UI, `--no-open` to skip the browser. |
1173
+ | `observer claude [-- args…]` | Launch Claude Code routed through the proxy for that session only (no config writes; fresh Pro/Max OAuth token re-exported so the SDK can't bypass). `--verify` runs pre-flight checks without launching. |
1174
+ | `observer codex [-- args…]` | Launch Codex routed through the proxy for that session only (injects `openai_base_url` via argv; no config writes). |
842
1175
  | `observer watch` | Long-running JSONL watcher only |
843
1176
  | `observer dashboard --addr ADDR` | HTTP dashboard only |
844
1177
  | `observer proxy start` | Reverse proxy only |
@@ -855,13 +1188,19 @@ Every command supports `--help` for the full surface.
855
1188
  | `observer summarize` | Roll-up summary across sessions |
856
1189
  | `observer export` | Export DB to xlsx / json |
857
1190
  | `observer prune` | Manual retention pass (delete old data) |
1191
+ | `observer db stats\|vacuum\|backup` | Storage manager. `stats`: per-table size breakdown (index + FTS5 shadow bytes folded into the table they serve). `vacuum`: reclaim free pages, reports bytes freed (can report SQLITE_BUSY against a busy daemon — re-run in a quiet moment). `backup <path>`: online snapshot via SQLite `VACUUM INTO` — safe while the daemon runs, refuses to overwrite. Restore is manual by design: stop daemon, swap file, start. |
1192
+ | `observer db import <path>` | Merge another `observer.db` into this node's (a stranded install from another OS or home directory). Set-based, single-transaction, idempotent — re-running adds nothing twice; project identities remap by root path. `--dry-run` executes the same transaction and rolls it back, so its per-table counts are exact. Migrates the source schema first — point it at a copy. Node-local state (cache tracking, watcher cursors, advisor state) doesn't transfer. |
858
1193
  | `observer backfill --is-sidechain` | Re-walk JSONL to populate `actions.is_sidechain` (added by migration 010) on pre-migration rows. |
859
1194
  | `observer backfill --cache-tier` | Re-walk JSONL to populate `cache_creation_1h_tokens` (added by migration 008) on pre-migration rows. Run once after upgrading to v1.4.16+ to correct historical 1h-tier cache writes that were silently billed at the cheaper 5m rate. |
860
1195
  | `observer backfill --message-id` | Re-walk JSONL to populate `message_id` on `actions` and `token_usage` (added by migration 012). Required by the per-message timeline view in the Sessions modal. |
861
1196
  | `observer backfill --all` | Run every supported backfill in one invocation. Idempotent — safe to re-run. |
1197
+ | `observer profile list\|show\|assign\|create\|delete\|set` | Compression profiles: inspect built-ins, reassign per traffic class or per tool (`assign tool:cline codex-safe`), create/edit custom profiles. Edits apply to new sessions hot. |
1198
+ | `observer config set <key> <value>` | Dotted-key config setter (`compression.conversation.enabled true`). `--project <root>` writes a per-repo override file instead. |
1199
+ | `observer advise` | Prescriptive cost/quality suggestions from captured activity (the Suggestions tab, in the CLI) |
1200
+ | `observer cache-health` | Prompt-cache engine health: grading gate, read:write consistency, cause concentration |
862
1201
  | `observer metrics` | Prometheus-format metrics endpoint |
863
1202
  | `observer serve` | MCP server (stdio JSON-RPC) — usually invoked by `observer init` registration |
864
- | `observer tail` | Live event stream |
1203
+ | `observer completion <shell>` | Generate shell completions (bash / zsh / fish / powershell) — e.g. `observer completion zsh > "${fpath[1]}/_observer"` |
865
1204
 
866
1205
 
867
1206
  ## Configuration
@@ -965,11 +1304,13 @@ its dirs get created and the watcher picks them up.
965
1304
 
966
1305
  ### Empty dashboard / "No proxy traffic"
967
1306
 
968
- The JSONL adapter populates passively after `observer init`, but
969
- ground-truth cost / compression numbers require the proxy. Set
970
- `ANTHROPIC_BASE_URL=http://127.0.0.1:8820` (Claude Code) or
971
- `OPENAI_BASE_URL=http://127.0.0.1:8820/v1` (Codex) in the shell
972
- that launches your AI client.
1307
+ Session/action data populates passively whenever `observer start` is
1308
+ running, but ground-truth cost / compression numbers require the
1309
+ proxy. Route your tool through it — the quickest way is the
1310
+ dashboard's Compression tab → **Proxy** banner → **Route through the
1311
+ observer proxy…** button; every other mechanism (wrappers, env vars,
1312
+ `observer init`) is listed under
1313
+ [Per-AI-client setup](#per-ai-client-setup).
973
1314
 
974
1315
  Verify with `observer status | grep api_turns` — count should
975
1316
  climb during AI-client activity.
@@ -82,30 +82,33 @@ the [main repo](https://github.com/marmutapp/superbased-observer).
82
82
  ## Five-minute quickstart
83
83
 
84
84
  ```bash
85
- # 1) Install. `observer init` is OPTIONAL — only run it if you want
86
- # the MCP server registered with your AI clients (gives them
87
- # on-demand tools like check_file_freshness / get_cost_summary
88
- # at the cost of ~1,800 tokens of schema per turn).
85
+ # 1) Install.
89
86
  npm install -g @superbased/observer
90
- observer init # OPTIONAL — interactive: pick clients;
91
- # writes MCP + codex proxy-route into AI client configs.
92
- # Skip this step for an MCP-free install.
93
-
94
- # 2) Start the long-running services (proxy + watcher + dashboard).
95
- # Auto-registers HOOKS for every detected AI tool on first launch.
96
- observer start &
97
-
98
- # 3) Engage the proxy by pointing your AI client at the local URL.
99
- # See "Per-AI-client setup" for the matching env var.
100
- export ANTHROPIC_BASE_URL=http://127.0.0.1:8820 # Claude Code
101
- export OPENAI_BASE_URL=http://127.0.0.1:8820/v1 # Codex / OpenAI
102
-
103
- # 4) Open the dashboard.
104
- open http://127.0.0.1:8081/ # macOS
105
- xdg-open http://127.0.0.1:8081/ # Linux
106
- start http://127.0.0.1:8081/ # Windows
87
+
88
+ # 2) Start everything: proxy + watcher + dashboard in one process.
89
+ # Hooks auto-register for every detected AI tool, and the
90
+ # dashboard opens in your browser (suppress with --no-open).
91
+ observer start
107
92
  ```
108
93
 
94
+ From here the dashboard drives:
95
+
96
+ 3. **Route your AI client through the proxy** — on the Compression
97
+ tab's **Proxy** banner, click your tool's status pill, then
98
+ **Route through the observer proxy…**. The button previews the
99
+ exact file change and writes only on confirm. (Every other
100
+ routing mechanism — `observer init`, the `observer claude` /
101
+ `observer codex` wrappers, plain env vars — is listed in
102
+ [Per-AI-client setup](#per-ai-client-setup).)
103
+ 4. **Use your AI tool as normal.** The Overview tab's onboarding
104
+ checklist tracks your first captured session; cost, compression,
105
+ and cache numbers populate within minutes of real activity.
106
+
107
+ `observer init` is OPTIONAL — run it only if you want the MCP server
108
+ registered with your AI clients (gives them on-demand tools like
109
+ `check_file_freshness` / `get_cost_summary`, at the cost of ~1,800
110
+ tokens of schema per turn). Skip it for an MCP-free install.
111
+
109
112
  **What `start` does vs what `init` adds:**
110
113
 
111
114
  | Step | Hooks | Proxy listening | Watcher | Dashboard | MCP in AI clients | Codex proxy route |
@@ -117,10 +120,6 @@ start http://127.0.0.1:8081/ # Windows
117
120
  MCP and codex routing are explicit-only because both write per-client
118
121
  config files. Hooks self-heal on every `start`.
119
122
 
120
- After ten minutes of normal AI-coding usage, the dashboard will be
121
- populated with cost over time, per-tool activity, compression
122
- savings, and stale-reread waste signals.
123
-
124
123
 
125
124
  <!-- @@INCLUDE:docs/distribution/README-body.md@@ -->
126
125
 
@@ -181,11 +180,13 @@ its dirs get created and the watcher picks them up.
181
180
 
182
181
  ### Empty dashboard / "No proxy traffic"
183
182
 
184
- The JSONL adapter populates passively after `observer init`, but
185
- ground-truth cost / compression numbers require the proxy. Set
186
- `ANTHROPIC_BASE_URL=http://127.0.0.1:8820` (Claude Code) or
187
- `OPENAI_BASE_URL=http://127.0.0.1:8820/v1` (Codex) in the shell
188
- that launches your AI client.
183
+ Session/action data populates passively whenever `observer start` is
184
+ running, but ground-truth cost / compression numbers require the
185
+ proxy. Route your tool through it — the quickest way is the
186
+ dashboard's Compression tab → **Proxy** banner → **Route through the
187
+ observer proxy…** button; every other mechanism (wrappers, env vars,
188
+ `observer init`) is listed under
189
+ [Per-AI-client setup](#per-ai-client-setup).
189
190
 
190
191
  Verify with `observer status | grep api_turns` — count should
191
192
  climb during AI-client activity.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@superbased/observer",
3
- "version": "1.8.2",
3
+ "version": "1.8.3",
4
4
  "description": "SuperBased Observer — capture, normalize, compress, and analyze AI coding tool activity across Claude Code, Codex, Cursor, Cline/Roo, and Copilot.",
5
5
  "keywords": [
6
6
  "ai",
@@ -36,11 +36,11 @@
36
36
  "LICENSE"
37
37
  ],
38
38
  "optionalDependencies": {
39
- "@superbased/observer-linux-x64": "1.8.2",
40
- "@superbased/observer-linux-arm64": "1.8.2",
41
- "@superbased/observer-darwin-x64": "1.8.2",
42
- "@superbased/observer-darwin-arm64": "1.8.2",
43
- "@superbased/observer-win32-x64": "1.8.2"
39
+ "@superbased/observer-linux-x64": "1.8.3",
40
+ "@superbased/observer-linux-arm64": "1.8.3",
41
+ "@superbased/observer-darwin-x64": "1.8.3",
42
+ "@superbased/observer-darwin-arm64": "1.8.3",
43
+ "@superbased/observer-win32-x64": "1.8.3"
44
44
  },
45
45
  "scripts": {
46
46
  "test": "node bin/observer.js --version"