@gi4nks/ph 1.4.0 → 1.6.0

package/docs/specs/SPEC-001-architecture-cli.md

@@ -0,0 +1,82 @@
+# SPEC-001: Architecture & CLI Dispatch
+
+- **ID**: SPEC-001
+- **Cluster**: Core
+- **Status**: Implemented (baseline)
+- **Date**: 2026-08-08
+- **Source**: reverse-engineered from code (files listed in §8)
+
+## 1. Purpose
+`ph` is a transparent observability layer for AI CLI tools: every prompt+response
+is captured to a local SQLite DB (SPEC-002), analyzed (SPEC-006), and made
+browsable via TUI (SPEC-012), CLI commands, MCP (SPEC-011) and HTTP (SPEC-010).
+This spec covers the entry point and the command dispatch architecture.
+
+## 2. Scope
+- **In scope**: cli.ts main(), command resolution, wrapper-mode fallback, config
+  loading order, TUI bootstrap.
+- **Out of scope**: individual commands (their own specs), the TUI internals.
+- **Entry points**: `ph` (no args → TUI), `ph <command>`, `ph <tool> "prompt"` (wrapper).
+
+## 3. Data Model
+- `argv` split into `command` (first non-flag arg) + `cmdArgs` + `preArgs`
+  (ph-specific flags before the command, e.g. `--ph-tag`).
+- `PhConfig` (src/config/index.ts:5) loaded before dispatch; `geminiApiKey`
+  is copied to `process.env.GEMINI_API_KEY` (cli.ts:153-155).
+- DB path resolution: `PH_DB` env > `cfg.dbPath` > `defaultPath()`
+  (cli.ts:157).
+
+## 4. Flows
+
+### 4.1 Dispatch (src/cli.ts:148-441)
+1. Load config; set env; resolve dbPath.
+2. No args + TTY → render BrowseApp in alt-screen, wait for exit, optional
+   rerun (cli.ts:159-186).
+3. Find first non-flag arg; flags `--ph-tag`/`--ph-role` consume their value
+   (cli.ts:190-203).
+4. Switch on the command name: ~25 cases (search, context, last, sessions,
+   stats, cluster, analyze*, cleanup*, star, export, import, analyze, mcp,
+   server, remote, log, embed-all, vacuum, config, _bg-analyze, browse, chat,
+   timeline, ollama-models, help).
+5. Unknown command → wrapper mode: `cmdWrap(dbPath, command, [...preArgs,
+   ...cmdArgs], cfg)` (cli.ts:436-439).
+
+### 4.2 Wrapper mode (SPEC-003)
+Unknown first arg = a real binary (`claude`, `gemini`, …): run it and capture.
+
+## 5. Invariants & Business rules
+- ESM only, `.js` import extensions (package.json `"type": "module"`).
+- Manual arg parsing (`parseFlags`, src/commands/_utils.ts:1) — no commander.
+- Every command handler owns its `PhDB` instance and closes it
+  (`new PhDB(...)` … `db.close()` per case).
+- `main()` catches everything: `ph: <message>` on stderr, exit 1 (cli.ts:443-446).
+
+## 6. UI / UX surface
+USAGE text is a single ~200-line template literal (cli.ts:38-140); `ph --help`,
+`-h`, `help` and non-TTY no-arg all print it.
+
+## 7. Acceptance criteria (Given/When/Then)
+- **G1**: Given `ph search foo`, When main() runs, Then the `search` case opens a
+  PhDB, calls `cmdSearch` and closes the DB (cli.ts:250-253).
+- **G2**: Given `ph claude "hello"` (unknown command), When main() runs, Then
+  `cmdWrap` receives tool=`claude` and args `["hello"]` (cli.ts:436-439).
+- **G3**: Given `ph` with no args on a non-TTY stdin, When main() runs, Then the
+  USAGE text is printed and the process exits 0 (cli.ts:183-186).
+- **G4**: Given `--ph-tag auth claude "x"`, When parsing, Then `auth` is consumed
+  as the tag value and `claude` is the command (cli.ts:192-194).
+
+## 8. Key implementation map
+| Concern | File(s) |
+|---|---|
+| Entry + dispatch + USAGE | src/cli.ts:38-441 |
+| Config load/save | src/config/index.ts:26-40 |
+| Flag parsing | src/commands/_utils.ts:1-30 |
+| Shared TUI bootstrap | src/cli.ts (`openBrowser`, `runRerun`) |
+
+## 9. Open questions / discrepancies
+- `--ph-debug` is listed in USAGE but only consumed by wrapper paths (inline/pty
+  debug logs); no validation if used with non-wrapper commands.
+
+## 10. Related
+- All other specs (commands dispatch into each), SPEC-012 (BrowseApp),
+  SPEC-003 (wrapper mode). CLI dispatch tests remain part of SPEC-ISSUES-007.

package/docs/specs/SPEC-002-database-layer.md

@@ -0,0 +1,105 @@
+# SPEC-002: Database Layer (PhDB)
+
+- **ID**: SPEC-002
+- **Cluster**: DB
+- **Status**: Implemented (baseline)
+- **Date**: 2026-08-08
+- **Source**: reverse-engineered from code (files listed in §8)
+
+## 1. Purpose
+All persistence for ph: prompts (raw history), FTS5 full-text index, sqlite-vec
+embeddings, memories, project summaries and the archive — behind a single
+`PhDB` class (src/db/index.ts, ~800 lines, ~50 public methods). WAL mode, best
+effort migrations, zero ORM.
+
+## 2. Scope
+- **In scope**: schema, migrations, insert/update/delete/search primitives,
+  memories + summaries + archive APIs, stats.
+- **Out of scope**: FTS query semantics (SPEC-005), semantic search (SPEC-008),
+  retention rules (SPEC-009), sync dedup (SPEC-010).
+- **Entry points**: `new PhDB(dbPath)`, `defaultPath()` (src/db/index.ts).
+
+## 3. Data Model
+- `prompts` (src/db/index.ts:56-66): id, timestamp, tool, prompt, args, workdir,
+  hostname, exit_code, metadata (JSON). NOTE: `response` column is added by ALTER
+  (SPEC-ISSUES-001).
+- `prompts_fts` FTS5 external-content (src/db/index.ts:68-73) with AFTER INSERT /
+  AFTER DELETE triggers (75-82).
+- `embeddings` (legacy BLOB, 87-91) + `vec_embeddings` vec0 float[768] (100-104).
+- `memories` (106-121) append-only per project; `project_summaries` (PhDB.PROJECT_SUMMARIES_SCHEMA, :123);
+  `prompts_archive` (PhDB.ARCHIVE_SCHEMA, :124).
+- `PromptEntry` / `PromptMetadata` / `MemoryEntry` / `ProjectSummary` types
+  (src/types.ts:1-52).
+
+## 4. Flows
+
+### 4.1 Open + migrate (src/db/index.ts:47-196)
+1. `new Database(dbPath)`; `PRAGMA journal_mode = WAL`; `sqliteVec.load(db)`.
+2. CREATE TABLE IF NOT EXISTS for all tables + triggers + indexes.
+3. Migrations: `$schema_version` backfill (127-136), legacy metadata column
+   (139-141), `response` column (143-146), BLOB→vec0 embedding migration
+   (148-180).
+
+### 4.2 Insert (src/db/index.ts:198-218)
+`insert(entry)` defaults `response ?? ''` and `metadata || '{}'`, INSERTs, FTS
+trigger keeps the index in sync. Returns lastInsertRowid.
+
+### 4.3 Read paths
+- `search(opts)` (238) — see SPEC-005.
+- `getById` (232), `getPromptsSince` (393), `getAllPrompts` (387),
+  `getPromptCount`/`getPromptCountSince` (399-410), `getStats` (410+).
+- `getPromptBySyncHash` (403-407) — used by sync dedup (SPEC-010).
+- Memories: `insertMemory` (535), `upsertProjectMemory` (612),
+  `upsertProjectSummary` (643), `getProjectSummary` (690) — SPEC-007.
+- Archive: `archivePrompts` (464), `searchArchive` (496), `purgeArchive` (506),
+  `getArchiveStats` (511) — SPEC-009.
+
+## 5. Invariants & Business rules
+- WAL mode always on; sqlite-vec loaded on every open.
+- FTS index maintained by triggers (never manually rebuilt except migration).
+- `metadata` is always valid JSON (`'{}'` fallback).
+- All counts/read queries are typed casts on prepared statements — no ORM.
+
+## 6. UI / UX surface
+None — library layer consumed by commands, TUI, MCP and HTTP server.
+
+## 7. Acceptance criteria (Given/When/Then)
+- **G1**: Given a fresh temp DB, When `insert()` runs with a prompt, Then the
+  row is retrievable via `getById` and count is 1 (src/db/index.ts:198-232).
+- **G2**: Given an insert without `response`, When read back, Then `response`
+  defaults to `''` (src/db/index.ts:199-201).
+- **G3**: Given two prompts with the same `tool|prompt|response`, When
+  `getPromptBySyncHash` runs, Then only the first match is returned
+  (src/db/index.ts:403-407).
+- **G4**: Given a delete, When `getById` runs, Then the entry is gone and the
+  FTS trigger removed it (src/db/index.ts:79-82, 427).
+- **G5**: Given `getStats()`, When run, Then total/totalMemories/byTool are
+  consistent with the tables (src/db/index.ts:410+).
+- **G6**: Given a fresh DB, When opened, Then `vec_embeddings` exists as a vec0
+  table with float[768] (src/db/index.ts:95-104).
+
+## 8. Key implementation map
+| Concern | File(s) |
+|---|---|
+| Schema + migrations | src/db/index.ts:47-196 |
+| CRUD + search + sync primitives | src/db/index.ts:198-531 |
+| Memories + summaries | src/db/index.ts:535-720 |
+| Archive | src/db/index.ts:464-518 |
+| Semantic vector index | src/db/semantic-index.ts |
+| Types | src/types.ts:1-52 |
+
+## 9. Open questions / discrepancies
+- **Architecture decision (2026-10-09)**: keep `PhDB` as the single SQLite adapter
+  for now. Context consumers depend on the focused `ProjectContextStore` interface
+  (`src/context/index.ts`). Vector encoding, compatibility storage, and KNN
+  ranking now live in `src/db/semantic-index.ts`, with `PhDB` retaining its public
+  methods as the adapter. Scope behavior has an integration test. Keep unrelated
+  CRUD/memory/archive methods together until a second adapter or concrete
+  independent migration/testing seam justifies further division.
+- `response` column via ALTER (SPEC-ISSUES-001).
+- `memories.prompt_ids` possibly unused after summaries (SPEC-ISSUES-014).
+- `delete` (427) and `deleteById` (435) coexist — duplicate deletion paths.
+
+## 10. Related
+- SPEC-005 (search), SPEC-007 (memory), SPEC-009 (retention), SPEC-010 (sync).
+  Public behavior is covered by Vitest integration tests (SPEC-ISSUES-007).

package/docs/specs/SPEC-003-capture-modes.md

@@ -0,0 +1,110 @@
+# SPEC-003: Capture Modes (PTY / Inline / Hooks / Log)
+
+- **ID**: SPEC-003
+- **Cluster**: Capture
+- **Status**: Implemented (baseline)
+- **Date**: 2026-08-08
+- **Source**: reverse-engineered from code (files listed in §8)
+
+## 1. Purpose
+The three ways a prompt+response reaches the DB: transparent PTY wrapping of an
+interactive tool, inline (non-interactive) capture, and direct logging via hooks
+or `ph log`. Plus the OpenCode plugin for real-time capture.
+
+## 2. Scope
+- **In scope**: runPTY (ESC/OSC state machine), runInline, resolveRealBinary,
+  ph log command, hooks, opencode plugin.
+- **Out of scope**: what happens after capture (analysis SPEC-006, background
+  push SPEC-010).
+- **Entry points**: `ph <tool> …` (wrapper), `ph log`, hook scripts, opencode plugin.
+
+## 3. Data Model
+- `PromptCallback(prompt, timestamp) => id` and `ResponseCallback(id, response)`
+  (src/pty/wrapper.ts:4-5).
+- Entry shape: `Omit<PromptEntry,'id'|'response'> & {response?: string}`
+  (PhDB.insert, SPEC-002).
+- Live captures and imports use `createCaptureRecord` (src/capture/index.ts)
+  to normalize timestamps, host, project/language/title metadata, and source
+  fields before persistence.
+
+## 4. Flows
+
+### 4.1 PTY wrapper (src/pty/wrapper.ts:23-169)
+1. `pty.spawn(binary, args, {name:'xterm-256color', cols, rows, cwd, env})`.
+2. Forward PTY output to stdout; buffer it as the "response".
+3. stdin in raw mode; a byte-level ESC/CSI/OSC state machine (wrapper.ts:74-145)
+   detects Enter (prompt capture), backspace, Ctrl-C/D, and ignores escape
+   sequences. On Enter: finalize previous response, call `onPrompt`.
+4. On PTY exit: finalize last response, restore stdin, resolve with exitCode.
+
+### 4.2 Inline (src/runner/inline.ts:37)
+`runInline` captures args → `extractPrompt(args)` (:8), runs the binary
+non-interactively (spawnSync), records exit code, then `db.insert`.
+
+### 4.3 Direct log (src/commands/log.ts:9-70)
+`ph log --tool <t> --prompt <p> [--response <r>]` builds the entry (project
+detection, language detection, topic extraction), inserts, then optionally
+spawns background analysis (SPEC-006) and a fire-and-forget remote push
+(SPEC-010).
+
+### 4.4 Hooks + plugin
+- `hooks/claude/ph-hook.sh`, `hooks/gemini/ph-hook.sh` — shell scripts invoked
+  post-session; they pipe JSON to `ph log`.
+- `hooks/codex/ph-hook.sh` — Codex `Stop` hook reads `transcript_path` from
+  stdin and imports the active rollout with `ph import codex --file`.
+- `hooks/opencode/ph-plugin.ts` — real-time capture via OpenCode hooks
+  (`chat.message` + `experimental.text.complete`), pairs prompts with streamed
+  responses, calls `ph log` in background; `hooks/opencode/install.sh`.
+
+## 5. Invariants & Business rules
+- PTY capture is byte-exact for prompts: ANSI sequences never leak into the
+  prompt text (ESC state machine).
+- `--ph-debug` writes a session log to `~/.ph_debug.log` via `debugLog`.
+- Capture never blocks the wrapped tool (background processes for analysis/push).
+- Binary resolution: `resolveRealBinary(tool)` maps known tool names
+  (src/runner/inline.ts:15).
+- All supported capture adapters produce the same normalized PromptEntry shape;
+  source-specific metadata remains attached to that record.
+
+## 6. UI / UX surface
+Transparent: the wrapped tool's own output is forwarded verbatim; ph itself
+prints only capture feedback.
+
+## 7. Acceptance criteria (Given/When/Then)
+- **G1**: Given a PTY session where the user types `hello` and presses Enter,
+  When the ESC state machine processes the bytes, Then `onPrompt` fires with
+  `"hello"` (src/pty/wrapper.ts:87-98).
+- **G2**: Given ANSI/OSC sequences (e.g. `\x1b]0;title\x07`), When the state
+  machine runs, Then the sequences are consumed and never appended to
+  `lineBuffer` (src/pty/wrapper.ts:114-143).
+- **G3**: Given `ph log --tool claude --prompt "x"` without `--response`, When
+  cmdLog runs, Then the entry is inserted with `response: ''` and project
+  metadata (src/commands/log.ts, SPEC-002 G2).
+- **G4**: Given `extractPrompt(["explain","goroutines"])`, When run, Then the
+  joined prompt text is returned (src/runner/inline.ts:8-14).
+- **G5**: Given a capture with a project marker and adapter metadata, When
+  `createCaptureRecord` builds it, Then it includes detected project/language,
+  source metadata, and common fields (src/capture/index.ts).
+
+## 8. Key implementation map
+| Concern | File(s) |
+|---|---|
+| PTY + ESC state machine | src/pty/wrapper.ts:23-169 |
+| Inline runner + binary resolution | src/runner/inline.ts:8-60 |
+| Direct log | src/commands/log.ts:9-85 |
+| Normalized capture record | src/capture/index.ts |
+| Hooks | hooks/claude/ph-hook.sh, hooks/gemini/ph-hook.sh, hooks/codex/ph-hook.sh |
+| OpenCode plugin | hooks/opencode/ph-plugin.ts, install.sh |
+
+## 9. Open questions / discrepancies
+- `cleanLine` (wrapper.ts:7-10) trims whitespace — prompts containing leading/
+  trailing spaces or multi-line pastes are collapsed; paste detection (bracketed
+  paste) is not implemented.
+- The PTY path is not exercised by any test (requires a PTY); the state machine
+  logic is pure enough to unit-test with byte arrays.
+
+## 10. Related
+- SPEC-001 (wrapper-mode dispatch), SPEC-002 (insert), SPEC-006 (background
+  analysis), SPEC-010 (background push). The normalized capture record is
+  covered by `src/capture/__tests__/record.test.ts`; PTY/integration coverage
+  remains open (SPEC-ISSUES-007).

package/docs/specs/SPEC-004-filter-metadata.md

@@ -0,0 +1,82 @@
+# SPEC-004: Filter Pipeline & Metadata
+
+- **ID**: SPEC-004
+- **Cluster**: Core
+- **Status**: Implemented (baseline)
+- **Date**: 2026-08-08
+- **Source**: reverse-engineered from code (files listed in §8)
+
+## 1. Purpose
+Keeps noise out of the history: `FilterPipeline` rejects too-short, binary,
+trivial and duplicate prompts before they reach the DB, and enriches entries
+with project/language/topic metadata at capture time.
+
+## 2. Scope
+- **In scope**: FilterPipeline checks, FilterResult, metadata enrichment
+  (detectProject, detectLanguage, extractTopic), filter config knobs.
+- **Out of scope**: LLM analysis (SPEC-006), retention (SPEC-009).
+- **Entry points**: FilterPipeline used by `ph log` / importers; metadata
+  helpers used across commands.
+
+## 3. Data Model
+- `FilterReason` union + `FilterResult` (src/filter/index.ts:4-11): `{ pass,
+  reason? }`.
+- `FilterOptions` (src/filter/index.ts:17): minLength, minRelevance.
+- Metadata JSON shape `PromptMetadata` (src/types.ts:14-27).
+
+## 4. Flows
+
+### 4.1 Filtering (src/filter/index.ts:34)
+`FilterPipeline.check(text)` runs 4 checks (AGENTS.md + filter/index.ts):
+1. too short (< `filterMinLength`, default 15)
+2. non-printable characters
+3. trivial pattern (e.g. single words, "ok", "yes")
+4. exact duplicate (against recent history)
+Optional relevance check when `minRelevance > 0`.
+
+### 4.2 Enrichment
+- `detectProject(startDir)` walks up looking for `.git`/`go.mod`/`package.json`/
+  `Cargo.toml` (src/runner/project.ts:25).
+- `detectLanguage(rootDir)` (src/runner/project.ts:40).
+- `extractTopic(prompt)` (src/utils/extractTopic.ts:5) — title-like extraction.
+- Assembled into `metadata` JSON before insert (e.g. src/commands/log.ts:50-58).
+
+## 5. Invariants & Business rules
+- Filtered prompts never reach the DB (importers and log both call the pipeline).
+- Metadata merge later (SPEC-006) preserves manual fields unless `force`.
+- `filterMinRelevance` default 3 — but relevance only exists after analysis, so
+  the check only bites on re-analysis paths.
+
+## 6. UI / UX surface
+Filter counts/reasons are logged at capture time; configurable via
+`ph config set filter-min-length <n>` etc.
+
+## 7. Acceptance criteria (Given/When/Then)
+- **G1**: Given a 5-char prompt with `minLength: 15`, When `FilterPipeline.check`
+  runs, Then `{ pass: false, reason: 'too short' }` (src/filter/index.ts:34+).
+- **G2**: Given a prompt with control bytes, When `check` runs, Then it is
+  rejected as non-printable (src/filter/index.ts).
+- **G3**: Given a prompt equal to an existing recent entry, When `check` runs,
+  Then it is rejected as duplicate (src/filter/index.ts).
+- **G4**: Given a dir containing `package.json`, When `detectProject` runs, Then
+  the project name is derived from the package (src/runner/project.ts:25-39).
+- **G5**: Given `extractTopic("fix the login bug")`, When run, Then a short
+  topic string is returned (src/utils/extractTopic.ts:5).
+
+## 8. Key implementation map
+| Concern | File(s) |
+|---|---|
+| FilterPipeline + reasons | src/filter/index.ts:4-80 |
+| Project/language detection | src/runner/project.ts:25-60 |
+| Topic extraction | src/utils/extractTopic.ts:5 |
+| Capture-time assembly | src/commands/log.ts:50-58 |
+
+## 9. Open questions / discrepancies
+- The exact trivial-pattern list lives inline in filter/index.ts; verify the
+  regexes when wiring GWT tests.
+- Duplicate check scope (recent history window vs all) is implementation detail
+  in filter/index.ts — confirm before writing the G3 test.
+
+## 10. Related
+- SPEC-002 (insert), SPEC-003 (capture), SPEC-013 (importers reuse the filter),
+  SPEC-015 (config keys). No tests exist (SPEC-ISSUES-007).

package/docs/specs/SPEC-005-search.md

@@ -0,0 +1,102 @@
+# SPEC-005: Search (FTS + Filters + Semantic)
+
+- **ID**: SPEC-005
+- **Cluster**: Search
+- **Status**: Implemented (baseline)
+- **Date**: 2026-08-08
+- **Source**: reverse-engineered from code (files listed in §8)
+
+## 1. Purpose
+Find prompts by full-text (FTS5 over prompt+response), by metadata filters
+(tool/project/language/role/tag/starred/quality/relevance/date range), or by
+semantic similarity (sqlite-vec, SPEC-008). Backs `ph search`, the TUI text
+filter, the MCP `search_prompts*` tools and the HTTP `/api/prompts/search`.
+
+## 2. Scope
+- **In scope**: `PhDB.search(opts)` dual-path (FTS vs scan), filter building,
+  semantic search, `ph search` command output.
+- **Out of scope**: embeddings generation (SPEC-008), archive search (SPEC-009),
+  TUI filter panel (SPEC-012).
+- **Entry points**: `ph search [--semantic] [--archive] [flags]`, TUI `/`, MCP tools, HTTP endpoints.
+
+## 3. Data Model
+- `SearchOptions` (src/types.ts:54-68): query, tool, project, language, role,
+  tag, starred, minQuality, minRelevance, since, until, limit, semantic.
+- Results: `PromptEntry[]`.
+
+## 4. Flows
+
+### 4.1 Dual-path query (src/db/index.ts:243-330)
+1. Build identical filters for two paths: FTS (JOIN with alias `p.`) and scan
+   (direct `prompts`). Filters use `json_extract` on metadata for
+   project/language/role/tag/starred/quality/relevance (db/index.ts:252-297).
+2. With a text query → FTS path: `prompts_fts JOIN prompts p` via MATCH, filters
+   prefixed `p.` (db/index.ts:300-320).
+3. Without text → scan path: `SELECT * FROM prompts WHERE … ORDER BY timestamp
+   DESC LIMIT ?` (db/index.ts:322-330).
+4. Date range: `since`/`until` become timestamp comparisons.
+
+### 4.2 Semantic search (src/db/index.ts)
+`searchSemantic(queryVector, limit, project?)` — vec0 KNN on `vec_embeddings`
+joined back to prompts. When `project` is supplied, matching is restricted to
+that project before the final result limit. Requires embeddings (SPEC-008).
+Because sqlite-vec applies `k` before ordinary metadata predicates, scoped
+queries currently examine all stored vectors before filtering and limiting.
+
+### 4.3 CLI (src/commands/search.ts)
+`ph search` prints via `printResults` (src/display/print.ts:33); `-i` opens the
+TUI (SPEC-001); `--archive` queries `prompts_archive` (SPEC-009).
+
+## 5. Invariants & Business rules
+- FTS path is used ONLY when a text query exists; otherwise full scan + filters.
+- `tag` filter is a JSON-array LIKE `%"tag"%` — substring match on the serialized
+  array, not a JSON array membership check (db/index.ts:276-282).
+- `limit` defaults to 50 in the CLI; results ordered timestamp DESC.
+- Project-scoped semantic search returns the nearest matching prompts within
+  that project, even when closer prompts from other projects exist.
+
+## 6. UI / UX surface
+`ph search` output: `#id tool date [tags] [exit:N]` + truncated prompt
+(printResults); `--full` shows the complete text.
+
+## 7. Acceptance criteria (Given/When/Then)
+- **G1**: Given prompts with text "goroutines" and "react", When `search({query:
+  "goroutines"})` runs, Then only the goroutines entry is returned via the FTS
+  path (src/db/index.ts:300-320).
+- **G2**: Given prompts from tools claude/gemini, When `search({tool: "gemini"})`
+  runs (no query), Then only gemini entries are returned via the scan path
+  (src/db/index.ts:322-330).
+- **G3**: Given entries with metadata `project: "ph"`, When
+  `search({project: "ph"})` runs, Then only those entries match via
+  `json_extract` (src/db/index.ts:258-263).
+- **G4**: Given a tag `auth` stored in `tags: ["auth","jwt"]`, When
+  `search({tag: "auth"})` runs, Then the entry matches via the LIKE filter
+  (src/db/index.ts:276-282).
+- **G5**: Given `since: 2026-01-01`, When `search` runs, Then entries older than
+  the date are excluded (src/db/index.ts:since handling).
+- **G6**: Given an unrelated project has the nearest vector, When
+  `searchSemantic(query, 1, "go-play")` runs, Then it returns the nearest
+  vector belonging to `go-play` (src/db/index.ts, project scope before limit).
+
+## 8. Key implementation map
+| Concern | File(s) |
+|---|---|
+| Dual-path search | src/db/index.ts:243-330 |
+| Semantic KNN | src/db/index.ts (`searchSemantic`) |
+| CLI output | src/commands/search.ts, src/display/print.ts:33-60 |
+| SearchOptions | src/types.ts:54-68 |
+
+## 9. Open questions / discrepancies
+- `tag` filter uses LIKE on the JSON string — matches `"auth2"` when filtering
+  `auth` (substring). JSON array membership would be stricter.
+- FTS path columns need the `p.` prefix; a new filter added to only one path
+  silently breaks the other (dual maintenance).
+- Project-scoped semantic search currently evaluates the full vector set
+  before metadata filtering because sqlite-vec's KNN limit precedes the join
+  predicate. A project-aware vector index or auxiliary metadata column may
+  reduce this cost if vector counts make it material.
+
+## 10. Related
+- SPEC-002 (primitives), SPEC-008 (embeddings), SPEC-009 (archive),
+  SPEC-011 (MCP), SPEC-010 (HTTP). Vitest coverage: `src/db/__tests__/phdb.test.ts`
+  covers G1-G6, including project-scoped nearest-neighbor behavior.

package/docs/specs/SPEC-006-analysis-pipeline.md

@@ -0,0 +1,98 @@
+# SPEC-006: Analysis Pipeline (LLM)
+
+- **ID**: SPEC-006
+- **Cluster**: AI
+- **Status**: Implemented (baseline)
+- **Date**: 2026-08-08
+- **Source**: reverse-engineered from code (files listed in §8)
+
+## 1. Purpose
+Turns raw prompts into structured knowledge: role/tags/relevance/quality plus
+summary/key_insights/technical_decisions via an LLM. Results feed metadata
+(SPEC-002), memories and project summaries (SPEC-007).
+
+## 2. Scope
+- **In scope**: ANALYSIS_PROMPT, parseAnalysisResponse, analyzePrompt, mergeMetadata,
+  analyzeAll, background analysis spawn, reusability scoring.
+- **Out of scope**: provider details (src/ai/*), memory persistence (SPEC-007).
+- **Entry points**: `ph analyze`, `ph analyze-reusability`, `_bg-analyze`,
+  auto-analysis after `ph log` when `backgroundAnalysis` is enabled.
+
+## 3. Data Model
+- `AnalysisResult` (src/analyzer/index.ts:8-18): project, language, role, tags,
+  relevance (0-10), quality (0-10), summary, key_insights, technical_decisions.
+- `PromptMetadata` (src/types.ts:14-27) — analysis merged into it.
+- `LLMProvider` interface (src/ai/provider.ts:5-12); factory `getProvider(cfg)`
+  (:14) returns Ollama or Gemini.
+
+## 4. Flows
+
+### 4.1 Prompt (src/analyzer/index.ts:20-50)
+`ANALYSIS_PROMPT(text)` demands a single JSON object with the exact field
+contract; input truncated to 2000 chars (:49). No markdown fences allowed.
+
+### 4.2 Parsing (src/analyzer/index.ts:56-120)
+`parseAnalysisResponse` strips fences, extracts the first `{…}` block
+(indexOf/lastIndexOf), JSON.parses; on failure returns `{}` (never throws).
+
+### 4.3 Analyze + merge (src/analyzer/index.ts:123-183)
+`analyzePrompt(text, provider)` → `parseAnalysisResponse(await provider.analyze…)`.
+`mergeMetadata(existing, analysis, force?)` merges fields, preserving manual
+metadata unless `force=true` (:136-183).
+
+### 4.4 Batch (src/analyzer/index.ts:205)
+`analyzeAll(db, provider, opts)` iterates unanalyzed prompts, updates metadata,
+writes memories (SPEC-007). CLI `ph analyze` with `--limit/--force/--prune/
+--dry-run`.
+
+### 4.5 Background (src/background/analyzer.ts:7)
+`spawnBackgroundAnalysis(id, dbPath)` spawns a detached `_bg-analyze` child —
+never blocks the capture path. Enabled via `ph config set background-analysis true`.
+
+### 4.6 Reusability (src/analyzer/reusability.ts)
+`ph analyze-reusability` scores prompts for reuse; `cleanup-reusability`
+deletes low-score ones (SPEC-009 cross-ref).
+
+## 5. Invariants & Business rules
+- Parsing NEVER throws — worst case is an empty AnalysisResult.
+- Analysis is additive: manual fields survive unless `force`.
+- Background analysis is detached: capture latency is never affected.
+- Input truncated to 2000 chars before the LLM.
+
+## 6. UI / UX surface
+`ph analyze` progress output; TUI shows `●` (analyzed) vs `○` (pending) in
+ListEntry (SPEC-012).
+
+## 7. Acceptance criteria (Given/When/Then)
+- **G1**: Given an LLM response with fenced JSON, When `parseAnalysisResponse`
+  runs, Then the JSON is extracted and parsed (src/analyzer/index.ts:56-65).
+- **G2**: Given a non-JSON response, When `parseAnalysisResponse` runs, Then
+  `{}` is returned without throwing (src/analyzer/index.ts:61-70).
+- **G3**: Given existing metadata with a manual role and analysis with a
+  different role, When `mergeMetadata(existing, analysis)` runs without force,
+  Then the manual role is preserved (src/analyzer/index.ts:136-183).
+- **G4**: Given `force=true`, When `mergeMetadata` runs, Then the analysis
+  values win (src/analyzer/index.ts:136-183).
+- **G5**: Given `backgroundAnalysis: true`, When `ph log` runs, Then
+  `spawnBackgroundAnalysis` is invoked with the new prompt id and dbPath
+  (src/commands/log.ts:61-63, src/background/analyzer.ts:7).
+
+## 8. Key implementation map
+| Concern | File(s) |
+|---|---|
+| Prompt + parse + merge + batch | src/analyzer/index.ts:20-275 |
+| Providers + factory | src/ai/provider.ts:5-20, ollama.ts:8, gemini.ts:36 |
+| Background spawn | src/background/analyzer.ts:7 |
+| Reusability | src/analyzer/reusability.ts |
+
+## 9. Open questions / discrepancies
+- `parseAnalysisResponse` uses first/last brace — nested braces or multiple JSON
+  blocks break extraction (same fragility class as lens's old regex plans).
+- `_bg-analyze` passes only (id, dbPath): the child reloads config; a config
+  change between capture and analysis silently changes the model used.
+- Reusability scoring lives in analyzer/reusability.ts — verify the formula when
+  wiring tests.
+
+## 10. Related
+- SPEC-002 (metadata update), SPEC-007 (memories), SPEC-015 (config),
+  SPEC-004 (filter runs before analysis). No tests (SPEC-ISSUES-007).