npm - @gi4nks/ph - Versions diffs - 1.7.0 → 1.8.0 - Mend

@gi4nks/ph 1.7.0 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (30) hide show

package/CHANGELOG.md +15 -1
package/README.md +34 -18
package/dist/cli.js +3383 -3700
package/dist/cli.js.map +1 -1
package/docs/evals/context-retrieval-golden.json +17 -1
package/docs/ph-architecture-review.html +179 -0
package/docs/ph-manual.md +88 -73
package/docs/ph-operational-memory-analysis.html +104 -0
package/docs/plans/2026-10-10-operational-memory.md +192 -0
package/docs/specs/SPEC-001-architecture-cli.md +100 -72
package/docs/specs/SPEC-002-database-layer.md +22 -3
package/docs/specs/SPEC-003-capture-modes.md +22 -9
package/docs/specs/SPEC-005-search.md +46 -11
package/docs/specs/SPEC-006-analysis-pipeline.md +8 -6
package/docs/specs/SPEC-007-memory-summaries.md +19 -19
package/docs/specs/SPEC-008-embeddings.md +8 -2
package/docs/specs/SPEC-009-retention-archive.md +2 -2
package/docs/specs/SPEC-010-remote-sync.md +77 -103
package/docs/specs/SPEC-011-mcp-server.md +7 -4
package/docs/specs/SPEC-012-tui-browser.md +30 -96
package/docs/specs/SPEC-015-config-env.md +4 -0
package/docs/specs/SPEC-016-operational-memory.md +48 -0
package/docs/specs/SPEC-017-architecture-consolidation.md +63 -0
package/docs/specs/SPEC-018-interaction-inspection-and-recall-integrity.md +164 -0
package/docs/specs/SPEC-019-network-capture-and-ranking-integrity.md +36 -0
package/docs/specs/SPEC-INDEX.md +12 -7
package/docs/specs/SPEC-ISSUES.md +104 -18
package/hooks/codex/README.md +1 -1
package/hooks/codex/ph-hook.sh +2 -1
package/package.json +1 -7

package/docs/ph-operational-memory-analysis.html ADDED Viewed

@@ -0,0 +1,104 @@
+<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <meta name="color-scheme" content="light dark">
+  <title>ph — From Prompt History to Operational Memory</title>
+  <style>
+    :root{font-family:Inter,-apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;color:#172033;background:#f4f6fa;font-synthesis:none;text-rendering:optimizeLegibility;--ink:#172033;--muted:#64748b;--line:#dbe2ec;--accent:#0969da;--soft:#eaf2ff;--good:#087443;--warn:#995c00;--card:#fff}
+    *{box-sizing:border-box}body{margin:0;line-height:1.58}main{max-width:1040px;margin:0 auto;padding:48px 28px 72px}header{padding:34px 38px;border-radius:20px;background:linear-gradient(135deg,#102a43,#174e7a);color:white}header p{max-width:720px;color:#d6e7f5}h1{font-size:clamp(2rem,5vw,3.3rem);line-height:1.08;letter-spacing:-.04em;margin:.2em 0}h2{margin:44px 0 12px;font-size:1.55rem;letter-spacing:-.025em}h3{margin:0 0 7px;font-size:1.05rem}p{margin:8px 0 14px}.eyebrow{text-transform:uppercase;letter-spacing:.14em;font-size:.75rem;font-weight:750;color:#a8d5ff}.meta{display:flex;gap:18px;flex-wrap:wrap;font-size:.9rem;color:#d6e7f5}.callout{background:var(--soft);border-left:4px solid var(--accent);padding:17px 20px;border-radius:0 12px 12px 0;margin:22px 0}.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:14px}.card{background:var(--card);border:1px solid var(--line);border-radius:14px;padding:18px}.card p{font-size:.94rem;color:var(--muted);margin-bottom:0}.tag{display:inline-block;background:#edf2f7;color:#334155;border-radius:999px;padding:3px 9px;font-size:.77rem;font-weight:700}.tag.good{background:#e8f6ee;color:var(--good)}table{width:100%;border-collapse:collapse;background:var(--card);border:1px solid var(--line);border-radius:14px;overflow:hidden;font-size:.91rem}th,td{text-align:left;vertical-align:top;padding:13px 14px;border-bottom:1px solid var(--line)}th{background:#edf2f7;font-size:.82rem;text-transform:uppercase;letter-spacing:.06em}tr:last-child td{border-bottom:0}td:first-child{font-weight:700;min-width:150px}.decision{border:2px solid var(--accent);border-radius:16px;padding:22px;background:var(--card)}.decision strong{color:var(--accent)}ol,ul{padding-left:1.25rem}li{padding:3px 0}.flow{display:grid;grid-template-columns:repeat(4,1fr);gap:10px;margin:18px 0}.step{padding:15px;border-radius:12px;background:var(--card);border:1px solid var(--line)}.step b{display:block;color:var(--accent);margin-bottom:5px}.small{font-size:.88rem;color:var(--muted)}code{background:#edf2f7;border-radius:5px;padding:2px 5px;font-size:.9em}footer{border-top:1px solid var(--line);margin-top:42px;padding-top:18px;color:var(--muted);font-size:.84rem}
+    @media(max-width:700px){main{padding:20px 14px 44px}header{padding:25px 22px}.flow{grid-template-columns:1fr 1fr}table{font-size:.82rem}th,td{padding:9px}}
+    @media(prefers-color-scheme:dark){:root{color:#e5edf7;background:#0c1421;--ink:#e5edf7;--muted:#a3b2c5;--line:#2c3b50;--soft:#132b45;--card:#121e2d}header{background:linear-gradient(135deg,#0c1c2b,#153d5d)}th{background:#1b2a3c}code,.tag{background:#253448;color:#d7e6f8}.tag.good{background:#123a2b;color:#86e0ae}}
+  </style>
+</head>
+<body>
+<main>
+  <header>
+    <div class="eyebrow">Product analysis · October 10, 2026</div>
+    <h1>From Prompt History<br>to Operational Memory</h1>
+    <p>Product question: How can ph turn prompts, responses, and analysis into knowledge that helps with the next task handled by an AI agent?</p>
+    <div class="meta"><span>Scope: ph</span><span>Status: direction approved; Phases 0–3 implemented</span><span>Horizon: first incremental release</span></div>
+  </header>
+  <div class="callout"><strong>Thesis:</strong> ph creates value when it prevents context from being repeated, decisions from being rediscovered, and failed attempts from being repeated. Conversation analysis is a means; the outcome is reliable memory brought into the workflow at the right time.</div>
+  <h2>Starting Point</h2>
+  <div class="grid">
+    <article class="card"><span class="tag good">Available</span><h3>Work evidence</h3><p>Prompts and responses are captured with project, tool, and Git context. Memories can retain the IDs of their source prompts.</p></article>
+    <article class="card"><span class="tag good">Available</span><h3>Project knowledge</h3><p>Append-only memories, curated summaries, MCP decision saving, and shared retrieval through <code>ph context</code>, chat, and MCP tools.</p></article>
+    <article class="card"><span class="tag">Gap</span><h3>No guided workflow</h3><p>There is no explicit step to prepare a task with relevant prior work and no review step to turn its outcome into verifiable knowledge.</p></article>
+  </div>
+  <p class="small">The existing ph evolution plan already established curated memory, MCP, and noise reduction. This proposal makes the operational cycle explicit without replacing those foundations.</p>
+  <h2>Options Considered</h2>
+  <table>
+    <thead><tr><th>Option</th><th>Benefits</th><th>Costs and risks</th><th>Assessment</th></tr></thead>
+    <tbody>
+      <tr><td>A · Analytics dashboard</td><td>Makes volume, roles, models, and trends visible; useful for inspecting history.</td><td>Does not bring knowledge into the workflow. It may optimize metrics that do not reduce repeated work.</td><td><span class="tag">Not a priority</span></td></tr>
+      <tr><td>B · Automatic injection everywhere</td><td>Context is always available with little user effort.</td><td>Can add noise, tokens, latency, and stale context to every request. Its usefulness is difficult to establish.</td><td><span class="tag">Avoid as the default</span></td></tr>
+      <tr><td>C · Prepare → work → approve</td><td>Makes reuse intentional, preserves sources and consent, and uses existing context, memory, and MCP capabilities.</td><td>Requires a brief user action and a useful proposal summary. Manual promotion adds a step.</td><td><span class="tag good">Recommended</span></td></tr>
+      <tr><td>D · Autonomous memory agent</td><td>Could extract and update knowledge without interrupting the workflow.</td><td>Harder to control; silent corrections, duplicates, and unapproved decisions could contaminate memory.</td><td><span class="tag">Consider after testing C</span></td></tr>
+    </tbody>
+  </table>
+  <h2>Decision</h2>
+  <section class="decision">
+    <p><strong>Build a deliberate, measurable workflow and reuse the current data model in the first release.</strong></p>
+    <p>Memory becomes useful when a user or agent consults it before acting and can save what it learned with references to the conversations that support it. Do not introduce a new knowledge graph or always-on agent in the first release.</p>
+    <p class="small">Decision record: direction approved in conversation; implementation details are in <a href="plans/2026-10-10-operational-memory.md">the operational memory work plan</a>.</p>
+  </section>
+  <h2>Implementable Workflow</h2>
+  <div class="flow">
+    <div class="step"><b>1 · Prepare</b><code>ph prepare "task"</code><br><span class="small">Detect project, retrieve relevant context, identify sources.</span></div>
+    <div class="step"><b>2 · Work</b><span class="small">The user or agent completes the task with essential context.</span></div>
+    <div class="step"><b>3 · Review</b><code>ph close</code><br><span class="small">Propose decisions or procedures with supporting prompt IDs.</span></div>
+    <div class="step"><b>4 · Promote</b><span class="small">Approve before saving or updating project memory.</span></div>
+  </div>
+  <ul>
+    <li><strong>Prepare:</strong> Reuse <code>getProjectContext()</code> and semantic search. Produce a short brief with decisions, gotchas, prior work, and source IDs. State clearly when there is no evidence.</li>
+    <li><strong>Close:</strong> Start from recent prompts or an explicitly selected range/session. Propose zero or more decisions and procedures. Show what would be saved and which <code>prompt_ids</code> support it.</li>
+    <li><strong>Approval before promotion:</strong> Reuse the existing <code>save_decision</code> and project summary paths, extending them to preserve sources. No permanent update without approval.</li>
+    <li><strong>Lightweight feedback:</strong> Later, consider useful / irrelevant / stale feedback for retrieved memories. Existing access counters are not an explicit usefulness judgment.</li>
+  </ul>
+  <h2>Trade-offs and Guardrails</h2>
+  <table>
+    <thead><tr><th>Product decision</th><th>Choice</th><th>Reason</th></tr></thead>
+    <tbody>
+      <tr><td>When to retrieve</td><td>On request through <code>prepare</code> and agent paths explicitly chosen by the user</td><td>Reduces irrelevant additions and keeps the workflow predictable.</td></tr>
+      <tr><td>What to keep</td><td>Confirmed decisions and procedures linked to sources</td><td>Reduces duplicates and makes a memory auditable and correctable.</td></tr>
+      <tr><td>Evidence status</td><td>Distinguish verified, inferred, and unverified claims</td><td>A fluent summary is not proof that a procedure still works.</td></tr>
+      <tr><td>Cross-project retrieval</td><td>Out of scope for the first release</td><td>Information from one project may be false or sensitive in another.</td></tr>
+    </tbody>
+  </table>
+  <h2>Cost, Rate Limits, and Latency</h2>
+  <div class="grid">
+    <article class="card"><h3>Prepare</h3><p>Lexical search and database reads are local. Semantic retrieval may call an Ollama embedding endpoint; there is no cloud generation cost, while latency and availability depend on the server.</p></article>
+    <article class="card"><h3>Close</h3><p>Proposal extraction can use one optional LLM call. OpenRouter may incur cost and rate limits; MLX-Serve depends on the tunnel and loaded model. Make the provider explicit and let the user skip proposals.</p></article>
+    <article class="card"><h3>Brief limits</h3><p>Apply a configurable character or token budget, show source counts, and truncate raw prompts first. Do not make extra LLM calls whenever the TUI opens.</p></article>
+  </div>
+  <h2>How to Measure Success</h2>
+  <p>Measure useful reuse, not the volume of analysis:</p>
+  <ul>
+    <li><strong>Prepare relevance:</strong> the first retrieved memories are relevant to the task and project in judged cases.</li>
+    <li><strong>Reuse rate:</strong> the user confirms that prior work changed or accelerated the task.</li>
+    <li><strong>Source reliability:</strong> every proposed decision links to verifiable prompts and responses; no source is attributed incorrectly.</li>
+    <li><strong>Workflow friction:</strong> prepare and close take few steps and can be skipped.</li>
+    <li><strong>Real task evaluation:</strong> expand the retrieval set in <code>docs/evals/context-retrieval-golden.json</code> with work queries from multiple projects.</li>
+  </ul>
+  <h2>Usage Check · October 10, 2026</h2>
+  <div class="callout"><strong>Result:</strong> The full workflow worked against an isolated temporary database. Semantic embeddings were unavailable, so <code>prepare</code> clearly fell back to recent project context. <code>close --dry-run</code> showed the selected source without a model call or database changes. With an SSH port forward to Parmenide, interactive <code>close</code> generated a proposal, accepted approval, and saved it with its source prompt ID.</div>
+  <p><code>ph context --memories-only</code> then returned the promoted knowledge and source reference. The generated proposal included plausible details beyond the sample evidence, so users should review and edit proposals before approval. All writes were limited to the temporary test database.</p>
+  <footer>
+    <strong>Local sources:</strong> <code>src/context/index.ts</code>, <code>src/types.ts</code>, <code>src/db/index.ts</code>, <code>src/mcp/server.ts</code>, <code>README.md</code>, <code>docs/superpowers/specs/2026-06-12-ph-evolution-design.md</code>, <code>docs/evals/context-retrieval.md</code>.<br>
+    This document records the product analysis and initial recommendation. Current implementation status is tracked in the linked work plan and SPEC-016.
+  </footer>
+</main>
+</body>
+</html>

package/docs/plans/2026-10-10-operational-memory.md ADDED Viewed

@@ -0,0 +1,192 @@
+# Work Plan: Making ph an Operational Memory
+**Date:** 2026-10-10
+**Status:** Phases 0–3 implemented; Phase 4 deferred until real usage feedback is available.
+**Product decision:** Guide the `prepare → work → review → promote` cycle by reusing existing retrieval, memories, summaries, and MCP tools.
+**Analysis:** [From Prompt History to Operational Memory](../ph-operational-memory-analysis.html)
+## Goal
+At the start of a task, ph should retrieve relevant prior work and show its sources. At the end, it should help preserve reusable decisions and procedures, always with explicit user approval.
+Success means reducing context that must be explained again and work that must be repeated, rather than increasing the number of analyzed prompts.
+## Current State and Implications
+- `getProjectContext()` is shared by `ph context`, `ph chat`, and MCP. It supports project summaries, memories, and relevant interactions.
+- Memories include `prompt_ids`, but MCP `save_decision` previously created decisions with `prompt_ids: []`.
+- `project_summaries` aggregates insights and decisions; `memories` preserves the detailed history.
+- `check_project_knowledge` and `get_project_context` are already available through MCP. The first slice should improve reuse of evidence rather than add a second memory system.
+- Deterministic context retrieval tests and a small golden set already exist. The evaluation set should be expanded before retrieval is optimized.
+## Architecture
+```text
+User task
+    │
+    ▼
+ph prepare ──► shared retrieval ──► compact brief + sources
+                                      │
+                                      ▼
+                                  AI work
+                                      │
+                                      ▼
+ph close ──► selected prompts ──► knowledge proposals
+                                      │
+                               user approval
+                                      │
+                                      ▼
+                    memories(prompt_ids) + project_summaries
+                                      │
+                                      └────► CLI / MCP / next prepare
+```
+## Phases
+### Phase 0 — Retrieval Contract and Evaluation
+**Purpose:** Define what makes a result useful and establish a baseline before changing search behavior.
+**Activities**
+- Expand `docs/evals/context-retrieval-golden.json` with queries across multiple projects, covering decisions, resolved bugs, procedures, and models/providers.
+- Label each case with relevant prompt or memory IDs, the project, and examples of misleading results.
+- Record a `Recall@3` baseline and manually review relevance; do not optimize for result volume.
+- Define source references using memory ID, prompt ID, date, tool, and project.
+**Acceptance criteria**
+- At least 12 judged queries across at least four projects.
+- Results distinguish “no evidence” from “evidence found.”
+- The evaluation test reports repeatable metrics without calling a generative LLM.
+### Phase 1 — `ph prepare`: Prepare for a Task
+**Interface**
+```bash
+ph prepare "add the Wise provider to ph"
+ph prepare "diagnose the SQLite lock" --project picot --limit 5
+ph prepare "evaluate MLX-Serve" --budget 3500
+```
+**Activities**
+- Add `src/commands/prepare.ts` and connect it to `src/cli.ts`.
+- Reuse `getProjectContext()` and `formatProjectContext()`; do not duplicate retrieval or formatting in a new service.
+- Detect the project from the current directory, with an explicit `--project` override.
+- Use the task as a semantic query and provide a clear fallback when Ollama embeddings are unavailable.
+- Print a compact brief with decisions, gotchas, prior work, and sources (`memory #N`, `prompt #N`, date, and tool).
+- Add an output budget. Truncate raw interactions first and preserve source IDs.
+- Keep `ph context` compatible and document the distinction: `context` explores; `prepare` creates a brief for a task.
+**Acceptance criteria**
+- `prepare` does not write to the database.
+- It makes no generative LLM call; local embeddings are used only for semantic retrieval.
+- Output is reproducible, bounded by the content budget, and has traceable sources.
+- An explicit `--project` prevents project mixing; a missing detected project gives a clear instruction.
+- A query with no results does not invent knowledge and says so.
+### Phase 2 — `ph close`: Propose What to Keep
+**Interface**
+```bash
+ph close --since 3h
+ph close --from 1670 --to 1682
+ph close --dry-run --since 3h
+```
+**Activities**
+- Add `src/commands/close.ts`; reuse `PhDB`, `LLMProvider`, and the existing `save_decision` persistence behavior where applicable.
+- Select inputs explicitly by project and time window or ID range. Show how many prompt/response pairs will be included before generation.
+- Make one optional analysis call to propose zero or more decisions/insights, without promoting them automatically.
+- Show proposed text, sources, and destination fields; allow approval, text editing, or cancellation.
+- After approval, append memories with their source prompt IDs and update `project_summaries`.
+- Extend MCP `save_decision` with optional `promptIds`; validate that IDs exist and belong to the declared project.
+- Avoid new tables or knowledge types in the first release. Store decisions and procedures in existing fields.
+**Acceptance criteria**
+- `--dry-run` performs no writes.
+- Cancelling review does not change `memories` or `project_summaries`.
+- Each approved proposal retains at least one source ID.
+- A prompt from another project cannot be attached accidentally.
+- The aggregate summary remains consistent and detailed memories remain append-only.
+- Provider errors, missing providers, and malformed responses do not cause partial promotion.
+### Phase 3 — Bring the Workflow to Agents Through MCP
+**Activities**
+- Make `check_project_knowledge` semantically relevant by reusing `getProjectContext()` instead of substring matching memories alone.
+- Include sources and a clear outcome: knowledge found, partial evidence, or no evidence.
+- Reuse MCP `save_decision` for end-of-task promotion, with source references.
+- Document a client-neutral agent workflow for Codex, OpenCode, and other MCP clients.
+- Do not inject context into every prompt automatically; the client or user invokes preflight when a meaningful task begins.
+**Acceptance criteria**
+- The same evaluation case returns the same evidence from `ph prepare` and MCP.
+- MCP tools do not expose content from other projects.
+- Promotion preserves source `prompt_ids` validated by the server.
+### Phase 4 — Feedback and Knowledge Maintenance
+**Deferred until Phases 1–3 have been used on real work.**
+- Add `ph feedback <memory-id> useful|irrelevant|stale` and an MCP equivalent only if agents need them.
+- Decide whether feedback belongs in a separate event table or updateable memory fields; preserve history if judgments can change over time.
+- Use feedback and access counts for ranking and review, not automatic deletion.
+- Add a review command for old or contradicted decisions, with sources and last verification date.
+**Gate:** proceed only after `prepare` is used on real tasks and some results are confirmed useful. Do not add complex ranking before feedback exists.
+## Decisions and Trade-offs
+| Topic | Decision | Rationale and cost |
+|---|---|---|
+| Entry points | Add `prepare` and `close`, keep `context` | The names describe two work moments; existing scripts remain compatible. |
+| Promotion | Require human approval before writes | Adds a step while preventing false or noisy permanent memory. |
+| Sources | Reuse `MemoryEntry.prompt_ids` | The field already exists; the main gap was MCP persistence. |
+| Data model | No new tables in Phases 0–3 | Existing summaries, insights, decisions, and sources are sufficient for the first validation. |
+| Retrieval | Prioritize relevance and project scoping before personalization | Reduces cross-project contamination and supports deterministic measurement. |
+| Automation | No global auto-injection or auto-promotion in the MVP | Avoids noise, wasted tokens, and silent changes. |
+## Cost, Rate Limits, and Latency
+- **Prepare:** SQLite is local. Semantic retrieval uses the configured embedding endpoint (Ollama by default); it adds a request and local-server latency, but no generative cloud call.
+- **Close:** One LLM request creates proposals. OpenRouter may incur cost and rate limits; MLX-Serve depends on the tunnel, loaded model, and available memory. Users choose the configured provider and approve before writing.
+- **Budget:** Limit included prompt/response text, show the number of sources, and apply an explicit budget to the `prepare` brief. Do not hide truncation.
+- **Degraded operation:** If embeddings or the provider are unavailable, keep search and manual saving usable and identify the unavailable step without fabricating results.
+## Definition of Done
+- Existing tests remain green. New tests cover sources/projects, output budget, dry-run, cancellation, and `promptIds` validation.
+- Golden set has at least 12 queries with an initial `Recall@3 ≥ 0.8` target and manual review of misses. Recalibrate if the labels show that the set is insufficient.
+- Three real tasks across different projects complete the workflow; the user can judge whether the brief prevented repeated decisions or investigation.
+- Every promoted item links to its evidence; unapproved work never changes curated memory.
+- README and manual document `prepare`, `close`, provider configuration, costs, and offline behavior.
+## Usage Check (2026-10-10)
+Using an isolated temporary database seeded with one sample prompt:
+- `ph prepare "project cache path" --project ph --limit 3 --budget 1200` displayed the matching prompt and response with their source ID. Ollama embeddings were unavailable, and the command clearly reported that it was using recent project context instead.
+- `ph close --from 1 --to 1 --project ph --dry-run` displayed the selected prompt and response, then confirmed that it made no model call and no database changes.
+- The configured direct localhost route to MLX-Serve failed. An SSH port forward to Parmenide (`18034 → 127.0.0.1:11234`) restored connectivity. Interactive `ph close` generated a proposal, accepted approval, and saved memory `#1` linked to prompt `#1` in the temporary database.
+- `ph context --memories-only --project ph` then returned the saved summary, insights, decisions, and `prompt #1` provenance. The generated proposal added plausible details beyond the sample evidence, reinforcing the need to inspect and edit proposals before approval.
+- The full usage path was exercised on the temporary database only; no user history or project memory was changed.
+## Delivery Order
+1. Phase 0: evaluation cases and source contract.
+2. Phase 1: read-only `prepare`.
+3. Phase 2: `close` with preview, approval, and provenance.
+4. Phase 3: shared retrieval and promotion through MCP.
+5. Phase 4: explicit feedback and review after real usage.
+Each phase can ship as a separate change. The first operational version is complete with Phases 0–3; Phase 4 depends on observing real usage and can be planned separately.

package/docs/specs/SPEC-001-architecture-cli.md CHANGED Viewed

@@ -2,81 +2,109 @@
 - **ID**: SPEC-001
 - **Cluster**: Core
-- **Status**: Implemented (baseline)
-- **Date**: 2026-08-08
-- **Source**: reverse-engineered from code (files listed in §8)
+- **Status**: Implemented (plain-text daily workflow)
+- **Date**: 2026-10-22
+- **Source**: current repository behavior
 ## 1. Purpose
-`ph` is a transparent observability layer for AI CLI tools: every prompt+response
-is captured to a local SQLite DB (SPEC-002), analyzed (SPEC-006), and made
-browsable via TUI (SPEC-012), CLI commands, MCP (SPEC-011) and HTTP (SPEC-010).
-This spec covers the entry point and the command dispatch architecture.
+PH captures AI CLI interactions, finds earlier work, retrieves project context,
+and supports explicit review of durable project knowledge. The everyday
+interface is a small plain-text CLI; maintenance and integration commands are
+available separately through `ph help --all`.
 ## 2. Scope
-- **In scope**: cli.ts main(), command resolution, wrapper-mode fallback, config
-  loading order, TUI bootstrap.
-- **Out of scope**: individual commands (their own specs), the TUI internals.
-- **Entry points**: `ph` (no args → TUI), `ph <command>`, `ph <tool> "prompt"` (wrapper).
-## 3. Data Model
-- `argv` split into `command` (first non-flag arg) + `cmdArgs` + `preArgs`
-  (ph-specific flags before the command, e.g. `--ph-tag`).
-- `PhConfig` (src/config/index.ts:5) loaded before dispatch; `geminiApiKey`
-  is copied to `process.env.GEMINI_API_KEY` (cli.ts:153-155).
-- DB path resolution: `PH_DB` env > `cfg.dbPath` > `defaultPath()`
-  (cli.ts:157).
-## 4. Flows
-### 4.1 Dispatch (src/cli.ts:148-441)
-1. Load config; set env; resolve dbPath.
-2. No args + TTY → render BrowseApp in alt-screen, wait for exit, optional
-   rerun (cli.ts:159-186).
-3. Find first non-flag arg; flags `--ph-tag`/`--ph-role` consume their value
-   (cli.ts:190-203).
-4. Switch on the command name: ~25 cases (search, context, last, sessions,
-   stats, cluster, analyze*, cleanup*, star, export, import, analyze, mcp,
-   server, remote, log, embed-all, vacuum, config, _bg-analyze, browse, chat,
-   timeline, ollama-models, help).
-5. Unknown command → wrapper mode: `cmdWrap(dbPath, command, [...preArgs,
-   ...cmdArgs], cfg)` (cli.ts:436-439).
-### 4.2 Wrapper mode (SPEC-003)
-Unknown first arg = a real binary (`claude`, `gemini`, …): run it and capture.
-## 5. Invariants & Business rules
-- ESM only, `.js` import extensions (package.json `"type": "module"`).
-- Manual arg parsing (`parseFlags`, src/commands/_utils.ts:1) — no commander.
-- Every command handler owns its `PhDB` instance and closes it
-  (`new PhDB(...)` … `db.close()` per case).
-- `main()` catches everything: `ph: <message>` on stderr, exit 1 (cli.ts:443-446).
-## 6. UI / UX surface
-USAGE text is a single ~200-line template literal (cli.ts:38-140); `ph --help`,
-`-h`, `help` and non-TTY no-arg all print it.
-## 7. Acceptance criteria (Given/When/Then)
-- **G1**: Given `ph search foo`, When main() runs, Then the `search` case opens a
-  PhDB, calls `cmdSearch` and closes the DB (cli.ts:250-253).
-- **G2**: Given `ph claude "hello"` (unknown command), When main() runs, Then
-  `cmdWrap` receives tool=`claude` and args `["hello"]` (cli.ts:436-439).
-- **G3**: Given `ph` with no args on a non-TTY stdin, When main() runs, Then the
-  USAGE text is printed and the process exits 0 (cli.ts:183-186).
-- **G4**: Given `--ph-tag auth claude "x"`, When parsing, Then `auth` is consumed
-  as the tag value and `claude` is the command (cli.ts:192-194).
-## 8. Key implementation map
+- **In scope**: CLI startup, concise and advanced help, command dispatch,
+  compatibility aliases, wrapper fallback, and task-aware context retrieval.
+- **Out of scope**: command internals outside context, MCP/HTTP protocols, and
+  import formats.
+- **Entry points**: `ph` prints a concise usage guide; `ph <tool> [args...]`
+  wraps and captures an AI CLI; `search`, `show`, `context`, and `close` serve the daily
+  history and memory workflows.
+## 3. Dispatch Rules
+1. Load config and resolve the database path.
+2. With no arguments, print the short guide and exit without opening a database
+   or starting an interactive UI.
+3. `ph --help` and `ph help` print the short guide. `ph --help --all` and
+   `ph help --all` print the full maintenance and integration catalog.
+4. `context` and the legacy `prepare` command share task-aware retrieval and
+   its recent-history fallback.
+5. `close` defaults to reviewing the last three hours; explicit time ranges
+   and prompt ID ranges remain supported.
+6. `browse` remains a text-only compatibility alias for recent interactions.
+   `search -i` remains accepted and prints normal text results.
+7. Unknown commands fall through to transparent wrapper mode.
+## 4. User-Facing Jobs
+| Job | Primary command | Contract |
+|---|---|---|
+| Setup | `ph setup [tool...]` | Detect supported installed tools by default and install native capture integrations while preserving existing configuration |
+| Capture | Native tool hook after setup; `ph <tool> [args...]` remains optional | Capture completed exchanges without changing the user's normal AI CLI invocation |
+| Find | `ph search <query>` | Search the current project, show five relevant matches; use `--all` to broaden |
+| Inspect | `ph show <id>` | Display the complete captured interaction and its analysis |
+| Resume | `ph context "task"` | Optional explicit retrieval of project knowledge and relevant interactions |
+| Review | `ph close` | Optional review of recent work; propose durable knowledge and require explicit approval |
+Daily use requires setup once, then normal use of the supported AI CLI; search
+is the primary PH command and `show` opens any result in full. `prepare` is a compatibility alias for
+`context`. There is no TUI in the runtime package. History can also be read
+with `last` or the `browse` compatibility alias.
+## 5. Invariants
+- ESM only; relative imports use `.js` extensions.
+- Existing advanced command names remain available unless separately removed.
+- Capture and full-text search do not require an AI provider.
+- `setup` writes only PH hook entries and preserves other JSON configuration;
+  rerunning setup does not duplicate its entries.
+- Setup does not import prior sessions or start an analysis provider.
+- Retrieval labels its recent-history fallback; it does not silently claim
+  semantic matching when embeddings are unavailable.
+- `close` does not call a provider or mutate storage without a TTY and explicit
+  approval of proposed knowledge.
+## 6. Acceptance Criteria
+- **G1**: Given `ph` in either a TTY or a pipe, when it starts, then it prints
+  the same short guide, exits successfully, and emits no alternate-screen or
+  interactive terminal controls.
+- **G2**: Given `ph help --all`, when help is requested, then the full advanced
+  command catalog is shown.
+- **G3**: Given `ph context "cache timeout"` in a detected project, when semantic
+  embeddings are unavailable, then recent project evidence is shown with an
+  explicit fallback label and no database writes occur.
+- **G4**: Given `ph prepare "cache timeout"`, when dispatched, then the same
+  task-aware context path and output are used.
+- **G5**: Given `ph close --dry-run` with no explicit range, when the project
+  contains interactions from the last three hours, then they are selected and
+  shown without provider calls or database writes.
+- **G6**: Given `ph browse` or `ph search -i "cache"`, when invoked, then
+  matching/recent plain-text results are printed without an interactive UI.
+- **G7**: Given an existing Claude/Codex/Gemini JSON configuration, when
+  `ph setup <tool>` runs, then its existing keys are preserved, one PH hook is
+  installed, and rerunning setup does not duplicate that hook; OpenCode's
+  existing `ph-plugin.ts` is left untouched if it differs from PH's plugin.
+- **G8**: Given a supported tool name, when `ph setup` is called with explicit
+  tool names, then only those integrations are installed; without names, only
+  installed supported CLIs are detected and configured.
+## 7. Implementation Map
 | Concern | File(s) |
 |---|---|
-| Entry + dispatch + USAGE | src/cli.ts:38-441 |
-| Config load/save | src/config/index.ts:26-40 |
-| Flag parsing | src/commands/_utils.ts:1-30 |
-| Shared TUI bootstrap | src/cli.ts (`openBrowser`, `runRerun`) |
-## 9. Open questions / discrepancies
-- `--ph-debug` is listed in USAGE but only consumed by wrapper paths (inline/pty
-  debug logs); no validation if used with non-wrapper commands.
-## 10. Related
-- All other specs (commands dispatch into each), SPEC-012 (BrowseApp),
-  SPEC-003 (wrapper mode). CLI dispatch tests remain part of SPEC-ISSUES-007.
+| Startup, help, dispatch | `src/cli.ts` |
+| Search and `-i` compatibility | `src/commands/search.ts` |
+| Task-aware context | `src/commands/context.ts`, `src/knowledge/task-context.ts` |
+| Three-hour review default | `src/commands/close.ts` |
+| Native capture setup | `src/commands/setup.ts` |
+| CLI / operational memory tests | `src/commands/__tests__/operational-memory.test.ts` |
+## 8. Related
+`SPEC-003` (capture wrapper), `SPEC-005` (search), `SPEC-016` (operational
+memory), and `SPEC-012` (retired history browser; retained as historical
+documentation).

package/docs/specs/SPEC-002-database-layer.md CHANGED Viewed

@@ -2,7 +2,7 @@
 - **ID**: SPEC-002
 - **Cluster**: DB
-- **Status**: Implemented (baseline)
+- **Status**: Implemented; internal repository extraction in SPEC-017
 - **Date**: 2026-08-08
 - **Source**: reverse-engineered from code (files listed in §8)
@@ -28,6 +28,9 @@ effort migrations, zero ORM.
 - `embeddings` (legacy BLOB, 87-91) + `vec_embeddings` vec0 float[768] (100-104).
 - `memories` (106-121) append-only per project; `project_summaries` (PhDB.PROJECT_SUMMARIES_SCHEMA, :123);
   `prompts_archive` (PhDB.ARCHIVE_SCHEMA, :124).
+- `ph_meta` stores a persistent database UUID and one-time migration markers.
+  The `search_metadata` backfill marker prevents reopening PH from rescanning
+  rows whose derived metadata is legitimately empty.
 - `PromptEntry` / `PromptMetadata` / `MemoryEntry` / `ProjectSummary` types
   (src/types.ts:1-52).
@@ -77,18 +80,28 @@ None — library layer consumed by commands, TUI, MCP and HTTP server.
   consistent with the tables (src/db/index.ts:410+).
 - **G6**: Given a fresh DB, When opened, Then `vec_embeddings` exists as a vec0
   table with float[768] (src/db/index.ts:95-104).
+- **G7**: Given an existing prompt database, When the searchable-metadata
+  migration completes, Then its marker is persisted and a later open does not
+  rerun the backfill query; the database UUID survives reopen and differs for
+  a separate database file.
 ## 8. Key implementation map
 | Concern | File(s) |
 |---|---|
 | Schema + migrations | src/db/index.ts:47-196 |
 | CRUD + search + sync primitives | src/db/index.ts:198-531 |
-| Memories + summaries | src/db/index.ts:535-720 |
-| Archive | src/db/index.ts:464-518 |
+| Memory and summary persistence | src/db/memory-repository.ts |
+| Archive persistence | src/db/archive-repository.ts |
+| Public facade and migrations | src/db/index.ts |
 | Semantic vector index | src/db/semantic-index.ts |
 | Types | src/types.ts:1-52 |
 ## 9. Open questions / discrepancies
+- Routine `PhDB` construction still runs schema probes and a JSON query across
+  prompts missing `$schema_version`. The search-metadata backfill is one-time,
+  but remaining migration checks scale with history and repeat for each command
+  and capture hook. Track schema migrations explicitly to bound normal startup
+  work (SPEC-ISSUES-027).
 - **Architecture decision (2026-10-09)**: keep `PhDB` as the single SQLite adapter
   for now. Context consumers depend on the focused `ProjectContextStore` interface
   (`src/context/index.ts`). Vector encoding, compatibility storage, and KNN
@@ -96,6 +109,12 @@ None — library layer consumed by commands, TUI, MCP and HTTP server.
   methods as the adapter. Scope behavior has an integration test. Keep unrelated
   CRUD/memory/archive methods together until a second adapter or concrete
   independent migration/testing seam justifies further division.
+- **Architecture decision (2026-10-22, SPEC-017)**: the atomic memory-promotion
+  operation now justifies internal responsibility seams. Keep prompt CRUD, FTS,
+  schema, and migrations together in `PhDB`; move memory/summary and archive
+  operations into internal repositories sharing its connection. Keep `PhDB`
+  source-compatible, preserve the single transaction boundary, and do not
+  change the schema.
 - `response` column via ALTER (SPEC-ISSUES-001).
 - `memories.prompt_ids` possibly unused after summaries (SPEC-ISSUES-014).
 - `delete` (427) and `deleteById` (435) coexist — duplicate deletion paths.

package/docs/specs/SPEC-003-capture-modes.md CHANGED Viewed

@@ -2,7 +2,7 @@
 - **ID**: SPEC-003
 - **Cluster**: Capture
-- **Status**: Implemented (baseline)
+- **Status**: Implemented; shared live lifecycle defined by SPEC-019
 - **Date**: 2026-08-08
 - **Source**: reverse-engineered from code (files listed in §8)
@@ -37,15 +37,20 @@ or `ph log`. Plus the OpenCode plugin for real-time capture.
    sequences. On Enter: finalize previous response, call `onPrompt`.
 4. On PTY exit: finalize last response, restore stdin, resolve with exitCode.
-### 4.2 Inline (src/runner/inline.ts:37)
-`runInline` captures args → `extractPrompt(args)` (:8), runs the binary
-non-interactively (spawnSync), records exit code, then `db.insert`.
+### 4.2 Inline (src/runner/inline.ts)
+`runInline` captures args → `extractPrompt(args)`, persists a normalized record
+with lifecycle status, spawns the tool, forwards stdout unchanged while
+collecting it as the interaction response, then persists response and exit
+code. The shared completion helper dispatches configured analysis and remote
+sync as the PTY and direct-log paths. Configured remote pushes have a
+five-second timeout; invalid URLs do not prevent local capture.
 ### 4.3 Direct log (src/commands/log.ts:9-70)
 `ph log --tool <t> --prompt <p> [--response <r>]` builds the entry (project
-detection, language detection, topic extraction), inserts, then optionally
-spawns background analysis (SPEC-006) and a fire-and-forget remote push
-(SPEC-010).
+detection, language detection, topic extraction), inserts, then dispatches
+configured analysis and a remote push with a five-second timeout through the
+shared live capture lifecycle (SPEC-019). Lifecycle states are stored on the
+interaction and shown by `ph status`.
 ### 4.4 Hooks + plugin
 - `hooks/claude/ph-hook.sh`, `hooks/gemini/ph-hook.sh` — shell scripts invoked
@@ -63,8 +68,10 @@ spawns background analysis (SPEC-006) and a fire-and-forget remote push
 - Capture never blocks the wrapped tool (background processes for analysis/push).
 - Binary resolution: `resolveRealBinary(tool)` maps known tool names
   (src/runner/inline.ts:15).
-- All supported capture adapters produce the same normalized PromptEntry shape;
-  source-specific metadata remains attached to that record.
+- All live adapters pass through the shared lifecycle preparation/completion
+  seam; source-specific metadata remains attached to the normalized record.
+- Inline wrapper output is both visible to the caller and stored as `response`;
+  child exit status is retained. Empty output is a valid completed response.
 ## 6. UI / UX surface
 Transparent: the wrapped tool's own output is forwarded verbatim; ph itself
@@ -85,6 +92,12 @@ prints only capture feedback.
 - **G5**: Given a capture with a project marker and adapter metadata, When
   `createCaptureRecord` builds it, Then it includes detected project/language,
   source metadata, and common fields (src/capture/index.ts).
+- **G6**: Given inline child stdout and a non-zero exit, When the wrapper
+  completes, Then stdout is forwarded and stored as response and the exit code
+  is retained (`src/runner/inline.test.ts`).
+- **G7**: Given analysis or remote sync is configured, When live capture enters
+  through log, inline, or PTY, Then all paths initialize and dispatch the same
+  lifecycle effects (`src/capture/lifecycle.ts`, SPEC-019).
 ## 8. Key implementation map
 | Concern | File(s) |