codetrap 0.1.6 → 0.1.8

package/README.md

@@ -53,18 +53,74 @@ codetrap list
 codetrap show 1
 ```
 
+## 5-Minute Agent Setup
+
+For a first AI-agent user, the fastest path is CLI-first project guidance:
+
+```bash
+# npm/source installs require Bun because the package entrypoint uses /usr/bin/env bun
+bun --version  # If this fails, install Bun first or use the binary install in docs/installation.md
+npm install -g codetrap
+
+# Initialize pitfall memory in the target project
+cd /path/to/project
+codetrap init
+codetrap doctor
+```
+
+Add the packaged agent guidance to `AGENTS.md` for Codex, or to `CLAUDE.md` for Claude Code:
+
+```bash
+cat "$(npm root -g)/codetrap/plugins/codetrap-agent/templates/AGENTS.codetrap.md" >> AGENTS.md
+# or:
+cat "$(npm root -g)/codetrap/plugins/codetrap-agent/templates/AGENTS.codetrap.md" >> CLAUDE.md
+```
+
+Then have the agent run this before non-trivial edits:
+
+```bash
+codetrap search "<task keywords>" --mode hybrid --json
+```
+
+Review the top 3 returned action cards, or all returned cards if fewer than 3, before deciding whether any trap applies. Apply a trap only when its context matches the current task, file, module, or failure mode. Severity alone is not enough to apply a trap. Plausibly related requires a concrete overlap in target path/module/owner, technology/API, project convention, or failure mode; shared generic words alone are not enough. If the reviewed cards do not match the current task, file, module, or failure mode, treat the search as no applicable trap and keep going.
+
+After a user correction, repeated test failure, or review finding, have the agent draft a candidate instead of writing confirmed memory:
+
+```bash
+cat <<'EOF' | codetrap session capture --trap-markdown - --kind review --json
+Title: <durable pitfall>
+Context: <when it triggers>
+Mistake: <what the agent did wrong>
+Fix: <what to do instead>
+EOF
+```
+
+Review pending candidates with `codetrap session status`, `codetrap session list`, `codetrap doctor`, or `codetrap web`; accepting a candidate remains an explicit human decision.
+
+Use the returned `candidate_id` and `session_id` to inspect and resolve the candidate:
+
+```bash
+codetrap session candidate <candidate-id> --session <session-id> --json
+
+# Only after explicit human approval:
+codetrap session accept <candidate-id> --session <session-id>
+
+# Or reject instead:
+codetrap session reject <candidate-id> --session <session-id> --reason "<reason>"
+```
+
 ## Features
 
 - **Structured trap recording** — title, category, context, mistake, fix, severity, tags, lifecycle, evidence, before/after code
 - **Session mode capture** — record implementation notes, promote explicit structured trap notes into candidates, and save only user-accepted lessons
 - **Dual scope** — project-scoped (`.codetrap/traps.db`) and global (`~/.codetrap/traps.db`)
 - **CLI-first agent API** — `search/show/list/stats/doctor --json` and stdin query support for shell-friendly automation
-- **Three search modes** — FTS (SQLite FTS5), semantic (Jina embeddings), hybrid (RRF fusion)
+- **Three search modes** — FTS (SQLite FTS5), semantic (local Ollama or Jina embeddings), hybrid (RRF fusion)
 - **Chinese + mixed-language search** — CJK bigram tokenizer, synonym map for Chinese-English terms
 - **MCP server** — optional tools + resources for AI agent integration
-- **Embedding cache** with freshness tracking — embeddings are rebuildable, stale ones auto-invalidated
+- **Embedding cache** with multi-profile freshness tracking — Jina/Ollama vectors can coexist and stale ones auto-invalidate
 - **Doctor diagnostics** — scope, database, and embedding health in text or JSON
-- **Schema migrations** — in-code migration system from v0 through current v5
+- **Schema migrations** — in-code migration system from v0 through current v6
 - **Single-binary builds** — `bun build --compile` produces standalone binaries in `dist/`
 
 ## Directory Structure
@@ -74,7 +130,7 @@ codetrap/
 ├── src/
 │   ├── index.ts              CLI entry point
 │   ├── mcp-server.ts         MCP server entry point
-│   ├── commands/router.ts    Thin CLI adapter + renderer
+│   ├── commands/router.ts    Optional thin CLI adapter + renderer
 │   ├── commands/workflow.ts  CLI command behavior
 │   ├── commands/command-result.ts  CLI command results + rendering
 │   ├── mcp/
@@ -87,9 +143,12 @@ codetrap/
 │   │   ├── store.ts          Project/global scope orchestration
 │   │   ├── trap-operations.ts Shared CLI/MCP operation semantics
 │   │   ├── session-operations.ts Session command semantics + accept/reject flow
+│   │   ├── session-review.ts Shared session review payloads + CLI conflict presenter
 │   │   ├── session-store.ts  Session files, active state, index, recaps
 │   │   ├── session-codec.ts  Session JSON/Markdown/candidate file conversion
-│   │   ├── session-capture.ts Candidate trap extraction from explicit structured notes
+│   │   ├── session-capture.ts Candidate draft normalization, extraction, and merge policy
+│   │   ├── session-candidate-document.ts Candidate document transition rules
+│   │   ├── session-candidate-scope.ts Candidate accepted-scope fallback
 │   │   ├── session-conflicts.ts Candidate vs active-trap conflict checks
 │   │   ├── trap-quality.ts   Deterministic candidate quality scoring
 │   │   ├── command-requests.ts CLI/MCP request normalization helpers
@@ -97,7 +156,9 @@ codetrap/
 │   │   ├── scope-context.ts  cwd/project/global DB context + repo selection
 │   │   ├── scope-migration.ts Safe project trap scope repair/migration
 │   │   ├── doctor.ts         Scope and embedding health diagnostics
+│   │   ├── embedding-runtime.ts Embedding provider runtime/config/status
 │   │   ├── embedding-health.ts  Fresh/stale/missing embedding summaries
+│   │   ├── embedding-management.ts Embedding profile command output
 │   │   ├── search-service.ts FTS/semantic/hybrid candidate retrieval
 │   │   ├── search-policy.ts  Applicability filtering, rerank, fusion signals
 │   │   ├── search-result-card.ts Compact agent-facing result cards
@@ -107,10 +168,13 @@ codetrap/
 │   │   ├── trap-json-fields.ts Tags/path/evidence JSON array codec
 │   │   ├── trap-codec.ts     Storage/JSON/archive/import shape conversion
 │   │   ├── trap-mutation-result.ts Mutation result + scope fallback semantics
+│   │   ├── string-list.ts    Shared string list de-duping
+│   │   ├── text-lines.ts     Shared line trimming helpers
+│   │   ├── value-types.ts    Shared runtime value guards
 │   │   ├── trap-scope-match.ts Path/module/owner applicability matching
 │   │   ├── trap-archive.ts   Import/export compatibility
 │   │   ├── trap-transfer.ts  DB-to-DB transfer for scope migration
-│   │   ├── embedder.ts       Jina Embeddings adapter
+│   │   ├── embedder.ts       Ollama and Jina Embeddings adapters
 │   │   ├── embedding-job.ts  Batch embedding generation
 │   │   ├── format.ts         CLI output formatting
 │   │   ├── scope.ts          Project root detection
@@ -121,6 +185,13 @@ codetrap/
 │   │   ├── queries.ts        CRUD, search, stats, import/export
 │   │   ├── embedding-queries.ts  Embedding storage SQL
 │   │   └── repository.ts     Single-DB facade
+│   ├── web/
+│   │   ├── server.ts         Thin Web API adapter over shared operations
+│   │   ├── static.ts         HTML/CSS shell
+│   │   ├── client-shell.ts   Pane sizing/collapse behavior
+│   │   ├── client-review.ts  Review queue + candidate draft/request model
+│   │   ├── client-script.ts  DOM composition and event wiring
+│   │   └── client-text.ts    Localized UI strings
 │   └── tests/
 │       ├── search-*.test.ts
 │       ├── trap-*.test.ts
@@ -130,8 +201,7 @@ codetrap/
 │       ├── scope-migration-cli.test.ts
 │       ├── import-export-cli.test.ts
 │       └── fixtures/search-eval.json
-├── skills/                   Agent skill definitions
-├── plugins/codetrap-agent/   Sample Codex plugin bundle
+├── plugins/codetrap-agent/   Codex plugin bundle with skills, MCP config, hooks, and templates
 ├── scripts/                  Release asset and preflight scripts
 ├── docs/                     Architecture + reference docs
 ├── package.json
@@ -144,7 +214,7 @@ codetrap/
 | Command | Description |
 |---|---|
 | `init` | Initialize `.codetrap/` in current project |
-| `add` | Record a new trap (`--json` structured input; interactive mode is not implemented) |
+| `add` | Record a confirmed trap (`--json` structured input; interactive mode is not implemented) |
 | `search <query>` | Search traps (--mode fts\|semantic\|hybrid, --category, --scope, --status, --limit, --path, --module, --owner, --no-rerank, --ranking-signals, --json; query can come from stdin) |
 | `list` | List traps (--category, --scope, --status, --path, --module, --owner, --limit, --json) |
 | `show <id>` | Show full trap details (--json) |
@@ -159,8 +229,10 @@ codetrap/
 | `doctor` | Diagnose cwd, scope, database paths, trap counts, and embedding health (--json) |
 | `repair-scope` | Move legacy mis-scoped project traps into the current project (dry-run by default, `--apply` to mutate, `--json`) |
 | `migrate-project` | Move project traps between initialized projects (`--from-project-path`, `--to-project-path`, dry-run by default, `--apply`, `--json`) |
-| `embed` | Generate embeddings (requires JINA_API_KEY) |
-| `session` | Start a development session, append notes, promote explicit structured trap notes into candidates, and accept/reject candidates |
+| `embed` | Generate embeddings (requires configured Ollama or Jina provider) |
+| `embeddings` | Manage embedding profiles (`status`, `list`, `use ollama|jina`, `reindex`) |
+| `session` | Start a development session, append notes, capture post-flight candidates, promote explicit structured trap notes into candidates, accept/reject candidates, and clean up session files |
+| `web` | Start the local review, trap library, insights, and Embeddings console |
 | `serve` | Start MCP server |
 
 ### Session Mode
@@ -170,15 +242,36 @@ Session mode stores temporary working memory in `.codetrap/sessions/`. It does n
 ```bash
 codetrap session start "implement agent harness" --spec docs/agent-harness-spec.md --module agent-runtime
 codetrap session note --kind decision --text "Defaulted tool calls to 30s because the spec does not define timeout behavior."
-codetrap session note --kind review --text $'Title: Do not parse nested tool calls with regex\nContext: When implementing parser logic for nested tool-call arguments.\nMistake: Using regex to split nested calls corrupts arguments.\nFix: Use a tokenizer/parser and add regression tests for nested calls.'
+cat <<'EOF' | codetrap session capture --trap-markdown - --kind review --json
+Title: Do not parse nested tool calls with regex
+Context: When implementing parser logic for nested tool-call arguments.
+Mistake: Using regex to split nested calls corrupts arguments.
+Fix: Use a tokenizer/parser and add regression tests for nested calls.
+Tags: parser, tool-calls
+Severity: error
+EOF
 codetrap session close --propose-traps
 codetrap session candidates
 codetrap session candidate cand-001
+
+# Only after explicit human approval:
 codetrap session accept cand-001
 ```
 
+`session capture` is the low-friction post-flight path: an agent drafts a structured Markdown or JSON candidate, codetrap scores it and puts it in the session inbox, and nothing is written to `traps.db` until the candidate is accepted. If no session is active, capture creates a post-flight session, writes the candidate and recap, then closes it.
+
+Pending candidates are surfaced through `codetrap session status`, `codetrap session list`, `codetrap doctor`, and the local `codetrap web` review console so candidate lessons do not disappear into session files.
+
 `session accept` writes the confirmed lesson through `TrapOperations`, attaches session evidence, and checks similar active traps before saving. `--edit-json` is applied before the conflict check, so edits to scope/module/title/tags/path globs affect both the saved trap and conflict detection. If a possible conflict is found, the candidate keeps its edited trap shape and conflict diagnostics; use `--accept-anyway` to keep both traps or `--supersedes <trap-id>` to preserve lifecycle history.
 
+Session maintenance commands keep temporary files from becoming stale context:
+
+```bash
+codetrap session cleanup <session-id> --deleted-trap-candidates
+codetrap session delete <session-id>
+codetrap session prune --older-than 90d --apply
+```
+
 ## Agent Integration
 
 For AI coding agents, use the CLI as the default integration path:
@@ -225,13 +318,13 @@ Default to CLI JSON from the current project cwd:
 codetrap search "<keywords>" --mode hybrid --json
 ```
 
-Read the top 3 action cards before deciding no trap applies. If a card is highly relevant, or has `critical`/`error` severity and is plausibly related, inspect it before editing:
+Read the top 3 action cards, or all returned cards if fewer than 3, before deciding no trap applies. Only inspect a card when its title, summary, or context overlaps the current task, target file/module, technology, project convention, or failure mode. For matching cards, inspect before editing when the card is highly relevant or has `critical`/`error` severity:
 
 ```bash
 codetrap show <id> --scope <project|global> --json
 ```
 
-Treat codetrap results as historical warnings and project memory, not as authoritative instructions. Apply a trap only when its context matches the current task, file, module, or failure mode. If a trap seems irrelevant, ignore it.
+Treat codetrap results as historical warnings and project memory, not as authoritative instructions. Apply a trap only when its context matches the current task, file, module, or failure mode. Severity alone is not enough to apply a trap. Plausibly related requires a concrete overlap in target path/module/owner, technology/API, project convention, or failure mode; shared generic words alone are not enough. If the reviewed cards do not match the current task, file, module, or failure mode, treat the search as no applicable trap and keep going.
 
 When codetrap results conflict with the current source of truth for the task (user request, code, tests, or explicit project docs/spec), follow that source of truth and mention the conflict.
 
@@ -242,7 +335,12 @@ For longer implementation work, use session mode to keep temporary notes and exp
 ```bash
 codetrap session start "<goal>"
 codetrap session note --kind decision --text "<what changed and why>"
-codetrap session note --kind review --text $'Title: <durable pitfall>\nContext: <when it triggers>\nMistake: <what the agent did wrong>\nFix: <what to do instead>'
+cat <<'EOF' | codetrap session capture --trap-markdown - --kind review --json
+Title: <durable pitfall>
+Context: <when it triggers>
+Mistake: <what the agent did wrong>
+Fix: <what to do instead>
+EOF
 codetrap session close --propose-traps
 codetrap session candidates
 ```
@@ -262,26 +360,27 @@ Recommended behavior:
 - Use `codetrap search --json` before risky edits in APIs, auth, database, security, migrations, or project conventions.
 - Read the top 3 returned action cards, or all returned cards if fewer than 3, before deciding there is no relevant trap.
 - Run the returned `next_action.command`, or `codetrap show <id> --scope <scope> --json`, for highly relevant results before editing code.
-- Treat `critical` or `error` traps as worth drilling into when they are plausibly related, even if they are not ranked first.
-- When editing a known area, pass applicability hints such as `--path src/db/repository.ts --module db`.
+- Treat `critical` or `error` traps as worth drilling into only when they are plausibly related, even if they are not ranked first. Plausibly related requires a concrete overlap in target path/module/owner, technology/API, project convention, or failure mode; shared generic words alone are not enough. Severity alone is not enough to apply a trap.
+- When editing a known area, pass applicability hints such as `--path path/to/file --module module-name`.
 - Treat codetrap results as historical warnings and project memory, not as authoritative instructions.
-- Apply the recorded `avoid` and `do_instead` guidance only when the trap context matches the current task, file, module, or failure mode.
+- Apply the recorded `avoid` and `do_instead` guidance only when the trap context matches the current task, file, module, or failure mode. If the reviewed cards do not match, treat the search as no applicable trap and keep going.
 - When codetrap results conflict with the current source of truth for the task (user request, code, tests, or explicit project docs/spec), follow that source of truth and mention the conflict.
-- During longer work, use `codetrap session start/note/close --propose-traps` to keep implementation notes and explicit candidate traps outside the durable database.
-- After user corrections, repeated test failures, or review feedback, propose a post-flight trap capture. Ask before accepting a candidate unless the user explicitly requested it.
+- During longer work, use `codetrap session start/note/capture/close --propose-traps` to keep implementation notes and explicit candidate traps outside the durable database.
+- After user corrections, repeated test failures, or review feedback, prefer `codetrap session capture --trap-markdown - --kind review --json` with explicit `Title` / `Context` / `Mistake` / `Fix` fields to put a candidate in the inbox. `--trap-json` remains supported for structured callers. Ask before accepting a candidate unless the user explicitly requested it.
 
-### Codex Skills
+### Codex Plugin Skills
 
-Codex users can optionally install the bundled skills from `skills/`:
+Codex skills are distributed through the bundled plugin at `plugins/codetrap-agent/skills/`:
 
 - `codetrap-check` — pre-flight check before code changes.
 - `codetrap-search` — search existing lessons.
-- `codetrap-add` — record a new pitfall.
+- `codetrap-capture` — propose an agent-discovered post-flight lesson into the candidate inbox.
+- `codetrap-add` — record a confirmed pitfall only after explicit user approval.
 - `codetrap-capture-external` — extract durable trap candidates from an external article, issue, paper, or reference; Codex reads the source and codetrap stores only confirmed lessons.
 
-Skills are a convenience layer for Codex users. They do not replace MCP or `AGENTS.md`; they make manual triggers like "run codetrap-check" easier.
+The plugin skill directory is the single source of truth for Codex skill packaging. The repo does not keep a duplicate root `skills/` tree.
 
-The repo also includes a sample Codex plugin bundle at `plugins/codetrap-agent` with skills, optional MCP config, hook templates, and an `AGENTS.md` snippet.
+Skills are a convenience layer for Codex users. They do not replace MCP or `AGENTS.md`; they make manual triggers like "run codetrap-check" easier.
 
 External lessons should keep codetrap local-first: let the agent read the URL or pasted source, ask which candidate traps to save, then attach the source as evidence instead of making the CLI crawl the web:
 
@@ -329,7 +428,11 @@ codetrap://project/recent?cwd=%2Fpath%2Fto%2Fproject
 
 | Env Variable | Required | Description |
 |---|---|---|
-| `JINA_API_KEY` | No | Jina AI API key for semantic/hybrid search. Without it, hybrid falls back to FTS. Get one at [jina.ai](https://jina.ai/api-dashboard/) |
+| `CODETRAP_EMBEDDING_PROVIDER` | No | Embedding provider for semantic/hybrid search: `ollama` or `jina`. Recommended local value: `ollama` |
+| `CODETRAP_OLLAMA_MODEL` | No | Ollama embedding model. Recommended: `qwen3-embedding:0.6b` |
+| `CODETRAP_OLLAMA_ENDPOINT` | No | Ollama endpoint. Default: `http://127.0.0.1:11434` |
+| `CODETRAP_OLLAMA_DIMENSIONS` | No | Ollama embedding dimensions. Default: `1024` for `qwen3-embedding:0.6b` |
+| `JINA_API_KEY` | No | Jina AI API key for optional cloud semantic/hybrid search. Get one at [jina.ai](https://jina.ai/api-dashboard/) |
 | `CODETRAP_SEARCH_MODE` | No | Default search mode: `fts`, `semantic`, or `hybrid` |
 | `CODETRAP_SEARCH_LIMIT` | No | Default search result limit |
 | `CODETRAP_SEARCH_SCOPE` | No | Default search scope: `project` or `global` |
@@ -344,6 +447,12 @@ Behavior preferences can also live in `~/.codetrap/config.json`; CLI args overri
     "limit": 20,
     "scope": "project",
     "rerank": true
+  },
+  "embeddings": {
+    "provider": "ollama",
+    "endpoint": "http://127.0.0.1:11434",
+    "model": "qwen3-embedding:0.6b",
+    "dimensions": 1024
   }
 }
 ```
@@ -364,61 +473,60 @@ Trap JSON supports optional applicability fields:
 
 Empty applicability fields mean the trap applies everywhere. `search` and `list` can filter with `--path`, `--module`, and `--owner`; matching scoped traps receive a small rerank boost.
 
-### Jina Embeddings Setup
+### Local Ollama Embeddings Setup
 
-codetrap works without a Jina API key. In that mode, search uses SQLite FTS keyword matching. If you want semantic search or stronger hybrid search, configure `JINA_API_KEY`.
+codetrap works with no embedding provider. In that mode, search uses local SQLite FTS keyword matching, and hybrid search falls back to FTS.
 
-1. Create a Jina API key from the [Jina AI dashboard](https://jina.ai/api-dashboard/).
+Recommended local semantic search uses Ollama with `qwen3-embedding:0.6b`. This keeps trap passages and query text on your machine.
 
-2. Add it to your shell environment.
-
-macOS or Linux with zsh:
+1. Install Ollama, then pull the 0.6B embedding model:
 
 ```bash
-echo 'export JINA_API_KEY="your-jina-api-key"' >> ~/.zshrc
-source ~/.zshrc
+ollama pull qwen3-embedding:0.6b
 ```
 
-macOS or Linux with bash:
+Do not omit `:0.6b`; `qwen3-embedding:latest` is much larger.
 
-```bash
-echo 'export JINA_API_KEY="your-jina-api-key"' >> ~/.bashrc
-source ~/.bashrc
-```
+2. Configure codetrap to use Ollama:
 
-Windows PowerShell:
-
-```powershell
-setx JINA_API_KEY "your-jina-api-key"
+```bash
+codetrap embeddings use ollama
+codetrap embeddings status
 ```
 
-After `setx`, open a new PowerShell window.
+This writes `~/.codetrap/config.json`. Environment variables such as `CODETRAP_EMBEDDING_PROVIDER` and `CODETRAP_OLLAMA_MODEL` are still supported for temporary shell or CI overrides.
 
-3. Verify that the key is visible without printing the secret:
+3. Verify Ollama embedding generation:
 
 ```bash
-bun -e 'console.log(process.env.JINA_API_KEY ? "has-key" : "no-key")'
+curl http://127.0.0.1:11434/api/embed -d '{"model":"qwen3-embedding:0.6b","input":"HTTP request timeout"}'
 ```
 
 4. Generate embeddings for the traps you want semantic search to use:
 
 ```bash
 cd /path/to/your/project
-codetrap embed --scope project
-codetrap embed --scope global
+codetrap embeddings reindex --scope project
+codetrap embeddings reindex --scope global
 ```
 
+`codetrap embed` remains as a short alias for reindexing. codetrap stores embeddings by profile, so switching between Jina and Ollama does not overwrite existing vectors; it creates or refreshes the selected profile.
+
+You can also run `codetrap web` and open the `Embeddings` view to inspect the active provider/profile, see project and global fresh/stale/missing counts, switch between Ollama and Jina, and reindex project or global embeddings from the web console. The web console does not save Jina API keys; Jina still reads `JINA_API_KEY` from the environment.
+
 5. Search with hybrid mode:
 
 ```bash
 codetrap search "HTTP request timeout" --mode hybrid
 ```
 
-If `JINA_API_KEY` is not set:
+Optional cloud provider: run `codetrap embeddings use jina` and set `JINA_API_KEY` to use Jina instead of Ollama. Privacy note: codetrap does not collect telemetry. FTS and Ollama search are local-only. When Jina is configured, reindexing sends trap passages to Jina, and semantic or hybrid search may send query text to Jina to compute embeddings.
+
+If no embedding provider is configured:
 
 - `codetrap search "<query>" --mode fts` works normally.
 - `codetrap search "<query>" --mode hybrid` works, but falls back to FTS.
-- `codetrap search "<query>" --mode semantic` and `codetrap embed` require `JINA_API_KEY`.
+- `codetrap search "<query>" --mode semantic` and `codetrap embed` require an embedding provider.
 
 ## Build
 
@@ -445,7 +553,7 @@ bun run eval:dogfood -- report --live  # Dogfood eval with configured embedding
 |---|---|
 | Runtime | Bun + TypeScript |
 | Database | SQLite (bun:sqlite) + FTS5 |
-| Embeddings | Jina AI (`jina-embeddings-v5-text-small`) |
+| Embeddings | Local Ollama (`qwen3-embedding:0.6b`) or Jina AI (`jina-embeddings-v5-text-small`) |
 | MCP | `@modelcontextprotocol/sdk` |
 | Search | FTS5 + cosine similarity + RRF fusion + generic rerank |