purecontext-mcp 1.1.1 → 1.1.2

package/docs/dev/PHASE14_TASKS.md

@@ -1,371 +0,0 @@
-# Phase 14 — Task Breakdown
-
-**Goal**: Fix PureContext's keyword search quality, which the Phase 14 competitive benchmark exposed as 0% Precision@1 and 0% Recall@5 against natural-language queries — while jcodemunch-mcp achieves 25% P@1 and 66.7% R@5 on the same task corpus. This phase closes that gap by wiring in the existing FTS5 infrastructure, adding a proper query preprocessor, and implementing a relevance re-ranking layer.
-
-**Scope rationale**: The benchmark revealed three root causes:
-
-| Root cause | Where | Impact |
-|---|---|---|
-| FTS5 table never populated | `index-manager.ts` never calls `bulkUpsertFtsSymbols` | No full-text index exists |
-| LIKE-only keyword search | `search-symbols.ts` uses `name LIKE '%query%'` | Multi-word queries find nothing |
-| No relevance ranking | Results sorted by name, not relevance | Wrong symbols ranked first |
-
-All three fixes are self-contained and carry no architectural risk — the FTS5 schema, `bulkUpsertFtsSymbols`, `ftsSearchSymbols`, and the FTS5 DDL already exist. This phase wires them together and adds the scoring layer.
-
-**Benchmark baseline (to beat):**
-
-| Metric | jcodemunch | PureContext before | PureContext target |
-|--------|-----------|-----------------|-----------------|
-| Search P@1 (keyword) | 25.0% | 0.0% | ≥ 50% |
-| Search R@5 (keyword) | 66.7% | 0.0% | ≥ 80% |
-| Token efficiency | 99.7% | 99.8% | ≥ 99.7% |
-
----
-
-## Task 108: FTS5 Population Pipeline
-
-Wire `bulkUpsertFtsSymbols` into the indexing pipeline so the FTS5 table is populated on every full index and incremental re-index.
-
-**Root cause**: `indexFolder` and `reindexFiles` in `src/core/index-manager.ts` insert symbols into the `symbols` table but never call `bulkUpsertFtsSymbols`. The `fts_symbols` table is therefore always empty for any repo indexed without the semantic path.
-
-**Deliverables:**
-
-- `src/core/index-manager.ts`:
-  - In the **full index path** (after `insertSymbols`): call `bulkUpsertFtsSymbols(db, repoId, allSymbols)` for the batch of newly inserted symbols.
-  - In the **incremental path** (`reindexFiles`): for each file being reindexed, call `deleteFtsByFile(db, repoId, filePath)` before re-inserting, then call `bulkUpsertFtsSymbols(db, repoId, newSymbols)` for the file's new symbols.
-  - When a file is deleted from the index, call `deleteFtsByFile` to remove stale FTS entries.
-
-- `src/core/db/symbol-store.ts`:
-  - Enrich the FTS content string: currently `${name} ${signature} ${summary}`. Extend to also include the file path stem (last path component without extension) so queries like "handler" find `typescript-handler.ts` symbols.
-    - New format: `${name} ${fileStem} ${signature} ${summary}`
-  - No schema change needed — content is a plain text column.
-
-- `test/core/fts-population.test.ts`:
-  - After a full index of `test/fixtures/basic-ts-project/`: verify `SELECT COUNT(*) FROM fts_symbols WHERE repo_id = ?` equals the symbol count.
-  - After an incremental re-index (modify one file): verify old FTS entries for that file are removed and new ones added.
-  - After a file deletion: verify its FTS entries are removed.
-
-**Key technical notes:**
-- FTS5 `INSERT` is fast (< 1ms per row typical). Even 5000 symbols adds < 50ms overhead.
-- The `bulkUpsertFtsSymbols` function already runs in a single transaction — no additional batching needed.
-- File stem in content is intentionally low-weight: it will rarely be the deciding factor but helps for queries like "discover files" → `file-discovery.ts`.
-
-**Verify:** Index the PureContext repo. `SELECT COUNT(*) FROM fts_symbols WHERE repo_id = ?` should equal `SELECT COUNT(*) FROM symbols WHERE repo_id = ?`.
-
-**Tests:** FTS population count matches symbol count. Incremental update replaces file entries. Deletion removes entries. File stem appears in FTS content.
-
----
-
-## Task 109: FTS5 Keyword Search Integration
-
-Replace the `name LIKE '%query%'` path in `search-symbols.ts` with `ftsSearchSymbols`, making multi-word natural-language queries work.
-
-**Root cause**: `search-symbols.ts` calls `searchSymbols(db, repoId, query)` which does:
-```sql
-SELECT * FROM symbols WHERE repo_id = ? AND name LIKE ?
-```
-with `%query%`. "orchestrate indexing pipeline" never matches any symbol name as a substring. `ftsSearchSymbols` uses FTS5 `MATCH` which tokenizes the query and finds documents containing those tokens anywhere in `name + fileStem + signature + summary`.
-
-**Deliverables:**
-
-- `src/server/tools/search-symbols.ts`:
-  - In the keyword mode path, replace `searchSymbols(db, ...)` with `ftsSearchSymbols(db, repoId, sanitizedQuery, limit)`.
-  - Keep `searchSymbols` (LIKE) as a fallback: if `ftsSearchSymbols` throws or the FTS table is empty (count = 0), fall back to LIKE. Log a warning when falling back.
-  - Add a `_meta.search_mode` field to the response: `'fts'` or `'like_fallback'` so callers can distinguish.
-
-- `src/core/db/symbol-store.ts` — `ftsSearchSymbols`:
-  - Apply `kind` filter by joining to the `symbols` table after FTS match (already does this via JOIN; add `AND s.kind = ?` when kind is provided).
-  - Apply `filePath` filter: `AND s.file_path LIKE ?` when filePath is provided.
-  - Use `bm25(fts_symbols)` for ordering instead of the default `rank` to get proper BM25 relevance ordering.
-
-- `src/core/db/symbol-store.ts` — empty FTS guard:
-  ```typescript
-  export function hasFtsIndex(db: Database.Database, repoId: string): boolean
-  ```
-  Returns true if `SELECT COUNT(*) FROM fts_symbols WHERE repo_id = ?` > 0. Used by search-symbols.ts to decide whether to use FTS or LIKE.
-
-- `test/server/search-fts.test.ts`:
-  - Fixture repo indexed → `search_symbols` with multi-word query returns expected symbol.
-  - Empty FTS table → falls back to LIKE, sets `search_mode: 'like_fallback'`.
-  - Kind filter applied correctly in FTS mode.
-  - FilePath filter applied correctly in FTS mode.
-
-**Key technical notes:**
-- FTS5 MATCH syntax: multi-word query `"orchestrate indexing pipeline"` → pass as `orchestrate indexing pipeline` (space-separated, FTS5 treats as implicit AND by default in SQLite's FTS5).
-- Special characters in user queries (`"`, `*`, `(`, `)`) must be escaped before passing to MATCH — handled in Task 110.
-- `bm25(fts_symbols)` returns a negative float; ORDER BY it ASC gives most-relevant first.
-
-**Verify:** Index the PureContext repo. `search_symbols("orchestrate indexing pipeline")` returns `indexFolder` in top 5.
-
-**Tests:** Multi-word query returns results. Single-word query still works. Kind filter works in FTS mode. Fallback triggers when FTS empty. bm25 ordering: exact name match ranked higher than summary-only match.
-
----
-
-## Task 110: Query Preprocessing & CamelCase Tokenization
-
-Add a query preprocessor that expands camelCase identifiers and normalizes queries before passing to FTS5, dramatically improving recall for code-specific queries.
-
-**Problem**: FTS5 tokenizes on whitespace and punctuation but not on camelCase boundaries. The query `parseFile` is one FTS5 token; it won't match a document containing `parse` and `file` separately. And a query like "parse source file" won't match the name `parseFile` unless it's in the signature/summary.
-
-**Deliverables:**
-
-- `src/core/search/query-preprocessor.ts` (new file):
-  ```typescript
-  export function preprocessQuery(raw: string): string
-  ```
-  Pipeline:
-  1. **Trim & lowercase** for comparison; keep original case for FTS (FTS5 is case-insensitive by default in SQLite).
-  2. **camelCase split**: `parseFile` → `parseFile OR parse OR file`. `HybridSearcher` → `HybridSearcher OR Hybrid OR Searcher`.
-  3. **snake_case split**: `get_symbol_source` → `get_symbol_source OR get OR symbol OR source`.
-  4. **Special char escape**: strip or escape `"()^*` that would break FTS5 MATCH.
-  5. **Short token filter**: drop tokens < 2 chars after splitting to avoid noise.
-  6. **Phrase handling**: if the raw query is a multi-word phrase (not a camelCase identifier), pass tokens as implicit AND: `orchestrate indexing pipeline` → `orchestrate indexing pipeline` (FTS5 already treats space as AND).
-  7. Return the preprocessed FTS5 query string.
-
-  Edge cases:
-  - All-caps acronyms (`MCP`, `HTTP`) → treat as single token, don't split.
-  - Numbers in identifiers (`phase14`, `v2`) → keep as-is.
-  - Already space-separated natural language → pass through (maybe with individual word expansion if words look like camelCase).
-
-- `src/server/tools/search-symbols.ts`:
-  - Apply `preprocessQuery(args.query)` before passing to `ftsSearchSymbols`.
-
-- `test/core/query-preprocessor.test.ts`:
-  - `parseFile` → includes `parse` and `file` in output.
-  - `HybridSearcher` → includes `hybrid` and `searcher`.
-  - `orchestrate indexing pipeline` → passes through unchanged (natural language).
-  - `get_symbol_source` → includes `get`, `symbol`, `source`.
-  - `"dangerous query"` → special chars escaped.
-  - Short tokens filtered: `a` dropped, `go` kept.
-
-**Key technical notes:**
-- The preprocessor produces an FTS5 query string, not a list of tokens. The format must be valid FTS5 MATCH syntax.
-- Use OR expansion (not AND) for camelCase splits so that a document matching any component is included — then the re-ranking layer (Task 111) promotes exact name matches.
-- This is stateless pure string transformation — no DB access, trivially unit testable.
-
-**Verify:** `search_symbols("parseFile")` returns `parseFile` in top 3. `search_symbols("indexFolder")` returns `indexFolder` at rank 1. `search_symbols("orchestrate indexing pipeline")` returns `indexFolder` in top 5.
-
-**Tests:** camelCase splitting, snake_case splitting, FTS5 escaping, natural language passthrough, short token filter. Integration: preprocessor hooked up in search-symbols.ts.
-
----
-
-## Task 111: Relevance Re-Ranking
-
-Add a post-FTS5 scoring layer that promotes exact and prefix name matches to the top, ensuring that when an agent queries `indexFolder` they get `indexFolder` at rank 1, not a summary-match ranked higher by BM25.
-
-**Problem**: BM25 ranks by term frequency across all FTS content fields. A symbol with "index" appearing 5 times in a long summary may outrank `indexFolder` whose name is an exact match. jcodemunch explicitly scores: exact name (20pts) > name contains (10pts) > name word overlap (5pts each) > signature match > summary match.
-
-**Deliverables:**
-
-- `src/core/search/relevance-ranker.ts` (new file):
-  ```typescript
-  export interface ScoredSymbol {
-    symbol: SymbolRecord;
-    score: number;
-    matchReason: 'exact_name' | 'prefix_name' | 'name_contains' | 'word_overlap' | 'content_match';
-  }
-
-  export function rankSymbols(symbols: SymbolRecord[], query: string): ScoredSymbol[]
-  ```
-
-  Scoring (additive, higher = more relevant):
-  | Rule | Points |
-  |------|--------|
-  | Exact name match (case-insensitive) | 100 |
-  | Name starts with query | 60 |
-  | Name contains query as substring | 40 |
-  | All query words appear in name | 30 |
-  | Any query word is exact name match | 20 |
-  | Any query word appears in name | 10 per word |
-  | Query phrase in signature | 8 |
-  | Any query word in signature | 2 per word |
-  | Query phrase in summary | 5 |
-  | Any query word in summary | 1 per word |
-
-  For camelCase queries, scoring runs against both the raw query and the split tokens.
-
-  Returns `ScoredSymbol[]` sorted by score descending; ties broken by FTS5 order (already BM25-ranked).
-
-- `src/server/tools/search-symbols.ts`:
-  - After `ftsSearchSymbols` returns results, apply `rankSymbols(results, args.query)`.
-  - Include `score` and `matchReason` in the response JSON for each symbol (helps agents understand why a result was returned).
-  - Apply ranking in LIKE fallback mode too (re-rank the LIKE results).
-
-- `test/core/relevance-ranker.test.ts`:
-  - `indexFolder` as query against a set containing `indexFolder`, `indexRepo`, `getBlastRadius` → `indexFolder` ranked 1st.
-  - Multi-word query "blast radius" → `getBlastRadius` (name words overlap) ranked above `buildGraph` (no overlap).
-  - Exact match always beats prefix which beats contains which beats word overlap.
-  - Tie-breaking: when scores equal, original FTS order preserved.
-
-**Key technical notes:**
-- Scoring is pure in-memory (no DB access). Input is the FTS5 result set (already filtered), typically ≤ 50 items.
-- The `matchReason` field documents the highest-scoring rule that fired (not all rules).
-- Including score in MCP responses is intentional — agents can use it to decide whether to fetch source for a result.
-
-**Verify:** `search_symbols("blast radius")` → `getBlastRadius` at rank 1, score ≥ 30. `search_symbols("indexFolder")` → `indexFolder` at rank 1 with matchReason `exact_name`.
-
-**Tests:** Scoring rules fire correctly. Ranking order matches expected. Multi-word queries. Tie-breaking. Score and matchReason in response.
-
----
-
-## Task 112: Symbol Count & IndexResult Stats Fix
-
-Fix the `getSymbolsByRepo` default-limit bug that causes the repos table to record a capped symbol count (1000) instead of the actual count.
-
-**Root cause (`src/core/index-manager.ts` line ~346):**
-```typescript
-const totalSymbols = getSymbolsByRepo(db, repoId).length;  // limit defaults to 1000
-```
-Any repo with > 1000 symbols stores 1000 in the repos table. The benchmark exposed this: a repo with 3832 symbols was reported as 1000.
-
-**Deliverables:**
-
-- `src/core/index-manager.ts`:
-  - Replace `getSymbolsByRepo(db, repoId).length` with a direct `COUNT(*)` query:
-    ```typescript
-    const totalSymbols = (
-      db.prepare<[string], { c: number }>('SELECT COUNT(*) AS c FROM symbols WHERE repo_id = ?')
-        .get(repoId)?.c ?? 0
-    );
-    ```
-  - Same fix for the `reindexFiles` path.
-
-- `src/core/types.ts` — `IndexResult`:
-  - Add `totalSymbolsInDb: number` — the COUNT(*) from the DB after indexing (not just this run's `symbolsFound`).
-  - Add `totalFilesInDb: number` — same for files.
-  - Keep existing `filesIndexed`, `symbolsFound` (incremental stats for this run).
-
-- `benchmarks/harness/run_purecontext.ts`:
-  - Remove the workaround `COUNT(*)` query added in the harness; use `indexResult.totalSymbolsInDb` instead.
-
-- `benchmarks/harness/compare.ts`:
-  - Fix "Winner" display for ties: when values are equal, show `—` instead of a tool name.
-
-- `test/core/index-stats.test.ts`:
-  - Index a fixture with > 10 symbols. Verify `IndexResult.totalSymbolsInDb` matches `SELECT COUNT(*) FROM symbols`.
-  - Verify repos table stores the correct count (not capped at 1000).
-  - Incremental re-index: verify totalSymbolsInDb reflects the full DB count, not just the files processed.
-
-**Key technical notes:**
-- This is a one-line fix but has visible impact on the benchmark and on any UI/tool that reads `repo.symbolCount`.
-- `IndexResult.symbolsFound` remains the incremental "how many new symbols did this run find" — useful for progress reporting. `totalSymbolsInDb` is the authoritative total.
-
-**Verify:** Index the PureContext repo (3800+ symbols). `list_repos` returns the correct symbol count. `IndexResult.totalSymbolsInDb` equals `SELECT COUNT(*) FROM symbols WHERE repo_id = ?`.
-
-**Tests:** Symbol count stored correctly for repos > 1000 symbols. `totalSymbolsInDb` vs `symbolsFound` semantics verified. Tie display in compare.ts.
-
----
-
-## Task 113: Ground Truth Expansion
-
-Expand the benchmark ground truth from 12 to 25+ tasks, add coverage for adapters and server tools, and re-run the benchmark to validate Phase 14 improvements.
-
-**Deliverables:**
-
-- `benchmarks/ground-truth.json`:
-  - Expand from 12 to 25+ tasks covering:
-    - **Core pipeline**: indexFolder, parseFile, discoverFiles, startWatching, buildGraph, getBlastRadius
-    - **Search layer**: searchSymbols, ftsSearchSymbols, HybridSearcher, rankSymbols (new)
-    - **Server tools**: handler functions in search-symbols, get-symbol-source, get-file-outline, find-dead-code
-    - **Adapters**: vuePreproccesor, expressAdapter, reactHandler (tests coverage of framework layer)
-    - **Config**: getConfig, loadConfig
-    - **Error types**: PureContextError, ParseError, StorageError
-  - Each task: `id`, `query` (natural language), `expected_symbol`, `expected_file`, `description`.
-  - Queries should represent realistic agent usage — not reverse-engineered from symbol names.
-
-- `benchmarks/harness/run_purecontext.ts`:
-  - After Phase 14 fixes, Dim 2 should report meaningful P@1 / R@5.
-  - Add `dim2.by_query` detail: for each task, show the actual top-5 names returned (aids debugging).
-  - Add latency stats: median and p95 query time across all Dim 2 tasks.
-
-- `benchmarks/harness/run_jmunch.py`:
-  - Mirror the `dim2.by_query` detail addition.
-
-- `benchmarks/harness/compare.ts`:
-  - In Dim 2 table: add "Top-5 results" column (expandable `<details>` per row) showing actual symbol names returned by each tool.
-  - Add Dim 2 improvement summary box showing delta vs Phase 14 baseline.
-
-- `benchmarks/results/comparison_v2.md`:
-  - Full benchmark re-run after all Phase 14 tasks complete.
-  - Expected improvement documented.
-
-**Key technical notes:**
-- Ground truth queries must be written before running the benchmark (not after looking at results) to avoid overfitting.
-- Negative precision is not tested in this phase — all 25+ tasks are positive (expected symbol should be found).
-- The `by_query` debug output in the JSON is diagnostic only; it shows what the tool actually returned, not what was expected.
-
-**Verify:** `npm run bench:purecontext` produces P@1 ≥ 50% and R@5 ≥ 80% on the expanded 25-task corpus.
-
-**Tests:** Ground truth file validates against JSON schema. All expected symbols exist in the indexed repo. Benchmark re-run completes without errors. Improvement over baseline documented.
-
----
-
-## Task 114: Phase 14 Integration Tests
-
-Validate all Phase 14 changes with focused integration tests that exercise the full search path from query string to ranked results.
-
-**Deliverables:**
-
-- `test/server/search-quality.test.ts`:
-  - Index the `test/fixtures/basic-ts-project/` fixture.
-  - For each of the 12 original ground-truth queries, call the `search_symbols` handler directly.
-  - Assert: expected symbol appears in top 5 results (R@5 gate).
-  - Assert: for the highest-quality queries (exact name fragments), symbol appears at rank 1 (P@1 gate).
-  - Assert: `_meta.search_mode` is `'fts'` (not `'like_fallback'`).
-  - Assert: each result includes `score` and `matchReason` fields.
-
-- `test/core/search-pipeline.test.ts`:
-  - End-to-end pipeline test: `preprocessQuery` → `ftsSearchSymbols` → `rankSymbols`.
-  - Test vectors covering: camelCase query, natural language query, snake_case query, single-word query, empty query (expect empty results, not error).
-
-- `test/core/fts-lifecycle.test.ts`:
-  - Full lifecycle: index → modify file → re-index → verify FTS reflects new symbols, old symbols removed.
-  - Delete file → verify FTS entries cleaned up.
-  - Empty query → safe (no FTS5 syntax error).
-
-- Regression: existing tests must still pass:
-  - `npm test` green.
-  - Token efficiency: Dim 1 benchmark should not regress below 99.5% reduction.
-
-**Key technical notes:**
-- The `basic-ts-project` fixture may need a few more symbols added to be useful as a search quality fixture — add 3–5 well-named functions with distinct summaries if coverage is thin.
-- P@1 gate in tests should be ≥ 8/12 (67%) — this is the minimum acceptable bar; Task 113's target is ≥ 50% on the harder 25-task corpus.
-- Integration tests are the acceptance criteria for this phase.
-
-**Verify:** `npm test` passes. Search quality tests achieve P@1 ≥ 67% on the 12-task fixture set. FTS lifecycle tests pass. Token efficiency not regressed.
-
----
-
-## Order of Execution
-
-```
-Task 108: FTS5 population pipeline          ████░░░░░░ Prerequisite (no FTS = nothing works)
-Task 109: FTS5 keyword integration          ██████░░░░ Core fix (wire FTS5 into search path)
-Task 110: Query preprocessing               ████████░░ Enhancement (camelCase, multi-word)
-Task 111: Relevance re-ranking              █████████░ Enhancement (exact match floats up)
-Task 112: Symbol count & stats fix          ██░░░░░░░░ Bug fix (parallel-able with 110/111) DONE
-Task 113: Ground truth expansion + re-run   ████████░░ Validation (after 108–112)
-Task 114: Integration tests                 ██████████ Gate (final acceptance)
-```
-
-**Critical path:** 108 → 109 → 110 → 111 → 113 → 114.
-Task 112 is independent and can be done in parallel with 110 or 111.
-Task 113 must run after all code changes are complete.
-
----
-
-## Phase 14 Completion
-
-With Phase 14 complete, PureContext's keyword search will be competitive with jcodemunch:
-
-**Before Phase 14:**
-- `searchSymbols`: LIKE substring match on name only. Multi-word queries return 0 results.
-- P@1: 0%, R@5: 0% on 12-task ground truth corpus.
-
-**After Phase 14:**
-- `ftsSearchSymbols`: FTS5 full-text match on name + fileStem + signature + summary, via BM25 ranking.
-- `preprocessQuery`: camelCase/snake_case expansion, FTS5 escaping.
-- `rankSymbols`: explicit relevance scoring (exact name > prefix > word overlap > content).
-- P@1 target: ≥ 50%, R@5 target: ≥ 80%.
-
-This makes PureContext a genuinely useful code navigation tool for natural-language AI agent queries, not just exact name lookups.

package/docs/dev/PHASE15_TASKS.md

@@ -1,256 +0,0 @@
-# Phase 15 — Task Breakdown
-
-**Goal**: Make PureContext viable for enterprise-scale repositories (10k–50k files) by eliminating three concrete bottlenecks in the indexing pipeline: redundant file I/O, redundant DB queries, and sequential single-threaded parsing.
-
-**Scope**: All changes are confined to `src/core/index-manager.ts` and new files under `src/core/`. No schema changes, no tool API changes, no adapter changes.
-
-**Baseline to beat:**
-The PureContext repo itself (~130 files, ~3800 symbols) indexes in ~400ms. A 10k-file enterprise repo indexing sequentially would take ~70s at ~7ms/file (1ms I/O + 5ms parse + 1ms extract). Target: ≤ 15s on the same corpus (4–5× speedup via parallelism + I/O fix).
-
----
-
-## Task 115: Fix Double File Reads
-
-**Problem**: `indexFolder` reads every changed file twice.
-
-- **Step 5** (line ~229): reads file to compute hash for change detection — then discards the `Buffer`, pushes only `relPath` to `toProcess`.
-- **Step 6** (line ~254): reads the same file again to parse it.
-- **Bonus**: `computeHash(content)` is also called a second time in step 6 (line ~280) for `upsertFile`. Hash was already computed in step 5.
-
-For a 10k-file repo with 5KB average file size, this is 100MB of redundant I/O and 10k redundant SHA-256 computations.
-
-**Deliverables:**
-
-- `src/core/index-manager.ts`:
-  - Change `toProcess` from `string[]` to `Array<{ relPath: string; content: Buffer; hash: string }>`.
-  - In step 5: populate `{ relPath, content, hash }` tuples (content already in memory, hash already computed).
-  - In step 6: use `entry.content` and `entry.hash` directly — no re-read, no re-hash.
-  - Remove the second `computeHash(content)` call (line ~280); use `entry.hash` in `upsertFile`.
-
-- `reindexFiles`: no change needed — it has no pre-pass hash step, so no double-read exists there.
-
-**Tests:**
-
-- `test/core/index-double-read.test.ts`:
-  - Spy on `readFileSync` (or `fs.readFile`). Index a fixture. Assert each file path was read exactly once.
-  - Assert `IndexResult.durationMs` is lower than a sequential double-read baseline (rough check via mocking with 1ms delay per read).
-
-**Verify:** `npm test` green. Index the PureContext repo — log shows same file count, symbol count as before.
-
----
-
-## Task 116: Consolidate Redundant `getSymbolsByRepo` Calls
-
-**Problem**: `indexFolder` calls `getSymbolsByRepo(db, repoId)` up to **three times** in the optional post-processing block:
-
-```typescript
-// Line ~303 — semantic indexer shouldIndex check
-const totalSoFar = getSymbolsByRepo(db, repoId).length;          // call 1
-if (options.semanticIndexer.shouldIndex(repoId, totalSoFar)) {
-  const allSymbols = getSymbolsByRepo(db, repoId);               // call 2
-  await options.semanticIndexer.index(repoId, allSymbols, db);
-}
-
-// Line ~326 — AI summarizer
-const candidateSymbols = getSymbolsByRepo(db, repoId);           // call 3
-```
-
-For a repo with 20k symbols, each call deserializes 20k rows. Three calls = 60k row deserializations that could be 20k.
-
-**Deliverables:**
-
-- `src/core/index-manager.ts`:
-  - Load `allSymbols` once, lazily, if either `semanticIndexer` or `aiSummarizer` is active:
-    ```typescript
-    const allSymbols =
-      options.semanticIndexer || options.aiSummarizer
-        ? getSymbolsByRepo(db, repoId)
-        : [];
-    ```
-  - Replace all three call sites with this single result:
-    - `shouldIndex(repoId, allSymbols.length)` — use `.length` on the already-loaded array.
-    - `semanticIndexer.index(repoId, allSymbols, db)` — pass `allSymbols` directly.
-    - `candidateSymbols` in the AI summarizer block → `allSymbols`.
-  - The lazy guard (`options.semanticIndexer || options.aiSummarizer`) ensures zero overhead when neither is enabled (the common case for most users).
-
-**Tests:**
-
-- `test/core/consolidate-queries.test.ts`:
-  - Spy on `getSymbolsByRepo`. Index a fixture with both `semanticIndexer` and `aiSummarizer` mocked. Assert `getSymbolsByRepo` called exactly once.
-  - With neither option: assert `getSymbolsByRepo` not called at all.
-  - With only `semanticIndexer`: assert called once and `shouldIndex` received correct count.
-
-**Verify:** `npm test` green. Behavior identical to before — just fewer DB round-trips.
-
----
-
-## Task 117: Worker Thread Pool for Parallel Parsing
-
-**Problem**: The parse loop in `indexFolder` (step 6, line ~250) processes files sequentially:
-
-```typescript
-for (const relPath of toProcess) {
-  const { symbols, imports } = await processFile(relPath, content, adapters);
-  // DB writes per file...
-}
-```
-
-`web-tree-sitter` is synchronous WASM. `await processFile` does not yield the event loop during parsing — it blocks. On a 16-core machine, only 1 core is used.
-
-**Architecture:**
-
-```
-Main thread
-├── Reads files (step 5, after Task 115: already has content)
-├── Creates WorkerPool(concurrency = min(cpuCount, 8))
-├── Dispatches {relPath, content (ArrayBuffer), adapterNames[]} to pool
-├── Collects all ParseResult[] back
-├── Single db.transaction() to write all symbols + imports
-└── Closes pool
-
-Worker thread (N instances)
-├── Lazy-initializes its own web-tree-sitter parser
-├── Instantiates adapters from adapterNames via adapter-registry
-├── processFile(relPath, content, adapters)
-└── Returns {relPath, symbols, imports} or {relPath, error}
-```
-
-**Key design decisions:**
-- `content` is transferred as `ArrayBuffer` (zero-copy via `Worker.postMessage(msg, [msg.content.buffer])`), not cloned.
-- Each worker has its own `Parser` instance — no shared state, no mutex.
-- DB writes happen **only on the main thread** in a single outer `db.transaction()` wrapping all file inserts. This replaces per-file transactions and is the second major speedup (especially for large repos on spinning disk).
-- `adapterNames: string[]` are serialized as plain strings. Each worker calls `getAdapterByName(name)` from the adapter registry to get the adapter instance.
-- Default concurrency: `Math.min(os.cpus().length, 8)`. Configurable via `IndexOptions.concurrency`.
-
-**Deliverables:**
-
-- `src/core/indexing-worker.ts` (new file):
-  - Worker entry point. Listens for `ParseJob` messages:
-    ```typescript
-    interface ParseJob {
-      relPath: string;
-      content: ArrayBuffer;   // transferable
-      adapterNames: string[];
-    }
-    interface ParseResult {
-      relPath: string;
-      symbols: SymbolRecord[];
-      imports: ImportRecord[];
-      error?: string;
-    }
-    ```
-  - Lazy-initializes parser on first message (`initParser()` once per worker).
-  - Calls `processFile(relPath, Buffer.from(content), adapters)`.
-  - Posts `ParseResult` back to main thread.
-  - Handles errors: catches `ParseError`, posts `{ relPath, symbols: [], imports: [], error: message }` — does not crash the worker.
-
-- `src/core/worker-pool.ts` (new file):
-  ```typescript
-  export interface WorkerPool {
-    run(jobs: ParseJob[]): Promise<ParseResult[]>;
-    terminate(): Promise<void>;
-  }
-  export function createWorkerPool(concurrency: number): WorkerPool
-  ```
-  - Creates `concurrency` Worker instances pointing at `indexing-worker.ts`.
-  - Maintains a job queue. Each idle worker pulls the next job.
-  - Returns a Promise that resolves when all jobs are complete.
-  - `terminate()`: sends `{ type: 'shutdown' }` to all workers, awaits their exit.
-
-- `src/core/index-manager.ts`:
-  - Add `concurrency?: number` to `IndexOptions` (default: `Math.min(os.cpus().length, 8)`).
-  - Replace the sequential step 6 loop with:
-    1. Build `ParseJob[]` from `toProcess` entries (using carried content from Task 115).
-    2. `const results = await pool.run(jobs)`.
-    3. Single `db.transaction()` that processes all `ParseResult[]`:
-       - For each result: `deleteByFile`, `insertSymbols`, `upsertFile` (batched).
-       - `allImports` accumulated from all results.
-  - Create pool before the loop, `terminate()` in a `finally` block.
-  - Log errors from workers alongside other per-file errors.
-
-- `src/core/db/symbol-store.ts` — optional micro-optimization:
-  - If `insertSymbols` is called once per file inside a loop that's already in a transaction, the inner `db.transaction()` becomes a savepoint (fine). No change required — outer transaction in index-manager is sufficient.
-
-- `IndexOptions` update in `src/core/types.ts`:
-  ```typescript
-  concurrency?: number;  // Worker thread count. Default: min(cpuCount, 8). Set to 1 to disable.
-  ```
-
-**Tests:**
-
-- `test/core/worker-pool.test.ts`:
-  - Pool with concurrency 2 processes 10 jobs in parallel (verify via timing: 10 jobs each taking 10ms should complete in ~50ms, not 100ms).
-  - Failed job (parse error) does not crash pool — error appears in results.
-  - `terminate()` cleans up all workers without hanging.
-
-- `test/core/parallel-indexing.test.ts`:
-  - Index `test/fixtures/basic-ts-project/` with `concurrency: 2`. Assert same symbol count as sequential path.
-  - Index with `concurrency: 1` (sequential mode). Assert identical results.
-  - Assert all symbols appear in DB after parallel index (regression against race conditions in DB writes).
-
-**Verify:**
-```bash
-# Time sequential (concurrency: 1) vs parallel (concurrency: 8) on a large repo
-node -e "
-  const { indexFolder } = await import('./dist/core/index-manager.js');
-  const r1 = await indexFolder('/path/to/large-repo', { concurrency: 1 });
-  const r2 = await indexFolder('/path/to/large-repo', { concurrency: 8 });
-  console.log('seq:', r1.durationMs, 'par:', r2.durationMs, 'speedup:', r1.durationMs / r2.durationMs);
-"
-```
-Expected: ≥ 3× speedup on repos with > 500 files.
-
----
-
-## Task 118: Performance Regression Tests
-
-Ensure the three fixes don't change observable behavior and lock in the performance improvements.
-
-**Deliverables:**
-
-- `test/core/indexing-correctness.test.ts`:
-  - Index `basic-ts-project` with all three optimizations active. Assert:
-    - Symbol count equals baseline (no symbols dropped or duplicated by parallel writes).
-    - Import edges count equals baseline.
-    - `IndexResult.totalSymbolsInDb` equals `SELECT COUNT(*) FROM symbols`.
-    - FTS table populated (count matches symbols table).
-  - Re-index same fixture (all files unchanged). Assert `filesIndexed = 0`, `filesSkipped = N`.
-
-- `test/core/performance-guard.test.ts`:
-  - Synthetic 200-file fixture (generated in `beforeAll` using a loop writing temp files).
-  - Index with `concurrency: 4`. Assert `durationMs < 5000` (generous upper bound, CI-safe).
-  - Assert no errors in `IndexResult.errors`.
-  - Cleanup temp files in `afterAll`.
-
-- `benchmarks/harness/run_purecontext.ts`:
-  - Add `Dimension 0 — Indexing Speed` to the benchmark output:
-    - Time `indexFolder` on the PureContext repo itself with `concurrency: 1` and `concurrency: auto`.
-    - Report: files/sec, symbols/sec, durationMs.
-
-**Verify:** `npm test` green. `npm run bench:purecontext` includes Dim 0 indexing speed numbers.
-
----
-
-## Order of Execution
-
-```
-Task 115: Fix double file reads          ██░░░░░░░░  Quick win, prerequisite for Task 117
-Task 116: Consolidate getSymbolsByRepo   ██░░░░░░░░  Independent quick win
-Task 117: Worker thread pool             ████████░░  Main feature (depends on 115 for content carry-through)
-Task 118: Performance regression tests   ██████████  Gate (after all code changes)
-```
-
-**Critical path:** 115 → 117 → 118. Task 116 is parallel with 115.
-
----
-
-## Phase 15 Completion
-
-**Before Phase 15 (10k-file repo estimate):**
-- Indexing: ~70s (sequential, double-read, triple DB load)
-- CPU utilization: 1 core
-
-**After Phase 15:**
-- Indexing: ~15s (8 workers, single-read, single DB transaction, one `getSymbolsByRepo`)
-- CPU utilization: up to 8 cores
-- Enterprise repos (50k files): ~75s vs ~350s — viable vs not viable