token-context-mcp 0.3.5__tar.gz

token_context_mcp-0.3.5/.gitignore

@@ -0,0 +1,38 @@
+.venv/
+__pycache__/
+.pytest_cache/
+.coverage
+coverage.xml
+/dist/
+/build/
+*.spec
+*.sqlite
+*.sqlite-shm
+*.sqlite-wal
+/indexes/
+/security/
+/.smoke/
+config/repos.toml
+config/indexes/
+__pycache__/
+.mcp.json
+
+# Agent session logs: contain source bodies from indexed repos, local paths and usernames
+evals/runs/
+
+# VS Code / Copilot MCP wiring: contains a machine-specific absolute path
+.vscode/mcp.json
+
+# Local planning and review notes; keep setup and benchmark result documents tracked.
+/docs/EXECUTION_PLAN.*.md
+/docs/REMEDIATION_AND_BENCHMARK_PLAN.*.md
+/docs/REVIEW_REMEDIATION.*.md
+
+# Superseded/duplicated documentation; SETUP is the canonical operational guide.
+/docs/ARCHITECTURE.md
+/docs/INTEGRATION.md
+/docs/PIPELINE_EXPLAINED*.md
+/tmp/
+
+# Launch campaign drafts and private notes (not published)
+/launch/

token_context_mcp-0.3.5/CHANGELOG.md

@@ -0,0 +1,294 @@
+# Changelog
+
+## 0.3.5 — docs & benchmarks: multilingual READMEs, Agentic Dark Green visuals, 59-task Head-to-Head harness (2026-10-26)
+
+**Upgrade:** no index rebuild needed. Documentation, packaging metadata, and benchmark reproduction suite update.
+
+Additions and improvements:
+
+- **Multilingual README suite (`README.md`, `README.en.md`, `README.cn.md`, `README.vi.md`)** — synchronized English, Simplified Chinese, and Vietnamese documentation with top-level language navigation and architecture/benchmark visual cards (`assets/cn/01..05`).
+- **59-task Head-to-Head benchmark harness (`launch/data/run_head_to_head.py`)** — reproducible `tiktoken` (`o200k_base`) evaluation across `bench_rich` (29 tasks) and `bench_fastify` (30 tasks) comparing interactive MCP pull (`search_source` + top-3 `get_symbol_context`) against static budget packing (`8k` / `32k`), alongside single-call `o200k_base` vs `bytes/4` cost audits (`launch/data/c1_o200k_summary.json`).
+- **Open-source launch & reproduction guides (`launch/`)** — added `launch/LINUXDO_POST_CN.md`, `launch/COMPETITIVE_DEEP_DIVE_CN.md`, and `launch/BENCHMARK_REPRO_GUIDE.md`.
+- **Package metadata (`pyproject.toml`)** — added `[project.urls]` (`Homepage`, `Repository`, `Documentation`, `Issues`, `Changelog`) and included `README.vi.md` in source distributions.
+
+
+## 0.3.4 — perf & robustness: AST byte-span elision, thinking-tag strip, TOML roundtrip preservation (2026-10-20)
+
+**Upgrade:** no index rebuild needed. All existing databases, caches, and configs remain 100% compatible.
+
+Enhancements and fixes:
+
+- **AST byte-span body elision (`sampling/skeleton_hybrid.py`)** — replaces regex line splitting with exact Tree-sitter byte ranges for Python, Java, TypeScript/JavaScript, Go, and C#. Class/method signatures and imports are preserved with 100% syntactic precision while irrelevant method bodies are cleanly elided. Falls back gracefully to regex on parse failure.
+- **Thinking LLM compatibility (`sampling/router.py`)** — automatically strips `<think>...</think>` reasoning tags before markdown fence removal and Pydantic validation, enabling seamless zero-crash integration with reasoning models (DeepSeek R1, Qwen-QwQ, etc.).
+- **TOML roundtrip preservation (`config.py`)** — `load_config` and `save_config` preserve unrecognized custom sections (e.g. `[plugins]`, `[harness]`) across repository updates and re-saves via `AppConfig.extra`.
+- **Dead code removal (`security/access_control.py`)** — removed unused `WRITE_OR_MUTATING_TOOLS` and verified complete classification of all registered tools against the access control plane.
+- **Golden contract test suite (`tests/golden/`)** — added comprehensive wire contract snapshots for `tools/list` across all 4 configurations (`core`, `+extensions`, `+admin`, `+extensions+admin`) and 2 schema profiles (`default`, `gemini_safe`), plus deterministic sanitized output regression snapshots.
+
+
+## 0.3.3 — fix: identity, governance, memory, windowed reads, Windows file locks (2026-10-03)
+
+**Upgrade:** no index rebuild needed (parser, FTS builder and resolver versions are unchanged). The memory FTS table is rebuilt lazily by the new query builder; existing memory entries stay readable.
+
+Behaviour changes (read before upgrading):
+
+- **Bound identity cannot be overridden** — when `TOKEN_CONTEXT_AGENT_ID` is set, an `agent_id` argument that differs from it is rejected (`agent_id does not match the identity bound to this server process`). Before, the argument silently replaced the bound identity, so any caller could pause-check, rate-limit or lock as someone else.
+- **`admin` is a reserved id** — it can no longer be self-declared or bound by env; only the internal `agent_control` path claims it.
+- **`anonymous` is rate limited** — the exemption is gone; it shares the default budget (120 calls/min) like any other id.
+- **`memory_lock` / `memory_unlock` use the resolved identity** — impersonating the holder to release or steal a lock no longer works.
+- **`memory_put` no longer uses `session_id` as the caller identity** (it only scopes the entry).
+- **`audit_logs`**: the `agent_id` argument is a filter only, not the caller identity.
+
+Fixes and additions:
+
+- **`agent_control(action="set_policy", agent_id, policy, custom_tools)`** — the `policy` argument was accepted but ignored; the action now exists (FULL_ACCESS / READ_ONLY / CUSTOM, CUSTOM needs a non-empty `custom_tools`; `admin` is protected).
+- **`memory_get(session_id=…)`** and **`memory_consolidate(namespace=…)`** — session-scoped entries are readable with the same `session_id` used to write them; consolidate prunes inside the namespace it read and reports a key as pruned only when a row was deleted (`delete` now returns whether anything was removed). Consolidate reads up to 500 entries (was the 100 oldest) and reports `source_entries_truncated`.
+- **`memory_consolidate(prune_transient=true)` deletes entries in every namespace** — deleting by the entry's own `scope` + `namespace` + `key` (FTS rows included) and listing only the keys removed.
+- **Tool catalog** — `get_module_dependents` now lists its `path` parameter next to `module` in `search_tools`/`list_available_tools` output, as the server already accepted both.
+- **Memory search** — `memory_fts` queries are phrase/token matches ranked by bm25 (identifiers with `_` or punctuation no longer fail or match loosely); expired rows are excluded in SQL and TTL cleanup also removes their FTS rows (orphans before).
+- **Windowed symbol bodies (`get_symbol_context(body_offset_line=N)`)** — a body that does not fit the budget is returned as a window (`body_window`: `start_line`, `end_line`, `total_lines`, `next_offset`) instead of being dropped; when the whole body is omitted the response carries `root_body_lines` and a paging hint. `inspect_symbol` output is unchanged.
+- **Windows file locks** — the index pointer swap, manifest replace, legacy-copy replace and snapshot move retry with back-off on `PermissionError` (WinError 32); a failed legacy copy is logged instead of swallowed.
+- **SQLite read pool** — connections owned by threads that have exited are closed when a new connection is registered.
+- **Docs** — Dynamic Tool Discovery and shared memory sections rewritten with real response shapes, catalog categories, measured sizes and comprehensive bilingual prompt templates.
+- **Tests** — `tests/test_guard.py` no longer pipes `git archive` through a shell (`git archive | tar` could hang on Windows); `tests/test_audit_fixes.py` (identity, governance, memory, pool, retry), `tests/test_memory_tools.py` (`test_memory_consolidate_prunes_namespaced_entries`) and `tests/test_parser_unicode_crlf_overloads.py` are new.
+
+Reported but not reproducible (guard tests added, no parser change): non-ASCII byte/char offsets (BUG-08), overloads sharing a symbol id (BUG-12), CRLF changing spans or signatures (BUG-14). On Python, JS, Java, C# and Go the spans, ids and bodies are correct and CRLF output equals LF output.
+
+Known limits: without `TOKEN_CONTEXT_AGENT_ID` the `agent_id` is self-declared, so pause/block/policy are advisory (bind the id per process for enforcement); `agent_control(action="status")` needs no admin token; `audit_logs` has no admin-token check; `tool_schemas_default.json` changed additively (`set_policy`, `custom_tools`, `body_offset_line`, `namespace`, `session_id`).
+
+## 0.3.2 — M13: Java and C# member resolution (overloads, overrides, packages, long identifiers) (2026-10-02)
+
+**Upgrade:** `PARSER_ARTIFACT_VERSION` 7 → 8, `RESOLVER_VERSION` 3 → 4, `FTS_BUILDER_VERSION` 2 → 3; the first `token-context index --all` after upgrading re-parses and re-indexes Java and C# files and rebuilds the search index. Python, JavaScript and TypeScript results are unchanged byte for byte (`rich`, `hono`, `fastify`, `tc-pinned` loc/edge gate: 0 differences).
+
+- **Typed member resolution for Java and C#** — the parser now records lexical scopes, argument types and receiver chains (`a.b().c`) per call site (`parse/jvm_types.py`, `parse/jvm_calls.py`). The resolver picks an overload by arity, then by argument types, and otherwise the first declared one; the nearest override wins; ancestors are searched level by level (classes and interfaces, methods before properties, delegate-typed C# properties); a sibling overload is preferred over recursion into the caller; extension methods, generic receivers and `Outer.Inner` qualified types are handled; receivers of known library types (`Map.Entry`, `List`, `String`, `Task`) are marked external instead of being guessed. `new X(...)` binds to the constructor or class (records included).
+- **Java packages** — `package` is extracted, and `same_package`, wildcard imports and narrowing use it instead of the directory (Maven `src/test/java` used to fall back to `global`).
+- **C# preprocessor** — files whose `#if/#else` branches break the parse are retried with the directives blanked or the `#else` branches pruned, keeping the variant with fewest syntax errors (Serilog `ILogger.cs`: 131 mangled symbols repaired).
+- **Search for long identifiers** — `FTS_BUILDER_VERSION` 3 splits C#/Java identifiers into sub-words and adds Porter stems to the search text, which is what the many-word names (`ConstructPositionalProperties`) and heavily overloaded names needed.
+- **Evidence (development sets, informative)** — unresolved ("ambiguous") edge share: CsvHelper 35 % → 23 %, jsoup 38 % → 16 %, Serilog 40 % → 16 %. Edge-gold recall at confidence ≥ 0.6: CsvHelper 0.40 → 0.80 (precision 1.00 → 0.80, two interface-dispatch calls counted wrong against implementation-labelled gold), jsoup 0.36 → 0.82 (precision 1.00), Serilog 0.76 → 0.94 (precision 1.00). R2 File Acc@5: CsvHelper 0.57 → 0.87, jsoup 0.67 → 0.83, Serilog 0.70 → 0.80.
+- **Evidence (fresh sets, measured once, not reviewed by the owner)** — gson (Java, 15 tasks, 12 call sites) and Newtonsoft.Json (C#, 15 tasks, 12 call sites): edge-gold recall 0.22 → 0.89 (gson) and 0.43 → 0.71 (Newtonsoft), but 2 of 10 resp. 2 of 7 edges at confidence ≥ 0.6 are wrong; R2 File Acc@5 0.80 → 0.93 and 0.73 → 0.80, Symbol Recall@10 0.59 → 0.66 and 0.52 → 0.66; all differences have wide intervals (n = 15). Details, misses and limits: `docs/BENCHMARK.md` (M13), `docs/progress/RUN_LEDGER_M13.md`, `evals/out/m13/`.
+- **Known limits** — type-parameter bounds, field chains, constructor chaining, generic element types, lambda parameter types and `await` unwrapping are not typed; same-named BCL extension methods (`Where`, `GetFields`) declared inside the repository resolve to the repository's copy; calls through a method returning a generic type (`delegate().write(...)`) stay unresolved.
+- **Harness** — `evals/guard.py` takes `TC_GUARD_MILESTONE` (default `m12`) so Rule 17 can check `m13-freeze` / `m13-base`; `evals/edge_gold_eval.py` treats `A.B` and `A.B.B` (constructor against class) as the same target; jsoup development sets (`bench_jsoup.json`, `edge_gold_jsoup.json`; three gold labels corrected after reading the source), fresh gson/Newtonsoft sets; tests `tests/test_m13_jvm.py` (typing, overloads, hierarchy, packages, preprocessor, I1 incremental = full).
+
+## 0.3.1 — fix: object-literal methods and chained assignments in JS/TS (2026-10-01)
+
+**Upgrade:** `PARSER_ARTIFACT_VERSION` 6 → 7; the first `token-context index --all` after upgrading re-parses JS/TS/TSX files (other languages produce the same artifact). Python results are unchanged.
+
+- **Regression introduced in 0.3.0 (M12.1), now fixed** — to avoid indexing a method twice, the parser skipped *every* `method_definition` inside an object literal. Methods of object literals that no pattern owns (an object passed as an argument such as `run("s", { test() {} })`, a returned object, a nested object, an object inside a function) therefore vanished from the index (0.3.0 holds 53 fewer symbols than 0.3.1 in hono, 195 fewer in fastify and 567 fewer in zod) and so did the edges that start in them. The generic path now skips only members that an owning pattern (`const x = {…}`, `X.prototype = {…}`, `module.exports = {…}`, `Object.defineProperty` descriptors) has already emitted. Tests: `tests/test_js_object_methods.py`.
+- **Chained assignments** — `res.set = res.header = function () {}` and `exports = module.exports = {…}` now bind the function (or the methods) to every left-hand side. 0.3.0 indexed none of them (K1 recall 0.933 on express, all misses chained); the same scan on 0.3.1 finds 90 of 90 assigned methods in express with 0 wrong in the 30-symbol sample (K1 met).
+- **Evidence** — 0.3.0 was measured on the held-out sets before this fix; those numbers belong to 0.3.0 (tag `m12-freeze`). 0.3.1 was re-measured on the development sets only, see `docs/BENCHMARK.md` (M12, "0.3.1 follow-up"). Rule 17's guard compares against the freeze tag and is expected to fail for this tree.
+
+## 0.3.0 — M12: JavaScript assigned methods, C# ranking, fewer ambiguous call edges (2026-10-01)
+
+**Upgrade:** the parser artifact (6), FTS builder (2) and edge resolver (3) versions changed; the first `token-context index --all` after upgrading re-parses and re-indexes every repository. Python results are unchanged byte for byte (`tc-pinned` loc/edge gate and `rich` bench: 0 differences).
+
+- **JS (M12.1)** — methods assigned through `X.prototype.m = function…`, `X.prototype = {…}`, `Object.defineProperty(X.prototype, …)`, `X.m = …`, `this.m = …` in constructor functions, `exports.m`/`module.exports.m` and module-level object literals are indexed as symbols (`docs/SYMBOL_NAMING.md`).
+- **C# (M12.2)** — implementation ranking over interface/abstract declarations, doc comments attached to the member instead of the container (`FTS_BUILDER_VERSION` 2) and interface doc inheritance.
+- **Call edges (M12.3)** — E1 implicit `this`, E2 overload resolution by arity/group, E3 C# namespaces, E4 local variable types, E5 fields without `this.`, E6 instantiation edges (`new X`, `new X()`), E7 missing C# call forms, E8 CommonJS module bindings, E10 TS property-signature field types (`RESOLVER_VERSION` 3).
+- **Fix after review of the M12 run** — the per-file wall-clock circuit breaker in the edge resolver made edge counts depend on machine load (the same repository indexed alone or with `--all` gave different graphs). It is replaced by a deterministic work budget (`FILE_EDGE_WORK_BUDGET`); `RESOLVER_VERSION` joins the parser fingerprint and the manifest, so a resolver change re-indexes (it did not before); the I1 test (incremental equals full) now also holds across a version bump.
+- **Harness** — Rule 17 guard hardened (`evals/guard.py`: held-out runs need tag `m12-freeze` and an empty diff on every protected path, or a baseline tree byte-identical to `m12-base`; real-git tests in `tests/test_guard.py`); `evals/edge_gold_eval.py`, `evals/sample_call_sites.py`; C3 v2 (`evals/run_c3*.py`, `c3_adapters.py`, `c3_grade.py`, `c3_locate_suite.py`): watchdog timeout, turn limit, case-preserving path grading, explicit dev/held-out suite roles; lint scripts honour `gold_pending_indexer`.
+- **Evidence** — dev before/after, held-out old/new, C3 with Claude Sonnet 5.5: `docs/BENCHMARK.md` (M12), `evals/out/m12/`. Four dev edge-gold labels for hono were wrong and were corrected (`evals/out/m12/edge_gold_hono_corrections.md`).
+
+## 0.2.0 — M6–M10: context packets, incremental index, client compatibility, GUI, Go (2026-09-29)
+
+Pushed to `main` on the owner's instruction; not tagged. Details are in the milestone sections below; the run ledger is `docs/progress/RUN_LEDGER_M6_M10.md`, the report `docs/reports/M6_M10_REPORT.vi.md`.
+
+- **M6** `inspect_symbol(view="full")` returns a context packet (`data.packet`: target body or truncated lines, callee/caller signatures, `more` refs, imports, class methods, file hashes) inside the response budget; `attr_param` receiver typing. Held-out, packet vs pre-M6 `full`: `evals/out/m6/packet_CMP_heldout_{2048,4096}.json`.
+- **M7** Schema **2.4** (re-index every repository once: `token-context index --all`); incremental index by `(size, mtime_ns)`, per-file parse artifacts, spawn parse pool, scoped edge resolution, delta snapshot write; cheap `get_index_status`; `index --all/--watch/--progress-format ndjson/--workers/--verify-hashes/--full`; `commit_sha` and `head_changed_since_index`. Measured gains and misses: `evals/out/m7/`.
+- **M9** `mcp_compat`, `serve --output-mode auto`, `serve --schema-profile gemini_safe`, real `get_tool_schema`, prompt-injection warnings inside each tool budget, `locate` profile 2048 tokens, heartbeat with client info, `docs/CLIENT_MATRIX.md`.
+- **M8** Desktop GUI: no I/O on the UI thread, child-process indexing with tree-kill Cancel, honest status badges, table model, running-server panel instead of Start/Stop, VACUUM only on mutable databases, stall watchdog.
+- **M10** Go grammar (`.go`; tree-sitter-go 0.25, `PARSER_ARTIFACT_VERSION = 2`: the first index after upgrading re-parses every file); one version source (`__version__`: pyproject, SBOM, `indexer_version`); `evals/bench_retrieval.py` and the public benchmark protocol in `docs/BENCHMARK.md`; first results on `rich` v15.0.0 with the owner-reviewed task set (`evals/out/m10/bench_rich_summary.json`): at equal cost R2 File Acc@5 0.90 against 0.57 for grep, R3 packets cover 98% of gold signatures/references with 91% fewer tokens than reading; graph expansion did not beat plain FTS (0.90 against 0.97).
+
+## Unreleased — M11: TypeScript benchmark (2026-09-29)
+
+- `evals/tasks/bench_hono.json` (hono v4.9.9, 30 locate + 10 packet tasks; authored by one Claude session, reviewed by another, not by the owner) and results in `evals/out/m11/`. Locate: R2 File Acc@5 0.80 against 0.23 for grep at equal cost (0.63 unbounded, 11x the tokens), Symbol Recall@10 0.66 against 0.13. Graph expansion beats plain FTS on symbols here (+0.11). **Packet KPIs missed on TypeScript**: sig/ref coverage 0.575 (targets 0.60/0.80), savings vs read 0.57 (target 0.70); coverage equals the reach ceiling, the index reports 45% ambiguous edges for hono. See `docs/BENCHMARK.md` (M11).
+- `evals/tasks/bench_fastify.json` (JavaScript) and `bench_csvhelper.json` (C#), 30 locate + 10 packet tasks each, authored and reviewed by separate Claude sessions (owner review pending); results in `evals/out/m11/`. File Acc@5 at equal cost: fastify R2 0.90 vs grep@R2 0.27 (unbounded grep 0.57); CsvHelper R2 0.60 vs 0.43 (unbounded grep 0.77, **R2 significantly worse**; behavioural queries ~0.1). Packet coverage 0.46 (JS) and 0.47 (C#), KPIs missed, tracking ambiguous edges (72 % / 90 %). JS indexer gap: prototype-assigned methods not indexed. Four-language table in `docs/BENCHMARK.md`.
+
+## Unreleased — M10: Go grammar, single version source, public benchmark harness (2026-09-29, branch `feat/m10-public-benchmark`)
+
+- **M10.2 Go (D8)** — chosen by data: the 18 real index snapshots hold 0 `unsupported` files among {go, rust, c/cpp, kotlin, php, ruby} (`evals/out/m10/real_unsupported_extensions.json`), so Go, as D8 prescribes. `SUPPORTED_EXTENSIONS[".go"] = "go"`, dependency `tree-sitter-go>=0.25.0,<0.26` (locked 0.25.0). Symbols: functions, methods (qualified `Type.Method`, pointer or value receiver, generic receivers), `struct`, `interface`, other `type` declarations and aliases; `is_private` is capitalisation; imports are the quoted paths (aliases and `_` imports included); calls are `name(...)` and `x.name(...)` and the receiver variable of a method carries the receiver type (`s.helper()` inside `func (s *Server)`). Edges are **name based** (no type inference for other receivers), so Go graphs are more ambiguous than Python's (gorilla/mux: 253 resolved, 395 ambiguous). `PARSER_ARTIFACT_VERSION` 1 → 2. Evidence: gorilla/mux @ `db9d1d0`, 17 `.go` files, parse_error rate 0% (`evals/out/m10/go_grammar_gomux.json`).
+- **Version** — `0.2.0` from `token_context_mcp.__version__` everywhere; `release.py` (SBOM) and `build_index` (`indexer_version`) no longer hard-code `0.1.0`; `tests/test_version_single_source.py`.
+- **`evals/bench_retrieval.py`** — deterministic arms R0-grep, R0-grep@R2 (same wire tokens as R2), R0-read, R1, R2 and R3 (packet vs reading), bootstrap CI95 (mean, seed 0), raw JSONL per arm plus a summary; refuses task sets that are not `reviewed: true`. Tested on fake data (`tests/test_bench_retrieval.py`). Frozen with the local tag `m10-freeze`.
+- **C3 on `bench-rich`** prepared (`evals/c3_prompts_rich.json`, `evals/c3_protocol_rich.md`); `run_c3_matrix --dry-run` lists all 45 runs (`evals/out/m10/c3_dry_run_rich.json`). Not executed (D6).
+- Regression gate (`tc-pinned`, loc A1final/A2 held-out @8192, `edge_eval`): identical, `evals/out/m10/regression_gate_m10.json`.
+
+## Unreleased — M8: the desktop GUI stops blocking (2026-09-29, branch `feat/m8-gui-offload`)
+
+- **M8.1/M8.2 `gui/workers.py::run_async(fn, *args, on_ok, on_err)`** on `QThreadPool.globalInstance()` (max 4 threads); callbacks are delivered on the UI thread through queued signals. Every tab has a pure `fetch()` (worker: reads only) and a `render()` (UI thread). `MainWindow._refresh_tab` runs them, shows `LoadingOverlay` until render completes, drops results of superseded refreshes; tabs no longer read anything in their constructors and the first refresh starts after the first paint.
+- **M8.3 Indexing in a child process** — `IndexProcess` (`QProcess`) runs `python -m token_context_mcp index --repo-id X | --all --progress-format ndjson`; progress reaches the UI at most 10×/s; Cancel kills the whole tree (`psutil` children recursively, then the process). New "Re-index all" button. `IndexWorker` is gone.
+- **M8.4 Log** — `setMaximumBlockCount(5000)`, appends batched every 100 ms.
+- **M8.5 Honest status** — `RepoManager.list_repositories` runs in a worker and uses `RetrievalService.status` (manifest aggregates). Badges `FRESH`, `STALE`, `DOCS_CHANGED` (only `changed_non_indexed`), `SCHEMA_OUTDATED`, `NOT_INDEXED`; the ambiguous rate is a 0–1 ratio everywhere and becomes a percentage only when displayed.
+- **M8.6 Repository table** — `QTableView` + `RepoTableModel` (`gui/models.py`), no `setCellWidget`; actions in a toolbar and a context menu.
+- **M8.7 Server panel (D2)** — Start/Stop/Restart and `ServerController.start_server/stop_server/restart_server` are removed. The dashboard lists running servers from `get_active_servers` (refreshed every 5 s in a worker). "Copy client config" for claude, claude-code, vscode, antigravity, codex with the flags of `docs/CLIENT_MATRIX.md` (`--output-mode`, `--schema-profile gemini_safe`).
+- **M8.8** `detect_ai_hardware()` runs at the start of `SystemMonitor.run()`; no `waitForStarted` on the UI thread.
+- **M8.9 VACUUM** — only `memory.sqlite`, `governance.sqlite`, `audit.sqlite` (never index snapshots); the user ticks the databases, it runs in a worker, `database is locked` is retried 3× (0.5 s backoff steps), the result of each database is reported. "Clean old snapshots" uses `gc_snapshots`.
+- **M8.10** `gui/perf.py::EventLoopWatchdog` (50 ms tick; with `TOKEN_CONTEXT_GUI_DEBUG=1` logs stalls >100 ms with faulthandler stacks) and `evals/gui_perf.py` (50 repos, tab switching, indexing synthetic-1k; report in `evals/out/m8/gui_perf.json`).
+- **Tests** — `tests/test_gui_m8.py` (27): run_async threading and errors, no SQLite/manifest/hardware/config I/O on the UI thread during refresh, badges end to end, subprocess index (single and `--all`), Cancel kills a grandchild, VACUUM never touches snapshots and retries locks, log cap, no start/stop API, client-config flags, hardware probe thread, watchdog. `tests/test_gui_widgets.py`/`test_gui_bridge.py` adapted to `fetch`/`render`.
+- Regression gate (`tc-pinned`, loc A1final/A2 held-out @8192, `edge_eval`): identical, `evals/out/m8/regression_gate_m8.json`.
+- Measured offscreen on the 2-core VM (`evals/out/m8/gui_perf.json`): 0 event-loop stalls >100 ms while indexing synthetic-1k through the GUI and while switching tabs; first paint ≈0.4 s; click→rendered with 50 repositories: tasks/cache/agents/settings 25–40 ms, dashboard ≈90 ms, repositories ≈126 ms (the UI thread itself stays free; the time is the worker's status reads) — the "<100 ms" soft KPI is missed for those two tabs.
+
+## Unreleased — M9: protocol and client compatibility (2026-09-29, branch `feat/m9-client-compat`)
+
+- **M9.1 SDK types** — `mcp_compat.py` is the only place that imports `CallToolResult`/`TextContent`; `serialization.py` and `server.py` use it (they imported from `mcp.types` and `mcp_types` respectively). `pyproject.toml`: `mcp>=2.0,<3`, `mcp-types>=2.0,<3`. Test: a result made by `ResultFinalizer` is an instance of the class the server returns.
+- **M9.2 Output mode per client (D11)** — new mode `auto` (config validation and the GUI combo accept it) and `serve --output-mode {auto,structured,text,legacy_dual}`; precedence flag > config > default. **The default stays `structured`.** `auto` = `structured` only when the client name is in `STRUCTURED_OK_CLIENTS` (`client_profile.py`, currently `claude-code`), otherwise `text`; no client info → `text`; never `legacy_dual`. The effective mode is logged at start-up and written to the heartbeat. `evals/stdio_smoke.py` now runs in `structured` and `text` and checks where the payload arrives.
+- **M9.3 Schema profiles** — `serve --schema-profile {auto,default,gemini_safe}`. Applied to `tools/list` only (server middleware rewriting the result; no site-packages edits). `gemini_safe`: no `null` branch / `default: null`, `type` arrays → non-null type, `$ref`/`$defs` inlined, arrays always have `items`, every property has a type; `memory_put.value` is declared `string` and, only in this profile, `json.loads`-ed on the server (a parse error keeps the string). `auto` → `gemini_safe` when the client name contains `gemini` or `antigravity`. `default` is byte-identical to before (snapshot `tests/fixtures/tool_schemas_default.json`, 22 tools). `evals/schema_compat.py --schema-profile gemini_safe`: 0 flags on all tools (default profile: 43 flags, `evals/out/m9/schema_compat_*.json`; static `schemas/*.json` files are documentation, not advertised, and are skipped in a profile audit).
+- **M9.4** `get_tool_schema` returns the registered input schema (after the active profile) instead of `{}`.
+- **M9.5 Tool descriptions** — `inspect_symbol` now says what `view='full'` returns and when to use it; `sample_summarize` no longer opens with "Hardware-aware"; `memory_consolidate` lost the "(learned from Google …)" aside. Test: every description ≤ 200 characters; catalog still matches the registered tools.
+- **M9.6 Prompt-injection warning** — `content_policy.scan_prompt_injection(text) -> list[int]` (line numbers) with the six patterns of the plan, case-insensitive. `search_source`, `get_symbol_context`, `get_file_skeleton`, `inspect_symbol` (`normal`, `full`) always set `untrusted_repository_content: true` in the envelope and, on a hit, add the warning `possible_prompt_injection` and `data.injection_hits` (`path:line`, ≤ 10). Nothing is redacted. The fields are inside each tool's measured budget (`inspect_symbol full` fixed-point loop; `normal` reserves them from the relationship share). **Contract note**: the `normal` view snapshots (20 of 40 files in `tests/fixtures/inspect_symbol_snapshots`) changed by exactly two lines — the new flag and `estimated_tokens`; `minimal` is byte-identical (it carries no repository text). Measured effect on `search_source` @2.048 over the 10 dev tasks: mean matches 13.2 → 13.2, mean tokens 1894 → 1903 (`evals/out/m9/inj_budget_{pre,post}.json`). Scan of the repository: 0 hits in tracked source; the only hits are the fixtures of this feature's own tests (11 lines in `tests/test_m9_client_compat.py`), as expected.
+- **M9.7 `locate` profile** — measured on the 10 dev tasks (`evals/locate_profile_probe.py`, `evals/out/m9/locate_probe.json`): shipped 1.024 tokens → mean 3.6 matches (min 3), File Acc@5 0.90, 862 tokens; (a) one snippet line per match → 3.9 matches, 0.90, 846 tokens; (b) budget 2.048 → 9.4 matches, 0.90, 1886 tokens. (a) does not reach 5 matches (the evidence object, not the snippet, dominates a match); (b) does, and costs about 200 tokens per match against about 240. **Decision: (b)**, `locate.budget_tokens = 2048` (callers can still pass `max_tokens=1024`); `snippet_lines` (1 or 2, default 2) stays available as a profile field. No match field was removed. `tests/test_index_retrieval.py` now expects the 2.048 budget.
+- **M9.8 Heartbeat** — `server_heartbeats` gains nullable `client_name`, `client_version`, `output_mode`, `schema_profile` (guarded `ALTER TABLE`, idempotent; a later heartbeat without the info keeps what was learned). Test: an old-schema database migrates and a pre-M9 `SELECT server_id, pid, last_seen, status …` still works.
+- **M9.9 Docs** — new `docs/CLIENT_MATRIX.md` (the check, one row per client, only real results); `docs/SETUP.*` §5.7 (flags, session-restart note, spec 2026-07-28 note, untrusted-content note), Antigravity entry uses `--schema-profile gemini_safe`, tool counts corrected (10 core, 20 with extensions, 22 with admin tools) in SETUP en/vi and README.
+- Regression gate (`tc-pinned`, loc A1final/A2 held-out @8192, `edge_eval`): see `evals/out/m9/regression_gate_m9.json`.
+
+## Unreleased — M7: incremental, parallel, commit-aware index — schema 2.4 (2026-09-29, branch `feat/m7-incremental-index`)
+
+- **Schema 2.4 (`INDEX_SCHEMA_VERSION = "2.4"`)** — new table `file_parse_artifacts(path, sha256, parser_version, language, calls_json, inheritance_json, imports_json, warnings_json, facts_json)`: the parse result of every file, keyed by content hash, is stored in the snapshot so the next run can skip parsing. **The first `index` after upgrading re-parses every file** (snapshots 2.1–2.3 have no artifacts; they are never used as a base and get a full rebuild). Older snapshots still open for reading and `get_index_status` warns to re-index.
+- **`PARSER_ARTIFACT_VERSION = 1` (`parse/treesitter.py`)** — bump it whenever the parser output for the same bytes can change (extraction rules, `CallRecord`/`ImportRecord` layout). `tests/test_parser_artifact_version.py` pins a golden hash of the parser source and the record layout and fails until the version is bumped. The tree-sitter package versions are also stored as `parser_fingerprint`; a different fingerprint forces a full re-parse.
+- **Incremental build (`index/runner.py`)** — files are classified by `(size, mtime_ns)` against the active snapshot: unchanged files are neither read nor hashed nor parsed; touched files with an identical hash are hashed but not parsed; only changed/new files go through `parse_source`. Deleted files disappear from every table. `--verify-hashes` reads and hashes every file, `--full` ignores the active snapshot. **Known limitation**: a same-size edit that keeps the mtime and is older than the previous scan start (racy window 2 s) is not detected without `--verify-hashes`.
+- **Parallel parse** — `ProcessPoolExecutor` with the `spawn` start method, `--workers N` / `TOKEN_CONTEXT_INDEX_WORKERS` (default `max(1, min(8, CPUs-1))`). Deviation from the plan, measured: spawn start-up costs ~0.8 s on the 2-core dev VM, so the pool is used only for ≥32 candidate files **and** ≥1 MB of source (`PARSE_POOL_MIN_FILES/BYTES`); smaller batches, and any pool failure (`BrokenProcessPool`, pickling, OS errors), fall back to the sequential path with identical output. SIGTERM/SIGBREAK/Ctrl-C terminate and reap the workers (test: no orphans).
+- **Scoped edge resolution** — edges of unchanged files are reused unless the class hierarchy / stubs fingerprint (`edge_context_sha256`) changed or the file mentions a symbol whose (qualified name, kind, signature) changed; ids of re-parsed files are remapped by `(qualified_name, kind)`. Any inconsistency falls back to re-resolving the file. Edge order is identical to a full build. `lexical_edges` keeps its 30 ms per-file circuit breaker in normal runs (the equivalence tool disables it so timing cannot change results).
+- **Write path** — no `VACUUM`; the legacy `<repo>.sqlite` copy is a hard link (copy fallback); when only a few files changed the new snapshot is a copy of the previous one with the changed files' FTS rows replaced (`write_mode = "delta"`, capped at 25 generations or 25 % free pages, otherwise `"rewrite"`).
+- **Performance work without behaviour change** — roles in O(n) (children index, lazy source reads), own-body via bisect, ranking loop hoisting (bit-identical), module-candidate cache. Byte-identical index vs. the pre-M7 code on `tc-pinned` (except the new table / schema metadata).
+- **Cheap `get_index_status`** — reads aggregates from the manifest (`symbols_with_roles`, `role_counts`, `edge_precision`, `import_count`, `importer_count`) instead of scanning tables (older snapshots fall back to the scan). New fields `commit_sha` (manifest, from `git rev-parse HEAD` with a 2 s timeout, `null` outside git) and `head_changed_since_index` (`true/false/null`, 5 s cache).
+- **CLI** — `index --repo-id X | --all` (mutually exclusive; `--all` prints a JSON summary `{repositories, indexed, failed}` and exits 1 when any repository failed), `--progress-format ndjson` (one JSON object per line on stdout, ≤10/s, final `{"stage":"done","manifest":...}`; logs stay on stderr; `--all` lines carry `repo_id` and end with `all_done`), `--workers`, `--verify-hashes`, `--full`, `--watch [--debounce S] [--poll-interval S]` (re-index after the tree has been quiet for the debounce; uses `watchdog` when installed — optional extra `watch` — otherwise polls; ignores `.gitignore`d and vendored paths).
+- **Evidence** — `evals/index_equivalence.py` (invariant I1: incremental == from-scratch after 5 chained edits — body edit, rename with cross-file caller, add, delete, import change — with exact `parse_source` counts 1/2/1/0/1; passes on `tc-pinned` and `synthetic-1k`), `evals/index_bench.py`, `evals/out/m7/`. Regression gate (loc A1final/A2 held-out @8192 + `edge_eval`, `tc-pinned` re-indexed): identical to baseline.
+- **Measured on the 2-core dev VM (median of 3)**: `tc-pinned` (137 code files) full 1.99 s → 2.09 s (parse-bound, no gain on 2 cores), no-op 2.26 s → 0.42 s, one file 2.22 s → 0.45 s; `synthetic-1k` (1050 files) full 8.28 s → 5.23 s (−37 %), no-op 0.77 s (15 % of full), one file 0.81 s; tracemalloc peak 44.9 → 36.4 MB (full); tree RSS 62–66 → ~70 MB full / 56 MB no-op; `get_index_status` p50 97 ms → 3.6 ms. Soft KPIs not met on this VM: `tc-pinned` full speed-up, full-index −40 % on synthetic-1k (−37 %), tree RSS −30 %; see `docs/reports/M6_M10_REPORT.vi.md`.
+
+## Unreleased — M6 Session 2: context packet in `inspect_symbol(view="full")` (2026-09-29, branch `feat/m6s2-packet`)
+
+- **Contract change (D9, approved by the owner)** — `inspect_symbol(view="full")` now returns a context packet. `minimal` and `normal` are unchanged (guarded by 40 JSON snapshots captured before any packet code, `tests/test_inspect_symbol_snapshots.py`). `full` returns:
+  - `data.symbol` (same fields as `normal`, no `content`), `data.packet`, `data.relationship_count` (filtered 1-hop relations, callee + caller, de-duplicated) and `data.relationships_filtered` (E14 meaning: edges excluded by the quality filter).
+  - `data.content` and `data.relationships` are **removed** from `full` (both live inside the packet; keeping them would count every token twice).
+  - `data.packet = {target, callees, callers, more, context, files, used_tokens, omitted}`:
+    - `target`: `{symbol_id (ref8), span, content, truncated_lines}`. A function/method body is complete when it fits; otherwise the kept lines are prefixed `"<line>: "`, always include the signature, the first docstring line and every line that calls a retained callee, and `truncated_lines` lists the omitted ranges. A class target is a skeleton (header + method signatures), not a body.
+    - `callees` / `callers`: `[ref8, "path:line", "<signature>", confidence]`; callees ordered by first call in the target body, callers regular code before `evals/` before `tests/` (then global rank, path, line), at most K per response. A constructor call resolves to the class, and the row carries the `__init__` signature (header when there is none).
+    - `more`: every remaining 1-hop relation as `[ref8, "path:line", "f name", "callee|caller|test"]`. `context`: `imports` used by the target and `class_methods` (sibling signatures). `files`: `path -> sha256[:12]` written once. `omitted`: rows shed for budget (`callees`, `callers`, `more`, `class_methods`, `imports`).
+    - Edge filter shared by the packet and every metric: callee edges satisfy `edge_is_traversable(e, 0.6)`, caller edges `edge_is_traversable(e, 0.5)`.
+    - `ref8` (first 8 hex of sha256(symbol_id)) is accepted as `symbol_id` by `get_symbol_context` and `get_impact_slice` (tested).
+    - A neighbour file that changed after indexing keeps its row and ref; the signature is `null` and `stale_content_unavailable` is added to `warnings`.
+- **Budget contract** — the response is built with a fixed-point loop: `_payload_tokens(response) <= _effective_budget(budget_tokens)` on the *whole* response (envelope included), `budget.estimated_tokens` is the measured number (no more `min(budget, …)` clamp), and a too-large response sheds `more -> context -> callers -> callees -> target lines`. Tested at 1.024 / 2.048 / 4.096 for short and >300-line targets, also through `server._result` in `structured` and `text` output modes.
+- **Algorithm (`retrieve/packet.py`, `retrieve/packet_inputs.py`)**: `RetrievalService.packet_inputs` gathers the material once per request (no budget); `build_context_packet` is a pure, deterministic function (same inputs -> same bytes). Pass 1 fills `target -> callees -> callers -> more -> context` by share with carry-over of unused bytes; pass 2 gives the remainder to sections that were still cut, in the same order.
+- **Frozen constants (tuned on the 8 dev tasks only, 24-config grid in `evals/out/m6/packet_tuning_dev_grid.json`, commit `freeze packet constants`)**: shares `target 0.30 / callees 0.25 / callers 0.25 / more 0.08 / context 0.12`, `CALLERS_K = 15` (initial values were 0.45 / 0.20 / 0.12 / 0.10 / 0.13 and K = 5).
+- **Evals**: new `evals/packet_eval.py` + `evals/packet_driver.py` (arms P0 = pre-M6 code `bb60f6e` on the same index, P1, READ, CMP; `--repo-id tc-pinned`), and `evals/bench_latency.py --inspect-tasks … --inspect-view full --inspect-budget 4096`. Metrics fixed before measuring: `sig_coverage` (primary), `ref_coverage`, `body_coverage` (report only: HCP packets carry no neighbour bodies), `reach_ceiling`, `wire_tokens`, `savings_vs_read`.
+  - Held-out (12 tasks, run once after the freeze), P1 vs P0: `ref_coverage` 1.000 vs 0.435 @4.096 and 0.824 vs 0.435 @2.048; `sig_coverage` 0.958 @4.096 and 0.725 @2.048 (P0 has no neighbour signatures); mean wire tokens 2.932 vs 2.622 @4.096 and 1.740 vs 1.602 @2.048; savings vs READ 92.3% / 95.1%; `reach_ceiling` 1.00. Dev with the frozen constants: `sig_coverage` 1.000 @4.096 and 0.825 @2.048.
+  - Latency (`inspect_symbol full@4096`, 20 targets, M5 protocol, VM, not noisy): 9.8 ms per call vs 5.3 ms for the pre-M6 `full` (`evals/out/m6/latency_m6s2.json`, `latency_m6s2_p0_full.json`).
+- **Regression gate (`tc-pinned`, re-indexed with this code)**: `loc_eval` A1final and A2 held-out @8.192 and `edge_eval` are identical to the baseline task by task (`evals/out/m6/regression_gate_m6s2.json`); @2.048 also identical. Full pytest: 359 collected (was 288), all passing, 1 pre-existing skip; `stdio_smoke` and the clean-room wheel pass.
+- Tests changed with the contract: `test_inspect_symbol_full_relationships_match_normal` (ids of the packet contain the ids of `normal`'s relationships), `test_token_efficiency` and `test_line_ending_normalization` (body now under `data.packet.target.content`). New: `tests/test_context_packet.py` (31 tests).
+
+## Unreleased — M6 Session 1: `attr_param` recall + `inspect_symbol` relationship contract fix (2026-09-27, branch `feat/m6-hcp-packet`)
+
+- **M6.0 — typed-parameter → self-attribute recall (`parse/treesitter.py`, `parse/lexical_edges.py`)**:
+  - `_extract_class_attributes` now also resolves `self.x = <param>` when `<param>` carries a type annotation on the enclosing method — both `typed_parameter` and `typed_default_parameter` AST nodes (e.g. constructor dependency injection: `def __init__(self, service: RetrievalService): self.service = service`). This was the single largest source of `unresolved_receiver` edges in the repo: the pattern is used throughout the GUI controllers and across `retrieve/workflows.py`.
+  - Container-typed parameters (`dict[...]`, `list[...]`, …) are skipped, and an attribute assigned from parameters of two different types across methods stays tainted/unresolved — same rules M4.3 already applied to `self.x = Cls(...)`.
+  - New edge scope `attr_param`, confidence 0.90 (provisional, uncalibrated: n=25 at introduction — see `evals/out/m6/edge_audit_after_m6.json`'s `scope_counts`).
+  - Effect on the real repo, isolated via `git stash`/`sync_index.sh`/eval so only this diff differs between the two measurements (`evals/out/m6/edge_audit_before_m6.json` vs `edge_audit_after_m6.json`): `scope:unresolved_receiver` 300 → 280 (-20), `scope:attr_param` 0 → 25 (new), `scope:attr_type` 64 → 65 (stable, no cannibalization), 0 active false positives before and after (`evals/out/m6/edge_eval_{before,after}_m6.json`, Part A `active_false_positives`). The 25 newly-resolved call sites (source path:line → target), including the motivating `CompositeWorkflowEngine._inspect_symbol_scoped` → `RetrievalService.find_symbols`/`symbol_context`/`symbol_relationships` example, are listed in full in `evals/out/m6/attr_param_new_edges_m6.json`.
+  - Note: the gold-set-relative Part A/B numbers in those same eval files (accuracy ~57-58%, Part B recall 30%) read far below the M4-era baseline (84.67%/88.33%) — confirmed pre-existing and orthogonal to this change (identical in both before/after measurements): `evals/gold/edges_token_context.json` is keyed by exact `source_symbol_id`/`source_path`+`source_line`, which has drifted as the repo grew across M5/M6. Tracked as a backlog item; not a M6.0 regression.
+- **E14 — `inspect_symbol` relationship contract fix (`retrieve/service.py`, `retrieve/workflows.py`)**: root cause was that `_inspect_symbol_scoped` fetched relationships via `impact_slice`, which has its own internal packing budget — at `budget_tokens < 2048` this gave `impact_slice` as little as 256 tokens, which could fail its own budget check and silently return zero edges regardless of how much of the *caller's* budget was actually free. Supersedes the partial mitigation in the M2/G4 entry above (raising `slice_tokens`), which reduced but did not eliminate the failure mode.
+  - Added `RetrievalService.symbol_relationships(repo_id, symbol_id, min_confidence=0.5)`: reads one-hop in/out edges directly from the cached `RepoGraph` and applies the shared `edge_is_traversable` predicate (promoted from `retrieve/expansion.py`, now used by both M5 graph-expansion and this method) — resolved, `backend=lexical`, real target, `confidence >= min_confidence`. No dependency on any other call's budget.
+  - `_inspect_symbol_scoped` now packs this pre-filtered, deterministically-sorted (confidence desc, then symbol_id) edge list into whatever budget remains after the symbol body, via a new `_pack_rows_to_budget` helper — a relationship is only ever dropped for genuinely not fitting, never by an unrelated internal budget.
+  - **Contract changes (approved for M6)**:
+    - `minimal` view `relationships`: compact list `[[symbol_id, "callee"|"caller", confidence], ...]`, quality-filtered as above.
+    - `normal` view: same filter; unchanged dict shape (`source`/`target`/`kind`/`confidence`).
+    - `full` view: `relationships` is now the same filtered/packed list as `normal` (previously: `normal`'s edges plus raw, unfiltered ones). `data.packet` is **not** part of this change — that is M6 Session 2 (M6.1/M6.2).
+    - New `data.relationships_filtered` field: count of edges excluded by the quality filter (ambiguous / stub / `confidence < 0.5`), kept distinct from the pre-existing `data.relationships_omitted` (budget-driven truncation).
+  - Test coverage: `tests/test_inspect_symbol_budget.py` (4 new tests against a fixture mirroring a pattern confirmed live on the real repo — a resolved same-file callee, an ambiguous unresolved-receiver call, and a stdlib `virtual_stub` call — plus 1 pre-existing test updated: 5→12 synthetic callees, because the fix itself made a 5-callee/256-token case no longer "tight").
+- Full pytest suite: 276 tests collected, all passing (1 pre-existing skip), no regressions from either change.
+
+## Unreleased — M5.2/M5.3 LARGER-lite graph expansion in `search_source` (2026-09-27, branch `feat/m5-larger-lite`)
+
+- **`search_source` contract (additive)**: new optional arguments `expand` (`auto` | `none` | `graph`, default `auto`), `expand_k` (1–8, default 3), `expand_hops` (1–2, default 1), `min_confidence` (0–1, default 0.6). Concrete types with defaults, no nullable `anyOf`.
+  - `expand="auto"` resolves to `graph` only for `profile="locate"`; calls without a profile keep the M5.1 behaviour (`none`). Output for `expand="none"`/`auto` without profile is byte-identical to M5.1 (tested).
+  - With `graph`, the response adds `neighbors: [[symbol_id, "path:line", "<kind> <name>", "callee|caller|test", score, anchor_id], ...]` and `retrieval: {expand_effective, profile_effective, anchors, neighbors, expand_k, expand_hops, min_confidence}`. The text summary shows `neighbors=<n>`.
+  - Neighbors take at most 25% of the packing budget and come out of the same `max_tokens`; neighbors whose anchor was packed out are dropped.
+- **Expansion (`retrieve/expansion.py`)**: top-5 matches are anchors; `<module>` anchors keep their rank but are not expanded. BFS over edges that are `resolved`, `backend=lexical`, point at a real symbol and have `confidence >= min_confidence` (excludes `global` 0.40, `unknown_receiver` 0.10, stubs). Score = best path product × 1.2 same file community × 0.5 utility hub (in-degree z > 3) × 0.5 test/eval path × (1 + 0.5·query terms in the name). Deterministic ordering (anchor rank, score, path, line). Relation labels: `callee`, `caller`, `test` only (the DB has `call` edges only; no `inherit`).
+- **File communities (M5.3)**: deterministic label propagation over the undirected in-repo import graph, computed lazily per index run and cached on `RepoGraph`. Deviation from plan: no `file_community` table and no schema bump; index schema stays 2.3. `SQLiteStore.import_pairs()` added.
+- **Ranking of AND/OR rows**: AND and OR top-up rows are now ranked together: non-test before `tests/`/`evals/`, AST symbols before `<module>`, AND before OR, then bm25. Fixes an eval file that embedded a task query outranking all source hits. Term-coverage ranking (F9) was implemented and **rejected on the dev split** (sym MRR 0.494 → 0.293 body-coverage, 0.423 name-coverage); kept behind `_COVERAGE_RANKING = "off"` for ablation only.
+- **Evals**: `loc_eval` now ranks each neighbor right after its anchor (was: after every match, so neighbors could never enter top-10) and reports `sym_recall_in_response` and `mean_neighbor_count`. New `evals/bench_search_source.py` replays the task queries per expand mode (M5 latency protocol). `evals/stdio_smoke.py` search check is now clean-room (temp repo) instead of depending on the local registry.
+- **Correction to the M5.1 entry below**: "latency p50 5.08 ms" was measured on a single easy query; on the 30 task queries M5.1 measured p50 ≈ 40–48 ms on Windows. See `evals/out/m5/latency_search_source_m5_2_vm.json` for the task-query protocol.
+
+## Unreleased — M5.1 Symbol-level FTS & Code Tokenization (2026-09-26, branch `feat/m5-larger-lite`)
+
+- **Index Schema 2.3 (`symbol_fts`)**:
+  - Added SQLite FTS5 virtual table `symbol_fts` using `unicode61 remove_diacritics 2 tokenchars '_'` tokenizer.
+  - Columns indexed: `symbol_id`, `path`, `name`, `code_tokens`, `docstring`, `body`.
+  - Nested body exclusion: symbols only index their `own_body` statements, stripping child symbol line spans to prevent parent classes or functions from diluting BM25 match density.
+  - File-level pseudo-symbols: created `<module>` symbol per file (`symbol_id = f"{lang}:{path}:<module>"`) to index top-level statements, imports, and constants.
+  - Snapshot migration: preserves backward compatibility with `source_bodies` in `sqlite_store.py`.
+- **Code Tokenization (`code_tokens.py`)**:
+  - Deterministic splitting of programming identifiers across `snake_case`, `camelCase`, `PascalCase`, numbers (`parseV2Config` -> `parse`, `v2`, `config`), acronyms (`HTTPServer` -> `http`, `server`), Vietnamese diacritics, and file path segments.
+- **Output Contract & Retrieval Optimization**:
+  - `search_source` contract refined: returns 1 entry per matching symbol (capped at 3 symbols per file) with new `lines: [[line_number, snippet_text], ...]` array preserving top matching spans.
+  - Backward compatibility: `start_line` and `snippet` fields preserved, pointing to the top scoring line of the symbol.
+  - Zero N+1 queries: `SQLiteStore.search_symbol_matches` performs `LEFT JOIN files` and `LEFT JOIN symbols` to eagerly retrieve file hashes and AST kinds within a single query (`last_query_count <= 3`).
+  - Search latency p50 reduced to 5.08 ms (< 40 ms threshold), symbol recall@10 jumped from 0.1500 to 0.7250 on heldout eval tasks.
+
+## Unreleased — M4 Edge Accuracy & Resolution Calibration (2026-09-26, branch `feat/m4-edge-accuracy`)
+
+- **Root Cause Fixes for False Positive Edges**:
+  - Eliminated naive substring matching (`imp in c.path`) in favor of exact segment suffix matching (`_path_segments` and `_import_matches_candidate`).
+  - Added receiver-type resolution via class-level attribute extraction and PyCG-style type inference (`attr_type` scope with confidence 0.95).
+  - Scope naming convention note:
+    - `import_match`: symbol imported directly (`from mod import func`) or imported module path segment match. Corresponds conceptually to contract specification `name_imported`. Retained to maintain index compatibility.
+    - `import_module_match`: call qualified by imported module alias/name (`mod.func()`). Corresponds conceptually to contract specification `module_exact`. Retained to maintain index compatibility.
+- **SCOPE_CONFIDENCE Calibration**:
+  - Calibrated based on empirical precision from evaluated gold set (150 samples: 114 TP, 36 FP; 0 active FP, precision 100.0%):
+    - `exact_receiver_type`: 0.95 (n=30, prec 1.0)
+    - `same_class`: 0.95 (n=22, prec 1.0)
+    - `import_match`: 0.95 (n=26, prec 1.0)
+    - `same_file`: 0.95 (n=23, prec 1.0)
+    - `attr_type`: 0.95 (receiver attribute type match)
+    - `import_module_match`: 0.75 (n=5, prec 1.0, conservative default for n < 10)
+    - `same_package`: 0.75 (n=3, conservative default for n < 10)
+    - `global`: 0.40 (n=5, elim FP 20, TP 2)
+    - `virtual_stub`: 0.90 (virtual external stubs)
+    - `unknown_receiver` / `unresolved_receiver`: 0.10 (ambiguous fallback)
+  - Ceiling rule: empirical precision is conservatively capped at 0.95 rather than 1.00 to account for statistical generalization.
+- **Top-20 In-Degree Normalization**:
+  - `MemoryStore.get` in-degree reduced from 96 (dominated by false generic calls like `dict.get`, `os.environ.get`) to 10 legitimate callers.
+  - Overall accuracy achieved 85.33% (128/150 raw eval); recall achieved 85.00% (1.159x baseline).
+- **Known Ranking Regression (dynamic_mode)**:
+  - Pruning of false positive edges shifted relative node degrees, causing `extract_with_trace` (rank 5 -> 8) and `run_with_diagnostics` (rank 7 -> 11) to drop in dynamic on-the-fly ranking, reducing dynamic nDCG@10 from 0.5248 to 0.4908 (-0.034).
+  - In contrast, production index-time `cached_mode` nDCG@10 improved significantly from 0.4224 to 0.5410 (+0.1186). M5 commits to no further regression.
+
+## Unreleased — M3 Query IO, Latency & Freshness Caching (2026-09-25, branch `feat/m3b-rank-cache`)
+
+- **FreshnessCache & ReadConnectionPool**: In-memory mtime cache with zero disk I/O when unchanged, snapshot-isolated read pool.
+- **PageRank & Personalized PageRank**: Index-time global PageRank and fast query-time local push PPR.
+- **RepoGraph Cache**: In-memory caching of adjacency graphs for sub-millisecond graph traversals.
+
+## Unreleased — M2-F Remediation & Bug Fixes (2026-09-25, branch `feat/m2-governance`)
+
+- **G1 (P0 Memory Migration & Auto-repair)**: `MemoryStore` v3 migration, automated detection and rebuild of broken legacy FTS tables lacking `namespace` column, backup to `.bak-v<ver>`, leaf string value re-redaction, refined SQLite error handling distinguishing syntax errors from structural errors.
+- **G2 (GUI Bridge Read/Write Separation)**: `AgentSecurityController` wired to `GovernanceStore`, `refresh_data()` made pure query via `merge_seen_agents()`, eliminating unexpected write mutations on GUI refresh.
+- **G3 (Agent Policy Synchronization)**: Added `policy` and `custom_tools_json` columns to `governance.sqlite` `agents` table, reduced access control cache TTL to 0.5s, cross-process policy sync verified within 0.6s.
+- **G4 (inspect_symbol Compact Relationships)**: Used compact edge dicts in `normal` view, increased edge retrieval ceiling (`slice_tokens`), added explicit `relationships_truncated` warning and `relationships_omitted` field when truncated.
+
+## Unreleased — M2 Governance & Memory Safety (2026-09-24, branch `feat/m2-governance`)
+
+- **Admin tools gate** (`enable_admin_tools: bool = False`): `agent_control` and `audit_logs` only registered when opt-in flag is set; `action=status` always public, all other actions require `TOKEN_CONTEXT_ADMIN_TOKEN`.
+- **Token protection**: admin token verified with `hmac.compare_digest`; never reflected in audit log; invalid token returns `permission_revoked` with opaque reason.
+- **Shared governance store** (`security/governance_store.py`): WAL-mode SQLite (`governance.sqlite`) with tables `agents`, `emergency`, `server_heartbeats`; 5 s busy-timeout; server records heartbeat every ≥15 s per call.
+- **Agent identity resolution** (`resolve_effective_agent_id`): reads `TOKEN_CONTEXT_AGENT_ID` env, validates `^[a-zA-Z0-9_-]{1,64}$`, falls back to `"anonymous"`; anonymous agent exempt from rate limiting but subject to all other controls.
+- **Atomic memory lock**: single `INSERT … ON CONFLICT DO UPDATE WHERE …` — one winner across any number of concurrent processes; `memory_unlock` tool added.
+- **Memory namespace isolation**: `key_values` schema v2 — `PRIMARY KEY (scope, namespace, key)`; existing DBs backed up before migration; `namespace=""` default preserves backward compatibility.
+- **Value redaction** (`_redact_value`): recursively applies `content_policy.redact_text` to all leaf strings before storage.
+- **Periodic TTL cleanup**: expired entries purged every 100 `put` calls.
+- **GUI read-only governance refresh** (`GovernanceRefreshWorker`): polls `governance.sqlite` every 5 s; never writes; `AgentSecurityController` starts it automatically; `active_servers_updated` signal added.
+- **Test coverage**: 161 passed, 6 skipped (GUI tests); new files `test_server_governance.py`, `test_governance_store.py`, `test_memory_lock_atomic.py`, `test_memory_namespace.py`; 3 new tests in `test_gui_bridge.py`.
+
+## Unreleased — POSIX hosts and snapshot privacy (2026-08-30)
+
+- Created the registry, snapshots, manifests and WAL sidecars owner-only (`0700`/`0600`) instead of inheriting the process umask, which left indexed source bodies world-readable under a default `0022`.
+- Added `token-context harden`, with `--check`, to repair existing files and to report the ACL principals of the config directory on Windows.
+- Kept repository-relative paths intact on POSIX, where a backslash is a filename character rather than a separator; the stricter Windows-shaped validation still applies on every host.
+- Documented Linux/macOS registry paths, VS Code Remote-SSH placement, and multi-user host guidance in the README and `SECURITY.md`.
+- Added `docs/PLATFORMS.en.md` and `docs/PLATFORMS.vi.md`: supported operating systems, remote/SSH placement, every enforced limit, and the permission model.
+
+## Unreleased — remediation after review 935511f (2026-08-28)
+
+- Restored the `token_context_mcp.security` package to the publishable source tree and added clean-room wheel checks.
+- Added cross-platform CI for source tests and an installed-wheel MCP stdio handshake.
+- Replaced regex import extraction with Tree-sitter extraction, including relative, aliased, re-export, and dynamic-import warnings.
+- Indexed TypeScript/TSX arrow functions, function expressions, anonymous default exports, and zero-symbol parse warnings.
+- Prevented stale symbol offsets from returning content under an old source hash.
+- Measured freshness-scan cost at the 1,000, 10,000, and 25,000-file scales.
+
+`v0.1.0` remains untagged until the required CI checks are green.

token_context_mcp-0.3.5/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2026 Token Context MCP contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+