weavatrix 0.2.5 → 0.2.7

package/README.md

@@ -5,7 +5,7 @@
 Grep sees text. Weavatrix sees structure. It builds a dependency graph of any local repository —
 files, symbols, and the imports/calls/inheritance connecting them — and serves it to Claude Code,
 Codex, or any MCP client: change impact, transitive dependents, health audit, clone detection,
-coverage mapping. **34 tools available; 31 enabled by the default offline profile. Local-first: with
+coverage mapping. **36 tools available; 33 enabled by the default offline profile. Local-first: with
 the defaults, no repository data leaves your machine.**
 
 - Website: [weavatrix.com](https://weavatrix.com)
@@ -22,12 +22,18 @@ answers grep can't produce:
   depends on them — with test coverage attached, so the **untested part of the blast radius** stands
   out before you ship.
 - *"Who calls this function?"* → `inspect_symbol` returns exact bounded TS/JS occurrences plus
-  source context; `get_dependents` walks the symbol-level reverse graph transitively. Opt into
+  source context; `context_bundle` adds grouped graph relations, exact re-export sites, and a
+  smaller source workset. `get_dependents` walks the symbol-level reverse graph transitively. Opt into
   `include_container_importers:true` only when a broader module-import radius is intended.
 - *"Did my refactor actually decouple anything?"* → `graph_diff base_ref=HEAD~1` builds an immutable
   baseline graph without checking it out, then reports the structural delta: new module
   dependencies, broken or introduced import cycles, and symbols that lost their last caller.
 
+- *"Is this change actually safe?"* -> `verified_change` accepts the task plus current diff/files and
+  returns one proof-carrying `PASS`, `BLOCKED`, or `UNKNOWN`: compact edit contexts, exact-symbol
+  impact, bounded call-argument flow, Git graph drift, architecture/duplicate/API ratchets, and
+  affected-test evidence.
+
 ## Quick start
 
 Requires Node ≥ 18. One command:
@@ -107,10 +113,11 @@ An agent skill with recipes ships in [skill/SKILL.md](skill/SKILL.md) — instal
 
 **graph** — `graph_stats`, `get_node`, `get_neighbors`, `query_graph`, `god_nodes`,
 `shortest_path`, `get_community`, `list_communities`, `module_map`, `get_dependents`,
-`change_impact`, `git_history`, `graph_diff`, `get_architecture_contract`,
+`change_impact`, `verified_change`, `git_history`, `graph_diff`, `get_architecture_contract`,
 `prepare_change`. Runtime dependencies, TypeScript type-only coupling and language
 compile-only edges (Rust module/use, Java imports) are reported separately where that distinction
-changes the result.
+changes the result. TypeScript type-space and value-space declarations keep distinct identities;
+classes and enums that inhabit both spaces are labelled `both`.
 
 Every current edge carries versioned provenance. The parser emits `EXTRACTED`, `RESOLVED`, and
 `INFERRED`; the built-in bounded TypeScript/JavaScript precision overlay upgrades only references
@@ -118,7 +125,7 @@ confirmed by its bundled `typescript-language-server` + TypeScript runtime to `E
 `CONFLICT` means evidence disagrees. `graph_stats` reports the provenance breakdown and the semantic
 provider's `COMPLETE`, `PARTIAL`, `UNAVAILABLE`, or `OFF` state; `OFF` means precision was explicitly
 disabled and only static evidence is active. Java and Rust language-server providers are not bundled
-in 0.2.5: their edges never become `EXACT_LSP`, even when a mixed repository reports a complete
+in 0.2.7: their edges never become `EXACT_LSP`, even when a mixed repository reports a complete
 TypeScript/JavaScript overlay.
 
 The bounded JS/TS provider is enabled by default for new graphs. Set `WEAVATRIX_PRECISION=off`
@@ -127,13 +134,15 @@ before starting the MCP server for parser-only operation from the first build, o
 choice as **TypeScript/JavaScript semantic precision**.
 
 **search / source** — `search_code` (ripgrep-backed, pure-Node fallback), `read_source` (a
-symbol's actual code in one hop), `inspect_symbol` (one exact bounded TS/JS reference query,
-logical containers, impact and source excerpts), `list_endpoints` (HTTP route inventory:
+symbol's actual code in one hop), `context_bundle` (definition plus grouped inbound/outbound
+containers, exact re-export sites and a few bounded source excerpts), `inspect_symbol` (one exact
+bounded TS/JS reference query, logical containers, impact and source excerpts), `list_endpoints` (HTTP route inventory:
 Express/Fastify/Nest/Flask/FastAPI/Go mux/Rust axum and actix-web …)
 
 **health** — `find_dead_code` (bounded review queue for statically unreferenced and test-only files,
-functions, methods, and symbols, with confidence/evidence and explicit public/framework/dynamic caveats),
-`run_audit` (unused files/exports/dependencies with an explicit manifest-check summary,
+functions, methods, and symbols, with evidence tiers, completed/remaining verification and explicit
+public/framework/dynamic caveats),
+`run_audit` (unused files/exports/dependencies with per-finding manifest/indexed-source/script/config verification,
 missing npm/Go/Python deps, runtime
 cycles, type-only/compile-only coupling, orphans, boundary rules, offline OSV vulnerabilities + typosquat +
 lockfile drift; accepts an immutable `base_ref`, `changed_files`, and `debt: new|existing|all` for
@@ -146,7 +155,9 @@ with separate graph/test risk), `verify_architecture`,
 `hot_path_review` ranks production symbols using parser-derived local time, memory, cyclomatic and
 call-site facts plus exact inside-loop allocation, copy, scan, sort and recursion evidence. Graph
 fan-in/fan-out and measured coverage (or explicitly labelled static test reachability) remain
-separate from local syntax cost. The ranking is not profiler data or interprocedural Big-O.
+separate from local syntax cost. Its focused default uses `min_score:85` plus a narrow strong-local
+fallback; pass `min_score:0` only when the full diagnostic queue is wanted. The ranking is not
+profiler data or interprocedural Big-O.
 
 **build** — `rebuild_graph` (reports the structural delta, keeps the prior state as
 `graph.prev.json`)
@@ -157,7 +168,8 @@ use `kinds:["method"]` or a `path` prefix to narrow it. `min_confidence:"low"` e
 public APIs and framework/dynamic/reflection-sensitive candidates with their warnings. For repository
 maintenance or branch debt, pair it with `run_audit category=unused debt=all` (or add an immutable
 `base_ref` with `debt=new`) to cover unused files, exports and dependencies. Neither tool authorizes
-deletion: inspect `read_source`, `get_dependents`, exact search, manifests/configuration and tests first.
+deletion: `STRONG_STATIC_EVIDENCE` means an exact zero-reference result, not permission to remove;
+follow each candidate's `remainingChecks`, then inspect source/dependents/configuration and run tests.
 
 `graph_diff` accepts `base_ref` (`HEAD~1`, `main`, `origin/main`, or another local Git ref) for a
 fresh baseline comparison. Without it, the tool compares against `graph.prev.json` saved by the last
@@ -169,6 +181,14 @@ manifest, alias, ignore/config, non-JS/TS, or unsafe merge cases deliberately fa
 rebuild. The result says whether freshness was `none`, `incremental`, or `full`. Graph artifacts stay
 in the per-user cache and never need to be committed to Git.
 
+`verified_change` is read-only by default. It may run only explicitly requested `package.json`
+scripts whose names are allowlisted as test/check/verify scripts, and only when both
+`run_tests:true` and `WEAVATRIX_ALLOW_TEST_RUNS=1` are present. It never accepts an arbitrary shell
+command. Cross-repository API proof is used only when the active profile includes `crossrepo`;
+without a configured architecture contract or complete evidence, the result is `UNKNOWN`, not a
+cosmetic pass. Its data-flow section maps bounded JS/TS call arguments to callee parameters; it is
+not CFG, value-propagation, or taint analysis.
+
 **retarget** *(included in `offline`; absent from `pinned`; every switch is an explicit local call)* —
 `open_repo`, `list_known_repos`; changes the active repository boundary
 
@@ -179,8 +199,11 @@ in the per-user cache and never need to be committed to Git.
 entry points. Resolved explicit seeds are exclusive by default; set `augment_seeds:true` only when
 question-derived seeds are also wanted. Broad bootstrap/tool-execution/routing questions now rank
 conventional executables and graph-declared production entry points ahead of site, documentation, benchmark
-and fixture matches. Broad ranking remains orientation evidence; use `seed_files` when the intended
-entry point is already known.
+and fixture matches. Production-first classification also applies during traversal, so unrelated
+tests/generated/docs/benchmarks do not leak back from a production seed; name a class in the
+question or set `include_classified:true` to include it. Unreferenced unmatched constant/field leaves
+are hidden unless `include_low_signal:true`. Broad ranking remains orientation evidence; use
+`seed_files` when the intended entry point is already known.
 
 **advisories** *(network, explicit opt-in)* — `refresh_advisories`
 
@@ -193,6 +216,46 @@ disclosed instead of silently guessed; and the server **hot-reloads its watched
 modules and catalog** when those files change — other MCP helpers and analysis engines require a
 reconnect.
 
+### 0.2.7 verified-change workflow
+
+- New `verified_change` composes task retrieval, exact symbol context, change impact, immutable Git
+  graph comparison, architecture/duplicate/API ratchets, and targeted-test evidence behind one
+  `PASS` / `BLOCKED` / `UNKNOWN` contract.
+- Intent-expanded graph retrieval is combined with exact changed-symbol seeds. A bounded JS/TS
+  interprocedural layer records call arguments and their callee parameters without claiming CFG or
+  taint completeness.
+- Test execution is a double opt-in and limited to existing test/check/verify package scripts;
+  default operation remains read-only.
+- Malware scanning now sweeps installed npm package roots instead of package-manager caches or
+  hosted release snapshots. Static heuristic findings are capped at `high`, expose unverified
+  execution/origin/lockfile/exposure state, and no longer prescribe secret rotation unless execution
+  or credential exposure has actually been confirmed.
+- `query_graph` and `verified_change` task retrieval now enforce production-first classification
+  through traversal/selection while retaining exact changed or pinned non-product symbols; query
+  output also suppresses unmatched constant/field leaves. `hot_path_review` defaults to a focused
+  85-point queue, while `min_score:0` remains the explicit full-diagnostic mode.
+- Dead-code candidates now carry evidence tiers and remaining checks; dependency findings carry
+  per-finding manifest plus indexed-source/script/config verification without authorizing removal.
+- `npm run benchmark:agent` reports routing success, false positives, token estimate and latency.
+  Codebase Memory and Serena results stay explicitly `MISSING` until independently collected data is
+  supplied under the [blind agent-change protocol](docs/agent-task-benchmark.md);
+  `benchmark:agent:release` fails closed without same-task change success, FP, token, and time results.
+
+Full patch notes: [docs/releases/v0.2.7.md](docs/releases/v0.2.7.md).
+
+### 0.2.6 compact-context and TypeScript identity patch
+
+- New `context_bundle` turns a symbol into a bounded workset: definition, grouped inbound/outbound
+  containers, reference evidence, exact re-export locations and a small number of source excerpts.
+- Re-export records retain their concrete file, line, alias, specifier, type-only flag and resolved
+  origin. Barrel propagation through `export *` stays exact and private declarations are not exposed.
+- TypeScript interfaces and type aliases are first-class type-space nodes. Same-named runtime values
+  keep separate identities, while classes and enums are explicitly marked as inhabiting both spaces.
+- Graph schema v5 and freshness gates rebuild older caches before these precision-sensitive results
+  are used. The changes remain local-only and add no runtime dependency.
+
+Full patch notes: [docs/releases/v0.2.6.md](docs/releases/v0.2.6.md).
+
 ### 0.2.5 exact-symbol and graph-fidelity patch
 
 - New `inspect_symbol` spends a bounded LSP query on one requested TS/JS declaration, groups exact
@@ -450,7 +513,7 @@ vulnerability findings. This is where each capability comes from and how it is c
 | Capability alert | Why it exists | Activation and boundary |
 |---|---|---|
 | Network access | `refresh_advisories` sends pinned package names and versions to OSV; `pull_architecture_contract` sends an opaque repository UUID and receives an owner-approved contract; `sync_graph` sends a normalized repository label plus an allowlisted graph/evidence payload. Evidence is derived locally from source and manifests, but source bodies, snippets, absolute paths, environment values, credentials and Git remotes are excluded | `offline` and `pinned` expose no network tools. `osv` exposes only advisory refresh. `hosted`/`full` expose all three; every request still requires an explicit tool call, and hosted calls require `WEAVATRIX_SYNC_URL` |
-| Shell access | Local `git` powers staleness/change impact; `rg` accelerates search; the bundled TLS/tsserver process supplies bounded JS/TS semantic evidence; timed-out Windows child trees may be terminated | Used only by the corresponding local operation. The semantic provider is package-pinned, disables automatic type acquisition, and never invokes a repository binary, script, installer or `npx` |
+| Shell access | Local `git` powers staleness/change impact; `rg` accelerates search; the bundled TLS/tsserver process supplies bounded JS/TS semantic evidence; timed-out Windows child trees may be terminated; `verified_change` can optionally run an existing package test/check/verify script | The semantic provider never invokes repository code. Test execution separately requires `run_tests:true` plus `WEAVATRIX_ALLOW_TEST_RUNS=1`, rejects arbitrary commands and shell-sensitive arguments, and should be enabled only for trusted repository scripts |
 | Debug / dynamic loading | Cache-busted `import()` hot-reloads watched MCP tool entry modules; `createRequire` loads package metadata and parser dependencies | Loads files from the installed package; no `eval` |
 | Environment access | Reads `WEAVATRIX_*` configuration; ordinary local helpers inherit a credential-stripped environment, while TLS/tsserver receives only allowlisted OS/temp/locale values and a constrained executable path | `WEAVATRIX_SYNC_TOKEN` is removed from every child-process and worker environment. TLS/tsserver also receives no registry/proxy/cloud credentials or `NODE_OPTIONS` |
 | Filesystem access | Reads the active repository, graph, lockfiles and coverage reports; writes derived graphs and advisory/architecture caches | Realpath containment blocks traversal and symlink/junction escapes. The `pinned` profile removes `open_repo`; the default `offline` profile permits only explicit local switching. The optional malware dependency scan may inspect installed dependency caches such as GOPATH |

package/SECURITY.md

@@ -28,6 +28,11 @@ dependency caches such as GOPATH. `offline` permits repository switching only th
 local `open_repo` call; select `pinned` to remove that tool, the global repository listing, and
 cross-repository API tracing, holding a hard startup-repository boundary.
 
+Installed-package malware pattern matching is static heuristic evidence. It cannot confirm execution,
+package compromise, or credential exposure and therefore cannot emit `critical`; heuristic severity
+is capped at `high` with explicit `NOT_VERIFIED` runtime/origin/lockfile/exposure fields. Independently
+confirmed malicious-package advisories remain separate vulnerability evidence and may be critical.
+
 The JS/TS precision overlay is enabled by default for new graphs and runs the package-pinned `typescript-language-server` and
 TypeScript runtime as local child processes. It never resolves a repository executable, invokes
 `npx`, runs package scripts, or installs dependencies. Automatic type acquisition is disabled, and
@@ -46,6 +51,15 @@ bounded graph-work drain, and closes or tree-terminates TLS/tsserver before exit
 Set `WEAVATRIX_PRECISION=off` before server startup (or select `off` in the MCPB semantic-precision
 setting) to keep new graphs parser-only from their first build.
 
+`verified_change` is read-only by default. Its optional targeted-test step can execute only an
+existing `package.json` script whose name matches the bounded test/check/verify allowlist. The call
+must set `run_tests:true` and the server operator must separately set
+`WEAVATRIX_ALLOW_TEST_RUNS=1`; otherwise no repository script runs. The tool accepts no command or
+shell string, caps scripts/arguments/time, rejects shell-sensitive argument characters, and launches
+the selected package-manager script with the credential-stripped child environment. Repository test
+scripts are repository-controlled code and may have arbitrary side effects, so enable this flag only
+for repositories and scripts you trust.
+
 Network capabilities are split by purpose. `osv` adds only `refresh_advisories`, which sends pinned
 package names and versions to OSV.dev when called. `hosted` / `full` also expose `sync_graph` and
 `pull_architecture_contract`; both require a user-configured endpoint, and contract pull requires

package/docs/agent-task-benchmark.md

@@ -0,0 +1,58 @@
+# Independent agent-change benchmark
+
+The 0.2.7 release includes two deliberately separate measurements.
+
+`npm run benchmark:agent` is a deterministic local routing microbenchmark. It measures whether the
+task retriever selects the expected symbol, plus false-positive selections, estimated output tokens,
+and local latency. It is a regression test, not evidence that an autonomous code change succeeded.
+
+The release comparison is an independently executed, blind agent-change benchmark. The evaluator
+must run the same pinned tasks and repository commits three times: with Weavatrix, with Codebase
+Memory, and with Serena. The product author must not adjudicate or repair a run while it is active.
+
+## Required task protocol
+
+Each task must provide a repository commit, natural-language request, allowed commands, timeout, and
+machine-verifiable acceptance tests. Keep task prompts and acceptance tests identical across systems;
+randomize system order. Record the entire agent input/output token count and wall-clock duration.
+
+A run is successful only when the requested behavior is implemented, all acceptance tests pass, and
+the final diff stays inside the task scope. A false positive is an evidenced warning, impacted symbol,
+or required edit that an evaluator confirms is unrelated to the task. Record a count, including zero;
+do not silently discard noisy output.
+
+Use a mix of at least:
+
+- behavior changes with indirect callers;
+- API route/client changes;
+- refactors with a duplicate or architecture regression trap;
+- changes whose nearest relevant test is not in the edited file;
+- one negative task where the correct action is to refuse or return unknown.
+
+## Result contract
+
+Save the blind evaluator output as JSON:
+
+```json
+{
+  "schemaVersion": "weavatrix.agent-change-results.v1",
+  "evaluator": "independent organization or person",
+  "systems": {
+    "weavatrix": {"runs": [{"taskId": "change-auth", "success": true, "falsePositives": 0, "tokens": 1234, "durationMs": 4567}]},
+    "codebase-memory": {"runs": [{"taskId": "change-auth", "success": true, "falsePositives": 1, "tokens": 1400, "durationMs": 5000}]},
+    "serena": {"runs": [{"taskId": "change-auth", "success": false, "falsePositives": 2, "tokens": 1600, "durationMs": 7000}]}
+  }
+}
+```
+
+Every system must contain the same unique task IDs. Every run must report success, false positives,
+tokens, and duration. Then run:
+
+```sh
+npm run benchmark:agent -- --independent-results path/to/results.json
+npm run benchmark:agent:release -- --independent-results path/to/results.json
+```
+
+The first command reports change-success rate, false positives per task, median tokens, and median
+duration for all three systems. The release command fails unless the independent comparison is
+complete and valid. Missing or malformed competitor data is `INCOMPLETE`, never a green comparison.

package/docs/releases/v0.2.7.md

@@ -0,0 +1,93 @@
+# Weavatrix 0.2.7
+
+## Proof-carrying changes
+
+`verified_change` is a new high-level MCP workflow for planning and verifying a code change. It
+accepts a natural-language task, an optional unified diff or changed-file hints, and an immutable Git
+baseline. Its stable result is one of:
+
+- `PASS`: every applicable enabled proof completed and no ratchet failed;
+- `BLOCKED`: a test failed, a new duplicate intersects changed code, a runtime cycle appeared, an
+  architecture ratchet failed, or bounded API evidence found an HTTP method mismatch;
+- `UNKNOWN`: required evidence is missing, partial, disabled, or not configured.
+
+The structured result contains intent-expanded task retrieval combined with exact changed-symbol
+seeds, up to five compact LSP/graph edit contexts, symbol-aware blast radius, affected-test paths,
+bounded JS/TS call-argument-to-parameter evidence, Git graph drift, architecture verification, a
+new-duplicate baseline comparison, and optional cross-repository HTTP contract evidence.
+
+The workflow respects the existing capability boundary. Exact source/LSP context requires `source`,
+duplicate comparison and architecture verification require `health`, and API tracing requires
+`crossrepo`. A narrow custom profile is never silently bypassed.
+
+## Safe targeted tests
+
+The workflow does not accept shell commands. It can execute at most five existing `package.json`
+scripts whose names match the test/check/verify allowlist. Execution requires both
+`run_tests:true` on the tool call and `WEAVATRIX_ALLOW_TEST_RUNS=1` in the server environment. The
+default remains read-only and returns the test plan or `UNKNOWN` when runtime evidence was requested
+but not authorized.
+
+## Malware scanner correction
+
+The npm content sweep now starts at actual installed package roots instead of the entire
+`node_modules` container. Hidden package-manager caches and hosted release snapshots are therefore
+not attributed as dependencies and cannot make Weavatrix flag its own malware-signature source.
+
+Heuristic remediation no longer says to treat a package as compromised or rotate all reachable
+secrets. It asks the operator to quarantine and inspect the package, verify origin and lockfile
+integrity, and rotate secrets only after execution or credential exposure is confirmed.
+
+Static heuristic findings now have a hard `high` severity ceiling. They expose
+`confirmedExecution:false` plus explicit runtime/origin/lockfile/exposure verification states;
+`critical` is reserved for independently confirmed malicious advisories or runtime evidence.
+
+Package-lock install-script drift now compares only `preinstall`, `install`, and `postinstall`.
+Development/publication hooks such as `prepare` no longer create false high-severity drift findings.
+
+## Signal-quality corrections
+
+- `query_graph` applies production-first path classification to the traversal, not only seed
+  selection. Tests, generated code, fixtures, docs, benchmarks and excluded paths require an
+  explicit question class or `include_classified:true`. Exact non-product seed files remain usable.
+- `verified_change` task retrieval uses the same production-first policy while always retaining
+  exact changed symbols. The bundled routing microbenchmark moved from one test-symbol false
+  positive (`0.25`) to zero (`0`) without losing any of its three expected production targets.
+- Unreferenced constant/field leaves that do not match the query are hidden by default; use
+  `include_low_signal:true` for diagnostic expansion.
+- `hot_path_review` now defaults to a focused score gate of 85 plus a narrow strong-local fallback.
+  On the 0.2.7 source tree this reduced the default queue from 717 diagnostic candidates to 116 of
+  1,043 analyzed symbols; `min_score:0` restores the full diagnostic set.
+- `find_dead_code` adds `STRONG_STATIC_EVIDENCE`, `BOUNDED_STATIC_EVIDENCE`, and
+  `HIGH_UNCERTAINTY` tiers, structured completed/remaining checks, and `autoDelete:false` on every
+  candidate. Strong static evidence means exact zero LSP references, not permission to delete.
+- npm unused/missing/duplicate dependency findings now carry manifest section, indexed-source,
+  script/config, package-scope and unresolved dynamic/plugin verification state. Unused dependencies
+  remain `REVIEW_REQUIRED`; missing dependencies retain the exact importing-file evidence.
+
+## Benchmark contract
+
+`npm run benchmark:agent` runs a deterministic task-to-symbol routing microbenchmark and reports task
+success, false-positive rate, estimated output tokens, and duration. This is intentionally labelled
+as a routing microbenchmark, not an autonomous end-to-end change benchmark.
+
+Independent Codebase Memory and Serena results may be supplied with
+`--independent-results <json>` using the blind evaluator contract in
+[`docs/agent-task-benchmark.md`](../agent-task-benchmark.md). Without same-task results for all three
+systems, comparison status is `INCOMPLETE`; `npm run benchmark:agent:release` fails closed. The
+repository does not manufacture competitor scores.
+
+## Scope and limitations
+
+- The interprocedural evidence maps call arguments to declared callee parameters for bounded JS/TS
+  graph call edges. It is not a CFG, value-propagation engine, or taint analysis.
+- An absent architecture contract produces `UNKNOWN`; provisional budgets are guidance, not policy.
+- A static path to a test is not proof that the test executed. A `PASS` that depends on runtime
+  behavior requires explicitly authorized test execution.
+- API proof is optional and registry-scoped. No client match is `UNKNOWN`, never proof of no external
+  consumers.
+
+## Tool catalog
+
+0.2.7 exposes 36 MCP tools. The default offline profile exposes 33; pinned exposes 30. Network tools
+remain absent from the offline and pinned profiles.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "weavatrix",
-  "version": "0.2.5",
+  "version": "0.2.7",
   "type": "module",
   "description": "Weavatrix — code graph & blast-radius MCP server for AI coding agents: change impact, transitive dependents, health audit, duplicate detection, and coverage mapping over any local repository.",
   "author": "Sergii Ziborov <sergii.ziborov@gmail.com>",
@@ -24,6 +24,9 @@
     "bin",
     "src",
     "skill",
+    "scripts/run-agent-task-benchmark.mjs",
+    "docs/agent-task-benchmark.md",
+    "docs/releases/v0.2.7.md",
     "README.md",
     "SECURITY.md",
     "LICENSE"
@@ -34,6 +37,8 @@
     "benchmark:quick": "node scripts/run-benchmark.mjs --quick",
     "benchmark:real": "node scripts/run-real-benchmark.mjs",
     "benchmark:real:release": "node scripts/run-real-benchmark.mjs --require-all",
+    "benchmark:agent": "node scripts/run-agent-task-benchmark.mjs",
+    "benchmark:agent:release": "node scripts/run-agent-task-benchmark.mjs --require-independent",
     "build:mcpb": "node scripts/build-mcpb.mjs",
     "verify:release": "node scripts/verify-release.mjs"
   },

package/scripts/run-agent-task-benchmark.mjs

@@ -0,0 +1,119 @@
+import {readFileSync} from 'node:fs'
+import {pathToFileURL} from 'node:url'
+import {performance} from 'node:perf_hooks'
+import {retrieveTaskContext} from '../src/analysis/task-retrieval.js'
+
+function graphFixture() {
+  const nodes = [
+    {id: 'src/auth/session.js#validateSession@10', label: 'validateSession()', source_file: 'src/auth/session.js', symbol_kind: 'function'},
+    {id: 'src/http/router.js#registerRoutes@20', label: 'registerRoutes()', source_file: 'src/http/router.js', symbol_kind: 'function'},
+    {id: 'src/cache/store.js#readCache@5', label: 'readCache()', source_file: 'src/cache/store.js', symbol_kind: 'function'},
+    {id: 'src/auth/session.test.js#sessionTest@3', label: 'sessionTest()', source_file: 'src/auth/session.test.js', symbol_kind: 'function'},
+  ]
+  return {nodes, byId: new Map(nodes.map((node) => [node.id, node])), out: new Map(), inn: new Map()}
+}
+
+const CASES = [
+  {task: 'validate authentication session', expected: 'src/auth/session.js#validateSession@10'},
+  {task: 'register HTTP API routes', expected: 'src/http/router.js#registerRoutes@20'},
+  {task: 'read values from cache store', expected: 'src/cache/store.js#readCache@5'},
+]
+
+function competitorTemplate(name) {
+  return {name, status: 'MISSING', reason: 'supply same-task blind evaluator output with --independent-results <json>'}
+}
+
+function readCompetitors(path) {
+  if (!path) return {}
+  const parsed = JSON.parse(readFileSync(path, 'utf8'))
+  return parsed?.competitors && typeof parsed.competitors === 'object' ? parsed.competitors : {}
+}
+
+const SYSTEMS = ['weavatrix', 'codebase-memory', 'serena']
+const median = (values) => {
+  const sorted = values.slice().sort((left, right) => left - right)
+  const middle = Math.floor(sorted.length / 2)
+  return sorted.length % 2 ? sorted[middle] : (sorted[middle - 1] + sorted[middle]) / 2
+}
+
+function independentComparison(path) {
+  if (!path) return {status: 'MISSING', reason: 'supply blind evaluator output with --independent-results <json>'}
+  try {
+    const input = JSON.parse(readFileSync(path, 'utf8'))
+    if (input?.schemaVersion !== 'weavatrix.agent-change-results.v1') throw new Error('unsupported schemaVersion')
+    const taskSets = []
+    const metrics = {}
+    for (const name of SYSTEMS) {
+      const runs = input.systems?.[name]?.runs
+      if (!Array.isArray(runs) || !runs.length) throw new Error(`${name} has no runs`)
+      const ids = runs.map((run) => String(run.taskId || '')).sort()
+      if (ids.some((id) => !id) || new Set(ids).size !== ids.length) throw new Error(`${name} task IDs are missing or duplicated`)
+      for (const run of runs) {
+        if (typeof run.success !== 'boolean' || !Number.isFinite(run.falsePositives) || !Number.isFinite(run.tokens) || !Number.isFinite(run.durationMs)) {
+          throw new Error(`${name}/${run.taskId || '?'} has incomplete metrics`)
+        }
+      }
+      taskSets.push(ids.join('\0'))
+      metrics[name] = {
+        tasks: runs.length, changeSuccessRate: runs.filter((run) => run.success).length / runs.length,
+        falsePositiveRate: runs.reduce((sum, run) => sum + Math.max(0, run.falsePositives), 0) / runs.length,
+        medianTokens: median(runs.map((run) => run.tokens)), medianDurationMs: median(runs.map((run) => run.durationMs)),
+      }
+    }
+    if (new Set(taskSets).size !== 1) throw new Error('systems were not evaluated on the same task IDs')
+    return {status: 'COMPLETE', evaluator: String(input.evaluator || 'unspecified'), metrics}
+  } catch (error) {
+    return {status: 'INVALID', reason: error instanceof Error ? error.message : String(error)}
+  }
+}
+
+export function runAgentTaskBenchmark({competitorResults, independentResults} = {}) {
+  const graph = graphFixture()
+  const started = performance.now()
+  const cases = CASES.map((item) => {
+    const semanticSeeds = graph.nodes.filter((node) => item.task.toLowerCase().split(/\W+/).some((word) => word.length > 3 && String(node.id).toLowerCase().includes(word)))
+    const result = retrieveTaskContext(graph, {task: item.task, semanticSeeds, maxSymbols: 2})
+    const selected = result.selected.map((entry) => entry.id)
+    return {task: item.task, expected: item.expected, selected, success: selected[0] === item.expected, falsePositives: selected.filter((id) => id !== item.expected).length}
+  })
+  const durationMs = performance.now() - started
+  const output = JSON.stringify(cases)
+  const imported = readCompetitors(competitorResults)
+  const independent = independentComparison(independentResults)
+  const competitors = independent.status === 'COMPLETE' ? {
+    'codebase-memory': {name: 'codebase-memory', status: 'COMPLETE', metrics: independent.metrics['codebase-memory']},
+    serena: {name: 'serena', status: 'COMPLETE', metrics: independent.metrics.serena},
+  } : {
+    'codebase-memory': imported['codebase-memory'] || competitorTemplate('codebase-memory'),
+    serena: imported.serena || competitorTemplate('serena'),
+  }
+  return {
+    schemaVersion: 'weavatrix.agent-task-benchmark.v1',
+    scope: 'deterministic task-to-symbol routing microbenchmark; not an end-to-end autonomous change benchmark',
+    weavatrix: {
+      status: cases.every((item) => item.success) ? 'PASS' : 'FAIL', cases,
+      metrics: {
+        taskSuccessRate: cases.filter((item) => item.success).length / cases.length,
+        falsePositiveRate: cases.reduce((sum, item) => sum + item.falsePositives, 0) / Math.max(1, cases.reduce((sum, item) => sum + item.selected.length, 0)),
+        estimatedOutputTokens: Math.ceil(Buffer.byteLength(output) / 4), durationMs: Number(durationMs.toFixed(3)),
+      },
+    },
+    competitors, independentComparison: independent,
+    comparisonStatus: independent.status === 'COMPLETE' ? 'COMPLETE' : 'INCOMPLETE',
+  }
+}
+
+function argument(name) {
+  const index = process.argv.indexOf(name)
+  return index >= 0 ? process.argv[index + 1] : null
+}
+
+if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
+  const report = runAgentTaskBenchmark({
+    competitorResults: argument('--competitor-results'),
+    independentResults: argument('--independent-results'),
+  })
+  process.stdout.write(`${JSON.stringify(report, null, 2)}\n`)
+  if ((process.argv.includes('--require-independent') || process.argv.includes('--require-competitors')) && report.comparisonStatus !== 'COMPLETE') process.exitCode = 1
+  if (report.weavatrix.status !== 'PASS') process.exitCode = 1
+}

package/skill/SKILL.md

@@ -38,15 +38,20 @@ Start from the task, not from the complete tool list:
 - **Review a branch/diff**: `change_impact` for changed-file blast radius; add
   `graph_diff base_ref=<merge-base>` for structural drift, then drill into exact candidates with
   `get_dependents` and `read_source`.
+- **Plan or verify a serious change**: call `verified_change phase=plan` with the natural-language
+  task and current diff/files. After editing, call it again with `phase=verify` and an immutable
+  `base_ref`. Treat `UNKNOWN` as missing proof, not success; inspect its blockers and unknowns instead
+  of manually orchestrating the lower-level graph, context, impact, ratchet, and test tools.
 - **Trace an API across repositories**: `list_known_repos` -> `trace_api_contract` with an explicit
   backend and client list; inspect each `graphReconciliation.buildMode` before using the verdict.
-- **Inspect exact symbol references**: start with `graph_stats`, then call `inspect_symbol` with an
-  exact node ID (or an unambiguous label). It spends a bounded point query beyond the broad overlay
-  cap and returns occurrence containers plus source context. Use `get_dependents` for transitive
+- **Inspect exact symbol references**: start with `graph_stats`, then call `context_bundle` with an
+  exact node ID (or an unambiguous label) for a compact definition, grouped relations, exact
+  re-export sites and source workset. Use `inspect_symbol` when the raw bounded point query is needed;
+  it spends evidence beyond the broad overlay cap. Use `get_dependents` for transitive
   graph impact and `find_dead_code` for bounded zero-reference
   candidates. Treat `PARTIAL`, `UNAVAILABLE`, `OFF`, or zero exact edges as incomplete evidence, not
   a compiler-exact result; `OFF` means the caller explicitly selected static-only mode. Java and Rust
-  providers are not bundled in 0.2.5, so their edges never become `EXACT_LSP` even when a mixed
+  providers are not bundled in 0.2.7, so their edges never become `EXACT_LSP` even when a mixed
   repository has a complete TypeScript/JavaScript overlay.
 - **Explore an architectural question**: use broad `query_graph` when entry points are unknown; its
   bootstrap/tool-execution ranking prefers production executables over docs, sites and fixtures.
@@ -86,16 +91,20 @@ Start from the task, not from the complete tool list:
   `.weavatrix.json` `classify.product`, for example `{"classify":{"product":["benchmarks/core/**"]}}`.
 - **Architectural queries**: broad bootstrap/tool-execution/routing questions rank conventional
   production and graph-declared entry points ahead of classified docs, sites, benchmarks and
-  fixtures. Inspect the returned seeds before trusting the traversal. Pass exact repository-relative
-  `seed_files` to `query_graph` when the intended entry points are already known.
+  fixtures. Production-first classification also applies during traversal. A class term in the
+  question enables only that class; `include_classified=true` enables all classified paths. Exact
+  non-product `seed_files` remain pinned. Unmatched unreferenced constant/field leaves are suppressed
+  unless `include_low_signal=true`. Inspect the returned seeds/policy before trusting the traversal.
 - **Coverage**: `coverage_map` reads an existing report. `unavailable` means no supported report was
   found, not 0% coverage; do not rank testing risk from that state.
 - **Local hot paths**: `hot_path_review` ranks parser-derived local syntax cost and reports graph
   fan-in/fan-out plus test evidence separately. `actualCoverage: NOT_AVAILABLE` is not zero coverage,
-  and the score is not profiler data or an interprocedural Big-O proof. Inspect its line evidence and
-  measure runtime before scheduling a performance rewrite.
+  and the score is not profiler data or an interprocedural Big-O proof. The default queue uses
+  `min_score=85` plus a narrow strong-local fallback; use `min_score=0` only for full diagnostics.
+  Inspect its line evidence and measure runtime before scheduling a performance rewrite.
 - **Audit completeness**: read `dependencyReport.status` plus its unused/missing counts, then
-  `checks.osv.status` and `checks.malware.status`. A dependency result from a partial graph is not a
+  each npm finding's `verification` (manifest, indexed source, scripts/config, unresolved dynamic
+  usage), then `checks.osv.status` and `checks.malware.status`. A dependency result from a partial graph is not a
   repository-wide clean zero. For OSV, `OK` is
   complete for the recorded dependency fingerprint; `PARTIAL` is incomplete or stale,
   `NOT_CHECKED` has no repository-specific result, and `ERROR` means the local check failed. Treat
@@ -120,8 +129,18 @@ Start from the task, not from the complete tool list:
   `offline`, absent from `pinned`, and available in a custom registration only when `retarget` is
   named. Concurrent non-mutating calls retain the graph/root snapshot with which they started.
 
+- **Test execution is separately authorized**: `verified_change` only executes named
+  `package.json` test/check/verify scripts when the call has `run_tests:true` and the server was
+  started with `WEAVATRIX_ALLOW_TEST_RUNS=1`. It never accepts arbitrary shell commands. Without
+  both gates it returns a plan or `UNKNOWN` evidence state.
+
 ## Recipes
 
+- **Proof-carrying refactor**: `verified_change task="..." phase=plan base_ref=<merge-base>` -> edit ->
+  `verified_change task="..." phase=verify base_ref=<same-ref> tests=[{"script":"test","args":[...]}]`.
+  Add `run_tests:true` only when test execution was authorized. `BLOCKED` means an evidenced ratchet
+  or test failure; `UNKNOWN` means at least one required proof is incomplete.
+
 - **Orient in the configured repo**: `module_map` → `list_communities` → `god_nodes`. Hub ranking
   is production-only by default; use `include_classified:true` only when tests/generated/build
   surfaces are deliberately part of the question.
@@ -151,7 +170,9 @@ Start from the task, not from the complete tool list:
   `run_audit category=unused debt=all` (or `base_ref=<merge-base> debt=new`) for unused exports and
   dependencies. Before deletion, run `read_source`, `get_dependents`, exact `search_code`, inspect
   framework discovery/annotations, package scripts, CLI/Docker/manifests, generated consumers and
-  test-only use, then run the repository tests. `REVIEW_REQUIRED` and `autoDelete:false` are hard
+  test-only use, then run the repository tests. Use `evidenceTier` and `remainingChecks` to order the
+  review; even `STRONG_STATIC_EVIDENCE` is exact zero-reference evidence, not deletion permission.
+  `REVIEW_REQUIRED` and `autoDelete:false` are hard
   safety semantics, not boilerplate. Use `.weavatrix-deps.json` entrypoints/nonRuntimeRoots only for
   verified conventions.
 - **API inventory**: `list_endpoints` (including Next.js App Router, Rust axum and actix-web).