weavatrix 0.2.6 → 0.2.8

package/README.md

@@ -5,7 +5,7 @@
 Grep sees text. Weavatrix sees structure. It builds a dependency graph of any local repository —
 files, symbols, and the imports/calls/inheritance connecting them — and serves it to Claude Code,
 Codex, or any MCP client: change impact, transitive dependents, health audit, clone detection,
-coverage mapping. **35 tools available; 32 enabled by the default offline profile. Local-first: with
+coverage mapping. **36 tools available; 33 enabled by the default offline profile. Local-first: with
 the defaults, no repository data leaves your machine.**
 
 - Website: [weavatrix.com](https://weavatrix.com)
@@ -29,6 +29,11 @@ answers grep can't produce:
   baseline graph without checking it out, then reports the structural delta: new module
   dependencies, broken or introduced import cycles, and symbols that lost their last caller.
 
+- *"Is this change actually safe?"* -> `verified_change` accepts the task plus current diff/files and
+  returns one proof-carrying `PASS`, `BLOCKED`, or `UNKNOWN`: compact edit contexts, exact-symbol
+  impact, bounded call-argument flow, Git graph drift, architecture/duplicate/API ratchets, and
+  affected-test evidence.
+
 ## Quick start
 
 Requires Node ≥ 18. One command:
@@ -108,7 +113,7 @@ An agent skill with recipes ships in [skill/SKILL.md](skill/SKILL.md) — instal
 
 **graph** — `graph_stats`, `get_node`, `get_neighbors`, `query_graph`, `god_nodes`,
 `shortest_path`, `get_community`, `list_communities`, `module_map`, `get_dependents`,
-`change_impact`, `git_history`, `graph_diff`, `get_architecture_contract`,
+`change_impact`, `verified_change`, `git_history`, `graph_diff`, `get_architecture_contract`,
 `prepare_change`. Runtime dependencies, TypeScript type-only coupling and language
 compile-only edges (Rust module/use, Java imports) are reported separately where that distinction
 changes the result. TypeScript type-space and value-space declarations keep distinct identities;
@@ -120,7 +125,7 @@ confirmed by its bundled `typescript-language-server` + TypeScript runtime to `E
 `CONFLICT` means evidence disagrees. `graph_stats` reports the provenance breakdown and the semantic
 provider's `COMPLETE`, `PARTIAL`, `UNAVAILABLE`, or `OFF` state; `OFF` means precision was explicitly
 disabled and only static evidence is active. Java and Rust language-server providers are not bundled
-in 0.2.6: their edges never become `EXACT_LSP`, even when a mixed repository reports a complete
+in 0.2.7: their edges never become `EXACT_LSP`, even when a mixed repository reports a complete
 TypeScript/JavaScript overlay.
 
 The bounded JS/TS provider is enabled by default for new graphs. Set `WEAVATRIX_PRECISION=off`
@@ -135,8 +140,9 @@ bounded TS/JS reference query, logical containers, impact and source excerpts),
 Express/Fastify/Nest/Flask/FastAPI/Go mux/Rust axum and actix-web …)
 
 **health** — `find_dead_code` (bounded review queue for statically unreferenced and test-only files,
-functions, methods, and symbols, with confidence/evidence and explicit public/framework/dynamic caveats),
-`run_audit` (unused files/exports/dependencies with an explicit manifest-check summary,
+functions, methods, and symbols, with evidence tiers, completed/remaining verification and explicit
+public/framework/dynamic caveats),
+`run_audit` (unused files/exports/dependencies with per-finding manifest/indexed-source/script/config verification,
 missing npm/Go/Python deps, runtime
 cycles, type-only/compile-only coupling, orphans, boundary rules, offline OSV vulnerabilities + typosquat +
 lockfile drift; accepts an immutable `base_ref`, `changed_files`, and `debt: new|existing|all` for
@@ -149,7 +155,9 @@ with separate graph/test risk), `verify_architecture`,
 `hot_path_review` ranks production symbols using parser-derived local time, memory, cyclomatic and
 call-site facts plus exact inside-loop allocation, copy, scan, sort and recursion evidence. Graph
 fan-in/fan-out and measured coverage (or explicitly labelled static test reachability) remain
-separate from local syntax cost. The ranking is not profiler data or interprocedural Big-O.
+separate from local syntax cost. Its focused default uses `min_score:85` plus a narrow strong-local
+fallback; pass `min_score:0` only when the full diagnostic queue is wanted. The ranking is not
+profiler data or interprocedural Big-O.
 
 **build** — `rebuild_graph` (reports the structural delta, keeps the prior state as
 `graph.prev.json`)
@@ -160,7 +168,8 @@ use `kinds:["method"]` or a `path` prefix to narrow it. `min_confidence:"low"` e
 public APIs and framework/dynamic/reflection-sensitive candidates with their warnings. For repository
 maintenance or branch debt, pair it with `run_audit category=unused debt=all` (or add an immutable
 `base_ref` with `debt=new`) to cover unused files, exports and dependencies. Neither tool authorizes
-deletion: inspect `read_source`, `get_dependents`, exact search, manifests/configuration and tests first.
+deletion: `STRONG_STATIC_EVIDENCE` means an exact zero-reference result, not permission to remove;
+follow each candidate's `remainingChecks`, then inspect source/dependents/configuration and run tests.
 
 `graph_diff` accepts `base_ref` (`HEAD~1`, `main`, `origin/main`, or another local Git ref) for a
 fresh baseline comparison. Without it, the tool compares against `graph.prev.json` saved by the last
@@ -172,6 +181,14 @@ manifest, alias, ignore/config, non-JS/TS, or unsafe merge cases deliberately fa
 rebuild. The result says whether freshness was `none`, `incremental`, or `full`. Graph artifacts stay
 in the per-user cache and never need to be committed to Git.
 
+`verified_change` is read-only by default. It may run only explicitly requested `package.json`
+scripts whose names are allowlisted as test/check/verify scripts, and only when both
+`run_tests:true` and `WEAVATRIX_ALLOW_TEST_RUNS=1` are present. It never accepts an arbitrary shell
+command. Cross-repository API proof is used only when the active profile includes `crossrepo`;
+without a configured architecture contract or complete evidence, the result is `UNKNOWN`, not a
+cosmetic pass. Its data-flow section maps bounded JS/TS call arguments to callee parameters; it is
+not CFG, value-propagation, or taint analysis.
+
 **retarget** *(included in `offline`; absent from `pinned`; every switch is an explicit local call)* —
 `open_repo`, `list_known_repos`; changes the active repository boundary
 
@@ -182,8 +199,11 @@ in the per-user cache and never need to be committed to Git.
 entry points. Resolved explicit seeds are exclusive by default; set `augment_seeds:true` only when
 question-derived seeds are also wanted. Broad bootstrap/tool-execution/routing questions now rank
 conventional executables and graph-declared production entry points ahead of site, documentation, benchmark
-and fixture matches. Broad ranking remains orientation evidence; use `seed_files` when the intended
-entry point is already known.
+and fixture matches. Production-first classification also applies during traversal, so unrelated
+tests/generated/docs/benchmarks do not leak back from a production seed; name a class in the
+question or set `include_classified:true` to include it. Unreferenced unmatched constant/field leaves
+are hidden unless `include_low_signal:true`. Broad ranking remains orientation evidence; use
+`seed_files` when the intended entry point is already known.
 
 **advisories** *(network, explicit opt-in)* — `refresh_advisories`
 
@@ -196,6 +216,52 @@ disclosed instead of silently guessed; and the server **hot-reloads its watched
 modules and catalog** when those files change — other MCP helpers and analysis engines require a
 reconnect.
 
+### 0.2.8 trust and precision patch
+
+- Dead-code review no longer labels a declaration `test-only` when it has a same-file production
+  use. Revision-bound positive evidence from an exact `inspect_symbol` point query also removes that
+  declaration from later dead-code candidates instead of remaining isolated in its cache.
+- Python dependency checks discover nested `requirements*.txt`, `requirements/*.txt`,
+  `pyproject.toml`, and `Pipfile` manifests and assign imports to their nearest manifest scope.
+  A root `test.py` is classified consistently as test code.
+- Endpoint extraction masks commented-out routes without shifting source offsets and rejects
+  primitive path-to-name maps such as `{'/.codex': 'claude'}` while retaining real route tables.
+- Rust symbols now expose concrete kinds, owner types, visibility/export state, selection ranges and
+  structural owner/member edges. This is parser evidence; Rust semantic/LSP precision remains
+  explicitly unavailable.
+- Natural-language retrieval honors explicit Rust, Python, TypeScript, JavaScript, Go, Java, and C#
+  intent in mixed repositories. Compact `verified_change` text now includes decisive exact reference
+  counts and bounded inbound caller names for every selected edit symbol.
+
+Full patch notes: [docs/releases/v0.2.8.md](docs/releases/v0.2.8.md).
+
+### 0.2.7 verified-change workflow
+
+- New `verified_change` composes task retrieval, exact symbol context, change impact, immutable Git
+  graph comparison, architecture/duplicate/API ratchets, and targeted-test evidence behind one
+  `PASS` / `BLOCKED` / `UNKNOWN` contract.
+- Intent-expanded graph retrieval is combined with exact changed-symbol seeds. A bounded JS/TS
+  interprocedural layer records call arguments and their callee parameters without claiming CFG or
+  taint completeness.
+- Test execution is a double opt-in and limited to existing test/check/verify package scripts;
+  default operation remains read-only.
+- Malware scanning now sweeps installed npm package roots instead of package-manager caches or
+  hosted release snapshots. Static heuristic findings are capped at `high`, expose unverified
+  execution/origin/lockfile/exposure state, and no longer prescribe secret rotation unless execution
+  or credential exposure has actually been confirmed.
+- `query_graph` and `verified_change` task retrieval now enforce production-first classification
+  through traversal/selection while retaining exact changed or pinned non-product symbols; query
+  output also suppresses unmatched constant/field leaves. `hot_path_review` defaults to a focused
+  85-point queue, while `min_score:0` remains the explicit full-diagnostic mode.
+- Dead-code candidates now carry evidence tiers and remaining checks; dependency findings carry
+  per-finding manifest plus indexed-source/script/config verification without authorizing removal.
+- `npm run benchmark:agent` reports routing success, false positives, token estimate and latency.
+  Codebase Memory and Serena results stay explicitly `MISSING` until independently collected data is
+  supplied under the [blind agent-change protocol](docs/agent-task-benchmark.md);
+  `benchmark:agent:release` fails closed without same-task change success, FP, token, and time results.
+
+Full patch notes: [docs/releases/v0.2.7.md](docs/releases/v0.2.7.md).
+
 ### 0.2.6 compact-context and TypeScript identity patch
 
 - New `context_bundle` turns a symbol into a bounded workset: definition, grouped inbound/outbound
@@ -466,7 +532,7 @@ vulnerability findings. This is where each capability comes from and how it is c
 | Capability alert | Why it exists | Activation and boundary |
 |---|---|---|
 | Network access | `refresh_advisories` sends pinned package names and versions to OSV; `pull_architecture_contract` sends an opaque repository UUID and receives an owner-approved contract; `sync_graph` sends a normalized repository label plus an allowlisted graph/evidence payload. Evidence is derived locally from source and manifests, but source bodies, snippets, absolute paths, environment values, credentials and Git remotes are excluded | `offline` and `pinned` expose no network tools. `osv` exposes only advisory refresh. `hosted`/`full` expose all three; every request still requires an explicit tool call, and hosted calls require `WEAVATRIX_SYNC_URL` |
-| Shell access | Local `git` powers staleness/change impact; `rg` accelerates search; the bundled TLS/tsserver process supplies bounded JS/TS semantic evidence; timed-out Windows child trees may be terminated | Used only by the corresponding local operation. The semantic provider is package-pinned, disables automatic type acquisition, and never invokes a repository binary, script, installer or `npx` |
+| Shell access | Local `git` powers staleness/change impact; `rg` accelerates search; the bundled TLS/tsserver process supplies bounded JS/TS semantic evidence; timed-out Windows child trees may be terminated; `verified_change` can optionally run an existing package test/check/verify script | The semantic provider never invokes repository code. Test execution separately requires `run_tests:true` plus `WEAVATRIX_ALLOW_TEST_RUNS=1`, rejects arbitrary commands and shell-sensitive arguments, and should be enabled only for trusted repository scripts |
 | Debug / dynamic loading | Cache-busted `import()` hot-reloads watched MCP tool entry modules; `createRequire` loads package metadata and parser dependencies | Loads files from the installed package; no `eval` |
 | Environment access | Reads `WEAVATRIX_*` configuration; ordinary local helpers inherit a credential-stripped environment, while TLS/tsserver receives only allowlisted OS/temp/locale values and a constrained executable path | `WEAVATRIX_SYNC_TOKEN` is removed from every child-process and worker environment. TLS/tsserver also receives no registry/proxy/cloud credentials or `NODE_OPTIONS` |
 | Filesystem access | Reads the active repository, graph, lockfiles and coverage reports; writes derived graphs and advisory/architecture caches | Realpath containment blocks traversal and symlink/junction escapes. The `pinned` profile removes `open_repo`; the default `offline` profile permits only explicit local switching. The optional malware dependency scan may inspect installed dependency caches such as GOPATH |

package/SECURITY.md

@@ -28,6 +28,11 @@ dependency caches such as GOPATH. `offline` permits repository switching only th
 local `open_repo` call; select `pinned` to remove that tool, the global repository listing, and
 cross-repository API tracing, holding a hard startup-repository boundary.
 
+Installed-package malware pattern matching is static heuristic evidence. It cannot confirm execution,
+package compromise, or credential exposure and therefore cannot emit `critical`; heuristic severity
+is capped at `high` with explicit `NOT_VERIFIED` runtime/origin/lockfile/exposure fields. Independently
+confirmed malicious-package advisories remain separate vulnerability evidence and may be critical.
+
 The JS/TS precision overlay is enabled by default for new graphs and runs the package-pinned `typescript-language-server` and
 TypeScript runtime as local child processes. It never resolves a repository executable, invokes
 `npx`, runs package scripts, or installs dependencies. Automatic type acquisition is disabled, and
@@ -46,6 +51,15 @@ bounded graph-work drain, and closes or tree-terminates TLS/tsserver before exit
 Set `WEAVATRIX_PRECISION=off` before server startup (or select `off` in the MCPB semantic-precision
 setting) to keep new graphs parser-only from their first build.
 
+`verified_change` is read-only by default. Its optional targeted-test step can execute only an
+existing `package.json` script whose name matches the bounded test/check/verify allowlist. The call
+must set `run_tests:true` and the server operator must separately set
+`WEAVATRIX_ALLOW_TEST_RUNS=1`; otherwise no repository script runs. The tool accepts no command or
+shell string, caps scripts/arguments/time, rejects shell-sensitive argument characters, and launches
+the selected package-manager script with the credential-stripped child environment. Repository test
+scripts are repository-controlled code and may have arbitrary side effects, so enable this flag only
+for repositories and scripts you trust.
+
 Network capabilities are split by purpose. `osv` adds only `refresh_advisories`, which sends pinned
 package names and versions to OSV.dev when called. `hosted` / `full` also expose `sync_graph` and
 `pull_architecture_contract`; both require a user-configured endpoint, and contract pull requires

package/docs/agent-task-benchmark.md

@@ -0,0 +1,58 @@
+# Independent agent-change benchmark
+
+The 0.2.7 release includes two deliberately separate measurements.
+
+`npm run benchmark:agent` is a deterministic local routing microbenchmark. It measures whether the
+task retriever selects the expected symbol, plus false-positive selections, estimated output tokens,
+and local latency. It is a regression test, not evidence that an autonomous code change succeeded.
+
+The release comparison is an independently executed, blind agent-change benchmark. The evaluator
+must run the same pinned tasks and repository commits three times: with Weavatrix, with Codebase
+Memory, and with Serena. The product author must not adjudicate or repair a run while it is active.
+
+## Required task protocol
+
+Each task must provide a repository commit, natural-language request, allowed commands, timeout, and
+machine-verifiable acceptance tests. Keep task prompts and acceptance tests identical across systems;
+randomize system order. Record the entire agent input/output token count and wall-clock duration.
+
+A run is successful only when the requested behavior is implemented, all acceptance tests pass, and
+the final diff stays inside the task scope. A false positive is an evidenced warning, impacted symbol,
+or required edit that an evaluator confirms is unrelated to the task. Record a count, including zero;
+do not silently discard noisy output.
+
+Use a mix of at least:
+
+- behavior changes with indirect callers;
+- API route/client changes;
+- refactors with a duplicate or architecture regression trap;
+- changes whose nearest relevant test is not in the edited file;
+- one negative task where the correct action is to refuse or return unknown.
+
+## Result contract
+
+Save the blind evaluator output as JSON:
+
+```json
+{
+  "schemaVersion": "weavatrix.agent-change-results.v1",
+  "evaluator": "independent organization or person",
+  "systems": {
+    "weavatrix": {"runs": [{"taskId": "change-auth", "success": true, "falsePositives": 0, "tokens": 1234, "durationMs": 4567}]},
+    "codebase-memory": {"runs": [{"taskId": "change-auth", "success": true, "falsePositives": 1, "tokens": 1400, "durationMs": 5000}]},
+    "serena": {"runs": [{"taskId": "change-auth", "success": false, "falsePositives": 2, "tokens": 1600, "durationMs": 7000}]}
+  }
+}
+```
+
+Every system must contain the same unique task IDs. Every run must report success, false positives,
+tokens, and duration. Then run:
+
+```sh
+npm run benchmark:agent -- --independent-results path/to/results.json
+npm run benchmark:agent:release -- --independent-results path/to/results.json
+```
+
+The first command reports change-success rate, false positives per task, median tokens, and median
+duration for all three systems. The release command fails unless the independent comparison is
+complete and valid. Missing or malformed competitor data is `INCOMPLETE`, never a green comparison.

package/docs/releases/v0.2.8.md

@@ -0,0 +1,82 @@
+# Weavatrix 0.2.8
+
+## Trust before surface area
+
+0.2.8 is a correctness release. It does not add another MCP tool. It fixes false positives and
+evidence hand-off failures found while running 0.2.7 against current JavaScript, TypeScript, Python,
+and Rust repositories.
+
+## Dead-code liveness
+
+- A production declaration is no longer classified as `test-only` merely because the same identifier
+  appears in tests. Same-file occurrences beyond the indexed declaration count as conservative
+  production-use evidence.
+- Exact positive point-query results are now revision/config fingerprint checked and consumed by
+  `find_dead_code`. If `inspect_symbol` found a reference, the symbol cannot remain in the dead queue.
+- Complete exact zero-reference point results can still raise internal candidates to strong static
+  evidence. Partial, stale, malformed, or configuration-drifted cache entries fail closed.
+
+These remain review candidates. Static absence never authorizes automatic deletion.
+
+## Python manifest ownership
+
+Python dependency collection now discovers nested `requirements*.txt`, `requirements/*.txt`,
+`pyproject.toml`, and `Pipfile` manifests. Each import is judged against its nearest ancestor Python
+manifest. A nested service therefore sees its own declared dependencies, while unrelated sibling code
+cannot inherit them accidentally.
+
+The conventional root filename `test.py` is now classified as test code by both path classification
+and dependency severity logic.
+
+## Endpoint false-positive guards
+
+Endpoint extraction masks line and block comments while preserving source offsets. Commented-out
+Express/Hono/Go/Rust/Java-style routes no longer enter the inventory.
+
+Direct object routes also require an executable handler value. Primitive path maps such as
+`{'/.codex': 'claude'}` are configuration data, not `ANY /.codex` endpoints.
+
+## Rust graph metadata
+
+Rust functions, methods, structs, enums, traits, types, modules, constants, statics, macros, and unions
+now carry concrete `symbol_kind` values. Impl/trait functions retain `member_of`, visibility, selection
+ranges and structural owner/member edges; public module declarations retain exported state.
+
+This improves graph navigation and dead-code classification but remains tree-sitter parser evidence.
+0.2.8 does not bundle rust-analyzer and does not claim exact Rust references.
+
+## Mixed-language retrieval and compact proof
+
+`query_graph` and workflows that reuse its seed ranking constrain explicit Rust, Python, TypeScript,
+JavaScript, Go, Java, or C# questions to matching source extensions. Multiple explicitly named
+languages form a union; exact `seed_files` continue to be the strongest pinning mechanism.
+
+Compact `verified_change` output now lists each selected symbol's exact/bounded reference occurrence
+count, referenced-file count, inbound-container count, and up to three inbound names. The short form
+therefore cannot hide that a supposedly unused symbol has a known caller.
+
+## Scope and remaining competitor gaps
+
+The local engine still has no vector embedding search, runtime trace ingestion, clone-to-ADR layer,
+Python/Rust/Java LSP integration, whole-program CFG, value propagation, or taint engine. Those are
+separate precision projects, not claims attached to this patch release. Independent end-to-end
+competitor scores also remain missing until collected under the blind benchmark protocol.
+
+## Verification
+
+- Full Node test suite: 487 tests, 484 passed, 3 platform skips, 0 failed.
+- Deterministic language/cross-repository/lifecycle benchmark: `PASS`.
+- Real pinned corpus: 6 of 6 repositories `PASS` across TypeScript, JavaScript, Python, Go, Java,
+  and Rust.
+- Agent routing microbenchmark: 3 of 3 expected symbols, 0 false positives. Independent competitor
+  comparison remains `INCOMPLETE` because no blind same-task result was supplied.
+- npm package dry-run succeeds for `weavatrix@0.2.8`; the portable MCPB validates, initializes its
+  staged TypeScript precision provider, passes the repository-plugin execution guard, and packs.
+- Direct dogfood checks confirm the original regressions: local `USER_HEADER` is not dead/test-only;
+  nested `autobot/requirements.txt` covers `slack_bolt`; commented `DELETE /:id` and primitive
+  `ANY /.codex` are absent; explicit Rust retrieval returns only `.rs` seeds.
+
+## Tool catalog
+
+0.2.8 exposes the same 36 MCP tools as 0.2.7. The default offline profile exposes 33; pinned exposes
+30. Network tools remain absent from offline and pinned profiles.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "weavatrix",
-  "version": "0.2.6",
+  "version": "0.2.8",
   "type": "module",
   "description": "Weavatrix — code graph & blast-radius MCP server for AI coding agents: change impact, transitive dependents, health audit, duplicate detection, and coverage mapping over any local repository.",
   "author": "Sergii Ziborov <sergii.ziborov@gmail.com>",
@@ -24,6 +24,9 @@
     "bin",
     "src",
     "skill",
+    "scripts/run-agent-task-benchmark.mjs",
+    "docs/agent-task-benchmark.md",
+    "docs/releases/v0.2.8.md",
     "README.md",
     "SECURITY.md",
     "LICENSE"
@@ -34,6 +37,8 @@
     "benchmark:quick": "node scripts/run-benchmark.mjs --quick",
     "benchmark:real": "node scripts/run-real-benchmark.mjs",
     "benchmark:real:release": "node scripts/run-real-benchmark.mjs --require-all",
+    "benchmark:agent": "node scripts/run-agent-task-benchmark.mjs",
+    "benchmark:agent:release": "node scripts/run-agent-task-benchmark.mjs --require-independent",
     "build:mcpb": "node scripts/build-mcpb.mjs",
     "verify:release": "node scripts/verify-release.mjs"
   },

package/scripts/run-agent-task-benchmark.mjs

@@ -0,0 +1,119 @@
+import {readFileSync} from 'node:fs'
+import {pathToFileURL} from 'node:url'
+import {performance} from 'node:perf_hooks'
+import {retrieveTaskContext} from '../src/analysis/task-retrieval.js'
+
+function graphFixture() {
+  const nodes = [
+    {id: 'src/auth/session.js#validateSession@10', label: 'validateSession()', source_file: 'src/auth/session.js', symbol_kind: 'function'},
+    {id: 'src/http/router.js#registerRoutes@20', label: 'registerRoutes()', source_file: 'src/http/router.js', symbol_kind: 'function'},
+    {id: 'src/cache/store.js#readCache@5', label: 'readCache()', source_file: 'src/cache/store.js', symbol_kind: 'function'},
+    {id: 'src/auth/session.test.js#sessionTest@3', label: 'sessionTest()', source_file: 'src/auth/session.test.js', symbol_kind: 'function'},
+  ]
+  return {nodes, byId: new Map(nodes.map((node) => [node.id, node])), out: new Map(), inn: new Map()}
+}
+
+const CASES = [
+  {task: 'validate authentication session', expected: 'src/auth/session.js#validateSession@10'},
+  {task: 'register HTTP API routes', expected: 'src/http/router.js#registerRoutes@20'},
+  {task: 'read values from cache store', expected: 'src/cache/store.js#readCache@5'},
+]
+
+function competitorTemplate(name) {
+  return {name, status: 'MISSING', reason: 'supply same-task blind evaluator output with --independent-results <json>'}
+}
+
+function readCompetitors(path) {
+  if (!path) return {}
+  const parsed = JSON.parse(readFileSync(path, 'utf8'))
+  return parsed?.competitors && typeof parsed.competitors === 'object' ? parsed.competitors : {}
+}
+
+const SYSTEMS = ['weavatrix', 'codebase-memory', 'serena']
+const median = (values) => {
+  const sorted = values.slice().sort((left, right) => left - right)
+  const middle = Math.floor(sorted.length / 2)
+  return sorted.length % 2 ? sorted[middle] : (sorted[middle - 1] + sorted[middle]) / 2
+}
+
+function independentComparison(path) {
+  if (!path) return {status: 'MISSING', reason: 'supply blind evaluator output with --independent-results <json>'}
+  try {
+    const input = JSON.parse(readFileSync(path, 'utf8'))
+    if (input?.schemaVersion !== 'weavatrix.agent-change-results.v1') throw new Error('unsupported schemaVersion')
+    const taskSets = []
+    const metrics = {}
+    for (const name of SYSTEMS) {
+      const runs = input.systems?.[name]?.runs
+      if (!Array.isArray(runs) || !runs.length) throw new Error(`${name} has no runs`)
+      const ids = runs.map((run) => String(run.taskId || '')).sort()
+      if (ids.some((id) => !id) || new Set(ids).size !== ids.length) throw new Error(`${name} task IDs are missing or duplicated`)
+      for (const run of runs) {
+        if (typeof run.success !== 'boolean' || !Number.isFinite(run.falsePositives) || !Number.isFinite(run.tokens) || !Number.isFinite(run.durationMs)) {
+          throw new Error(`${name}/${run.taskId || '?'} has incomplete metrics`)
+        }
+      }
+      taskSets.push(ids.join('\0'))
+      metrics[name] = {
+        tasks: runs.length, changeSuccessRate: runs.filter((run) => run.success).length / runs.length,
+        falsePositiveRate: runs.reduce((sum, run) => sum + Math.max(0, run.falsePositives), 0) / runs.length,
+        medianTokens: median(runs.map((run) => run.tokens)), medianDurationMs: median(runs.map((run) => run.durationMs)),
+      }
+    }
+    if (new Set(taskSets).size !== 1) throw new Error('systems were not evaluated on the same task IDs')
+    return {status: 'COMPLETE', evaluator: String(input.evaluator || 'unspecified'), metrics}
+  } catch (error) {
+    return {status: 'INVALID', reason: error instanceof Error ? error.message : String(error)}
+  }
+}
+
+export function runAgentTaskBenchmark({competitorResults, independentResults} = {}) {
+  const graph = graphFixture()
+  const started = performance.now()
+  const cases = CASES.map((item) => {
+    const semanticSeeds = graph.nodes.filter((node) => item.task.toLowerCase().split(/\W+/).some((word) => word.length > 3 && String(node.id).toLowerCase().includes(word)))
+    const result = retrieveTaskContext(graph, {task: item.task, semanticSeeds, maxSymbols: 2})
+    const selected = result.selected.map((entry) => entry.id)
+    return {task: item.task, expected: item.expected, selected, success: selected[0] === item.expected, falsePositives: selected.filter((id) => id !== item.expected).length}
+  })
+  const durationMs = performance.now() - started
+  const output = JSON.stringify(cases)
+  const imported = readCompetitors(competitorResults)
+  const independent = independentComparison(independentResults)
+  const competitors = independent.status === 'COMPLETE' ? {
+    'codebase-memory': {name: 'codebase-memory', status: 'COMPLETE', metrics: independent.metrics['codebase-memory']},
+    serena: {name: 'serena', status: 'COMPLETE', metrics: independent.metrics.serena},
+  } : {
+    'codebase-memory': imported['codebase-memory'] || competitorTemplate('codebase-memory'),
+    serena: imported.serena || competitorTemplate('serena'),
+  }
+  return {
+    schemaVersion: 'weavatrix.agent-task-benchmark.v1',
+    scope: 'deterministic task-to-symbol routing microbenchmark; not an end-to-end autonomous change benchmark',
+    weavatrix: {
+      status: cases.every((item) => item.success) ? 'PASS' : 'FAIL', cases,
+      metrics: {
+        taskSuccessRate: cases.filter((item) => item.success).length / cases.length,
+        falsePositiveRate: cases.reduce((sum, item) => sum + item.falsePositives, 0) / Math.max(1, cases.reduce((sum, item) => sum + item.selected.length, 0)),
+        estimatedOutputTokens: Math.ceil(Buffer.byteLength(output) / 4), durationMs: Number(durationMs.toFixed(3)),
+      },
+    },
+    competitors, independentComparison: independent,
+    comparisonStatus: independent.status === 'COMPLETE' ? 'COMPLETE' : 'INCOMPLETE',
+  }
+}
+
+function argument(name) {
+  const index = process.argv.indexOf(name)
+  return index >= 0 ? process.argv[index + 1] : null
+}
+
+if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
+  const report = runAgentTaskBenchmark({
+    competitorResults: argument('--competitor-results'),
+    independentResults: argument('--independent-results'),
+  })
+  process.stdout.write(`${JSON.stringify(report, null, 2)}\n`)
+  if ((process.argv.includes('--require-independent') || process.argv.includes('--require-competitors')) && report.comparisonStatus !== 'COMPLETE') process.exitCode = 1
+  if (report.weavatrix.status !== 'PASS') process.exitCode = 1
+}

package/skill/SKILL.md

@@ -38,6 +38,10 @@ Start from the task, not from the complete tool list:
 - **Review a branch/diff**: `change_impact` for changed-file blast radius; add
   `graph_diff base_ref=<merge-base>` for structural drift, then drill into exact candidates with
   `get_dependents` and `read_source`.
+- **Plan or verify a serious change**: call `verified_change phase=plan` with the natural-language
+  task and current diff/files. After editing, call it again with `phase=verify` and an immutable
+  `base_ref`. Treat `UNKNOWN` as missing proof, not success; inspect its blockers and unknowns instead
+  of manually orchestrating the lower-level graph, context, impact, ratchet, and test tools.
 - **Trace an API across repositories**: `list_known_repos` -> `trace_api_contract` with an explicit
   backend and client list; inspect each `graphReconciliation.buildMode` before using the verdict.
 - **Inspect exact symbol references**: start with `graph_stats`, then call `context_bundle` with an
@@ -47,7 +51,7 @@ Start from the task, not from the complete tool list:
   graph impact and `find_dead_code` for bounded zero-reference
   candidates. Treat `PARTIAL`, `UNAVAILABLE`, `OFF`, or zero exact edges as incomplete evidence, not
   a compiler-exact result; `OFF` means the caller explicitly selected static-only mode. Java and Rust
-  providers are not bundled in 0.2.6, so their edges never become `EXACT_LSP` even when a mixed
+  providers are not bundled in 0.2.7, so their edges never become `EXACT_LSP` even when a mixed
   repository has a complete TypeScript/JavaScript overlay.
 - **Explore an architectural question**: use broad `query_graph` when entry points are unknown; its
   bootstrap/tool-execution ranking prefers production executables over docs, sites and fixtures.
@@ -87,16 +91,20 @@ Start from the task, not from the complete tool list:
   `.weavatrix.json` `classify.product`, for example `{"classify":{"product":["benchmarks/core/**"]}}`.
 - **Architectural queries**: broad bootstrap/tool-execution/routing questions rank conventional
   production and graph-declared entry points ahead of classified docs, sites, benchmarks and
-  fixtures. Inspect the returned seeds before trusting the traversal. Pass exact repository-relative
-  `seed_files` to `query_graph` when the intended entry points are already known.
+  fixtures. Production-first classification also applies during traversal. A class term in the
+  question enables only that class; `include_classified=true` enables all classified paths. Exact
+  non-product `seed_files` remain pinned. Unmatched unreferenced constant/field leaves are suppressed
+  unless `include_low_signal=true`. Inspect the returned seeds/policy before trusting the traversal.
 - **Coverage**: `coverage_map` reads an existing report. `unavailable` means no supported report was
   found, not 0% coverage; do not rank testing risk from that state.
 - **Local hot paths**: `hot_path_review` ranks parser-derived local syntax cost and reports graph
   fan-in/fan-out plus test evidence separately. `actualCoverage: NOT_AVAILABLE` is not zero coverage,
-  and the score is not profiler data or an interprocedural Big-O proof. Inspect its line evidence and
-  measure runtime before scheduling a performance rewrite.
+  and the score is not profiler data or an interprocedural Big-O proof. The default queue uses
+  `min_score=85` plus a narrow strong-local fallback; use `min_score=0` only for full diagnostics.
+  Inspect its line evidence and measure runtime before scheduling a performance rewrite.
 - **Audit completeness**: read `dependencyReport.status` plus its unused/missing counts, then
-  `checks.osv.status` and `checks.malware.status`. A dependency result from a partial graph is not a
+  each npm finding's `verification` (manifest, indexed source, scripts/config, unresolved dynamic
+  usage), then `checks.osv.status` and `checks.malware.status`. A dependency result from a partial graph is not a
   repository-wide clean zero. For OSV, `OK` is
   complete for the recorded dependency fingerprint; `PARTIAL` is incomplete or stale,
   `NOT_CHECKED` has no repository-specific result, and `ERROR` means the local check failed. Treat
@@ -121,8 +129,18 @@ Start from the task, not from the complete tool list:
   `offline`, absent from `pinned`, and available in a custom registration only when `retarget` is
   named. Concurrent non-mutating calls retain the graph/root snapshot with which they started.
 
+- **Test execution is separately authorized**: `verified_change` only executes named
+  `package.json` test/check/verify scripts when the call has `run_tests:true` and the server was
+  started with `WEAVATRIX_ALLOW_TEST_RUNS=1`. It never accepts arbitrary shell commands. Without
+  both gates it returns a plan or `UNKNOWN` evidence state.
+
 ## Recipes
 
+- **Proof-carrying refactor**: `verified_change task="..." phase=plan base_ref=<merge-base>` -> edit ->
+  `verified_change task="..." phase=verify base_ref=<same-ref> tests=[{"script":"test","args":[...]}]`.
+  Add `run_tests:true` only when test execution was authorized. `BLOCKED` means an evidenced ratchet
+  or test failure; `UNKNOWN` means at least one required proof is incomplete.
+
 - **Orient in the configured repo**: `module_map` → `list_communities` → `god_nodes`. Hub ranking
   is production-only by default; use `include_classified:true` only when tests/generated/build
   surfaces are deliberately part of the question.
@@ -152,7 +170,9 @@ Start from the task, not from the complete tool list:
   `run_audit category=unused debt=all` (or `base_ref=<merge-base> debt=new`) for unused exports and
   dependencies. Before deletion, run `read_source`, `get_dependents`, exact `search_code`, inspect
   framework discovery/annotations, package scripts, CLI/Docker/manifests, generated consumers and
-  test-only use, then run the repository tests. `REVIEW_REQUIRED` and `autoDelete:false` are hard
+  test-only use, then run the repository tests. Use `evidenceTier` and `remainingChecks` to order the
+  review; even `STRONG_STATIC_EVIDENCE` is exact zero-reference evidence, not deletion permission.
+  `REVIEW_REQUIRED` and `autoDelete:false` are hard
   safety semantics, not boilerplate. Use `.weavatrix-deps.json` entrypoints/nonRuntimeRoots only for
   verified conventions.
 - **API inventory**: `list_endpoints` (including Next.js App Router, Rust axum and actix-web).

package/src/analysis/allowed-test-runner.js

@@ -0,0 +1,59 @@
+import {existsSync, readFileSync} from 'node:fs'
+import {join} from 'node:path'
+import {createRepoBoundary} from '../repo-path.js'
+import {runCommand} from '../process.js'
+
+const SAFE_SCRIPT = /^(?:test(?::|$)|(?:check|verify)(?::|$)|[^:]+:(?:test|check|verify)(?::|$))/i
+const UNSAFE_SHELL_ARG = /[\0\r\n&|<>^%!`\"]/
+const tail = (value, limit = 8000) => String(value || '').slice(-limit)
+
+function manifestAt(repoRoot) {
+  const resolved = createRepoBoundary(repoRoot).resolve('package.json')
+  if (!resolved.ok) return null
+  try { return JSON.parse(readFileSync(resolved.path, 'utf8')) } catch { return null }
+}
+
+function packageManager(repoRoot) {
+  if (existsSync(join(repoRoot, 'pnpm-lock.yaml'))) return 'pnpm'
+  if (existsSync(join(repoRoot, 'yarn.lock'))) return 'yarn'
+  return 'npm'
+}
+
+export function validateTestRequests(repoRoot, requests = []) {
+  const manifest = manifestAt(repoRoot)
+  if (!manifest) return {ok: false, reason: 'package.json is missing or unreadable', tests: []}
+  const tests = []
+  for (const request of requests.slice(0, 5)) {
+    const script = String(request?.script || '')
+    const args = Array.isArray(request?.args) ? request.args.slice(0, 40).map(String) : []
+    if (!SAFE_SCRIPT.test(script)) return {ok: false, reason: `script ${script || '(missing)'} is outside the test/check/verify allowlist`, tests}
+    if (!Object.hasOwn(manifest.scripts || {}, script)) return {ok: false, reason: `package.json has no script named ${script}`, tests}
+    if (args.some((arg) => arg.length > 300 || UNSAFE_SHELL_ARG.test(arg))) return {ok: false, reason: `script ${script} has an invalid or shell-sensitive argument`, tests}
+    tests.push({script, args})
+  }
+  return {ok: true, tests, packageManager: packageManager(repoRoot)}
+}
+
+export async function runAllowedTests(repoRoot, requests = [], {enabled = false, timeoutMs = 60_000} = {}) {
+  const checked = validateTestRequests(repoRoot, requests)
+  if (!checked.ok) return {state: 'BLOCKED', reason: checked.reason, results: []}
+  if (!checked.tests.length) return {state: 'NOT_REQUESTED', reason: 'no package scripts were requested', results: []}
+  if (!enabled || process.env.WEAVATRIX_ALLOW_TEST_RUNS !== '1') return {
+    state: 'DISABLED', reason: 'set WEAVATRIX_ALLOW_TEST_RUNS=1 and pass run_tests:true to execute allowlisted package scripts',
+    plan: checked.tests, results: [],
+  }
+  const results = []
+  const timeout = Math.max(1000, Math.min(300_000, Number(timeoutMs) || 60_000))
+  for (const test of checked.tests) {
+    const start = Date.now()
+    const separator = checked.packageManager === 'yarn' ? [] : ['--']
+    try {
+      const run = await runCommand(checked.packageManager, ['run', test.script, ...separator, ...test.args], {cwd: repoRoot, timeoutMs: timeout})
+      results.push({script: test.script, status: run.exitCode === 0 ? 'PASS' : 'FAIL', exitCode: run.exitCode, durationMs: Date.now() - start, stdoutTail: tail(run.stdout), stderrTail: tail(run.stderr)})
+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error)
+      results.push({script: test.script, status: /timed out/i.test(message) ? 'TIMEOUT' : 'FAIL', exitCode: null, durationMs: Date.now() - start, stdoutTail: '', stderrTail: tail(message)})
+    }
+  }
+  return {state: results.every((result) => result.status === 'PASS') ? 'PASS' : 'FAIL', packageManager: checked.packageManager, results}
+}