woods 2.0.0.beta2 → 2.0.0.beta3

data/docs/MCP_SERVERS.md

@@ -27,6 +27,15 @@ bin/rails woods:validate
 bin/rails woods:stats
 ```
 
+For embedding-free ranked retrieval, start Index MCP with
+`WOODS_RETRIEVAL_MODE=lexical`. This opt-in reads the published extraction units;
+it does not probe providers or load vectors. `woods_status.retriever.mode` reports
+`lexical`, and inactive embedding fields are `null`. The default semantic mode
+keeps its existing embedding setup and failure behavior. See
+[retrieval modes](RETRIEVAL_GUIDE.md#embedding-free-lexical-retrieval) for scoring,
+query limits and measured tradeoffs. Both packaged stdio and HTTP launches honor
+the setting; put it in the MCP process's environment, not just a Rails initializer.
+
 The stdio server can then run outside Rails. Point it at the index root (`tmp/woods/` by default), not at an internal generation or payload directory.
 
 ### Configure a stdio client
@@ -47,6 +56,10 @@ Prefer the application's bundle and a project-scoped configuration:
 
 `woods-mcp-start` checks that the directory and published manifest exist, then replaces itself with `woods-mcp`. It does not install dependencies or restart a crashed process.
 
+`woods_status.index.woods_version` identifies the last publisher of the served
+manifest; `server.version` identifies the running MCP reader. Missing writer
+provenance is `null`. See [manifest writer provenance](PUBLISHED_INDEX.md#manifest-writer-provenance).
+
 You can launch the server directly when the client already handles preflight:
 
 ```bash
@@ -57,6 +70,12 @@ Keep stdout reserved for MCP protocol messages. Diagnose startup failures from s
 
 ### Client configuration locations
 
+Supporting development builds offer preview/apply/update/remove ownership for
+Claude Code project or explicit user configuration. See
+[managed configuration](AGENT_SETUP.md#managed-claude-code-configuration) for
+`woods-agent-config`, host/Compose preflight, conflict handling, and recovery.
+Manual configuration remains available for older gems and other clients.
+
 MCP clients expose project or user-level server settings in different locations. Use project scope when available, preserve the `command`, `args`, and absolute `cwd` semantics above, and translate only the surrounding client-specific format. Woods is model-independent: compatibility depends on the client supporting MCP stdio or Streamable HTTP, not on whether the connected model is from OpenAI, Anthropic, Google, xAI, or another provider.
 
 Client configuration formats can change independently of Woods. If a client rejects otherwise valid JSON, check that client's current MCP documentation.
@@ -99,6 +118,27 @@ Reconnect the client, then call:
 
 Prefer a real MCP client's connection flow over a hand-written JSON-RPC pipe. Modern MCP 2026-07-28 requests carry per-request protocol metadata and can use `server/discover` without an initialization handshake; older clients still use `initialize`. A valid raw smoke test must implement one complete flow rather than sending an isolated `tools/list` or `tools/call` request.
 
+### Initialization guidance
+
+The Index Server supplies a short, client-neutral `instructions` field through
+the SDK's `initialize` response and modern `server/discover`. It describes the
+status → discovery → inspection → bounded traversal → source-verification
+workflow, and lists only the tools actually registered by this server.
+Instructions are stable for unchanged tool registration and bounded to 2,048
+UTF-8 bytes across supported configurations. Building the text does not probe
+providers, extract code, or write configuration.
+
+Registration alone does not establish retrieval readiness: check `woods_status`
+before using `codebase_retrieve`, including after reload. The guidance grants
+no extraction, configuration-change, or Console authorization. Detailed usage
+belongs in the [agent guide](AGENT_GUIDE.md).
+
+This addition is unreleased after `2.0.0.beta2`. The SDK omits `instructions`
+when negotiating protocol `2024-11-05`; that behavior is preserved. Older gems,
+legacy clients, and clients that do not show server instructions can use the
+agent guide or investigation skill. Leave protocol negotiation enabled rather
+than pinning a newer version solely to obtain guidance.
+
 ### Tools (29 — 14 registered in the packaged default)
 
 The Index Server defines 29 schemas across core and conditional capabilities. The normal packaged executable registers the 14 tools below; the remaining schemas require the specialized wiring described afterward.
@@ -108,8 +148,8 @@ The Index Server defines 29 schemas across core and conditional capabilities. Th
 | `woods_status` | Index health, generation, counts, and retrieval readiness |
 | `search` | Discover identifiers by regex, prefix, suffix, source, or metadata |
 | `lookup` | Fetch one exact unit with source, metadata, and relationships |
-| `dependencies` | Traverse what a unit depends on (`depth`, `types`, `via` narrow; `limit`, `offset` page) |
-| `dependents` | Traverse what depends on a unit (`depth`, `types`, `via` narrow; `limit`, `offset` page) |
+| `dependencies` | Traverse what a unit depends on (`depth`, `types`, `via` narrow; `max_nodes`, `max_edges` budget work; `limit`, `offset` page) |
+| `dependents` | Traverse what depends on a unit (`depth`, `types`, `via` narrow; `max_nodes`, `max_edges` budget work; `limit`, `offset` page) |
 | `structure` | Summarize structural relationships around a unit |
 | `trace_flow` | Follow a request, job, mail, or other execution flow |
 | `framework` | Inspect relevant Rails or installed gem source |
@@ -118,7 +158,29 @@ The Index Server defines 29 schemas across core and conditional capabilities. Th
 | `domain_clusters` | Discover connected domains in the graph |
 | `pagerank` | Find structurally central units |
 | `reload` | Reload a newly published generation without restarting the client |
-| `codebase_retrieve` | Natural-language retrieval; returns a configuration error until embeddings exist |
+| `codebase_retrieve` | Natural-language retrieval with embeddings or explicit lexical mode over extraction output |
+
+When an identifier appears in multiple extraction types, `framework` reads its
+framework-source bucket and `recent_changes` reads each selected type bucket.
+Their paths and metadata belong to that selected bucket. When session tracing
+is configured, newly recorded requests use the dispatched controller's runtime
+class name; the fallback for requests without an instance respects Rails acronym
+inflections. Existing trace records are unchanged. Controller lookup and root
+outgoing-edge selection preserve the controller type. Downstream references and the shared context pool still use
+bare identifiers. If a dependency encountered within the requested depth has
+multiple published types, `session_trace` returns an `ambiguous_identity` tool
+error naming the identifier and candidate types, with no partial context. This
+also prevents an earlier dependency from occupying a later controller’s context
+key. Unrelated collisions do not block a trace, and a known controller root keeps
+its controller identity. A controller absent from the index remains in the
+timeline without a source reference, so another type cannot fill that reference.
+Candidate discovery and source reads use one pinned
+generation. Corrupt or missing listed artifacts retain the `internal_error`
+failure boundary; they do not prove uniqueness or become `ambiguous_identity` errors.
+Use `depth: 0` for the request timeline, or inspect candidates with typed `lookup`
+calls. Re-extraction does not remove a legitimate cross-type collision. Successful
+traces retain their existing identifiers and response shape; target identity has
+not been migrated globally. These corrections are unreleased after `2.0.0.beta2`.
 
 The server also exposes MCP resources and resource templates for indexed units. Tool descriptions returned by MCP are the parameter-level source of truth; [Agent guide](AGENT_GUIDE.md) explains selection strategy.
 
@@ -130,6 +192,126 @@ error and continues serving the previous aligned generation; it never swaps in a
 partial or empty replacement. Grant write access for live reloads, or restart the MCP
 process after publishing a new embedded index.
 
+### Search completeness
+
+Search responses retain `query`, `result_count`, and `results`; `result_count`
+is the number returned, not an estimated total. The additive `completeness`
+object describes the requested types, literal filters, and fields in the pinned
+generation. This contract is unreleased after `2.0.0.beta2`.
+
+| `reason` | `status` | `has_more` | `total_matches` |
+|---|---|---|---|
+| `exhausted` | `complete` | `false` | Exact count |
+| `result_limit` | `partial` | `true` | `null` (unknown) |
+| `scan_budget` or `regex_timeout` | `partial` | `null` (unknown) | `null` (unknown) |
+
+`matched_lower_bound` counts distinct observed `(type, identifier)` matches,
+including at most one lookahead match beyond `limit`. A result-limit response
+therefore establishes another match; an exactly full page can instead be
+complete if the requested domain is exhausted. Deep lookahead shares
+`WOODS_SEARCH_MAX_SCAN` with the initial scan and retains round-robin scanning
+across types. Search does not count the entire omitted tail or offer pagination.
+The existing `types` filter and result labels name directory families:
+`rails_source` includes both Rails and gem source units. Deep reads accept those
+two stored types only in that shared directory; `lookup` and lexical retrieval
+retain the unit's actual `rails_source` or `gem_source` type.
+
+All partial responses retain `partial: true` and include a narrowing `hint`.
+JSON exposes these fields; Markdown, plain text, and Claude formats label the
+returned count, stopping reason, known/unknown remainder, and total explicitly.
+Narrow `types`, literal `exact_prefix`/`exact_suffix`, or deep `fields` before
+using discovery as exhaustive evidence. Completeness applies to this index and
+query domain, not to unindexed application code.
+
+Detected missing, unreadable, or corrupt artifacts remain `isError: true` with
+`_meta.error_code: "corrupt_artifact"`. Their `_meta.completeness` has
+`status: "unknown"`, `reason: "unreadable_or_corrupt_source"`, and `null` for
+`has_more`, `total_matches`, and `matched_lower_bound`; no successful empty
+result is substituted. Inspect `woods_status` and run `woods:validate`.
+
+### Dependency traversal budgets
+
+`dependencies` and `dependents` walk breadth-first in stored graph order. The
+walk defaults to `max_nodes: 1000` (including the root) and `max_edges: 10000`;
+callers can select 1–10,000 nodes and 1–100,000 edge checks. The node budget
+counts distinct nodes admitted after filters. Every candidate edge is charged
+before filtering, including duplicates, cycles, and the forward-edge checks
+needed to match a reverse `via` filter. Thus restrictive filters cannot bypass
+the edge budget. Nodes at the requested `depth` are recorded without reading
+their adjacency lists.
+
+When further expansion would exceed a budget, JSON reports `partial: true`,
+`partial_reason: "node_budget"` or `"edge_budget"`, and `traversal_budget` with
+`max_nodes`, `max_edges`, `visited_nodes`, and `visited_edges`. Text renderers
+also identify the partial traversal. Already discovered nodes remain in the
+answer, but an empty `deps` array in a partial answer does not prove a leaf.
+Exact-budget walks that finish all requested work are complete and have no
+`partial` marker.
+
+`limit` (default 50) and `offset` only page that discovered result; they never
+change the walk budget or depth. On a partial traversal, `nodes_total`, when
+present for pagination, counts the discovered prefix, **not the full reachable
+graph**. Paging beyond that prefix stays partial. To explore more, narrow
+`depth`/`types`/`via`, choose another root, or increase the traversal budget within
+its maximum. Keep the root, filters, budgets, and published generation unchanged
+for stable pages. No wall-clock deadline is used, so cutoffs are deterministic.
+
+Budgets cover traversal work after per-generation graph loading and cache
+preparation (JSON parsing, typed-edge normalization, node types and database
+metadata). They do not cap that initial load, elapsed time, or total process
+memory. These arguments are unreleased in Woods 2.0.0.beta2; check the connected
+server's tool schema before sending them to an older installation.
+
+### Traversal explanations
+
+Supporting development versions accept `explain: true` on `dependencies` and
+`dependents`. Check the connected schema first; this option is unreleased after
+2.0.0.beta2. Omitted or false keeps the existing compact response.
+
+The additive `explanation` object contains:
+
+- `direction`: `forward` or `reverse`, plus the requested `root` identity.
+- `edges`: records keyed by response-local IDs such as `e0`. Every record keeps
+  the original **source → target** direction, even during reverse traversal.
+  `source` contains its recorded `identifier` and `type`; `target` contains its
+  identifier and the unique type when the published graph establishes one.
+  `via`, `through`, `through_db`, and `disable_joins` preserve recorded values;
+  absent legacy attributes are null (shown as unknown in text), including an
+  unrecorded `disable_joins` rather than an invented false value.
+- `witnesses`: one shortest breadth-first predecessor per admitted identifier,
+  keyed by identifier. Each has `parent`, `edge_id`, `impact` (`root`, `direct`,
+  or `transitive`), and `typed_path_complete`. Follow parent references to the
+  root to reconstruct one witness; alternative paths are not enumerated.
+
+A target name shared by several types has `type: null`,
+`resolution: "ambiguous"`, and sorted `candidate_types`. An unresolved target has
+`resolution: "unresolved"` and an empty candidate list. Forward artifacts do not
+record target types, so the response cannot choose among candidates. A witness
+through an ambiguous or unresolved identity sets `typed_path_complete: false`;
+it describes identifier-level reachability, never a uniquely typed path.
+`types` filters retain the compact traversal's identifier-level semantics: any
+registered type can qualify a name, while edge evidence keeps its actual source
+owner. Multiple relationship kinds between the same endpoints remain separate.
+
+Direct witnesses establish a recorded root relationship; transitive witnesses
+represent inferred downstream reachability through recorded relationships.
+Neither establishes observed execution, confidence, call order, or test coverage.
+
+Node pagination retains required ancestor witnesses once, marked `context: true`
+when outside the page; returned rows have `context: false`. Context records do
+not increase the result-row count. Page evidence retains the witness edges and
+other observed relationships among its visible/context endpoints; an empty page
+has empty edge/witness maps. Edge IDs are local to this traversal response.
+
+All examined evidence shares the existing edge budget, before `via`/`types`
+filtering. Current `reverse_via` buckets allow direct reverse evidence lookup;
+legacy recovery charges each reverse candidate and every inspected forward edge.
+The shared predecessor forest and emitted records remain bounded by admitted
+nodes and inspected edges. Per-generation JSON loading and the cached
+O(nodes + variants) ownership/type preparation are outside the walk budget;
+explanation mode never flattens all forward edges as per-request preparation.
+Partial traversal and pagination metadata retain the budget contract above.
+
 ### Conditional Index capabilities
 
 The Ruby server builder contains 15 additional schemas for sessions, pipeline operations, retrieval feedback, temporal snapshots, and Notion sync. They register only when their required collaborators or configuration are wired.
@@ -140,6 +322,7 @@ The normal packaged executable does not wire pipeline-operator or feedback-store
 
 Use HTTP only for a deliberate shared or remote deployment. It expands the network boundary and requires authentication, origin restrictions, and TLS termination. Follow [MCP HTTP transport](MCP_HTTP_TRANSPORT.md); do not translate the stdio example into an unauthenticated public listener.
 
+
 ## Console Server
 
 The Console Server launches a Rails process through direct, Docker, or SSH connection configuration. It reads live data and must be treated as a separate security decision.
@@ -151,11 +334,14 @@ Console MCP is disabled by default because it reads live application data. Enabl
 ```ruby
 Woods.configure do |config|
   config.console_mcp_enabled = true
-  config.console_mcp_token = ENV["WOODS_CONSOLE_MCP_TOKEN"]
+  config.console_mcp_http_enabled = false # stdio-only
 end
 ```
 
-The bearer token authenticates HTTP clients and is not sent over stdio. Rails still validates Console configuration while booting: production requires a token of at least 32 characters whenever Console is enabled, including for a stdio-only client. Keep it in the application's secret store, not in the initializer.
+This explicitly disables HTTP Console while retaining stdio access; no HTTP
+token is needed at boot. Existing configurations default to HTTP enabled.
+For HTTP deployment, enable the HTTP flag and configure its token, origins
+and TLS using the [Console setup guide](CONSOLE_MCP_SETUP.md#option-c-http-rack-middleware).
 
 Without a console connection file, the executable then launches the Rails task directly from its `cwd`:
 
@@ -229,3 +415,33 @@ Report vulnerabilities privately through [SECURITY.md](../SECURITY.md).
 5. Check [Troubleshooting](TROUBLESHOOTING.md) for the exact stderr message.
 
 For agent query behavior after connection, continue to [Agent guide](AGENT_GUIDE.md).
+
+### Explicit retrieval and discovery scope
+
+On a server whose tool schema advertises them, `packages` and `source_paths` narrow
+`search` and `codebase_retrieve` before candidate limits. These are per-call
+arguments, not configuration settings. Inspect applied scope and completeness;
+a narrow graph query can omit relevant cross-boundary dependencies. See the
+[scope contract](RETRIEVAL_GUIDE.md#explicit-package-and-source-path-scopes) for
+root/nested ownership, path normalization, errors, storage support, and cost.
+
+### Source freshness in status
+
+`woods_status` accepts optional `source_check: "quick"` (default, 250ms scan) or
+`"deep"` (five seconds). `index.source_freshness` describes the served generation
+as `current`, `drifted` or `unknown`; missing source/key and incomplete capture
+never count as current. No Rails initialization or provider call is needed.
+See [source freshness](SOURCE_FRESHNESS.md) for scope, private-key handling and
+fresh-process extraction. Existing HEAD/dirty fields remain separate diagnostics.
+
+### Explicit source evidence modes
+
+When advertised by the installed schema, `lookup` and `codebase_retrieve` accept
+`evidence: 'compact'` or `'outline'`; omitted/`'full'` preserves existing behavior.
+Retrieval uses its original query. Compact lookup accepts optional `query` and an
+estimated `budget` (default 2000); full lookup remains complete. `lookup` also
+accepts an actual `type` and a `source_sha256` guard for typed, byte-verified
+follow-up from an excerpt. Compact modes cannot be combined with metadata-only
+lookup controls. Structured provenance stays within the existing closed output
+schema's `data` field. Read the [evidence contract](RETRIEVAL_GUIDE.md#compact-published-evidence-and-api-outlines)
+before interpreting published line ranges as physical source locations.

data/docs/MCP_TOOL_COOKBOOK.md

@@ -255,13 +255,24 @@ The `metadata.inlined_concerns` array lists which concerns were resolved:
 }
 ```
 
-**What you'll get:** A BFS tree of everything that references `User`, controllers, services, jobs, mailers, up to 2 hops out. Set `depth: 1` for direct dependents only.
+**What you'll get:** A BFS traversal of units that reference `User`, such as controllers, services, jobs, and mailers, up to 2 hops out. Set `depth: 1` for direct dependents only.
 
-The answer is bounded to 50 nodes. When it is cut, the response ends with a
+The answer is paged to 50 nodes by default. When it is cut, the response ends with a
 `Showing N of M (truncated)` line, the same one `graph_analysis` prints. Reach
 for `depth`, `types` and `via` first: they make the answer smaller. `limit` and
 `offset` only page what those leave, so a hub read one page at a time still
-costs every page.
+repeats the walk. A separate traversal budget can return `partial: true`;
+that marker means the reachable graph is incomplete even after the final page.
+See [traversal budgets](MCP_SERVERS.md#dependency-traversal-budgets) before
+changing `max_nodes` or `max_edges`.
+
+To explain why a row is affected, check the connected schema and add
+`"explain": true`. The response preserves source-to-target labels even while
+walking dependents. Its predecessor witnesses distinguish direct relationships
+from transitive inferred reachability, retain ancestor context across pages, and
+mark ambiguous types explicitly. See the
+[explanation contract](MCP_SERVERS.md#traversal-explanations); these witnesses are
+not proof of observed execution.
 
 To find only which jobs depend on `User`:
 
@@ -323,26 +334,30 @@ column.
 }
 ```
 
-**Example response:**
+**Example JSON data** (inside the MCP response envelope):
 
 ```json
-[
-  {
-    "identifier": "PaymentsController",
-    "type": "controller",
-    "file_path": "app/controllers/payments_controller.rb",
-    "metadata": {
-      "actions": ["create", "show", "webhook"],
-      "routes": [
-        { "verb": "POST", "path": "/payments", "action": "create" },
-        { "verb": "POST", "path": "/payments/webhook", "action": "webhook" }
-      ]
-    }
+{
+  "query": "payment",
+  "result_count": 1,
+  "results": [
+    { "identifier": "PaymentsController", "type": "controller", "match_field": "identifier" }
+  ],
+  "completeness": {
+    "status": "complete",
+    "reason": "exhausted",
+    "has_more": false,
+    "total_matches": 1,
+    "matched_lower_bound": 1
   }
-]
+}
 ```
 
-Search `source_code` when you want semantic matches, not just naming matches.
+Use `lookup` on the returned identifier for source, actions, and routes. Search
+`source_code` for textual matches beyond names. Supporting versions distinguish
+exact totals from a bounded result prefix; `partial` means this is discovery,
+not an exhaustive list. Completeness metadata is unreleased after `2.0.0.beta2`;
+see the [search contract](MCP_SERVERS.md#search-completeness).
 
 ---
 

data/docs/NOTION_INTEGRATION.md

@@ -34,6 +34,19 @@ Sync is incremental. A **sync manifest** (`<output_dir>/notion_sync_manifest.jso
 
 Manifest entries for models/columns that vanished from the current extraction are pruned, but **no Notion page is ever deleted** by the sync, there is no delete path. A renamed or removed model just leaves its old page in Notion untouched.
 
+Models and migration dates are loaded by `(identifier, type)`. A same-named
+PORO, library unit, or other extracted type cannot supply a model's table or
+column payload. Public identifiers, page titles, and manifest keys are unchanged.
+If a listed model or migration cannot be read with its exact identity, the
+affected sync refuses before mapping pages or pruning its manifest. Preserve
+the error, validate the index, and regenerate it before retrying; force sync
+does not bypass identity checks. Each public sync method, including standalone
+model or column sync, uses one validated generation snapshot through mapping
+and API calls. Native readers validate the published index before selecting
+model/migration payloads. Custom readers must provide complete published
+enumeration or complete per-bucket listings plus
+`find_unit(identifier, type:)` returning the requested identity.
+
 If the manifest is missing (first run, or a CI cache miss), the exporter falls back to the full lookup/create path for every page and rebuilds the manifest, correct, just more API calls than a steady-state run.
 
 ### Escape hatch

data/docs/OBSIDIAN_INTEGRATION.md

@@ -8,7 +8,7 @@ ways at once:
   Obsidian [Bases](https://help.obsidian.md/bases) table, and drill into a single unit's note with its
   dependencies and dependents as clickable wikilinks.
 - **By agents**: load the entire dependency topology from a single `_woods/` sidecar (one read,
-  no per-note fan-out), with a stable `id → note path` manifest for navigation.
+  no per-note fan-out), with a stable typed-unit → note-path manifest for navigation.
 
 Unlike the Notion and Unblocked exporters, this one writes **local files only**: there is no API
 token, no network call, and no rate limit. An Obsidian vault is just a folder.
@@ -95,6 +95,33 @@ Wikilinks are path-qualified with an alias (`[[models/Account|Account]]`): the t
 sanitized vault path so the link always resolves, and the alias shows the original identifier. The
 note's `# H1` carries the clean identifier so the sanitized filename never shows as the title.
 
+## Sidecar identity and schema versions
+
+A unit keeps its original `id` and `type` in note frontmatter. Different types may
+share an identifier: a database view and a factory called `reports` export to
+`database_views/reports.md` and `factories/reports.md`. Existing unambiguous note
+paths and display aliases stay unchanged.
+
+Check `schema_version` before consuming `_woods/manifest.json`:
+
+- **Version 1:** the graph has no cross-type identifier collisions. The existing
+  `notes[id]` and `paths[path]` maps are unchanged.
+- **Version 2:** `notes[id]` retains the graph's primary type (when exported), and
+  `variants` lists each additional exported unit as
+  `{ "identifier": "reports", "type": "factory", "path": "factories/reports.md" }`.
+  Combine primary entries and variants using **`(identifier, type)`** as identity.
+  `paths[path]` still gives the original identifier, so several paths may return
+  the same value. Resolve a path against both collections; `notes[id]` alone is
+  incomplete. A consumer supporting only version 1 must refuse version 2.
+
+The exporter reads each variant's own outgoing graph edges and derives incoming
+links from them. Persisted targets are bare identifiers: if a target names several
+types, the exporter omits that relationship from note links and counts it in a
+progress diagnostic. It does this even when one sibling is excluded or unreadable.
+Human association links use the same ambiguity rule. Bare PageRank and graph-analysis
+annotations are omitted for ambiguous identifiers, rather than assigned to either
+type. The verbatim graph sidecar retains all original records for inspection.
+
 ## The three visualizer surfaces
 
 | Surface | Best for | Notes |
@@ -114,7 +141,7 @@ credential):
 | `WOODS_OUTPUT` | `config.output_dir` (`tmp/woods`) | extraction directory to read from |
 | `WOODS_OBSIDIAN_VAULT` | `<output>/obsidian_vault` | where to write the vault |
 | `WOODS_OBSIDIAN_INCLUDE_SOURCE` | off | embed each unit's source code (credential-scrubbed) in its note |
-| `WOODS_OBSIDIAN_INCLUDE_FRAMEWORK` | off | include `rails_source` units (large; off by default) |
+| `WOODS_OBSIDIAN_INCLUDE_FRAMEWORK` | off | include `rails_source` and readable `gem_source` units (large; off by default) |
 | `WOODS_OBSIDIAN_FORCE_PURGE` | off | bypass the 30% mass-deletion guard during the stale-note sweep |
 
 ```bash
@@ -141,6 +168,28 @@ The exporter **fully regenerates** the vault on every run (no incremental manife
 cheap). Output is deterministic: re-running against an unchanged extraction produces byte-identical
 notes, so unchanged units never show up in a git diff.
 
+Before writing, Woods renders the output and checks **every destination**. An existing note or index
+must carry `woods_managed: true`; a `.woods-vault` sentinel alone never authorizes replacing your
+notes or settings. Conflicts stop the export before any writes or sweep, appear in `errors`, and make
+`woods:obsidian` exit nonzero. `WOODS_OBSIDIAN_FORCE_PURGE` does not bypass ownership checks.
+Child symlinks, directories in place of files, and special files are refused.
+
+Generated machine assets are tracked by SHA-256 in `_woods/ownership.json`: the three sidecar JSON
+files, three `.obsidian/` JSON settings files, and `Units.base`. Existing assets must match their
+recorded bytes or the newly generated bytes. This receipt does not change the public sidecar manifest
+schema. Modified settings are preserved by refusing the export; the sentinel is not an override.
+
+**Upgrading an older vault:** without a receipt, byte-identical generated assets can be adopted. Run
+once against the same extraction to establish ownership before updating the index. If the index has
+already changed, Woods may refuse the old sidecars because their ownership cannot be proved. Inspect
+and back up the named files before moving them aside, or export into a new directory. Do not remove
+personal content or manufacture a receipt to bypass a conflict.
+
+Preflight prevents known destination conflicts from causing partial exports. This is not a multi-file
+transaction or a lock against concurrent editors: a later I/O failure can leave some generated files
+updated. Such failures report errors, suppress the sweep, and return zero completed-export counts;
+inspect the vault before retrying. Avoid editing the destination while an export runs.
+
 Notes Woods manages carry `woods_managed: true` in their frontmatter. On each run, after all notes are
 written successfully, a **sweep** removes managed notes whose unit no longer exists, so deletions in
 your code propagate. Several guards make the sweep safe to point at a real vault:
@@ -150,21 +199,20 @@ your code propagate. Several guards make the sweep safe to point at a real vault
 - It refuses to delete more than 30% of managed notes at once (the signature of a partial extraction)
   unless `WOODS_OBSIDIAN_FORCE_PURGE=1` is set.
 - It resolves symlinks and confirms every deletion target is inside the vault root.
-- It is skipped entirely if any note failed to write that run (a stale note is harmless; a deleted
-  reviewed note is not).
+- It is skipped entirely if any candidate unit is unreadable or returns a mismatched identity,
+  or any note failed to write. `force_purge` does not bypass this incomplete-export guard.
 
-The same ownership check guards the `.obsidian/` config: Woods will **not** overwrite an existing,
-foreign `.obsidian/` folder, it leaves your Obsidian settings untouched and warns instead.
+In a foreign vault, Woods leaves `.obsidian/` configuration untouched and skips the sweep. Notes and
+sidecars are added only after the same per-file ownership preflight. In a Woods-owned vault, existing
+configuration assets still require a matching ownership receipt or byte-identical generated content.
 
 ## Limitations
 
 - **Bases needs Obsidian ≥ 1.9** (≥ 1.10 for card/list views). The `.base` file is harmless on older
   versions, it simply doesn't render.
-- **`gem_source` units are not exported.** They aren't reachable through the index reader; only
-  `rails_source` is covered by `include_framework`.
 - **Hand-edits diverge.** The vault is meant to be regenerated. Editing a note's properties in
   Obsidian rewrites its frontmatter, after which a re-export will overwrite your changes.
 - **Nested vaults ignore the shipped config.** Open the generated folder as its own vault for graph
   colors, Bases, and link-format settings to apply.
 
-See `docs/AGENT_GUIDE.md` for how an agent consumes the `_woods/` sidecar.
+For MCP-based exploration instead of local vault files, see [the agent guide](AGENT_GUIDE.md).

data/docs/PUBLISHED_INDEX.md

@@ -1,5 +1,8 @@
 # Reading a published index from Ruby
 
+For shell, Python, shipping snapshots, or direct JSON access, use the
+[published filesystem layout contract](INDEX_LAYOUT.md).
+
 `Woods::PublishedIndex` is the stable, read-only API for tools that are not MCP clients: RuboCop cops, CI gate scripts, and the `woods:check:*` tasks. It needs no Rails, opens one published generation, and never moves off it for the life of the reader.
 
 ```ruby
@@ -24,6 +27,58 @@ Woods::PublishedIndex.open(Rails.root.join('tmp/woods')) do |index|
 end
 ```
 
+## Validate a published generation
+
+```ruby
+require 'woods/resilience/index_validator'
+
+report = Woods::Resilience::IndexValidator.new(index_dir: 'tmp/woods').validate
+report.valid? # false when artifact or semantic graph errors were found
+report.errors
+report.warnings
+```
+
+`woods:validate` uses the same checker. Supporting development versions validate
+raw graph relationships as well as JSON, content hashes, and indexed files; see
+the [semantic invariants and limits](INDEX_LAYOUT.md#semantic-graph-validation).
+Errors retain typed identifiers and artifact paths. Writer-version and source-path
+warnings keep their existing advisory behavior.
+
+Each call resolves and pins one published generation for all checks. Concurrent
+publication can advance the pointer, but retention cannot remove the payload
+being validated. The next call sees the new generation. Locks release on success
+and errors; malformed pointers fail instead of falling back to stale root files.
+Legacy flat layouts remain supported but cannot provide immutable-generation
+isolation against in-place writes. Bare type-directory fixtures without a
+manifest retain structural-only validation.
+
+The reusable `Woods::Resilience::GraphInvariantValidator` accepts raw string-keyed
+`graph:` data and typed `index_entries:` and returns an array of errors without
+modifying either input. Callers supplying raw data must keep it within one pinned
+generation and verify the unit artifacts themselves, as `IndexValidator` does.
+Validation is a read-only diagnostic, not automatic repair or proof of runtime
+execution.
+
+## Manifest writer provenance
+
+The published `manifest.json` records `woods_version`, a string naming the Woods gem version
+that last published that manifest. Full extraction, changed incremental runs,
+targeted refreshes, and the static Woods self-map write it. A no-op leaves the
+published manifest and its version unchanged. Resolve the manifest through the
+generation pointer, as Woods readers do.
+
+Older manifests may omit the field or contain `null`; that means unknown.
+`woods_status.index.woods_version` reports this value from the served manifest,
+while `woods_status.server.version` identifies the running MCP reader. The reader
+never substitutes its own version for missing writer provenance.
+
+`woods:validate` warns when a present writer version is malformed or its major
+version differs from the installed validator. The warning is advisory and does
+not invalidate an otherwise structurally valid index. Re-run full extraction
+when investigating a major-version mismatch. Matching versions do not certify
+compatibility: an incremental publisher can retain units written by an older
+version. This field does not replace the [upgrade procedure](UPGRADING_TO_2.md).
+
 ## One generation, pinned for the reader's whole life
 
 Unlike `Woods::MCP::IndexReader`, a `PublishedIndex` never refreshes between calls. It resolves one generation at `.new`/`.open` time and every fact it returns, `units`, the table map, `generation_number`, `external_dependency_checksum`, comes from that one generation for as long as the reader is open. There is no `reload` and no auto-refresh: open a new reader to see a later publish.

data/docs/README.md

@@ -36,10 +36,13 @@ and stdio or Streamable HTTP endpoints directly.
 - [MCP tool cookbook](MCP_TOOL_COOKBOOK.md): scenario-based calls with parameters and expected response shapes.
 - [Console MCP setup](CONSOLE_MCP_SETUP.md): Console transports, blocked tables, credential scanning, redaction, SQL validation, and production safeguards.
 - [MCP HTTP transport](MCP_HTTP_TRANSPORT.md): shared/remote Index Server transport, authentication, origins, and protocol details.
+- [Edit client adapters](CLIENT_HOOKS.md): opt-in Claude/OpenCode registration, complete path batches, and recovery.
 - [MCP worktree setup](MCP_WORKTREE_SETUP.md): register Woods correctly when agents work in linked git worktrees.
 
 ## Index lifecycle
 
+- [Source freshness](SOURCE_FRESHNESS.md): verify dirty source against a served generation, establish a fresh-process baseline, and understand bounded unknown results.
+
 - [Retrieval guide](RETRIEVAL_GUIDE.md): configure embeddings and understand semantic retrieval, ranking, and token budgets.
 - [Embedding models](EMBEDDING_MODELS.md): choose and size local Ollama models.
 - [Upgrade to Woods 2.0](UPGRADING_TO_2.md): identifier changes, atomic payloads, durable-store reconciliation, and rollback.
@@ -48,8 +51,10 @@ and stdio or Streamable HTTP endpoints directly.
 
 - [Why Woods](WHY_WOODS.md): the problems runtime introspection solves.
 - [Internals](INTERNALS.md): extraction, publication, graph, storage, retrieval, and MCP components.
+- [Ruby runtime trace enrichment](RUNTIME_TRACING.md): record observed Ruby callers and merge trace evidence into method units.
 - [Extractor reference](EXTRACTOR_REFERENCE.md): what each extractor produces and the edge cases it handles.
 - [Reading a published index from Ruby](PUBLISHED_INDEX.md): the `Woods::PublishedIndex` Ruby API for cops, gate scripts, and `woods:check:*` tasks (including the moved-message check).
+- [Published index layout](INDEX_LAYOUT.md): the filesystem contract for non-Ruby readers, with Bash/jq and Python examples, retention pins, and snapshot-copy rules.
 - [Evaluation](EVALUATION.md): retrieval scoring, baselines, and the agent-level index on/off ablation.
 - [Backend matrix](BACKEND_MATRIX.md): implemented provider/store combinations and their operational requirements.
 - [Token benchmark](TOKEN_BENCHMARK.md): evidence behind Woods token-estimation defaults.
@@ -89,6 +94,8 @@ Use this map when changing behavior or documentation. Update the owner first; ot
 | Contributor policy | [CONTRIBUTING.md](../CONTRIBUTING.md) |
 | Coding-agent repository instructions | [AGENTS.md](https://github.com/lost-in-the/woods/blob/main/AGENTS.md) |
 | Non-MCP Ruby access to a published index | [PUBLISHED_INDEX.md](PUBLISHED_INDEX.md) |
+| Generation-bound source evidence and fresh extraction | [SOURCE_FRESHNESS.md](SOURCE_FRESHNESS.md) |
+| Published filesystem layout for external readers | [INDEX_LAYOUT.md](INDEX_LAYOUT.md) |
 | Evaluation harnesses | [EVALUATION.md](EVALUATION.md) |
 
 The current public surface is generated from 35 extractors. Counts and capability claims must match `.Codex/release-v2/surface-inventory.json`, which is generated from the code and verified in CI.