woods 2.0.0.beta2 → 2.0.0.beta4

data/README.md

@@ -4,38 +4,23 @@
 
 # Woods
 
-**Give AI coding agents a runtime-accurate map of your Rails application.**
+**Give coding agents the Rails context that source files alone leave out.**
 
 [![Gem Version](https://img.shields.io/gem/v/woods)](https://rubygems.org/gems/woods)
 [![CI](https://github.com/lost-in-the/woods/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/lost-in-the/woods/actions/workflows/ci.yml)
 [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE.txt)
 
-<!-- release-state:version-banner -->
-> **This tree documents version 2.0.0.** It is a major update from 1.x: read [what changed and how to upgrade](docs/UPGRADING_TO_2.md) before updating. The full history is in the [CHANGELOG](CHANGELOG.md).
->
-> `main` is the development branch and can run ahead of the latest published gem. The gem badge above shows the latest published version; documentation for a published version lives on its tag.
->
-> ### Version: 2.0.0.beta2 is published as a prerelease; `main` documents 2.0.0
->
-> | Line | Version | Documentation |
-> |---|---|---|
-> | Documented here | **2.0.0**, unreleased | this README and the [documentation index](docs/README.md) |
-> | Latest prerelease | **2.0.0.beta2** | [the v2.0.0.beta2 tag](https://github.com/lost-in-the/woods/tree/v2.0.0.beta2) |
-> | Latest published gem | **1.6.1** | [the v1.6.1 tag](https://github.com/lost-in-the/woods/tree/v1.6.1) |
->
-> RubyGems treats 2.0.0.beta2 as a prerelease, so `gem "woods", "~> 2.0"` does not resolve it. Install it explicitly with `gem "woods", "2.0.0.beta2"`. The released constraint stays `gem "woods", "~> 1.6"`.
-<!-- release-state:end -->
+Woods boots your Rails application, extracts its resolved structure, and publishes an index that coding agents can query through the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/). It brings together database schema, associations, callbacks, concerns, routes, and source code so an agent can inspect how Rails assembles your application.
 
-Woods boots your Rails app, extracts the behavior Rails assembles at runtime, and serves it to AI tools through the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/). Agents can inspect resolved routes, schema, associations, callbacks, included concerns, dependencies, and execution flows instead of guessing from source files alone.
+Supports **Ruby 3.0+ and Rails 6.0–8.x**, using a Ruby version supported by your Rails release. The application must boot and connect to its database. Structural queries need no embedding provider or vector database.
 
-Woods 2.0 supports Ruby 3.0 or later and Rails 6.0 through 8.x. It connects AI coding tools and agents through MCP.
+[Get started](#five-minute-setup) · [Documentation](docs/README.md) · [Agent setup](docs/AGENT_SETUP.md) · [Upgrade from 1.x](docs/UPGRADING_TO_2.md)
 
 ## What Woods adds
 
-A Rails model rarely lives in one file. Its real behavior can include database schema, generated methods, framework defaults, and concerns loaded from elsewhere:
+Consider a model whose behavior is spread across Rails, the database, and a concern:
 
 ```ruby
-# app/models/order.rb
 class Order < ApplicationRecord
   include Auditable
   belongs_to :customer
@@ -43,40 +28,58 @@ class Order < ApplicationRecord
 end
 ```
 
-Woods turns that runtime class into one connected unit with:
+Woods can give an agent one unit containing its columns and indexes, association metadata, resolved callbacks, and included concern source. Recorded relationships connect that unit to other parts of the application.
 
-- column types, indexes, and foreign keys from the live database;
-- associations, validations, scopes, enums, and resolved callbacks;
-- source from included concerns, kept beside the owning class;
-- callback side effects such as jobs, mailers, and columns written;
-- forward dependencies and reverse dependents;
-- route, controller, view, job, and service relationships.
+An agent can then ask:
 
-The result is a codebase index an agent can query by exact name, pattern, dependency path, graph structure, or natural language.
+> Find the Order model, inspect its callbacks and associations, and show its recorded dependents. Cite the indexed evidence and check source code for callers the graph may miss.
 
-Still weighing it? [Why Woods](docs/WHY_WOODS.md) makes the case against grep, cloud indexers, and IDE language servers.
+Models are one part of the index: Woods also extracts controllers, routes, jobs, mailers, views, components, GraphQL types, service objects, tests, and more. See the [extractor reference](docs/EXTRACTOR_REFERENCE.md) for coverage and the [agent guide](docs/AGENT_GUIDE.md) for query examples.
 
 ## Five-minute setup
 
-The default setup provides structural code intelligence. It does not require an embedding provider, vector database, or access to live application records.
+For agent-led setup, use the [agent installation option](#let-an-agent-install-it) and its runbook. For a new manual installation, follow the steps below.
 
-### 1. Install Woods
+**Already using Woods?** If you are upgrading from 1.x, follow the [upgrade guide](docs/UPGRADING_TO_2.md). For an existing 2.x installation, go directly to [retrieval modes](#retrieval-with-or-without-embeddings), [MCP configuration](docs/MCP_SERVERS.md), or the [configuration reference](docs/CONFIGURATION_REFERENCE.md). Preserve your initializer, index path, provider settings, and other client entries. Changing only the MCP launch configuration or retrieval mode does not require rerunning the installer or rebuilding the structural index.
 
-```ruby
-# Gemfile
-group :development do
-  gem "woods", "~> 2.0"
-end
-```
+Run installation and extraction commands from your Rails application root in its normal development environment. **Using Docker?** Follow [Docker setup](docs/DOCKER_SETUP.md) first: run those commands inside the application container and use paths visible to the process that runs MCP.
+
+### 1. Install and configure
+
+These steps are for **Woods 2.x**. Choose a published 2.x version from the release information below and confirm it on [RubyGems](https://rubygems.org/gems/woods/versions). If only prereleases are available, use an exact prerelease pin; `~> 2.0` will not select one. Follow the chosen version's tag documentation rather than assuming every feature on `main` is published. If you choose 1.x, use its tag documentation instead of this quickstart.
+
+<details>
+<summary>Release information and Gemfile version constraints</summary>
+
+<!-- release-state:version-banner -->
+> **This tree documents version 2.0.0.** It is a major update from 1.x: read [what changed and how to upgrade](docs/UPGRADING_TO_2.md) before updating. The full history is in the [CHANGELOG](CHANGELOG.md).
+>
+> `main` is the development branch and can run ahead of the latest published gem. The gem badge above shows the latest published version; documentation for a published version lives on its tag.
+>
+> ### Version: this tree declares prerelease 2.0.0.beta4; `main` documents 2.0.0
+>
+> | Line | Version | Documentation |
+> |---|---|---|
+> | Documented here | **2.0.0**, unreleased | this README and the [documentation index](docs/README.md) |
+> | Declared prerelease | **2.0.0.beta4** | [the v2.0.0.beta4 tag](https://github.com/lost-in-the/woods/tree/v2.0.0.beta4) |
+> | Latest published gem | **1.6.2** | [the v1.6.2 tag](https://github.com/lost-in-the/woods/tree/v1.6.2) |
+>
+> RubyGems treats 2.0.0.beta4 as a prerelease, so `gem "woods", "~> 2.0"` does not resolve it. Once published, install it explicitly with `gem "woods", "2.0.0.beta4"`. The released constraint stays `gem "woods", "~> 1.6"`.
+<!-- release-state:end -->
+
+</details>
+
+Expand the release information above, then add its appropriate `gem "woods", …` declaration to your Gemfile's `:development` group and run:
 
 ```bash
 bundle install
+bundle exec ruby -rwoods/version -e 'puts Woods::VERSION'
 bin/rails generate woods:install
 ```
 
-**Do not run the generated migration for a new default installation.** The generator creates an annotated `config/initializers/woods.rb` plus a legacy application migration for `woods_units`, `woods_edges`, and `woods_embeddings`. Woods 2's shipped structural index and storage backends do not use those application tables. Remove the migration before continuing; keep and run it only when deliberately preserving an older/custom integration that uses them.
+**For a new default installation, remove the generated `db/migrate/*_create_woods_tables.rb` migration without running it.** Those legacy application tables are unused by the shipped index and storage backends. Keep the generated `config/initializers/woods.rb`; its defaults are sufficient. Only retain the migration for a deliberate older/custom integration. See [Getting started](docs/GETTING_STARTED.md#2-generate-and-review-configuration).
 
-### 2. Extract and verify the codebase
+### 2. Extract and validate
 
 ```bash
 bin/rails woods:extract
@@ -84,11 +87,11 @@ bin/rails woods:validate
 bin/rails woods:stats
 ```
 
-Extraction must run where Rails can boot. The default index lives at `tmp/woods/`.
+Run these where your Rails application can boot. The default output is `tmp/woods/`; keep this generated directory out of source control.
 
-### 3. Connect the Index Server
+### 3. Connect your MCP client
 
-Add this to your MCP client's project configuration. The configuration location varies by client:
+Adapt this example to your MCP client's project configuration format, using your application path and preserving other server entries. See [client configuration locations](docs/MCP_SERVERS.md#client-configuration-locations) for guidance:
 
 ```json
 {
@@ -102,175 +105,91 @@ Add this to your MCP client's project configuration. The configuration location
 }
 ```
 
-Restart or reconnect your MCP client, then ask it to call `woods_status`. A ready response with non-zero unit counts confirms the path from Rails extraction to the MCP client.
-
-The Index Server reads the published index from disk. It does not boot Rails or query application records.
-
-> **Using Docker?** Run Rails commands inside the application container. If Woods is installed only there, launch the Index Server through that container too; a host-side server requires a host Ruby bundle and host-visible index. Follow [Docker setup](docs/DOCKER_SETUP.md).
-
-The complete walkthrough, including expected output and first questions to ask, is in [Getting started](docs/GETTING_STARTED.md).
-
-## Let an agent install it
-
-Woods is built to be agent-operated, and the fastest path is handing installation to the coding agent that will use it. Claude Code users can install the distributed skills once — they trigger on install, upgrade, configuration, investigation, and diagnosis on their own:
-
-```bash
-/plugin marketplace add lost-in-the/plugins
-/plugin install woods-plugin@lost-in-the-plugins
-```
-
-With any coding agent (no plugin needed), paste this into a session opened at your Rails app's root:
-
-```text
-Install or upgrade the woods gem in this Rails application by following
-https://github.com/lost-in-the/woods/blob/main/docs/AGENT_SETUP.md.
-Structural setup only: add the gem to the development group, run the
-installer, extract and validate the index, and register the Index MCP
-server for this app. Do not run the generated legacy migration, and do
-not add embedding providers, vector databases, Console/live-data access,
-or secrets without asking me first. If woods 1.x is already installed,
-follow the upgrade runbook in docs/UPGRADING_TO_2.md instead and plan a
-clean re-index. Finish by reporting the installed version, files
-changed, commands run, and one verified woods_status call through the
-registered MCP server.
-```
-
-The runbook holds the agent to the same guardrails the skills enforce: a version preflight, minimal diffs, and explicit approval before anything beyond the structural index. Prefer doing it by hand? The five-minute setup above is the same procedure as commands.
-
-## Choose your path
-
-| Goal | Start here |
-|---|---|
-| Install Woods yourself | [Getting started](docs/GETTING_STARTED.md) |
-| Ask a coding agent to install Woods safely | [Agent setup runbook](docs/AGENT_SETUP.md) |
-| Configure an MCP client, Docker, or HTTP | [MCP servers](docs/MCP_SERVERS.md) |
-| Teach an agent how to query Woods effectively | [Agent guide](docs/AGENT_GUIDE.md) |
-| Add semantic search with OpenAI or local Ollama | [Retrieval guide](docs/RETRIEVAL_GUIDE.md) |
-| Query live Rails data through the optional Console Server | [Console MCP setup and security](docs/CONSOLE_MCP_SETUP.md) |
-| Keep the index current automatically while coding | [Watch daemon](docs/WATCH_DAEMON.md) |
-| Upgrade an existing 1.x installation | [Upgrade to Woods 2.0](docs/UPGRADING_TO_2.md) |
-| Diagnose a failure | [Troubleshooting](docs/TROUBLESHOOTING.md) |
-
-## Upgrading from 1.x
-
-Woods 2.0 is a major release: identifiers, the on-disk layout, the MCP surface, and task failure posture all changed. [Upgrade to Woods 2.0](docs/UPGRADING_TO_2.md) holds the full what-changed table, the step-by-step runbook with backups and rollback, and an agent-operated upgrade prompt.
+Reconnect the client and ask it to call `woods_status`. Confirm the index path and non-zero unit counts. Then use `search` to discover a known class and `lookup` with its identifier **and type** to inspect it.
 
-## Optional Claude Code workflows
+The Index Server reads the published index without booting Rails or querying application records. See [MCP servers](docs/MCP_SERVERS.md) for client-specific configuration and HTTP transport.
 
-Woods itself is MCP-client and model independent. The separately packaged Woods plugin (install commands under [Let an agent install it](#let-an-agent-install-it)) gives Claude Code five guided skills: setup and upgrade, MCP configuration, index-driven investigation, repository agent enablement, and diagnosis. Other MCP clients do not need it; follow the human or agent runbooks linked above and configure either stdio or Streamable HTTP directly.
+If Woods is installed only inside Docker, launch MCP through that container too. Host-side launch needs a host bundle and a host-visible index; see the [Docker process and path rule](docs/MCP_SERVERS.md#docker-process-and-path-rule).
 
-## Two servers, two trust boundaries
+## Retrieval: with or without embeddings
 
-Woods ships two MCP servers. Most users only need the Index Server.
+Exact lookup, pattern search, and graph queries work immediately after extraction. For ranked retrieval through `codebase_retrieve`, choose a mode:
 
-| | Index Server | Console Server |
+| Mode | Setup | What it searches |
 |---|---|---|
-| Purpose | Query pre-extracted code context | Query live Rails models and schema |
-| Data source | Files under `tmp/woods/` | A booted Rails process and its database |
-| Default tools | 14 | 9 |
-| Optional tools | Semantic retrieval activates after embedding; advanced Ruby embeddings can wire more collaborators | `console_sql` and `console_query` raise the total to 11 when explicitly enabled |
-| Default posture | Read-only index | Disabled; live-data access requires deliberate setup |
-
-The 14 Index tools cover health, exact lookup, search, dependency traversal, flow tracing, graph analysis, framework source, change recency, and optional semantic retrieval. The Console Server exposes nine supported model/schema tools by default. Nineteen Tier 2/3 Console schemas (9 Tier 2, 10 Tier 3) and `console_eval` exist as source inventory but do not register in any supported mode.
-
-See [MCP servers](docs/MCP_SERVERS.md) for the callable tool lists and client configuration.
+| **Lexical** | Set `WOODS_RETRIEVAL_MODE=lexical` in the MCP process environment and restart the server | Published extraction units, ranked by field-aware keyword matching; no provider or embeddings |
+| **Semantic** (default mode) | Configure a local or hosted embedding provider, then run `bin/rails woods:embed` | Embedded code context, ranked by semantic similarity |
 
-## Optional semantic search
+For the stdio configuration above, add `"env": {"WOODS_RETRIEVAL_MODE": "lexical"}` inside the `woods` server entry to choose lexical mode. Confirm the active retriever with `woods_status`.
 
-Exact search, lookup, graph traversal, and flow tools work after extraction alone. Natural-language retrieval through `codebase_retrieve` also needs embeddings:
+To switch back to semantic retrieval, remove the lexical environment override or set `WOODS_RETRIEVAL_MODE=semantic`, configure the provider and embedding artifacts, then restart the MCP server and verify `woods_status`. Switching to lexical does not delete existing vectors or provider configuration.
 
-```ruby
-# config/initializers/woods.rb
-Woods.configure_with_preset(:local)
-```
-
-The `:local` preset uses SQLite metadata, in-memory vectors persisted under the index, and a local Ollama service. It needs the `sqlite3` gem in the application bundle plus an installed, running Ollama service, but no cloud API key. Pull the default model before the first embed:
-
-```bash
-ollama pull nomic-embed-text
-```
-
-MySQL/PostgreSQL applications that do not bundle `sqlite3` can use `:shared_filesystem` for local persisted stores instead. PostgreSQL/OpenAI, Qdrant/OpenAI, and shared-filesystem configurations are documented in the [backend matrix](docs/BACKEND_MATRIX.md) and [configuration reference](docs/CONFIGURATION_REFERENCE.md).
-
-For dense Ruby source, add `gem "tokenizers", "~> 0.5"` for exact WordPiece token counting. Without it, Woods uses a character estimate that can over-pack some Ollama chunks.
-
-```bash
-bin/rails woods:embed
-```
-
-Reconnect the Index Server after the first embed, then check `woods_status` before using `codebase_retrieve`.
+The [lexical guide](docs/RETRIEVAL_GUIDE.md#embedding-free-lexical-retrieval) and [semantic setup](docs/RETRIEVAL_GUIDE.md#configuring-retrieval) cover configuration, ranking, and response budgets. Lexical matching depends on shared vocabulary; semantic mode requires the configured provider and embedding artifacts.
 
 ## Keeping the index current
 
-Run a full extraction after installation or broad configuration changes:
-
-```bash
-bin/rails woods:extract
-```
-
-For automatic maintenance during development, run the watcher as a dedicated process:
+Run a watcher alongside your development processes:
 
 ```bash
 bin/rails woods:watch
 ```
 
-Add it to your development process manager so it starts beside Rails:
+It catches up on changes, publishes complete generations, and lets the Index Server refresh on later tool calls. Use a process supervisor for changes that require the watcher to restart. Without a watcher, run `bin/rails woods:incremental` after edits or `bin/rails woods:extract` for a full rebuild.
 
-```text
-# Procfile.dev
-web:   bin/rails server
-woods: bundle exec rake woods:watch
-```
-
-The watcher catches up changes made while it was stopped, batches new file changes, reloads Rails code when safe, and publishes complete generations atomically. The Index Server notices a new generation on its next tool call and refreshes itself. **After the initial extraction, ordinary code changes need no manual re-extraction or MCP restart.**
+Incremental cost depends on the affected code and relationships; broad changes can cost as much as a full extraction. Semantic embeddings have a separate update step. See [Watch daemon](docs/WATCH_DAEMON.md), [incremental extraction](docs/INCREMENTAL_EXTRACTION.md), and [source freshness](docs/SOURCE_FRESHNESS.md).
 
-Changes to boot-captured state, including dependencies, initializers, database configuration, credentials, or schema, make the watcher exit with status 75 so a process supervisor can restart it cleanly. If semantic retrieval is enabled, the watcher keeps structural context current; run `bin/rails woods:embed_incremental` to update vectors.
-
-In CI on Rails 8.1, add one step to `config/ci.rb` so the index the gates read matches the commit under test:
+## Two servers, two trust boundaries
 
-```ruby
-step "Woods: refresh", "bin/rails woods:incremental"
-```
+| | Index Server | Console Server |
+|---|---|---|
+| Purpose | Inspect extracted code context | Query live Rails models and schema |
+| Reads | Published index files | A booted application and its database |
+| Packaged tools | 14; retrieval usable when configured | 9; 11 with embedded read tools enabled |
+| Setup | The workflow above | Optional, disabled by default |
 
-Claude Code users with the Woods plugin can opt into the same refresh from a `PostToolUse` hook, plus a `SessionStart` warning scoped to commit timestamps (it does not see uncommitted edits or an older checkout). Both ship disabled; set `WOODS_HOOKS_ENABLED=1` to turn them on. See [Watch daemon](docs/WATCH_DAEMON.md#hooks-for-agent-sessions).
+Extraction itself boots and eager-loads your application, so its boot-time behavior still runs. Treat the generated index as confidential application source. Enabling hosted embeddings sends the embedded content to that provider. MCP responses also contain application source, which your client may send to its model provider even when Woods uses lexical retrieval or local embeddings.
 
-Without a resident watcher, run `bin/rails woods:incremental` after changes. See [Watch daemon](docs/WATCH_DAEMON.md) for Docker polling, failure behavior, and restart triggers.
+The optional Console Server can access live data. Review its [setup and security model](docs/CONSOLE_MCP_SETUP.md) before enabling it. Report vulnerabilities privately through [SECURITY.md](SECURITY.md).
 
-## What gets indexed
+## What the index can and cannot establish
 
-Woods recognizes the Rails application as a connected system, including:
+- **It is a snapshot.** Check freshness against your working tree before relying on it for a change.
+- **Relationships are recorded evidence, not a complete call graph.** Arbitrary method-body constant references are not exhaustively indexed. No recorded dependents does not prove that a class has no callers or is safe to delete.
+- **A traced flow is not proof of execution.** Follow the tool's evidence and limits, and verify behavior in the application when it matters.
 
-- models, concerns, controllers, routes, middleware, and engines;
-- services, interactors, commands, jobs, mailers, and scheduled work;
-- ERB views, Phlex components, ViewComponents, and navigation edges;
-- GraphQL types, mutations, resolvers, and fields;
-- policies, serializers, decorators, validators, state machines, and events;
-- migrations, database views, factories, tests, configuration, and installed framework source.
+Use Woods to locate and connect evidence, then confirm the relevant source and tests. The [agent guide](docs/AGENT_GUIDE.md) describes this workflow.
 
-Read the [extractor reference](docs/EXTRACTOR_REFERENCE.md) for the complete per-type contract and [internals](docs/INTERNALS.md) for how extraction, storage, retrieval, and MCP fit together.
+## Let an agent install it
 
-## Security boundary
+Use the [agent setup runbook](docs/AGENT_SETUP.md) for a copyable installation prompt and verification checklist. Woods works with MCP-capable clients independently of a specific model or editor.
 
-Woods extraction reads application code, resolved Rails configuration, and database schema. Treat the generated index as source code: do not publish it unless the source itself may be published.
+Claude Code users can optionally install the companion workflows:
 
-The optional Console Server has a larger trust boundary because it can read live application data. It is disabled by default and adds table blocking, credential scanning, column redaction, SQL validation, and rolled-back transactions when enabled. Those controls reduce risk; they do not turn production data access into a harmless default. Review [Console MCP security](docs/CONSOLE_MCP_SETUP.md#safety-model) before enabling it.
+```text
+/plugin marketplace add lost-in-the/plugins
+/plugin install woods-plugin@lost-in-the-plugins
+```
 
-Report vulnerabilities privately through [SECURITY.md](SECURITY.md).
+The plugin guides installation, MCP configuration, investigation, repository agent setup, and diagnosis. It is distributed separately from the gem.
 
 ## Documentation
 
-Use the [documentation index](docs/README.md) to find guides by task or audience. Frequently used references include:
+| Task | Guide |
+|---|---|
+| Install and verify | [Getting started](docs/GETTING_STARTED.md) |
+| Configure clients, Docker, or HTTP | [MCP servers](docs/MCP_SERVERS.md) |
+| Query effectively | [Agent guide](docs/AGENT_GUIDE.md) and [tool cookbook](docs/MCP_TOOL_COOKBOOK.md) |
+| Configure Woods | [Configuration reference](docs/CONFIGURATION_REFERENCE.md) |
+| Choose retrieval and storage | [Retrieval guide](docs/RETRIEVAL_GUIDE.md) and [backend matrix](docs/BACKEND_MATRIX.md) |
+| Upgrade from 1.x | [Upgrade guide](docs/UPGRADING_TO_2.md) |
+| Diagnose a failure | [Troubleshooting](docs/TROUBLESHOOTING.md) |
 
-- [Configuration reference](docs/CONFIGURATION_REFERENCE.md)
-- [MCP tool cookbook](docs/MCP_TOOL_COOKBOOK.md)
-- [FAQ](docs/FAQ.md)
-- [Troubleshooting](docs/TROUBLESHOOTING.md)
-- [Upgrade to Woods 2.0](docs/UPGRADING_TO_2.md)
+See the [documentation index](docs/README.md) for all guides and canonical reference pages.
 
 ## Contributing
 
-Read [CONTRIBUTING.md](CONTRIBUTING.md) before opening an issue or pull request. Coding agents working in the source repository should also read [AGENTS.md](https://github.com/lost-in-the/woods/blob/main/AGENTS.md).
+Use [GitHub issues](https://github.com/lost-in-the/woods/issues) for bugs and feature requests. Read [CONTRIBUTING.md](CONTRIBUTING.md) before submitting a pull request; coding agents should also read [AGENTS.md](https://github.com/lost-in-the/woods/blob/main/AGENTS.md).
 
 ## License
 
-Woods is available under the [MIT License](LICENSE.txt).
+[MIT](LICENSE.txt).

data/SECURITY.md

@@ -60,11 +60,12 @@ Woods runs inside your Rails application and has access to:
 - **Application source code**: extracted and written to the output directory as JSON
 - **Database schema**: column names, types, indexes, and foreign keys (no row data)
 - **Git metadata**: commit history, contributors, file change frequency
-- **Runtime state** (Console MCP Server only), live database queries within a rolled-back transaction
+- **Optional session traces**: session/trace identifiers, request paths and controller/action timelines when session tracing is configured
+- **Live database rows** (Console MCP Server), queries within a rolled-back transaction
 
 ### Output Directory
 
-Extracted data is written to `tmp/woods/` by default. This directory contains your application's source code and schema in structured JSON format. Treat it with the same sensitivity as your source code, do not expose it to untrusted parties.
+Extracted data is written to `tmp/woods/` by default. This directory contains your application's source code and schema in structured JSON format. Source literals, comments and configuration metadata may contain sensitive values. Treat the index with the same sensitivity as your source code; do not expose it to untrusted parties. Optional session stores can be configured under this directory and add request metadata that may be more sensitive than the source itself. Apply the [session tracing access and retention guidance](docs/CONFIGURATION_REFERENCE.md#session-tracer-options) to those stores too.
 
 ### Console Server
 
@@ -80,13 +81,15 @@ If extraction output leaks, what can an attacker do with it?
 
 **What the output contains.** Application source code (inlined concerns, callback-resolved behavior), database schema (column names, types, indexes, foreign keys), route tables, migration history, gem versions, and git metadata (commit history, contributor emails, file change frequency).
 
-**What the output does not contain.** No row-level data from your database. Woods extracts schema only. No environment variables, no `Rails.application.credentials`, no API keys, no session state, no request logs, no customer data.
+**Structural extraction boundaries.** Structural extraction collects schema rather than dumping live database rows, and does not intentionally collect environment variables or the Rails encrypted credential store. This is not a guarantee that output is free of secrets or customer information: extracted source can contain hardcoded values, comments or examples, and Index tools can return that source. Console redaction and export-specific scrubbing do not sanitize the structural index.
+
+**Optional session data.** When configured, session tracing records session/trace identifiers, request paths and controller/action timelines. Request paths and identifiers can contain sensitive user information. The configured session store controls where these records live; configured Index session tools can return them. Session storage and access need their own retention and authorization controls. Console MCP is a separate live-data boundary described above.
 
 | Leak scenario | Attacker gains | Attacker does not gain |
 |---|---|---|
-| `tmp/woods/` directory exfiltrated | Source code + schema equivalent to a git clone + `rails db:schema:dump` | Database rows, secrets, tokens, customer data |
-| MCP Index Server token leaked (HTTP transport) | Read-only query access to the extracted index, no write or execution paths | Shell access, database row data, secrets |
+| `tmp/woods/` directory exfiltrated | Source code, schema and metadata, including sensitive values present in source; session records if their store is configured here | No additional live database or shell access merely from possessing these files |
+| MCP Index Server token leaked (HTTP transport) | Access to published source/metadata and configured session tools, including sensitive content they contain | The packaged default does not boot Rails or provide live database queries or shell execution; custom tool wiring has its own access boundary |
 | Notion sync database compromised | Model and column summaries synced to Notion | Anything not mirrored, source code stays local |
-| Console MCP Server exposed (dev/staging) | Read-only database access through a rolled-back transaction, bounded by TableGate + Redactor + SqlValidator | Write access (rolled back), full credentials (redacted), blocked tables |
+| Console MCP Server exposed (dev/staging) | Live database access constrained by TableGate, credential scanning, Redactor and SqlValidator; this remains admin-trust access | Supported packaged modes expose no write or eval tools; rollback and redaction have the limits described in the Console guide |
 
 **Mitigation.** Treat `tmp/woods/` as source-equivalent, keep it out of world-readable directories and public container images. Rotate `WOODS_MCP_HTTP_TOKEN` on compromise. Keep `console_mcp_enabled = false` in production regardless of environment, since the console layers are defense-in-depth and not primary controls.

data/docs/AGENT_GUIDE.md

@@ -2,6 +2,14 @@
 
 This guide is for coding agents using an already connected Woods MCP server. Woods is evidence from the running Rails application and its extracted graph; it complements file search, tests, git history, and direct source inspection.
 
+Supporting servers also send a concise version of this workflow in MCP
+initialization/discovery instructions, without requiring an installed plugin.
+Check the connected server's version and registered tools; this feature is
+unreleased after `2.0.0.beta2`, and protocol `2024-11-05` omits the field.
+The [initialization contract](MCP_SERVERS.md#initialization-guidance) describes
+availability. This guide remains the detailed reference when instructions are
+absent or the client does not display them.
+
 ## Start every session with status
 
 Call `woods_status` before relying on the index. Check:
@@ -13,6 +21,11 @@ Call `woods_status` before relying on the index. Check:
 
 If status is unhealthy, report the evidence and ask the owner to extract or refresh. Do not fill gaps by asserting that Woods found nothing.
 
+Compare `index.woods_version` (last manifest publisher, unknown for older indexes)
+with `server.version` (the MCP reader). A major-version difference warrants a
+full extraction and upgrade review; a match does not prove every retained unit
+was rewritten. See [manifest writer provenance](PUBLISHED_INDEX.md#manifest-writer-provenance).
+
 ## The default query loop
 
 Use this four-step loop for most codebase questions:
@@ -103,22 +116,70 @@ fields: ["identifier", "source_code", "metadata"]
 
 Start with identifier search. Add source or metadata only when name discovery fails. Restrict types and keep result limits small enough to inspect.
 
+On supporting versions, read `completeness` before calling search exhaustive.
+`result_count` counts returned rows. `result_limit` proves one more match exists;
+`scan_budget` and `regex_timeout` leave that unknown. Only `exhausted` establishes
+an exact total within the requested index/query domain. Narrow types, literal
+prefix/suffix filters, or deep fields when `partial` is true. A detected artifact
+failure remains an error with unknown completeness, never proof of no matches.
+
+This metadata is unreleased after `2.0.0.beta2`; older servers may omit it.
+Do not infer completeness from a full page or missing metadata. See the
+[search response contract](MCP_SERVERS.md#search-completeness).
+
 ## Traverse deliberately
 
-`dependencies` means “what this unit uses.” `dependents` means “what uses this unit.” Both default to bounded breadth-first traversal and accept type or relationship filters.
+`dependencies` shows recorded relationships from this unit; `dependents` shows
+recorded relationships to it. Both default to bounded breadth-first traversal and
+accept type or relationship filters. They are not exhaustive source-reference or
+call graphs: selective scanning can miss arbitrary method-body constant references,
+including generic PORO and library targets. No dependents or test-only dependents
+do not establish absence of production callers. Check source before claiming absence.
+The `graph_coverage` notice makes this scope explicit in supporting responses;
+that metadata is unreleased after `2.0.0.beta3`.
 
 Start at depth 1 or 2. A deeper unfiltered traversal can obscure the direct evidence that matters. Common relationship values include associations (`belongs_to`, `has_many`, `has_one`), code references, renders, redirects, form actions, and navigation links.
 
-Both return at most 50 nodes and say so with a `Showing N of M (truncated)`
-line. Narrow with `depth`, `types` and `via` before paging with `limit` and
+Both return at most 50 nodes by default and say so with a `Showing N of M (truncated)`
+line for completed walks. Budget-limited answers in supporting versions instead
+say `Showing N of at least M (total unknown: <budget reason>)`. Narrow with `depth`, `types` and `via` before paging with `limit` and
 `offset`: narrowing answers the question, paging only splits the same answer
 across turns. In a multi-database app each row names the unit's database.
 
-Use returned relationship labels as evidence. Do not infer call order from a dependency edge alone.
-
-## Use semantic retrieval only when ready
-
-`codebase_retrieve` answers natural-language questions with token-budgeted context. Use it when `woods_status` reports a configured embedding provider and current vector data.
+On supporting servers, inspect `structuredContent.data` for traversal nodes,
+`graph_coverage`, exactness, budgets and optional explanation witnesses, regardless
+of the text renderer. This packaged stdio/HTTP payload is unreleased after
+`2.0.0.beta3`; check the actual response and use its text when structured data is
+absent. There is no traversal `format` argument. See the
+[response contract](MCP_SERVERS.md#dependency-graph-coverage).
+
+A traversal can also stop at its independent node or edge budget. Treat
+`partial`/`partial_reason` as incomplete graph evidence even on the final page;
+paging cannot recover nodes the walk never reached. Supporting responses include
+`total_is_exact: false` for a cutoff and true for a finished walk, independently of
+pagination. This field is unreleased after `2.0.0.beta3`; on older servers inspect
+`partial` directly. `nodes_total` remains the root-inclusive admitted prefix count,
+not the full reachable total when partial. Even an exact count covers only the
+requested root, depth, filters and published graph generation. Check the connected schema
+before using `max_nodes`/`max_edges`, and follow the
+[budget contract](MCP_SERVERS.md#dependency-traversal-budgets).
+
+When the connected schema supports `explain`, request `explain: true` to see
+recorded source-to-target relationships and a shared shortest witness to each
+row. Report `direct` relationships separately from `transitive` inferred impact.
+Follow `parent`/`edge_id` references; `context: true` ancestors are outside the
+current result page. Unknown labels and ambiguous candidate types stay unknown;
+`typed_path_complete: false` does not establish a uniquely typed path. True means
+only unambiguous witness types, not complete source coverage. Supporting text
+responses label this `witness types unambiguous` (unreleased after `2.0.0.beta3`). See the
+[explanation contract](MCP_SERVERS.md#traversal-explanations).
+
+Use recorded relationship labels as evidence. Do not infer execution or call
+order from a dependency edge alone.
+
+## Use ranked retrieval only when ready
+
+`codebase_retrieve` answers natural-language questions with token-budgeted context. Use it when `woods_status` reports explicit lexical mode over a current published index, or a configured embedding provider and current vector data in semantic mode. Lexical mode explains matching terms/fields and does not infer synonyms absent from the text; a no-match response is not proof of missing behavior.
 
 Important parameters:
 
@@ -202,3 +263,43 @@ Verification: <source/test/history checked or still needed>
 - [Extractor reference](EXTRACTOR_REFERENCE.md): indexed unit and edge contracts.
 - [Retrieval guide](RETRIEVAL_GUIDE.md): embeddings, ranking, and token budgets.
 - [Troubleshooting](TROUBLESHOOTING.md): stale indexes, disabled retrieval, and startup failures.
+
+### Explicit retrieval and discovery scope
+
+On a server whose tool schema advertises them, `packages` and `source_paths` narrow
+`search` and `codebase_retrieve` before candidate limits. These are per-call
+arguments, not configuration settings. Inspect applied scope and completeness;
+a narrow graph query can omit relevant cross-boundary dependencies. See the
+[scope contract](RETRIEVAL_GUIDE.md#explicit-package-and-source-path-scopes) for
+root/nested ownership, path normalization, errors, storage support, and cost.
+
+### Verify source content before relying on freshness
+
+When supported by the installed version, inspect `woods_status.index.source_freshness`.
+Repeated edits can leave the porcelain fingerprint unchanged. A quick-budget
+`unknown` can justify one explicit `source_check: "deep"` call; persistent unknown
+needs the reported limitation resolved, not repeated status polling. Use
+`bundle exec woods-extract full` to establish verified preboot source evidence.
+A named refresh does not certify unrelated consumers or external runtime state.
+Follow [source freshness](SOURCE_FRESHNESS.md) and keep ordinary query scopes narrow.
+
+### Recovering relevant code under a small context budget
+
+Check the installed schemas before requesting `evidence: 'compact'` on retrieval
+or lookup, or `evidence: 'outline'` for API orientation. These modes preserve
+complete selected spans and explicitly report omissions. An outline is not proof
+of implementation behavior. Follow the returned typed `full_evidence` call when
+you need full source; its SHA guard refuses changed source instead of validating
+a different publication accidentally. Published-unit line/byte ranges can include
+synthesized or commented concern source and are not physical file coordinates.
+See the [evidence contract](RETRIEVAL_GUIDE.md#compact-published-evidence-and-api-outlines).
+
+### Optional hook hints
+
+When explicitly enabled on a supporting installed gem, Claude hook context offers
+a small served-generation orientation and post-edit candidate dependents. Treat
+pre-refresh, unknown freshness, truncation and ambiguous identity labels as limits
+on the evidence. Verify direct and inferred downstream candidates using typed
+lookup and `dependents explain:true`; suggested tests do not prove coverage.
+Silence does not establish no impact. See [bounded context hints](WATCH_DAEMON.md#optional-bounded-context-hints)
+for opt-in, independent refresh controls, limits and repeat suppression.