woods 2.0.0.beta3 → 2.0.0

data/docs/CONFIGURATION_REFERENCE.md

@@ -208,6 +208,25 @@ config.vector_store_options = {
 }
 ```
 
+Woods uses an HNSW index over pgvector's `vector` representation, which supports
+**1–2,000 dimensions** ([pgvector's HNSW limits](https://github.com/pgvector/pgvector#hnsw)).
+Included in Woods `2.0.0`: the adapter rejects wider dimensions before any
+SQL, and `woods:pgvector` rejects invalid widths before writing a migration.
+There is no automatic vector truncation or half-precision conversion.
+
+The default `text-embedding-3-large` output is 3,072 dimensions. With pgvector,
+explicitly request a supported provider output width, for example:
+
+```ruby
+config.embedding_model = 'text-embedding-3-large'
+config.embedding_options = { dimensions: 1536 }
+```
+
+Keep the provider, `vector_store_options[:dimensions]` (when set), and generated
+migration width equal. Changing a stored width requires a compatible new table
+or an intentional index rebuild; changing the setting does not resize old data.
+Use another backend if you need the full 3,072-dimensional output.
+
 Requires the pgvector extension. Run the generator to create migrations:
 
 ```bash
@@ -263,7 +282,12 @@ in the promoted dump; the index MCP server loads that snapshot at startup or
 reload. Incremental embedding publishes changes to paths, dependencies, and
 other unit metadata even when unchanged source needs no new embedding. A run
 with no content or metadata changes keeps the existing dump and retention
-window. This is a reasonable default for hosts that don't bundle `sqlite3`.
+window. Included in Woods `2.0.0`: a full `Indexer#index_all` run replaces
+the published corpus even when a custom caller reuses in-memory vector and
+metadata stores. Deleted units, including metadata-only records, are removed;
+an empty full rebuild publishes an empty dump. Failed embedding leaves the
+previous promoted dump and checkpoint intact. Incremental purge guards remain
+unchanged. This is a reasonable default for hosts that don't bundle `sqlite3`.
 
 ## Retrieval cache options
 
@@ -335,7 +359,7 @@ The embed run writes `woods.json` + `dumps/<ISO8601>/vectors.bin` + `metadata.ms
 
 Requirements:
 - `output_dir` must be set and readable by both the embed process and the MCP server.
-- The MCP server must know the same `output_dir` (pass via `woods-mcp <DIR>` or set `WOODS_DIR`).
+- The MCP server must know the same `output_dir` (pass via `woods-mcp <DIR>` or set `WOODS_DIR`; see MCP path precedence below).
 
 ## Presets
 
@@ -418,11 +442,26 @@ finding qualifying edges. Run extraction again after changing either setting;
 MCP reads the published report. These findings remain informational, never a
 release or architecture gate.
 
-When the JSON snapshot fallback is in use, malformed JSON, top-level values
-other than objects, and files that cannot be read (including concurrent retention
-removals) are warned about and treated as absent. Snapshot lists and unit history
-omit them; direct lookup returns no snapshot, and a diff with an unavailable
-snapshot returns empty added, modified, and deleted lists.
+When the JSON snapshot fallback is in use, malformed JSON, invalid snapshot
+shapes, and files that cannot be read (including concurrent retention removals)
+are warned about and treated as absent. A snapshot needs a hexadecimal string
+`git_sha` matching its filename; `extracted_at` may be a string, null, or omitted.
+When present and non-null, `units` must be an object whose records are objects.
+A malformed record invalidates the entire snapshot, rather than exposing partial
+history. Legacy bare identifier keys, omitted/null unit collections, and optional per-unit hash
+fields remain supported; timestamp strings are not restricted to a new format.
+Unit history limits count matching unit records, not the most recent snapshots
+searched (JSON fallback correction included in Woods `2.0.0`). A unit
+missing from newer snapshots can still have retained history.
+Snapshot lists and unit history omit unusable files; direct lookup returns no
+snapshot, and a diff with an unavailable snapshot returns empty added, modified,
+and deleted lists. An empty diff in this case is not proof that nothing changed.
+New captures compare against the latest usable snapshot. Unusable SHA-named files
+still count toward retention and are pruned first when the limit is exceeded;
+reading alone does not delete them. Valid legacy snapshots with null or omitted
+timestamps are retained ahead of corrupt files, then treated as oldest among
+usable snapshots. Direct lookup and diff still reject invalid
+caller-supplied SHA paths with an argument error.
 
 `incremental_blast_radius_depth` is unbounded by default because a unit two hops
 out really can have content that depends on the changed file. An STI grandchild
@@ -473,6 +512,18 @@ config.session_store = Woods::SessionTracer::FileStore.new(
 config.session_exclude_paths = ['/health', '/metrics', '/assets']
 ```
 
+### File session retention
+
+`FileStore` accepts `ttl:` in seconds (default `nil`, expiration disabled),
+`max_sessions:` (default `1000`), and `max_requests_per_session:` (default `1000`).
+TTL expires a file when the store clock reaches its modification time plus the
+TTL. Recording after expiry starts a fresh history; expired events are discarded
+before appending or migrating legacy filenames, under the same store lock.
+When legacy and encoded files coexist, each expires independently before any
+surviving histories are merged. Clearing a session is idempotent for supported
+IDs, including Unicode and punctuation, and removes both filename formats when
+applicable.
+
 ### Redis session index compatibility
 
 `RedisStore` lists and clears both legacy SET indexes and recency ZSET indexes.
@@ -652,7 +703,8 @@ These variables are read by the gem and its MCP servers at runtime. They complem
 | Variable | Default | Purpose |
 |----------|---------|---------|
 | `WOODS_RETRIEVAL_MODE` | `semantic` | Explicit packaged MCP retrieval mode: `semantic` or `lexical`. Lexical reads extraction unit JSON without provider autodetection, credentials or vector artifacts. |
-| `WOODS_DIR` | `Dir.pwd` | Path to the extraction output directory. |
+| `WOODS_DIR` | unset | MCP extraction-index path, after a positional argument and before `WOODS_OUTPUT`. See precedence below. |
+| `WOODS_OUTPUT` | unset | MCP index-path fallback when neither a positional path nor `WOODS_DIR` is set; included in Woods `2.0.0`. |
 | `WOODS_REQUIRE_INDEX` | unset | Set to `"1"` to fail closed: the server refuses to boot (raises `MissingArtifact`) unless a real index (`woods.json`) is present. By default an extract-only host boots in pattern/structural mode without it. Explicit lexical mode requires a valid published extraction index, not `woods.json`. |
 | `WOODS_ALLOW_AUTODETECT` | unset | **Deprecated no-op.** Auto-detect is now the default; accepted for backward compatibility only. |
 | `WOODS_SEARCH_MAX_SCAN` | `500` | Cap on unit files loaded during a phase-2 (metadata/source_code) `search`. Hitting the cap sets `partial: true` in the response. |
@@ -668,6 +720,20 @@ These variables are read by the gem and its MCP servers at runtime. They complem
 | `WOODS_QDRANT_URL`, `WOODS_QDRANT_COLLECTION`, `WOODS_QDRANT_API_KEY` | n/a | Override/require Qdrant connection settings when a pgvector/Qdrant-backed index is served outside its host application (no `Woods.configuration` available). |
 | `WOODS_PG_URL` | n/a | Required when a pgvector-backed index is served outside its host application. |
 
+**MCP index path precedence (included in Woods `2.0.0`):** positional
+argument → `WOODS_DIR` → `WOODS_OUTPUT` → current directory for `woods-mcp`
+and `woods-mcp-http`. `woods-mcp-start` still requires one of the first three;
+it never silently selects the current directory. An explicitly empty
+`WOODS_DIR` remains an invalid override rather than falling through. Earlier
+versions accept the positional path or `WOODS_DIR`; use an explicit path for
+portable client configuration.
+
+Paths are resolved in the MCP process's working directory and filesystem.
+A published index can have `generation.json` pointing to a payload's
+`manifest.json`; a root `manifest.json` is only the legacy flat layout. If
+startup cannot find a manifest, check the reported directory and point at the
+existing index before deciding another extraction is needed.
+
 ### Rake tasks
 
 | Variable | Default | Purpose |
@@ -705,10 +771,24 @@ These variables are read by the gem and its MCP servers at runtime. They complem
 | `WOODS_WATCH_CATCH_UP` | `1` (enabled) | Set to `"0"` to skip generation-watermark catch-up on daemon start. |
 | `WOODS_WATCH_TRUST_FOREIGN_HOST` | unset (disabled) | Set to `"1"` in each task/MCP reader to trust a foreign daemon's heartbeat for up to 15 minutes, without a local pid check. See [cross-host liveness](WATCH_DAEMON.md#cross-host-liveness) for clock bounds, degraded coverage, and startup limitations. |
 
+### Managed watcher startup
+
+**Included in Woods `2.0.0`.** `woods-watch` wraps the raw task for native
+Foreman/Puma lifecycle management. `--root PATH` selects the application working
+directory; `--boot-timeout SECONDS` defaults to `300` and bounds boot/handshake,
+not extraction. Explicit child argv follows `--`. Output-directory precedence
+remains `WOODS_OUTPUT`, then `Woods.configuration.output_dir`.
+
+Managed mode requires `WOODS_WATCH_IDLE_TIMEOUT` to be unset, preserves one owner,
+and never takes over a conflicting daemon. The optional Puma adapter only starts
+in its finalized development environment. See [startup and installation](WATCH_DAEMON.md#managed-development-startup)
+for the generator's explicit modes, portable receipt, update/removal, and the
+separate supervision status. Raw task settings above remain compatible.
+
 ### Opt-in plugin refresh hooks
 
 These settings control the plugin shell worker. Check installed
-`woods:hook_refresh` support first; the task is unreleased after 2.0.0.beta2.
+`woods:hook_refresh` support first; the task is included in Woods `2.0.0`.
 See [hook coverage and retry](WATCH_DAEMON.md#hooks-for-agent-sessions) and
 [optional context limits](WATCH_DAEMON.md#optional-bounded-context-hints).
 
@@ -741,26 +821,23 @@ tasks for manual refreshes; hook transport is not a general shell execution API.
 | `GITHUB_BASE_REF` | unset (GitHub Actions) | Build the diff range `origin/<ref>...HEAD` for `woods:incremental`; an unfetched ref makes the range unresolvable, same exit behavior. |
 | `RAILS_ENV` | `development` | Rails environment the rake tasks boot in. |
 | `WOODS_PROFILE` | unset | Set to `"1"` to log disjoint `[Woods] [profile] <phase> in N.NNs` durations, including git enrichment, reconciliation, payload sync, pointer publication (`publish`) and retention (`payload prune`). Separate `[profile total]` lines report whole extraction wall time, including unprofiled setup and failed runs; never add these totals to phase durations. Excludes process/Rails boot before extraction. Off by default. |
-| `WOODS_GIT_DIR` | unset | Absolute path to the canonical git directory. Wins over the repository Woods would otherwise find, at all three of its git call sites: per-unit `commit_count`/`change_frequency` (enrichment), `manifest.json`'s `git_branch`/`git_sha` (provenance), and the `woods:incremental` diff range. All three build their command line with `Woods::GitCommand.argv`. |
+| `WOODS_GIT_DIR` | unset | Absolute path passed directly to Git's `--git-dir`, selecting that directory's `HEAD`. For a linked worktree use its `worktrees/<id>` directory inside the complete shared Git layout. Wins over the repository Woods would otherwise find, at all three of its git call sites: per-unit `commit_count`/`change_frequency` (enrichment), `manifest.json`'s `git_branch`/`git_sha` (provenance), and the `woods:incremental` diff range. All three build their command line with `Woods::GitCommand.argv`. |
 | `GIT_BRANCH`, `GIT_SHA` | unset | Provenance for a checkout with no `.git` at all (a source tarball, a Docker `COPY` that excludes it). Ignored when a `.git` is present but unresolvable, so a stale build arg cannot mask a worktree. |
 
-**`GIT_DIR` alone is not enough for a linked git worktree.** Woods runs git as a
-subprocess, so git's own `GIT_DIR` and `GIT_COMMON_DIR` are honored wherever git
-honors them. But pointing `GIT_DIR` at a worktree's *private* git directory only
-moves the failure: that directory reaches the shared object store through a
-relative `commondir` pointer, which still resolves outside a container mount,
-and `GIT_COMMON_DIR` does not override it. `git rev-parse --git-dir` then
-succeeds while no ref resolves.
-
-Woods refuses to enrich in that state rather than writing `commit_count: 0` and
-`change_frequency: "new"` on every unit: the git keys are omitted, provenance is
-`"unknown"`, and one warning names the cause. Point `WOODS_GIT_DIR` at the
-canonical git directory (the one a worktree's `gitdir:` pointer ultimately leads
-to) and mount it:
-
-```bash
-WOODS_GIT_DIR=/canonical-git bundle exec rake woods:extract
-```
+For linked worktrees in containers, preserve access to the complete shared
+Git directory and the worktree-specific HEAD. A same-path mount that resolves
+the existing `.git` pointer needs no override. For a relocated complete layout,
+set `WOODS_GIT_DIR=/mounted-common/worktrees/<id>`, deriving `<id>` from Git's
+worktree metadata rather than the branch name. Selecting `/mounted-common`
+itself selects the primary checkout's HEAD and can produce incorrect history,
+provenance, and incremental paths. See the canonical
+[worktree mount and verification steps](TROUBLESHOOTING.md#git-directory-mounts-for-linked-worktrees).
+
+Git subprocesses inherit Git's own environment variables, so inspect existing
+`GIT_DIR` and `GIT_COMMON_DIR` settings when resolving layout problems.
+If HEAD cannot be resolved, enrichment omits the Git keys and provenance is
+`"unknown"` for a present but unresolvable `.git`; one warning names the cause.
+After correcting the layout, run a full extraction to refresh retained metadata.
 
 ### Exporters
 
@@ -780,11 +857,10 @@ The `woods-mcp` bootstrapper emits a one-line STDERR banner at startup indicatin
 
 ## Git enrichment history
 
-Current source requires **Git 2.31 or newer** for optional per-unit git
+Woods 2.0 requires **Git 2.31 or newer** for optional per-unit git
 metadata. Extraction still succeeds when git is unavailable or history cannot
 be read completely. Git enrichment is omitted in either case; a failed or
 incomplete streamed history read logs a warning.
-This requirement and the history policy below are unreleased after 2.0.0.beta2.
 
 Per-unit enrichment also requires a non-shallow repository. A shallow checkout
 or a failed repository-depth probe omits enrichment with one warning per

data/docs/CONSOLE_MCP_SETUP.md

@@ -48,7 +48,7 @@ its token, allowed origins and TLS as described in [Option C](#option-c-http-rac
 
 The rake task does two things before starting the MCP server:
 
-1. **Captures stdout before Rails boots.** Rails boot emits OpenTelemetry warnings, gem notices, and other output to stdout. An MCP client cannot parse these as JSON-RPC, they break the protocol. The rake task redirects stdout → stderr immediately, saves the real stdout fd, and restores it after boot completes.
+1. **Captures stdout before Rails boots.** Rails boot emits OpenTelemetry warnings, gem notices, and other output to stdout. An MCP client cannot parse these as JSON-RPC. The rake task saves the protocol output and redirects application stdout to stderr. In the runtime isolation fix (included in Woods `2.0.0`), that redirection remains active throughout the server's lifetime; only the MCP transport writes to the saved protocol pipe. Rails loggers, `puts`, and writes to standard output during queries stay on stderr.
 2. **Calls `Rails.application.eager_load!`** to load all application models. Without eager loading, only the models that happen to be autoloaded before the first query appear in the registry.
 
 ### MCP client configuration
@@ -83,7 +83,7 @@ rake woods:console
   │    ├─ Rails.application.eager_load!
   │    ├─ build model registry from ActiveRecord::Base.descendants
   │    ├─ Server.build_embedded(model_validator:, safe_context:, ...)
-  │    └─ MCP::Server::Transports::StdioTransport.new(server).open
+  │    └─ Woods::Console::StdioTransport.new(server, output: protocol_out).open
   │
   └─ MCP server responds to tool calls via stdin/stdout
 ```
@@ -198,6 +198,12 @@ the Rails server environment. The middleware stack registers automatically via
 the gem's Railtie and requires `Authorization: Bearer <token>` on every Console
 request. Missing or incorrect tokens receive `401 Unauthorized`.
 
+The HTTP authentication scheme is ASCII case-insensitive (`Bearer`, `bearer`,
+or `BEARER`); the token remains case-sensitive and must match exactly after one
+space. This applies to both Console HTTP and `woods-mcp-http`. Case-insensitive
+scheme support is included in Woods `2.0.0`; use the canonical `Bearer`
+spelling in client configuration for compatibility with earlier releases.
+
 For non-loopback access, `console_mcp_allowed_origins` must include the public
 Rails/MCP host. If a browser-based client sends an `Origin` header from a
 different host, include that exact origin too. This allow-list controls both
@@ -760,12 +766,12 @@ Each transaction sets a statement timeout before any query runs. The default is
 
 `SqlValidator` rejects non-read-only SQL at the string level, before any database interaction.
 
-Validation runs **once**, inside the executor, with the dialect of the live adapter. There is deliberately no earlier dialect-blind pre-check in the tool handler: a validator built without a dialect is the conservative MySQL+PostgreSQL union, and running it first meant a MySQL host rejected statements whose `\'`/backtick grammar produces a spuriously forbidden PostgreSQL view — the adapter-aware acceptance below could never be reached on a real transport. The executor raises `SqlValidationError` for anything it refuses, which the dispatch pipeline renders as a tool error, so nothing is ungated.
+Validation runs **once**, inside the executor, with the dialect of the live adapter. There is deliberately no earlier dialect-blind pre-check in the tool handler: a validator built without a dialect is the conservative union of supported dialects, and running it first meant a MySQL host rejected statements whose `\'`/backtick grammar produces a spuriously forbidden PostgreSQL view — the adapter-aware acceptance below could never be reached on a real transport. The executor raises `SqlValidationError` for anything it refuses, which the dispatch pipeline renders as a tool error, so nothing is ungated.
 
 
 - **Allowed prefixes:** `SELECT`, `WITH...SELECT`, and plain `EXPLAIN`. `EXPLAIN ANALYZE` is rejected, it executes the query rather than just planning it (both the whitespace and `EXPLAIN (ANALYZE, …)` option-list spellings).
 - **Rejected prefixes:** `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `DROP`, `ALTER`, `TRUNCATE`, `CREATE`, `GRANT`, `REVOKE`
-- **Rejected anywhere in query:** `UNION`, `INTO`, `COPY`; row-lock clauses (`FOR UPDATE`, `FOR NO KEY UPDATE`, `FOR SHARE`, `FOR KEY SHARE`, `FOR UPDATE NOWAIT`/`SKIP LOCKED`, MySQL `LOCK IN SHARE MODE`) — these take live row locks even inside the rolled-back transaction. The lock check is adapter-aware: `console_sql` validates with the active adapter's dialect, including MySQL double-quoted strings/backtick identifiers and PostgreSQL quoted identifiers/E-strings. Unknown adapters conservatively scan both normalizations. Every view is scanned under both MySQL executable-comment (`/*!...*/`) semantics, so `#` comments and version-guarded comments cannot split a clause apart.
+- **Rejected anywhere in query:** `UNION`, `INTO`, `COPY`; row-lock clauses (`FOR UPDATE`, `FOR NO KEY UPDATE`, `FOR SHARE`, `FOR KEY SHARE`, `FOR UPDATE NOWAIT`/`SKIP LOCKED`, MySQL `LOCK IN SHARE MODE`) — these take live row locks even inside the rolled-back transaction. The lock check is adapter-aware: `console_sql` validates with the active adapter's dialect, including MySQL double-quoted strings/backtick identifiers and PostgreSQL quoted identifiers/E-strings. Unknown adapters conservatively scan all supported normalizations. Every view is scanned under both MySQL executable-comment (`/*!...*/`) semantics, so `#` comments and version-guarded comments cannot split a clause apart.
 - **Function allowlist (the authoritative function control):** every function-call-shaped identifier must appear in `ALLOWED_FUNCTIONS`, a conservative set of pure read-only functions (aggregates, window functions, string/number/date/JSON readers) kept portable across MySQL, PostgreSQL, and SQLite. Anything else is rejected by name, quoted forms (`"pg_terminate_backend"(…)`) included. This is an allowlist because a denylist cannot enumerate every side-effecting function (`nextval`, `pg_advisory_lock`, `pg_terminate_backend`, …). A legacy `DANGEROUS_FUNCTIONS` denylist (`pg_sleep`, `lo_import`, `lo_export`, `pg_read_file`, `pg_write_file`, `load_file`, `sleep`, `benchmark`) still runs first as belt-and-suspenders.
 - **Rejected patterns:** multiple statements (semicolons), writable CTEs (every `AS (...)` body is checked, so a writable CTE in any WITH position is refused — `WITH a AS (SELECT 1), b AS (DELETE FROM users RETURNING *) SELECT * FROM b`), a CTE list attached to top-level DML (`WITH a AS (SELECT 1) DELETE FROM users RETURNING *`), comment-hidden injections
 
@@ -793,13 +799,15 @@ to enforce their narrower parameterized scope grammar.
 - **HTTP:** Check that the Rails server is running and listening on the expected port. An unauthenticated `curl http://localhost:3000/mcp/console` should return `401` when the enabled middleware and bearer-auth guard are mounted. A request with the configured bearer token proceeds to MCP protocol handling.
 - **All modes:** Run `bundle exec rake woods:console` directly in a terminal. It should hang (waiting for MCP protocol input) rather than exit immediately. If it exits, check the error output.
 
-### Rails boot noise breaks MCP protocol
+<a id="rails-boot-noise-breaks-mcp-protocol"></a>
+
+### Rails logs break MCP protocol
+
+Prefer `bundle exec rake woods:console`: it captures output before the Rails environment boots. Direct `rails runner` invocation can redirect output only after Rails has booted; it cannot recover an already contaminated protocol stream.
 
-The rake task redirects stdout to stderr before Rails boots specifically to prevent this. If you see JSON parse errors from the MCP client, check:
+The runtime isolation fix is included in Woods `2.0.0`. Earlier builds restore stdout after boot, so a logger that writes there can interleave SQL or application logs with MCP responses. On those builds, configure the Console process's application logger to write to stderr or a file. Discarding stderr does not fix logs written to stdout.
 
-1. You are using `bundle exec rake woods:console`, not `rails runner exe/woods-console` directly (the runner path handles this too, but via a different mechanism).
-2. No `puts` or `print` calls run at boot in your initializers before the task can capture stdout.
-3. Try running `bundle exec rake woods:console 2>/dev/null` to isolate, the MCP protocol output goes to stdout, Rails noise goes to stderr.
+With a build containing the fix, application output remains on stderr during tool calls. If protocol contamination persists, inspect wrapper scripts and output emitted before the rake task starts; reserve stdout for MCP and retain stderr for diagnosis. Do not disable Console redaction or credential scanning to troubleshoot transport logging.
 
 ### Models not visible to `console_status`
 
@@ -853,7 +861,44 @@ quote and comment rules. MySQL also reads the executing session's `ANSI_QUOTES`
 and `NO_BACKSLASH_ESCAPES` settings for validation, protected-column scanning, and
 table gating; adjacent subtraction operators are not assumed to begin a comment.
 Direct scanner callers without session settings use conservative quote-mode scans.
+SQLite read SQL accepts simple ASCII bare, double-quoted, or backtick identifiers
+(letters, digits, and underscores, starting with a letter or underscore).
+Use whitespace after `FROM` and `JOIN`, and use `SELECT` subqueries rather than
+parenthesized table groups. Bracket-quoted and string-quoted names, quoted names
+containing punctuation, and unsupported table-reference syntax are refused before
+execution, because they cannot be reliably checked against the configured table
+policy. Ordinary string literals remain supported. This restriction is part of
+2.0.0.beta4; keep read tools disabled on older versions when this policy is
+needed. Confirm that the release is available before selecting it. The Rails-version
+integration lane checks these boundaries on real SQLite.
+
 The contributor live-backend lane exercises these boundaries
 through Console requests against PostgreSQL and MySQL. Keep read tools disabled
 unless live SQL access is needed, and retain the configured blocked-table and
 redaction policies when diagnosing a rejected request.
+
+## Console policy corrections in 2.0.0.beta4
+
+In `2.0.0.beta4`, the default model-reading tools check the resolved relation
+against `console_blocked_tables` before fetching records or counts. This includes
+application-defined default scopes and the parent lookup for association counts.
+The checked relation is reused for execution so a dynamic default scope is not
+resolved twice. These checks do not change which Console tools are enabled.
+
+Response handling redacts protected fields before invoking serializers, converts
+the remaining response to JSON-compatible values, then redacts and scans that
+normalized tree before either JSON or Markdown rendering. Symbol values and custom
+JSON serializers therefore receive the same credential checks as ordinary strings.
+Custom values in Markdown now use their JSON-compatible representation. Numbers,
+booleans, nulls, and ordinary record shapes retain their existing meanings.
+
+For SQLite SQL, keyword spellings receive function-policy exceptions only where
+supported query grammar requires them. PostgreSQL reserved-keyword grammar is
+preserved. Unsupported parenthesized offset expressions on other dialects may
+require a plain numeric offset. Keep the configured access and credential policies
+in place when adjusting a query.
+
+These corrections require `2.0.0.beta4` or a reviewed development revision that
+contains them. Confirm that a patched release is available before selecting it.
+On affected versions, disable Console where these policies are required; Index MCP
+can stay enabled because it reads the published code index separately.

data/docs/DOCKER_SETUP.md

@@ -87,9 +87,24 @@ docker compose exec app bundle exec rake woods:extract_framework
 
 Run the watcher as its own development service or process-manager entry, not as a one-off terminal command. Docker Desktop bind mounts may not deliver reliable native filesystem events; set `WOODS_WATCH_POLL=1` for polling when needed. The watcher updates structural generations automatically, while semantic vectors still require `woods:embed_incremental`.
 
+Use the application's actual Rails task entrypoint. If its root `Rakefile` wraps
+Compose, the container may need `bundle exec rails woods:watch`. Keep the raw
+task under one external restart owner: `restart: unless-stopped` recovers after
+Docker restarts while respecting an intentional stop; `on-failure` covers failed
+process exits but not Docker restart. Leave idle TTL unset for continuous work.
+
+Validate `docker compose config`: explicit YAML anchor keys can replace inherited
+mounts/environment, and short `depends_on` does not establish database readiness.
+Preserve the existing source/bundle mounts and use the application's healthcheck
+convention. With Grove, include the watcher in the applicable shared or isolated
+services list and align source/index mounts to the selected worktree. Follow
+[Docker and Grove verification](AUTOMATIC_MAINTENANCE.md#docker-verify-the-resolved-service)
+instead of copying a generic service that loses required settings.
+
 When host-side tasks or one-off containers read the daemon's shared index,
 `WOODS_WATCH_TRUST_FOREIGN_HOST=1` lets those readers trust its recent heartbeat.
 Set it in each reader process; Docker does not forward host variables by default.
+Ordinary Index MCP reads do not need this trust or the extraction writer lock.
 See [cross-host liveness](WATCH_DAEMON.md#cross-host-liveness) for the 15-minute
 crash-detection bound and single-supervisor requirement.
 
@@ -468,6 +483,6 @@ To register `console_sql` and `console_query`, enable
 
 ### Woods MCP tools not available in a git worktree
 
-When working in a git worktree, subagents may not find the woods MCP servers because `.mcp.json` discovery is path-based and the worktree has a different root directory. See [MCP_WORKTREE_SETUP.md](MCP_WORKTREE_SETUP.md) for the fix and verification steps.
+A separate Claude Code session launched in a worktree may have different project-scoped MCP registrations. Subagents inherit the parent session's MCP tools, subject to tool restrictions; changing their working directory does not select a different Woods index. Check registration, the Compose service's mounted checkout, and the served index using [MCP worktree setup](MCP_WORKTREE_SETUP.md).
 
 See [CONSOLE_MCP_SETUP.md](CONSOLE_MCP_SETUP.md) for detailed console server documentation.

data/docs/EVALUATION.md

@@ -90,8 +90,8 @@ bundle exec ruby -Ilib bench/evaluation/runner.rb
   strategy selection also fail. The baseline format is developer-only and is
   **not** the `EVAL_BASELINE_FILE` aggregate-threshold format.
 
-B-190/B-191 recapture on Ruby 4.0.6 (five warmed pipeline repetitions per query; Ruby 3.3.1
-and 3.4.10 replay the same answers):
+B-190/B-191 baseline, with the #549 output-label refresh captured on Ruby 4.0.6
+(five warmed pipeline repetitions per query):
 
 | Strategy | Queries | Precision@5 | Recall | MRR | Mean actual context tokens |
 |---|---:|---:|---:|---:|---:|
@@ -99,8 +99,14 @@ and 3.4.10 replay the same answers):
 | Vector | 4 | 0.313 | 0.375 | 0.625 | 1,041.2 |
 | Graph | 8 | 0.813 | 0.519 | 1.000 | 1,041.1 |
 | Hybrid | 4 | 0.750 | 0.396 | 1.000 | 1,058.8 |
-| Direct with type filtering | 4 | 0.375 | 0.750 | 0.625 | 551.0 |
-| Within-type vector fallback | 4 | 0.400 | 1.000 | 1.000 | 1,250.8 |
+| Direct with type filtering | 4 | 0.375 | 0.750 | 0.625 | 552.0 |
+| Within-type vector fallback | 4 | 0.400 | 1.000 | 1.000 | 1,251.8 |
+
+The clearer `Retrieval metadata records` heading adds one exact `cl100k_base`
+token to each of the eight direct/fallback contexts. The other 20 contexts,
+retrieved identifiers and order, annotations, and quality metrics are unchanged.
+The refresh reuses captured vectors and the pinned `tiktoken 0.11.0` tokenizer;
+it performs no new model inference.
 
 Precision@5 divides relevant hits by the actual returned slice size (up to five),
 not always by five. Recall divides retrieved relevant units by all annotated

data/docs/EXTRACTOR_REFERENCE.md

@@ -2,7 +2,7 @@
 
 Woods ships **35 extractor classes** producing **39 distinct unit types**: one for each meaningful category of Rails code. This doc covers what each extractor captures, how to configure them, and the shape of the data they produce.
 
-> **Counts explained.** `lib/woods/extractors/` contains 42 files: 35 extractor classes (each ending in `_extractor.rb`) plus 7 supporting utilities (`shared_utility_methods`, `shared_dependency_scanner`, `callback_analyzer`, `behavioral_profile`, `route_helper_resolver`, `ast_source_extraction`, `source_nesting`). The 39 unit types comes from some extractors emitting multiple categories, `GraphQLExtractor` alone produces four (`graphql_type`, `graphql_mutation`, `graphql_resolver`, `graphql_query`), and `RailsSourceExtractor` produces both `rails_source` and `gem_source`. Supporting utilities enrich existing extractors (callback side-effects, behavioral config, AST-based source slicing, nested-namespace resolution) but are not themselves extractors and do not appear in the unit type enumeration. The authoritative mapping is `Woods::Extractor::TYPE_TO_EXTRACTOR_KEY` in `lib/woods/extractor.rb`.
+> **Counts explained.** `lib/woods/extractors/` contains 35 extractor classes (each ending in `_extractor.rb`) plus supporting utilities such as `shared_utility_methods`, `shared_dependency_scanner`, `callback_analyzer`, `behavioral_profile`, `route_helper_resolver`, `ast_source_extraction`, `source_nesting`, and `declared_parent`. The 39 unit types comes from some extractors emitting multiple categories, `GraphQLExtractor` alone produces four (`graphql_type`, `graphql_mutation`, `graphql_resolver`, `graphql_query`), and `RailsSourceExtractor` produces both `rails_source` and `gem_source`. Supporting utilities enrich existing extractors (callback side-effects, behavioral config, AST-based source slicing, nested-namespace resolution) but are not themselves extractors and do not appear in the unit type enumeration. The authoritative mapping is `Woods::Extractor::TYPE_TO_EXTRACTOR_KEY` in `lib/woods/extractor.rb`.
 
 ---
 
@@ -32,6 +32,14 @@ Extractors discover code one of two ways:
 
 Some extractors combine both (e.g., `JobExtractor` scans directories first, then supplements with `ApplicationJob.descendants`).
 
+Discovery is not exhaustive. A standalone module under `app/models` that is
+called through singleton methods is not discovered by the model/PORO paths;
+conventional concerns and app modules included by live models have separate
+concern discovery. This gap is tracked in [#552](https://github.com/lost-in-the/woods/issues/552).
+Separately, dependency scanning does not capture every method-body constant
+reference ([#475](https://github.com/lost-in-the/woods/issues/475)). A missing unit
+or edge is not proof of unused code; cross-check the application source.
+
 ### Identifier naming (source-derived units)
 
 File-based extractors derive an identifier in three steps, first match wins:
@@ -65,6 +73,7 @@ Every extractor returns `Array<ExtractedUnit>`. An `ExtractedUnit` is a self-con
 - Reads Rails' per-event callback chains (`_save_callbacks`, `_create_callbacks`, and the other lifecycle events), preserving each chain's order and each entry's `kind`, filter and conditions. The public `type` combines kind and event, such as `before_save` or `after_create`; Rails' separate `before_commit` event is reported as `before_commit`, not `before_before_commit`. `callback_count` equals the emitted callback list's length. The list includes framework-registered callbacks; it is runtime metadata, not an application-only filter or a cross-event execution trace. Regenerate the index after upgrading to pick up corrected callback metadata.
 - Proc/lambda filters, including Rails-generated association callbacks, use stable source-site labels in both metadata and callback chunks: `#<Proc app/models/post.rb:12>` (or `lambda`). App paths are relative to `Rails.root`; external paths are retained and native procs use `native`. Rails 6's numeric filter identity is resolved through `raw_filter`. These labels describe location and callable kind, not captured closure state; callbacks are never executed. Model condition labels retain their existing format.
 - Default callback-object representations omit process addresses: an instance becomes `#<CleanupCallback>`, an anonymous class becomes `#<Class>`, and its instance becomes `#<#<Class>>`. Anonymous namespace prefixes are normalized too (for example, `#<Module>::CleanupCallback`). Named classes and custom `to_s` labels retain their text. These labels do not distinguish arbitrary object state; separate registered callbacks remain separate entries even when their descriptive labels match. Controller object-filter formatting is unchanged.
+- Direct Proc/lambda validation option values (for example inclusion/exclusion membership or message callables) use the same stable kind/source-site labels without execution. Validation order and duplicates, condition formats (`if`, `unless`, `on`), and non-Proc values are unchanged. Nested arrays/hashes are not recursively normalized, and labels do not serialize captured closure state. Run a full extraction after upgrading to refresh retained validation metadata; the stored schema is unchanged.
 - Callback side-effects are analyzed via `CallbackAnalyzer`: detects columns written (`self.col =`), jobs enqueued (`perform_later`), and services called
 - Reflects model class and instance methods after reading the schema, so Rails schema-loading optimizations produce the same method metadata in cold and warmed runs. Application-defined constructors remain visible; Rails versions that install an optimized singleton `new` during schema loading consistently include it in `class_methods`.
 - Automatically skips HABTM join models and anonymous classes
@@ -255,6 +264,10 @@ class PageView < AnalyticsRecord; end   # metadata[:database] => "analytics"
 - Discovery and direct extraction accept only mailers backed by an existing app-owned source file, excluding dependency mailers and fabricated convention paths.
 - Each mailer action corresponds to an email template, template paths are recorded in metadata
 - Extracts `default from:`, `layout`, and per-action subject patterns
+- Action names are sorted consistently in metadata, the generated header, template discovery and action chunks. Callback chain order and duplicate registrations are preserved.
+- Direct Proc-valued defaults and Proc callback filters use source-location/kind labels without executing them; application paths are relative to `Rails.root`. Default containers, literal strings and non-Proc values retain their existing types. This does not serialize closure captures or recursively normalize arbitrary nested objects.
+- Object callback filters use descriptive labels with addresses removed only from Ruby's default representation, including anonymous classes and namespaces. Custom labels and literal hexadecimal text are preserved. Labels do not serialize callback object state, and extraction never invokes callbacks.
+- After upgrading, run a full extraction to refresh retained mailer units. Stabilized headers and callable labels can cause a one-time source-hash change; stored index schemas are unchanged.
 
 ---
 
@@ -382,6 +395,7 @@ class PageView < AnalyticsRecord; end   # metadata[:database] => "analytics"
 - Scans `app/models` for files that don't define an `ActiveRecord::Base` descendant
 - Common examples: value objects, form objects placed in `app/models`, domain structs
 - Excludes concerns (those go to ConcernExtractor)
+- `parent_class` and the generated Parent annotation describe the selected unit declaration only. Nested or sibling classes cannot supply its parent. An implicit `Object` parent, a dynamic superclass expression, or unparseable source produces `nil`; explicit constant-path parents retain their source names.
 
 ---
 
@@ -421,6 +435,7 @@ class PageView < AnalyticsRecord; end   # metadata[:database] => "analytics"
 - Produces unit types: `graphql_type`, `graphql_mutation`, `graphql_resolver`, `graphql_query`
 - Extracts field metadata (types, descriptions, complexity, arguments), authorization patterns (Pundit, CanCan, `authorized?`), and dependencies on models/services
 - Since all GraphQL units come from one extractor, incremental re-extraction handles them via `extract_graphql_file`
+- `parent_class` and summary chunks describe the selected declaration's explicit constant-path superclass, preserving its written qualification. Nested or sibling declarations and literal text cannot supply a parent. Implicit Object, module interfaces, dynamic superclass expressions, unavailable source, and invalid source have no declared parent (`null` metadata; `unknown` in summaries). This is source declaration metadata, not resolved runtime ancestry.
 
 **Example output (abbreviated):**
 
@@ -677,6 +692,7 @@ Every app-owned unit under a package root carries `metadata[:package]` with the
 **Key details:**
 - Excludes `lib/tasks/` (covered by RakeTaskExtractor) and `lib/generators/`
 - File-based scanning; no assumption about class hierarchy
+- `parent_class` and the generated Parent annotation describe the selected unit declaration only. Nested or sibling classes cannot supply its parent. An implicit `Object` parent, a dynamic superclass expression, or unparseable source produces `nil`; explicit constant-path parents retain their source names.
 
 ---
 
@@ -738,11 +754,15 @@ History is limited to commits reachable from `HEAD` in the past 365 days,
 including merged branch history. Unmerged branches, remote refs, and tool
 checkpoint refs do not contribute. Commands run against the application root;
 when `WOODS_GIT_DIR` is set, `HEAD` belongs to that explicitly selected git
-directory, which may differ from a linked worktree's HEAD.
+directory. For a linked worktree, select its `worktrees/<id>` directory within
+the complete shared Git layout; selecting the shared root instead uses the
+primary checkout's HEAD. See the [worktree mount guide](TROUBLESHOOTING.md#git-directory-mounts-for-linked-worktrees).
 
 After upgrading from a version that included all refs, run a full
 `woods:extract` to replace previously published git metadata. Incremental
-extraction refreshes only the units it rewrites.
+extraction refreshes only the units it rewrites. A commit alone does not
+necessarily trigger the source-file watcher; run full extraction when current
+Git history is required.
 
 | Field | Description |
 |-------|-------------|

data/docs/FAQ.md

@@ -134,7 +134,7 @@ When a model includes a concern, the behavior defined in that concern is part of
 
 ### How do I update the index after code changes?
 
-Use incremental mode, which re-extracts only files that have changed since the last run:
+Use incremental mode to dispatch a selected batch of changed paths:
 
 ```bash
 bundle exec rake woods:incremental
@@ -143,7 +143,12 @@ bundle exec rake woods:incremental
 docker compose exec app bundle exec rake woods:incremental
 ```
 
-Incremental mode is ideal for CI pipelines and local development workflows. It is typically 5-10× faster than a full extraction. Nine unit types, `route`, `middleware`, `engine`, `scheduled_job`, `state_machine`, `factory`, `event`, `database_view`, and `rails_source`, don't map to individual files, so incremental mode re-runs their extractor **wholesale** whenever the relevant trigger path changes (e.g. `config/routes.rb` for routes, `Gemfile.lock` for middleware/engines/rails_source; `rails_source` participates only when `include_framework_sources` is enabled). You never need to run a full extraction just because one of these changed, see [TROUBLESHOOTING.md](TROUBLESHOOTING.md) for details.
+The default Git range is `HEAD~1`; CI variables, an explicit range, or
+`CHANGED_FILES` can select another batch. Incremental extraction also refreshes
+affected concern consumers and re-runs whole-app extractors when their trigger
+paths change. It can reduce work, but has no universal speedup: Rails boot, graph
+work, and publication still take time. See the [incremental contract](INCREMENTAL_EXTRACTION.md)
+for scope, whole-app triggers, and cases that require a full extraction.
 
 ---
 
@@ -411,7 +416,13 @@ When you run `rake woods:embed`, Woods generates embedding vectors for each extr
 
 ### What is the `codebase_retrieve` tool for?
 
-`codebase_retrieve` is the primary semantic search tool on the Index Server. It accepts a natural-language description of what you're looking for ("find where user email validation happens", "which services send Stripe API calls") and returns the most relevant extracted units as formatted context. It requires embedding configuration, without an embedding provider the tool responds with an error (`isError`, code `:not_configured`) and a remediation hint covering provider setup and the `search` tool for pattern-based matching in the meantime. Token budget is controlled by `config.max_context_tokens` (default: 8000).
+`codebase_retrieve` ranks extracted units for a natural-language query, such as
+"find where user email validation happens". Default semantic mode requires an
+embedding provider and vector store. Explicit lexical mode ranks published
+extraction text without either: set `WOODS_RETRIEVAL_MODE=lexical` in the MCP
+process environment and restart it. There is no automatic fallback between
+modes. See the [retrieval guide](RETRIEVAL_GUIDE.md#embedding-free-lexical-retrieval)
+for setup, ranking evidence, and the estimated text-budget contract.
 
 ---
 

data/docs/GETTING_STARTED.md

@@ -12,15 +12,15 @@ If an agent will perform the installation, use the safety and handoff checklist
 
 ## 1. Install the gem
 
-Use the [README release table](../README.md) to choose a version, then confirm
-that **exact version is published** on the [RubyGems versions page](https://rubygems.org/gems/woods/versions)
-before editing the Gemfile. A prepared release checkout can update the README
+Choose a version from the [RubyGems versions page](https://rubygems.org/gems/woods/versions)
+and confirm that **exact version is published** before editing the Gemfile.
+A prepared release checkout can update the documentation
 before its gem is published; if the version is absent, choose an available
 version or wait for publication.
 
 If the published 2.x line has only beta or release-candidate versions, use an
-exact pin to the published prerelease, following the README's prerelease
-instructions; `~> 2.0` does not select prereleases. Follow the selected version's
+exact pin to the published prerelease; `~> 2.0` does not select prereleases.
+Follow the selected version's
 tag documentation. The `main` guides may describe features absent from the
 published gem.
 
@@ -148,21 +148,22 @@ Reconnect the MCP server and check `woods_status`. For OpenAI, pgvector, Qdrant,
 
 ### Keep the index current
 
-For automatic maintenance, keep a watcher running beside the Rails development process:
-
-```bash
-bin/rails woods:watch
-```
-
-```text
-# Procfile.dev
-web:   bin/rails server
-woods: bundle exec rake woods:watch
-```
+Enable one watcher through the application's normal development startup. The
+[managed startup guide](WATCH_DAEMON.md#managed-development-startup) covers
+opt-in Puma integration for a simple Rails application, an owned Foreman entry
+for existing Procfile workflows, and external supervision for Docker/Grove.
+The managed launcher and generator are **included in Woods `2.0.0`**;
+check the installed commands before using them. Older packages can run the raw
+`bin/rails woods:watch` task under an external restart-capable supervisor.
 
 On startup it reconciles changes made since the last successful generation. While running it batches file events, reloads Rails code when safe, extracts affected units, and publishes atomically. The Index Server detects the new generation on its next call and reloads automatically. After the initial extraction, ordinary code edits need no manual extraction or MCP restart.
 
-When dependencies, initializers, database configuration, credentials, or schema change, Rails cannot safely reload all captured state. The watcher records a degraded reason and exits with status 75 so the process manager can restart it. Docker bind mounts may require polling; follow [Watch daemon](WATCH_DAEMON.md).
+When dependencies, initializers, database configuration, credentials, or schema
+change, the raw task exits 75 to request a fresh Rails boot. The managed launcher
+handles that restart internally; an external supervisor must handle it for the
+raw task. Do not add the bare task to Foreman. Docker bind mounts may require
+polling. See the [low-interaction workflow](AUTOMATIC_MAINTENANCE.md) for ownership,
+hooks, and the checks that establish automatic maintenance is active.
 
 The watcher maintains the structural index. If semantic retrieval is enabled, also run `bin/rails woods:embed_incremental` to update vectors. Without a resident watcher, run `bin/rails woods:incremental` after changes. Use a full `woods:extract` after major upgrades or when validation reports drift. CI and shared-artifact patterns are covered in [Incremental extraction](INCREMENTAL_EXTRACTION.md).