woods 2.0.0.beta2 → 2.0.0.beta4

data/docs/AGENT_SETUP.md

@@ -40,13 +40,23 @@ If the worktree contains unrelated changes, preserve them. Do not overwrite an e
 
 Use structural-only setup when the user wants code navigation, runtime Rails structure, dependencies, flows, or blast-radius analysis. Fourteen tools register in the normal packaged launch without an embedding provider.
 
-Discuss semantic retrieval only if the user needs natural-language `codebase_retrieve`. The choice depends on whether they prefer local Ollama or hosted OpenAI and which vector store fits their environment. See [Backend matrix](BACKEND_MATRIX.md).
+If the user wants ranked discovery through `codebase_retrieve`, offer [explicit lexical mode](RETRIEVAL_GUIDE.md#embedding-free-lexical-retrieval) over the published index without a provider or embeddings. Check that the installed version supports it, set `WOODS_RETRIEVAL_MODE=lexical` in the MCP process environment, restart that server, and verify `woods_status.retriever.mode`. Keep structural-only setup as the default unless this mode is requested.
+
+For semantic matching, discuss local Ollama or hosted OpenAI and the appropriate vector store separately; adding a provider still requires authorization. See [Backend matrix](BACKEND_MATRIX.md).
 
 Do not infer permission to configure Console MCP from a request to “set up Woods” or “set up MCP.” The Index Server reads generated code context; the Console Server can read live data.
 
 ## 3. Install on a branch
 
-Create or switch to the branch requested by the repository owner. Add only the development dependency:
+Create or switch to the branch requested by the repository owner. Select the
+published version using the [installation guide](GETTING_STARTED.md#1-install-the-gem).
+Before stable 2.x is published, use the exact published prerelease constraint
+from the README release table; `~> 2.0` will not select a beta or release candidate.
+Use the selected version's tag documentation and verify its capabilities before
+configuring features described on `main`.
+
+Add only the development dependency. The following constraint applies **after a
+stable 2.x release is published**:
 
 ```ruby
 # Gemfile
@@ -120,15 +130,94 @@ For Docker, extraction runs inside the Rails container. If Woods is installed on
 
 Reconnect the client and call `woods_status`. Confirm a current generation and non-zero unit counts before claiming setup works.
 
+### Managed Claude Code configuration
+
+`woods-agent-config` is available from Woods `2.0.0.beta3`.
+Check `bundle exec woods-agent-config --help` in the selected application bundle;
+use the manual client configuration below when it is absent. The supported
+client format is Claude Code (tested with 2.1.267).
+
+Create a private plan, inspect its paths and diff, then apply that same plan:
+
+```bash
+bundle exec woods-agent-config setup --client claude --scope project \
+  --root "$PWD" --instructions CLAUDE.md,AGENTS.md --plan /tmp/woods-setup.json --diff
+bundle exec woods-agent-config apply /tmp/woods-setup.json \
+  --client claude --scope project --root "$PWD"
+```
+
+Choose a new, unused plan filename for each preview. Preview writes only the
+requested plan file; it does not edit managed configuration. Plans contain the
+complete replacement bytes, including unrelated settings, and use mode 0600:
+keep them private and remove them when no longer needed. `show FILE` prints its
+summary; `show FILE --diff` checks the original snapshots and prints a unified
+diff. Applying a changed snapshot fails rather than replacing the new content.
+A repeated identical setup makes no configuration edits.
+
+| Selection | Managed files |
+|---|---|
+| `--scope project` | `<root>/.mcp.json`, explicitly selected `<root>/CLAUDE.md` and/or `AGENTS.md`, `<root>/.woods-agent-config.json` ownership receipt |
+| `--scope user` | `~/.claude.json`, explicitly selected `~/.claude/CLAUDE.md`, application-specific receipt in `~/.claude/` |
+
+With `CLAUDE_CONFIG_DIR`, user scope uses that directory's `.claude.json`,
+`CLAUDE.md`, and receipt instead. Instruction edits are opt-in with
+`--instructions`; existing selections carry forward on update. The command
+configures the Index Server. Client trust and project approval remain Claude
+Code settings; apply does not change them.
+
+Preflight runs the selected installed bundle, validates its index, and checks
+its actual registered capabilities. It does not boot Rails or contact an
+embedding provider. The bundle must already resolve in frozen mode; prepare
+its lockfile separately if Bundler reports a mismatch. Host mode uses the
+application's absolute Gemfile and index paths. `--index tmp/woods` is relative
+to the selected root. For Compose, also select `--mode compose --service web
+--container-root /app`; run the configuration command where Docker Compose can
+access that project. Preflight verifies the index and installed gem inside that
+service. Both host and container subprocesses have time limits.
+
+Use `update --plan FILE` with the same client/scope/root and the desired launch
+options to change the owned entry or instruction selection. Update explicitly
+records the current template and installed-gem evidence; background hooks never
+update configuration. `remove --plan FILE` previews deletion of owned content
+and does not require the application bundle or index to remain available.
+Use `--name NAME` consistently if the installation uses a nondefault server name.
+Apply each operation's saved plan with the same explicit client/scope/root.
+
+Ownership comes from the receipt and exact managed section, not from a server
+named `woods`. Existing unowned names, edited managed content, malformed JSON,
+duplicate markers, symlinks, and concurrent edits cause conflicts. Preserve the
+receipt for future update/removal. Unrelated servers, hooks, settings,
+instruction text, permissions, and line-ending conventions are retained;
+changing JSON may reformat its whitespace.
+
+Unreleased after `2.0.0.beta3`: apply and recovery coordinate on the actual
+managed file paths, including user configuration and shared instruction files.
+Two application roots sharing those files cannot apply overlapping plans at the
+same time. A competing operation reports a conflict; after it finishes, create a
+fresh preview if the saved plan's snapshots changed. Both applications keep
+their own ownership receipts. Do not delete an active coordination lock.
+
+Writes use atomic replacement per file and a private recovery journal beside
+the receipt. The plan summary names all adjacent `.woods.lock` files, the
+receipt `.lock`, and the `.pending` journal;
+a lock file may remain after completion. Multiple files are not one atomic
+transaction. An ordinary write failure restores original files when safe; an
+interruption or concurrent edit can retain the journal. Resolve reported
+conflicts, then use `recover --client claude --scope project --root "$PWD"`
+(or the original user scope). Recovery refuses to overwrite concurrent edits.
+Keep journals private because they contain original configuration bytes. A plan
+whose recovery journal would exceed 8 MiB is refused before any managed file
+is changed; reduce the selected configuration before applying.
+
 ## 7. Verify useful behavior
 
 Use a class known to exist in the application:
 
-1. Call `search` to obtain its exact identifier.
-2. Call `lookup` to confirm source and metadata are present.
+1. Call `search` to obtain its exact identifier and type.
+2. Call `lookup` with that identifier and type to confirm source and metadata are present.
 3. Call `dependents` with depth 1 or 2 to confirm graph edges are queryable.
 
-If `codebase_retrieve` reports that semantic search is disabled, that is expected for structural-only setup. Do not configure credentials merely to remove the message.
+If `codebase_retrieve` reports that semantic search is disabled, that is expected for structural-only setup. Do not configure credentials merely to remove the message. If lexical retrieval was requested, verify its mode with `woods_status` and make one `codebase_retrieve` call against the published index.
 
 ## 8. Offer automatic index maintenance
 
@@ -184,7 +273,7 @@ Verified capabilities:
 - Index Server connected: yes/no
 - woods_status current: yes/no
 - search/lookup/dependents checked: yes/no
-- semantic retrieval: disabled/enabled (provider)
+- retrieval: disabled/lexical/semantic (provider when semantic)
 - Console MCP: disabled/enabled (authorization)
 - automatic structural updates: disabled/enabled (process manager)
 
@@ -195,10 +284,12 @@ Never report a capability as enabled solely because its schema exists in source.
 
 ## Copyable prompt for an installation agent
 
-> Install Woods 2.x in this Rails repository using `docs/AGENT_SETUP.md`. Start with read-only preflight and preserve unrelated changes. Default to the structural Index Server; do not enable embeddings, Console MCP, HTTP transport, secrets, or purge overrides without asking me. Inspect generated files before migrating, run extraction and validation in the app's normal execution environment, configure a project-scoped MCP server in the same filesystem context as the application bundle and index, and verify `woods_status`, `search`, `lookup`, and `dependents`. Finish with the runbook's handoff report.
+> Install Woods 2.x in this Rails repository using https://github.com/lost-in-the/woods/blob/main/docs/AGENT_SETUP.md. Select a published version and follow that version's tag documentation and supported capabilities. Start with read-only preflight and preserve unrelated changes. Default to the structural Index Server; do not enable embeddings, Console MCP, HTTP transport, secrets, or purge overrides without asking me. Inspect generated files before migrating, run extraction and validation in the app's normal execution environment, configure a project-scoped MCP server in the same filesystem context as the application bundle and index, and verify `woods_status`, `search`, `lookup` with the discovered identifier and type, and `dependents`. Finish with the runbook's handoff report.
 
 ## Related guides
 
+- [Edit client adapters](CLIENT_HOOKS.md) for separately opt-in Claude/OpenCode edit hooks; MCP setup does not enable them.
+
 - [Getting started](GETTING_STARTED.md) for the human walkthrough.
 - [MCP servers](MCP_SERVERS.md) for client-specific configuration and server boundaries.
 - [Upgrade to Woods 2.0](UPGRADING_TO_2.md) for an existing 1.x installation.

data/docs/BACKEND_MATRIX.md

@@ -96,6 +96,11 @@ CREATE INDEX IF NOT EXISTS idx_woods_vectors_embedding_hnsw
   ON woods_vectors USING hnsw (embedding vector_cosine_ops);
 ```
 
+**Dimension limit:** Woods uses `vector_cosine_ops` HNSW, limited to 2,000 dimensions.
+The default 3,072-dimensional `text-embedding-3-large` output needs an explicit
+smaller provider output width or another backend. See the
+[pgvector configuration contract](CONFIGURATION_REFERENCE.md#pgvector-postgresql).
+
 **Performance notes:**
 - HNSW: ~5ms search at 10K vectors, ~20ms at 100K. Memory: ~1.5x vector size.
 - For codebase indexing (~1000-5000 units, potentially 5000-20000 chunks), HNSW is appropriate.
@@ -271,6 +276,12 @@ contract tests; it does not represent semantic quality. Other values raise
 
 `build_metadata_store` accepts `:in_memory` and `:sqlite`. Nothing else is implemented.
 
+Both adapters search Boolean fields as the JSON words `true` and `false`,
+with case-insensitive substring matching. Numeric values `1` and `0` remain
+separate from Booleans. Strings are searched without JSON quotes; objects and
+arrays use JSON text. Null or absent fields never match a field-scoped query.
+Whole-record search (`fields: nil`) searches serialized JSON, including keys.
+
 ### SQLite
 
 **Best for:** Local development, zero-dependency setups, testing, and every shipped preset except pure in-memory.
@@ -284,6 +295,11 @@ contract tests; it does not represent semantic quality. Other values raise
 - Single writer at a time
 - No network access
 
+Metadata search uses literal, ASCII-case-insensitive substring matching. Selected
+string fields include embedded NUL characters in the searchable text. With no
+field selection, search operates on serialized JSON, where NUL is represented
+as `\u0000`; a literal NUL query therefore does not match that escaped text.
+
 ### In-memory
 
 **Best for:** Testing, evaluation, small codebases.
@@ -318,6 +334,15 @@ A recursive-CTE graph store (MySQL 8.0+ or PostgreSQL, storing edges in a table
 
 Indexing can be triggered synchronously (rake task, inline) or from a background job. The pipeline itself is job-system-agnostic, it's synchronous Ruby, and the wrapper below is just scheduling and concurrency control. Use `Woods.extract!` for a full run; incremental runs need a changed-file list, so a job usually just shells out to `rake woods:incremental` (which computes that list from git) rather than calling `Woods.extract_changed!` directly.
 
+Both Ruby helpers hold the same heartbeat-maintained extraction lock as the
+tasks and watch daemon. `WOODS_LOCK_WAIT` controls the wait (600 seconds by
+default); timeout raises `Woods::Coordination::LockError`. Failed generation
+publication raises `Woods::ExtractionError`, allowing job retries instead of
+reporting unpublished work as success. The low-level `Woods::Extractor` remains
+an orchestration building block: callers using it directly own locking and
+publication-failure handling. Do not wrap the public helpers in a second Woods
+extraction lock.
+
 ### Sidekiq
 
 ```ruby

data/docs/CLIENT_HOOKS.md

@@ -0,0 +1,111 @@
+# Edit hooks for Claude Code and OpenCode
+
+Edit hooks are optional. MCP reads and `woods:watch` work independently of them.
+Check the installed gem exposes `woods:hook_refresh` before enabling these
+unreleased adapters; updating the plugin does not update the application gem.
+Start the client from the Rails application root, with an existing index.
+
+## Supported client contracts
+
+| Client | Verified version | Events covered |
+|---|---|---|
+| Claude Code | 2.1.267 | Successful `PostToolUse` for `Write` and `Edit`, using `tool_input.file_path` |
+| Claude legacy compatibility | Existing single-file `MultiEdit` shape | One `tool_input.file_path`; this is not multi-file patch support |
+| OpenCode | 1.18.27 | `tool.execute.after` for `apply_patch`, `write`, and `edit` |
+
+The OpenCode patch adapter reads the successful tool's `metadata.files` array.
+It includes every add, update, delete and move; a move becomes deletion of the
+old path plus addition of the new path. `write` uses `metadata.filepath` and
+`metadata.exists`; `edit` uses `metadata.filediff.file`. Patch text, source bytes,
+diagnostics, session identifiers and arbitrary shell commands are not parsed
+or placed in the queue. Actual client captures and their provenance live under
+`spec/fixtures/hooks/`; the exact metadata contract is pinned to
+[OpenCode v1.18.27 source](https://github.com/anomalyco/opencode/tree/v1.18.27/packages/opencode/src/tool).
+See the primary [Claude hook reference](https://code.claude.com/docs/en/hooks)
+and [OpenCode plugin reference](https://opencode.ai/docs/plugins/).
+
+Other clients and arbitrary mutation tools are unsupported. Unknown shapes for
+registered edit tools produce a short diagnostic instead of claiming refresh.
+Use the resident watcher or an explicit extraction for unsupported operations.
+OpenCode session-start/context hooks are not provided by this edit adapter.
+
+## Claude Code registration
+
+The Woods Claude plugin registers `woods-post-edit.sh` through its existing
+`hooks/hooks.json`. The wrapper selects the explicit Claude parser, then the
+shared runner handles queueing and extraction. Do not install a second copy of
+the same hook. Existing single-file `MultiEdit` compatibility remains registered.
+
+Enable the existing environment settings in the process launching the client:
+
+```bash
+export WOODS_HOOKS_ENABLED=1
+# Optional; relative to the application root:
+export WOODS_OUTPUT=tmp/woods
+```
+
+`WOODS_HOOKS_DISABLED=1` always wins. Enablement does not authorize Console MCP,
+provider calls, user configuration writes, or daemon lifecycle changes.
+
+## OpenCode project registration
+
+Keep the complete Woods `plugin/` directory at a stable path visible to the
+client. The Claude marketplace registration does not install a native OpenCode
+plugin. Create a project-local `.opencode/plugins/woods.js` containing this one
+import, replacing the path with that stable plugin location:
+
+```javascript
+export { default } from "/absolute/path/to/woods/plugin/hooks/woods-opencode.mjs";
+```
+
+OpenCode automatically loads project `.js` and `.ts` plugin files at startup.
+Use the `.js` wrapper name above; copying a standalone `.mjs` file into the
+autoload directory does not register it. The wrapper imports the complete
+adapter and shared runner; copying only the shell entry point is insufficient.
+Restart OpenCode after registration and set `WOODS_HOOKS_ENABLED=1` in its launch
+environment. This setup does not require npm packages or a host Rails bundle.
+
+The adapter uses OpenCode's `directory` as the application root and verifies
+that it belongs to the supplied git `worktree`. A Rails application nested
+inside a repository therefore uses its own index. Each affected path must stay
+inside that application root. Linked worktrees keep separate queues and indexes.
+
+## Queue, paths and recovery
+
+Both adapters use the same extraction eligibility, queue, command prefix,
+deadline, locks and active-daemon behavior described in
+[watch hook operation](WATCH_DAEMON.md#hooks-for-agent-sessions).
+`WOODS_HOOK_RAKE="docker compose exec -T app bundle exec rake"` runs extraction
+inside the application container. The host needs Bash 3.2 or later, Unix tools,
+and either jq or Ruby; OpenCode supplies its own JavaScript runtime.
+
+The complete event is validated before queueing. Empty/NUL paths, traversal,
+foreign-project paths and symlink path components are rejected. Deleted paths
+need not exist; contained symlinks are deliberately unsupported too. Spaces,
+commas, newlines and Unicode paths remain intact. Adapter input is limited to
+1 MiB and 1,000 affected paths; the OpenCode handoff is additionally limited to
+48 KiB. An unsupported oversized event requires explicit extraction or watch.
+Raw input uses a private temporary file during validation so the shell cannot
+silently remove bytes. The runner removes that file before publishing the
+path-only queue record, and cleans it up on handled exits.
+
+One immutable queue file holds a multi-file event. The owner batches up to
+16 files, 1,000 paths and 48 KiB without splitting an event. Existing single-path
+queue records remain readable. A successful task, including a confirmed no-op,
+acknowledges the batch; failure, timeout or daemon exit 75 retains every path.
+At-least-once delivery can repeat work after a crash or event replay.
+
+The OpenCode callback hands the event to a detached runner, which owns its
+existing deadline. Callback completion means handoff, not index publication.
+Inspect `<output>/hook.log`, the pending queue and the published generation to
+confirm refresh. Malformed input diagnostics omit the original tool payload.
+For rollback, disable hooks before removing registration; preserve pending
+multi-file records until a compatible runner has consumed them.
+
+## Optional Claude context
+
+Claude has a separate opt-in for bounded synchronous orientation and change-impact
+reminders. It does not change either client’s refresh queue contract and is not
+enabled by `WOODS_HOOKS_ENABLED`. Check the installed helper, timing, path-mapping
+and uncertainty contract in [bounded context hints](WATCH_DAEMON.md#optional-bounded-context-hints).
+OpenCode refresh support does not imply context delivery support.