woods 2.0.0.beta3 → 2.0.0

data/docs/RETRIEVAL_GUIDE.md

@@ -63,7 +63,7 @@ Keyword results are scored by how many distinct fields matched (identifier, sour
 
 ## Configuring Retrieval
 
-Retrieval requires an embedding provider and a vector store. Set these in `config/initializers/woods.rb`.
+Semantic retrieval requires an embedding provider and a vector store. Configure these in `config/initializers/woods.rb` before embedding. For provider-free ranked retrieval over extraction output, use [explicit lexical mode](#embedding-free-lexical-retrieval) in the MCP process environment instead.
 
 ### Presets (recommended)
 
@@ -180,7 +180,12 @@ WOODS_RETRIEVAL_MODE=lexical bundle exec woods-mcp-start ./tmp/woods
 For a Ruby-built retriever, set `config.retrieval_mode = :lexical` and supply a
 populated metadata store to `Builder#build_retriever`. The packaged MCP server
 loads published unit JSON itself; it does not boot Rails or read current source
-files. Neither path constructs an embedding provider or vector adapter. The
+files. Neither path constructs an embedding provider or vector adapter. If a
+no-provider error suggests only OpenAI, Ollama or `search`, explicit lexical
+mode is still available from `2.0.0.beta3`: set the variable in the MCP client
+configuration and restart that server. Confirm `woods_status.retriever.mode`
+is `lexical`; setting it only in a Rails initializer does not configure a
+separate MCP process. The
 existing `:semantic` mode remains the default; provider failures never switch
 modes automatically. A Rails initializer is not loaded by the standalone MCP
 process, so set the environment variable in that process's client configuration.
@@ -191,15 +196,25 @@ and validations). Exact full identifiers rank first, with ambiguous typed owners
 retained. Other ties are deterministic. Responses name the lexical mode and
 matching fields/terms; runtime-field hits include the selected published runtime
 values. Lexical Ruby results leave the semantic-only `type_rank_context` table
-`nil`; they do not report a global vector rank or vector fallback. The top 20 eligible positive matches are considered for the
-output budget; this is ranked discovery, not an exhaustive match listing. Explicit
+`nil`; they do not report a global vector rank or vector fallback. The top 20
+eligible positive matches form the candidate shortlist for the output budget.
+The lexical header reports `sources included`, `candidates considered`, and
+`candidate limit: 20`. Included sources count the actual returned entries;
+considered candidates count the shortlist after filtering and the limit, not
+all matches or all documents examined. This is ranked discovery, not an
+exhaustive match listing. Explicit
 `types` filters override default exclusions, as in semantic retrieval, and apply
 before that limit. A query with no lexical evidence returns no matches; unrelated
 graph hubs are never added. Query-seeded graph ranking is evaluation-only.
 
 The budget covers headers, matching explanations and truncation notices using a
 labelled character-based estimate, not an exact provider tokenizer. Full source
-remains available through `lookup`. Very small budgets can omit all sources. The
+remains available through `lookup`. Count text is charged before source selection;
+final counts do not trigger a second selection pass. Very small budgets can omit
+all sources or clip the header itself. Zero candidates means no lexical matches;
+positive candidates with zero included sources means no source entry fit the
+available budget. The same counts apply to full, compact, outline and scoped
+retrieval, agreeing with returned source attribution. The
 reader pins one published generation for building and querying its immutable
 lexical snapshot, rebuilding after publication. Corrupt units fail explicitly;
 they cannot quietly become a successful partial index. Older flat indexes rebuild
@@ -276,16 +291,65 @@ result    = retriever.retrieve("what validations does Order have?")
 
 ---
 
-## Degradation Tiers
-
-Retrieval degrades gracefully when components are unavailable. The Retriever itself does not implement explicit fallback tiers, degradation happens naturally through how each component handles errors:
+## Semantic corpus diagnostics
+
+Extraction and embedding publish different data. `woods_status.ready` describes
+the structural index; neither that flag nor bootstrap `hydrated` proves that
+semantic retrieval has indexed records. Provider detection alone can succeed
+before the first embedding run.
+
+Supporting readers report `woods_status.retriever.corpus`:
+
+- `state`: `empty`, `metadata_only`, `vectors_only`, `nonempty`, or `unknown`.
+- `vectors` and `metadata`: each has `count`, `by_type`, and `untyped_count`.
+  Counts describe stored entries, including chunks; they are not distinct
+  extracted-unit counts or a completeness certificate. Missing type labels are
+  reported separately rather than assigned to a guessed type.
+- Unknown counts are `null`. Diagnostics use an explicit local-store capability;
+  they do not query remote stores to discover their counts. A missing capability
+  does not mean the backend is empty or broken.
+
+When both semantic stores are known empty, `codebase_retrieve` reports
+`empty_index` with recovery guidance instead of presenting an empty match as
+evidence that application code is absent. Run `woods:embed` in the application
+with the intended provider and storage configuration, then reload or restart
+the reader. Restart after changing provider or store configuration; reload
+refreshes the stores of the existing retriever. Alternatively, explicitly select `WOODS_RETRIEVAL_MODE=lexical`
+in the MCP process and restart for ranked retrieval over the published units.
+Woods does not change retrieval mode automatically.
+
+Metadata-only stores can still answer some keyword, direct, and graph queries;
+they are not blocked by this diagnostic. Source-empty units deliberately retain
+metadata without vectors, so the two counts need not match. Nonempty stores can
+still have missing types, stale vectors, or provider failures. Inspect the
+query result and embedding evidence before claiming coverage. The type-rank
+table's metadata count describes the retrieval metadata store, not all units
+in the structural index.
+
+Older readers may omit `retriever.corpus`; record the reader revision separately
+from the index writer version and verify the embedding artifacts directly.
+Explicit lexical mode does not use or report semantic corpus counts.
 
-- **Embedding provider unavailable**: `codebase_retrieve` returns a structured configuration error. Check `woods_status` for retrieval readiness.
-- **Vector store unavailable**: vector and hybrid strategies fail at query time. Keyword and graph strategies remain available for direct calls to `SearchExecutor`.
-- **Metadata store error**: the structural context overview (unit counts by type) is silently omitted; `Retriever#build_structural_context` rescues `StandardError` and returns `nil`. The retrieval result is still returned without the overview.
-- **Graph store unavailable**: graph expansion in hybrid strategy produces no graph candidates; vector and keyword candidates are still ranked and returned.
+## Degradation Tiers
 
-In all cases, errors in individual components produce empty candidate sets for that source rather than raising through the `Retriever`. Configure circuit breakers via `Woods::Resilience::CircuitBreaker` on external providers (Qdrant, OpenAI) for production deployments.
+The MCP boundary distinguishes missing configuration, empty stores, and failed
+stores. A failure is not evidence that no application code matches:
+
+- **No embedding provider configured:** `codebase_retrieve` reports a configuration
+  error with embedding and explicit lexical-mode options.
+- **Both semantic stores known empty:** the tool reports `empty_index`; see the
+  [corpus diagnostics](#semantic-corpus-diagnostics) above.
+- **Failed dump hydration:** the tool reports `degraded_index` rather than serving
+  a clean empty result. Repair or regenerate the named embedding artifact.
+- **Query-time storage failures:** vector, metadata, and graph adapter exceptions
+  are translated into store errors and reported as `degraded_index` by MCP.
+  This includes failures building the metadata overview; that failure is not
+  silently omitted. Direct Ruby callers should handle `Woods::Retriever::StoreError`.
+
+Provider failures and missing metadata for returned candidates have their own
+error paths. Preserve their diagnostics; do not silently switch modes or treat
+an exception as an empty match. Positive corpus counts do not override these
+checks.
 
 ---
 
@@ -353,6 +417,10 @@ bundle exec rake woods:embed
 | `text-embedding-3-small` (default) | 1536 |
 | `text-embedding-3-large` | 3072 |
 
+Woods' pgvector HNSW adapter supports at most 2,000 dimensions. For the large
+model, request a supported output width explicitly or choose another backend;
+see [pgvector configuration](CONFIGURATION_REFERENCE.md#pgvector-postgresql).
+
 **Ollama default model:** `nomic-embed-text`. Dimensions are detected dynamically on first embed.
 
 ---

data/docs/SOURCE_FRESHNESS.md

@@ -4,7 +4,7 @@
 application inputs with source bytes visible to the reader. It is separate from
 index age, HEAD equality, daemon liveness and external database/runtime state.
 
-This capability is unreleased after `2.0.0.beta2`. Check the installed gem's
+This capability is included in Woods `2.0.0`. Check the installed gem's
 `woods-extract --help`, `rake -T woods:source_status`, and `woods_status` schema
 before using it; upgrading the plugin alone does not upgrade Woods.
 

data/docs/TOKEN_BENCHMARK.md

@@ -13,7 +13,8 @@
 > When the optional [`tokenizers`](https://github.com/ankane/tokenizers-ruby)
 > gem is installed, the Ollama path uses the real BERT WordPiece tokenizer
 > (`Woods::Embedding::TokenCounter`) instead of this heuristic. The 4.0
-> divisor below is what the gem falls back to everywhere else.
+> divisor applies to the OpenAI/default path; Ollama falls back to 1.5 when
+> its tokenizer is unavailable.
 
 This is a historical record of the benchmark that picked 4.0 over the
 original 3.5 divisor. It is cited from five places in `lib/` as the evidence
@@ -35,17 +36,21 @@ for that choice, keep the numbers below intact if you edit this doc.
 | 3.8 | 16.2% | 42.5% |
 | **4.0 (shipped)** | **10.6%** | **35.4%** |
 
-Mean chars/token across the corpus was **4.41** (range 3.94–5.42). The
-heuristic always overestimated, never underestimated, across all 19 files,
-which is what makes it safe for token-limit enforcement even at its worst
-case. Code lines and comment/YARD lines had similar ratios (4.38 vs. 4.27
-chars/token), no separate handling needed for either.
+Mean chars/token across the corpus was **4.41** (range 3.94–5.42). These
+aggregate results favor the 4.0 divisor for this sample; they do not establish
+an upper bound on token counts. The recorded range includes values below 4.0,
+so the heuristic can underestimate. Code lines and comment/YARD lines had
+similar ratios (4.38 vs. 4.27 chars/token) in this sample.
+
+The character estimate covers only the text passed to the counter. It does not
+bound the serialized MCP response: text rendering, structured output, provenance,
+and JSON framing can add bytes and tokens beyond that input.
 
 ## What shipped
 
 **The divisor changed from 3.5 to 4.0.** It roughly halves the mean
-overestimate (26.2% → 10.6%) while keeping the conservative
-always-overestimates property, at zero new runtime dependencies. The
+error (26.2% → 10.6%) at zero new runtime dependencies. It remains an estimate,
+not a guarantee that arbitrary input fits a model's token limit. The
 constant lives in one place now (`Woods::TokenUtils::CHARS_PER_TOKEN_BY_PROVIDER`),
 not scattered across call sites, see `lib/woods/token_utils.rb` for the
 current definition and `docs/EMBEDDING_MODELS.md` for the Ollama-side ratio.
@@ -53,8 +58,9 @@ current definition and `docs/EMBEDDING_MODELS.md` for the Ollama-side ratio.
 **tiktoken_ruby was deliberately not added as a runtime dependency.** A 10.6%
 mean error is acceptable for chunking decisions, budget estimates, and
 truncation; a native-extension dependency for marginal accuracy gains wasn't
-worth it. The optional `tokenizers` gem covers the case where exact counts
-matter more (see above).
+worth it. The optional `tokenizers` gem provides counts for its supported BERT
+WordPiece tokenizer, not every model. Strict token-limit enforcement requires
+the tokenizer used by the target model.
 
 ## Reproducing this benchmark
 

data/docs/TROUBLESHOOTING.md

@@ -8,7 +8,7 @@ This guide covers the most common problems encountered when installing, extracti
 
 | Error message | Cause | Fix |
 |---------------|-------|-----|
-| `No manifest.json found` | Wrong index path or no published generation | Use the path visible to the server process; run `woods:validate` |
+| `Could not resolve a published Woods index` (older versions: `No manifest.json found`) | Wrong index path or unresolved published generation | Select the existing index using a path visible to the server process; see [startup diagnostics](#index-cannot-be-resolved-at-startup) |
 | `uninitialized constant Rails` | Not running inside Rails app | Run via `bundle exec rake` in Rails root |
 | `type "vector" does not exist` | pgvector not installed | `CREATE EXTENSION vector` in PostgreSQL |
 | `Connection refused (localhost:11434)` | Ollama not running | `ollama serve` |
@@ -23,6 +23,7 @@ This guide covers the most common problems encountered when installing, extracti
 | `No such container` | Wrong container name | Check with `docker ps --format '{{.Names}}'` |
 | `JSON parse errors` (MCP) | Rails boot noise on stdout | Remove `puts` calls from initializers |
 | Query timeout | Large table, no scope | Add scope conditions to narrow results |
+| `Extraction failed for …; the previous generation remains active` | A consumer handled a source error during incremental extraction or refresh (included in Woods `2.0.0`) | Fix the logged source error and retry the [complete batch](INCREMENTAL_EXTRACTION.md#handled-source-errors-and-retry); watch keeps it pending |
 | Empty extraction output | `eager_load!` failure | Check for `NameError` in boot output |
 | Git metadata missing | Shallow clone in CI | Use `fetch-depth: 0` for complete history |
 | Parallel tool calls all fail | MCP client batches calls | Send calls sequentially, validate params first |
@@ -52,13 +53,44 @@ version. See [manifest writer provenance](PUBLISHED_INDEX.md#manifest-writer-pro
 
 If a tool call fails with **"Tool not found: … not available in the installed Woods v…"**, the client is asking for a tool a newer gem provides. Run `bundle update woods` and reconnect the MCP server, then retry.
 
+### Watcher startup or planned restart fails
+
+Managed `woods-watch` startup is **included in Woods `2.0.0`**; record the
+loaded version/path and revision, then verify executable and generator help.
+If changing an initializer stops every Foreman process, replace a bare
+`woods:watch` entry with the [managed setup](WATCH_DAEMON.md#managed-development-startup).
+
+Read launcher logs and `woods_status` supervision records separately from daemon
+liveness and index freshness. `retrying` means the last generation remains usable
+while boot is retried. A parked ownership/protocol conflict requires correcting
+the selected owner or installed command and restarting that owner; do not delete
+claim files or kill PIDs taken from status. No index-visible record exists before
+the first boot resolves the application's output directory.
+
+Unset `WOODS_WATCH_IDLE_TIMEOUT` in managed modes. If the boot deadline is reached,
+diagnose Bundler/initializer startup before increasing `--boot-timeout`; a valid
+long extraction has a separate readiness state and is not bounded by that clock.
+If setup created a Procfile but normal `bin/dev` still only launches Rails, choose
+Puma or explicitly run the selected Foreman command. The generator never rewrites
+`bin/dev` or starts services during preview.
+
+If installation reports a pending transaction, use `woods:watch --operation
+recover` through the Rails generator, initially with `--pretend`; see
+[owned setup recovery](WATCH_DAEMON.md#ownership-updates-and-removal). That Rails
+command boots the application first. For broken initializers use the documented
+direct bundled Ruby helper, which does not boot Rails or require task discovery.
+Both refuse to overwrite intervening edits. A Puma setup refusal for
+`config/puma/development.rb` means the default
+configuration would bypass the generated plugin; select an external/Foreman
+arrangement instead of installing an inactive directive.
+
 ### Semantic graph validation errors
 
 In development versions containing #413, `woods:validate` rejects graphs that
 parse as JSON but disagree with their indexes. Errors name the section and
 identity, for example `reverse["http_api"]: missing "Order"`, a duplicate typed
-variant, or an indexed unit absent from `nodes`. This is unreleased after
-`2.0.0.beta2`; check the installed gem before expecting these diagnostics.
+variant, or an indexed unit absent from `nodes`. This is included in Woods
+`2.0.0`; check the installed gem before expecting these diagnostics.
 
 Keep the failing generation and report the exact errors. Run a full extraction
 in a fresh application process with the intended bundle, then validate again.
@@ -87,7 +119,7 @@ Scoped resets leave corrupt state untouched. Valid state keeps any unrelated
 operation entries, and missing state remains a no-op without creating a file.
 A permission failure must be corrected before repair can succeed.
 
-This recovery is unreleased after `2.0.0.beta2`; check the installed version.
+This recovery is included in Woods `2.0.0`; check the installed version.
 Older versions report corrupt state as nothing to repair. Stop pipeline writers,
 back up the configured guard state's `pipeline_guard.json`, and remove only that
 file before restarting, or upgrade to a version containing the fix.
@@ -159,7 +191,12 @@ For subsequent runs, use incremental mode instead of full extraction:
 bundle exec rake woods:incremental
 ```
 
-Incremental extraction only re-extracts files that changed since the last run. It skips unchanged units and is typically 5-10× faster.
+Incremental extraction dispatches the selected changed paths, including affected
+concern consumers and whole-app extractors whose trigger paths changed. The default
+Git range is `HEAD~1`; pass an explicit range or `CHANGED_FILES` for other batches.
+It can reduce extraction work, but Rails boot, graph rebuilding, and publication
+still contribute to runtime. Measure the improvement in your application; Woods
+does not guarantee a speedup. See the [incremental contract](INCREMENTAL_EXTRACTION.md).
 
 ---
 
@@ -227,7 +264,7 @@ dependents after an incremental run.
 identities when restoring and updating the graph.
 
 **Fix:** Check whether the installed version includes B-193; this fix is
-unreleased. After upgrading to a version containing the fix, run
+included in Woods `2.0.0`. After upgrading to a version containing the fix, run
 `bundle exec rake woods:extract` once to rebuild lost reverse dependencies.
 Loading an already damaged graph does not restore discarded entries. See the
 [incremental graph contract](INCREMENTAL_EXTRACTION.md#the-contract).
@@ -238,7 +275,7 @@ Loading an already damaged graph does not restore discarded entries. See the
 most files as `change_frequency: new` in a shallow CI checkout.
 
 **Cause:** A shallow clone truncates HEAD ancestry. The shallow-checkout guard is
-unreleased after 2.0.0.beta2: current source omits git enrichment and warns once,
+included in Woods `2.0.0`: Woods omits git enrichment and warns once,
 rather than treating the truncated history as complete. If repository depth
 cannot be verified, enrichment is also omitted; check git access and version.
 
@@ -255,12 +292,33 @@ clone), then run full extraction to replace retained metadata:
 Two commits can suffice for an incremental diff, but do not establish the full
 ancestry needed for churn metadata.
 
+### Git executable is missing from the extraction environment
+
+**Symptom:** Extraction logs `Git history unavailable: git executable was not
+found in PATH`, and newly extracted units have no `metadata.git`. Older builds
+can omit this enrichment silently when the executable is missing.
+
+**Fix:** Run `git --version` in the same container and environment that runs
+extraction. Install Git 2.31 or newer there, ensure its executable is on `PATH`,
+then run full `woods:extract` to refresh every unit's history. A working Git
+installation on the host does not provide Git inside an application container.
+
+Extraction continues without inventing zero-commit history. The warning appears
+once per extractor instance when the application has a `.git` entry or an
+explicit `WOODS_GIT_DIR`/`GIT_DIR` setting. A source archive with neither remains
+supported and quiet. `GIT_BRANCH`/`GIT_SHA` provenance fallback is unchanged;
+those values identify a build but cannot supply per-file history.
+
+This diagnostic is emitted during extraction. `woods_status.ready` and a
+manifest Git SHA do not establish that per-unit history was available, and
+`recent_changes` returning no results does not prove no files changed.
+
 ---
 
 ### Git enrichment warns that history could not be read completely
 
-Current source uses an explicit merge-diff mode requiring **Git 2.31 or newer**.
-This is unreleased after 2.0.0.beta2: first confirm the installed Woods version.
+Woods 2.0 uses an explicit merge-diff mode requiring **Git 2.31 or newer**.
+First confirm the installed Woods version.
 Check `git --version` inside the same container/process environment as extraction,
 and upgrade git if it is older. On a supported version, check that the application's
 `HEAD` and object store can be read using the same `WOODS_GIT_DIR` setting.
@@ -292,29 +350,62 @@ When it does not, the git keys are omitted from every unit, provenance records
 `"unknown"`, and one warning names git's own reason. Absent keys mean "not
 known"; they never mean "brand new".
 
-**Fix:** Point `WOODS_GIT_DIR` at the *canonical* git directory, the one the
-worktree's `gitdir:` pointer ultimately leads to, and make sure it is mounted:
+**Fix:** Restore access to both the worktree-specific Git directory and the
+shared objects and refs using the mount layouts below, then run a full
+`woods:extract` to replace retained metadata.
+
+### Git directory mounts for linked worktrees
+
+`WOODS_GIT_DIR` is passed directly to Git's `--git-dir`. It selects that
+directory's `HEAD` for manifest provenance, per-file history, and incremental
+diff ranges. **For a linked worktree, pointing it at the shared `.git` root
+selects the primary checkout's HEAD.** A successful Git command alone does
+not prove Woods is reading the intended branch.
+
+First inspect Git metadata on the host, from the intended worktree:
 
 ```bash
-# docker-compose.yml, mounting the parent repository's git directory
-#   volumes:
-#     - /path/to/repo/.git:/canonical-git:ro
-WOODS_GIT_DIR=/canonical-git bundle exec rake woods:extract
+git -C /path/to/worktree rev-parse --absolute-git-dir
+# Example: /path/to/repo/.git/worktrees/wt
+git -C /path/to/worktree rev-parse --path-format=absolute --git-common-dir
+# Example: /path/to/repo/.git
+git -C /path/to/worktree rev-parse --abbrev-ref HEAD
+git -C /path/to/worktree rev-parse HEAD
 ```
 
-`WOODS_GIT_DIR` wins over whatever the worktree pointer says, and applies to
-every git call Woods makes: per-unit enrichment, `manifest.json` provenance,
-and the diff range `woods:incremental` resolves. All three run through
-`Woods::GitCommand.argv`, so the override cannot reach two of them and miss the
-third.
+The worktree ID in this example is `wt`. Use the ID returned by Git metadata;
+it need not match the branch name. Choose one of these layouts:
+
+- **Same-path mount:** mount the complete shared directory read-only at its
+  original absolute path (`/path/to/repo/.git:/path/to/repo/.git:ro`). With the
+  application's existing `.git` pointer resolvable, leave `WOODS_GIT_DIR`
+  unset and remove conflicting Git-directory overrides from the environment.
+- **Relocated mount:** mount that complete directory read-only at a new path
+  (`/path/to/repo/.git:/mounted-common:ro`), including `objects`, `refs`, and
+  `worktrees`. Select the worktree-specific directory inside it:
+
+  ```bash
+  WOODS_GIT_DIR=/mounted-common/worktrees/wt bundle exec rake woods:extract
+  ```
 
-**`GIT_DIR` alone is not enough for a linked worktree.** Woods honors git's own
-`GIT_DIR` and `GIT_COMMON_DIR` because git does, but setting `GIT_DIR` to a
-worktree's private git directory only moves the failure: the `commondir`
-pointer inside it is relative, so it still resolves to a path that is not
-mounted, and `GIT_COMMON_DIR` does not override it. Either mount the canonical
-git directory at the same absolute path the pointer names, or use
-`WOODS_GIT_DIR`.
+Mounting only the private worktree directory can leave its `commondir` pointer
+without access to shared objects and refs. Git's own environment variables are
+inherited by the subprocess; check any existing `GIT_DIR` and `GIT_COMMON_DIR`
+settings when diagnosing the effective layout. The complete layouts above
+preserve both worktree identity and shared storage.
+
+In the extraction container, verify the relocated selection against the host
+branch and exact SHA before extracting (replace `/app` and `wt` as needed):
+
+```bash
+git --git-dir=/mounted-common/worktrees/wt --work-tree=/app -C /app rev-parse --abbrev-ref HEAD
+git --git-dir=/mounted-common/worktrees/wt --work-tree=/app -C /app rev-parse HEAD
+```
+
+After fixing the selection, run full `woods:extract` and verify the published
+manifest. Incremental extraction can retain older per-file Git metadata.
+A commit alone does not necessarily trigger the source-file watcher; run a
+full extraction when current history and provenance are required.
 
 ---
 
@@ -374,29 +465,39 @@ retries after a later filesystem event.
 
 ### `manifest.json` shows the wrong branch (or `git_branch: "unknown"`) in a worktree
 
-**Symptom:** `git_branch` / `git_sha` in `manifest.json` name a different branch than the worktree is actually on, or report `"unknown"`. The extracted units themselves are correct, only the provenance metadata is off.
+**Symptom:** `git_branch` / `git_sha` in `manifest.json` name a different
+branch or SHA than the intended worktree, or report `"unknown"`.
 
-**Cause:** In a linked git worktree, `.git` is a *file* containing a `gitdir:` pointer to the real git directory, often an absolute host path. When extraction runs where that path can't be resolved (e.g. inside a container where the host path isn't mounted), git can't read the ref. Woods now reports `"unknown"` in that case rather than emitting a stale, misleading value (previously it fell back to a baked `GIT_BRANCH`/`GIT_SHA` build arg).
+**Cause:** An unreachable `.git` file's `gitdir:` pointer prevents Git from
+resolving the worktree's HEAD. An override selecting the shared `.git` root
+instead resolves the primary checkout's HEAD successfully. That wrong selection
+also affects per-file history and HEAD-based incremental ranges.
 
-**Fix:** Make the worktree's git directory reachable from the extraction environment, for example, mount the parent repository (the directory the `gitdir:` pointer references) into the container, or run extraction from a normal (non-worktree) checkout. With the real git directory reachable, `git_branch`/`git_sha` resolve correctly. If the checkout legitimately ships without a `.git` at all (a source tarball, or a Docker `COPY` that excludes it), set `GIT_BRANCH` / `GIT_SHA` explicitly. Woods honors these when there is no `.git` at the root (or no git binary), but suppresses them when a `.git` *is* present but unresolvable (so a stale build arg can't mask a worktree).
+**Fix:** Follow [Git directory mounts for linked worktrees](#git-directory-mounts-for-linked-worktrees),
+compare the selected branch and exact SHA in the extraction environment, then
+run a full extraction. Compare the newly published manifest, not a retained
+generation. A commit without a source edit may leave the watcher idle.
 
-When the canonical git directory is mounted but not at the path the pointer
-names, set `WOODS_GIT_DIR` to where it actually is. It wins over the pointer for
-provenance and for unit-level git metadata both. Setting git's own `GIT_DIR` to
-the worktree's private git directory does not work: its `commondir` pointer is
-relative and resolves outside the mount.
+For a checkout legitimately shipped without `.git` (such as a source tarball),
+`GIT_BRANCH` / `GIT_SHA` can supply provenance. They are fallbacks only when
+`.git` is absent or Git is unavailable; a present but unresolvable `.git`
+reports `"unknown"` instead of substituting stale build arguments.
 
 ---
 
 ## MCP Server Problems
 
-### "No manifest.json" error when starting the Index Server
+<a id="no-manifestjson-error-when-starting-the-index-server"></a>
+
+### Index cannot be resolved at startup
 
-**Symptom:** `woods-mcp-start` exits with an error like `No manifest.json found at /path/to/...` even though extraction completed.
+**Symptom:** An Index MCP executable exits with `Could not resolve a published Woods index in: /path/to/...` even though extraction completed. This headline is included in Woods `2.0.0`; older versions say `No manifest.json found`. Both mean the selected index could not resolve its manifest, not that an atomic index needs a root manifest.
 
-**Cause:** The Index Server is using the container-internal path rather than the host-side path to the volume-mounted output. The server runs on the host and cannot access container filesystem paths.
+Embedded Index MCP startup through `IndexReader` also raises an `ArgumentError` with the selected directory and layout guidance when the marker cannot resolve a manifest, including malformed marker shapes such as `[]` or a numeric `payload` (included in Woods `2.0.0`). Earlier builds may expose a raw `TypeError` or `NoMethodError` for those shapes. Inspect the marker and preserve the failing index before attempting recovery.
 
-**Fix:** Use the host path in your `.mcp.json`:
+**Cause:** The selected directory is not the published index root, the published generation cannot be resolved, or the path is not visible to the MCP process. A container path is appropriate for a container process; a host process needs the host-visible path.
+
+**Fix:** Point at the existing index before extracting again. Check the examined directory in the error and the [MCP path precedence](CONFIGURATION_REFERENCE.md#environment-variables). For a host-side launch whose working directory contains `tmp/woods`, for example:
 
 ```json
 {
@@ -409,13 +510,7 @@ relative and resolves outside the mount.
 }
 ```
 
-Verify the output is accessible from the host:
-
-```bash
-ls ./tmp/woods/manifest.json
-```
-
-**Since Woods 2.0, a healthy index may not have `manifest.json` at the output root at all.** Extraction publishes each generation into an immutable `payloads/gen-<N>/` directory and points to it from `generation.json`. If the flat path is missing, check the payload path instead before assuming extraction failed:
+**Since Woods 2.0, a healthy index may not have `manifest.json` at the output root at all.** Extraction publishes each generation into an immutable `payloads/gen-<N>/` directory and points to it from `generation.json`. Inspect the marker and its payload in the MCP process's filesystem before assuming extraction failed (use the generation named by your marker):
 
 ```bash
 cat ./tmp/woods/generation.json                    # {"number": 42, "payload": "payloads/gen-42", ...}
@@ -427,7 +522,7 @@ Update their gate using the [filesystem layout contract](INDEX_LAYOUT.md), which
 includes Bash/jq and Python readers. An upload must pin and copy one complete
 payload before publishing its captured pointer; keep a failed copy unpublished.
 
-`woods-mcp-start` and `IndexReader` already resolve this automatically, this is only for manual inspection. If neither path has a manifest, your Docker volume mount is not configured correctly. See [DOCKER_SETUP.md](DOCKER_SETUP.md).
+`woods-mcp-start` and `IndexReader` resolve this automatically; these commands are for manual inspection. Legacy flat indexes use a root `manifest.json`. If neither layout resolves, check the selected path, pointer, payload and any volume mount. See [DOCKER_SETUP.md](DOCKER_SETUP.md) for container launches.
 
 ---
 

data/docs/UPGRADING_TO_2.md

@@ -2,12 +2,10 @@
 
 Woods 2.0 changes observable index identifiers, publication layout, vector-store reconciliation, and the supported MCP surface. Plan a clean re-index. Do not upgrade a shared or durable index in place without a backup and a rollback window.
 
-This guide assumes the last v1 release, 1.6.1, and targets 2.0.0.
+This guide covers the supported 1.6.x line and targets 2.0.0. Use the latest
+published 1.6.x security patch as the rollback version.
 
 <!-- release-state:upgrade-availability -->
-> RubyGems lists 2.0.0.beta3 as a prerelease. Pin it explicitly with
-> `gem "woods", "2.0.0.beta3"`; `~> 2.0` resolves only once
-> 2.0.0 is published.
 <!-- release-state:end -->
 
 ## Upgrade outcome
@@ -95,7 +93,11 @@ Also back up managed Obsidian/Unblocked destinations before allowing a mass stal
 
 ### 3. Choose a rollback point
 
-Keep the v1 Gemfile/lockfile commit and all durable-store backups until v2 extraction, MCP calls, retrieval, and exports are verified. Downgrading the gem does not translate v2 identifiers back to v1.
+Record and test a Gemfile/lockfile selecting the latest published 1.6.x security
+patch as the rollback bundle. If the current installation is older, verify that
+patched v1 bundle before beginning the v2 migration. Keep its commit and all
+durable-store backups until v2 extraction, MCP calls, retrieval, and exports are
+verified. Downgrading the gem does not translate v2 identifiers back to v1.
 
 ## Upgrade the application
 
@@ -142,6 +144,10 @@ bin/rails woods:validate
 bin/rails woods:stats
 ```
 
+Included in Woods `2.0.0`: `woods:clean` removes index artifacts but keeps
+the output directory and its hidden extraction guard. This stable guard lets
+concurrent writers coordinate safely; its presence does not mean an index remains.
+
 The clean extract is required for corrected identifier shapes. Do not use an incremental run as the first v2 extraction: after `woods:clean` there is no baseline, and v2 `woods:incremental` refuses that state rather than publishing a near-empty index as the application's complete truth.
 
 An interrupted extraction leaves readers on the last complete generation because Woods publishes `generation.json` only after the payload is complete. Re-run the task; do not delete a partial directory speculatively. A run that completes its payload but cannot publish the marker now fails loudly instead of reporting success, so treat a non-zero exit as work to redo rather than as a partial success.
@@ -327,7 +333,7 @@ Complete every applicable check:
 If verification fails:
 
 1. stop v2 MCP, watcher, embedding, and exporter processes;
-2. restore the v1 Gemfile and lockfile or deploy the recorded v1 commit;
+2. restore the tested, patched v1 Gemfile and lockfile or deploy its recorded commit;
 3. run the v1 `woods:clean` before restoring anything under the configured output directory;
 4. either restore the complete pre-upgrade v1 output-directory backup, or run a fresh v1 extraction and then restore its v1 `dumps/` and configuration artifacts;
 5. restore external vector-store and managed export backups when v2 modified them;