woods 2.0.0.beta2 → 2.0.0.beta4

data/docs/RETRIEVAL_GUIDE.md

@@ -26,6 +26,16 @@ query
 | Assembly | `Woods::Retrieval::ContextAssembler` | Fills a token budget with ranked units, sectioned into structural / primary / supporting / framework blocks |
 | Orchestration | `Woods::Retriever` | Coordinates all four stages; returns a `RetrievalResult` with `context`, `sources`, `strategy`, `tokens_used`, and `trace` |
 
+### PageRank importance
+
+When a graph store supplies PageRank, ranking converts its scores to ordinal
+percentiles: the highest-ranked unit receives 1.0 and the lowest receives 1/n,
+where n is the number of entries in the PageRank map.
+Equal PageRank scores are ordered lexically by identifier before assigning
+percentiles. This preserves distinct ordinal weights while making ties independent
+of graph insertion order and Ruby version. It does not assign equal importance
+weights to tied units. Missing graph entries retain the metadata-based fallback.
+
 ### Search strategies
 
 `SearchExecutor` selects one of five strategies based on query classification:
@@ -38,13 +48,22 @@ query
 | `:hybrid` | `comprehensive` or `exploratory` scope | Runs vector + keyword + graph expansion, deduplicates |
 | `:direct` | `locate`/`reference` + `pinpoint` scope | Looks up identifiers directly in metadata store; falls back to keyword |
 
+Graph queries preserve identifier spelling and namespaces: `trace Billing::Invoice`,
+`trace ReviewAssignment`, and `trace review_assignment` resolve the subject before
+traversing its dependencies and dependents. An exact identifier takes precedence;
+an unqualified name without an exact match can seed up to three namespace matches
+in storage-identifier order. Use the qualified name to disambiguate. A missing
+qualified identifier does not fall back to a different namespace. For unqualified
+prose, metadata fallback searches subject terms without instructions such as
+`trace`, `follow`, or `who calls`.
+
 Keyword results are scored by how many distinct fields matched (identifier, source, metadata), not by the store's result order. Each matched field adds 0.25, capped at 1.0, so a result matching on identifier and source scores higher than one matching source alone.
 
 ---
 
 ## Configuring Retrieval
 
-Retrieval requires an embedding provider and a vector store. Set these in `config/initializers/woods.rb`.
+Semantic retrieval requires an embedding provider and a vector store. Configure these in `config/initializers/woods.rb` before embedding. For provider-free ranked retrieval over extraction output, use [explicit lexical mode](#embedding-free-lexical-retrieval) in the MCP process environment instead.
 
 ### Presets (recommended)
 
@@ -124,13 +143,95 @@ bundle exec rake woods:extract
 bundle exec rake woods:embed
 ```
 
+### Input integrity and source-empty units
+
+Embedding validates a native published generation's manifest, type listings and
+unit payloads before changing vector storage, metadata or checkpoints. A missing,
+corrupt or mismatched listed unit raises `Embedding input incomplete`; repair the
+extraction (or run a full `woods:extract`) before retrying embedding.
+`WOODS_ALLOW_PURGE=1` permits intentional mass deletion; it does not bypass this
+integrity check. Corpus reads hold one generation pin while collecting the input.
+
+Legacy flat indexes retain arbitrary unit filenames and do not require a manifest
+or type listing. A `payloads/` directory requires a valid publication pointer;
+missing or null pointers refuse embedding even when stale flat files remain.
+Malformed JSON now refuses embedding instead of silently dropping
+that file. Without an authoritative listing, a missing legacy file still denotes
+a deletion; regenerate extraction into the native publication layout for stronger
+completeness checks.
+
+When a current unit has no source text to embed, Woods retains its metadata and
+removes its superseded vectors, including old chunks. It records the current
+source hash without calling the provider; later incremental runs verify that the
+unit still prepares no text before accepting a matching no-vector checkpoint.
+Empty-vector reconciliation waits until all batches succeed, so a later provider
+failure does not retire those old vectors.
+
+---
+
+## Embedding-free lexical retrieval
+
+Opt into ranked retrieval over an extract-only published index:
+
+```bash
+WOODS_RETRIEVAL_MODE=lexical bundle exec woods-mcp-start ./tmp/woods
+```
+
+For a Ruby-built retriever, set `config.retrieval_mode = :lexical` and supply a
+populated metadata store to `Builder#build_retriever`. The packaged MCP server
+loads published unit JSON itself; it does not boot Rails or read current source
+files. Neither path constructs an embedding provider or vector adapter. If a
+no-provider error suggests only OpenAI, Ollama or `search`, explicit lexical
+mode is still available from `2.0.0.beta3`: set the variable in the MCP client
+configuration and restart that server. Confirm `woods_status.retriever.mode`
+is `lexical`; setting it only in a Rails initializer does not configure a
+separate MCP process. The
+existing `:semantic` mode remains the default; provider failures never switch
+modes automatically. A Rails initializer is not loaded by the standalone MCP
+process, so set the environment variable in that process's client configuration.
+
+Lexical results use field-aware BM25 scoring over identifiers, source paths,
+published source and selected runtime metadata (including callbacks, associations
+and validations). Exact full identifiers rank first, with ambiguous typed owners
+retained. Other ties are deterministic. Responses name the lexical mode and
+matching fields/terms; runtime-field hits include the selected published runtime
+values. Lexical Ruby results leave the semantic-only `type_rank_context` table
+`nil`; they do not report a global vector rank or vector fallback. The top 20
+eligible positive matches form the candidate shortlist for the output budget.
+The lexical header reports `sources included`, `candidates considered`, and
+`candidate limit: 20`. Included sources count the actual returned entries;
+considered candidates count the shortlist after filtering and the limit, not
+all matches or all documents examined. This is ranked discovery, not an
+exhaustive match listing. Explicit
+`types` filters override default exclusions, as in semantic retrieval, and apply
+before that limit. A query with no lexical evidence returns no matches; unrelated
+graph hubs are never added. Query-seeded graph ranking is evaluation-only.
+
+The budget covers headers, matching explanations and truncation notices using a
+labelled character-based estimate, not an exact provider tokenizer. Full source
+remains available through `lookup`. Count text is charged before source selection;
+final counts do not trigger a second selection pass. Very small budgets can omit
+all sources or clip the header itself. Zero candidates means no lexical matches;
+positive candidates with zero included sources means no source entry fit the
+available budget. The same counts apply to full, compact, outline and scoped
+retrieval, agreeing with returned source attribution. The
+reader pins one published generation for building and querying its immutable
+lexical snapshot, rebuilding after publication. Corrupt units fail explicitly;
+they cannot quietly become a successful partial index. Older flat indexes rebuild
+on each query because they lack an immutable generation identity.
+
+See [evaluation](EVALUATION.md) for measured query coverage and limits. Lexical
+matching cannot infer synonyms absent from the published text; a miss is not proof
+that application behavior is absent. Static Woods self-maps can opt into the same
+mode but remain static source maps, not resolved Rails runtime evidence.
+
 ---
 
 ## Running Retrieval
 
 ### MCP tool: `codebase_retrieve`
 
-The primary interface for agents. Available in the Index Server when an embedding provider is configured and `rake woods:embed` has been run.
+The primary interface for agents. Available with explicit lexical mode over extraction output, or with an embedding provider and completed `rake woods:embed` in the default semantic mode.
 
 ```
 codebase_retrieve(query: "how does billing work?")
@@ -165,6 +266,12 @@ result.sources      # => [{ identifier: "User", type: "model", score: 0.91, ...
 result.trace        # => RetrievalTrace with elapsed_ms, candidate_count, etc.
 ```
 
+`result.tokens_used` and `result.trace.tokens_used` count the final returned
+`context`, including the optional formatter output and type-rank table. Counting
+uses the same injected token counter or chars-per-token estimate as assembly;
+it does not guarantee an exact count for the downstream model. `budget` limits
+context assembly, so postprocessing can make the final count exceed it.
+
 Override the token budget per call:
 
 ```ruby
@@ -201,18 +308,29 @@ In all cases, errors in individual components produce empty candidate sets for t
 
 ### `similarity_threshold`
 
-Controls which vector search results are considered. Range: `0.0`–`1.0`. Default: `0.7`.
-
-```ruby
-config.similarity_threshold = 0.6  # Include less similar results (broader)
-config.similarity_threshold = 0.8  # Require higher similarity (narrower)
-```
-
-Lower values return more candidates, which can improve recall for broad queries at the cost of precision. Raise it if results seem loosely related.
+Deprecated and inert. Woods retains numeric validation (`0.0`–`1.0`) and
+readback for compatibility, but setting this option emits a warning. It has not
+filtered retrieval results; changing it does not change scores or candidates.
+Use explicit type, package or source-path scopes to select eligible units, and
+inspect returned ranking evidence to assess relevance. No new score cutoff is
+introduced by this deprecation.
 
 ### `max_context_tokens`
 
-Sets the default token budget for context assembly. Default: `8000`. The `budget` parameter on `codebase_retrieve` and `Retriever#retrieve` overrides this per call.
+Sets the default token budget for context assembly. Default: `8000`. Builder
+captures this value when constructing semantic or lexical retrievers, including
+cached retrievers. The `budget` parameter on `codebase_retrieve` and
+`Retriever#retrieve` overrides it per call; omitted and explicitly equal budgets
+share a cache entry. Changing configuration affects newly constructed retrievers.
+Direct `Retriever.new` callers can pass `default_budget:`; otherwise they retain
+8000. Custom MCP collaborators without `default_budget` also retain the legacy
+8000 fallback.
+
+The setting applies to the serving process's configuration. It is not embedded
+in `woods.json`; a standalone MCP process that does not load the host initializer
+uses its own default unless a tool call supplies `budget`. Token accounting uses
+the configured estimator, and very small budgets can still include formatting
+overhead.
 
 ```ruby
 config.max_context_tokens = 12_000  # More context per retrieval
@@ -250,6 +368,10 @@ bundle exec rake woods:embed
 | `text-embedding-3-small` (default) | 1536 |
 | `text-embedding-3-large` | 3072 |
 
+Woods' pgvector HNSW adapter supports at most 2,000 dimensions. For the large
+model, request a supported output width explicitly or choose another backend;
+see [pgvector configuration](CONFIGURATION_REFERENCE.md#pgvector-postgresql).
+
 **Ollama default model:** `nomic-embed-text`. Dimensions are detected dynamically on first embed.
 
 ---
@@ -264,4 +386,143 @@ bundle exec rake woods:embed
 | Empty results for a known class name | Keyword strategy not finding the identifier | Try a conceptual query with `codebase_retrieve`; or use `search` for exact name lookup |
 | Very slow retrieval | Large vector index without HNSW index, or Qdrant cold start | For pgvector: create an HNSW index (see `BACKEND_MATRIX.md`). For Qdrant: check collection status |
 | `codebase_retrieve` tool listed but disabled | Embedding provider not configured or API key missing | Set `embedding_provider`, run `woods:embed`, and check `woods_status` |
-| Results clustered around one type | Diversity penalty insufficient for codebase shape | Lower `similarity_threshold` slightly and widen the query scope |
+| Results clustered around one type | Diversity penalty insufficient for codebase shape | Use explicit type filters or a more specific query; inspect ranking evidence |
+
+## Explicit package and source-path scopes
+
+Both `codebase_retrieve` and `search` accept `packages` and `source_paths` arrays.
+The Ruby API uses the same keyword arguments:
+
+```ruby
+retriever.retrieve('How are payments collected?', budget: 1200,
+                   packages: ['packs/billing'], source_paths: ['packs/billing/app'])
+```
+
+These filters select eligible published units **before candidate limits**. Each
+list uses OR; package, path, and type restrictions combine with AND. Empty or
+omitted scope lists add no restriction. Existing unscoped calls keep their behavior.
+
+- **Packages:** exact, case-sensitive published nearest owners. A parent package
+  excludes its nested packages unless both names are requested. `.` selects only
+  root-owned units. Missing ownership does not match a package restriction;
+  it can still match a source path. Unknown package names return an argument error;
+  a declared package with no eligible units is a valid empty scope.
+- **Paths:** application-relative directory prefixes, compared at segment
+  boundaries. `packs/billing` includes descendants, including nested packages,
+  and excludes `packs/billing_admin`. `.` covers all published application-relative
+  paths. Repeated separators and `.` are normalized; internal `..` segments are
+  resolved, while attempts to escape the root, absolute paths, Windows paths,
+  backslashes, and NUL are rejected. External gem paths and missing paths do not
+  match. No host `realpath` or current source-file reads determine ownership.
+- **Types and graph:** existing inclusion/exclusion rules remain authoritative.
+  Scoped discovery uses actual published unit types, including `gem_source` units
+  in the shared `rails_source` directory. Graph seeds and expansion stay within
+  eligibility. Existing bare graph edges can still be ambiguous between typed
+  units sharing an identifier; scope does not establish a missing edge target type.
+
+`RetrievalResult#applied_scope` reports normalized lists, `eligible_units`,
+`candidate_count`, `returned_units`, and `outcome` (`empty_scope`, `no_match`, or
+`matched`). A matched candidate need not fit a tiny output budget. Scoped MCP
+retrieval exposes this object in `structuredContent.data.applied_scope` and `_meta`,
+with typed source provenance in `structuredContent.data.sources`; the budget continues
+to apply to the context text. Scoped results omit `type_rank_context`, whose global
+rank fields would otherwise describe the wrong population.
+
+Search adds `applied_scope` beside its existing completeness evidence. Zero
+eligible units means an empty scope; positive eligibility with a complete zero-match
+search means no match. A partial zero-match search remains inconclusive.
+
+### Storage and cost
+
+Scope preparation reads the complete metadata snapshot from the selected store
+bundle. Packaged discovery and lexical retrieval keep that read and the query on
+one published generation. Discovery's deep-field scan budget applies to matching
+after scope preparation; it does **not** cap the initial metadata read.
+
+Vector search enumerates eligible raw vector IDs, including typed and chunk IDs,
+and searches every batch of at most 100 IDs before merging the best candidates.
+It embeds the query once per vector execution. In-memory, pgvector, and Qdrant
+adapters support this without adding package fields or re-embedding old vectors.
+Scoped pgvector queries materialize eligible rows before exact distance ranking;
+scoped Qdrant requests use exact search. Custom adapters must explicitly support
+native ID filters and ID enumeration; unsupported adapters return a degraded
+error instead of falling back to global results. Broad scopes can cost more than
+ordinary approximate search. Automatic package balancing is not enabled.
+
+See [scope evaluation](EVALUATION.md#explicit-scope-comparison) for the matched
+budget replay and its cross-boundary recall tradeoff.
+
+## Compact published evidence and API outlines
+
+`codebase_retrieve` and `Retriever#retrieve` accept an explicit `evidence` mode:
+
+- `full` (default) preserves existing source formatting and budget truncation.
+- `compact` chooses complete query-relevant methods and published concern display
+  blocks within each ranked unit, then relevant resolved runtime metadata.
+- `outline` returns declared method names, kinds, lexical owners and published
+  line ranges. It is an API orientation aid, not reconstructed Ruby signatures.
+
+```ruby
+retriever.retrieve('How are payments refunded?', budget: 1200, evidence: 'compact')
+# MCP:
+codebase_retrieve(query: 'How are payments refunded?', budget: 1200, evidence: 'compact')
+lookup(identifier: 'Billing::Invoice', type: 'model', evidence: 'outline', budget: 600)
+lookup(identifier: 'Billing::Invoice', type: 'model', evidence: 'compact', query: 'refund', budget: 800)
+```
+
+Ranking, candidate limits and explicit package/path scopes are unchanged. Selection
+uses the original query: method-name term matches weigh more than body matches;
+source order breaks ties. With no matching terms, or no lookup query, source order
+provides deterministic orientation. Nested blocks and definitions stay inside their
+complete containing method. A method that cannot fit is omitted whole; compact mode
+never substitutes a broken prefix. If a heredoc body lies outside its method's
+syntactic range, a `whole_source_fallback` span retains the complete published
+source or omits it whole when it cannot fit. Metadata fields are also included whole. More
+unit names fitting in an outline does not establish that their implementation was
+shown or that retrieval quality improved.
+
+The existing retrieval token counter also charges compact headers, notices and
+source spans. Compact lookup defaults to 2000 estimated tokens, using four
+characters per token. These are text-context budgets; MCP JSON envelopes and
+structured provenance add transport/output tokens. Very small budgets may return
+no evidence text. Check omission counts and increase the budget or request full
+source. `query` and `budget` apply only to compact/outline lookup; combining those
+modes with `include_source: false` or nonempty `sections` is an argument error.
+
+### Provenance and full-source verification
+
+MCP retrieval exposes evidence under `structuredContent.data.sources[].evidence`;
+compact lookup uses `structuredContent.data.evidence`. Ruby retrieval carries the
+same data in `result.sources`. Every record includes the typed unit owner, original
+published display path, source SHA256, generation status, selected span hashes,
+zero-based byte offsets (exclusive end), one-based line ranges and omitted spans.
+Span owners are **lexical declarations**, not inferred runtime dispatch owners.
+
+Coordinates are explicitly `published_unit`: they refer to the exact published
+`source_code` bytes. Physical source coordinates are unavailable. Model/controller
+units may contain synthesized headers or commented inlined concerns; the excerpt
+preserves those comments verbatim and labels concern display blocks. Inherited
+behavior may be described by runtime metadata without a corresponding source span.
+The server never opens host application source files to fill these gaps.
+
+Published lexical retrieval and lookup report the pinned generation when present.
+Legacy flat indexes and standalone semantic metadata stores do not establish a
+publication generation and report it unavailable. A current reader generation must
+not be substituted for an older semantic artifact's unknown generation.
+
+Each record supplies a `full_evidence` lookup call with the actual `type`,
+`identifier`, `evidence: 'full'` and `source_sha256`. Pass that hash back to verify
+the same source bytes; changed source produces a typed `stale_index` refusal.
+Ordinary full lookup remains available without the hash when deliberately
+inspecting the newest publication. Optional `type` disambiguates names shared by
+multiple unit types and preserves actual types within mixed storage directories.
+Full source is never disabled by compact mode. See [evaluation](EVALUATION.md#compact-evidence-comparison-403)
+for measured tradeoffs and the limits of returned-unit metrics.
+
+### Source-owner selection remains an offline experiment
+
+The [owner-overlap evaluation](EVALUATION.md#source-owner-overlap-experiment-412)
+compares complete-span overlap against current score order. It adds no retrieval
+configuration or MCP option. Display paths alone cannot prove original ownership,
+especially for inlined concern display; representative runtime task evidence is
+still needed before changing which units receive context budget.

data/docs/RUNTIME_TRACING.md

@@ -0,0 +1,71 @@
+# Ruby runtime trace enrichment
+
+`Woods::RubyAnalyzer::TraceEnricher` records Ruby method events during a block
+and can attach that evidence to Ruby method units. This is a Ruby API; Rails
+session tracing and the MCP `trace_flow` tool are separate features.
+
+```ruby
+require 'woods/ruby_analyzer'
+
+traces = Woods::RubyAnalyzer::TraceEnricher.record { application.run }
+units = Woods::RubyAnalyzer.analyze(paths: ['lib'], trace_data: traces)
+# Or enrich units already analyzed:
+Woods::RubyAnalyzer::TraceEnricher.merge(units: units, trace_data: traces)
+```
+
+## Caller evidence
+
+Each event contains `class_name`, `method_name`, `event` (`call` or `return`),
+`method_kind`, `path`, `line`, `caller_class`, `caller_method`,
+`caller_method_kind`, and `return_class`.
+For an observed `Caller#invoke` calling `Callee#run`, the callee's call and
+return events record `caller_class: 'Caller'` and `caller_method: 'invoke'`.
+The caller method field contains the method name, not a qualified backtrace label.
+
+The caller is the **nearest Ruby method observed by this recording**. The
+recorder does not capture native method frames or block frames, and does not
+infer methods already on the stack when recording begins. An unknown caller
+has all three caller fields set to `nil`; merge omits those unknown caller entries.
+This is scoped execution evidence, not a complete application call graph.
+
+Stacks belong to one recording and are separate for each fiber. Recording
+captures the current thread, including fibers running on it, and excludes
+other threads. To record work in another thread, invoke `record` in that thread.
+Recursion and Ruby return events during exception or nonlocal unwinds balance
+the stack. An unmatched return, such as a fiber method entered before recording,
+has no inferred caller. Escaping exceptions still propagate and disable the
+recorder; a later recording starts with fresh stacks.
+
+`merge` adds `metadata[:trace]` with `call_count`, `callers`, and `return_types`.
+A return event during exception unwinding does not establish normal completion;
+`return_types` summarizes the return-event values Ruby exposes. Recording does
+not change existing method identifiers.
+
+## Method identity
+
+`method_kind` is `instance` or `singleton`. Merge matches the defining owner,
+method name, and kind: `Example#run` and `Example.run` keep separate call counts
+and return types. This identity survives JSON serialization; symbol keys and
+symbol kind values are accepted too. Caller entries retain `caller_method_kind`
+when present, so same-name instance and singleton callers remain distinct.
+
+Singleton methods use their defining named class or module. If `Child.run`
+inherits `Parent.run`, its events belong to `Parent.run`. Methods supplied by
+an extended module retain that module's instance-method identity. Anonymous
+owners and singleton methods on individual objects have no named unit;
+their `class_name` is `nil`, and merge skips them.
+
+## Older recordings
+
+Older Woods recordings derived `caller_class` from the callee receiver and
+`caller_method` from an interpreter-dependent backtrace offset. Their caller
+fields can disagree or describe the callee itself. Record again to obtain
+corrected caller evidence; merging old JSON cannot reconstruct missing callers.
+
+Recordings without `method_kind` match **instance methods only**. The older
+recorder used a normal owner name for instance methods and an opaque string
+such as `#<Class:Example>` for singleton methods. Those opaque singleton-owner
+records are skipped; record again to obtain a usable singleton identity.
+Handwritten fixtures for `Example.run` must specify `method_kind: 'singleton'`.
+Merge never guesses the kind from which units happen to be supplied. Unknown
+explicit kinds are skipped. Old caller entries without a kind remain untyped.

data/docs/SOURCE_FRESHNESS.md

@@ -0,0 +1,143 @@
+# Source freshness
+
+`woods_status.index.source_freshness` compares the served generation's captured
+application inputs with source bytes visible to the reader. It is separate from
+index age, HEAD equality, daemon liveness and external database/runtime state.
+
+This capability is unreleased after `2.0.0.beta2`. Check the installed gem's
+`woods-extract --help`, `rake -T woods:source_status`, and `woods_status` schema
+before using it; upgrading the plugin alone does not upgrade Woods.
+
+## Read the result
+
+| State | Meaning | Next step |
+|---|---|---|
+| `current` | All covered inputs match their consumer baselines, with a verified fresh boot boundary and complete checks. A dirty checkout can be current. | Use the indexed facts within the coverage below. |
+| `drifted` | At least one captured input differs, was removed, or a relevant input was added. | Inspect the changed paths; choose a full run or a justified targeted refresh. |
+| `unknown` | Evidence is incomplete: for example an old index, missing source/key, scan limit, opaque symlink directory, or unproved boot/consumer boundary. | Inspect `reasons`; use a deep check or fresh full capture as appropriate. |
+
+Drift can coexist with incomplete coverage. `reasons` reports both; absence of a
+listed change never proves an omitted input is unchanged. `complete` describes
+capture/traversal completion, while boot and consumer qualifications remain in
+`reasons`. `counts` contains total observed added/changed/removed paths; each
+`changes` list contains at most 30 paths and `truncated` marks longer lists.
+
+```json
+{"source_check":"deep"}
+```
+
+Pass this to `woods_status` for an explicit five-second content scan. The default
+`quick` scan has a 250ms budget. Both read and HMAC source bytes; there is no
+stat-only `current` shortcut, so same-size/same-mtime edits are detected. Limits
+also cap traversal at 50,000 visited files and 128 MiB read. Budgets are checked
+between filesystem operations; an operating-system read that itself stalls can
+outlast the scan deadline. A limit produces unknown coverage, never current.
+
+The result is tied to the **served** generation, including a reader holding an
+older generation during a concurrent publication. It is recomputed on each call;
+a source edit does not require an index generation change to become visible.
+
+## Establish a fresh baseline
+
+Run the launcher through the application's installed bundle:
+
+```bash
+bundle exec woods-extract full
+bundle exec woods-extract incremental app/services/checkout.rb app/views/orders/show.html.erb
+bundle exec woods-extract refresh routes controllers
+```
+
+`--root PATH` selects the application root. `--output PATH` selects the index
+(default `WOODS_OUTPUT`, otherwise `tmp/woods`); relative output paths resolve
+under the application root. Repeat `--source-root PATH` to include additional
+application-relative directories used by custom loaders. Use the same declaration
+on subsequent launcher runs. Paths are separate arguments, preserving spaces,
+commas and newlines. Refresh accepts known extractor names. Invalid arguments
+fail before extraction; the launcher propagates the child's failure or daemon
+stand-down exit 75. Split oversized incremental batches or choose full.
+
+The launcher captures source before a **fresh child** evaluates its Gemfile,
+Rakefile, Rails boot and eager loading. A private one-use handoff binds the capture
+to root, output, action, rules, nonce and parent process. It waits for the child
+and removes the handoff when the child exits. Source is checked again before
+publication; edits during boot/extraction retain the earlier identity and are
+reported, rather than being silently adopted as a current baseline.
+
+Existing Rake tasks, direct `Extractor` calls and the watch daemon remain usable.
+Their post-boot capture is marked `unverified_boot_boundary`; current bytes alone
+cannot prove what a previously booted Rails process consumed. A fresh launcher
+full run replaces that uncertainty. Hooks continue their existing refresh flow;
+enabling hooks does not implicitly restart or replace a daemon.
+
+## Scope and partial extraction
+
+Coverage follows the shared file/whole-app dispatch rules and reload policy:
+application and lib Ruby, known views/locales/tests/packages/schedules/schema and
+boot configuration. Source-only boot coverage also includes `Rakefile`,
+`config.ru`, root gemspecs and otherwise-unclassified Ruby helpers under `config/`.
+Normal generated/hidden directories are pruned before traversal, using the watch
+scanner's exclusions. Explicit source roots override generic exclusions; the
+index output is always excluded. Contained file symlinks are checked for stable
+resolution; directory symlinks and escaping/unreadable inputs leave uncertainty.
+
+Every consuming scope keeps its own identities. An events scan can reread a
+service file while its service unit remains untouched; refreshing events does
+not certify the retained service unit. Successful file/whole-extractor work
+updates only its scopes, including negative results and confirmed deletion.
+Unchanged scopes keep their earlier baseline. Boot inputs advance only on a full
+run. Partial runtime changes retain an explicit `runtime_consumption` uncertainty
+when Woods cannot prove every retained reflected fact was re-serialized. Named
+framework refreshes do not certify unrelated application inputs. A handled
+extractor error retains an explicit `extractor:<name>` uncertainty even if the
+extractor returns an empty result. Successful consumers keep their own evidence;
+a later full run without that failure can replace the uncertainty.
+
+Custom loader source outside captured roots, missing eager-load coverage and
+uncaptured application-owned unit paths remain unknown. This is application
+source evidence: it does not certify external database schemas/data, remote
+configuration, installed gem bytes, provider state or live runtime services.
+
+## Containers and hooks
+
+Run the launcher inside the application container when that is where the bundle
+and source exist, for example `docker compose exec -T app bundle exec woods-extract full`.
+An MCP reader must see the source and original private key; otherwise it reports
+unknown. `woods:source_status` has no Rails environment prerequisite and uses the
+same verifier without Rails initialization or provider work. Its optional
+Base64-encoded JSON transport supports `output`, `root` (an explicit reader-side
+source mapping), and `mode` (`quick` or `deep`).
+
+The opt-in SessionStart hook uses `WOODS_HOOK_RAKE` and `WOODS_OUTPUT`, including a
+Docker command prefix without requiring a host application bundle. It prints
+an actionable drift or unknown warning and stays quiet for current evidence.
+Its ten-second process deadline includes command startup; the scan uses quick
+mode. Missing older tasks and failed/timed-out commands report unknown. Cancelling
+Docker exec does not itself prove the process inside the container stopped.
+A quiet session hook does not acknowledge deferred PostToolUse queue entries.
+
+## Artifact and cost
+
+Each atomic payload contains versioned `source_inputs.json`. It records a compact
+identity table shared by per-consumer scope/path maps, root, rules fingerprint,
+generation, coverage and scan metrics. All file identities use HMAC-SHA256 with
+`<output>/.source-inputs.key`, a 32-byte, owner-only private file outside payloads.
+Low-entropy secret-bearing input files receive the same protection as ordinary
+source; no source bytes or raw secret hashes are added to this artifact. Do not
+publish the key with an index. Missing, insecure or mismatched keys produce
+unknown; Woods does not silently repair permissions or rotate keys. Independent
+outputs have different identities and must be compared using their own keys and
+consumer semantics, not raw manifest equality.
+
+Failed/no-op extraction does not advance the artifact's published generation.
+Flat fallback and older indexes lack verified atomic source evidence.
+`WOODS_PROFILE=1` reports `source capture` and `source verification` separately.
+Capture/recheck each allow up to ten seconds with the same file/byte caps.
+
+September 2026 fixture measurements: a pinned Writebook source tree (456 visited
+files, 223 hashed, 233KB) completed quick scans in median 36ms native / 45ms on a
+Linux container bind mount. Discourse (26,133 visited, 6,209 hashed, 56.4MB) needed
+about 1.6–2 seconds; quick scans returned unknown, while five-second scans completed
+traversal and still reported an opaque directory symlink. Native Ruby 4.0 and
+container Ruby 3.4 differed, so these are a budget envelope, not a filesystem
+speed comparison or a macOS virtiofs benchmark. Measure your own application;
+a large or slow source tree may need a full extraction rather than a longer read.