woods 2.0.0 → 2.1.0

data/docs/CONFIGURATION_REFERENCE.md

@@ -115,7 +115,7 @@ Columns:
 | `include_framework_sources` | Boolean | `true` | user-settable | Extract Rails and gem source code |
 | `concurrent_extraction` | Boolean | `false` | user-settable | Enable parallel extraction (experimental) |
 | `vector_store` / `metadata_store` / `graph_store` / `embedding_provider` | Symbol | n/a | preset-derived | Adapter types. Set by presets; override individually to mix stacks. |
-| chars-per-token ratio (used by ContextAssembler, TextPreparer, Builder, cost_model) | Float | `4.0` (OpenAI) / `1.5` (Ollama) | computed | Derived from the active embedding provider via `Woods::TokenUtils.chars_per_token_for(...)`. Not directly user-settable; change `embedding_provider` to change the ratio. |
+| chars-per-token ratio (used by ContextAssembler, TextPreparer, Builder, cost_model) | Float | `4.0` (OpenAI) / `1.5` (Ollama) | computed | A sizing estimate derived from the active provider via `Woods::TokenUtils.chars_per_token_for(...)`, not a proof of token count. Complete embedding admission uses the provider-specific bound described below. |
 
 ## Embedding options
 
@@ -136,6 +136,26 @@ config.embedding_options = {
 }
 ```
 
+`dimensions:` explicitly requests an output width; use it to reduce a
+`text-embedding-3-small` or `text-embedding-3-large` vector. The legacy singular
+`dimension:` remains an alias for this explicit request; if both are supplied,
+they must agree. `expected_dimensions:` records an expected output width
+without requesting a reduction. Standalone snapshot restoration uses that
+separate channel. Provider responses must match the expected width before
+Woods accepts them.
+
+`text-embedding-ada-002` has a fixed width of 1,536. A matching declared width
+is accepted without sending the API's unsupported `dimensions` parameter;
+other widths are refused before a request.
+
+Embedding snapshots store observed `dimension` separately from
+`requested_dimensions`. Standalone readers restore both. For legacy snapshots
+without the latter field, Woods restores a reduction only when the model is
+exactly `text-embedding-3-small` or `text-embedding-3-large` and the recorded
+width is below that model's native width. It does not infer support for
+reductions from arbitrary model names. Store and snapshot width checks still
+apply; changing models or widths requires rebuilding a compatible vector store.
+
 OpenAI embedding batches are sent in slices of at most 36 texts, preserving
 input order. For inputs within the API's 8,192-token per-text limit, this stays
 below both the 2,048-input limit and the 300,000-token total request limit.
@@ -160,14 +180,20 @@ config.embedding_options = {
 
 The provider reads `model:`, `host:`, and `num_ctx:` from `embedding_options`. `num_ctx` is auto-selected from a per-model registry (`nomic-embed-text` → 2048, `bge-m3` → 8192, `mxbai-embed-large` → 512, `snowflake-arctic-embed` → 512, `snowflake-arctic-embed2` → 8192, `all-minilm` → 512). Unknown models fall back to 2048, matching Ollama's conservative embedding default. Set `num_ctx:` explicitly only when running a model with a known-larger native context that isn't in the registry yet.
 
+An explicit `dimensions:` option (or its legacy `dimension:` alias) is sent to
+Ollama and requires server/model support. A stored width is restored through
+`expected_dimensions:` instead.
+Discovering a width through a probe never adds `dimensions` to later requests.
+
 **Why `num_ctx` is capped at the native context.** Ollama has an open regression ([ollama/ollama#14186](https://github.com/ollama/ollama/issues/14186)) where `options.num_ctx` does not lift the effective ceiling on `/api/embed` for models whose native context is smaller than the override. Woods advertises the native ceiling so the chunker sizes inputs to what Ollama will actually accept.
 
-**Optional exact tokenization.** Install the [`tokenizers`](https://github.com/ankane/tokenizers-ruby) gem alongside Woods to get BERT WordPiece token counting. Without it, Woods falls back to a chars/token ratio, which under-counts dense Ruby source (CamelCase constants, callback DSLs) and can silently over-pack chunks. Recommended for any Ollama setup.
-
-```ruby
-# Gemfile (optional)
-gem 'tokenizers', '~> 0.5'
-```
+**Input counting (included in Woods 2.1).** Complete inputs include metadata
+prefixes. Known OpenAI embedding models use a conservative byte upper bound;
+Ollama/custom model counts remain estimates. Woods requests `truncate: false`
+from Ollama and refuses/splits oversized inputs without dropping source.
+Installing `tokenizers` no longer implicitly downloads BERT or establishes an
+exact count for an unrelated model. A caller-supplied local tokenizer must match
+the selected model. See the model guide for limitations.
 
 See [EMBEDDING_MODELS.md](EMBEDDING_MODELS.md) for the full model comparison and the procedure for adding a new model to the registry.
 
@@ -188,6 +214,39 @@ Anything responding to `#embed` and `#embed_batch` can be assigned directly, it
 config.embedding_provider = MyCompany::CustomEmbedder.new(endpoint: internal_url)
 ```
 
+Included in Woods 2.1: snapshots capture the effective settings of injected
+Ollama, OpenAI, and Fake instances, including through Woods' retry and embedding
+cache wrappers. An injected instance takes precedence over unused
+`embedding_options`. Ollama retains its model, host, context size, read timeout,
+and vector settings; OpenAI and Fake retain their model and vector settings.
+Capturing constructor settings makes no requests. When a live `provider:` is
+passed to `ResolvedConfig.from_configuration`, its existing dimension-discovery
+behavior remains available. The explicit request width remains separate from
+the observed width.
+
+API keys are never stored; standalone OpenAI restoration still requires
+`OPENAI_API_KEY`. Only plain HTTP(S) origins can be persisted as Ollama hosts
+(an optional trailing `/` is allowed). URLs containing user information, a path
+prefix, query, or fragment are omitted and marked `requires_host_provider`.
+Those components may be legitimate routing settings, but can also carry
+credentials; Woods refuses to reconstruct a different endpoint by dropping them.
+The same restriction applies when reading older snapshot endpoints.
+
+**Reader compatibility:** snapshots marked `requires_host_provider` need a
+supporting reader. Older schema-1 readers cannot enforce that marker and may
+silently select defaults; do not use them for implicit restoration of these
+snapshots. Ordinary built-in snapshots with non-secret settings remain
+schema-1 compatible.
+
+Custom providers and custom wrappers are also marked `requires_host_provider`;
+Woods does not serialize arbitrary client state or infer a built-in from a custom
+class name. Embedding in the configured host continues to work. To serve such a
+snapshot semantically, provide an explicitly configured compatible provider in
+the reader's host initializer. Implicit standalone restoration reports a
+configuration error instead of choosing defaults. Explicit lexical retrieval
+does not restore or call an embedding provider. Record the loaded revision
+before relying on these snapshot improvements.
+
 ## Storage options
 
 | Option | Type | Default | Description |
@@ -264,6 +323,15 @@ worktree when overriding it. Existing databases at the old configured output
 path are not moved or deleted; run `woods:embed` for the selected index after
 upgrading to populate its default metadata database.
 
+**Included in Woods 2.1:** embedding with dump-backed vectors reconciles the
+complete SQLite identity inventory, including records without vectors and
+same-name units with distinct types. A full rebuild removes vanished records,
+including when the new corpus is empty. Incremental runs retain the existing
+empty-input and 30% purge guards. Both modes still refuse incomplete extraction
+input before changing stores. Standalone MCP opens the index's SQLite metadata
+before restoring type filters on hydrated vectors, using that same metadata
+store for retrieval. No database migration or embedding format change is required.
+
 Requires the `sqlite3` gem in your host bundle. Rails apps backed by
 MySQL or PostgreSQL won't have it by default, selecting `:sqlite`
 without it raises `Woods::ConfigurationError` with install
@@ -326,9 +394,33 @@ overrides the wrapper defaults for `:embeddings` (24 hours) and `:context`
 (15 minutes). `:memory` accepts `max_entries` (default 500); it ignores
 `default_ttl` because each wrapper write supplies its domain TTL.
 
+**Included in Woods 2.1:** retrieval contexts are scoped to one retriever
+instance, including when Redis or Solid Cache is shared by applications or
+worktrees. Restarting the retriever starts a fresh context namespace. Reload
+retires only that instance's namespace, including results still in flight;
+already running requests may finish against the previous corpus. Retired entries
+expire according to their configured TTL or backend eviction; disabling both
+can retain unused entries indefinitely. Embedding-vector caches are separate and survive context invalidation.
+
+Embedding entries are scoped to the provider family, endpoint, model, requested
+and declared widths, and relevant input options. Cache lookups never call a
+provider's probing `dimensions` method. Keys contain hashes rather than raw
+endpoint credentials or API keys. Malformed cached vectors and known-width
+mismatches are refused. This configuration key format starts a fresh embedding
+cache after upgrading; old entries expire under their existing TTLs.
+
+Custom provider objects may expose a pure, JSON-compatible `cache_identity`
+containing every setting that affects their vectors, plus a non-probing
+`configured_dimensions` reader for width validation. Otherwise each cache
+wrapper has its own namespace; cached vectors are still checked for valid
+numeric content and consistent batch widths. Configuration changes require a
+new provider and wrapper. A remotely replaced model under an unchanged name
+still requires an explicit cache/model-version change and re-embedding.
+
 `Woods::Cache.cache_key` length-prefixes every component, including a single
-component, so different argument counts cannot share a response. Existing
-multi-component keys used by Woods' wrappers remain unchanged. Custom callers
+component, so different argument counts cannot share a response. This component
+encoding is unchanged and independent of the retriever's context namespace;
+the embedding wrapper uses the new identity above. Custom callers
 using single-component keys must clear their affected persistent cache domain
 when upgrading, since older unprefixed entries can alias the new encoding;
 subsequent calls refill it normally. Namespace clearing still covers both formats.
@@ -496,22 +588,35 @@ there is no full population count to publish. Both caps must be a positive Integ
 
 | Option | Type | Default | Description |
 |--------|------|---------|-------------|
-| `session_tracer_enabled` | Boolean | `false` | Enable session tracing middleware |
+| `session_tracer_enabled` | Boolean | `false` | Enable session tracing middleware; set with its store in `config/application.rb` before Railtie initialization |
 | `session_tracer_allow_production` | Boolean | `false` | Explicitly allow session tracing in `Rails.env.production?`. Without this opt-in, the Railtie warns and leaves the tracer disabled even when `session_tracer_enabled` is true. Review trace contents, retention, and access controls before enabling it. |
 | `session_store` | Object | `nil` | Store backend: `FileStore`, `RedisStore`, or `SolidCacheStore` |
 | `session_id_proc` | Proc | `nil` | Custom proc to extract session ID from requests |
 | `session_exclude_paths` | Array<String> | `[]` | Path patterns to exclude from tracing |
 
+Configure the complete tracer setup in `config/application.rb`, after Woods is
+required and before Rails initialization. A `config/initializers/woods.rb`
+assignment is too late to mount this middleware; Woods warns and tracing stays
+disabled. Restart the server after changing it.
+
 ```ruby
-require 'woods/session_tracer/file_store' # the stores are not autoloaded
+# config/application.rb, after Bundler.require(*Rails.groups)
+require 'woods/session_tracer/file_store'
 
-config.session_tracer_enabled = true
-config.session_store = Woods::SessionTracer::FileStore.new(
-  base_dir: Rails.root.join('tmp/session_traces')
-)
-config.session_exclude_paths = ['/health', '/metrics', '/assets']
+Woods.configure do |config|
+  config.session_tracer_enabled = true
+  config.session_store = Woods::SessionTracer::FileStore.new(
+    base_dir: File.expand_path('../tmp/session_traces', __dir__)
+  )
+  config.session_exclude_paths = ['/health', '/metrics', '/assets']
+end
 ```
 
+The Rails middleware records traces. Reading them with `session_trace` additionally
+requires a custom Index Server process configured with the same store; the
+packaged Index executable does not load application initializers. See
+[conditional capabilities](MCP_SERVERS.md#conditional-index-capabilities).
+
 ### File session retention
 
 `FileStore` accepts `ttl:` in seconds (default `nil`, expiration disabled),
@@ -682,8 +787,8 @@ deployment guide including defense layers.
 | `console_mcp_enabled` | Boolean | `false` | Master switch. When `false`, stdio exits and the mounted Console middleware passes requests through to Rails. |
 | `console_mcp_http_enabled` | Boolean | `true` | HTTP transport switch; effective only while the master switch is on. Set `false` for stdio-only use without HTTP token validation or an active HTTP endpoint. Read at request time. |
 | `console_mcp_token` | String | `ENV['WOODS_CONSOLE_MCP_TOKEN']` or `nil` | Bearer token required on every enabled Console HTTP request. With both Console flags enabled, production boot raises on a missing token; other environments warn and requests fail closed with 401. A configured token shorter than 32 characters raises at boot while HTTP is enabled. Explicit stdio-only configurations skip HTTP token validation. Generate with `SecureRandom.hex(32)`. |
-| `console_mcp_allowed_origins` | Array\<String\> | `%w[http://localhost http://127.0.0.1 http://[::1]]` | `OriginGuard` allowlist. Port is stripped before comparison, so `http://localhost` matches any localhost port. Override for tunneled / internal-dashboard access. |
-| `console_mcp_path` | String | `/mcp/console` | URL path the Rack middleware responds on. |
+| `console_mcp_allowed_origins` | Array\<String\> | `%w[http://localhost http://127.0.0.1 http://[::1]]` | Shared guard/SDK allowlist (included in Woods 2.1). Exact cross-origin entries include the port; portless entries also permit same-authority traffic. Include the actual browser origin and non-loopback endpoint authority. Restart after changes. |
+| `console_mcp_path` | String | `/mcp/console` | URL path captured when Rack middleware mounts. Set a custom path in `config/application.rb` before Railtie initialization, then restart. |
 | `console_embedded_read_tools` | Boolean | `false` | Register `console_sql` and `console_query` in supported stdio and Rack modes. |
 | `console_blocked_tables` | Array\<String\> | `Woods::DEFAULT_CONSOLE_BLOCKED_TABLES` | TableGate denylist (case-insensitive). Bare names match every schema; qualified names (`schema.table`) match exactly. |
 | `console_redacted_columns` | Array\<String\> | `Woods::DEFAULT_CONSOLE_REDACTED_COLUMNS` | Column names whose values are replaced with `[REDACTED]` in responses, and which are refused as aggregate, scope, find, and order inputs. |
@@ -785,6 +890,11 @@ in its finalized development environment. See [startup and installation](WATCH_D
 for the generator's explicit modes, portable receipt, update/removal, and the
 separate supervision status. Raw task settings above remain compatible.
 
+**Included in Woods 2.1 (#591):** blank/whitespace-only idle timeouts count as
+unset consistently. `woods-watch --recover-claim INDEX --claim-token TOKEN`
+provides explicit recovery only for a matching abandoned managed lifetime lease;
+see [ownership recovery](WATCH_DAEMON.md#recovering-an-abandoned-managed-claim).
+
 ### Opt-in plugin refresh hooks
 
 These settings control the plugin shell worker. Check installed
@@ -813,6 +923,11 @@ tasks for manual refreshes; hook transport is not a general shell execution API.
 
 ### Extraction rake tasks
 
+The separate `woods-extract` launcher captures before Rails configuration runs:
+custom `output_dir` applications must supply matching `--output` or `WOODS_OUTPUT`.
+Its implicit default is `tmp/woods`; the Woods 2.1 #591 guard refuses a finalized
+configuration mismatch before publication. See [launcher output selection](SOURCE_FRESHNESS.md#establish-a-fresh-baseline).
+
 | Variable | Default | Purpose |
 |----------|---------|---------|
 | `WOODS_OUTPUT` | `Woods.configuration.output_dir` | Overrides the output directory for extraction/watch and embedding tasks without editing the initializer; embedding also places its default SQLite metadata database there. Explicit database options take precedence. |

data/docs/CONSOLE_MCP_SETUP.md

@@ -209,7 +209,18 @@ Rails/MCP host. If a browser-based client sends an `Origin` header from a
 different host, include that exact origin too. This allow-list controls both
 DNS-rebinding Host checks and browser CORS; keep Rails' own `config.hosts`, TLS,
 and proxy rules aligned with it. Server-to-server clients normally omit
-`Origin`, but their request `Host` must still be allowed.
+`Origin`, but a present request `Host` must still be allowed. Supporting security
+revisions deliberately allow configured non-loopback Hosts through the SDK too;
+2.0.0 could refuse them at that inner layer despite the Woods allowlist. This
+widening is limited to the configured authorities and retains bearer auth. The
+same immutable policy makes preflight acceptance agree with SDK dispatch. List exact browser ports for
+cross-port traffic, including loopback; a portless origin additionally permits
+same-authority traffic, not arbitrary cross-port access. Console defaults remain
+HTTP loopback, while the Index Server defaults include HTTP and HTTPS loopback.
+Default HTTP(S) ports are equivalent to their omitted form. An explicit list
+replaces default browser origins, including loopback origins; add loopback entries
+if those clients are needed. Invalid entries fail at boot naming the entry.
+Restart after changing the list. See [HTTP origin matching](MCP_HTTP_TRANSPORT.md#browser-origins-dns-rebinding-defense).
 
 Do not mount `Woods::Console::RackMiddleware` by itself. The Railtie composes
 `OriginGuard`, `BearerAuth`, and the Console middleware in the supported order.
@@ -329,6 +340,21 @@ user: deploy
 command: cd /app && bundle exec rake woods:console
 ```
 
+**Included in Woods 2.1:** direct mode prefers an existing executable `bin/rake`
+in the application's selected directory, falling back to `bundle exec rake`.
+Relative `directory` values resolve from the launcher's initial working directory.
+Explicit `command` settings win. Docker and SSH modes retain their remote default;
+set `command` explicitly when that application uses another task entry point.
+
+**Included in Woods 2.1:** the launcher rejects unknown keys, nested
+`connection:`/`console:` sections, blank or non-string values, and options that
+do not apply to the selected mode. Keep `mode` and `command` at the top level;
+use `directory` for direct mode, `container` for Docker, and `host`/`user` for SSH.
+Configure access controls and redaction in the Rails initializer, not this
+process-launch file. Move legacy nested launch settings to the matching example
+above. An empty file or mapping still selects the default direct command;
+supported `command`/`directory` overrides can omit `mode` for direct execution.
+
 Override config path with environment variable:
 
 ```bash
@@ -376,6 +402,13 @@ servers register only executable tools: the 9 Tier 1 tools by default, plus
 | `console_association_count` | Count associated records for a specific record |
 | `console_recent` | Recently created/updated records (max 50) |
 
+**Included in Woods 2.1:** `console_association_count` returns `0` or `1` for
+`belongs_to` and `has_one`, including missing targets. It counts the association's
+relation, applying `scope` to the target model; collection associations retain
+their ordinary count. Target-table and scope-column checks still apply before
+association reads. The tool does not call a custom association getter to count
+an already loaded model object.
+
 ### Tier 2: Domain-aware (9 tools): inventory only, not executable
 
 | Tool | Description |
@@ -420,9 +453,21 @@ servers register only executable tools: the 9 Tier 1 tools by default, plus
 
 ## Configuration options
 
-Set these in your Rails initializer:
+Set the request-time flags and protection settings below in your Rails initializer.
+A **custom endpoint path** must be set earlier, after Woods is required in
+`config/application.rb`, because the Railtie captures it while mounting middleware:
 
 ```ruby
+# config/application.rb, after Bundler.require(*Rails.groups)
+Woods.configure { |config| config.console_mcp_path = '/internal/woods-console' }
+```
+
+Restart after changing the path. Leaving the default `/mcp/console` requires no
+path assignment. Session tracing also needs early configuration; follow
+[session tracer options](CONFIGURATION_REFERENCE.md#session-tracer-options).
+
+```ruby
+# config/initializers/woods.rb
 Woods.configure do |config|
   # Master on/off switch for the Console MCP feature (Layer 0). Default: false.
   # Applies to every transport: stdio, launcher wrapper, and Rack.
@@ -432,9 +477,6 @@ Woods.configure do |config|
   # after configuring the layers below that match your threat model.
   config.console_mcp_enabled = true
 
-  # URL path for the Rack middleware endpoint. Default: '/mcp/console'.
-  config.console_mcp_path = '/mcp/console'
-
   # HTTP Origin + Host allow-list. Defaults to loopback only. Non-loopback
   # Rack deployments must include their public MCP host; browser clients from
   # another origin need that exact origin listed too.
@@ -596,6 +638,10 @@ Direct, unaliased selection of a redacted column stays allowed: the output heade
 
 `console_sql` applies a stricter form of the same rule because arbitrary SQL can rename output headers. A protected identifier is accepted only as a direct, unaliased outer `SELECT` column; aliases, aggregates, predicates, CTE shapes, ordering/grouping uses, and an EAV value without its paired key are refused before adapter execution. Use `console_query` when a protected column must participate in a more complex structured query.
 
+The security correction refuses PostgreSQL Unicode-escaped identifiers in
+`console_sql` before execution. Use ordinary identifiers or standard quoted
+identifiers instead. This restriction does not change the structured query tools.
+
 `console_query`'s `having` is covered by the same oracle rule: an aggregate over a protected column (`MAX(amount) > ?`), a bare predicate on a redacted column (`salary > ?`), or a predicate on an EAV value column is refused, since repeated guesses reveal the protected value from whether a row is returned. EAV key predicates stay allowed so callers can select the rows whose paired values need redaction.
 
 **Ships with a curated credential default list** (`Woods::DEFAULT_CONSOLE_REDACTED_COLUMNS`, 31 columns) covering Devise, Doorkeeper, Rodauth, has_secure_password, devise-two-factor, and common hand-rolled auth shapes: `password`, `password_digest`, `password_salt`, `encrypted_password`, `crypted_password`, `salt`, `otp_secret`, `encrypted_otp_secret`, `two_factor_secret`, `backup_codes`, `consumed_timestep`, `reset_password_token`, `confirmation_token`, `unlock_token`, `remember_token`, `invitation_token`, `access_token`, `refresh_token`, `auth_token`, `api_token`, `api_key`, `bearer_token`, `client_secret`, `webhook_secret`, `signing_secret`, `session_secret`, `private_key`, `encrypted_private_key`, `key_hash`, `token`, `secret`.
@@ -636,6 +682,9 @@ Redaction is defense-in-depth, prefer not storing plaintext secrets in database
 
 ### `console_redacted_key_values`
 
+See [read policy compatibility](#read-policy-compatibility) for SQL column-list
+restrictions, conservative typed masking, exact key spelling and binary columns.
+
 Column-name redaction falls short when credentials are stored in a **key-value (EAV)** table, e.g. a Stripe Connect `authorizations` row of `{key: "stripe_access_token", value: "sk_live_..."}`. The column holding the secret is called `value`, which is generic: adding `value` to `console_redacted_columns` would over-redact every unrelated row in the table.
 
 `console_redacted_key_values` takes one or more patterns that describe "when a row has `key_column` set to one of these names, redact its `value_column`":
@@ -700,7 +749,7 @@ these controls, in order:
 
 1. `SqlValidator` rejects DML/DDL (`INSERT`/`UPDATE`/`DELETE`/`MERGE`/`DROP`/`TRUNCATE`/`ALTER`/`CREATE`/`REPLACE`), row-lock clauses (`FOR UPDATE`, `FOR SHARE`, `LOCK IN SHARE MODE`), writable CTEs (every `AS (...)` body, not just the first), `UNION`/`INTO`/`COPY`, multi-statement and comment-hidden injections, and most administrative keywords (`DO`, `SET`, `LISTEN`, `NOTIFY`, `CALL`, `LOAD`, `VACUUM`, `PREPARE`, transaction control, `EXPLAIN ANALYZE`) at the string level. Enforces a read-only **function allowlist** (`ALLOWED_FUNCTIONS`), anything not on it is rejected by name, quoted forms (`"pg_terminate_backend"(…)`) included. Only `SELECT`, `WITH…SELECT`, and plain `EXPLAIN` pass.
 2. `TableGate` refuses any SQL, model, or join that touches a `console_blocked_tables` entry.
-3. `SafeContext` wraps every request in a rolled-back transaction with a short statement timeout. **It does NOT cover async side effects**: ActiveJob `perform_later`, ActionMailer `deliver_later`, direct HTTP egress, `Thread.new`-spawned work, `after_rollback` callbacks, and writes through a different shard all execute as live. Treat the Console MCP as an admin-trust boundary, not a sandbox.
+3. `SafeContext` wraps every request in a rolled-back transaction with an adapter-dependent statement timeout. **It does NOT cover async side effects**: ActiveJob `perform_later`, ActionMailer `deliver_later`, direct HTTP egress, `Thread.new`-spawned work, `after_rollback` callbacks, and writes through a different shard all execute as live. Treat the Console MCP as an admin-trust boundary, not a sandbox.
 4. `CredentialScanner` + column/EAV redaction scrub results.
 
 Keep the flag off when the host requires a narrower database capability.
@@ -718,7 +767,7 @@ supported transport (stdio, Docker/SSH launcher, and HTTP).
 | 1 | Blocked tables | `console_blocked_tables` | Tool dispatch, before executor | Reject any tool call that touches a named table (model, table, or sql arg) |
 | 2 | Credential scanner | `console_disabled_scanner_patterns` (`[:all]` to disable entirely) | After executor, before render | Content-shape redaction of credential-shaped strings anywhere in the response tree |
 | 3 | Column + EAV redaction | `console_redacted_columns`, `console_redacted_key_values` | After executor, before Layer 2 | Identity-based redaction by column name and by key/value row shape |
-| 4 | SqlValidator + SafeContext | built-in | Inside executor | SQL deny-list for `console_sql`; transaction-rollback for every request |
+| 4 | SqlValidator + SafeContext | built-in | Inside executor | SQL validation and function allowlist for `console_sql`; transaction rollback for every request |
 
 Layers 0–3 are configured via `Woods.configure`. Layer 4 is always on and has no knobs. Observability hooks, `console.table_gate.rejected` for Layer 1, `console.credential_scan.hits` for Layer 2, emit structured log lines via `Woods::Observability::StructuredLogger` so operators can audit enforcement without scraping MCP wire traffic.
 
@@ -754,13 +803,20 @@ boundary remain necessary.
 
 ### Statement timeout
 
-Each transaction sets a statement timeout before any query runs. The default is **5000ms** (5 seconds). Timeout enforcement is adapter-specific:
+SafeContext attempts a **5000ms** (5-second) timeout. Support depends on the
+adapter and server; an unsupported setting is skipped and logged when a Rails
+logger is available.
 
 | Adapter | Mechanism | Scope |
 |---------|-----------|-------|
-| PostgreSQL | `SET statement_timeout = '5000ms'` | All statement types |
-| MySQL | `SET max_execution_time = 5000` (session scope; the prior value is restored after the transaction) | SELECT only (MySQL limitation) |
-| Other | Best-effort (skipped gracefully) | n/a |
+| PostgreSQL | `SET LOCAL statement_timeout = '5000ms'` | Transaction-local; discarded on rollback |
+| MySQL | `SET max_execution_time = 5000` | SELECT only; previous session value restored in `ensure` |
+| MariaDB | `SET max_statement_time = 5.0` | Seconds; previous session value restored in `ensure` |
+| SQLite / unrecognized family | No supported per-statement timeout | Do not rely on a query time limit |
+
+Rollback remains active when a timeout setting is unsupported. Recognition of a
+MySQL-family adapter alone does not establish that its server supports the
+corresponding timeout variable.
 
 ### SQL validation (tier 4 `console_sql`)
 
@@ -771,7 +827,7 @@ Validation runs **once**, inside the executor, with the dialect of the live adap
 
 - **Allowed prefixes:** `SELECT`, `WITH...SELECT`, and plain `EXPLAIN`. `EXPLAIN ANALYZE` is rejected, it executes the query rather than just planning it (both the whitespace and `EXPLAIN (ANALYZE, …)` option-list spellings).
 - **Rejected prefixes:** `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `DROP`, `ALTER`, `TRUNCATE`, `CREATE`, `GRANT`, `REVOKE`
-- **Rejected anywhere in query:** `UNION`, `INTO`, `COPY`; row-lock clauses (`FOR UPDATE`, `FOR NO KEY UPDATE`, `FOR SHARE`, `FOR KEY SHARE`, `FOR UPDATE NOWAIT`/`SKIP LOCKED`, MySQL `LOCK IN SHARE MODE`) — these take live row locks even inside the rolled-back transaction. The lock check is adapter-aware: `console_sql` validates with the active adapter's dialect, including MySQL double-quoted strings/backtick identifiers and PostgreSQL quoted identifiers/E-strings. Unknown adapters conservatively scan all supported normalizations. Every view is scanned under both MySQL executable-comment (`/*!...*/`) semantics, so `#` comments and version-guarded comments cannot split a clause apart.
+- **Rejected anywhere in query:** `UNION`, `INTO`, `COPY`; row-lock clauses (`FOR UPDATE`, `FOR NO KEY UPDATE`, `FOR SHARE`, `FOR KEY SHARE`, `FOR UPDATE NOWAIT`/`SKIP LOCKED`, MySQL `LOCK IN SHARE MODE`) — these take live row locks even inside the rolled-back transaction. The lock check is adapter-aware: `console_sql` validates with the active adapter's dialect, including MySQL double-quoted strings/backtick identifiers and PostgreSQL quoted identifiers/E-strings. Direct validator calls without a dialect conservatively scan all supported normalizations; packaged `console_sql` refuses unrecognized adapter families. Every view is scanned under both MySQL executable-comment (`/*!...*/`) semantics, so `#` comments and version-guarded comments cannot split a clause apart.
 - **Function allowlist (the authoritative function control):** every function-call-shaped identifier must appear in `ALLOWED_FUNCTIONS`, a conservative set of pure read-only functions (aggregates, window functions, string/number/date/JSON readers) kept portable across MySQL, PostgreSQL, and SQLite. Anything else is rejected by name, quoted forms (`"pg_terminate_backend"(…)`) included. This is an allowlist because a denylist cannot enumerate every side-effecting function (`nextval`, `pg_advisory_lock`, `pg_terminate_backend`, …). A legacy `DANGEROUS_FUNCTIONS` denylist (`pg_sleep`, `lo_import`, `lo_export`, `pg_read_file`, `pg_write_file`, `load_file`, `sleep`, `benchmark`) still runs first as belt-and-suspenders.
 - **Rejected patterns:** multiple statements (semicolons), writable CTEs (every `AS (...)` body is checked, so a writable CTE in any WITH position is refused — `WITH a AS (SELECT 1), b AS (DELETE FROM users RETURNING *) SELECT * FROM b`), a CTE list attached to top-level DML (`WITH a AS (SELECT 1) DELETE FROM users RETURNING *`), comment-hidden injections
 
@@ -843,6 +899,23 @@ fail closed with `Woods::ConfigurationError`.
 
 The middleware lazy-initializes the MCP server on the first request, which includes `Rails.application.eager_load!`. This can take several seconds on large apps. Subsequent requests are fast. If you want to pre-warm, call a health check endpoint that touches the middleware path at app startup.
 
+**Included in Woods 2.1:** if that eager load raises `NameError` (including a
+Zeitwerk naming error), HTTP Console returns a stable `503` and records one
+`console.eager_load.failed` diagnostic. That worker remains unavailable until
+restart; it does not serve a partially populated model registry or repeatedly
+retry loading on incoming requests. The response and diagnostic omit the raw
+application exception. Reproduce it in the same application environment:
+
+```bash
+bundle exec rails runner 'Rails.application.eager_load!'
+```
+
+Fix the application error, then restart the Rails workers. Authentication and
+requests outside the Console mount retain their normal behavior. Other exception
+types retain Rails' usual failure handling. The existing stdio entry point warns
+on eager-load `NameError` and continues; its model list may be incomplete, so fix
+the same application error there too.
+
 ### Timeout errors on large models
 
 The default statement timeout is 5000ms (5 seconds). If you are hitting timeouts on models with millions of rows, use `scope` to narrow the query:
@@ -902,3 +975,59 @@ These corrections require `2.0.0.beta4` or a reviewed development revision that
 contains them. Confirm that a patched release is available before selecting it.
 On affected versions, disable Console where these policies are required; Index MCP
 can stay enabled because it reads the published code index separately.
+
+## Console corrections included in 2.1
+
+Woods 2.1 includes the reviewed Console corrections also released in the
+2.0.1 and 1.6.4 security patches. For Git/path installations, verify the loaded
+revision and gem path. Plugin updates alone do not update the Console server.
+See the published [security advisory](https://github.com/lost-in-the/woods/security/advisories/GHSA-wxxx-6hqc-qm8g).
+
+The patch tightens SQL policy for adapter-specific comment and quoting forms,
+whole-row and multi-source redaction, typed key-value records, and per-mount HTTP
+guards. Supported structured reads remain available. Raw `console_sql` requires
+a recognized PostgreSQL, MySQL-family, or SQLite adapter; compatible subclasses
+are recognized by ancestry. Unsupported raw SQL receives a clear refusal without
+disabling structured tools.
+
+Malformed HTTP origin configuration now refuses at boot with the offending entry.
+Fix the allowlist and restart; do not relax authentication or redaction to make a
+refused request succeed. For stdio-only use, keep HTTP disabled.
+
+### Read policy compatibility
+
+These rules describe Woods 2.1 and the 2.0.1 security patch. Differences from
+the 1.6.4 backport are called out below. Verify the installed version and, for
+Git/path installations, the revision and loaded gem path.
+
+- **Column alias lists:** when either `console_redacted_columns` or
+  `console_redacted_key_values` is nonempty, `console_sql` refuses relation
+  and CTE column alias lists before execution, including lists on base tables,
+  derived tables, parenthesized `VALUES` sources and table functions. This applies
+  even when the selected names are not protected and independently of function
+  validation. Ordinary relation
+  aliases, CTEs without column lists and allowed scalar functions remain subject
+  to the normal SQL policy. Use explicit, unaliased protected columns or a
+  structured Console tool.
+- **Conservative typed masking:** EAV type lookup matches the final source-table
+  name case-insensitively and includes every matching registered model, even
+  across schemas. Any matching type can cause masking. This deliberately may
+  mask extra values when table names differ only by case or share that final
+  name; qualifying the table does not narrow that type set.
+- **Exact sensitive values:** `sensitive_keys` compares the stored and cast key
+  values with exact case after string conversion. Configure their actual raw or
+  cast spelling. `CredentialIndex` also matches credential substrings with exact
+  case; it does not decode hexadecimal binary output such as PostgreSQL `bytea`.
+  Binary cells have no general text-scanning guarantee: non-UTF-8 data can fail
+  JSON normalization before scanning. Put binary secret columns in
+  `console_redacted_columns` so they are masked before serialization.
+- **Adapter boundary:** raw `console_sql` requires PostgreSQL, MySQL-family
+  (including Mysql2, MariaDB and Trilogy), or SQLite classification. Compatible
+  subclasses are recognized by ancestry. Unknown families receive a validation
+  refusal for raw SQL; structured tools remain available under their normal
+  gates. See [statement timeouts](#statement-timeout) for adapter limits.
+- **SQL functions and select entries:** 2.x enforces a read-only function
+  allowlist; 1.6.x retains a function denylist. For `console_query`, provide one
+  expression per `select` array entry. 2.x refuses comma-combined entries that
+  1.6.4 splits before validation. Execution and typed redaction use the same
+  validated projection on all patched lines.

data/docs/EMBEDDING_MODELS.md

@@ -37,8 +37,8 @@ Woods.configure do |config|
     model: 'bge-m3',
     host: 'http://localhost:11434'
   }
-  # Everything else, chunker sizing, token counting, num_ctx, is picked
-  # up automatically from the model name.
+  # Model context and initial chunk sizing follow the registry.
+  # Local counts are estimates; the server enforces its actual limit.
 end
 ```
 
@@ -64,21 +64,20 @@ override. For `nomic-embed-text` (native 2048) the server rejects inputs above
 that with a `400 "the input length exceeds the context length"` regardless of
 `num_ctx`.
 
-Woods works around this two ways:
-
-1. **Advertise the native ceiling, not the override.** `Provider::Ollama` keeps
-   a model → native-context registry (`MODEL_CONTEXT_LENGTHS`) so the chunker
-   sizes inputs to what Ollama will actually accept. `num_ctx` is still passed
-   through the request body in case the regression is fixed upstream, but
-   nothing relies on it.
-2. **Verify client-side with the real tokenizer.** When the
-   [`tokenizers`](https://github.com/ankane/tokenizers-ruby) gem is installed,
-   `Embedding::TokenCounter` loads the `bert-base-uncased` WordPiece tokenizer
-   (the one every BERT-family embedding model is built on) and the chunker
-   re-verifies every slice. WordPiece fragments CamelCase and `::` separators
-   differently than character-based estimation suggests, this verification is
-   what catches the 10–20 % gap between our estimate and Ollama's internal
-   count.
+**Included in Woods 2.1:** Woods checks complete inputs, including their
+metadata prefixes, and splits source without discarding characters. OpenAI's
+known byte-BPE embedding models use a conservative UTF-8 byte upper bound at
+8191 tokens; this may create more chunks than an exact tokenizer would.
+Ollama and custom models use explicitly labelled estimates. Ollama requests
+set `truncate: false`, so an actual overflow fails instead of losing source.
+A prefix that cannot fit or one unsplittable source character refuses before
+vector/checkpoint writes; inspect the reported unit, model and limit.
+
+Installing `tokenizers` alone no longer downloads or selects BERT for every
+model. Advanced callers can inject a local model-matched tokenizer into
+`Embedding::TokenCounter`; the caller must establish that it matches the server.
+No automatic tokenizer download occurs. These checks do not prove that estimated
+inputs fit an arbitrary tokenizer: the provider remains authoritative.
 
 ## Adding a new model to the registry
 
@@ -112,8 +111,6 @@ If you want Woods to auto-pick `num_ctx` for a model we don't ship support for:
 
 - Indexing a real-world Rails app where concern-inlined models or long service
   objects routinely exceed 2048 tokens.
-- You already pay the `tokenizers` gem install cost and want the tighter
-  coupling between client-side verification and server-side enforcement.
 - Multilingual content (Arctic Embed 2 is BGE M3 with a stronger non-English
   story).