gemchat 0.1.0

data/docs/plans/gemchat-local-bundle-index.md

@@ -0,0 +1,1900 @@
+# GemChat Local — Bundle-Scoped Gem Documentation Index
+
+**Repo:** `/Users/zoras/work/gemchat` (new `bundle gem gemchat` skeleton)
+**Gem name:** `gemchat` (RubyGems name confirmed available; overlap with gemchat.org is intentional)
+**Status:** `init`, `index`, `reindex`, `search`, `vsearch`, `query`, `embed`,
+`status`, `version`, and a working trust-gated, sha256-gated auto-index plugin.
+Next up: the lexical arm of the fusion — see §11.4, which is designed and
+eval-gated but not built.
+
+## 1. Summary
+
+A local, offline, agent-facing search index over the gems in *your*
+`Gemfile.lock`, modelled on [qmd](https://github.com/tobi/qmd). Ships as a
+Ruby gem with a CLI on `PATH`.
+
+It lives in a **separate repo** from the Rails app (`/Users/zoras/work/gemchat_app`)
+because it must:
+
+- install and start in milliseconds without booting Rails,
+- have no PostgreSQL/pgvector dependency,
+- version and release independently of the hosted service.
+
+The problem it actually solves is **bundle awareness**. A Rails `Gemfile.lock`
+is 500–1500 lines; making an agent read it to discover your stack burns context
+for something a deterministic script does better. Secondary wins: exact-lockfile
+version fidelity, privacy (your dependency graph stays local), and offline use.
+
+This is a **retrieval** tool. It returns chunks and citations. It does not
+generate prose answers.
+
+## 2. End-user experience
+
+The goal in one sentence: **an agent working in my repo can answer "how do I
+use *my* pinned version of this gem" without me pasting a 1500-line
+`Gemfile.lock` into its context, and without my dependency graph leaving my
+machine.**
+
+### Path A — local (default)
+
+```bash
+$ bundle add gemchat --group development   # or: gem install gemchat
+$ gemchat init
+Appends `plugin "gemchat"` to your Gemfile and writes .gemchat.yml. [y/N] y
+Run `bundle install` once to activate it.
+
+$ bundle install && gemchat index
+ri: 0/27 found — Bundler installs without documentation. Generating locally…
+[12/27] sidekiq 8.1.7  1,482 chunks  (18s)
+Indexed 27 gems · 4,312 chunks · 14MB · ri: 27/27 generated
+
+$ gemchat search "retry policy"              # instant, BM25, offline, no model
+$ gemchat query "configure sidekiq retries" # hybrid, exact 7.3.9
+→ [sidekiq 7.3.9] lib/sidekiq/options.rb:42
+```
+
+### Path B — hosted, zero local footprint
+
+For users who don't want ri generated, a 146MB model, or the disk at all:
+
+```bash
+$ gem install gemchat
+$ gemchat init --hosted        # records hosted as this project's default
+$ export GEMCHAT_API_KEY=…     # or a `credentials` file in the config home
+$ gemchat query "configure sidekiq retries"
+```
+
+`init` does not prompt for a key: it edits the Gemfile, and the key is read from
+the environment or from `credentials` under the config home. That is deliberate
+— a prompt here would put a secret on a tty in a command that otherwise only
+reads a Gemfile.
+
+Three cost tiers on the local path, in the order they arrive:
+
+| Moment | What works | Cost |
+|---|---|---|
+| after `gemchat index` | `search` — BM25, exact version, fully offline | seconds + one ri pass |
+| after `gemchat embed` | `vsearch`, and `query`'s vector arm | 146MB, once |
+| always | `query` hybrid with RRF | both arms, and it degrades to BM25 alone if the model is absent |
+
+Note the second row. `vsearch` and `query` do **not** download the model; only
+`gemchat embed` does. A search command that silently pulls 146MB is a different
+product from one that does not, and it would be a nasty surprise in CI. When the
+model is absent they say so and answer from BM25 — or, for `vsearch`, exit 1
+rather than answer a different question than the one asked.
+
+**The first useful result costs nothing** — no account, no API key, no model
+download.
+
+## 3. How we got here
+
+| Thread | Outcome |
+|---|---|
+| Chat markdown table styling | `b3e1d51` — unrelated, done |
+| rubydex native ext unusable on Alpine/musl | `3866330` lazy-load → `372e57e` compile from Rust source; `available? => true` in production |
+| Agent Skills spike (Hyperdrive interop) | `b9d1491` grounded `SKILL.md` generation, spec-validated |
+| **This plan** | Local bundle index, with a hosted tier for users who want neither |
+
+Two findings shape the architecture, and neither came from the rubydex work
+itself. Reading `SearchService` showed it is unportable (pgvector + PostgreSQL
+FTS + HNSW), which is *why* a separate local store exists. The hosted app's
+`AGENTS.md` records `RERANKING_ENABLED=false` as the production default, which
+is why no reranker is planned here. What the rubydex work did contribute is the
+lesson that a Ruby dev tool has to install cleanly on musl, which shaped the
+dependency set (§4.4) and the embedder choice (§4.3).
+
+The Agent Skills work is reused directly: §16 ships the same
+Hyperdrive-compatible `SKILL.md` format that `b9d1491` validated.
+
+## 4. Decisions
+
+| Decision | Choice | Rationale |
+|---|---|---|
+| Language / packaging | Ruby gem, `gemchat` | Reuses RDoc/ri/Ruby-aware parsing; no Bun/GGUF toolchain |
+| Embedding default | `rllama` + downloaded GGUF | Works on a bare machine after `gem install` |
+| ri content | ri is the corpus, always indexed | 98.5% of the hosted corpus; carries the API surface |
+| Prose content | READMEs and `guides/`, indexed under `source_type = "readme"` | ri answers "what does this take"; a README answers "how is this used", which is the question ri is worst at. 1,301 of 86,945 chunks, but that ratio counts chunks, not usefulness |
+| **ri acquisition** | **Generate it ourselves — bundler does not** | Measured: `bundle install` produces no ri (§4.1) |
+| ri pollution | Document `lib/`+`ext/` only, drop `--all` | Structural filter, not a heuristic |
+| Reranker | Excluded from v1 | Hosted app ships with it disabled; eval-gated follow-up |
+| Index scope | Direct deps by default, `--all` opt-in | ~14.2k chunks / ~108MB vs ~105k / ~308MB (§6) |
+| Prism | Unconditional `prism` gem dependency | Identical behaviour on Ruby 3.2 and 3.3+ |
+| **ActiveSupport** | **Not a dependency** | Measured: 13 extra gems (`concurrent-ruby`, `i18n`, `tzinfo`, `drb`, `connection_pool`, …) for a CLI that uses none of them. The single thing wanted — boolean env parsing — is `Gemchat::Env` (~15 lines) |
+| rubydex | **Not a dependency** | Cost is musl-only, not universal; and the join is worth less here because we index the ri and Prism halves independently |
+| Core/stdlib ri | Generate from local `rubylibdir` — **not** `rdoc-data` | See §8.4 |
+| **Hosted tier** | **In v1, explicit opt-in only** | Low-disk / no-ri / no-model users still get value |
+| Degradation | Diagnose and stop; never auto-switch | A local privacy tool must not silently phone home |
+| Staleness trigger | Bundler plugin on `add`/`install`/`update`, trust-gated | All three change the lockfile; sha256-gating handles them uniformly |
+| Setup verb | Separate `gemchat init` | `index` must stay non-interactive and never mutate the Gemfile (CI safety) |
+| Store drift | Refuse on `schema_version`, tolerate `gemchat_version` | A newer layout would return wrong answers silently; a point release must not strand an index |
+| Agent discoverability | Ship `skills/gemchat/SKILL.md` in the gem | An agent cannot use a CLI it does not know exists |
+| Install path | **Both** Gemfile (dev group) and global | Reproducibility for projects, lightness for one-offs |
+
+### 4.1 Measured correction: `bundle install` does not generate ri
+
+The original plan assumed "ri documentation is usually installed in the user's
+system unless `--no-document` is set". **That is true for `gem install` and
+false for `bundle install`.** Verified with isolated installs:
+
+| Method | ri generated? |
+|---|---|
+| `gem install zeitwerk` (no flags at all) | **yes** |
+| `gem install zeitwerk --document=ri` | yes |
+| `bundle install` via isolated `BUNDLE_PATH` | **no** |
+
+Corroborating evidence on a dev machine: 3 of 411 installed gems have ri, and
+all three (`bundler`, `rubygems`, `rubygems-update`) ship with Ruby itself. No
+`~/.gemrc` and no `.bundle/config` — this is not a user setting. Bundler skips
+documentation generation deliberately, because it is slow across hundreds of
+gems.
+
+**Consequence:** for bundler users — essentially the whole target audience —
+Tier 0 (§8.1) hits ~0%. ri generation is therefore the **main path, not a
+fallback**, and the resumable generation subsystem (§13) is a core phase rather
+than a conditional one. An earlier draft of this plan got this backwards.
+
+The user's original counterpoint still holds in narrower form: `gem rdoc --all
+--ri --no-rdoc` *does* generate ri for every installed gem, so tier 0 is worth
+keeping as a free fast path and the remedy is worth printing.
+
+### 4.2 `rdoc-data` must not be a dependency
+
+Looks like the obvious route to core/stdlib ri, but the gem is a dead end:
+
+- latest version `4.1.0`, published **2015-12-29**; nothing since
+- runtime dependency `rdoc ~> 4.0`, while current RDoc is `8.0.0` — adding it
+  would pin a decade-old RDoc onto a Ruby 3.2+/4.x install
+- payload covers Ruby 1.8–2.3 only, and its own description says that data
+  "are not needed for typical installs of C Ruby"
+- `require "rdoc/data"` fails on this machine's Ruby 4.0.7
+
+Instead: `RbConfig::CONFIG["rubylibdir"]` already contains **727 stdlib `.rb`
+files** locally, so core/stdlib ri is generated offline with the same RDoc
+recipe. No dependency, no download, no version-manager intervention. See §8.4.
+
+### 4.3 Embedding default flipped from Ollama to rllama
+
+The original reasoning was wrong: it optimised for *this* machine, where
+`bin/pull_models` already pulled `nomic-embed-text`. A user who just ran
+`gem install gemchat` has Ruby, their bundle, and nothing else. Defaulting to
+Ollama means a second install, a running daemon, and a port. Sizes are
+near-identical (`nomic-embed-text` 274MB, `embeddinggemma-300M-Q8_0` ~300MB) so
+the differentiator is one step versus two, not bytes.
+
+Superseded in one respect: the `rllama` default is now
+`nomic-embed-text-v1.5` rather than `embeddinggemma-300M-Q8_0`, because the
+latter is gated on HuggingFace. See §10.1. The one-step-versus-two argument
+above still holds.
+
+### 4.4 Why rubydex is not a dependency
+
+rubydex resolves one specific mismatch: RDoc reports *mixin-flattened* owners
+(`Sidekiq::Job.perform_async`, via `extend ClassMethods`) while Prism reports
+the *lexical* owner (`Sidekiq::Job::ClassMethods`). Measured in the Rails app
+on sidekiq 8.1.7: **+8 of 771 chunks, ~1.1 percentage points** (88.7% → 89.8%).
+
+The cost side is weaker than it first appears, and the value side is lower here
+than in the Rails app:
+
+- **Distribution cost is musl-specific, not universal.** rubydex publishes
+  precompiled builds for `x86_64-linux`, `aarch64-linux`, `x86_64-darwin`,
+  `arm64-darwin`, and `x64-mingw-ucrt`, so `gem install rubydex` is a plain
+  download on macOS, glibc Linux, and Windows x64 — no Rust, no build. The gap
+  is musl: no `-linux-musl` variant is published, so Alpine users would need a
+  Rust toolchain — and an unbuildable dependency fails the *entire*
+  `bundle install`, not just gemchat. Real severity, but for a minority.
+- **The value is lower here than in the Rails app.** We generate the ri store
+  ourselves *and* run Prism over the same source, so both artifacts are ours
+  and both are indexed. The Rails app joined them to enrich a third-party
+  corpus; here search can surface either half independently. The 1.1% is an
+  upper bound, not a floor.
+- **It would not fix the remaining ~10%**, which is mostly
+  `attr_reader`/`attr_accessor` (a `CallNode`, not a `DefNode`) and dynamic
+  definitions.
+- **The loss is nearly harmless.** Reporting
+  `Sidekiq::Job::ClassMethods#perform_async` rather than
+  `Sidekiq::Job.perform_async` is arguably *more* precise — it tells the agent
+  the method comes from a mixin.
+
+Prism alone is pure Ruby, dependency-free, and the sole source of file:line and
+signature metadata in this design.
+
+The decision rests on the value argument, not the cost one. An earlier draft of
+this section claimed rubydex would impose a Rust build on every end user; that
+was wrong, and generalised from the Rails app's own Alpine deploy onto a
+population that mostly does not resemble it.
+
+### 4.5 Four defects the test suite found
+
+Worth recording because all four passed a manual end-to-end check and a clean
+`standardrb` run. A smoke test proves the happy path works; it does not prove
+the *state* is correct, and these were all state bugs.
+
+1. **`Gemchat.root` was memoised.** It derives from `GEMCHAT_HOME`, and a cached
+   environment value means the first reader wins for the life of the process. Any
+   later `GEMCHAT_HOME` silently did nothing, and the test suite was writing into
+   the developer's real `~/.gemchat`. The join is cheap; do not cache it. This
+   class of bug is why the first suite run failed only in one order.
+2. **`Store#search` returned `{**row, score: …}`.** With `results_as_hash` set,
+   every row already carries integer keys, so the splat leaked `0, 1, 2…` into
+   results and made `score` a Symbol while every other column was a String.
+   Fixed by selecting an explicit column list and building the hash by hand.
+3. **Blank chunk fields were indexed as dead tokens.** `searchable_text` emitted
+   `method_name: ` for class-level chunks. Class chunks outnumber method chunks,
+   so this padded a large share of the corpus. Now dropped.
+4. **`index` reported work it had not done.** A cached ri store plus an unchanged
+   content digest printed `generated … 0 chunks`, which reads like success and
+   like corruption equally. `Store#replace_gem` now returns an `Outcome`
+   (`:indexed` / `:updated` / `:unchanged`) and the CLI prints `unchanged`
+   instead of a chunk count. A no-op and a run that lost its content are very
+   different events and must not look the same.
+
+The generalisable lesson: `:unchanged` is a *status*, and
+"wrote 0 rows" is not a substitute for one. Minitest 6 removing `Object#stub`
+and `minitest/mock` also forced a small local `with_stub` helper — check the
+loaded minitest version before assuming mocking exists.
+
+## 5. What already exists (reuse, don't rebuild)
+
+Read-only references in `gemchat_app`:
+
+| Asset | Location | Reuse as |
+|---|---|---|
+| `read_ri_from_path` | `app/services/gem_fetcher_service.rb:176-183` | Port the `RDoc::RI::Driver` reader (~50 lines, Rails-free) |
+| ri generation recipe | `app/services/gem_fetcher_service.rb:339-369` | Port verbatim, including both gotchas |
+| ri tuning notes | `docs/ri-doc-generation.md` | Source of the `--all` decision |
+| Prism symbol pass | `RubySourceDefinitionService` | Port for `class_name`/`method_name`/signature |
+| `find_ri_path` | `app/services/gem_fetcher_service.rb:259-266` | **Fix, don't copy** — see §8.1 |
+| Hybrid weights | 0.7 vector / 0.3 FTS, `RRF_K = 60.0` | Identical to qmd's RRF k |
+| Source-prior rebalancing | `RAG_SOURCE_PRIORS_ENABLED` | Reuse the idea for ri/readme balancing |
+| Core/stdlib generation | `app/jobs/ruby_core_stdlib_index_job.rb:92-120` | Reference for the tier-3 path |
+| Skill artifact format | `b9d1491` in this thread | Reuse the `SKILL.md` shape for §16 |
+| Plugin trust pattern | qmd `.qmd/index.yml` gating | Model for §15 + §17 |
+| **Hosted MCP tools** | `query_docs`, `answer_extractive`, `get_index_status`, `read_changelog` | The backend for §12 |
+
+**Not reusable:** `SearchService` (806 lines) — pgvector `<=>`,
+`websearch_to_tsquery`, generated `tsvector`, `pg_trgm`, HNSW
+`vector_cosine_ops`. That constraint is precisely *why* a local store exists.
+
+## 6. Research findings (measured, not assumed)
+
+Measured against the 124-gem local corpus:
+
+| Metric | Value |
+|---|---|
+| ri chunks | **85,644** |
+| readme chunks | 1,301 |
+| ri share | **98.5%** |
+| per-gem chunk counts | median 176 · p90 1,482 · max 12,663 |
+| 150-gem bundle (est.) | ~105,000 chunks · 308MB of vectors |
+| ~25 direct deps (**superseded** — see below) | ~4,400 chunks · ~~14MB~~ of vectors |
+
+**Conclusion:** ri is the corpus, not an optional extra. The hosted index
+already answered this empirically. The cost objection dissolves at direct-deps
+scope. The 308MB figure applies only to `--all`.
+
+Install-method measurement (§4.1):
+
+| Finding | Value |
+|---|---|
+| `gem install` generates ri | yes, no flags needed |
+| `bundle install` generates ri | **no** |
+| Local machine: gems with ri | 3 of 411 (all Ruby-shipped) |
+| `~/.gemrc` / `.bundle/config` | absent — not user-configured |
+
+ri generation cost, measured across the **49 resolvable direct dependencies of a
+real Rails app** (`gemchat_app`, 52 direct deps, 223 total specs), public API
+only, `lib/`+`ext/`:
+
+| Metric | Value |
+|---|---|
+| generation time | **9.9s total · 0.20s/gem mean · 0.09s/gem median** |
+| chunks produced | **14,229 · 290/gem mean · 101/gem median** |
+| ri store on disk | **66 MB** |
+| vectors @768d f32 | **42 MB** |
+| 4-core wall clock | **~2s** |
+| skipped (no `lib`/`ext`) | `rails` (meta-gem), `rubocop-rails-omakase`, `tzinfo-data` |
+| largest single gems | brakeman 1,597 · yard 1,473 · rouge 1,229 · simplecov 1,214 · capybara 1,019 chunks |
+
+This **resolves the plan's largest stated risk**. ri generation is seconds, not
+minutes, so the resumable design (§13) is good hygiene rather than a
+load-bearing necessity. Chunk count is ~3x the plan's earlier 4,400 estimate,
+so the real local footprint is ~108MB (ri + vectors) rather than ~14MB — still
+comfortable, and the 308MB figure remains the `--all`-on-everything case.
+
+`--all` cost, measured on a separate 9-gem sample: adds **3–46%** more chunks
+(roo +46%, pg_query +23%, selenium-webdriver +20%, prism +3%) and roughly
+proportional generation time. Dropping it (§9) is worth keeping, though the win
+is smaller than the noise argument alone suggests.
+
+RDoc and SQLite behaviour found empirically while building the spike:
+
+- **RDoc refuses to write into an output directory that already exists.** Do not
+  pre-create the target; create only its parent. This constrains §13's atomic
+  rename — RDoc creates the directory, we then move it into place.
+- **`RDoc::RI::Driver` takes `--doc-dir`**, not `--ri`/`--op`.
+- `Dir.chdir(gem_dir)` with relative source dirs is required (§8.3); absolute
+  `--op` is fine alongside it.
+- **`benchmark` is not a default gem in Ruby 4.0** — use
+  `Process.clock_gettime(Process::CLOCK_MONOTONIC)` for all timing.
+- Gems with no `lib/` and no `ext/` (meta-gems like `rails`) must be skipped
+  rather than passed to RDoc, which errors.
+
+Environment verification:
+
+- `sqlite3` 2.9.6 present, **FTS5 working**, precompiled natives for
+  `x86_64-linux-musl`, `x86_64-linux-gnu`, `x86_64-darwin` (confirm
+  `aarch64-linux` during the spike)
+- `sqlite-vec` **not** present, and not needed (§11.1)
+- `rllama` 1.2.0 exists; single runtime dependency `ffi >= 1.0`
+- `prism` 1.9.0 exists; no runtime dependencies
+- `rdoc-data` 4.1.0 is legacy (2015) and constrains `rdoc ~> 4.0` (§4.2)
+- `rubydex` 0.4.1 publishes `ruby`, `x86_64-linux`, `aarch64-linux`,
+  `x86_64-darwin`, `arm64-darwin`, `x64-mingw-ucrt` — **no `-linux-musl`
+  variant**, which is part of why it is excluded (§4.4). For contrast `sqlite3`
+  publishes the full matrix including `x86_64-linux-musl` and
+  `aarch64-linux-musl`, so Alpine coverage is a packaging choice some gems make
+  and rubydex has not.
+- stdlib doc dir **absent** on this machine (mise Ruby without ri data)
+- Bundler 4.0.21 supports the `plugin` DSL and defines `Bundler::Plugin`
+
+## 7. Architecture
+
+```
+~/.gemchat/index.sqlite     # global store: every gem ever indexed, keyed name+version
+~/.gemchat/models/          # downloaded GGUF embedding models
+~/.gemchat/ri/              # ri stores WE generate (the main path, §8.3)
+~/.gemchat/credentials      # hosted MCP API key, chmod 600
+~/.gemchat/trusted.json     # trust approvals for project-local config and plugin
+<project>/.gemchat.yml      # lockfile sha256 + scope (trust-gated, §17)
+```
+
+**Global store + per-project manifest** mirrors qmd's collection model and
+avoids re-embedding `rails` in every project. Config is XDG-aware
+(`$XDG_CONFIG_HOME` honoured, plus `GEMCHAT_CONFIG_DIR`).
+
+Two backends share one CLI: `local` (default, §8–§11) and `hosted`
+(gemchat.org, §12). The backend is a stored setting, never inferred.
+
+### 7.1 Store identity
+
+One `meta` table records everything that would make a store unreadable or
+silently wrong:
+
+| Key | Guards against |
+|---|---|
+| `schema_version` | store written by a newer/older gemchat |
+| `gemchat_version` | global install drifting from the bundled one (§14) |
+| `embedding_model` | mixing vectors from different models |
+| `embedding_dimensions` | a model swap with a different output width |
+| `backend` | local vs hosted confusion in output |
+
+On mismatch the store **refuses to open** and prints the exact remediation
+(`gemchat reindex`, or the specific field that changed). This generalises the
+"never mix embedding models" rule the hosted app already enforces, and is the
+reason dual install paths (§14) are safe.
+
+## 8. ri strategy — four tiers
+
+Because `bundle install` produces no ri (§4.1), tier 0 is a rare fast path and
+tier 2 is the main event. The ordering below reflects that honestly.
+
+### 8.1 Tier 0 — opportunistic read (rare fast path)
+
+Worth keeping: `gem install` users, anyone who has run `gem rdoc`, and gems
+that ship prebuilt docs all hit this instantly for free.
+
+Resolve the **exact locked version**. The hosted app's lookup must *not* be
+copied verbatim:
+
+```ruby
+# WRONG for this tool — gemchat_app:259-266 globs any version
+Dir.glob(File.join(gem_path, "doc", "#{name}-*", "ri"))
+
+# RIGHT — exact locked version
+Dir.glob(File.join(gem_path, "doc", "#{name}-#{version}", "ri"))
+```
+
+Search `Gem.path` **plus** `Bundler.bundle_path` (bundler's install path is
+frequently absent from `Gem.path`). Fast path first:
+`Gem::Specification.find_by_name(name, version)&.doc_dir`.
+
+### 8.2 Tier 1 — diagnose, and print the remedy
+
+Expected to fire for most bundler users, so it must read as normal rather than
+as breakage. State the cause plainly:
+
+> ri: 0/27 found — Bundler installs without documentation by default.
+
+Then the remedy, which genuinely works and is cheap to run once:
+
+```bash
+gem rdoc --all --ri --no-rdoc   # generate ri for every installed gem
+```
+
+Offer both routes — run that, or let `gemchat index` generate ri per-gem
+itself (§8.3), or switch to the hosted backend (§12). The user picks.
+
+### 8.3 Tier 2 — generate per-gem into a private cache (main path)
+
+```bash
+rdoc --all --ri --no-rdoc --op ~/.gemchat/ri/<name>-<version>
+```
+
+Generates into **our** cache. Never writes into the user's gem directories and
+never installs. This is the expected path for a fresh project, and its
+cost profile drives the resumable design in §13.
+
+Two gotchas ported verbatim from `gem_fetcher_service.rb`:
+
+1. **`Dir.chdir(source_dir)` with relative source dirs.** RDoc 8 computes
+   `full_names` relative to cwd, then chdirs into `--op` when saving. Absolute
+   paths from a different cwd yield escaping relative paths
+   (`./../../../../var/...`) and writes land *outside* the output directory.
+2. **`rescue SystemStackError` and delete the partial output.** RDoc recurses on
+   symlink cycles and deeply nested modules. `SystemStackError < Exception`, so
+   a bare `rescue => e` misses it, and a truncated ri store must never be read.
+
+Also applied here (from §9): document only `lib/` and `ext/`, and **omit
+`--all`** — the latter halves the noise and is the single most important lever
+on output size and speed.
+
+### 8.4 Tier 3 — core and stdlib
+
+Do **not** depend on the `rdoc-data` gem (§4.2). Instead:
+
+1. If the system store exists, read it
+   (`RbConfig::CONFIG["rubylibdir"]/../doc`).
+2. Otherwise generate from `RbConfig::CONFIG["rubylibdir"]` — 727 stdlib `.rb`
+   files ship with Ruby, so `rdoc --ri --op <cache> <rubylibdir>` is fully
+   offline. Cache under a synthetic `ruby-core-stdlib` key.
+3. Surface the version-manager remedy for users who prefer a prebuilt store
+   (rbenv `rdoc` package, mise, brew) rather than generating.
+
+## 9. Fixing ri pollution without excluding ri
+
+Two flags, both already established in `gemchat_app`:
+
+1. **Document only `lib/` and `ext/`.** Structurally excludes Rakefiles,
+   `spec/`, `.github/`, `bin/` — the "Sidekiq's own HERB linting config"
+   failure mode observed in the agent-skills spike.
+2. **Drop `--all`.** The tuning table in `docs/ri-doc-generation.md` names
+   `--all` (private/protected methods) as the noise source and states that
+   removing it limits output to the public API.
+
+At query time, source priors bias toward README/guides for *how-to* questions
+while preserving ri for *API signature* questions — the same rebalancing the
+hosted app already performs.
+
+## 10. Embedding
+
+| Provider | Model | User cost | Status |
+|---|---|---|---|
+| `rllama` | `nomic-embed-text-v1.5` Q8_0 (146MB) | auto-download | **default** |
+| `ollama` | `nomic-embed-text` (274MB) | must install and run Ollama | opt-in |
+| `openai` | `text-embedding-3-small` | key + network + per-query cost | opt-in |
+
+`rllama` ships prebuilt llama.cpp binaries in the gem; its only dependency is
+`ffi`. Users compile nothing.
+
+**`gemchat search` must work with zero models downloaded.** This is qmd's
+behaviour and the reason it feels instant: BM25 works immediately after
+`index`, and the model is only pulled when semantic recall is wanted — either
+via the explicit `gemchat embed`, or lazily on the first `vsearch`/`query`.
+First vector use downloads with progress, resume, and checksum verification.
+
+Model identity and dimensionality live in the store's `meta` table (§7.1).
+**Refuse to mix** and require an explicit re-embed.
+
+### 10.1 The default model changed: embeddinggemma is gated
+
+The table above originally named `embeddinggemma-300M-Q8_0`. That model is
+**`gated: manual` on HuggingFace**: fetching it needs an account, a licence
+click, and a personal token. An unattended `gemchat embed` cannot do any of
+that, so it was not a default, it was a wall. `nomic-embed-text-v1.5` is
+ungated, Apache-2.0, 768-dimensional (matching `gemchat_app`'s vectors, so the
+two agree on what good retrieval looks like), and 146MB rather than 300MB.
+
+The digest is pinned rather than described, so "checksum verification" means
+something specific:
+
+```
+nomic-ai/nomic-embed-text-v1.5-GGUF / nomic-embed-text-v1.5.Q8_0.gguf
+146,146,432 bytes
+sha256 3e24342164b3d94991ba9692fdc0dd08e3fd7362e0aacc396a9a5c54a544c3b7
+```
+
+`Models.installed?` compares **size, not digest**, because SHA-256 over 146MB on
+every `gemchat status` and every `gemchat search` is not affordable. The digest
+is enforced in `download` before the file is moved into place, so a wrong file
+cannot be installed by that path.
+
+### 10.2 Prefixes are part of the model identity
+
+Nomic models are trained with task prefixes. Measured on the rake corpus, the
+correct hit's similarity for the query *"stop the process when the OS sends a
+signal"* rose from **0.515 to 0.580** once `search_document: ` and
+`search_query: ` were applied to the two sides.
+
+The prefixes are declared per model in the registry rather than hardcoded, and
+`+prefixed` is part of the stored identity string — because changing the prefixes
+changes what the vectors *mean*, so an existing index has to be rebuilt rather
+than extended. Omitting them is not a crash, it is a silent quality loss, which
+is the harder failure to notice.
+
+### 10.3 The native embedder has to be supervised, and has a size cliff
+
+rllama 1.2.0's prebuilt llama.cpp **aborts the process** on this machine rather
+than returning an error. Measured, not assumed: the same command failed four
+times in a row and later passed 20 of 20 with no configuration change, and
+`GGML_METAL_DISABLE=1` made no reliable difference. A `SIGILL` cannot be
+rescued — it descends from nothing `StandardError` catches — so in-process there
+is no way to detect it, retry it, or degrade. Embedding therefore runs in a
+**supervised child process**, retried once with Metal disabled.
+
+There is also a size cliff, and its exact position could not be pinned:
+
+| Input | Result |
+|---|---|
+| 128 real chunks, one batch | pass |
+| 216 real chunks, ~90KB, one batch | crash |
+| 160 real chunks, 44,624 bytes | pass |
+| 400 short synthetic chunks, 23,200 bytes | pass |
+| one real chunk at 1634 bytes | pass |
+| one real chunk at 1665 bytes | **crash** |
+
+It is not a byte count — 2500 bytes of repeated `"word "` is fine. It is not
+ASCII-related — stripping to pure ASCII does not help. It is not cleanly a token
+count either. The cause was not isolated, so the defence is two-layered rather
+than tuned: a per-item ceiling of **1200 bytes** (margin under the observed
+cliff) and a per-batch budget of **16KB**, with any batch that still fails
+**halved and retried**, down to a single chunk.
+
+The cost is real and stated: a long prose chunk is represented by its opening,
+and its tail stays reachable through BM25 but not through `vsearch`. Prose is
+already split at the chunker, so this affects the largest sections only. On the
+236-chunk rake index the result is **236/236 embedded in ~6s**, from a clean
+auto-download, with nothing left pending.
+
+### 10.4 Partial success is a first-class outcome
+
+A batch that cannot be embedded is left **pending**, not skipped and not faked.
+`Supervised#embed` returns `[index, vector]` pairs rather than a bare vector
+list precisely so a failed chunk cannot be written with a neighbour's vector or
+dropped from the count and silently misalign every row after it. A run where
+*nothing* embeds raises rather than reporting a successful no-op.
+
+`embed --force` discards the vectors and re-embeds everything. This is not
+cosmetic: `unembedded_chunks` only returns rows where the vector `IS NULL`, so
+without the reset a model change cannot be reconciled by re-running `embed` at
+all, and the mismatch error would be telling the user to run a command that does
+nothing.
+
+## 11. Search (local backend)
+
+| Command | Engine | Requires models |
+|---|---|---|
+| `gemchat search` | FTS5 BM25 | no |
+| `gemchat vsearch` | vector cosine | yes |
+| `gemchat query` | hybrid: typed expansion → BM25 + vector → RRF k=60 | yes |
+
+### 11.1 No ANN index needed — but two claims here were wrong, and are corrected
+
+**The first version of this section claimed the full scan "ranks in
+milliseconds".** Measured on a real 22,141-chunk index (120 locally installed
+gems, 33 with ri, 21,253 ri chunks) it takes **915ms per vector query**, not
+milliseconds:
+
+| Query | Full scan | FTS-narrowed to 200 | to 500 |
+|---|---|---|---|
+| how do I parse a ruby file into an abstract syntax tree | 915.3ms | 9.2ms | 22.2ms |
+| what raises a cop offense for a long line | 913.1ms | 9.2ms | 22.3ms |
+
+915ms is the real number, and it is because a UDF that unpacks two 3072-byte
+blobs into Ruby arrays is called once per row: ~34M array elements allocated per
+query. The 236-chunk index every other measurement in this project used reports
+~5ms, which is exactly how the wrong claim survived.
+
+**The second claim was that two-stage retrieval "cannot be expressed in SQL".**
+That is false, and the correction is the useful part. A candidate CTE joined to
+`chunks` limits which rows the UDF is called for, so "the top N by FTS, then rank
+those" is one statement:
+
+```sql
+WITH candidates AS (
+  SELECT rowid FROM chunks_fts WHERE chunks_fts MATCH ?
+  ORDER BY bm25(chunks_fts) LIMIT 500
+)
+SELECT c.title, -dot(c.embedding, ?) AS rank
+FROM candidates JOIN chunks c ON c.id = candidates.rowid
+WHERE c.embedding IS NOT NULL
+ORDER BY rank LIMIT 10
+```
+
+40x faster, and **not free**. Measured against the full scan on eight questions:
+
+| Pool | recall@200 of the full-scan top-200 | top-1 identical |
+|---|---|---|
+| 200 | 33.5% | 7/8 |
+| 500 | 48.8% | **8/8** |
+| 1000 | 59.9% | 8/8 |
+
+So the answer the user sees is preserved; the tail is not. And there is a cliff
+the numbers do not show: the candidate generator is lexical, so a paraphrase
+sharing no vocabulary with its answer produces few or no candidates and returns
+nothing, where the full scan would have found it. The OR mode is a *terrible*
+ranker (§11.3) and may still be an adequate *recall filter*, because the
+question's rare terms still have to match. That is the trade worth making
+deliberately rather than by default.
+
+**Not implemented.** It is recorded here as a measured, specified optimisation
+with its cost stated, rather than slipped in on the strength of a 40x number. The
+`sqlite-vec` conclusion still stands — unnecessary at bundle scale — but for the
+right reason: the bottleneck is a Ruby-level UDF, not the absence of an ANN index.
+
+Two details the full scan cost anyway, both silent:
+
+- **sqlite3 2.x ignores the block's return value** and reads `FunctionProxy#result`
+  instead. Returning the float makes every rank `NULL`, which sorts as `0.0` and
+  returns an arbitrary slice of the corpus as if it were ranked output.
+- **A mis-sized query** makes `dot` return `NULL` for every row. `vector_search`
+  refuses up front rather than returning junk.
+
+Vectors are stored as packed float32 BLOBs, 3072 bytes for 768 dimensions.
+
+### 11.2 Ranking
+
+RRF with `k = 60`, matching both `SearchService`'s `RRF_K` and qmd. Typed query
+expansion copied from qmd — `lex` routes to BM25, `vec` and `hyde` route to
+vector search, routed exclusively rather than sent to both.
+
+**No reranker in v1.** Corroborated by the hosted app's
+`RERANKING_ENABLED=false` production default. Revisit only if eval shows top-1
+errors concentrated in ri-heavy queries.
+
+### 11.3 There is no usable keyword arm yet, and the fusion is built around that
+
+The fusion is implemented and correct, but it is only ever as good as the weaker
+arm, so the weaker arm had to be measured before the feature was called done.
+
+**`search` is an exact-phrase matcher wearing BM25's name.** `fts_query` quotes
+each whitespace-separated term — which is load-bearing, because an unquoted
+`Puma::Server` parses as an FTS5 column filter and fails with "no such column",
+a systematic collision with Ruby naming. But joining those quoted terms with a
+*space* is FTS5's AND-of-a-phrase, and that was a side effect of the escaping
+rather than a decision. Measured on a ~22k-chunk corpus (a synthetic lockfile
+built from locally installed gems; see §11.4 for the exact configuration):
+
+| Query | `:phrase` (shipped) | `:any` (OR'd) |
+|---|---|---|
+| any natural-language question | **0 rows** | 5 rows |
+| `Rake::Task#enhance` | 1 row | 1 row |
+
+So the obvious fix — OR the terms, which is what a keyword search means —
+measures **worse, not better**. On five known-answer queries, each with a
+definitely-existing answer in the corpus, the OR'd arm located the right chunk in
+**1 of 5**:
+
+| Query | Expected | OR'd top-1 |
+|---|---|---|
+| write a rubocop cop that checks line length | `RuboCop::Cop::Layout::LineLength` | `RuboCop::Cop::MinBodyLength` |
+| generate documentation from source comments | `RDoc::Task` | `Parser::Source::Comment#document?` |
+| run tests in parallel with processes | `Minitest::Parallel::Executor` | `Minitest::Parallel::Executor` ✓ |
+| parse ruby source into an abstract syntax tree | `Parser::CurrentRuby` | `prism › docs › Ruby API > API` |
+| configure the default rake task | `Rake::Task.define_task` | — |
+
+The mechanism is length normalisation: a rare query term inside a short ri
+signature scores enormously under BM25, so `MinBodyLength` beats a long prose
+chunk that actually discusses line length. Corpus size does not save it — the
+22k-chunk run above has perfectly good IDF statistics.
+
+**Therefore `:any` is not the default, and this matters more for fusion than for
+search alone.** RRF trusts rank position, so a confidently wrong arm is not
+ignored — it is *promoted*. Feeding OR'd BM25 to the fusion changed the top-1 on
+4 of 5 rake questions and made it worse on most, e.g. `syntax highlighting in
+the console` went from `Rakefile Format > Comments` to `Why rake? · 4/4`.
+
+What ships instead: `:phrase` everywhere, so the lexical arm contributes only
+when it has an exact hit — which *should* rank first anyway. Measured effect on
+the rake index:
+
+- Six natural-language questions: fused output **agrees with the vector arm on
+  all 6** (3/3 shared every time), because the phrase arm correctly has nothing
+  to contribute. No regression.
+- `Rake::FileList`: the vector arm's top-1 was `Rake::FileList::[]`, a class
+  method. The phrase arm found `#==`, `#resolve`, `#*` as literal matches, and
+  fusion promoted the doubly-matched `#==` and `#*` above it.
+
+That is the honest characterisation: **fusion earns its keep on lexical queries,
+where the two engines disagree, and is a harmless no-op on questions.** Every
+result carries `matched`, so the caller can see which case they are in.
+
+A real keyword arm needs term selection before it goes near `:any` — drop the
+common words, and require at least one genuinely rare term so that a lone
+incidental match cannot outrank prose. That is the next piece of work, and it
+should be eval-gated rather than assumed. §11.4 is that work, written down.
+
+### 11.4 A keyword arm: what already exists, what is missing, what to do
+
+**Status: designed, not built.** Everything below was verified against the
+SQLite that ships in this environment (3.53.2 via sqlite3 2.9.6) rather than
+assumed from the FTS5 documentation.
+
+The question that prompted this section was whether `websearch_to_tsquery` could
+be borrowed from `gemchat_app`, which uses
+`websearch_to_tsquery('english', …)` against a `tsvector` column generated from
+`searchable_text`. It cannot be borrowed — it is PostgreSQL, and gemchat is
+SQLite — but the underlying idea is right, and separating its three parts
+changes what the work actually is:
+
+| `websearch_to_tsquery('english', …)` gives you | gemchat status |
+|---|---|
+| stemming / root-word matching | **already present** |
+| stopword removal | **absent** |
+| injection-safe parsing | **already present** |
+
+#### Stemming is already on
+
+`chunks_fts` is declared `tokenize='porter unicode61'`, and FTS5 applies a
+table's tokenizer to **both** sides of a match. Verified:
+
+| Query | Document text | Result |
+|---|---|---|
+| `run` | `running a stream…` | matches — stemmed |
+| `streams` | `stream` | matches — stemmed |
+| `parse` | `parser` | **no match** — over-stemmed |
+| `the`, `of`, `in`, `a` | present | matches — **indexed, not stopwords** |
+
+So "switch to stem/root word matching" is already done. The single real gap is
+the last row: FTS5 has no `stopwords=` option, so `the`/`of`/`how` sit in the
+index and match. On an OR'd query those are exactly the terms that hit nearly
+every chunk, which is the noise measured in §11.3 as
+`RuboCop::Cop::MinBodyLength` outranking prose about line length.
+
+Two fidelity caveats that no amount of work here removes:
+
+- **FTS5's `porter` is Porter-1980; PostgreSQL's `english` is Snowball.** They
+  are different algorithms. Local and hosted retrieval will never stem
+  identically, and this is not fixable by adopting a list.
+- **Porter over-stems Ruby-shaped words.** `parser` reduces to `pars` while
+  `parse` does not, so the two never match. Ruby docs are full of `-er`/`-or`
+  nouns, so this is a real recall cost and must be re-flagged in the docs rather
+  than left to look solved.
+
+The rest of the `websearch_to_tsquery` feature set is either already covered or
+free, and is deliberately **out of scope** for this slice:
+
+- *Injection safety* — `fts_query` quotes every term, so `:` and FTS5 operators
+  cannot escape. Already have it; the systematic `Puma::Server` collision is
+  handled.
+- *Phrase / `OR` / `-exclusion` / prefix* — FTS5 supports all of these natively
+  (verified: `OR`, `NOT`, `NEAR`, implicit AND, `parse*`, column filters). They
+  are not *exposed*: `fts_query` quotes each whitespace token, so a user typing
+  `foo OR bar` gets the literal phrase `"foo OR bar"`. Exposing them is a
+  hand-rolled parser with its own escaping edge cases — deferred, see the
+  out-of-scope table in §11.4.
+- *Trigram* — available and working (bare-term substring: `FileLis` → 1 hit).
+  This is the lever for identifier near-misses, and `gemchat_app` already scores
+  on it. It needs a second FTS table plus sync in `replace_gem` — deferred, see
+  the out-of-scope table in §11.4.
+
+#### Stage 1 — query-side stopwords in `:any` mode only
+
+**Do not reuse `gemchat_app`'s stopword list.** At
+`app/services/agentic/gem_similarity_service.rb:132` it is tuned for SEO title
+work and drops `gem`, `ruby`, `library`, `tool`, `management`, `system` — all of
+which are signal in a documentation index. Use Snowball's list instead, vendored
+from PostgreSQL's `src/backend/snowball/stopwords/english.stop`: **127 words**,
+counted, and containing no domain vocabulary.
+
+Change:
+
+- New `lib/gemchat/stopwords.rb` exposing `Gemchat::Stopwords::ENGLISH` as a
+  frozen `Set`. The comment records the source and the rejection above, so the
+  list is not quietly swapped later.
+- `Store.fts_query(raw, mode:)` — in `:any` mode only, drop a term when **every**
+  alphanumeric token of it is a stopword. That rule rather than a plain
+  `include?` check matters: `unicode61` splits on non-alphanumerics, so `how-to`
+  and `don't` are multi-token terms whose tokens are all stopwords, and dropping
+  them is what actually removes the noise. If *every* term is dropped, fall back
+  to the unfiltered set — a stopword-only query must not silently return nothing,
+  because the caller asked a real question.
+
+`:phrase` is **not** touched. Dropping terms from a literal phrase changes what
+it matches, so `search` and `lex:` keep exact behaviour and the flagship command
+carries zero regression risk. `search` still works with no model and no stopword
+table, which matters for a CLI that must run on a bare `bundle install`.
+
+#### Tests
+
+New `test/stopwords_test.rb`:
+
+- `ENGLISH.size == 127` — pins a truncated or mis-pasted list, which is the most
+  likely way this rots silently.
+- Every entry is lowercase and alphanumeric.
+- **The `gemchat_app` guard:** assert `rake`, `task`, `file`, `gem`, `method`,
+  `class` are *absent*. This is the test that stops someone "fixing" the list by
+  copying the app's.
+- `fts_query(mode: :any)` drops `how do I` from a question and keeps `list`,
+  `tasks`.
+- Multi-token handling: `how-to` and `don't` dropped, `task-manager` kept.
+- An all-stopwords query falls back instead of returning `nil`.
+- `fts_query` `:phrase` output is unchanged for a fixed input — an explicit
+  regression guard on `search`.
+
+Plus one end-to-end check in `test/store_test.rb` that stopwords no longer match
+via `search(query, mode: :any)`.
+
+#### Eval gate
+
+Build the corpus and run the five known-answer queries from §11.3 against
+`fts_query` in both modes. Configuration, so the number is reproducible:
+
+- A synthetic `Gemfile.lock` built from **33 locally installed gems**, ~21s to
+  index, **22,141 chunks**.
+- **`gemchat` is excluded, deliberately.** It is `path:`-sourced, so its own
+  `docs/plans/*.md` gets indexed as prose — and those documents quote the eval
+  queries verbatim. The first run included it and `:phrase` "scored" 0/5 by
+  matching the plan that documents the failing queries. A corpus containing the
+  project evaluating itself is not a baseline. Excluding it left both numbers
+  unchanged, which is worth knowing but should not have needed checking.
+
+| Arm | Known-answer result |
+|---|---|
+| `:phrase` | **0/5** |
+| `:any` (current) | **1/5** |
+| `:any` + stopwords | **1/5** |
+| `:all` — AND of the content terms (probe, never shipped) | **0/5** |
+
+**The gate was not met. `:any` remains off by default, and `:phrase` stays.**
+
+Stopwords changed nothing, which is worth more than a marginal number. It
+rules out the explanation §11.3 offered — that the OR'd arm was drowning in
+common words — and leaves the real one. Look at what actually wins:
+
+| Query | Content terms | `:any` top-1 |
+|---|---|---|
+| write a rubocop cop that checks line length | `write rubocop cop checks line length` | `RuboCop::Cop::MinBodyLength` |
+| generate documentation from source comments | `generate documentation source comments` | `Parser::Source::Comment#document?` |
+| run tests in parallel with processes | `run tests parallel processes` | `Minitest::Parallel::Executor` ✓ |
+
+`MinBodyLength` and `Comment#document?` are short ri signatures containing one
+query term. They beat the prose chunk that actually answers the question,
+because BM25's length normalisation rewards exactly that shape. Removing `the`
+and `of` never touched the terms doing the damage — `length`, `document`,
+`task` were already the discriminating ones.
+
+And the obvious correction makes it worse, not better. Requiring **all** the
+content terms (`:all`, FTS5's implicit AND) drops to **0/5** — it
+over-constrains, and `"parse" "ruby" "source" "abstract" "syntax" "tree"` matches
+nothing in a 22k-chunk corpus at all.
+
+#### What this rules out, and what it means
+
+Four combinations of FTS5's boolean semantics were measured, and the spread is
+0/5, 1/5, 1/5, 0/5. **Query-side term selection is exhausted.** Stopwords,
+AND, OR and phrase are the whole vocabulary available, and none of them fixes a
+paraphrase question, because the problem is not which terms are selected — it
+is that the question and the answer share no vocabulary. The chunk that answers
+*generate documentation from source comments* is RDoc's README, which never
+uses the phrase "from source comments".
+
+So Stage 2 as originally written — a rare-term requirement, whether via
+`fts5vocab` IDF or a second intersecting `MATCH` — is **falsified**, not merely
+deferred. It was going to attack term *selection* inside a lexical arm, and
+term selection is not the failure. A paraphrase query needs a different
+mechanism entirely: document or query expansion, a learned lexical expansion
+model, or a reranker over a candidate set. That is well beyond a stopword list
+and is not planned here.
+
+The honest conclusion, which matches what §11.3 already measured from the other
+direction:
+
+- The **lexical arm is for identifiers**, where the query terms appear
+  literally. `Rake::FileList` and friends, where fusion genuinely earns its keep.
+- The **vector arm is for paraphrase**. It answers all five of these questions
+  correctly, unaided.
+- A hybrid that fuses both is correct for a query that is *partly* one and
+  partly the other, and the fusion is currently a harmless no-op when the query
+  is purely a question. That is not a defect to be engineered away; it is the
+  correct behaviour, and it is now measured rather than assumed.
+
+`Stopwords` stays in the codebase. It is ~15 tested lines behind an explicit
+`mode:`, it changed no shipped behaviour, and `:any` is still the mode a future
+expansion or reranking experiment would reach for. It simply is not the fix, and
+the tests now say so by pinning the count and the domain-vocabulary guard rather
+than by implying a quality win it does not deliver.
+
+To make the gate enforceable rather than a claim in a commit body, add
+`bin/eval-fusion` alongside `bin/sandbox`: it builds the corpus, runs the query
+set through all three arms, and prints the table.
+
+#### Explicitly out of scope
+
+| Stage | Why not now |
+|---|---|
+| ~~Rare-term requirement~~ | **Falsified.** Measured above: AND of the content terms scores 0/5, worse than OR's 1/5. Term *selection* is not the failure, so per-term IDF and a second intersecting `MATCH` were aimed at the wrong thing. |
+| Document-side stopword removal | Superseded. Stopwords were not the cause, so removing them from the index would not have helped either. |
+| Document or query expansion | The mechanism that *would* address a paraphrase question — the vocabulary mismatch is the actual blocker. Beyond a stopword list and not planned here. |
+| Expose `OR` / `NOT` / `NEAR` / prefix | A parser with its own escaping edge cases, and the eval above says more boolean surface is not the lever. |
+| Trigram table | Second FTS table plus sync in `replace_gem`. The one deferred item that targets identifiers, which *is* where the lexical arm is strong. |
+
+### 11.5 Source locations from Prism
+
+`Gemchat::Symbols` is the sole source of `file:line`, and the only part of this
+design that ri cannot supply. ri reports a class and a method; it does not report
+which file the method is defined in, or on which line. That is the one thing a
+result cannot be used without and cannot be inferred from, so it is printed after
+every ri hit:
+
+```
+ 1. [rake           13.4.2    ri     ] Rake::FileList#exclude
+    exclude(*patterns, &block)  (lib/rake/file_list.rb:150)
+```
+
+Ported from the app's `RubySourceDefinitionService`, minus Rails — no
+`Rails.logger`, no `blank?`/`presence`, and the namespace stack is plain array
+manipulation. The visitor logic is unchanged, including its known imprecision: a
+method defined inside `extend SomeModule` is reported under its *lexical* owner
+rather than the mixin. §4.4 argues that is arguably more precise, since it tells
+the reader the method arrives via a mixin.
+
+Measured on rake 13.4.2: **146 of 179 ri chunks (82%) resolve to a file and
+line.** The remainder are C extensions, `define_method`, and macros that
+generate methods, none of which are recoverable from parsing alone. An unmatched
+chunk keeps a nil location rather than borrowing a neighbouring chunk's path,
+because an invented citation is worse than an absent one.
+
+Three details that cost time:
+
+- **Paths are relative to the gem root**, so the indexer passes `gem_dir` and
+  paths come out `lib/rake/file_list.rb`. Passing `lib/` yields `file_list.rb` —
+  correct for the argument given, and a different contract.
+- **The lookup key normalises a missing owner to `""`.** A top-level `def` has no
+  owner, so its `class_name` is `nil`, and an ri chunk for it also has `nil`.
+  Both sides `.to_s`, which is what lets the two meet; without it every top-level
+  method silently misses.
+- **The content digest now covers the location.** A definition that moves files is
+  a changed chunk, and reusing the old row would leave a citation pointing at the
+  wrong line.
+
+`schema_version` moves 1 → 2, so existing stores need `gemchat reindex` — the
+`Store::IdentityError` path already names that command.
+
+## 12. Hosted tier (gemchat.org)
+
+For users who don't want ri generated, a 146MB model, or the disk at all. In v1,
+explicit opt-in only. **Built**; see §12.6 for the two corrections this section
+needed.
+
+### 12.1 Activation is always explicit
+
+| Mechanism | Scope |
+|---|---|
+| `gemchat init --hosted` | stores the default for the project (`.gemchat.yml`) |
+| `gemchat query --hosted "..."` | one invocation |
+| `GEMCHAT_HOST=gemchat.org` | environment override |
+| `GEMCHAT_API_KEY` | environment, for CI |
+| `$GEMCHAT_CONFIG_HOME/credentials` | `GEMCHAT_API_KEY=…`, chmod 600 |
+
+The key is read from the **config home**, not the data home, for the same reason
+trust and settings are: wiping the index must not discard a key.
+
+**Never automatic.** If the local index is empty or unusable, gemchat
+**diagnoses and stops** (§12.4). A local-first privacy tool must not silently
+send queries to a server.
+
+### 12.2 Command mapping
+
+| Local | Transport | Notes |
+|---|---|---|
+| `query` | `POST /api/v1/query` | hybrid retrieval, cited, ~$0.000001, no LLM |
+| `search`, `vsearch` | — | **local-only**, no hosted equivalent |
+| `status` | — | names the configured backend and, if unset, the settings URL |
+| — | `POST /api/mcp` (`query_docs`) | the agent-facing surface, unchanged |
+
+`lex:` is BM25 and therefore local-only; `query --hosted lex: …` refuses and
+names `gemchat search` rather than quietly returning something else.
+
+The mapping is imperfect and the docs must say so: `search` and `vsearch` simply
+do not exist on the hosted path.
+
+### 12.3 Honest limits
+
+State these in the README, in `gemchat status`, and in the shipped skill:
+
+- **Version drift — but less than this section originally claimed.** The hosted
+  API accepts `gem_versions`, so the local CLI sends this lockfile's exact pins
+  and the answer is about the version actually installed. The response discloses
+  any pin it could not apply (`scoped to 8.0.2; no per-version index for rake`),
+  so drift is visible rather than silent. A gem with no per-version index still
+  falls back to that gem's full docs, which is the residual limit.
+- **The dependency graph leaves the machine.** The whole point of Path A is that
+  it does not.
+- **Requires a key and a network.** Not zero-config.
+- **Different ranking.** Postgres hybrid + source priors vs local RRF. Results
+  will not be identical between backends, and that is expected.
+
+### 12.4 Diagnose and stop
+
+`gemchat status` shows the full tier ladder so the user always knows where they
+stand:
+
+```
+gemchat:   0.1.0
+store:     ~/.gemchat/index.sqlite
+index:     27 gems · 4,312 chunks
+vectors:   not downloaded (146MB) → gemchat embed
+hosted:    not configured → gemchat.org/settings
+```
+
+The `ri:` line from an earlier draft is gone. It suggested
+`gem rdoc --all --ri --no-rdoc` as the remedy for a missing ri store, but §6
+measured that `--all` adds 3–46% more chunks of private machinery, and
+§8.3 established that `gemchat index` generates ri itself — so the remedy is
+simply to index. Showing a command that inflates the index by up to half in
+order to avoid an index is not a remedy.
+
+The `backend:` and `MCP_API_KEY` phrasing was also dropped: the tier ladder
+should name the action (`gemchat embed`, the settings URL), not the variable
+that gates it.
+
+When a query cannot be served, gemchat names the tier, states the remedy, and
+stops. It never escalates on the user's behalf.
+
+### 12.5 Product and coupling notes
+
+This gives the `gemchat` gem a reason to exist for people who never index
+anything: a first-party, authenticated CLI for the hosted service. That is a
+real product surface, not just a fallback.
+
+The cost is coupling — the gem becomes a client of gemchat.org, so hosted
+breaking changes reach the gem. Treat gemchat.org as a **versioned API**: pin
+the tool contract, and make the client tolerant of unknown parameters (qmd's own
+MCP server silently ignores unknown params, which is the right precedent).
+
+### 12.6 Built, and two things this section got wrong
+
+Shipped: `Hosted` (the client), `query --hosted`, `init --hosted`, and the
+`status` tier line. `GEMCHAT_HOST` accepts a bare host or a full URL and always
+lands on the versioned path, because §12.1 names a host.
+
+**The transport is JSON, not MCP.** This section originally mapped local commands
+onto hosted *MCP* tools. The transport turned out not to be the problem — it is a
+plain JSON-RPC POST, no SSE, no session, so a client is about thirty lines. The
+problem is the response: `query_docs` returns **Markdown prose**, which is right
+for a model to read and wrong for a CLI, because a cosmetic server-side edit
+breaks every client with no error on either side. So the hosted tier speaks
+`POST /api/v1/query`, which returns rows.
+
+MCP is untouched and remains the agent surface. Both transports call one
+`HostedQuery` service in `gemchat_app`, so the ranking is implemented once rather
+than twice and drifting. The rendered MCP Markdown is byte-identical to before,
+which is why its existing tests pass unchanged.
+
+**Auto-indexing on a miss is off for the API, on for MCP.** That is a policy
+difference, not a bug: MCP's caller is an agent that is told what happened and
+can poll `get_index_status` and retry, whereas a CLI that silently enqueued
+indexing would spend the caller's quota on every miss with nothing in the
+response to show for it. The API requires `auto_index=true`.
+
+Two bugs worth recording, both invisible in review because the code reads
+correctly:
+
+- **`lex:` was not refused.** `cmd_query` splits the type off the query and then
+  called `split_type` again on the bare question, which of course could not see
+  it, so the guard never fired and `lex:` silently got a hosted answer. The type
+  is now threaded through. This is exactly the failure the skill warns agents
+  about, shipped in the thing that tells agents to avoid it.
+- **The credentials path printed empty.** The instruction used the *parsed* key
+  path rather than the path itself, so with no file present it said "write a key
+  to " and stopped.
+
+## 13. Indexing and resumability
+
+**Scope:** direct deps by default; `--all` for transitive. ri + README +
+guides all on.
+
+**Delta gates**, cheapest first:
+
+1. `Gemfile.lock` sha256 unchanged → no-op
+2. per-gem `version` + `source_hash` match a completed row → skip
+3. per-chunk `content_digest` → re-embed only changed chunks
+
+**Resumable ri generation** — because §8.3 is the main path, this is a core
+subsystem, not a contingency. Foreground process, state persisted in SQLite (not
+a daemon; a daemon buys little and costs a lot of complexity):
+
+- `ri_jobs(name, version, state, attempts, error, chunk_count, source_hash)`
+- states `pending → generating → done | failed | skipped`
+- reclaim stale `generating` rows on startup, recovering Ctrl-C and crashes
+- generate into a temp directory, then **atomically rename** into place before
+  recording `done`, so a half-written store is never visible
+- SIGINT returns the in-flight gem to `pending` and exits cleanly
+- **fork a small RDoc worker pool (`nproc`), not threads** — RDoc is CPU-bound.
+  Never fork across an `rllama`/llama.cpp boundary: load it only in the parent
+  process. FFI state plus `fork` is a segfault waiting to happen.
+
+## 14. Install paths
+
+Both supported, with different tradeoffs.
+
+| | A. Gemfile (development group) | B. Global |
+|---|---|---|
+| Command | `bundle add gemchat --group development` | `gem install gemchat` |
+| Invocation | `bundle exec gemchat` | `gemchat` |
+| Version | pinned per project, reproducible | floats with `gem update` |
+| Store format | locked to the project's gemchat version | may drift |
+| Best for | teams, CI, reproducibility | solo work, one-off queries |
+
+- **Never** in the `:default`/runtime group. Development tool; must not ship to
+  production.
+- When both are present, `bundle exec` disambiguates and the plugin (§15)
+  prefers the bundled one.
+- Drift is caught by `gemchat_version` in the store identity table (§7.1).
+- The shipped skill (§16) must say which invocation to prefer.
+
+## 15. Bundler plugin (staleness trigger)
+
+Adding a gem to a Gemfile is inert. Unlike qmd's notes, **the bundle changes on
+every `bundle add` and `bundle update`**, so the index goes stale constantly.
+
+### 15.1 `gem` and `plugin` are different directives
+
+Both lines are required, and they are not interchangeable:
+
+```ruby
+# Gemfile
+gem "gemchat", group: :development   # → DEPENDENCIES. Provides `bundle exec gemchat`.
+plugin "gemchat"                     # → PLUGINS. Provides the bundle-install hook.
+```
+
+| | `gem "gemchat"` | `plugin "gemchat"` |
+|---|---|---|
+| Lockfile section | `DEPENDENCIES` | **no lockfile section at all** (§15.6) |
+| Purpose | bundle dependency, version-pinned | hooks Bundler's own lifecycle |
+| Gives you | the CLI on the load path | auto-index on `bundle install` |
+| Required extra | — | the gem must ship a top-level **`plugins.rb`** |
+
+They are parsed in separate passes: during the plugins-only pre-parse,
+`Bundler::Plugin::DSL#gem` is a no-op. So `plugin` does not imply `gem`.
+`plugins.rb` sits at the top level of the gem and registers a hook:
+
+```ruby
+require_relative "lib/gemchat"
+Bundler::Plugin.add_hook("after-install-all") { |_deps| Gemchat::Bundler.auto_index }
+```
+
+**"Top level of the gem", emphatically — not of the project.** Measured, and the
+distinction is easy to get backwards because the file name is identical:
+
+- A `plugins.rb` in the **project root** is **never executed**. Verified by
+  putting a `$stderr.puts` at the top of one and running `bundle install`: zero
+  output, while the installed plugin's own `plugins.rb` ran and printed.
+- Bundler only ever resolves the file inside the installed gem.
+  `Plugin#load_plugin` reads `path = index.plugin_path(name)` and then
+  `load path.join(PLUGIN_FILE_NAME)`; `validate_plugin!` asserts the file
+  exists at `Pathname.new(spec.full_gem_path)`. No code path consults the app
+  directory.
+- So a root `plugins.rb` is silently inert — the worst failure mode available,
+  because the hook appears installed and simply never fires. Anyone following
+  the usual "drop a plugins.rb in your app" advice will ship exactly that.
+
+The event name must be a **string**. `Events::GEM_AFTER_INSTALL_ALL` is
+literally that same String — `define :GEM_AFTER_INSTALL_ALL, "after-install-all"` —
+so passing the constant also works. Only the underscored symbol
+`:after_install_all` is rejected, and it is rejected as `ArgumentError`, which
+`register_plugin` rescues and rewraps as `MalformattedPlugin`.
+`after-install-all` is the right event: the `*-require` hooks fire on every
+`bundle exec`, which is far too often.
+
+### 15.1a Four ways a plugin looks installed and does nothing
+
+All four measured against `gemchat`'s own `plugins.rb`. Every one of them
+produces a *successful* `bundle install`, which is what makes them dangerous —
+there is no error to notice.
+
+1. **A `plugins.rb` in the project root.** Never executed (§15.1). Bundler only
+   loads `<installed gem>/plugins.rb`.
+2. **A `plugins.rb` that only runs, and never calls `add_hook`.** Executed
+   exactly once — during the plugin phase of the install that just installed it —
+   and never again. Bundler records *registered hooks* into
+   `.bundle/plugin/index`, and `hook_plugins` gates every later invocation on
+   that record. Observed directly: the index showed `hooks:` empty while the
+   plugin had already printed a line during `bundle install`, so it looked alive.
+   `bundle add` and `bundle update` were then complete silent no-ops.
+3. **A bare `Bundler::Plugin` reference.** Bundler loads the file with
+   `load(path, true)`, wrapping it in an anonymous module, while
+   `register_plugin` is mid-flight inside `module Bundler::Plugin`. A bare
+   constant can resolve against the wrapper and raise
+   `NameError: uninitialized constant …::Bundler::Plugin` — measured, and it
+   aborted the install. Every Bundler reference is root-scoped `::Bundler`.
+4. **An `RAILS_ENV == "development"` guard.** Copied from Hyperdrive, where the
+   host is a Rails engine. `gemchat` is not a Rails tool, so in any plain Ruby
+   project `RAILS_ENV` is unset, the guard is permanently false, and the hook is
+   permanently silent. Corrected to a `CI`-unset check only.
+
+The common thread: **the failure mode of a plugin is silence, not an error.**
+A shim that appears installed and does nothing is strictly worse than one that
+refuses to install, because the user has no signal to act on. `test/plugin_shim_test.rb`
+asserts each of these mechanically, since all four are invisible in review.
+
+### 15.2 Setup is a separate `gemchat init`
+
+Following the Hyperdrive pattern. Its `hyperdrive:install` generator ends with
+`append_to_file "Gemfile", "plugin \"bundler-rails-hyperdrive\"\n"`, after first
+checking for an existing line so the write is idempotent.
+
+- `gemchat init` — the once-per-project setup verb. Appends
+  `plugin "gemchat"` with an interactive confirm, writes `.gemchat.yml`,
+  records trust, prints the `gem rdoc` remedy if ri is missing (§8.2), and can
+  set the hosted backend (§12). **The only command that may modify the
+  Gemfile.**
+- `gemchat index` — syncs the index. Non-interactive, never writes to the
+  project.
+
+Separate on purpose: a command named `index` that mutates your Gemfile is a
+surprise, and in CI or a pre-commit hook it would either block on a prompt —
+hanging the build — or silently edit a file it has no business touching.
+
+**Activation is not immediate.** Bundler only activates a newly declared plugin
+on the next `bundle install` (Hyperdrive documents the same chicken-and-egg). So
+`init` must tell the user to run `bundle install` once to arm the hook.
+
+### 15.3 Fires on `add`, `install`, and `update` — when Bundler installs
+
+The qualifier is the whole correction. `after-install-all` fires when Bundler
+performs an install, so `bundle update --all` with nothing to change installs
+nothing and the hook correctly stays silent. A test that asserted the hook "fires
+on update" while the update was a no-op was asserting the wrong thing; it now adds
+a gem to the Gemfile first, which is the ordinary reason anyone runs `update`.
+
+Earlier this claim was measured against the *published* `bundler-rails-hyperdrive`
+plugin, watching for that plugin's own output, gated behind an env var so it
+skipped by default. It never exercised gemchat's hook and it failed when actually
+run — leaving `bundle add` untested while the plan claimed all three. It now runs
+gemchat's own path-sourced plugin offline via `test/support/sandbox.rb`, and
+passes.
+
+Measured end-to-end against the published `bundler-rails-hyperdrive` plugin, one
+throwaway project, observing the hook's own output:
+
+| Command | `after-install-all` fired | Index may need |
+|---|---|---|
+| `bundle install` | **yes** | new/changed gems |
+| `bundle add <gem>` | **yes** | the new gem immediately |
+| `bundle update` | **yes** | often many changed versions |
+
+All three mutate `Gemfile.lock`, so the sha256 gate handles them uniformly and
+there is no need to detect which command ran. The plugin hooks the post-install
+point and asks one question — did the lockfile digest change?
+
+`bundle add` is the case that matters: that is exactly when a developer has just
+adopted a gem. It is now confirmed rather than assumed.
+
+### 15.4 The plugin is a stateless shim, not the indexer
+
+Measured: dual-declaring one gem installs **two physical copies** — one under
+the bundle's `gems/`, one under the project's `.bundle/plugin/gems/` — and
+Bundler 4.0.21 writes **no `PLUGINS` section** into `Gemfile.lock`, so nothing
+ties the plugin copy to the dependency version. Drift is therefore possible.
+
+Rails Hyperdrive already solves this, and its solution is the reason to copy its
+shape rather than invent one. `rails-hyperdrive` (the engine) does **not** ship
+`plugins.rb`; a separate tiny gem, `bundler-rails-hyperdrive` is the plugin.
+Its two files, read from the installed copy:
+
+```ruby
+# plugins.rb — the only thing Bundler loads, and all of it is registration
+require_relative "lib/bundler/hyperdrive"
+
+Bundler::Plugin.add_hook("after-install-all") do |_dependencies|
+  Bundler::Hyperdrive.auto_install
+end
+```
+
+```ruby
+# lib/bundler/hyperdrive.rb — the logic, never executed from the plugin copy
+require_relative "hyperdrive/version"
+
+module Bundler
+  module Hyperdrive
+    HOST_GEM = "rails-hyperdrive".freeze
+    SUPPORTED_RAILS_HYPERDRIVE = Gem::Requirement.new(">= 0.2")
+    PREFIX = "[hyperdrive] ".freeze
+    DEVELOPMENT = "development".freeze
+
+    module_function
+
+    # Runs inside `bundle install`: every failure must degrade to a printed
+    # line, never a raised error that fails the install.
+    def auto_install
+      return unless development_environment?
+      # ... locates HOST_GEM in the bundle and requires THAT copy's lib
+    rescue StandardError => e
+      report "auto-install skipped (#{e.class}: #{e.message})"
+    end
+  end
+end
+```
+
+The split matters and is the actual lesson. `plugins.rb` holds *only* the
+`add_hook` registration; the logic lives under `lib/` and is reached through the
+host gem found in the bundle. That is what makes the plugin copy stateless —
+there is no second implementation of anything, so there is nothing to drift.
+
+`gemchat/plugins.rb` follows this shape, with one deliberate deviation recorded
+in §15.1a: the `RAILS_ENV` guard is dropped, because `DEVELOPMENT` is a Rails
+concept and gemchat is not a Rails tool.
+
+Three transferable properties:
+
+1. **The shim carries no logic.** It locates the host gem *in the bundle* and
+   requires **that** copy. The plugin's own copy of the code never runs, so
+   version drift between the two copies is harmless. `gemchat` should do the
+   same: `plugins.rb` finds the bundled `gemchat` spec via
+   `Bundler.load.specs` and requires its `lib`. The only contract that must
+   stay stable across versions is *locate and require*.
+2. **A declared supported range.** `SUPPORTED_RAILS_HYPERDRIVE = ">= 0.2"` lets
+   an old shim politely skip a new engine instead of misbehaving. `gemchat`
+   should declare the same for its own version pair.
+3. **A cheap environment pre-filter.** Its `development_environment?` returns
+   early unless the env is development *and* `CI` is unset, with the comment
+   that the hook fires on every install — CI and deploys included — where output
+   would be noise. Adopt that as a first check, before any trust lookup: it
+   costs nothing and removes pointless work.
+
+### 15.5 Contract
+
+1. **Never fails the bundle.** Any error is swallowed; exit status unaffected.
+   This is not optional-by-choice. Measured on `gemchat`'s own `plugins.rb`, all
+   three of these fail the install:
+
+   | `plugins.rb` | Result |
+   |---|---|
+   | raises | `MalformattedPlugin`, **exit 29**, no lockfile written |
+   | calls `add_hook(:after_install_all)` | `MalformattedPlugin` (from `ArgumentError`), exit 29 |
+   | missing entirely | `MalformattedPlugin: plugins.rb was not found`, exit 29 |
+
+   "Aborts" is imprecise in one respect worth recording: the gems themselves do
+   get installed and `bundle list` shows them. What fails is the run — non-zero
+   exit, no `Gemfile.lock` — so the project is left without a usable bundle.
+   Note also that the *gems* land before the plugin phase, which is why a broken
+   plugin leaves a half-configured project rather than an untouched one.
+2. **Non-blocking.** Runs on `after-install-all`, never during install.
+3. **Silent by default.** One line at most, prefixed, and only when work was
+   done. Hyperdrive uses the literal prefix `[hyperdrive] `.
+4. **sha256-gated.** Compares the lockfile digest against `.gemchat.yml` and
+   no-ops when unchanged. Covers `add`, `install`, and `update` uniformly
+   (§15.3), so no command detection is needed.
+5. **Opt-out.** `bundle config set --local no_auto-index true` (Bundler also
+   honours a global `plugins false`, which disables the hook entirely), or
+   `gemchat init --no-auto-index`.
+
+**The plugin must also be trust-gated.** A Bundler plugin is *checked-in code
+that executes on `bundle install`* — `git clone && bundle install` on a
+repository you do not own would run it. Hyperdrive does not address this; qmd's
+trust model does, and it is the stronger precedent. Same hazard class as the
+checked-in `.gemchat.yml` in §17.
+
+The gate is low-friction for a project's own author: **the first explicit
+gemchat command run inside a project grants trust for that project**, recorded
+in `~/.gemchat/trusted.json`. Someone who just added `plugin "gemchat"` to
+their own Gemfile gets auto-indexing with no ceremony. Someone who *clones* that
+repository gets a no-op that prints one line, because they have not asked.
+
+The `CI` pre-filter (§15.4) runs first, so unattended contexts exit before any
+trust lookup. `gemchat trust` pre-approves explicitly — the only way to enable
+auto-index in CI on purpose.
+
+### 15.6 Plugin mechanics — measured, not assumed
+
+| Question | Answer | How |
+|---|---|---|
+| Dual `gem` + `plugin` of one gem: one copy or two? | **two** | `.bundle/plugin/gems/` + bundle `gems/`, published gem |
+| Does a `#` inside a fenced code block split a section? | it does, unless handled | `Markdown` tracks fence state |
+| Does Bundler 4 pin the plugin copy? | **no** | no `PLUGINS` section in `Gemfile.lock` |
+| Is `Bundler::Plugin` a class or module? | **module** | `module Bundler; module Plugin` |
+| `add_hook` signature | `(event, &block)` | event is a String key of `Events` |
+| Malformed `plugins.rb` | exit 29, no lockfile | raises / bad event name / missing file |
+| Project-root `plugins.rb` | **never executed** | `load_plugin` reads `index.plugin_path(name)` only |
+| Fires on `add`/`install`/`update`? | **all three**, but only when Bundler *installs* | gemchat's own path-sourced plugin, `Sandbox` |
+| Registration without `add_hook` | fires **once**, then never | `.bundle/plugin/index` records `hooks:` |
+
+### 15.6a Testing the plugin against a local checkout
+
+Before release, the shim has to be exercised for real. The mechanism is a
+Bundler local override, and the exact form matters:
+
+```sh
+bundle config set --local local.gemchat /path/to/gemchat   # does NOT work yet
+```
+
+`local.<gem>` is a *remote-source override*: Bundler still resolves the gem from
+rubygems.org first and only substitutes the local path for a version it finds
+there. For an unpublished gem it fails with "Could not find gem 'gemchat'".
+Pinning a version does not help.
+
+Two forms that also do not do what they look like:
+
+- `bundle config --local set path …` is the **deprecated** `config` form. It
+  writes `BUNDLE_SET: "path …"` and silently has no effect; Bundler prints a
+  deprecation warning naming the replacement. The modern
+  `bundle config set --local path …` sets `BUNDLE_PATH`, which is the
+  *install location* — pointing it at the gemchat checkout made even `rake`
+  unresolvable.
+- `bundle config set --local local.<gem> …`, for the reason above.
+
+What actually works is a path source in the Gemfile:
+
+```ruby
+gem    "gemchat", path: "/path/to/gemchat"
+plugin "gemchat", path: "/path/to/gemchat"
+```
+
+Verified end to end with this: the plugin installs, `plugins.rb` is loaded from
+the checkout, `.bundle/plugin/index` records `after-install-all: [gemchat]`, the
+hook fires on `bundle install`, `bundle add` and `bundle update`, and it stays
+silent under `CI=1` and `GEMCHAT_NO_AUTO_INDEX=1`.
+
+**The one thing this cannot test is the dual-copy condition.** A `path:` source is
+referenced in place — no second physical copy is made, and `.bundle/plugin/gems/`
+stays empty. So the two-copies finding in §15.4 must be re-verified against a
+genuinely *published* gem before release; the local path harness exercises every
+other mechanic but structurally cannot reproduce that one.
+
+Verified against Bundler 4.0.21 and a real published plugin gem
+(`bundler-rails-hyperdrive` 0.1.0), not from memory:
+
+| Question | Answer |
+|---|---|
+| Dual declaration: one copy or two? | **Two.** `<project>/.bundle/plugin/gems/<name>-<ver>` and `<bundle>/gems/<name>-<ver>` |
+| Does the lockfile pin the plugin copy? | **No.** Bundler 4.0.21 writes **no `PLUGINS` section**; only `DEPENDENCIES` |
+| How does a plugin register? | `Bundler::Plugin.add_hook("<event>") { \|args\| }` in a top-level `plugins.rb` |
+| What form does `<event>` take? | The **string** key of `Bundler::Plugin::Events` (`"after-install-all"`). The constant or an underscored symbol raise |
+| `Bundler::Plugin` a class or module? | A **module** — you cannot subclass it |
+| Available events | `before/after-install`, `before/after-install-all`, `before/after-require`, `before/after-require-all` |
+| Does a malformed plugin break the bundle? | **Yes.** `MalformattedPlugin` → `PluginInstallError` aborts `bundle install` before anything is written |
+| Does the hook fire on `bundle add`? | Yes — it fires on the post-install path, which `add` and `update` share |
+| Can plugins be disabled? | Yes — `Plugin.hook` returns early unless `Bundler.settings[:plugins]`, so `bundle config set --local plugins false` disables it globally |
+
+Consequence for §7.1: because two copies exist and the lockfile does not tie
+them, the plugin **must not** be the thing that owns the store. §15.4's
+stateless-shim design is what makes that safe — the shim requires the bundled
+copy, so only one version ever runs. Store identity is then checked by the
+running copy against `meta`, and a mismatch is a clear error rather than silent
+corruption.
+
+## 16. Agent skill
+
+The gem ships `skills/gemchat/SKILL.md` — a CLI is worthless to an agent that
+does not know it exists. Reuses the artifact format validated in `b9d1491`
+against `agentskills.io/specification`: frontmatter with `name` +
+`description`, body as agent-facing instructions.
+
+The skill must encode:
+
+- **When to use it** — questions about how a gem in *this* project works, its
+  API, configuration, version differences.
+- **Which invocation** — `bundle exec gemchat` when the gem is in the Gemfile,
+  otherwise bare `gemchat` (§14).
+- **Check `gemchat status` first**, and act on what it reports rather than
+  guessing. It states backend, index size, ri coverage, model state, and hosted
+  availability (§12.4).
+- **Version fidelity matters here.** If the local backend is active, prefer it;
+  if the user is on the hosted backend, answers are about gemchat.org's indexed
+  version, not the lockfile pin. Say so rather than presenting a drifted answer
+  as exact.
+- **Division of labor.** Gem documentation → this tool. *This application's*
+  runtime state, routes, schema, or logs → Rails Hyperdrive.
+- **Cost expectations** — `search` is instant and free with no model on disk;
+  `vsearch` and `query` need `gemchat embed` to have run first and do **not**
+  download anything themselves. The one-time download is **146MB**, not the
+  300MB this section originally said, and only `embed` performs it.
+
+Two corrections to this section as written, both made when the skill was
+authored:
+
+- The 300MB figure predates §10.1; the shipped default is 146MB.
+- This section says `status` reports "hosted availability", which it does — as
+  the `hosted: not configured → gemchat.org/settings` line. But §12 is not built,
+  so there is no hosted backend to be available yet. The skill says so rather
+  than describing a tier that does not exist.
+
+Side benefit: because it ships in a `skills/` directory, it is also consumable
+by Rails Hyperdrive's `enabled:` opt-in, which installs skills from any bundled
+gem regardless of native Hyperdrive support.
+
+## 17. Configuration and trust
+
+A checked-in `.gemchat.yml` travels with a `git clone` and must not be trusted
+unattended. **Four** things can reach outside the project or execute code:
+
+- command hooks
+- collection/store paths outside the project
+- custom model URIs
+- **the Bundler plugin auto-index hook** (§15)
+
+Two ways to grant approval, recorded in `~/.gemchat/trusted.json`:
+
+- **Implicitly, by use** — the first explicit `gemchat` command run inside a
+  project approves that project. The common path, no ceremony.
+- **Explicitly, via `gemchat trust`** — pre-approves a project you have
+  reviewed, and is the only way to enable auto-index in CI on purpose.
+
+Non-interactive contexts never reach the implicit grant and therefore **skip**
+gated behaviour, continuing with in-project indexing. `gemchat trust list` and
+`gemchat trust revoke` inspect and withdraw approvals. Editing any gated field
+invalidates a prior approval and re-asks, matching qmd.
+
+## 18. CLI surface
+
+```
+gemchat init [--hosted|--no-local] [--no-auto-index]   # once per project; only cmd that writes the Gemfile
+gemchat index [--all] [--dry-run] [--jobs N] [--ri-only]
+gemchat embed [--force] [--all]        # vectors only; downloads the model on first use
+gemchat search   <query> [-n N] [--json|--files] [--min-score] [--explain]
+gemchat vsearch  <query> [-n N] [--json|--files]
+gemchat query    <query> [-n N] [--json|--files] [--explain] [--hosted]
+gemchat status                                    # backend, delta, disk, store identity, tier ladder
+gemchat trust [list|revoke]
+gemchat reindex                                  # force, after a store-identity change
+gemchat version
+```
+
+`index` and `embed` are separate verbs, following qmd (`qmd update` vs
+`qmd embed`). That separation is what lets `search` work instantly after
+`index` without a 300MB download; `embed` is the explicit opt-in to semantic
+recall. `vsearch` and `query` also trigger it lazily on first use, so the
+explicit command is a convenience rather than a requirement.
+
+Agent-first output: `--json`, `--files`, `--explain` (retrieval score traces),
+`--min-score`, and `gemchat://` URIs. Defaults tuned for agents, not humans.
+
+## 19. Repository work
+
+Now done, and verified by a `gem build` that produces a non-empty package:
+
+- **`exe/gemchat` exists.** `bindir = "exe"` and `executables` greps
+  `spec.files` for `exe/`, so without it there is no binary. The convention
+  splits `exe/` (ships, lands on `PATH`) from `bin/` (dev scripts, explicitly
+  rejected from `spec.files` — already holds `bin/console` and `bin/setup`).
+- **Gemspec metadata filled in** — `summary`, `description`, `homepage`,
+  `source_code_uri`, `changelog_uri`, and `rubygems_mfa_required = "true"`.
+- **The description is unambiguous about local-vs-hosted.** Users will see
+  `gemchat` in a Gemfile and reasonably expect the gemchat.org *service*. It
+  states plainly that this is a local indexer with an optional hosted mode: no
+  account needed for local use.
+- **Runtime dependencies declared**: `sqlite3` (FTS5, precompiled natives),
+  `rllama` (prebuilt llama.cpp binaries, so no compiler on install), `prism`
+  (unconditional, so the symbol pass behaves identically on Ruby 3.2), `rdoc`.
+  The omission of these from the skeleton gemspec was not caught by the manual
+  smoke test, only by running the suite under `bundle exec` — where
+  `require "sqlite3"` failed, because an undeclared gem happens to be present
+  outside a bundle. **A smoke test that runs outside `bundle exec` will not
+  catch a missing runtime dependency.**
+- **Linter is `standardrb`** (`.standard.yml`), not rubocop.
+- `spec.files` derives from `git ls-files -z`, so untracked files are invisible
+  to the built gem. `git add` before `gem build` still matters — this will cover
+  `skills/gemchat/SKILL.md` and `plugins.rb` when they arrive.
+- **`plugins.rb` is shipped** at the top level of the gem, in the Hyperdrive
+  shape: the file holds only the `add_hook` registration, and the logic lives in
+  `lib/`. Verified against a local `path:` checkout — installs, records
+  `after-install-all` in `.bundle/plugin/index`, fires on install/add/update,
+  silent under `CI` and `GEMCHAT_NO_AUTO_INDEX`. It currently only reports that
+  auto-index is not implemented yet (§20 phase 2). The dual-copy condition is
+  still unverified for a *published* gemchat; see §15.6a.
+- Optional `spec.metadata["hyperdrive_targets"]` for Hyperdrive `discover`
+  interop (§16).
+- **Prism** as an unconditional runtime dependency. Ruby 3.3+ bundles it, but
+  depending on the gem unconditionally keeps behaviour identical on 3.2.
+- **No test framework** declared. Minitest, to match the Rails app. (Done in
+  Phase 1: 77 examples, no skips.)"
+- Runtime dependencies: `sqlite3`, `rllama`, `prism`, `rdoc`. **Not**
+  `rdoc-data` (§4.2) and **not** `rubydex` (§4.4) — the latter is a marginal
+  owner-resolution gain and would break `bundle install` for musl users.
+
+## 20. Phases
+
+1. ~~**Spike — ri resolution, chunking, FTS5, and the plugin mechanics.**~~
+   **Done 2026-09-26.** ri generation cost measured across 49 real direct deps
+   (9.9s total, ~2s on 4 cores — see §6); `read_ri_from_path` ported; chunker and
+   FTS5 store built; `gemchat index` + `gemchat search` working end-to-end with
+   exact signatures; all plugin mechanics answered in §15.6, including the
+   dual-copy result that forces the shim design in §15.4.
+   **Test suite added alongside it:** 77 Minitest examples, no skips, covering
+   the FTS quoting rules, chunk caps and digests, the store's
+   indexed/updated/unchanged outcomes, RDoc failure cleanup, and CLI exit codes.
+   Four implementation defects were found by writing those tests: a memoised
+   `Gemchat.root` that made the *first* `GEMCHAT_HOME` reader win for the life of
+   the process (tests were writing to the real `~/.gemchat`), `Store#search`
+   leaking sqlite3's integer row keys into results, blank fields being indexed as
+   dead tokens, and `index` reporting `generated … 0 chunks` on a no-op. Fixed
+   in this phase; see §4.5.
+2. ~~**Resumable ri generation** (§13) — **largely superseded.** The phase-1
+    measurement showed generation is seconds, so this was never a necessity. The
+    specific cost it was meant to fix — a repeat run re-reading and re-chunking
+    every gem, 0.19s for irb and 0.04s for rake — is now handled one level up:
+    the `.gemchat.yml` digest gate means `gemchat index` is only reached at all
+    when the lockfile moved, and the per-gem store digest skips unchanged gems
+    after that. Worth revisiting only if a real bundle makes indexing visibly
+    slow, not as planned work.
+3. ~~**Delta indexer + `.gemchat.yml` manifest + store identity (§7.1) +
+   trust**~~ — **Done 2026-09-27.** `5dac123` (trust + sha256 gate), `c0f432d`
+   (`gemchat init`, `reindex`, store identity). The detail is in the "Also landed
+   with the plugin" note below.
+4. ~~**Embeddings via rllama + `gemchat embed` + model download UX + store-identity
+   guard**~~ — **Done 2026-09-27.** `bfc730f` (model download), `0fa0477` (vector
+   storage and ranking), `8c2f6be` (`embed`, `vsearch`). The default model
+   changed from the plan's `embeddinggemma-300M` to `nomic-embed-text-v1.5`
+   because the former is gated; §10.1 has the reasoning and §10.3 the crash
+   measurements.
+5. ~~**Hybrid RRF + source priors + Prism symbol metadata**~~ — **Done
+   2026-09-27**, and not how the phase was written.
+   - *Hybrid RRF* landed in `34d137b`. §11.3 records what it measured: a no-op on
+     questions, because the lexical arm is a phrase matcher, and §11.4 records
+     the designed-but-falsified fix.
+   - *Source priors* were measured before being built, and **not built**. Against
+     an evenly-split ri/prose eval of 10 known-answer questions, the app's own
+     1.15 prose prior is neutral on hit@1 (6/10 either way) and worse on MRR
+     (0.744 → 0.705); 1.30 gives 5/10, and a 1.15 ri prior gives 4/10 with MRR
+     0.547. Recall@10 is 10/10 in every case, so a prior only ever reorders, and
+     always for the worse. A prior shifts a whole class of documents and cannot
+     tell a good prose chunk from a bad one, so the app's constant does not
+     transfer to a bundle-scoped index.
+   - *Prism symbol metadata* landed as `Gemchat::Symbols`.
+6. **Hosted tier** (§12) — `--hosted`, command mapping, the tier ladder in
+   `status`
+7. ~~**`skills/gemchat/SKILL.md`**~~ — **Done 2026-09-27.** Written against the
+   real CLI rather than this section, which had drifted: it says 146MB, not
+   300MB, and it does not claim `vsearch` may trigger a download, because only
+   `embed` downloads. The most valuable thing in the file is the routing table
+   and the warning that `search` is a phrase matcher, because that is the one
+   thing an agent cannot infer and the one that silently returns nothing.
+   `test/skill_test.rb` guards it mechanically: the skill must name only
+   commands that exist, must document every command that does, and must not
+   regress to a stale model size.
+8. MCP **server** — deferred; the hosted gemchat.org already owns that surface
+
+Also landed with the plugin: `Gemchat::Lockfile` (the sha256), `Gemchat::Manifest`
+(`.gemchat.yml`), `Gemchat::Trust` (`trusted.json`), and `Gemchat::Hook` (the
+logic, reached through the bundled copy). `gemchat index` grants trust for the
+current project and records the manifest, so opting in is simply running it
+once; the hook then indexes on any later `install`/`add`/`update` that moves the
+lockfile digest, and stays completely silent when nothing has.
+
+Also landed: `gemchat init` (§15.2), the only command that may modify a Gemfile.
+It adds `gem "gemchat", group: :development` and `plugin "gemchat"`, idempotently
+and matched per directive, grants trust, and tells the user to run `bundle install`
+once because Bundler does not activate a newly declared plugin until the next
+install. `gemchat reindex` is the remediation `Store::IdentityError` names.
+`gemchat status` now reports the lockfile digest, manifest state, trust, and
+whether the hook is actually *armed* — read from Bundler's own plugin index,
+because a Gemfile line only says the plugin was requested, not that it runs.
+
+Store identity (§7.1) is in: `schema_version` is a hard gate and the store
+refuses to open on a mismatch, `gemchat_version` drift is tolerated and surfaced
+in `status`, and `embedding_model`/`embedding_dimensions` are reserved for the
+embedding phase.
+
+### 20.1 Prose ingestion — `Gemchat::Prose`
+
+`Gemchat::Markdown` splits prose into heading-scoped sections, handling ATX,
+setext and RDoc headings and treating a `#` inside a fenced block as code rather
+than a heading. `Gemchat::Prose` discovers the sources below and labels each one,
+so a gem's rows never overwrite each other and search output says what kind of
+document a hit came from.
+
+| Source | Path | `source_type` |
+|---|---|---|
+| README | `README.{md,markdown,mkd,rdoc,txt}`, bare `README` | `readme` |
+| Guides | `guides/**`, `guide/**` | `guides` |
+| Reference docs | `doc/**` prose | `doc` |
+
+**`readme`, `guides` and `doc` stay distinct types, grouped only at query time.**
+This is the vocabulary `gemchat_app` already uses, and it is not incidental: its
+`search_service.rb` gives `guides` a prior of `1.15` — ranked *up*, because guides
+are better than baseline for the questions they suit — and derives a
+`readme_guides` group via `source_group` purely to enforce prose diversity
+(`WORKFLOW_MIN_README_GUIDES`). Storing them separately and grouping at query time
+is the model. Folding `guides/` into `readme` would also make local `guides` mean
+"bundled `guides/`" while hosted `guides` means "web-crawled reference pages" —
+drift that is miserable to unpick later. `doc/` is likewise kept separate from
+hosted `guides`: the local tool's whole claim is offline and local, and conflating
+"shipped in the gem" with "fetched from the internet" would muddy that.
+
+`guides/` is **0 of 409** gems in the measured population, so its own type is
+justified by vocabulary and ranking headroom, not by data. `doc/` is 3.9%. Stated
+plainly so the split is not mistaken for evidence-driven.
+
+**`doc/` is hand-written prose, and an earlier draft of this plan asserted the
+opposite.** The claim was that `doc/` holds generated ri and that indexing it
+would "double the corpus with the same text `ri` already covers". Measured across
+every installed gem with a `doc/`:
+
+| | |
+|---|---|
+| gems with `doc/` | 12 |
+| files | 96 |
+| prose (`.md`/`.rdoc`/`.txt`) | **83** |
+| generated ri | **0** |
+| files byte-identical to the README | **0** |
+| subdirectories | all hand-written: `example`, `irb`, `releases`, `markup_reference`, `ja`, `en` |
+
+The generated shape, `doc/<name>-<version>/ri`, appears in none of them. What is
+there is exactly what ri cannot produce: `irb/Index.md` (23KB), `irb/EXTEND_IRB.md`,
+`irb/Configurations.md`, `rake/glossary.rdoc`, `test/how-to.md`,
+`rubygems/UPGRADING.md`, `racc/doc/en/grammar.en.rdoc`. The claim could not be
+traced to any source — not `ri-doc-generation.md`, and `gemchat_app`'s own
+AGENTS.md lists `guides` as *web-crawled*, not read from `doc/`. It appears to be
+an over-generalisation from the ri-*generation* context, where RDoc's default
+target happens to be called `doc/`. Asserting a plausible rule without measuring
+it is the exact failure mode this document keeps warning about.
+
+The generated-ri case is still excluded, specifically: any `**/ri/**` path under
+`doc/`, because `gem install` with documentation does write there, and it costs
+nothing. So is generated web output (`.html`, `.js`, `.css`). This is the
+narrow, correct version of the earlier rule rather than a rejection of it.
+
+**Non-English documents are skipped by default**, with `--all-languages` to opt
+in. `doc/` carries substantial translations — `irb.rd.ja` (15KB),
+`grammar.ja.rdoc` (14.5KB), `usage.ja.html` (18KB) — and Japanese text pollutes
+BM25 with tokens no English query can match, while still costing embedding time
+later. Detection is by filename and directory marker (`ja`, `zh`, `ko`, and a
+documented short list), which is predictable and covers every observed case:
+`racc/doc/ja/`, `irb/doc/irb/irb.rd.ja`, `typeprof/doc.ja.md`. `en` is always
+kept. This also fixes `racc`, whose *only* README is `README.ja.rdoc` and which
+therefore currently indexes Japanese into an English index.
+
+**Long sections are split, not truncated.** `MAX_BODY_CHARS` is 2000, tuned for ri
+method bodies where that is naturally the whole unit. Applied to prose it silently
+dropped **13% of all `doc/` prose**, concentrated in files with long sections:
+`tutorial.rdoc` (40KB, 22% lost), `master_toc.rdoc` (40%), `doc.md` (13%),
+truncating mid-sentence. Sections over the cap are now split on blank lines into
+consecutive chunks, titled with a ` · 2/5` suffix so a search returning three
+parts of one section does not look like three unrelated hits. The cap itself is
+unchanged, so there is no new constant and ri chunks are unaffected.
+
+Sections are stored under their own `source_type` so a gem gaining or losing a
+README does not rewrite its much larger ri row. A gem then has one row per source
+type, so `Store#stats` counts distinct name+version rather than rows, or every gem
+with a README would be reported twice.
+
+Measured after the change, across every installed gem with a `doc/`: **1,406
+chunks** and **601KB** of prose, `doc` 45 / `readme` 12 / `ri` 179 for a
+two-gem rake-only bundle. `rake/doc/command_line_usage.rdoc` is now findable —
+`rake › doc › Rake Command Line Usage` — which was unreachable before.
+
+Still ahead: vectors, hybrid RRF and Prism metadata, the hosted tier, and
+`skills/gemchat/SKILL.md`.
+
+## 21. Risks
+
+- **The Bundler plugin is a code-execution surface.** A checked-in
+  `plugin "gemchat"` runs for anyone who clones the repo. Mitigated by the trust
+  gate defaulting to a no-op print (§15.4), but it remains the sharpest edge
+  here and warrants explicit review before any release.
+- **Plugin copy may drift from the dependency copy.** Resolved: the two copies
+  are *confirmed* and nothing pins them (§15.4), so the answer is not to
+  deduplicate but to make the plugin copy carry no logic — it requires the
+  bundled copy, so drift is harmless by construction. The shim is not written
+  yet (phase 7), so this is a design commitment, not a tested result.
+- ~~**ri generation cost is unmeasured.**~~ **Resolved in phase 1:** 9.9s
+  across 49 real direct dependencies, 0.09s median, ~2s wall clock on 4 cores
+  (§6). The resumable design is good hygiene rather than a necessity. The
+  remaining open question is different and smaller — whether unchanged gems can
+  be skipped *before* re-reading and re-chunking their ri (§20 phase 2).
+- **Version fidelity.** Serving the wrong version's ri defeats the tool's
+  purpose; the exact-version glob is a correctness requirement.
+- **Hosted tier couples the gem to gemchat.org.** Hosted breaking changes reach
+  the gem. Mitigate by treating gemchat.org as a versioned API and tolerating
+  unknown parameters (§12.5).
+- **Silent escalation risk.** If the local path ever auto-falls-back to hosted,
+  the tool becomes a privacy leak. The "diagnose and stop" rule (§12.4) is a
+  hard requirement, not a default.
+- **Windows** has no prebuilt `rllama` binary. Requires an explicit "use
+  `--embedder ollama` or `--embedder openai`" message — never a crash.
+- **300MB model download** is the largest user-visible cost. BM25 without models
+  is the mitigation.
+- **RDoc `SystemStackError`** on pathological gems.
+- **Name collision with the hosted service.** Same name, two products. Mitigated
+  only by documentation discipline (§19).
+- **Drift** from `gemchat_app`'s parsing and ranking logic, which is read-only
+  reference. Accept divergence when the local store's constraints differ;
+  re-sync deliberately.
+
+## 22. Out of scope
+
+- Cross-gem discovery, alternatives, gem guides
+- MCP **server** (phase 8, deferred)
+- Answer generation / chat — this tool returns chunks and citations
+- Cross-project global search — the manifest scopes queries to the bundle
+- Windows support for the default embedder (opt-in providers only)
+- Multi-user or shared index stores
+- Automatic background indexing while no bundle command runs
+
+## 23. Open questions
+
+- ~~Per-gem ri generation cost on a realistic bundle.~~ **Answered in phase 1:**
+  9.9s for 49 direct deps, 0.09s median, ~2s on 4 cores (§6). Carried forward as
+  a narrower question in phase 2: whether unchanged gems can be skipped before
+  re-reading and re-chunking their ri, rather than after.
+- ~~Should `vsearch` warn when the store has no vectors yet, or silently fall back
+  to BM25?~~ **Answered by the shipped code: it warns and exits 1.** "no vectors
+  yet → run `gemchat embed`" on stderr, with the reminder that `gemchat search`
+  still works. The silent-fallback option is the one §12 explicitly rejects for
+  the hosted tier — "a user who asked for hosted results and got local ones
+  cannot tell the difference" — and the same reasoning applies here: `vsearch`
+  would be answering a different question than the one asked.
+- ~~Should a `--all` run hard-cap per-gem ri chunks?~~ **Answered 2026-09-27: no,
+  and the framing was wrong.** Measured across 120 local gems, 33 with ri:
+
+  | | chunks |
+  |---|---|
+  | median gem | 32 |
+  | p90 | 32 |
+  | max (`parser`) | 8,631 |
+  | total ri | 21,253 |
+
+  The distribution is not a long tail, it is bimodal: a median of 32 and a
+  maximum of 40% of the corpus in one gem. A cap of 5,000 would keep 82.7% of
+  chunks, i.e. **delete 3,631 real methods of `parser`** — not "a few percent of
+  one gem" as the question assumed, but most of a gem the user has installed,
+  and silently.
+
+  What the cap was meant to bound, measured on that same 22,141-chunk index:
+
+  | | |
+  |---|---|
+  | `gemchat index` | 20s |
+  | `gemchat embed` | 494s (8.2min), **22,134 of 22,141 embedded, 7 pending** |
+  | vector query | 915ms (§11.1) |
+  | store size | 13.5MB |
+
+  99.97% embed success on a 94x larger corpus than anything else here was
+  measured on, which is the strongest single validation of the supervised-child
+  design — it was built for a cliff that could not be located, and it absorbed
+  22k chunks without one.
+
+  So there is no failure for a cap to prevent, and implementing one would trade a
+  large, silent loss of real API surface for a bound nobody needs. **No default
+  cap.** If a pathological gem is ever observed — a vendored mega-gem, say —
+  the trigger is visible in these numbers: a single gem above ~10,000 chunks is
+  5% of this corpus and worth capping then, loudly and overridably, not silently
+  and now.
+- **Whether Tier 3 (core/stdlib) belongs in v1.** **Answered 2026-09-27: not in
+  v1.** The argument for is that it is genuinely free — 727 stdlib `.rb` files
+  under `RbConfig::CONFIG["rubylibdir"]`, no download, no dependency, ~20s to
+  index by the same recipe. The argument against is scope: the corpus is defined
+  as *the gems in your bundle*, and stdlib is not in the bundle. Indexing it
+  answers questions about methods the project does not use, diluting every
+  result with 727 files' worth of otherwise-unreferenced API. Answering
+  "what does `FileUtils#split_all` do" for a gem you do not have installed is
+  what the hosted tier exists for. Revisit if someone asks; the recipe is in
+  §8.4 and nothing else has to change.
+- **Whether to add a `gemchat changelog <gem>` subcommand.** **Deferred, and the
+  plan's reasoning is now wrong.** It said the hosted `read_changelog` tool
+  "supports it for free" — true when the tier spoke MCP, false now. The tier is a
+  row API (§12.6), so a changelog subcommand needs a second endpoint in
+  `HostedQuery` plus a new subcommand, and neither is free. Worth doing when
+  something else needs the endpoint.