rails-ai-context 5.32.2 → 5.34.0

data/docs/TROUBLESHOOTING.md

@@ -11,7 +11,7 @@
 ---
 
 > [!TIP]
-> Always start with `rails ai:doctor`. It runs 25 checks and catches most issues automatically.
+> Always start with `rails ai:doctor`. It checks the app, its databases, the MCP configs and the context files, and catches most issues automatically.
 
 ## Diagnostics first
 
@@ -20,7 +20,7 @@ rails ai:doctor          # In-Gemfile
 rails-ai-context doctor  # Standalone
 ```
 
-This runs 25 checks and returns an AI readiness score (0-100). Each failed check includes a fix suggestion.
+This runs every check that applies to the app ([the list](GUIDE.md#diagnostics)) and returns an AI readiness score (0-100). Each check that fails or warns includes a fix suggestion.
 
 ---
 
@@ -70,9 +70,10 @@ ls -la .mcp.json .cursor/ .vscode/ .github/
    - OpenCode: `opencode.json`
    - Codex: `.codex/config.toml`
 
-2. Verify the command works:
+2. Verify the command the config holds works:
    ```bash
-   bundle exec rails ai:serve
+   bundle exec rails-ai-context serve   # in-Gemfile
+   rails-ai-context serve               # standalone
    # Should output nothing (waiting for stdio input)
    # Ctrl+C to stop
    ```
@@ -82,18 +83,66 @@ ls -la .mcp.json .cursor/ .vscode/ .github/
    rails generate rails_ai_context:install
    ```
 
+### "Could not write .vscode/mcp.json - that tool will not auto-discover the MCP server"
+
+The install never replaces a config it cannot write back faithfully: one that does not parse as JSON, one holding comments that writing it back would drop, or one that is not UTF-8. Trailing commas, which VS Code and OpenCode accept, are read the way they read them under json 2.9 or later; an older json refuses them, as JSON does. The line after the warning says which, and ends with the exact entry to add, as `Add {"servers":{...}} to it by hand`. Add that entry yourself, or remove the comments (and, under an older json, the trailing commas) and run the install again. `doctor` names the same entry for a file whose comments keep the install out, rather than sending you back to the install, and asks for one that is not UTF-8 to be saved as UTF-8 first.
+
+Claude Code reads `.mcp.json` with a plain JSON parser, and loads no server from a file holding a comment or a trailing comma ("MCP config is not a valid JSON"); Cursor's `.cursor/mcp.json` is held to plain JSON too. Either one is reported as `Claude Code cannot read it: it holds a trailing comma, which JSON does not allow`, by the install and by `doctor`, however current the entry in it. Make the file plain JSON and run the install again. `doctor` reports one that does not parse, or holds no object to merge into, a workspace's above the app included. Dropping an AI tool leaves such a file alone the same way and names the entries to remove.
+
+### Codex: "TOML parse error ... duplicate key"
+
+Codex refuses a `config.toml` that declares a server twice. An earlier install could write a second `[mcp_servers.rails-ai-context]` table beside the server spelled another way: a quoted name (`[mcp_servers."rails-ai-context"]`), an inline table under `[mcp_servers]`, or dotted keys. `doctor` names such a file. Two tables are merged into one when the install runs again; for an inline table or dotted keys, delete one of the two by hand.
+
+The install now finds the server however its name is quoted and replaces its table where it stands. An entry of the gem's written as an inline table or with dotted keys is left as it is, with a warning, as is a file whose `mcp_servers` is one inline table: write it as a `[mcp_servers.rails-ai-context]` table, or delete it, and the next run keeps it current. `doctor` gives that same edit for any problem in such an entry that only a rewrite fixes, a stale env snapshot included.
+
+### Codex: "stream did not contain valid UTF-8"
+
+Codex reads `config.toml` as TOML, which is UTF-8, and loads nothing from a file holding a byte that is not - one in a comment included. The install still writes its table into such a file, keeping every byte around it, and says `Codex CLI cannot read it: it is not UTF-8, which TOML is`; `doctor` fails it in the same words. Save the file as UTF-8.
+
 ### "MCP server fails to start"
 
+`rails ai:doctor` (or `rails-ai-context doctor`) checks the command each config holds and says which one cannot start and why. The usual causes:
+
+- `rails-ai-context is not currently included in the bundle`: the config runs `bundle exec rails-ai-context serve`, but the gem left the app's Gemfile. A standalone install runs `rails-ai-context serve`: run `rails-ai-context init` to rewrite the configs.
+- The config runs `rails-ai-context serve` while the app's Gemfile.lock carries the gem: it needs a copy installed outside the bundle as well, which hands the run to the bundle's. Run `rails generate rails_ai_context:install` to write `bundle exec rails-ai-context serve`.
+- `can't find executable rails-ai-context for gem rails-ai-context`: the bundle's copy of the gem lists no executable, as a `path:` copy of 5.32.2 or earlier outside a git checkout does. Point the Gemfile at a git checkout or a release.
+- On Windows, a config that runs `bundle` or `rails-ai-context` itself rather than through `cmd /c`: those are batch files there, which Claude Code, Codex and the other clients that start a server without a shell do not run (`doctor` warns about it). Run the install again; on Windows it writes `cmd /c bundle exec rails-ai-context serve`.
+- On macOS or Linux, a config that runs `cmd /c ...`: it was written on Windows, and there is no `cmd` here (`doctor` fails it). Run the install again on this machine, and keep the MCP configs out of git if the team works on both.
+
 Check for Bundler/Ruby path issues:
 
 ```bash
 which bundle
-which rails
+which rails-ai-context
 ruby -v
 ```
 
 For Codex CLI specifically, the env section in `.codex/config.toml` must match your current Ruby environment. Re-run install if you changed Ruby versions.
 
+### Answers say "Boot did not finish within 20s"
+
+The app's boot hung, usually an initializer waiting on a service that is
+down (Redis, Elasticsearch, a secrets store). A stdio server gives the boot 20
+seconds, because its client stops waiting for the first answer after about 30
+(Claude Code, the MCP Inspector), and then serves the static tier; every other
+command gives it 60. Start the service, or, for an app that is healthy but
+slow to boot, raise both limits in the server's MCP entry: the boot's with
+`RAILS_AI_CONTEXT_BOOT_TIMEOUT` (seconds) in its `env`, and the client's with
+its own setting (Claude Code reads `MCP_TIMEOUT`, in milliseconds). Raising
+only the boot's leaves a client that gives up first. `rails ai:serve` has no
+static tier, so it exits with the same message instead.
+
+### A server set up for a folder of apps fails to start
+
+`rails-ai-context init` run in a folder of apps writes entries that name each app relative to that folder (`--app-path shop`, and for an in-Gemfile app `BUNDLE_GEMFILE=shop/Gemfile`, or the monorepo's shared Gemfile its `config/boot.rb` names). Claude Code, Codex and OpenCode start a server in the folder they were launched in, so:
+
+1. Start the AI tool in the folder of apps, not inside one of them. Launched inside an app, the tool still reads the folder's config above it, and the server says `--app-path shop is read from ...; it names an app from ...`. An in-Gemfile entry fails earlier, in Bundler, with a Gemfile it cannot find. To work in one app on its own, run `rails-ai-context init` inside it too.
+2. Check the Ruby. Every server in the folder runs under the Ruby your version manager picks in the folder, and `init` warns about an app that declares another. `Your Ruby version is ..., but your Gemfile specified ...` from an in-Gemfile entry is that case: install the app's bundle for that Ruby, or give the app its own config with `init` inside it.
+
+### "No Rails app found in ..., and 2 below it"
+
+You ran a command in a folder that holds several apps. The error lists the exact command for each, with `--app-path`. To set them all up at once, run `rails-ai-context init` in that folder.
+
 ### "Tools return empty results"
 
 1. Check that your Rails app boots: `rails runner "puts 'ok'"`
@@ -110,7 +159,8 @@ dependency graph, or launching from inside another Bundler project's
 directory. Fixes:
 
 1. Launch `rails-ai-context serve` with the app root as the working directory
-   (the generated MCP configs do this by default).
+   (the generated MCP configs do this by default; a folder of apps' configs
+   start in that folder and name the app with `--app-path`).
 2. Make sure `GEM_PATH` covers the complete default gem path
    (`gem env path`), not just `gem env gemdir` - Ruby's bundled gems
    (racc, etc.) live in a separate directory.
@@ -120,10 +170,29 @@ directory. Fixes:
 ### MCP server responds slowly
 
 - Check `config.cache_ttl` - lower values mean more frequent re-introspection
+- Without the `listen` gem, each call first checks the app's files for edits, about 100 ms at 10,000 files; adding `listen` to the development group replaces that check with live reload
 - Check `config.preset` - `:standard` is faster than `:full`
 - Large schema files (>10MB) slow down schema introspection
 - Run `rails ai:doctor` - it checks schema size and view count
 
+### The stdio server stops with "MCP stdio connection closed: stdio frame exceeds 4194304 bytes"
+
+The MCP SDK reads one JSON-RPC message per line and refuses a line of 4 MiB or more. The server answers the client with a JSON-RPC `-32600` error, prints the reason on stderr and exits 1. The message is too large for any client to need; look for a tool argument the client filled with a whole file.
+
+### HTTP answers an HTML error page while a migration is pending
+
+In development Rails answers every request with its pending-migration page, and the mounted engine is a route like any other. Run the migration, or use `auto_mount`, stdio or the standalone server, which all answer ahead of that check; `rails_runtime_info` lists the pending migrations.
+
+### HTTP answers 403 "Forbidden: Invalid Host header"
+
+The standalone server (`serve --transport http`, `rails ai:serve_http`) answers only requests whose `Host` header is `127.0.0.1`, `::1` or `localhost`, as the MCP SDK does, to stop DNS rebinding. A client that reaches it by the machine's IP or hostname is refused, whatever `http_bind` says. Connect to `localhost`: through a published container port, or an SSH tunnel (`ssh -L 6029:localhost:6029 your-host`).
+
+The mounted engine and `auto_mount` answer the hosts the app's `config.hosts` allows, as the rest of the app does. Add the name the client uses to `config.hosts`, the way you would for any other route.
+
+### HTTP answers 403 "does not serve MCP over HTTP in production"
+
+The mounted engine and `auto_mount` refuse every request outside development and test (production, staging) unless the app sets `config.allow_http_in_production = true`. Set it only with the endpoint behind your app's authentication; see [Security](SECURITY.md#mcp-http-transport).
+
 ---
 
 ## Context file issues
@@ -151,7 +220,11 @@ rails ai:context
 
 ### "My custom content in CLAUDE.md was overwritten"
 
-The gem uses section markers (`<!-- BEGIN/END rails-ai-context -->`) to preserve user content. Content outside these markers is preserved. If markers were removed, the gem may overwrite.
+The gem writes its section between `<!-- BEGIN rails-ai-context -->` and `<!-- END rails-ai-context -->` and replaces only what lies between them, so text above and below the markers survives every run; text inside them does not. A file with no markers at all gets the section added at the top, and its own text kept below.
+
+The per-directory `app/models/AGENTS.md` and `app/controllers/AGENTS.md` that OpenCode and Codex read follow the same rule. One the gem generated whole carries `> Auto-generated by rails-ai-context. Do not edit manually.` and is rewritten whole; one you wrote yourself keeps its text, and the listing goes between the markers above it.
+
+If one marker goes missing, or the file holds a second pair, nothing says where the gem's section ends. The run then leaves the file as it is and reports it as `unpaired rails-ai-context marker, left as it is`. Put the lost marker back where the gem's section ends, or delete the leftover section with its markers, and the next run updates the file again.
 
 ### "Generated files are too large"
 
@@ -218,6 +291,9 @@ brew install ripgrep
 
 # Ubuntu
 apt install ripgrep
+
+# Windows
+winget install BurntSushi.ripgrep.MSVC
 ```
 
 The gem falls back to Ruby regex if ripgrep isn't available. Search still works, just slower on large codebases.
@@ -238,7 +314,7 @@ Standalone mode boots the app with a small shim, then restores the `$LOAD_PATH`
 
 The file is read from the app root, once, at boot, and an initializer does not replace it: a `configure` block wins only the keys it assigns, key by key ([Precedence](CONFIGURATION.md#precedence)). In standalone mode the initializer contributes nothing at all, since the gem is not loaded while `config/initializers` runs. Check:
 
-1. You are in the app root, or passed `--app-path`.
+1. You are inside the app, or passed `--app-path`. A command run from a subdirectory reads the config of the app root it walks up to, and names it on stderr.
 2. The key is one the gem knows. An unknown key warns on stderr and is ignored.
 3. The file is readable and parses. Corrupted YAML degrades gracefully with a warning.
 
@@ -262,10 +338,12 @@ Without it, the tool reports "not installed" but the gem works fine otherwise.
 
 The scan runs under the app's own bundle, so a brakeman installed globally is
 not on its load path. The tool falls back on its own: it runs the installed
-brakeman as a separate process outside the bundle and says so under the
-results. This message means that fallback produced no report either - the gem
-is there and the run failed. Run `brakeman` in the app directory to see what
-it hit, or add it to the Gemfile so the scan runs in-process:
+gem's `bin/brakeman` with the app's Ruby, as a separate process outside the
+bundle, and says so under the results. This message means that fallback
+produced no report either - the gem is there and the run failed. `rails
+ai:doctor` runs the same script's `--version` and names what it printed. Run
+`brakeman` in the app directory to see what it hit, or add it to the Gemfile
+so the scan runs in-process:
 
 ```bash
 bundle add brakeman --group development

data/docs/adr/0003-shared-tool-cache-semantics.md

@@ -8,10 +8,16 @@ Status: accepted
 
 **One client's session record was served to another.** `SESSION_CONTEXT[:queries]` was a single flat hash. Over stdio that is right - one process is one conversation - but the middleware, the engine controller and the standalone HTTP server each serve every client from one process, so `rails_session_context` listed calls the asking client never made. The record is now bucketed per conversation: all three HTTP entry points wrap each request in `BaseTool.with_session(<Mcp-Session-Id>)`, and stdio falls through to a single default bucket, unchanged.
 
+**The scope ended before the tool ran.** The SDK answers a request inside a session with a streaming body, and the tool runs when the web server calls that body - after the entry point has returned, so the scope around `handle_request` was already gone and the middleware and the standalone server recorded every client in the default bucket again. `McpEdge.serve` hands the web server a body that re-enters the client's session. The engine controller calls the body itself, inside its own scope.
+
+**One client's reset emptied every client's record.** `action: "reset"` cleared the whole hash. It clears the calling conversation's bucket now (`current_session_reset!`). `session_reset!` still clears them all, for a change that makes every earlier answer stale: live reload.
+
 **`session_queries` handed out live entries.** It returned `values.dup`, a shallow copy, so the entry hashes stayed live inside the record and kept being mutated by later calls - a caller's snapshot changed under it. It now copies each entry.
 
 **The record grew without bound, keyed by a client-controlled header.** Bucketing per conversation introduced a hash whose keys come from `Mcp-Session-Id`, in a process that stays up. Three things were wrong at once: the hash had a default block, so merely *reading* a session's history created a bucket; nothing capped the id's length; and nothing evicted. Reading no longer writes, ids are truncated to `MAX_SESSION_ID_LENGTH`, and the number of remembered conversations is capped at `MAX_SESSIONS`.
 
+**A composite tool's inner calls were recorded as the client's.** `rails_get_context`, `rails_diagnose` and `rails_review_changes` answer by calling other tools, and every call recorded itself, so one `rails_get_context` read as four calls the client never made, and the first inner call took the outer call's params with it. SafeCall now marks the outermost call on the thread, and only that call is recorded, once, with its own params.
+
 Eviction is least-recently-used, not oldest-created. Recording re-inserts the session so hash order tracks use. Ordering by creation instead would evict the conversation that has run for hours ahead of a hundred idle newcomers - backwards, and worst on exactly the long-lived transports that made bucketing necessary.
 
 ## Intended semantics, not defects
@@ -21,3 +27,5 @@ Eviction is least-recently-used, not oldest-created. Recording re-inserts the se
 **`deep_dup` also happens under the mutex.** Every caller gets an independent copy, so a tool that mutates its context cannot corrupt the next tool's view. Copying inside the lock adds to hold time, and moving it outside would hand out the cached object itself between the read and the copy. Isolation is worth more than the contention; the copy is cheap next to the introspection it protects.
 
 Both are pinned by `spec/lib/rails_ai_context/tools/shared_cache_concurrency_spec.rb`, which also drives the middleware and the engine controller concurrently against both caches.
+
+**Every server checks the files outside the mutex, at the start of each call.** Within the TTL the fast path trusts the cache, so something has to drop it on an edit before a call reads it. Live reload cannot: `listen` delivers a change after its debounce, and with no `listen` in the bundle nothing delivers it at all. So `refresh_if_files_changed!` checks the files itself (`Fingerprinter::Snapshot`, a stat of each watched directory and file), and on a change reloads the app's code and resets the caches. That check cannot share the mutex: the code reload waits for running calls to finish, and those may be waiting on the mutex. Only a check that began after a call arrived can stand in for the call's own - an earlier one may have passed a file before it was edited - so calls made at once wait for one check together, and calls one after another each check. Sharing any check made within ten walks' time, as the first version did, answered a call made right after an edit from before it.

data/docs/adr/0005-workspace-mcp-entries.md

@@ -0,0 +1,39 @@
+# A workspace's MCP entries name their app by a relative path
+
+Status: accepted
+
+`rails-ai-context init` run in a folder of apps (a workspace, see `CONTEXT.md`) writes one server per app into the folder's own MCP configs (#429). Those files may be committed, so each entry has to find its app on any machine, and an AI tool gives a stdio server nothing to find it with but the command line, the environment and the directory it starts the server in.
+
+## What the tools do
+
+Checked against each tool's documentation and source in October 2026:
+
+| Tool | Starts the server in | Names the config's folder |
+|------|----------------------|---------------------------|
+| Claude Code (`.mcp.json`) | the folder it was launched in; it also reads a `.mcp.json` above that folder | no; no `cwd` key either |
+| Codex CLI (`.codex/config.toml`) | its session directory | no; a `cwd` key, itself relative to the session directory |
+| OpenCode (`opencode.json`) | the folder it was opened in | no; a `cwd` key only from v1.17.4, and older versions reject unknown keys |
+| VS Code (`.vscode/mcp.json`) | the workspace folder | `${workspaceFolder}` |
+| Cursor (`.cursor/mcp.json`) | undocumented; users report `/` or `~` | `${workspaceFolder}`, the folder holding `.cursor/` |
+
+## Decision
+
+- `--app-path` and an in-Gemfile app's `BUNDLE_GEMFILE` are written relative to the workspace. Where the tool names its config's folder (Cursor, VS Code: `AiTool`'s `folder_variable`), they start from that name; elsewhere they are bare relative paths, which those tools resolve against the folder they were started in. An absolute path would hold only on the machine that wrote it.
+- Each app keeps its own install form: the bare binary for a standalone app, `bundle exec` for an in-Gemfile one, decided from that app's own Gemfile.lock (or its Gemfile, before the bundle is installed) rather than the process's bundle.
+- A re-run writes the current set: an entry under a name the gem gives an app in the folder (its folder name or its path, never a numbered one) is dropped when that app's folder is gone, or when a twin's arrival renamed it. A folder this machine lacks reads as gone: the gem cannot tell an app removed from one never checked out. An entry counts as the gem's by its name and by running `rails-ai-context serve`, or, under the bare name every version has written, by running a command at all; an entry with no command (an HTTP one), a second entry named by hand for an app, a numbered one (as likely made by hand), one naming its app through a variable the tool expands, and one naming an app outside the folder are left as somebody's own.
+- An in-Gemfile app's `BUNDLE_GEMFILE` names the Gemfile its bundle reads: its own, by the name it has in the app even when it links elsewhere, since Bundler names the lockfile after the path it is given; else the one its `config/boot.rb` names, which is how a monorepo's apps share one bundle; else the one `bundle exec` run inside the app would find. With none of these the entry names none.
+- A config the gem cannot write back faithfully - one JSON does not parse, which VS Code and OpenCode accept with trailing commas, one whose comments a rewrite would drop, or one that is not UTF-8 - is left as it is, with the entry to add by hand, said where the install speaks. A folder's config is the likelier to be kept by hand.
+- Each entry is `rails-ai-context-<app folder>` and announces `<app folder>-rails-ai-context`, through the `RAILS_AI_CONTEXT_SERVER_NAME` environment variable rather than a flag: an app whose bundle pins an older gem ignores an unknown variable and starts, where an unknown flag would stop it. The variable only takes the place of the default name, so a `server_name` the app configured wins. VS Code names every tool after the announced name, sanitised and cut to 13 characters, so a name that starts with the gem's would give every app's tools the same `mcp_rails-ai-cont_` prefix and leave the model unable to tell them apart. Names are capped at 30 characters (a hash keeps a longer one unique): Cursor drops a tool whose server and tool names together pass 60, Codex up to v0.100 capped `mcp__<server>__<tool>` at 64, and the longest built-in tool name is 27.
+
+## Consequences
+
+The entries work when the tool is started in the workspace folder, which is the setup they are written for. A tool started inside one of the apps that still reads the workspace's config (Claude Code, Codex, OpenCode) starts each server there, where a relative path misses. The binary then names the folder the path was written for instead of guessing a different app; an in-Gemfile entry fails earlier, in Bundler, before the gem runs. An app opened on its own gets its own config from `init` run inside it.
+
+Every server in a workspace runs under the Ruby the tool resolves in the workspace folder, which a version manager picks by that folder, not by the app. `init` warns when an app declares another; a standalone app that cannot boot under it answers from the static tier, and an in-Gemfile entry may not start.
+
+## Alternatives not taken
+
+- **Absolute paths**: correct on one machine, wrong in every clone.
+- **A `cwd` key per entry**: Claude Code has none, OpenCode's breaks its older versions, and Codex's is relative to the same directory a bare path already is.
+- **Walking up for a relative `--app-path` that misses**: it would make the binary's `--app-path` mean different things in different directories, and would still leave `bundle exec` entries failing before the gem runs.
+- **Re-launching an in-Gemfile app's server from inside the app** (the global binary resolving the app, then `exec`ing `bundle exec` there): it fixes the Ruby choice, but needs the binary installed outside every bundle, cannot `exec` in place on Windows, and fights Codex's environment snapshot, which pins the Ruby `init` ran under.