ragx-cli 0.6.0__tar.gz → 0.8.0__tar.gz

{ragx_cli-0.6.0 → ragx_cli-0.8.0}/CHANGELOG.md

@@ -4,6 +4,73 @@ All notable changes to `ragx-cli` are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [0.8.0] — 2026-10-20
+
+### Changed
+
+- **`query` defaults to vector search + graph traversal.** Query expansion and
+  cross-encoder rerank are now opt-in via `--expand` and `--rerank` (`QueryOptions`
+  defaults `expand=False`, `rerank=False`). A default query makes no LLM call and loads
+  no reranker. `--no-graph` is unchanged. `--no-expand` and `--no-rerank` still parse
+  as deprecated no-ops with a stderr warning. For the previous behaviour pass
+  `--expand --rerank`.
+- **Default corpus globs index source code and text docs only.** `corpus.include` lists
+  ~70 source extensions, `Dockerfile`, `Makefile`, and `*.md`/`*.mdx`/`*.rst`/`*.txt`/
+  `*.adoc`/`*.org`; `corpus.exclude` skips lockfiles, `requirements*.txt`,
+  `constraints*.txt`, licenses, minified files, source maps, and `dist/`, `build/`,
+  `target/`, `coverage/`, `vendor/`. Applies to new `ragx-cli init` configs; an existing
+  `ragx.toml` keeps its lists.
+
+- **Query results are one entry per file (`ragx.query.v2`, breaking).** A file whose
+  chunks would appear several times in the ranked list becomes one entry with up to 3
+  sub-entries; its score is their sum, so it ranks higher. Only chunks within the visible
+  window (ranked chunks until `--top` distinct files are seen) count. JSON results are
+  `{file, score, chunks: [...]}`; chunks no longer repeat `file`. `--files-only`
+  (`ragx.files.v1`) keeps its shape and now returns the same entries. On the notes eval
+  corpus: baseline MRR .646 → .665, graph config r@5 .808 → .923 and MRR .432 → .653.
+- **`--top` counts result entries (files), default 20** (`query.top`, was 8 chunks).
+  Existing `ragx.toml` files keep their own `query.top`. `ragx-cli eval` no longer
+  over-fetches `top × 3` chunks.
+
+### Added
+
+- `ragx-cli init` prints a note that an AI agent can tailor `corpus.include`/`exclude`
+  to the project and add a CLAUDE.md/AGENTS.md section on using `ragx-cli query`.
+
+## [0.7.0] — 2026-09-17
+
+### Added
+
+- **`ragx-cli doctor`**: runs the configured providers for real — embeds a probe string,
+  scores a probe pair with the reranker, and checks the index manifest and corpus drift —
+  then reports every stage with an actionable hint. Starts whatever ragx manages, exactly
+  as `index`/`query` would. Exit `0` healthy, `1` a check failed, `2` doctor could not run
+  (no corpus, unreadable config); warnings (no index yet, drift, a model the server does
+  not list but may JIT-load) do not fail the run. `--json` emits `ragx.doctor.v1` with a
+  `status`/`detail`/`hint`/`elapsed_ms` entry per check.
+- **LM Studio autostart**: with `provider = "openai"` and a local `base_url`, `index`,
+  `query`, and `doctor` now run `lms server start` and `lms load <model>` before their
+  first request, so a cold machine needs no manual setup. ragx already spawned the
+  `llama-server` engines; LM Studio is an app ragx hands off to, so it is started but
+  never stopped. Only local URLs are touched (a cloud endpoint, including one set via
+  `OPENAI_BASE_URL`, is never started), an already-running server is left alone, and a
+  model that is not downloaded fails loud naming the models the machine does have.
+  Opt out with `embeddings.autostart` / `expansion.autostart` (default `true`, settable
+  machine-wide via `ragx-cli config set --global`).
+
+### Fixed
+
+- **`llama-server` returned 500 on every real query.** The server was spawned without
+  `-b`/`-ub`/`-c`, leaving llama.cpp's default 512-token physical batch in place; a rerank
+  or embedding input is non-causal and cannot be split across ubatches, so ragx's
+  ~800-token chunks overflowed it ("input is too large to process"). Two-document smoke
+  tests passed, which is why it went unnoticed. Now spawned with
+  `--parallel 1 -c 0 -b 8192 -ub 8192` — one slot with the model's full trained context.
+  Fixes the `llama-server` embedding engine at default chunk size as well as rerank.
+- **`ragx-cli models --quality balanced` could not download BGE-M3.** `gaianet/bge-m3-GGUF`
+  now answers 401 from the Hugging Face API; both refs point at `gpustack/bge-m3-GGUF@Q8_0`
+  (same model, already the catalog's reranker source, pinned to the benchmarked quant).
+
 ## [0.6.0] — 2026-07-15
 
 ### Changed

{ragx_cli-0.6.0 → ragx_cli-0.8.0}/CLAUDE.md

@@ -27,6 +27,11 @@ API contracts for every core module. Deferred feature specs live in `docs/`.
   with the llama-server rerank engine, LM Studio is downloader-only at query time.
   Env fallbacks: `OPENAI_BASE_URL` (only while base_url is still the default) and
   `OPENAI_API_KEY` (only when `api_key_env` is unset); explicit config always wins.
+  With `provider="openai"` and a LOCAL base_url, `index`/`query`/`doctor` autostart LM
+  Studio first (`providers/lmstudio_process.py`: `lms server start` + `lms load` when the
+  model isn't in `lms ps`; never stops it — it's an app, not our child). Remote URLs and
+  `<section>.autostart=false` skip it. Tests are disarmed by an autouse conftest fixture
+  that stubs `ensure_for_section` — never let a test reach the real `lms`.
 - Machine-level provider settings live in `~/.ragxrc` (TOML; ONLY the embeddings/
   expansion/rerank sections, unknown keys fail loud). Precedence: DEFAULTS < corpus
   ragx.toml < ~/.ragxrc — the rc OVERRIDES corpus values and logs a stderr warning
@@ -63,22 +68,26 @@ src/ragx/
     indexer.py     #   run_index: discover->hash->chunk->embed->HNSW+store->kNN edges
     query.py       #   run_query: expansion->fan-out->RRF->traversal->rerank->combine; JSON serializers
     eval.py        #   recall@5/@10 + MRR over file-level ranking, injected query_fn
+    doctor.py      #   run_doctor: config/embeddings/expansion/rerank/index health checks
+                   #   (injected provider factories; backends get started here too)
     catalog.py     #   curated embedding/reranker catalog (tiers, prefixes) + spec detect
     lmstudio.py    #   `lms` CLI wrapper: find/download (`lms get --yes`)/`lms ls --json`;
                    #   LM Studio can't download plain safetensors — rerankers stay on HF
   providers/       # base.py protocols (Embedder/Generator/Reranker); openai_compat.py;
-                   # st_reranker.py; llama_process.py (managed llama-server lifecycle) +
+                   # st_reranker.py; lmstudio_process.py (LM Studio autostart);
+                   # llama_process.py (managed llama-server lifecycle) +
                    # llama_server.py (rerank) + llama_embedder.py (embeddings);
                    # registry.py factories (env-var + api_key_env resolution)
   cli/             # thin shells only: app.py (init/status/config + registration),
                    # pipeline.py (index/query), inspect_cmd.py, eval_cmd.py, models_cmd.py
-                   # (recommend/download/configure models), output.py
-tests/             # 177 tests; mocked HTTP (respx), FakeEmbedder integration tests, no live network
+                   # (recommend/download/configure models), doctor_cmd.py, output.py
+tests/             # 253 tests; mocked HTTP (respx), FakeEmbedder integration tests, no live network
 ```
 
 ## Flows
 
-**Index** (`ragx-cli index [--changed]`): discover files (include/exclude globs, root .gitignore,
+**Index** (`ragx-cli index [--changed]`): discover files (include/exclude globs — defaults in
+`core/corpus_globs.py`: source + text docs, minus lockfiles/requirements/build output — root .gitignore,
 skip node_modules/.venv/hidden/binaries/>2MB) → xxhash diff (`--changed` = incremental, default
 = full rebuild) → chunk (~800 tok target, chunks are exact byte slices, sub-100-char fragments
 merge into neighbors) → embed batched → HNSW add → kNN edges per new chunk (k=8, cos ≥ 0.55)
@@ -108,8 +117,13 @@ no-op on failure) → embed all variants, HNSW top-20 each → RRF merge = seeds
 (2 hops, decay 0.5, heat = max not sum, neighbor admitted only if cos(query) ≥ 0.35 floor,
 frontier ≤ 150) → cross-encoder rerank of top-100 shortlist → final =
 0.6·rerank + 0.25·heat + 0.15·vector (each min-max normalized; weights renormalize when a
-stage is off). `--no-expand/--no-graph/--no-rerank` skip stages; `--files-only` ranks files
-by sum of top-3 chunk scores; `--explain` attaches seed→parent→edge_weight→hop traces.
+stage is off). Since 0.8.0 expansion and rerank are OPT-IN (`--expand`/`--rerank`;
+`QueryOptions` defaults expand=False, rerank=False; `--no-expand/--no-rerank` are hidden
+deprecated no-ops); graph stays on, `--no-graph` skips it. Output is folded per file
+(`core/grouping.py`, schema `ragx.query.v2`): walk ranked chunks until `top` (default 20)
+distinct files are seen, each file keeps its ≤3 chunks inside that window, entry score = their
+sum. The window is load-bearing — summing ALL candidates let weak graph-neighbour chunks pad
+files (graph MRR .43 → .38; windowed .65). `--files-only` = same entries, no text; `--explain` attaches seed→parent→edge_weight→hop traces.
 
 ## State & benchmarks (as of 2026-07-10)
 
@@ -121,7 +135,8 @@ full .759/**.889**/**.613**. Key finding: graph-only HURTS precision (near-dupli
 displace direct hits — parameter sweeps plateau below baseline MRR; corpus property, not a
 tuning miss), but graph+rerank together deliver the recall win — one Dutch query's target is
 unreachable by vector search or rerank-alone, surfaced only via a hop-1 edge then reranked
-19→4. Ship graph+rerank together; `--no-graph --no-rerank` is the explicit fast mode.
+19→4. Graph+rerank belong together for quality (`--rerank`); the 0.8.0 default (graph, no
+rerank) is the measured "graph-only" row — chosen for speed, not quality.
 
 UPDATE 2026-07-13: production embedding model is now `text-embedding-bge-m3` (Q8 GGUF, empty
 doc/query prefixes) — notes-repo corpus config + index flipped in place from the worktree-built
@@ -132,6 +147,10 @@ mostly on Dutch/multilingual queries. Details: `research/bge-m3-dense-q8-vs-nomi
 
 ## Gotchas
 
+- llama-server spawns MUST pass `--parallel 1 -c 0 -b/-ub 8192` (`llama_process.py`):
+  llama.cpp's default 512-token physical batch cannot hold a ~800-token chunk, and a
+  non-causal rerank/embedding input can't split across ubatches — without them every real
+  query 500s ("input is too large to process") while a 2-doc smoke test passes.
 - Reasoning models as expansion LLM: answers land in `content` only after thinking;
   expansion uses max_tokens=4096 to leave room for that. Don't lower it.
 - Changing `embeddings.model` invalidates the index; run_query fails loud on manifest
@@ -158,7 +177,7 @@ mostly on Dutch/multilingual queries. Details: `research/bge-m3-dense-q8-vs-nomi
 
 ## Dev loop
 
-`uv sync --group dev --extra rerank` · `uv run pytest -q` (177 pass, ~5 s) ·
+`uv sync --group dev --extra rerank` · `uv run pytest -q` (253 pass, ~5 s) ·
 `uv run ruff check src tests` · file soft cap ~150 lines · expected failures raise `RagxError`
 (CLI maps to exit 2). Live smoke: LM Studio must be running with the configured embedding model.
 Changes that impact usage (CLI flags, config keys/precedence, output schemas, install steps)

{ragx_cli-0.6.0 → ragx_cli-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: ragx-cli
-Version: 0.6.0
+Version: 0.8.0
 Summary: Similarity-graph RAG CLI: index a corpus into a chunk-level embedding graph and query it
 Project-URL: Repository, https://github.com/alber70g/ragx-cli
 Project-URL: Issues, https://github.com/alber70g/ragx-cli/issues
@@ -53,6 +53,7 @@ ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph (
 ragx-cli query "why did we switch build tools?" --json --files-only
 ragx-cli index --full          # full rebuild: re-chunk + re-embed everything
 ragx-cli models                # recommend + download an embedding/reranker combo via LM Studio
+ragx-cli doctor                # check the configured providers start and answer, and the index matches
 ragx-cli --version             # version, repo, and the effective models (corpus config + ~/.ragxrc overrides)
 ```
 
@@ -70,6 +71,7 @@ your files (gitignore it) — like `.git/`, delete `.ragx/` and the corpus is un
   - [Does it actually help? (benchmarks)](#does-it-actually-help-benchmarks)
     - [Parameter study: what each knob actually does (2026-07)](#parameter-study-what-each-knob-actually-does-2026-07)
     - [BGE-M3 embedding study (2026-07-12)](#bge-m3-embedding-study-2026-07-12)
+  - [Checking your setup: `ragx-cli doctor`](#checking-your-setup-ragx-cli-doctor)
   - [Agent-first conventions](#agent-first-conventions)
   - [Using ragx-cli from a coding agent (CLAUDE.md / AGENTS.md)](#using-ragx-cli-from-a-coding-agent-claudemd--agentsmd)
     - [Pointing ragx-cli at your LLM — local or online](#pointing-ragx-cli-at-your-llm--local-or-online)
@@ -128,20 +130,30 @@ since the last index) so you can see when a re-index is due without running one.
 
 ### Querying
 
-Every stage is individually skippable (`--no-expand`, `--no-graph`, `--no-rerank`) so callers
-can trade quality for latency.
+A default query runs vector search + graph traversal: no LLM call, no cross-encoder.
+Expansion (`--expand`) and rerank (`--rerank`) are opt-in; `--no-graph` skips traversal,
+which leaves pure vector search. Callers trade latency for quality per query.
+
+Results are **one entry per file**. The pipeline ranks chunks, walks that list until it has
+seen `--top` distinct files (default `query.top = 20`), and folds each file's chunks inside
+that window into one entry: up to 3 sub-entries, entry score = their sum. A file that would
+appear several times in a flat list therefore ranks higher. Chunks below the window never
+count, so graph traversal's weak neighbour chunks cannot pad a file (measured on the notes
+eval corpus: graph-config MRR .43 flat-list, .38 summing all chunks, .65 windowed).
 
 ```mermaid
 flowchart TD
     Q[query] --> X["1 · expansion (optional, one LLM call)<br/>2–4 reformulations + HyDE passage"]
     X --> S["2 · fan-out vector search<br/>top-20 per variant"]
-    Q -. "--no-expand" .-> S
+    Q -. "default (no --expand)" .-> S
     S --> R["3 · Reciprocal Rank Fusion<br/>merged seed set"]
     R --> H["4 · heat propagation over the graph<br/>2 hops · decay 0.5 · max-aggregation<br/>query-similarity floor · frontier cap"]
     R -. "--no-graph" .-> K
-    H --> K["5 · cross-encoder rerank<br/>(query, chunk) pairs, capped shortlist"]
+    H --> K["5 · cross-encoder rerank (--rerank)<br/>(query, chunk) pairs, capped shortlist"]
+    H -. "default (no --rerank)" .-> F
     K --> F["6 · combined score<br/>α·rerank + β·heat + γ·vector"]
-    F --> O["ranked chunks (or files via --files-only)<br/>+ --explain traversal trace"]
+    F --> G["7 · fold per file<br/>a file's best ≤3 chunks → one entry, score = their sum"]
+    G --> O["top 20 entries (or paths via --files-only)<br/>+ --explain traversal trace"]
 ```
 
 **Heat propagation** is what sets ragx-cli apart from plain RAG: seed chunks (from vector search)
@@ -208,8 +220,10 @@ output for that result shows the exact seed → edge → chunk path.
 **Honest caveat:** graph traversal *alone* hurts precision on this corpus (MRR 0.593 → 0.522) —
 near-duplicate neighbors (e.g. adjacent meeting notes) displace weaker direct hits. A parameter
 sweep over decay/floor/weights plateaued below baseline MRR, so this is a property of
-similarity-only edges, not a tuning miss. Conclusion baked into the defaults: **graph and
-rerank ship together**. Use `--no-graph --no-rerank` as the explicit fast mode.
+similarity-only edges, not a tuning miss. Conclusion: **graph and rerank belong together**.
+Since 0.8.0 the CLI default is the fast path (vector + graph, no rerank), which is the
+"graph-only" row above; pass `--rerank` for the measured best quality, or `--no-graph` for
+pure vector search.
 
 The parameter study below explains the mechanism behind this caveat — and shows that once the
 scoring weights stop putting heat into the final score, the graph win gets much bigger.
@@ -381,16 +395,69 @@ Why not `jina-embeddings-v5-text-nano`? Excellent model, but its EuroBERT archit
 Jina's llama.cpp fork — stock LM Studio can't load it (and the license is CC-BY-NC). See
 `research/lm-studio-model-download-api-and-curated-embedding-reranker-catalog-for-ragx-models-command.md`.
 
+## Checking your setup: `ragx-cli doctor`
+
+`ragx-cli doctor` runs the configured pipeline's providers for real — it embeds a probe string,
+scores a probe pair with the reranker, and checks the index on disk against the config — then
+reports each stage:
+
+```
+✓ config      /corpus/ragx.toml — embeddings=openai/text-embedding-bge-m3 · expansion=openai/qwen3.5-9b · rerank=sentence-transformers/BAAI/bge-reranker-v2-m3
+✗ embeddings  request to http://localhost:1234/v1/embeddings failed after 3 attempts: [Errno 61] Connection refused
+  → ragx does not start this server — launch it at http://localhost:1234/v1 (LM Studio: `lms server start`),
+    or hand the process to ragx with `ragx-cli models --embed-engine llama-server`
+✗ expansion   no server answered at http://localhost:1234/v1/models
+  → turn it off with `ragx-cli config set expansion.enabled false`
+✓ rerank      sentence-transformers/BAAI/bge-reranker-v2-m3 scored a probe pair (-8.14, 4.72)  3.1s
+✓ index       634 files, 1247 chunks, built with 'text-embedding-bge-m3', no drift
+```
+
+Exit codes: `0` healthy, `1` at least one check failed, `2` doctor itself could not run (no
+corpus, unreadable config). Warnings (no index yet, corpus drift, a model the server does not
+list but may JIT-load) do not fail the run. `--json` emits a `ragx.doctor.v1` document with a
+`status`/`detail`/`hint`/`elapsed_ms` entry per check.
+
+**Which servers does ragx start?** All of the local ones:
+
+| engine (`ragx-cli models`) | config | lifecycle |
+| --- | --- | --- |
+| `llama-server` | `provider="llama-server"` (embeddings and/or rerank) | spawned on demand from the configured GGUF and **killed at exit** — ragx owns the process |
+| `lm-studio` | `embeddings.provider="openai"` + a local `base_url` | `lms server start` + `lms load <model>` when nothing answers — LM Studio **keeps running** afterwards (it is an app, not ragx's child) |
+| `sentence-transformers` | `rerank.provider="sentence-transformers"` | in-process, no server |
+
+`index`, `query`, and `doctor` all do this before their first request, so a cold machine needs
+no manual setup. What was started is logged to stderr and shown in doctor's report:
+
+```
+✓ embeddings  started the LM Studio server on port 1234; loaded 'text-embedding-nomic-embed-text-v1.5' into LM Studio — openai/text-embedding-nomic-embed-text-v1.5 answered, dim=768  14.8s
+```
+
+Rules it follows: only a **local** `base_url` is ever touched (a cloud endpoint, including one
+set via `OPENAI_BASE_URL`, is never ragx's to start); an already-running server is left alone,
+and if LM Studio isn't installed but something else answers on that port, ragx just uses it; a
+model that isn't downloaded fails loud naming the models the machine does have. Turn it off per
+section with `ragx-cli config set embeddings.autostart false` (or machine-wide via
+`--global`) — then ragx only ever talks to servers you started.
+
+`ragx-cli models` still only downloads models and writes `ragx.toml`; it never launches
+anything. If a run dies with "connection refused", `doctor` names the stage and the fix.
+
 ## Agent-first conventions
 
-- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v1`,
-  `ragx.files.v1`, `ragx.status.v1`, `ragx.eval.v1`, `ragx.inspect.*.v1`); logs go to stderr.
+- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v2`,
+  `ragx.files.v1`, `ragx.status.v1`, `ragx.doctor.v1`, `ragx.eval.v1`, `ragx.inspect.*.v1`); logs go
+  to stderr.
 - Exit codes: `0` results, `1` success-but-empty, `2` error.
-- Every chunk carries `file`, `line_start/line_end`, `byte_start/byte_end` — agents jump to the
-  exact source location and read the full text themselves (JSON chunk text is truncated).
-- `--files-only` aggregates chunk scores per file (sum of top-3) — the mode coding agents use most.
+- `ragx.query.v2` results are one entry per file: `{file, score, chunks: [...]}`. Each chunk
+  carries `chunk_id`, `line_start/line_end`, `byte_start/byte_end`, `score`, `breakdown`, `text`,
+  `truncated` — agents jump to the exact source location and read the full text themselves
+  (JSON chunk text is truncated). v1 (one flat entry per chunk) is gone since 0.8.0.
+- `--files-only` returns the same entries without text (`file`, `score`, `chunk_ids`) — the
+  mode coding agents use most.
 - `ragx-cli status` includes a `drift` object (`new`/`changed`/`deleted` file counts vs the index) —
   agents check it to decide whether to run `ragx-cli index` before querying.
+- `ragx-cli doctor --json` verifies the providers answer before a batch of queries (`ok: false`
+  plus a per-check `hint` when something is down); exit `1` = a check failed.
 - `ragx-cli query -` reads the query from stdin; `ragx-cli inspect chunk|file|neighbors|communities|community`
   debugs the graph.
 
@@ -406,10 +473,11 @@ This repo has a ragx-cli index (`.ragx/`). Prefer it over grep for "where is X d
 questions; fall back to grep for exact identifiers.
 
 - Find relevant files: `ragx-cli query "<natural-language question>" --json --files-only`
-- Get chunks with exact locations: `ragx-cli query "..." --json --top 8` — each result carries
-  `file` + `line_start/line_end`; the JSON `text` is truncated, so read the file yourself
-  for full context.
-- Fast mode (no LLM call, no cross-encoder): add `--no-expand --no-rerank`.
+- Get chunks with exact locations: `ragx-cli query "..." --json` — one result per `file`
+  (default 20, `--top N`), each with up to 3 `chunks` carrying `line_start/line_end`; the JSON
+  `text` is truncated, so read the file yourself for full context.
+- The default query makes no LLM call and loads no cross-encoder. For higher precision add
+  `--rerank`; add `--expand` for an LLM query-expansion pass (one extra LLM call).
 - After adding or editing files: `ragx-cli index` (incremental and cheap — hash-based).
   `ragx-cli status --json` shows `drift` counts if you want to check staleness first.
 - stdout is exactly one JSON document; logs are on stderr.
@@ -559,18 +627,23 @@ non-thinking model is the better expansion choice; detection is a best-effort na
 heuristic, the APIs expose no capability flag), and corpus include/exclude
 (gitignore-style globs, comma-separated). Every prompt is pre-filled with the defaults
 below — Enter accepts them all. With piped stdin (agents), `--yes`, or
-`--no-interactive`, it writes the defaults unchanged, exactly as before. Key defaults:
+`--no-interactive`, it writes the defaults unchanged, exactly as before. After writing the
+config, `init` prints a note: an AI agent can tailor `corpus.include`/`exclude` to the
+project, and add a CLAUDE.md/AGENTS.md section on searching it with `ragx-cli query`
+(see [Using ragx-cli from a coding agent](#using-ragx-cli-from-a-coding-agent-claudemd--agentsmd)).
+Key defaults:
 
 | section | defaults |
 |---|---|
+| `[corpus]` | `include` = source code (`*.py`, `*.ts`, `*.go`, `*.rs`, … ~70 extensions, plus `Dockerfile`/`Makefile`) and text docs (`*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.adoc`, `*.org`); `exclude` = lockfiles (`*.lock`, `package-lock.json`, `pnpm-lock.yaml`, `go.sum`, …), `requirements*.txt`/`constraints*.txt`, `LICENSE*`/`COPYING*`, `*.min.js`/`*.min.css`/`*.map`, and `dist/`, `build/`, `target/`, `coverage/`, `vendor/`; `respect_gitignore=true`. Full lists: `src/ragx/core/corpus_globs.py`. Existing `ragx.toml` files keep their own lists |
 | `[chunking]` | `size_tokens=800`, `overlap=0.15` |
 | `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="chunk"`, `subchunk_size_tokens=128`, `near_dup_sim=0.9` |
 | `[traversal]` | `hops=2`, `decay=0.5`, `query_floor=0.35`, `max_frontier=150` |
 | `[communities]` | `resolution=1.0`, `seed=42` — recomputed every index run; changing these never invalidates the index |
 | `[fusion]` | `rrf_k=60`, `per_query_top=20` |
 | `[scoring]` | `alpha_rerank=0.6`, `beta_heat=0.25`, `gamma_vector=0.15` |
-| `[embeddings]` | `provider="openai"`, `base_url="http://localhost:1234/v1"`, prefixes for nomic-style models, `api_key_env=""` |
-| `[expansion]` | optional LLM for multi-query/HyDE; reasoning models supported (4096-token budget); `api_key_env=""` |
+| `[embeddings]` | `provider="openai"`, `base_url="http://localhost:1234/v1"`, prefixes for nomic-style models, `api_key_env=""`, `autostart=true` (start LM Studio + load the model when the URL is local) |
+| `[expansion]` | optional LLM for multi-query/HyDE; reasoning models supported (4096-token budget); `api_key_env=""`, `autostart=true` |
 | `[rerank]` | `BAAI/bge-reranker-v2-m3` via sentence-transformers (`uv tool install 'ragx-cli[rerank]'`) |
 
 ## Features & roadmap

{ragx_cli-0.6.0 → ragx_cli-0.8.0}/README.md

@@ -25,6 +25,7 @@ ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph (
 ragx-cli query "why did we switch build tools?" --json --files-only
 ragx-cli index --full          # full rebuild: re-chunk + re-embed everything
 ragx-cli models                # recommend + download an embedding/reranker combo via LM Studio
+ragx-cli doctor                # check the configured providers start and answer, and the index matches
 ragx-cli --version             # version, repo, and the effective models (corpus config + ~/.ragxrc overrides)
 ```
 
@@ -42,6 +43,7 @@ your files (gitignore it) — like `.git/`, delete `.ragx/` and the corpus is un
   - [Does it actually help? (benchmarks)](#does-it-actually-help-benchmarks)
     - [Parameter study: what each knob actually does (2026-07)](#parameter-study-what-each-knob-actually-does-2026-07)
     - [BGE-M3 embedding study (2026-07-12)](#bge-m3-embedding-study-2026-07-12)
+  - [Checking your setup: `ragx-cli doctor`](#checking-your-setup-ragx-cli-doctor)
   - [Agent-first conventions](#agent-first-conventions)
   - [Using ragx-cli from a coding agent (CLAUDE.md / AGENTS.md)](#using-ragx-cli-from-a-coding-agent-claudemd--agentsmd)
     - [Pointing ragx-cli at your LLM — local or online](#pointing-ragx-cli-at-your-llm--local-or-online)
@@ -100,20 +102,30 @@ since the last index) so you can see when a re-index is due without running one.
 
 ### Querying
 
-Every stage is individually skippable (`--no-expand`, `--no-graph`, `--no-rerank`) so callers
-can trade quality for latency.
+A default query runs vector search + graph traversal: no LLM call, no cross-encoder.
+Expansion (`--expand`) and rerank (`--rerank`) are opt-in; `--no-graph` skips traversal,
+which leaves pure vector search. Callers trade latency for quality per query.
+
+Results are **one entry per file**. The pipeline ranks chunks, walks that list until it has
+seen `--top` distinct files (default `query.top = 20`), and folds each file's chunks inside
+that window into one entry: up to 3 sub-entries, entry score = their sum. A file that would
+appear several times in a flat list therefore ranks higher. Chunks below the window never
+count, so graph traversal's weak neighbour chunks cannot pad a file (measured on the notes
+eval corpus: graph-config MRR .43 flat-list, .38 summing all chunks, .65 windowed).
 
 ```mermaid
 flowchart TD
     Q[query] --> X["1 · expansion (optional, one LLM call)<br/>2–4 reformulations + HyDE passage"]
     X --> S["2 · fan-out vector search<br/>top-20 per variant"]
-    Q -. "--no-expand" .-> S
+    Q -. "default (no --expand)" .-> S
     S --> R["3 · Reciprocal Rank Fusion<br/>merged seed set"]
     R --> H["4 · heat propagation over the graph<br/>2 hops · decay 0.5 · max-aggregation<br/>query-similarity floor · frontier cap"]
     R -. "--no-graph" .-> K
-    H --> K["5 · cross-encoder rerank<br/>(query, chunk) pairs, capped shortlist"]
+    H --> K["5 · cross-encoder rerank (--rerank)<br/>(query, chunk) pairs, capped shortlist"]
+    H -. "default (no --rerank)" .-> F
     K --> F["6 · combined score<br/>α·rerank + β·heat + γ·vector"]
-    F --> O["ranked chunks (or files via --files-only)<br/>+ --explain traversal trace"]
+    F --> G["7 · fold per file<br/>a file's best ≤3 chunks → one entry, score = their sum"]
+    G --> O["top 20 entries (or paths via --files-only)<br/>+ --explain traversal trace"]
 ```
 
 **Heat propagation** is what sets ragx-cli apart from plain RAG: seed chunks (from vector search)
@@ -180,8 +192,10 @@ output for that result shows the exact seed → edge → chunk path.
 **Honest caveat:** graph traversal *alone* hurts precision on this corpus (MRR 0.593 → 0.522) —
 near-duplicate neighbors (e.g. adjacent meeting notes) displace weaker direct hits. A parameter
 sweep over decay/floor/weights plateaued below baseline MRR, so this is a property of
-similarity-only edges, not a tuning miss. Conclusion baked into the defaults: **graph and
-rerank ship together**. Use `--no-graph --no-rerank` as the explicit fast mode.
+similarity-only edges, not a tuning miss. Conclusion: **graph and rerank belong together**.
+Since 0.8.0 the CLI default is the fast path (vector + graph, no rerank), which is the
+"graph-only" row above; pass `--rerank` for the measured best quality, or `--no-graph` for
+pure vector search.
 
 The parameter study below explains the mechanism behind this caveat — and shows that once the
 scoring weights stop putting heat into the final score, the graph win gets much bigger.
@@ -353,16 +367,69 @@ Why not `jina-embeddings-v5-text-nano`? Excellent model, but its EuroBERT archit
 Jina's llama.cpp fork — stock LM Studio can't load it (and the license is CC-BY-NC). See
 `research/lm-studio-model-download-api-and-curated-embedding-reranker-catalog-for-ragx-models-command.md`.
 
+## Checking your setup: `ragx-cli doctor`
+
+`ragx-cli doctor` runs the configured pipeline's providers for real — it embeds a probe string,
+scores a probe pair with the reranker, and checks the index on disk against the config — then
+reports each stage:
+
+```
+✓ config      /corpus/ragx.toml — embeddings=openai/text-embedding-bge-m3 · expansion=openai/qwen3.5-9b · rerank=sentence-transformers/BAAI/bge-reranker-v2-m3
+✗ embeddings  request to http://localhost:1234/v1/embeddings failed after 3 attempts: [Errno 61] Connection refused
+  → ragx does not start this server — launch it at http://localhost:1234/v1 (LM Studio: `lms server start`),
+    or hand the process to ragx with `ragx-cli models --embed-engine llama-server`
+✗ expansion   no server answered at http://localhost:1234/v1/models
+  → turn it off with `ragx-cli config set expansion.enabled false`
+✓ rerank      sentence-transformers/BAAI/bge-reranker-v2-m3 scored a probe pair (-8.14, 4.72)  3.1s
+✓ index       634 files, 1247 chunks, built with 'text-embedding-bge-m3', no drift
+```
+
+Exit codes: `0` healthy, `1` at least one check failed, `2` doctor itself could not run (no
+corpus, unreadable config). Warnings (no index yet, corpus drift, a model the server does not
+list but may JIT-load) do not fail the run. `--json` emits a `ragx.doctor.v1` document with a
+`status`/`detail`/`hint`/`elapsed_ms` entry per check.
+
+**Which servers does ragx start?** All of the local ones:
+
+| engine (`ragx-cli models`) | config | lifecycle |
+| --- | --- | --- |
+| `llama-server` | `provider="llama-server"` (embeddings and/or rerank) | spawned on demand from the configured GGUF and **killed at exit** — ragx owns the process |
+| `lm-studio` | `embeddings.provider="openai"` + a local `base_url` | `lms server start` + `lms load <model>` when nothing answers — LM Studio **keeps running** afterwards (it is an app, not ragx's child) |
+| `sentence-transformers` | `rerank.provider="sentence-transformers"` | in-process, no server |
+
+`index`, `query`, and `doctor` all do this before their first request, so a cold machine needs
+no manual setup. What was started is logged to stderr and shown in doctor's report:
+
+```
+✓ embeddings  started the LM Studio server on port 1234; loaded 'text-embedding-nomic-embed-text-v1.5' into LM Studio — openai/text-embedding-nomic-embed-text-v1.5 answered, dim=768  14.8s
+```
+
+Rules it follows: only a **local** `base_url` is ever touched (a cloud endpoint, including one
+set via `OPENAI_BASE_URL`, is never ragx's to start); an already-running server is left alone,
+and if LM Studio isn't installed but something else answers on that port, ragx just uses it; a
+model that isn't downloaded fails loud naming the models the machine does have. Turn it off per
+section with `ragx-cli config set embeddings.autostart false` (or machine-wide via
+`--global`) — then ragx only ever talks to servers you started.
+
+`ragx-cli models` still only downloads models and writes `ragx.toml`; it never launches
+anything. If a run dies with "connection refused", `doctor` names the stage and the fix.
+
 ## Agent-first conventions
 
-- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v1`,
-  `ragx.files.v1`, `ragx.status.v1`, `ragx.eval.v1`, `ragx.inspect.*.v1`); logs go to stderr.
+- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v2`,
+  `ragx.files.v1`, `ragx.status.v1`, `ragx.doctor.v1`, `ragx.eval.v1`, `ragx.inspect.*.v1`); logs go
+  to stderr.
 - Exit codes: `0` results, `1` success-but-empty, `2` error.
-- Every chunk carries `file`, `line_start/line_end`, `byte_start/byte_end` — agents jump to the
-  exact source location and read the full text themselves (JSON chunk text is truncated).
-- `--files-only` aggregates chunk scores per file (sum of top-3) — the mode coding agents use most.
+- `ragx.query.v2` results are one entry per file: `{file, score, chunks: [...]}`. Each chunk
+  carries `chunk_id`, `line_start/line_end`, `byte_start/byte_end`, `score`, `breakdown`, `text`,
+  `truncated` — agents jump to the exact source location and read the full text themselves
+  (JSON chunk text is truncated). v1 (one flat entry per chunk) is gone since 0.8.0.
+- `--files-only` returns the same entries without text (`file`, `score`, `chunk_ids`) — the
+  mode coding agents use most.
 - `ragx-cli status` includes a `drift` object (`new`/`changed`/`deleted` file counts vs the index) —
   agents check it to decide whether to run `ragx-cli index` before querying.
+- `ragx-cli doctor --json` verifies the providers answer before a batch of queries (`ok: false`
+  plus a per-check `hint` when something is down); exit `1` = a check failed.
 - `ragx-cli query -` reads the query from stdin; `ragx-cli inspect chunk|file|neighbors|communities|community`
   debugs the graph.
 
@@ -378,10 +445,11 @@ This repo has a ragx-cli index (`.ragx/`). Prefer it over grep for "where is X d
 questions; fall back to grep for exact identifiers.
 
 - Find relevant files: `ragx-cli query "<natural-language question>" --json --files-only`
-- Get chunks with exact locations: `ragx-cli query "..." --json --top 8` — each result carries
-  `file` + `line_start/line_end`; the JSON `text` is truncated, so read the file yourself
-  for full context.
-- Fast mode (no LLM call, no cross-encoder): add `--no-expand --no-rerank`.
+- Get chunks with exact locations: `ragx-cli query "..." --json` — one result per `file`
+  (default 20, `--top N`), each with up to 3 `chunks` carrying `line_start/line_end`; the JSON
+  `text` is truncated, so read the file yourself for full context.
+- The default query makes no LLM call and loads no cross-encoder. For higher precision add
+  `--rerank`; add `--expand` for an LLM query-expansion pass (one extra LLM call).
 - After adding or editing files: `ragx-cli index` (incremental and cheap — hash-based).
   `ragx-cli status --json` shows `drift` counts if you want to check staleness first.
 - stdout is exactly one JSON document; logs are on stderr.
@@ -531,18 +599,23 @@ non-thinking model is the better expansion choice; detection is a best-effort na
 heuristic, the APIs expose no capability flag), and corpus include/exclude
 (gitignore-style globs, comma-separated). Every prompt is pre-filled with the defaults
 below — Enter accepts them all. With piped stdin (agents), `--yes`, or
-`--no-interactive`, it writes the defaults unchanged, exactly as before. Key defaults:
+`--no-interactive`, it writes the defaults unchanged, exactly as before. After writing the
+config, `init` prints a note: an AI agent can tailor `corpus.include`/`exclude` to the
+project, and add a CLAUDE.md/AGENTS.md section on searching it with `ragx-cli query`
+(see [Using ragx-cli from a coding agent](#using-ragx-cli-from-a-coding-agent-claudemd--agentsmd)).
+Key defaults:
 
 | section | defaults |
 |---|---|
+| `[corpus]` | `include` = source code (`*.py`, `*.ts`, `*.go`, `*.rs`, … ~70 extensions, plus `Dockerfile`/`Makefile`) and text docs (`*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.adoc`, `*.org`); `exclude` = lockfiles (`*.lock`, `package-lock.json`, `pnpm-lock.yaml`, `go.sum`, …), `requirements*.txt`/`constraints*.txt`, `LICENSE*`/`COPYING*`, `*.min.js`/`*.min.css`/`*.map`, and `dist/`, `build/`, `target/`, `coverage/`, `vendor/`; `respect_gitignore=true`. Full lists: `src/ragx/core/corpus_globs.py`. Existing `ragx.toml` files keep their own lists |
 | `[chunking]` | `size_tokens=800`, `overlap=0.15` |
 | `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="chunk"`, `subchunk_size_tokens=128`, `near_dup_sim=0.9` |
 | `[traversal]` | `hops=2`, `decay=0.5`, `query_floor=0.35`, `max_frontier=150` |
 | `[communities]` | `resolution=1.0`, `seed=42` — recomputed every index run; changing these never invalidates the index |
 | `[fusion]` | `rrf_k=60`, `per_query_top=20` |
 | `[scoring]` | `alpha_rerank=0.6`, `beta_heat=0.25`, `gamma_vector=0.15` |
-| `[embeddings]` | `provider="openai"`, `base_url="http://localhost:1234/v1"`, prefixes for nomic-style models, `api_key_env=""` |
-| `[expansion]` | optional LLM for multi-query/HyDE; reasoning models supported (4096-token budget); `api_key_env=""` |
+| `[embeddings]` | `provider="openai"`, `base_url="http://localhost:1234/v1"`, prefixes for nomic-style models, `api_key_env=""`, `autostart=true` (start LM Studio + load the model when the URL is local) |
+| `[expansion]` | optional LLM for multi-query/HyDE; reasoning models supported (4096-token budget); `api_key_env=""`, `autostart=true` |
 | `[rerank]` | `BAAI/bge-reranker-v2-m3` via sentence-transformers (`uv tool install 'ragx-cli[rerank]'`) |
 
 ## Features & roadmap

{ragx_cli-0.6.0 → ragx_cli-0.8.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ragx-cli"
-version = "0.6.0"
+version = "0.8.0"
 description = "Similarity-graph RAG CLI: index a corpus into a chunk-level embedding graph and query it"
 readme = "README.md"
 license = "MIT"
@@ -44,7 +44,7 @@ build-backend = "hatchling.build"
 packages = ["src/ragx"]
 
 [dependency-groups]
-dev = ["pytest>=8", "ruff>=0.6", "respx>=0.21"]
+dev = ["pytest>=8", "ruff>=0.15,<0.16", "respx>=0.21"]
 
 [tool.ruff]
 line-length = 100

{ragx_cli-0.6.0 → ragx_cli-0.8.0}/skills/ragx-cli/SKILL.md

@@ -11,8 +11,9 @@ description: |
 # ragx-cli — semantic search over an indexed corpus
 
 Indexes files into a chunk-level embedding similarity graph; answers queries via vector search +
-graph traversal + cross-encoder rerank. Agent-first: one JSON doc on stdout, logs on stderr,
-deterministic exit codes, byte-exact source locations.
+graph traversal, with optional cross-encoder rerank (`--rerank`) and LLM expansion (`--expand`).
+Agent-first: one JSON doc on stdout, logs on stderr, deterministic exit codes, byte-exact source
+locations.
 
 ## When
 
@@ -29,24 +30,28 @@ A corpus is ragx-indexed if its root has `ragx.toml` and `.ragx/`.
 ```bash
 ragx-cli status --json          # root, model, counts, and drift (new/changed/deleted vs index)
 ragx-cli query "question" --json --files-only   # which files are relevant — most-used mode
-ragx-cli query "..." --json --top 8             # ranked chunks with exact locations
-ragx-cli query "..." --json --no-expand --no-rerank   # fast: skip LLM call + cross-encoder
+ragx-cli query "..." --json --top 10            # one entry per file, best chunks + exact locations
+ragx-cli query "..." --json --rerank            # higher precision: cross-encoder rerank (slower)
 echo "..." | ragx-cli query - --json            # long/multiline query from stdin
 ragx-cli index                  # incremental reindex (hash-based, cheap) — the default
 ragx-cli index --full           # full rebuild: re-chunks + re-embeds everything
 ```
 
-Other query flags: `--no-graph`, `--hops N`, `--explain`.
+Other query flags: `--expand` (one LLM call: multi-query + HyDE), `--no-graph` (pure vector),
+`--hops N`, `--explain`. `--no-expand`/`--no-rerank` are deprecated no-ops (both stages are off
+by default).
 
 ## Output
 
-- stdout is exactly one JSON doc (`ragx.query.v1` / `ragx.files.v1` / `ragx.status.v1`); logs go
+- stdout is exactly one JSON doc (`ragx.query.v2` / `ragx.files.v1` / `ragx.status.v1`); logs go
   to stderr. Never parse stderr.
 - Exit codes: `0` results, `1` success-but-empty (report "nothing found", don't retry as failure),
   `2` error.
-- Chunk results carry `file`, `line_start/end`, `byte_start/end`, `score`, `breakdown`. JSON
-  `text` is truncated — read the file at the line range for full context.
-- Typical flow: `--files-only` to pick files → `--top 8` for locations → read files at ranges.
+- `ragx.query.v2` results are one entry per file (default 20): `file`, `score` (sum of its
+  chunks), `chunks` (up to 3, each with `line_start/end`, `byte_start/end`, `score`,
+  `breakdown`, `text`). JSON `text` is truncated — read the file at the line range for full
+  context.
+- Typical flow: `--files-only` to pick files → `--json` for line ranges → read files at ranges.
 
 ## Freshness