ragx-cli 0.8.0__tar.gz → 0.10.0__tar.gz

{ragx_cli-0.8.0 → ragx_cli-0.10.0}/CHANGELOG.md

@@ -4,6 +4,53 @@ All notable changes to `ragx-cli` are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [0.10.0] — 2026-10-21
+
+### Added
+
+- `ragx-cli index --edges-only` — rebuilds sub-chunks, graph edges and communities from the
+  stored chunks per the current `[graph]` config, without re-chunking or re-embedding files.
+  Lets an existing index switch `graph.edge_source` (or `subchunk_size_tokens`) cheaply: chunk
+  embeddings and the vector index are kept; a chunk that stays whole reuses its own stored
+  vector, so only the sub-chunk pieces a split chunk actually produces go through the embedder.
+  Mutually exclusive with `--full`/`--changed`. The incremental-index manifest mismatch message
+  for `edge_source`/`subchunk_size_tokens` now also suggests `--edges-only`.
+
+### Changed
+
+- `ragx-cli web` no longer requires an index built with `graph.edge_source = "subchunk"`: it
+  renders whatever edges the index has — sub-chunk similarity when the index has them, otherwise
+  the stored whole-chunk cosine edges. An index with zero edges renders as a node-only cloud.
+
+### Fixed
+
+- `ragx-cli web` layout: ForceAtlas2 now uses edge weights (`edgeWeightInfluence = 1`), so more
+  similar chunks sit closer together. Before, every edge pulled equally.
+
+## [0.9.0] — 2026-10-21
+
+### Added
+
+- `ragx-cli web [--host] [--port] [--open/--no-open]` — serves an interactive graph view
+  (nodes/edges/communities) plus a query box over plain `http.server`. Requires an index built
+  with `graph.edge_source = "subchunk"`.
+- Grep-style `-B`/`-A`/`-C` context lines on `ragx-cli query` results (`query.context_lines`,
+  default 5), and `explain.path` (the seed→parent chain) in `--explain` output.
+- `ragx.query.v3` schema (breaking): adds the `context` and `explain.path` fields to each result
+  chunk.
+
+### Changed
+
+- **`graph.edge_source` defaults to `"subchunk"` (was `"chunk"`); `graph.subchunk_size_tokens`
+  defaults to 256 (was 128). Breaking: an index built under the old defaults mismatches the
+  manifest — `ragx-cli index --full` rebuilds it, or pin `graph.edge_source = "chunk"` /
+  `graph.subchunk_size_tokens = 128` in `ragx.toml` to keep it.** A corpus whose `ragx.toml` was
+  written by `ragx-cli init` already pins both keys explicitly and is unaffected; only a
+  hand-written `ragx.toml` missing them picks up the new default. `ragx-cli query` is unaffected
+  either way — it doesn't read sub-chunk data. Measured tie with `"chunk"` on the scoped notes eval corpus;
+  256 chosen as the cheapest size that ties:
+  `research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md`.
+
 ## [0.8.0] — 2026-10-20
 
 ### Changed

{ragx_cli-0.8.0 → ragx_cli-0.10.0}/CLAUDE.md

@@ -73,6 +73,10 @@ src/ragx/
     catalog.py     #   curated embedding/reranker catalog (tiers, prefixes) + spec detect
     lmstudio.py    #   `lms` CLI wrapper: find/download (`lms get --yes`)/`lms ls --json`;
                    #   LM Studio can't download plain safetensors — rerankers stay on HF
+    graph_export.py #  build_graph: ragx.graph.v1 payload for `web`; renders whatever edges the
+                   #   index has, subchunk or chunk-mode cosine (see Flows)
+    reedge.py      #   rebuild_edges: `index --edges-only` — rebuilds sub-chunks/edges/
+                   #   communities from stored chunks, no re-chunk/re-embed (see Flows)
   providers/       # base.py protocols (Embedder/Generator/Reranker); openai_compat.py;
                    # st_reranker.py; lmstudio_process.py (LM Studio autostart);
                    # llama_process.py (managed llama-server lifecycle) +
@@ -80,37 +84,52 @@ src/ragx/
                    # registry.py factories (env-var + api_key_env resolution)
   cli/             # thin shells only: app.py (init/status/config + registration),
                    # pipeline.py (index/query), inspect_cmd.py, eval_cmd.py, models_cmd.py
-                   # (recommend/download/configure models), doctor_cmd.py, output.py
-tests/             # 253 tests; mocked HTTP (respx), FakeEmbedder integration tests, no live network
+                   # (recommend/download/configure models), doctor_cmd.py, web_cmd.py, output.py
+  web/             # second thin shell over ragx.core (MCP server is the planned third):
+                   # server.py (build_server: ThreadingHTTPServer + cached graph bytes),
+                   # handlers.py (routing), query_runner.py (lazy providers, lock-serialized
+                   # queries), static_files.py (path-traversal-safe static serving);
+                   # static/ (frontend-owned: index.html, JS/CSS, vendored sigma.js)
+tests/             # 301 tests; mocked HTTP (respx), FakeEmbedder integration tests, no live network
 ```
 
 ## Flows
 
-**Index** (`ragx-cli index [--changed]`): discover files (include/exclude globs — defaults in
+**Index** (`ragx-cli index`): discover files (include/exclude globs — defaults in
 `core/corpus_globs.py`: source + text docs, minus lockfiles/requirements/build output — root .gitignore,
-skip node_modules/.venv/hidden/binaries/>2MB) → xxhash diff (`--changed` = incremental, default
-= full rebuild) → chunk (~800 tok target, chunks are exact byte slices, sub-100-char fragments
+skip node_modules/.venv/hidden/binaries/>2MB) → xxhash diff (incremental by default since 0.8.0;
+`--full` rebuilds everything; `--changed` is a deprecated no-op alias for the default) → chunk
+(~800 tok target, chunks are exact byte slices, sub-100-char fragments
 merge into neighbors) → embed batched → HNSW add → kNN edges per new chunk (k=8, cos ≥ 0.55)
 + recompute edge lists of pre-existing neighbors that appear in new lists.
-Experimental `graph.edge_source="subchunk"` (default `"chunk"`): chunks additionally split into
-~128-tok sentence-aligned sub-chunks, embedded separately (float32 blobs in the SQLite
+`graph.edge_source="subchunk"` (default since 2026-10-21; `"chunk"` is the opt-out): chunks
+additionally split into ~`subchunk_size_tokens`-tok (default 256) sentence-aligned sub-chunks,
+embedded separately (float32 blobs in the SQLite
 `subchunks` table, cascade-deleted with their chunk; NOT in the query-time HNSW); edge weight =
 max cos over sub-chunk pairs; edges with whole-chunk cos ≥ `graph.near_dup_sim` (0.9) dropped.
 Query side untouched. In subchunk mode `graph.k` means links per SUB-chunk (no per-chunk cap —
-edge budget scales with concept count; graph gets ~2.2x denser). The manifest guards
-`edge_source`/`subchunk_size_tokens` like the embedding model — `--changed` after a flip fails
-loud. Rationale + literature: `research/fine-grained-sub-chunk-edges-*.md`. MEASURED 2026-07-12
-(`.lab` experiments #10–#11): on the scoped notes eval corpus the dense concept graph at hops2
-TIES the tuned chunk-graph baseline (r@10 .923 / MRR .718 vs .717) at ~4.5x index embed cost —
-no win because the corpus's short single-topic chunks have no dilution headroom (near-dup guard
-pruned ~3 edges). hops3 + uncapped budget floods RERANK_CAP (candidates 643). Verdict: default
-stays `edge_source="chunk"`; subchunk is opt-in for corpora with long multi-concept chunks, and
-if used, drop traversal to hops=2. RE-MEASURED 2026-07-13 under BGE-M3: same tie (rerank MRR
-.7548 vs .755) at 2x index time / 2.24x edges — verdict stands across embedding models. That
-ablation also found that with tuned β=0 the pre-rerank shortlist beyond the RRF seeds is filled
-by an ascending-chunk-id tie-break (graph-only candidates all have vector=0), so graph/hops
-changes barely reach the reranker — latent design smell, see
+edge budget scales with concept count; graph gets ~1.4x denser at the 256-tok default). The
+manifest guards `edge_source`/`subchunk_size_tokens` like the embedding model — an incremental
+`index` after a flip fails loud, naming the old/new value; `index --edges-only` (`core/reedge.py`)
+rebuilds just sub-chunks/edges/communities from the stored chunks — no re-chunk/re-embed, reusing
+each whole chunk's stored vector — or `index --full` rebuilds everything, or pin the old value in
+`ragx.toml`. Rationale + literature: `research/fine-grained-sub-chunk-edges-*.md`.
+MEASURED 2026-07-12 (`.lab` experiments #10–#11): on the scoped notes eval corpus the dense
+concept graph at hops2 TIES the tuned chunk-graph baseline (r@10 .923 / MRR .718 vs .717) at
+~4.5x index embed cost (128-tok sub-chunks) — no win because the corpus's short single-topic
+chunks have no dilution headroom (near-dup guard pruned ~3 edges). hops3 + uncapped budget
+floods RERANK_CAP (candidates 643). Verdict at the time: default stays `edge_source="chunk"`;
+subchunk opt-in, drop traversal to hops=2 if used. RE-MEASURED 2026-07-13 under BGE-M3: same tie
+(rerank MRR .7548 vs .755) at 2x index time / 2.24x edges (128-tok) — verdict stands across
+embedding models. That ablation also found that with tuned β=0 the pre-rerank shortlist beyond
+the RRF seeds is filled by an ascending-chunk-id tie-break (graph-only candidates all have
+vector=0), so graph/hops changes barely reach the reranker — latent design smell, see
 `research/bge-m3-subchunk-edge-ablation-2026-07-13-ties-again-and-beta-zero-shortlist-mechanism.md`.
+RE-VERDICT 2026-10-21 (`research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md`):
+sweep of 64/96/128/192/256-tok sub-chunks ties `edge_source="chunk"` at every size on a second
+eval corpus — size choice is driven by cost, not quality. Default flipped to
+`edge_source="subchunk"`, `subchunk_size_tokens=256` (cheapest tying size: fewest edges,
+~2x index time vs ~2.8–4.7x at 128/64-tok).
 
 **Query** (`ragx-cli query "..."`): optional expansion (1 LLM call, strict-JSON parse, graceful
 no-op on failure) → embed all variants, HNSW top-20 each → RRF merge = seeds → heat propagation
@@ -120,10 +139,18 @@ frontier ≤ 150) → cross-encoder rerank of top-100 shortlist → final =
 stage is off). Since 0.8.0 expansion and rerank are OPT-IN (`--expand`/`--rerank`;
 `QueryOptions` defaults expand=False, rerank=False; `--no-expand/--no-rerank` are hidden
 deprecated no-ops); graph stays on, `--no-graph` skips it. Output is folded per file
-(`core/grouping.py`, schema `ragx.query.v2`): walk ranked chunks until `top` (default 20)
+(`core/grouping.py`, schema `ragx.query.v3`): walk ranked chunks until `top` (default 20)
 distinct files are seen, each file keeps its ≤3 chunks inside that window, entry score = their
 sum. The window is load-bearing — summing ALL candidates let weak graph-neighbour chunks pad
 files (graph MRR .43 → .38; windowed .65). `--files-only` = same entries, no text; `--explain` attaches seed→parent→edge_weight→hop traces.
+Each result chunk also carries grep-style `-B`/`-A`/`-C` context lines (`core/context.py`,
+`query.context_lines=5` default, counts non-empty lines only); `--files-only` skips the file
+reads since no text is shown.
+
+**Web** (`ragx-cli web`): builds the `ragx.graph.v1` payload once at startup (`graph_export.py`
+— renders the index's stored edges, sub-chunk or chunk-mode cosine; fails loud only when no
+index exists) and serves it plus `/api/chunk/{id}` and `/api/query` (same pipeline as `query`,
+providers built lazily and cached, one query at a time behind a lock) over stdlib `http.server`.
 
 ## State & benchmarks (as of 2026-07-10)
 
@@ -169,7 +196,8 @@ mostly on Dutch/multilingual queries. Details: `research/bge-m3-dense-q8-vs-nomi
 - Phase 4 (plan §5): Leiden community creation is DONE (index-time, `communities.py`,
   read-only via `status`/`inspect communities`/`inspect community`). Remaining: community
   labels, `query --global` for corpus-level questions.
-- Phase 5: MCP server as a second thin shell over `ragx.core` (split already enforced).
+- Phase 5: MCP server as a third thin shell over `ragx.core` (split already enforced; `web/`
+  is the second, see Layout/Flows).
 - Temporal weighting: full decided spec in `docs/feature-temporal-weighting.md` — opt-in
   `--since/--until/--temporal recent|oldest`, date cascade filename/frontmatter → git → mtime,
   schema v2 migration. Build only after it can be measured against the eval baseline.
@@ -177,7 +205,7 @@ mostly on Dutch/multilingual queries. Details: `research/bge-m3-dense-q8-vs-nomi
 
 ## Dev loop
 
-`uv sync --group dev --extra rerank` · `uv run pytest -q` (253 pass, ~5 s) ·
+`uv sync --group dev --extra rerank` · `uv run pytest -q` (300 pass + 1 skipped, ~9 s) ·
 `uv run ruff check src tests` · file soft cap ~150 lines · expected failures raise `RagxError`
 (CLI maps to exit 2). Live smoke: LM Studio must be running with the configured embedding model.
 Changes that impact usage (CLI flags, config keys/precedence, output schemas, install steps)

{ragx_cli-0.8.0 → ragx_cli-0.10.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ragx-cli
-Version: 0.8.0
+Version: 0.10.0
 Summary: Similarity-graph RAG CLI: index a corpus into a chunk-level embedding graph and query it
 Project-URL: Repository, https://github.com/alber70g/ragx-cli
 Project-URL: Issues, https://github.com/alber70g/ragx-cli/issues
@@ -52,6 +52,7 @@ ragx-cli init                  # create ragx.toml next to your corpus (interacti
 ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph (incremental after the first run)
 ragx-cli query "why did we switch build tools?" --json --files-only
 ragx-cli index --full          # full rebuild: re-chunk + re-embed everything
+ragx-cli index --edges-only    # rebuild sub-chunks/edges/communities only; keeps chunk embeddings
 ragx-cli models                # recommend + download an embedding/reranker combo via LM Studio
 ragx-cli doctor                # check the configured providers start and answer, and the index matches
 ragx-cli --version             # version, repo, and the effective models (corpus config + ~/.ragxrc overrides)
@@ -89,21 +90,28 @@ embedded, and stored in an HNSW index. The similarity graph then falls out almos
 one kNN pass over the vectors that are already in memory — each chunk gets edges to its top-k
 nearest neighbors above a similarity floor.
 
-Experimental: `graph.edge_source = "subchunk"` derives edge weights from sentence-aligned
+`graph.edge_source = "subchunk"` (default) derives edge weights from sentence-aligned
 **sub-chunks** instead of whole-chunk cosine — each chunk is split into ~`subchunk_size_tokens`
-windows, embedded separately (stored in SQLite, never in the query-time HNSW), and the edge
-weight between two chunks becomes the *max* similarity over their sub-chunk pairs, so a chunk
-mixing several concepts gets one sharp edge per concept instead of a diluted average. Edges
+(256) token windows, embedded separately (stored in SQLite, never in the query-time HNSW), and
+the edge weight between two chunks becomes the *max* similarity over their sub-chunk pairs, so a
+chunk mixing several concepts gets one sharp edge per concept instead of a diluted average. Edges
 between near-duplicate chunks (whole-chunk cosine ≥ `near_dup_sim`) are dropped — those are the
-measured precision-killers. Costs roughly 5–10× more embedding calls at index time; retrieval
-units, traversal, and query flow are unchanged. Switching `edge_source` (or the sub-chunk size)
-requires `index --full`; incremental runs fail loud on the mismatch. Literature grounding:
+measured precision-killers. Costs ~2× more embedding calls at index time than `edge_source =
+"chunk"`; retrieval units, traversal, and query flow are unchanged. Switching `edge_source` (or
+the sub-chunk size) requires either `index --edges-only` (cheap: rebuilds sub-chunks, edges and
+communities from the stored chunks — chunk embeddings and the HNSW index are kept) or
+`index --full`; incremental runs fail loud on the mismatch, naming the old/new value and both
+fixes. Literature grounding:
 `research/fine-grained-sub-chunk-edges-with-coarse-chunk-nodes-multi-granularity-graph-rag-literature-validation.md`.
 In this mode `k` counts links per *sub-chunk* with no per-chunk cap, so the graph is denser and
-shallower traversal suffices. Measured on the tuned eval corpus (2026-07): at `hops=2` it ties
-the chunk-edge baseline exactly (same recall, same MRR) at ~4.5× the indexing cost — that
-corpus's short, single-topic chunks leave no concept-dilution headroom to exploit. Worth trying
-only on corpora with long, genuinely multi-concept chunks (use `hops=2`); the default stays `"chunk"`.
+shallower traversal suffices (the default `hops=2` already matches this). Quality ties
+`edge_source="chunk"` exactly at every sub-chunk size tried, 64–256 tokens (2026-07 and
+2026-10-21 sweeps); the corpora measured so far have short, single-topic chunks with no
+concept-dilution headroom to exploit. `256` is the default because it's the cheapest size that
+ties — fewest edges, ~2× index time vs ~2.8–4.7× at 128/64 tokens. Set `graph.edge_source =
+"chunk"` back if a corpus's long, genuinely multi-concept chunks make the extra edges worth the
+cost — no corpus has measured a win yet. Sweep data:
+`research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md`.
 
 After edge construction, a Leiden partition (`graspologic-native`, seeded, deterministic) is
 computed over the whole edge list every index run and stored read-only — see
@@ -124,9 +132,24 @@ flowchart TD
 re-embeds only new/modified files, drops deleted ones, and repairs only the edge lists those
 chunks touch. Content hashes make `touch`-ed but unchanged files free, and include/exclude glob
 changes converge to the same file set a rebuild would produce. `--full` rebuilds from scratch;
-changing `embeddings.model`, `chunking.*`, or `graph.edge_source` requires it — incremental runs
-fail loud on the manifest mismatch. `ragx-cli status` reports **drift** (files new/changed/deleted
-since the last index) so you can see when a re-index is due without running one.
+changing `embeddings.model` or `chunking.*` requires it — incremental runs fail loud on the
+manifest mismatch, naming the changed key and its old/new value. Changing `graph.edge_source`/
+`subchunk_size_tokens` alone is cheaper: `--edges-only` rebuilds just the sub-chunks, edges and
+communities from the chunks already stored — it skips chunking and embedding entirely, reusing
+each chunk's stored vector (via the HNSW index) for chunks that stay whole, and only calls the
+embedder for the sub-chunk pieces a long chunk splits into. It fails loud the same way `--full`
+does if `embeddings.model`/`chunking.*` don't match the stored manifest (those invalidate the
+chunks it reuses), but `edge_source`/`subchunk_size_tokens` are exactly what it's meant to change.
+`ragx-cli status` reports **drift** (files new/changed/deleted since the last index) so you can
+see when a re-index is due without running one.
+
+Upgrading from an older `ragx-cli` after a default change (e.g. the `graph.edge_source` flip to
+`"subchunk"`): `ragx-cli init` always writes every `[graph]` key explicitly into `ragx.toml`
+(`Config.save` bakes the full default set, not just overrides), so an init-created corpus already
+pins its `edge_source`/`subchunk_size_tokens` and is unaffected by a later default change. Only a
+hand-written `ragx.toml` missing those keys picks up the new default; its next incremental
+`index` then fails loud — `index --edges-only` rebuilds just the graph (cheapest), `index --full`
+rebuilds everything, or pin the old value back in `ragx.toml`.
 
 ### Querying
 
@@ -134,6 +157,16 @@ A default query runs vector search + graph traversal: no LLM call, no cross-enco
 Expansion (`--expand`) and rerank (`--rerank`) are opt-in; `--no-graph` skips traversal,
 which leaves pure vector search. Callers trade latency for quality per query.
 
+Every result chunk also comes with grep-style `-B`/`-A`/`-C` context lines, on by default
+(`query.context_lines=5` on each side). `-B/--before N` and `-A/--after N` set each side
+independently; `-C/--context N` sets both; explicit `-B`/`-A` win over `-C`, which wins over
+the default. Counting rule: `N` counts **non-empty** lines walking outward from the chunk —
+blank lines don't count toward `N` but are kept in the returned span if they fall between the
+chunk and the Nth non-empty line; the span stops exactly at that line, so blank lines beyond it
+are excluded. `N=0` disables that side; fewer than `N` non-empty lines in the file returns what
+exists; none returns `null`. Context is read fresh from disk at query time, never truncated by
+`query.max_chunk_chars`, and `--files-only` skips it (no file reads, since the text isn't shown).
+
 Results are **one entry per file**. The pipeline ranks chunks, walks that list until it has
 seen `--top` distinct files (default `query.top = 20`), and folds each file's chunks inside
 that window into one entry: up to 3 sub-entries, entry score = their sum. A file that would
@@ -173,7 +206,10 @@ flowchart LR
 ```
 
 Because every admitted chunk records which seed and edge produced it, `--explain` can print the
-full justification: *seed → edge(weight) → chunk*, per result.
+full justification: *seed → edge(weight) → chunk*, per result. The explain object also carries
+`path`: the seed-first chain of chunk ids along the traversal's own parent links
+(`[seed, ..., chunk_id]`), so a UI can highlight the exact path the traversal took instead of
+guessing a shortest path through the graph.
 
 ---
 
@@ -442,16 +478,51 @@ section with `ragx-cli config set embeddings.autostart false` (or machine-wide v
 `ragx-cli models` still only downloads models and writes `ragx.toml`; it never launches
 anything. If a run dies with "connection refused", `doctor` names the stage and the fix.
 
+## Browsing the graph: `ragx-cli web`
+
+`ragx-cli web` serves an interactive view of the chunk graph plus a query box, over plain
+`http.server` (no web framework, no build step for the backend):
+
+```
+ragx-cli web [--host 127.0.0.1] [--port 8765] [--open/--no-open]
+```
+
+Renders whatever edges the index has: sub-chunk similarity when it was built with
+`graph.edge_source = "subchunk"` (the default), whole-chunk cosine when `"chunk"`. Fails loud
+(exit 2) only when no index exists yet. To switch an existing index to sub-chunk edges without a
+full re-embed, set the `[graph]` keys in `ragx.toml` and run `ragx-cli index --edges-only`.
+`--open` (default) launches the browser; Ctrl-C shuts the server down cleanly.
+
+Endpoints (JSON; errors are `{"error": "<message>"}`):
+
+- `GET /` and `GET /static/<path>` — the viewer's static assets.
+- `GET /api/graph` — `ragx.graph.v1`: `meta` (root, edge_source, subchunk_size_tokens,
+  embedding_model, nodes/edges/communities counts), `nodes` (id, file, line_start/line_end,
+  label, community, degree), `edges` (`[src, dst, weight]`). Built once at startup, cached,
+  gzip-encoded when the request sends `Accept-Encoding: gzip`.
+- `GET /api/chunk/{id}` — a chunk's file/lines/text/community plus its neighbors
+  (`{chunk_id, weight}`, weight descending). 404 for an unknown id.
+- `POST /api/query` — body `{"text", "top", "expand", "graph", "rerank", "hops", "explain",
+  "before", "after"}` (only `text` required; the rest default like `ragx-cli query`). Returns
+  `ragx.query.v3` (same shape as `--json` query output) plus `elapsed_ms`. 400 on a malformed
+  body, 502 when the query pipeline raises (backend down, manifest mismatch).
+
+Providers (embedder, reranker, expansion LLM) are built lazily on first use of their stage and
+cached for the server's lifetime; queries run one at a time behind a lock.
+
 ## Agent-first conventions
 
-- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v2`,
+- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v3`,
   `ragx.files.v1`, `ragx.status.v1`, `ragx.doctor.v1`, `ragx.eval.v1`, `ragx.inspect.*.v1`); logs go
   to stderr.
 - Exit codes: `0` results, `1` success-but-empty, `2` error.
-- `ragx.query.v2` results are one entry per file: `{file, score, chunks: [...]}`. Each chunk
+- `ragx.query.v3` results are one entry per file: `{file, score, chunks: [...]}`. Each chunk
   carries `chunk_id`, `line_start/line_end`, `byte_start/byte_end`, `score`, `breakdown`, `text`,
   `truncated` — agents jump to the exact source location and read the full text themselves
   (JSON chunk text is truncated). v1 (one flat entry per chunk) is gone since 0.8.0.
+- Each chunk also carries `context: {before, after}` (grep-style `-B`/`-A` lines, default 5
+  each; see [Querying](#querying)) — each side is `{line_start, line_end, text}` or `null`.
+  The key is omitted entirely when both `--before`/`-B` and `--after`/`-A` are 0.
 - `--files-only` returns the same entries without text (`file`, `score`, `chunk_ids`) — the
   mode coding agents use most.
 - `ragx-cli status` includes a `drift` object (`new`/`changed`/`deleted` file counts vs the index) —
@@ -637,7 +708,7 @@ Key defaults:
 |---|---|
 | `[corpus]` | `include` = source code (`*.py`, `*.ts`, `*.go`, `*.rs`, … ~70 extensions, plus `Dockerfile`/`Makefile`) and text docs (`*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.adoc`, `*.org`); `exclude` = lockfiles (`*.lock`, `package-lock.json`, `pnpm-lock.yaml`, `go.sum`, …), `requirements*.txt`/`constraints*.txt`, `LICENSE*`/`COPYING*`, `*.min.js`/`*.min.css`/`*.map`, and `dist/`, `build/`, `target/`, `coverage/`, `vendor/`; `respect_gitignore=true`. Full lists: `src/ragx/core/corpus_globs.py`. Existing `ragx.toml` files keep their own lists |
 | `[chunking]` | `size_tokens=800`, `overlap=0.15` |
-| `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="chunk"`, `subchunk_size_tokens=128`, `near_dup_sim=0.9` |
+| `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="subchunk"`, `subchunk_size_tokens=256`, `near_dup_sim=0.9` |
 | `[traversal]` | `hops=2`, `decay=0.5`, `query_floor=0.35`, `max_frontier=150` |
 | `[communities]` | `resolution=1.0`, `seed=42` — recomputed every index run; changing these never invalidates the index |
 | `[fusion]` | `rrf_k=60`, `per_query_top=20` |
@@ -645,6 +716,7 @@ Key defaults:
 | `[embeddings]` | `provider="openai"`, `base_url="http://localhost:1234/v1"`, prefixes for nomic-style models, `api_key_env=""`, `autostart=true` (start LM Studio + load the model when the URL is local) |
 | `[expansion]` | optional LLM for multi-query/HyDE; reasoning models supported (4096-token budget); `api_key_env=""`, `autostart=true` |
 | `[rerank]` | `BAAI/bge-reranker-v2-m3` via sentence-transformers (`uv tool install 'ragx-cli[rerank]'`) |
+| `[query]` | `top=20`, `max_chunk_chars=1200`, `context_lines=5` (default for `-B`/`-A`/`-C`) |
 
 ## Features & roadmap
 
@@ -659,9 +731,10 @@ unchecked ones are next up. Release history: [CHANGELOG.md](CHANGELOG.md).
 - [x] **Interactive `init`**: LM Studio/Ollama server + model detection, guided embeddings/expansion/corpus setup (`--yes` for defaults)
 - [x] **Committable config**: `ragx.toml` at the corpus root (0.3.0; was `.ragx/config.toml`), index data stays gitignored in `.ragx/`
 - [x] **Offline-friendly rerank**: graceful degradation + pre-seeding docs when huggingface.co is unreachable
-- [x] **Sub-chunk edges** (opt-in `graph.edge_source="subchunk"`): edge weight = max cosine over sentence-aligned sub-chunk pairs, for corpora with long multi-concept chunks
+- [x] **Sub-chunk edges** (default since the 2026-10-21 default flip; `graph.edge_source="subchunk"`): edge weight = max cosine over sentence-aligned sub-chunk pairs, for corpora with long multi-concept chunks; set `graph.edge_source="chunk"` for the whole-chunk-cosine alternative
 - [x] **Incremental by default** (0.4.0): `index` hash-diffs, `--full` rebuilds, `status` reports corpus drift, chunking params are manifest-guarded
 - [x] **`ragx-cli models`**: curated embedding/reranker recommendation (quality tier + RAM detection), downloads via LM Studio's `lms get`, serving via llama-server or LM Studio
+- [x] **`ragx-cli web`**: interactive sub-chunk graph viewer + query UI over stdlib `http.server`
 - [ ] **Community labels**: name the Leiden communities so they're browsable without reading member chunks
 - [ ] **query --global** for corpus-level questions (answer from community summaries, not individual chunks)
 - [ ] **MCP server**: a second thin shell over `ragx.core` (the core/CLI split it needs is already enforced)

{ragx_cli-0.8.0 → ragx_cli-0.10.0}/README.md

@@ -24,6 +24,7 @@ ragx-cli init                  # create ragx.toml next to your corpus (interacti
 ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph (incremental after the first run)
 ragx-cli query "why did we switch build tools?" --json --files-only
 ragx-cli index --full          # full rebuild: re-chunk + re-embed everything
+ragx-cli index --edges-only    # rebuild sub-chunks/edges/communities only; keeps chunk embeddings
 ragx-cli models                # recommend + download an embedding/reranker combo via LM Studio
 ragx-cli doctor                # check the configured providers start and answer, and the index matches
 ragx-cli --version             # version, repo, and the effective models (corpus config + ~/.ragxrc overrides)
@@ -61,21 +62,28 @@ embedded, and stored in an HNSW index. The similarity graph then falls out almos
 one kNN pass over the vectors that are already in memory — each chunk gets edges to its top-k
 nearest neighbors above a similarity floor.
 
-Experimental: `graph.edge_source = "subchunk"` derives edge weights from sentence-aligned
+`graph.edge_source = "subchunk"` (default) derives edge weights from sentence-aligned
 **sub-chunks** instead of whole-chunk cosine — each chunk is split into ~`subchunk_size_tokens`
-windows, embedded separately (stored in SQLite, never in the query-time HNSW), and the edge
-weight between two chunks becomes the *max* similarity over their sub-chunk pairs, so a chunk
-mixing several concepts gets one sharp edge per concept instead of a diluted average. Edges
+(256) token windows, embedded separately (stored in SQLite, never in the query-time HNSW), and
+the edge weight between two chunks becomes the *max* similarity over their sub-chunk pairs, so a
+chunk mixing several concepts gets one sharp edge per concept instead of a diluted average. Edges
 between near-duplicate chunks (whole-chunk cosine ≥ `near_dup_sim`) are dropped — those are the
-measured precision-killers. Costs roughly 5–10× more embedding calls at index time; retrieval
-units, traversal, and query flow are unchanged. Switching `edge_source` (or the sub-chunk size)
-requires `index --full`; incremental runs fail loud on the mismatch. Literature grounding:
+measured precision-killers. Costs ~2× more embedding calls at index time than `edge_source =
+"chunk"`; retrieval units, traversal, and query flow are unchanged. Switching `edge_source` (or
+the sub-chunk size) requires either `index --edges-only` (cheap: rebuilds sub-chunks, edges and
+communities from the stored chunks — chunk embeddings and the HNSW index are kept) or
+`index --full`; incremental runs fail loud on the mismatch, naming the old/new value and both
+fixes. Literature grounding:
 `research/fine-grained-sub-chunk-edges-with-coarse-chunk-nodes-multi-granularity-graph-rag-literature-validation.md`.
 In this mode `k` counts links per *sub-chunk* with no per-chunk cap, so the graph is denser and
-shallower traversal suffices. Measured on the tuned eval corpus (2026-07): at `hops=2` it ties
-the chunk-edge baseline exactly (same recall, same MRR) at ~4.5× the indexing cost — that
-corpus's short, single-topic chunks leave no concept-dilution headroom to exploit. Worth trying
-only on corpora with long, genuinely multi-concept chunks (use `hops=2`); the default stays `"chunk"`.
+shallower traversal suffices (the default `hops=2` already matches this). Quality ties
+`edge_source="chunk"` exactly at every sub-chunk size tried, 64–256 tokens (2026-07 and
+2026-10-21 sweeps); the corpora measured so far have short, single-topic chunks with no
+concept-dilution headroom to exploit. `256` is the default because it's the cheapest size that
+ties — fewest edges, ~2× index time vs ~2.8–4.7× at 128/64 tokens. Set `graph.edge_source =
+"chunk"` back if a corpus's long, genuinely multi-concept chunks make the extra edges worth the
+cost — no corpus has measured a win yet. Sweep data:
+`research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md`.
 
 After edge construction, a Leiden partition (`graspologic-native`, seeded, deterministic) is
 computed over the whole edge list every index run and stored read-only — see
@@ -96,9 +104,24 @@ flowchart TD
 re-embeds only new/modified files, drops deleted ones, and repairs only the edge lists those
 chunks touch. Content hashes make `touch`-ed but unchanged files free, and include/exclude glob
 changes converge to the same file set a rebuild would produce. `--full` rebuilds from scratch;
-changing `embeddings.model`, `chunking.*`, or `graph.edge_source` requires it — incremental runs
-fail loud on the manifest mismatch. `ragx-cli status` reports **drift** (files new/changed/deleted
-since the last index) so you can see when a re-index is due without running one.
+changing `embeddings.model` or `chunking.*` requires it — incremental runs fail loud on the
+manifest mismatch, naming the changed key and its old/new value. Changing `graph.edge_source`/
+`subchunk_size_tokens` alone is cheaper: `--edges-only` rebuilds just the sub-chunks, edges and
+communities from the chunks already stored — it skips chunking and embedding entirely, reusing
+each chunk's stored vector (via the HNSW index) for chunks that stay whole, and only calls the
+embedder for the sub-chunk pieces a long chunk splits into. It fails loud the same way `--full`
+does if `embeddings.model`/`chunking.*` don't match the stored manifest (those invalidate the
+chunks it reuses), but `edge_source`/`subchunk_size_tokens` are exactly what it's meant to change.
+`ragx-cli status` reports **drift** (files new/changed/deleted since the last index) so you can
+see when a re-index is due without running one.
+
+Upgrading from an older `ragx-cli` after a default change (e.g. the `graph.edge_source` flip to
+`"subchunk"`): `ragx-cli init` always writes every `[graph]` key explicitly into `ragx.toml`
+(`Config.save` bakes the full default set, not just overrides), so an init-created corpus already
+pins its `edge_source`/`subchunk_size_tokens` and is unaffected by a later default change. Only a
+hand-written `ragx.toml` missing those keys picks up the new default; its next incremental
+`index` then fails loud — `index --edges-only` rebuilds just the graph (cheapest), `index --full`
+rebuilds everything, or pin the old value back in `ragx.toml`.
 
 ### Querying
 
@@ -106,6 +129,16 @@ A default query runs vector search + graph traversal: no LLM call, no cross-enco
 Expansion (`--expand`) and rerank (`--rerank`) are opt-in; `--no-graph` skips traversal,
 which leaves pure vector search. Callers trade latency for quality per query.
 
+Every result chunk also comes with grep-style `-B`/`-A`/`-C` context lines, on by default
+(`query.context_lines=5` on each side). `-B/--before N` and `-A/--after N` set each side
+independently; `-C/--context N` sets both; explicit `-B`/`-A` win over `-C`, which wins over
+the default. Counting rule: `N` counts **non-empty** lines walking outward from the chunk —
+blank lines don't count toward `N` but are kept in the returned span if they fall between the
+chunk and the Nth non-empty line; the span stops exactly at that line, so blank lines beyond it
+are excluded. `N=0` disables that side; fewer than `N` non-empty lines in the file returns what
+exists; none returns `null`. Context is read fresh from disk at query time, never truncated by
+`query.max_chunk_chars`, and `--files-only` skips it (no file reads, since the text isn't shown).
+
 Results are **one entry per file**. The pipeline ranks chunks, walks that list until it has
 seen `--top` distinct files (default `query.top = 20`), and folds each file's chunks inside
 that window into one entry: up to 3 sub-entries, entry score = their sum. A file that would
@@ -145,7 +178,10 @@ flowchart LR
 ```
 
 Because every admitted chunk records which seed and edge produced it, `--explain` can print the
-full justification: *seed → edge(weight) → chunk*, per result.
+full justification: *seed → edge(weight) → chunk*, per result. The explain object also carries
+`path`: the seed-first chain of chunk ids along the traversal's own parent links
+(`[seed, ..., chunk_id]`), so a UI can highlight the exact path the traversal took instead of
+guessing a shortest path through the graph.
 
 ---
 
@@ -414,16 +450,51 @@ section with `ragx-cli config set embeddings.autostart false` (or machine-wide v
 `ragx-cli models` still only downloads models and writes `ragx.toml`; it never launches
 anything. If a run dies with "connection refused", `doctor` names the stage and the fix.
 
+## Browsing the graph: `ragx-cli web`
+
+`ragx-cli web` serves an interactive view of the chunk graph plus a query box, over plain
+`http.server` (no web framework, no build step for the backend):
+
+```
+ragx-cli web [--host 127.0.0.1] [--port 8765] [--open/--no-open]
+```
+
+Renders whatever edges the index has: sub-chunk similarity when it was built with
+`graph.edge_source = "subchunk"` (the default), whole-chunk cosine when `"chunk"`. Fails loud
+(exit 2) only when no index exists yet. To switch an existing index to sub-chunk edges without a
+full re-embed, set the `[graph]` keys in `ragx.toml` and run `ragx-cli index --edges-only`.
+`--open` (default) launches the browser; Ctrl-C shuts the server down cleanly.
+
+Endpoints (JSON; errors are `{"error": "<message>"}`):
+
+- `GET /` and `GET /static/<path>` — the viewer's static assets.
+- `GET /api/graph` — `ragx.graph.v1`: `meta` (root, edge_source, subchunk_size_tokens,
+  embedding_model, nodes/edges/communities counts), `nodes` (id, file, line_start/line_end,
+  label, community, degree), `edges` (`[src, dst, weight]`). Built once at startup, cached,
+  gzip-encoded when the request sends `Accept-Encoding: gzip`.
+- `GET /api/chunk/{id}` — a chunk's file/lines/text/community plus its neighbors
+  (`{chunk_id, weight}`, weight descending). 404 for an unknown id.
+- `POST /api/query` — body `{"text", "top", "expand", "graph", "rerank", "hops", "explain",
+  "before", "after"}` (only `text` required; the rest default like `ragx-cli query`). Returns
+  `ragx.query.v3` (same shape as `--json` query output) plus `elapsed_ms`. 400 on a malformed
+  body, 502 when the query pipeline raises (backend down, manifest mismatch).
+
+Providers (embedder, reranker, expansion LLM) are built lazily on first use of their stage and
+cached for the server's lifetime; queries run one at a time behind a lock.
+
 ## Agent-first conventions
 
-- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v2`,
+- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v3`,
   `ragx.files.v1`, `ragx.status.v1`, `ragx.doctor.v1`, `ragx.eval.v1`, `ragx.inspect.*.v1`); logs go
   to stderr.
 - Exit codes: `0` results, `1` success-but-empty, `2` error.
-- `ragx.query.v2` results are one entry per file: `{file, score, chunks: [...]}`. Each chunk
+- `ragx.query.v3` results are one entry per file: `{file, score, chunks: [...]}`. Each chunk
   carries `chunk_id`, `line_start/line_end`, `byte_start/byte_end`, `score`, `breakdown`, `text`,
   `truncated` — agents jump to the exact source location and read the full text themselves
   (JSON chunk text is truncated). v1 (one flat entry per chunk) is gone since 0.8.0.
+- Each chunk also carries `context: {before, after}` (grep-style `-B`/`-A` lines, default 5
+  each; see [Querying](#querying)) — each side is `{line_start, line_end, text}` or `null`.
+  The key is omitted entirely when both `--before`/`-B` and `--after`/`-A` are 0.
 - `--files-only` returns the same entries without text (`file`, `score`, `chunk_ids`) — the
   mode coding agents use most.
 - `ragx-cli status` includes a `drift` object (`new`/`changed`/`deleted` file counts vs the index) —
@@ -609,7 +680,7 @@ Key defaults:
 |---|---|
 | `[corpus]` | `include` = source code (`*.py`, `*.ts`, `*.go`, `*.rs`, … ~70 extensions, plus `Dockerfile`/`Makefile`) and text docs (`*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.adoc`, `*.org`); `exclude` = lockfiles (`*.lock`, `package-lock.json`, `pnpm-lock.yaml`, `go.sum`, …), `requirements*.txt`/`constraints*.txt`, `LICENSE*`/`COPYING*`, `*.min.js`/`*.min.css`/`*.map`, and `dist/`, `build/`, `target/`, `coverage/`, `vendor/`; `respect_gitignore=true`. Full lists: `src/ragx/core/corpus_globs.py`. Existing `ragx.toml` files keep their own lists |
 | `[chunking]` | `size_tokens=800`, `overlap=0.15` |
-| `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="chunk"`, `subchunk_size_tokens=128`, `near_dup_sim=0.9` |
+| `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="subchunk"`, `subchunk_size_tokens=256`, `near_dup_sim=0.9` |
 | `[traversal]` | `hops=2`, `decay=0.5`, `query_floor=0.35`, `max_frontier=150` |
 | `[communities]` | `resolution=1.0`, `seed=42` — recomputed every index run; changing these never invalidates the index |
 | `[fusion]` | `rrf_k=60`, `per_query_top=20` |
@@ -617,6 +688,7 @@ Key defaults:
 | `[embeddings]` | `provider="openai"`, `base_url="http://localhost:1234/v1"`, prefixes for nomic-style models, `api_key_env=""`, `autostart=true` (start LM Studio + load the model when the URL is local) |
 | `[expansion]` | optional LLM for multi-query/HyDE; reasoning models supported (4096-token budget); `api_key_env=""`, `autostart=true` |
 | `[rerank]` | `BAAI/bge-reranker-v2-m3` via sentence-transformers (`uv tool install 'ragx-cli[rerank]'`) |
+| `[query]` | `top=20`, `max_chunk_chars=1200`, `context_lines=5` (default for `-B`/`-A`/`-C`) |
 
 ## Features & roadmap
 
@@ -631,9 +703,10 @@ unchecked ones are next up. Release history: [CHANGELOG.md](CHANGELOG.md).
 - [x] **Interactive `init`**: LM Studio/Ollama server + model detection, guided embeddings/expansion/corpus setup (`--yes` for defaults)
 - [x] **Committable config**: `ragx.toml` at the corpus root (0.3.0; was `.ragx/config.toml`), index data stays gitignored in `.ragx/`
 - [x] **Offline-friendly rerank**: graceful degradation + pre-seeding docs when huggingface.co is unreachable
-- [x] **Sub-chunk edges** (opt-in `graph.edge_source="subchunk"`): edge weight = max cosine over sentence-aligned sub-chunk pairs, for corpora with long multi-concept chunks
+- [x] **Sub-chunk edges** (default since the 2026-10-21 default flip; `graph.edge_source="subchunk"`): edge weight = max cosine over sentence-aligned sub-chunk pairs, for corpora with long multi-concept chunks; set `graph.edge_source="chunk"` for the whole-chunk-cosine alternative
 - [x] **Incremental by default** (0.4.0): `index` hash-diffs, `--full` rebuilds, `status` reports corpus drift, chunking params are manifest-guarded
 - [x] **`ragx-cli models`**: curated embedding/reranker recommendation (quality tier + RAM detection), downloads via LM Studio's `lms get`, serving via llama-server or LM Studio
+- [x] **`ragx-cli web`**: interactive sub-chunk graph viewer + query UI over stdlib `http.server`
 - [ ] **Community labels**: name the Leiden communities so they're browsable without reading member chunks
 - [ ] **query --global** for corpus-level questions (answer from community summaries, not individual chunks)
 - [ ] **MCP server**: a second thin shell over `ragx.core` (the core/CLI split it needs is already enforced)

{ragx_cli-0.8.0 → ragx_cli-0.10.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ragx-cli"
-version = "0.8.0"
+version = "0.10.0"
 description = "Similarity-graph RAG CLI: index a corpus into a chunk-level embedding graph and query it"
 readme = "README.md"
 license = "MIT"