ragx-cli 0.3.2__tar.gz → 0.4.0__tar.gz

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/CHANGELOG.md

@@ -4,6 +4,34 @@ All notable changes to `ragx-cli` are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
 [Semantic Versioning](https://semver.org/).
 
+## [0.4.0] — 2026-07-15
+
+### Changed (BREAKING)
+
+- **`ragx-cli index` is now incremental by default** (previously a full rebuild).
+  It hash-diffs the corpus and re-processes only new/modified/deleted files. The old
+  behavior moved to `ragx-cli index --full`; `--changed` remains as a deprecated,
+  hidden alias for the (now default) incremental mode.
+- **Chunking params joined the index manifest**: changing `chunking.size_tokens` or
+  `chunking.overlap` now fails loud on an incremental run (previously it silently left
+  old files chunked under the old params). Fix is `ragx-cli index --full`.
+
+### Added
+
+- **`ragx-cli status` reports corpus drift**: a `drift` object
+  (`{"new": N, "changed": N, "deleted": N}`, same in text output) diffing the corpus
+  on disk against the stored file hashes, so agents and humans can see when a
+  re-index is due without running one.
+
+## [0.3.3] — 2026-07-15
+
+### Added
+
+- **Early hint when the reranker model download stalls**: the huggingface_hub retry
+  loop inside the cross-encoder load can spin for minutes before failing. The first
+  retry warning now also prints a one-shot stderr pointer to the README's "Reranker on
+  restricted networks" section (manual copy, mirror, or `rerank.enabled false`).
+
 ## [0.3.2] — 2026-07-15
 
 _0.3.1 was tagged but never published — its release workflow referenced a nonexistent
@@ -95,6 +123,7 @@ Initial release: Phases 0–3 of [the plan](ragx-cli-plan.md), built and validat
   on stderr, exit codes `0` results / `1` empty / `2` error.
 - MIT license; PyPI packaging as `ragx-cli` with trusted-publishing release workflow.
 
+[0.3.3]: https://github.com/alber70g/ragx-cli/compare/v0.3.2...v0.3.3
 [0.3.2]: https://github.com/alber70g/ragx-cli/compare/v0.3.0...v0.3.2
 [0.3.0]: https://github.com/alber70g/ragx-cli/compare/v0.2.0...v0.3.0
 [0.2.0]: https://github.com/alber70g/ragx-cli/compare/v0.1.1...v0.2.0

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ragx-cli
-Version: 0.3.2
+Version: 0.4.0
 Summary: Similarity-graph RAG CLI: index a corpus into a chunk-level embedding graph and query it
 Project-URL: Repository, https://github.com/alber70g/ragx-cli
 Project-URL: Issues, https://github.com/alber70g/ragx-cli/issues
@@ -49,9 +49,9 @@ that can justify every result via the exact graph path that produced it.
 ```bash
 uv tool install ragx-cli --with ragx-cli[rerank]  # install and use `ragx-cli`
 ragx-cli init                  # create ragx.toml next to your corpus (interactive on a TTY; --yes for defaults)
-ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph
+ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph (incremental after the first run)
 ragx-cli query "why did we switch build tools?" --json --files-only
-ragx-cli index --changed       # incremental: only new/modified/deleted files
+ragx-cli index --full          # full rebuild: re-chunk + re-embed everything
 ```
 
 Runs against any OpenAI-compatible embedding endpoint (LM Studio, Ollama, OpenAI). Reranking
@@ -93,7 +93,7 @@ mixing several concepts gets one sharp edge per concept instead of a diluted ave
 between near-duplicate chunks (whole-chunk cosine ≥ `near_dup_sim`) are dropped — those are the
 measured precision-killers. Costs roughly 5–10× more embedding calls at index time; retrieval
 units, traversal, and query flow are unchanged. Switching `edge_source` (or the sub-chunk size)
-requires a full re-index; `--changed` fails loud on the mismatch. Literature grounding:
+requires `index --full`; incremental runs fail loud on the mismatch. Literature grounding:
 `research/fine-grained-sub-chunk-edges-with-coarse-chunk-nodes-multi-granularity-graph-rag-literature-validation.md`.
 In this mode `k` counts links per *sub-chunk* with no per-chunk cap, so the graph is denser and
 shallower traversal suffices. Measured on the tuned eval corpus (2026-07): at `hops=2` it ties
@@ -116,8 +116,13 @@ flowchart TD
     G --> F
 ```
 
-Incremental runs (`--changed`) re-embed only changed files and repair only the edge lists that
-those chunks touch. Content hashes make `touch`-ed but unchanged files free.
+`ragx-cli index` is **incremental by default** (0.4.0; `--changed` is a deprecated alias): it
+re-embeds only new/modified files, drops deleted ones, and repairs only the edge lists those
+chunks touch. Content hashes make `touch`-ed but unchanged files free, and include/exclude glob
+changes converge to the same file set a rebuild would produce. `--full` rebuilds from scratch;
+changing `embeddings.model`, `chunking.*`, or `graph.edge_source` requires it — incremental runs
+fail loud on the manifest mismatch. `ragx-cli status` reports **drift** (files new/changed/deleted
+since the last index) so you can see when a re-index is due without running one.
 
 ### Querying
 
@@ -322,6 +327,8 @@ Full write-ups: `research/bge-m3-dense-q8-vs-nomic-q4-benchmark-2026-07-12-workt
 - Every chunk carries `file`, `line_start/line_end`, `byte_start/byte_end` — agents jump to the
   exact source location and read the full text themselves (JSON chunk text is truncated).
 - `--files-only` aggregates chunk scores per file (sum of top-3) — the mode coding agents use most.
+- `ragx-cli status` includes a `drift` object (`new`/`changed`/`deleted` file counts vs the index) —
+  agents check it to decide whether to run `ragx-cli index` before querying.
 - `ragx-cli query -` reads the query from stdin; `ragx-cli inspect chunk|file|neighbors|communities|community`
   debugs the graph.
 
@@ -341,7 +348,8 @@ questions; fall back to grep for exact identifiers.
   `file` + `line_start/line_end`; the JSON `text` is truncated, so read the file yourself
   for full context.
 - Fast mode (no LLM call, no cross-encoder): add `--no-expand --no-rerank`.
-- After adding or editing files: `ragx-cli index --changed` (cheap, hash-based).
+- After adding or editing files: `ragx-cli index` (incremental and cheap — hash-based).
+  `ragx-cli status --json` shows `drift` counts if you want to check staleness first.
 - stdout is exactly one JSON document; logs are on stderr.
   Exit codes: 0 = results, 1 = no results (not an error), 2 = error.
 - Why did this result appear? `ragx-cli query "..." --explain`.
@@ -499,7 +507,7 @@ Checked features are built and validated per [the implementation plan](ragx-cli-
 unchecked ones are next up. Release history: [CHANGELOG.md](CHANGELOG.md).
 
 - [x] **CLI & storage**: typer CLI, SQLite schema, provider abstraction (Embedder/Generator/Reranker)
-- [x] **Baseline vector RAG**: discovery, chunking, embeddings, HNSW search, incremental `--changed`
+- [x] **Baseline vector RAG**: discovery, chunking, embeddings, HNSW search, incremental indexing
 - [x] **Similarity graph**: kNN edge construction, heat-propagation traversal, `inspect`, `--explain`
 - [x] **Quality & measurement**: multi-query/HyDE expansion, RRF fusion, cross-encoder rerank, `eval` harness
 - [x] **Communities**: Leiden detection over the edge list (index-time, read-only via status/inspect)
@@ -507,11 +515,12 @@ unchecked ones are next up. Release history: [CHANGELOG.md](CHANGELOG.md).
 - [x] **Committable config**: `ragx.toml` at the corpus root (0.3.0; was `.ragx/config.toml`), index data stays gitignored in `.ragx/`
 - [x] **Offline-friendly rerank**: graceful degradation + pre-seeding docs when huggingface.co is unreachable
 - [x] **Sub-chunk edges** (opt-in `graph.edge_source="subchunk"`): edge weight = max cosine over sentence-aligned sub-chunk pairs, for corpora with long multi-concept chunks
+- [x] **Incremental by default** (0.4.0): `index` hash-diffs, `--full` rebuilds, `status` reports corpus drift, chunking params are manifest-guarded
 - [ ] **Community labels**: name the Leiden communities so they're browsable without reading member chunks
 - [ ] **query --global** for corpus-level questions (answer from community summaries, not individual chunks)
 - [ ] **MCP server**: a second thin shell over `ragx.core` (the core/CLI split it needs is already enforced)
 - [ ] **[Temporal weighting](docs/feature-temporal-weighting.md)**: opt-in `--since`/`--until`/`--temporal recent|oldest`, date cascade filename/frontmatter → git → mtime
 - [x] **Release**: publish to PyPI as `ragx-cli` (plain `ragx` is name-blocked, too similar to an existing project) so `uvx ragx-cli` works out of the box
 
-Development: `uv sync --group dev && uv run pytest`. 177 tests; module contracts live in
+Development: `uv sync --group dev && uv run pytest`. 183 tests; module contracts live in
 `CONTRACTS.md` / `CONTRACTS-PHASE23.md`.

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/README.md

@@ -21,9 +21,9 @@ that can justify every result via the exact graph path that produced it.
 ```bash
 uv tool install ragx-cli --with ragx-cli[rerank]  # install and use `ragx-cli`
 ragx-cli init                  # create ragx.toml next to your corpus (interactive on a TTY; --yes for defaults)
-ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph
+ragx-cli index                 # chunk -> embed -> HNSW + kNN similarity graph (incremental after the first run)
 ragx-cli query "why did we switch build tools?" --json --files-only
-ragx-cli index --changed       # incremental: only new/modified/deleted files
+ragx-cli index --full          # full rebuild: re-chunk + re-embed everything
 ```
 
 Runs against any OpenAI-compatible embedding endpoint (LM Studio, Ollama, OpenAI). Reranking
@@ -65,7 +65,7 @@ mixing several concepts gets one sharp edge per concept instead of a diluted ave
 between near-duplicate chunks (whole-chunk cosine ≥ `near_dup_sim`) are dropped — those are the
 measured precision-killers. Costs roughly 5–10× more embedding calls at index time; retrieval
 units, traversal, and query flow are unchanged. Switching `edge_source` (or the sub-chunk size)
-requires a full re-index; `--changed` fails loud on the mismatch. Literature grounding:
+requires `index --full`; incremental runs fail loud on the mismatch. Literature grounding:
 `research/fine-grained-sub-chunk-edges-with-coarse-chunk-nodes-multi-granularity-graph-rag-literature-validation.md`.
 In this mode `k` counts links per *sub-chunk* with no per-chunk cap, so the graph is denser and
 shallower traversal suffices. Measured on the tuned eval corpus (2026-07): at `hops=2` it ties
@@ -88,8 +88,13 @@ flowchart TD
     G --> F
 ```
 
-Incremental runs (`--changed`) re-embed only changed files and repair only the edge lists that
-those chunks touch. Content hashes make `touch`-ed but unchanged files free.
+`ragx-cli index` is **incremental by default** (0.4.0; `--changed` is a deprecated alias): it
+re-embeds only new/modified files, drops deleted ones, and repairs only the edge lists those
+chunks touch. Content hashes make `touch`-ed but unchanged files free, and include/exclude glob
+changes converge to the same file set a rebuild would produce. `--full` rebuilds from scratch;
+changing `embeddings.model`, `chunking.*`, or `graph.edge_source` requires it — incremental runs
+fail loud on the manifest mismatch. `ragx-cli status` reports **drift** (files new/changed/deleted
+since the last index) so you can see when a re-index is due without running one.
 
 ### Querying
 
@@ -294,6 +299,8 @@ Full write-ups: `research/bge-m3-dense-q8-vs-nomic-q4-benchmark-2026-07-12-workt
 - Every chunk carries `file`, `line_start/line_end`, `byte_start/byte_end` — agents jump to the
   exact source location and read the full text themselves (JSON chunk text is truncated).
 - `--files-only` aggregates chunk scores per file (sum of top-3) — the mode coding agents use most.
+- `ragx-cli status` includes a `drift` object (`new`/`changed`/`deleted` file counts vs the index) —
+  agents check it to decide whether to run `ragx-cli index` before querying.
 - `ragx-cli query -` reads the query from stdin; `ragx-cli inspect chunk|file|neighbors|communities|community`
   debugs the graph.
 
@@ -313,7 +320,8 @@ questions; fall back to grep for exact identifiers.
   `file` + `line_start/line_end`; the JSON `text` is truncated, so read the file yourself
   for full context.
 - Fast mode (no LLM call, no cross-encoder): add `--no-expand --no-rerank`.
-- After adding or editing files: `ragx-cli index --changed` (cheap, hash-based).
+- After adding or editing files: `ragx-cli index` (incremental and cheap — hash-based).
+  `ragx-cli status --json` shows `drift` counts if you want to check staleness first.
 - stdout is exactly one JSON document; logs are on stderr.
   Exit codes: 0 = results, 1 = no results (not an error), 2 = error.
 - Why did this result appear? `ragx-cli query "..." --explain`.
@@ -471,7 +479,7 @@ Checked features are built and validated per [the implementation plan](ragx-cli-
 unchecked ones are next up. Release history: [CHANGELOG.md](CHANGELOG.md).
 
 - [x] **CLI & storage**: typer CLI, SQLite schema, provider abstraction (Embedder/Generator/Reranker)
-- [x] **Baseline vector RAG**: discovery, chunking, embeddings, HNSW search, incremental `--changed`
+- [x] **Baseline vector RAG**: discovery, chunking, embeddings, HNSW search, incremental indexing
 - [x] **Similarity graph**: kNN edge construction, heat-propagation traversal, `inspect`, `--explain`
 - [x] **Quality & measurement**: multi-query/HyDE expansion, RRF fusion, cross-encoder rerank, `eval` harness
 - [x] **Communities**: Leiden detection over the edge list (index-time, read-only via status/inspect)
@@ -479,11 +487,12 @@ unchecked ones are next up. Release history: [CHANGELOG.md](CHANGELOG.md).
 - [x] **Committable config**: `ragx.toml` at the corpus root (0.3.0; was `.ragx/config.toml`), index data stays gitignored in `.ragx/`
 - [x] **Offline-friendly rerank**: graceful degradation + pre-seeding docs when huggingface.co is unreachable
 - [x] **Sub-chunk edges** (opt-in `graph.edge_source="subchunk"`): edge weight = max cosine over sentence-aligned sub-chunk pairs, for corpora with long multi-concept chunks
+- [x] **Incremental by default** (0.4.0): `index` hash-diffs, `--full` rebuilds, `status` reports corpus drift, chunking params are manifest-guarded
 - [ ] **Community labels**: name the Leiden communities so they're browsable without reading member chunks
 - [ ] **query --global** for corpus-level questions (answer from community summaries, not individual chunks)
 - [ ] **MCP server**: a second thin shell over `ragx.core` (the core/CLI split it needs is already enforced)
 - [ ] **[Temporal weighting](docs/feature-temporal-weighting.md)**: opt-in `--since`/`--until`/`--temporal recent|oldest`, date cascade filename/frontmatter → git → mtime
 - [x] **Release**: publish to PyPI as `ragx-cli` (plain `ragx` is name-blocked, too similar to an existing project) so `uvx ragx-cli` works out of the box
 
-Development: `uv sync --group dev && uv run pytest`. 177 tests; module contracts live in
+Development: `uv sync --group dev && uv run pytest`. 183 tests; module contracts live in
 `CONTRACTS.md` / `CONTRACTS-PHASE23.md`.

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ragx-cli"
-version = "0.3.2"
+version = "0.4.0"
 description = "Similarity-graph RAG CLI: index a corpus into a chunk-level embedding graph and query it"
 readme = "README.md"
 license = "MIT"

ragx_cli-0.4.0/skills/ragx-cli/SKILL.md

@@ -0,0 +1,84 @@
+---
+name: ragx-cli
+description: |
+  Query a ragx-cli semantic index instead of grepping. Use when the user asks a conceptual or
+  natural-language question about a corpus ("where is X discussed/decided?", "find notes about Y"),
+  when you are about to grep/rg for a concept (not an exact identifier) in a directory that contains
+  a `.ragx/` folder or `ragx.toml`, or when the user says `/ragx`, "search my notes", or "semantic
+  search". Covers running queries, interpreting the JSON output, and keeping the index fresh.
+---
+
+# ragx-cli — semantic search over an indexed corpus
+
+Indexes files into a chunk-level embedding similarity graph; answers queries via vector search +
+graph traversal + cross-encoder rerank. Agent-first: one JSON doc on stdout, logs on stderr,
+deterministic exit codes, byte-exact source locations.
+
+## When
+
+A corpus is ragx-indexed if its root has `ragx.toml` and `.ragx/`.
+
+- Use ragx-cli for conceptual, natural-language questions ("where do we discuss auth tradeoffs?",
+  cross-language queries). Full questions beat keywords.
+- Use grep/rg for exact identifiers and literal strings. ragx does not replace it.
+- Not on PATH: `uvx ragx-cli …` works (PyPI name is `ragx-cli`). Run from anywhere inside the
+  corpus — root is found upward.
+
+## Commands
+
+```bash
+ragx-cli status --json          # root, model, counts, and drift (new/changed/deleted vs index)
+ragx-cli query "question" --json --files-only   # which files are relevant — most-used mode
+ragx-cli query "..." --json --top 8             # ranked chunks with exact locations
+ragx-cli query "..." --json --no-expand --no-rerank   # fast: skip LLM call + cross-encoder
+echo "..." | ragx-cli query - --json            # long/multiline query from stdin
+ragx-cli index                  # incremental reindex (hash-based, cheap) — the default
+ragx-cli index --full           # full rebuild: re-chunks + re-embeds everything
+```
+
+Other query flags: `--no-graph`, `--hops N`, `--explain`.
+
+## Output
+
+- stdout is exactly one JSON doc (`ragx.query.v1` / `ragx.files.v1` / `ragx.status.v1`); logs go
+  to stderr. Never parse stderr.
+- Exit codes: `0` results, `1` success-but-empty (report "nothing found", don't retry as failure),
+  `2` error.
+- Chunk results carry `file`, `line_start/end`, `byte_start/end`, `score`, `breakdown`. JSON
+  `text` is truncated — read the file at the line range for full context.
+- Typical flow: `--files-only` to pick files → `--top 8` for locations → read files at ranges.
+
+## Freshness
+
+- `status` includes `drift` counts; nonzero means the index is stale → run `ragx-cli index`
+  (incremental, safe, converges even after include/exclude glob changes).
+- Changing `embeddings.model`, `chunking.*`, or `graph.edge_source` invalidates the index:
+  commands fail loud on the manifest mismatch, and the fix is `ragx-cli index --full` —
+  re-embeds everything, so confirm before running on a large corpus.
+
+## Why did/didn't a result appear?
+
+- `ragx-cli query "..." --explain` — per-result trace: seed, graph edge, hop.
+- `ragx-cli inspect neighbors <chunk_id>` / `inspect chunk <id>` / `inspect file <path>` /
+  `inspect communities` / `inspect community <id>`.
+
+## Gotchas
+
+- The embedding provider must be reachable (default: LM Studio at `localhost:1234`; Ollama and
+  any OpenAI-compatible endpoint via config). On connection error, report it and suggest starting
+  the provider — never silently fall back to grep.
+- Query expansion degrades gracefully (LLM down → no-op); embeddings do not.
+- `--changed` is a deprecated alias — incremental is the default now.
+- The corpus may be someone's personal notes repo: never `git add`/commit there. Querying and
+  `ragx-cli index` are safe.
+
+## New corpus (only when asked)
+
+```bash
+ragx-cli init
+ragx-cli config set embeddings.model <model-id>   # match what the provider serves
+ragx-cli index
+```
+
+Provider recipes are in the ragx-cli README. Check `ragx-cli status` and
+`curl -s http://localhost:1234/v1/models` before guessing model ids.

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/src/ragx/cli/app.py

@@ -93,10 +93,13 @@ def _counts(root: Path) -> tuple[int, int, int, int]:
 
 @app.command()
 def status(json_out: bool = typer.Option(False, "--json")) -> None:
-    """Show corpus root, embedding config, and index counts."""
+    """Show corpus root, embedding config, index counts, and drift vs the corpus on disk."""
+    from ragx.core.indexer import corpus_drift
+
     root = _require_root()
     cfg = _load_config(root)
     files, chunks, edges, communities = _counts(root)
+    drift = corpus_drift(root, cfg)
     doc = {
         "schema": "ragx.status.v1",
         "root": str(root),
@@ -106,6 +109,7 @@ def status(json_out: bool = typer.Option(False, "--json")) -> None:
         "chunks": chunks,
         "edges": edges,
         "communities": communities,
+        "drift": drift,
     }
     if json_out:
         emit_json(doc)
@@ -113,6 +117,10 @@ def status(json_out: bool = typer.Option(False, "--json")) -> None:
         typer.echo(f"root: {doc['root']}")
         typer.echo(f"embedding: {doc['embedding_provider']}/{doc['embedding_model']}")
         typer.echo(f"files: {files}  chunks: {chunks}  edges: {edges}  communities: {communities}")
+        line = f"drift: {drift['new']} new, {drift['changed']} changed, {drift['deleted']} deleted"
+        if any(drift.values()):
+            line += "  (run `ragx-cli index`)"
+        typer.echo(line)
     if files == 0 and chunks == 0 and edges == 0:
         raise typer.Exit(code=1)
 

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/src/ragx/cli/pipeline.py

@@ -22,14 +22,21 @@ def register(app: typer.Typer) -> None:
 
 def index(
     path: Path | None = typer.Argument(None),
-    changed: bool = typer.Option(False, "--changed", help="only re-process new/modified/deleted files"),
+    full: bool = typer.Option(False, "--full", help="full rebuild: re-chunk and re-embed everything"),
+    changed: bool = typer.Option(
+        False, "--changed", hidden=True, help="deprecated: incremental is now the default"
+    ),
     json_out: bool = typer.Option(False, "--json"),
 ) -> None:
-    """Chunk, embed, and index the corpus (full rebuild unless --changed)."""
+    """Chunk, embed, and index the corpus (incremental by default; --full rebuilds)."""
+    if changed and full:
+        fail("--changed and --full are mutually exclusive")
+    if changed:
+        print("warning: --changed is deprecated; incremental is now the default", file=sys.stderr)
     try:
         root = require_root(path)
         cfg = Config.load(root, confirm=migrate_confirm())
-        stats = run_index(root, cfg, make_embedder(cfg), changed_only=changed)
+        stats = run_index(root, cfg, make_embedder(cfg), full=full)
     except RagxError as exc:
         fail(str(exc))
     doc = {"schema": "ragx.index.v1", **stats.__dict__}

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/src/ragx/core/indexer.py

@@ -35,42 +35,40 @@ class IndexStats:
     communities_total: int = 0
 
 
-def run_index(root: Path, cfg: Config, embedder: Embedder, *, changed_only: bool = False) -> IndexStats:
-    """Index the corpus at `root`. Full rebuild by default; hash-diff when changed_only."""
+def run_index(root: Path, cfg: Config, embedder: Embedder, *, full: bool = False) -> IndexStats:
+    """Index the corpus at `root`. Hash-diff incremental by default; rebuild when full."""
     edge_source = cfg.get("graph.edge_source")
     if edge_source not in ("chunk", "subchunk"):
         raise RagxError(f"graph.edge_source must be 'chunk' or 'subchunk', got {edge_source!r}")
     sub_size = cfg.get("graph.subchunk_size_tokens")
+    size_tokens = cfg.get("chunking.size_tokens")
+    overlap = cfg.get("chunking.overlap")
     db_path(root).parent.mkdir(parents=True, exist_ok=True)
     with Store(db_path(root)) as store:
-        _check_manifest(store, embedder, edge_source, sub_size, allow_rewrite=not changed_only)
+        _check_manifest(
+            store, embedder, edge_source, sub_size, size_tokens, overlap, allow_rewrite=full
+        )
         dim = embedder.dimension()
         store.set_meta("embedding_model", embedder.model)
         store.set_meta("embedding_dim", str(dim))
         store.set_meta("edge_source", edge_source)
         store.set_meta("subchunk_size_tokens", str(sub_size))
+        store.set_meta("chunk_size_tokens", str(size_tokens))
+        store.set_meta("chunk_overlap", str(overlap))
 
-        current = {
-            rel: hash_file(root / rel)
-            for rel in discover_files(
-                root,
-                cfg.get("corpus.include"),
-                cfg.get("corpus.exclude"),
-                cfg.get("corpus.respect_gitignore"),
-            )
-        }
+        current = _discover_hashes(root, cfg)
         known = store.get_file_hashes()
 
-        if changed_only:
-            to_index = [p for p, h in current.items() if known.get(p) != h]
-            to_delete = [p for p in known if p not in current]
-            index = _open_vectors(root, dim)
-        else:
+        if full:
             to_index, to_delete = list(current), []
             for p in known:
                 store.delete_file(p)
             vectors_path(root).unlink(missing_ok=True)
             index = _open_vectors(root, dim)
+        else:
+            to_index = [p for p, h in current.items() if known.get(p) != h]
+            to_delete = [p for p in known if p not in current]
+            index = _open_vectors(root, dim)
 
         stats = IndexStats(files_unchanged=len(current) - len(to_index))
 
@@ -80,8 +78,6 @@ def run_index(root: Path, cfg: Config, embedder: Embedder, *, changed_only: bool
             stats.files_deleted += 1
             stats.chunks_deleted += len(removed)
 
-        size_tokens = cfg.get("chunking.size_tokens")
-        overlap = cfg.get("chunking.overlap")
         new_ids: list[int] = []
         for path in sorted(to_index):
             if path in known:  # changed file: drop old chunks first
@@ -134,9 +130,21 @@ def run_index(root: Path, cfg: Config, embedder: Embedder, *, changed_only: bool
 
 
 def _check_manifest(
-    store: Store, embedder: Embedder, edge_source: str, sub_size: int, *, allow_rewrite: bool
+    store: Store,
+    embedder: Embedder,
+    edge_source: str,
+    sub_size: int,
+    size_tokens: int,
+    overlap: float,
+    *,
+    allow_rewrite: bool,
 ) -> None:
-    checks = [("embedding_model", embedder.model), ("edge_source", edge_source)]
+    checks = [
+        ("embedding_model", embedder.model),
+        ("edge_source", edge_source),
+        ("chunk_size_tokens", str(size_tokens)),
+        ("chunk_overlap", str(overlap)),
+    ]
     if edge_source == "subchunk":  # size only shapes the graph when subchunk edges are on
         checks.append(("subchunk_size_tokens", str(sub_size)))
     for key, current in checks:
@@ -144,10 +152,36 @@ def _check_manifest(
         if prev and prev != str(current) and not allow_rewrite:
             raise ManifestMismatchError(
                 f"index was built with {key}={prev!r} but config says {current!r}; "
-                "run a full `ragx-cli index` to rebuild"
+                "run `ragx-cli index --full` to rebuild"
             )
 
 
+def _discover_hashes(root: Path, cfg: Config) -> dict[str, str]:
+    return {
+        rel: hash_file(root / rel)
+        for rel in discover_files(
+            root,
+            cfg.get("corpus.include"),
+            cfg.get("corpus.exclude"),
+            cfg.get("corpus.respect_gitignore"),
+        )
+    }
+
+
+def corpus_drift(root: Path, cfg: Config) -> dict[str, int]:
+    """Diff the corpus on disk against the stored file hashes. Read-only."""
+    current = _discover_hashes(root, cfg)
+    known: dict[str, str] = {}
+    if db_path(root).exists():
+        with Store(db_path(root)) as store:
+            known = store.get_file_hashes()
+    return {
+        "new": sum(1 for p in current if p not in known),
+        "changed": sum(1 for p, h in current.items() if p in known and known[p] != h),
+        "deleted": sum(1 for p in known if p not in current),
+    }
+
+
 def _unit(vec: Sequence[float]) -> list[float]:
     norm = math.sqrt(sum(v * v for v in vec)) or 1.0
     return [v / norm for v in vec]
@@ -193,7 +227,7 @@ def _subchunk_edge_fn(
     if len(chunk_ids) != store.chunk_count():
         raise RagxError(
             "sub-chunk vectors missing for some chunks (index predates graph.edge_source="
-            "'subchunk'); run a full `ragx-cli index` to rebuild"
+            "'subchunk'); run `ragx-cli index --full` to rebuild"
         )
     chunk_vectors = dict(zip(chunk_ids, index.get_vectors(chunk_ids)))
 

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/src/ragx/core/query.py

@@ -42,7 +42,7 @@ def _check_model(store: Store, embedder: Embedder) -> None:
     if built_with != embedder.model:
         raise RagxError(
             f"index built with {built_with!r}, config says {embedder.model!r}; "
-            "run `ragx-cli index` to rebuild"
+            "run `ragx-cli index --full` to rebuild"
         )
 
 

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/src/ragx/providers/st_reranker.py

@@ -2,10 +2,36 @@
 
 from __future__ import annotations
 
+import logging
+import sys
 from typing import Sequence
 
 from ragx.core.errors import RagxError
 
+_MANUAL_INSTALL_HINT = (
+    "downloading the reranker model from huggingface.co is failing — to install the "
+    "model manually, use a mirror, or disable reranking, see "
+    "https://github.com/alber70g/ragx-cli#reranker-on-restricted-networks-no-huggingfaceco"
+)
+
+
+class _DownloadHintHandler(logging.Handler):
+    """Emits the manual-install hint once, on the first huggingface_hub retry warning."""
+
+    def __init__(self) -> None:
+        super().__init__(level=logging.WARNING)
+        self._fired = False
+
+    def emit(self, record: logging.LogRecord) -> None:
+        if self._fired:
+            return
+        message = record.getMessage().lower()
+        if "retry" in message or "thrown while requesting" in message:
+            self._fired = True
+            # logging drops re-entrant calls made while a record is being
+            # handled, so the hint must go to stderr directly.
+            print(f"WARNING: {_MANUAL_INSTALL_HINT}", file=sys.stderr)
+
 
 class STReranker:
     """Reranker implementing providers.base.Reranker via a sentence-transformers CrossEncoder."""
@@ -20,6 +46,9 @@ class STReranker:
                 "sentence-transformers not installed — install with: uv tool install ragx-cli --with ragx-cli[rerank]"
             ) from exc
         self.model = model
+        hint = _DownloadHintHandler()
+        hub_logger = logging.getLogger("huggingface_hub")
+        hub_logger.addHandler(hint)
         try:
             self._encoder = CrossEncoder(model)
         except Exception as exc:
@@ -30,6 +59,8 @@ class STReranker:
                 "reranking (rerank.enabled false); see README 'Reranker on restricted "
                 "networks (no huggingface.co)'"
             ) from exc
+        finally:
+            hub_logger.removeHandler(hint)
 
     def score(self, query: str, texts: Sequence[str]) -> list[float]:
         pairs = [[query, text] for text in texts]

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/tests/test_cli.py

@@ -174,6 +174,7 @@ def test_status_json_shape_and_stdout_purity(tmp_path, monkeypatch):
     assert doc["files"] == 0
     assert doc["chunks"] == 0
     assert doc["edges"] == 0
+    assert doc["drift"] == {"new": 0, "changed": 0, "deleted": 0}
 
 
 def test_status_human_readable(tmp_path, monkeypatch):
@@ -186,6 +187,27 @@ def test_status_human_readable(tmp_path, monkeypatch):
     assert "embedding:" in result.stdout
 
 
+def test_status_reports_drift(tmp_path, monkeypatch):
+    runner.invoke(app, ["init", str(tmp_path)])
+    (tmp_path / "a.md").write_text("# A\n\nSome text.\n")
+    monkeypatch.chdir(tmp_path)
+
+    result = runner.invoke(app, ["status", "--json"])
+    doc = json.loads(result.stdout)
+    assert doc["drift"] == {"new": 1, "changed": 0, "deleted": 0}
+
+    result = runner.invoke(app, ["status"])
+    assert "drift: 1 new, 0 changed, 0 deleted" in result.stdout
+    assert "ragx-cli index" in result.stdout
+
+
+def test_index_changed_and_full_mutually_exclusive(tmp_path, monkeypatch):
+    runner.invoke(app, ["init", str(tmp_path)])
+    monkeypatch.chdir(tmp_path)
+    result = runner.invoke(app, ["index", "--changed", "--full"])
+    assert result.exit_code == 2
+
+
 def test_status_without_init(tmp_path, monkeypatch):
     monkeypatch.chdir(tmp_path)
     result = runner.invoke(app, ["status"])

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/tests/test_integration.py

@@ -10,7 +10,7 @@ import pytest
 from ragx.core.config import Config, db_path, write_default_config
 from ragx.core.errors import ManifestMismatchError
 from ragx.core.store import Store
-from ragx.core.indexer import run_index
+from ragx.core.indexer import corpus_drift, run_index
 from ragx.core.query import QueryOptions, run_query, to_files_json, to_query_json
 
 
@@ -74,15 +74,15 @@ def test_incremental_reindex(tmp_path):
     emb = FakeEmbedder()
     run_index(tmp_path, cfg, emb)
 
-    # unchanged corpus: nothing re-processed
-    stats = run_index(tmp_path, cfg, emb, changed_only=True)
+    # unchanged corpus: nothing re-processed (incremental is the default)
+    stats = run_index(tmp_path, cfg, emb)
     assert stats.files_indexed == 0
     assert stats.files_unchanged == 3
 
     # edit one file, delete another
     (tmp_path / "pets.md").write_text("# Pets\n\nDogs bark. Hamsters run on wheels.\n")
     (tmp_path / "space.md").unlink()
-    stats = run_index(tmp_path, cfg, emb, changed_only=True)
+    stats = run_index(tmp_path, cfg, emb)
     assert stats.files_indexed == 1
     assert stats.files_deleted == 1
 
@@ -154,7 +154,7 @@ def test_subchunk_incremental_reindex(tmp_path):
 
     (tmp_path / "pets.md").write_text("# Pets\n\nDogs bark. Hamsters run on wheels.\n")
     (tmp_path / "space.md").unlink()
-    stats = run_index(tmp_path, cfg, emb, changed_only=True)
+    stats = run_index(tmp_path, cfg, emb)
     assert stats.files_indexed == 1
     assert stats.files_deleted == 1
 
@@ -166,7 +166,7 @@ def test_subchunk_incremental_reindex(tmp_path):
     assert out.results[0].chunk.file_path == "pets.md"
 
 
-def test_changed_only_fails_loud_on_edge_source_flip(tmp_path):
+def test_incremental_fails_loud_on_edge_source_flip(tmp_path):
     make_corpus(tmp_path)
     write_default_config(tmp_path)
     cfg = Config.load(tmp_path)
@@ -175,7 +175,53 @@ def test_changed_only_fails_loud_on_edge_source_flip(tmp_path):
 
     cfg.set("graph.edge_source", "subchunk")
     with pytest.raises(ManifestMismatchError):
-        run_index(tmp_path, cfg, emb, changed_only=True)
+        run_index(tmp_path, cfg, emb)
+    stats = run_index(tmp_path, cfg, emb, full=True)  # explicit rebuild clears the mismatch
+    assert stats.files_indexed == 3
+
+
+def test_incremental_fails_loud_on_chunking_change(tmp_path):
+    make_corpus(tmp_path)
+    write_default_config(tmp_path)
+    cfg = Config.load(tmp_path)
+    emb = FakeEmbedder()
+    run_index(tmp_path, cfg, emb)
+
+    cfg.set("chunking.size_tokens", "400")
+    with pytest.raises(ManifestMismatchError):
+        run_index(tmp_path, cfg, emb)
+    stats = run_index(tmp_path, cfg, emb, full=True)
+    assert stats.files_indexed == 3
+
+
+def test_corpus_drift(tmp_path):
+    make_corpus(tmp_path)
+    write_default_config(tmp_path)
+    cfg = Config.load(tmp_path)
+    emb = FakeEmbedder()
+
+    assert corpus_drift(tmp_path, cfg) == {"new": 3, "changed": 0, "deleted": 0}
+    run_index(tmp_path, cfg, emb)
+    assert corpus_drift(tmp_path, cfg) == {"new": 0, "changed": 0, "deleted": 0}
+
+    (tmp_path / "pets.md").write_text("# Pets\n\nParrots repeat words.\n")
+    (tmp_path / "space.md").unlink()
+    (tmp_path / "gardens.md").write_text("# Gardens\n\nTomatoes ripen in summer sun.\n")
+    assert corpus_drift(tmp_path, cfg) == {"new": 1, "changed": 1, "deleted": 1}
+
+
+def test_drift_reflects_exclude_change(tmp_path):
+    make_corpus(tmp_path)
+    write_default_config(tmp_path)
+    cfg = Config.load(tmp_path)
+    emb = FakeEmbedder()
+    run_index(tmp_path, cfg, emb)
+
+    cfg.set("corpus.exclude", "cooking.md")
+    assert corpus_drift(tmp_path, cfg) == {"new": 0, "changed": 0, "deleted": 1}
+    stats = run_index(tmp_path, cfg, emb)  # incremental converges on the new glob set
+    assert stats.files_deleted == 1
+    assert corpus_drift(tmp_path, cfg) == {"new": 0, "changed": 0, "deleted": 0}
 
 
 def test_index_persists_communities(tmp_path):
@@ -210,7 +256,7 @@ def test_changed_reindex_recomputes_communities(tmp_path):
     with Store(db_path(tmp_path)) as store:
         deleted_ids = set(store.chunk_ids_for_file("space.md"))
     (tmp_path / "space.md").unlink()
-    run_index(tmp_path, cfg, emb, changed_only=True)
+    run_index(tmp_path, cfg, emb)
 
     with Store(db_path(tmp_path)) as store:
         members = set()

{ragx_cli-0.3.2 → ragx_cli-0.4.0}/tests/test_providers.py

@@ -200,6 +200,42 @@ def test_make_reranker_model_load_failure_returns_none_and_warns(monkeypatch, ca
     assert "HF_ENDPOINT" in err
 
 
+def test_st_reranker_first_retry_warning_emits_manual_install_hint(monkeypatch, capsys):
+    """The first HF download-retry warning triggers a one-shot README pointer."""
+    import logging
+    import sys
+    import types
+
+    class _RetryingCrossEncoder:
+        def __init__(self, model):
+            hub_log = logging.getLogger("huggingface_hub.utils._http")
+            for _ in range(2):
+                hub_log.warning(
+                    "'_ssl.c:1015: The handshake operation timed out' thrown while "
+                    "requesting HEAD https://huggingface.co/x/resolve/main/modules.json"
+                )
+                hub_log.warning("Retrying in 1s [Retry 1/5].")
+            raise OSError("timed out")
+
+    fake = types.ModuleType("sentence_transformers")
+    fake.CrossEncoder = _RetryingCrossEncoder
+    monkeypatch.setitem(sys.modules, "sentence_transformers", fake)
+
+    from ragx.core.errors import RagxError
+    from ragx.providers.st_reranker import STReranker, _DownloadHintHandler
+
+    with pytest.raises(RagxError):
+        STReranker("BAAI/bge-reranker-v2-m3")
+
+    err = capsys.readouterr().err
+    assert err.count("install the model manually") == 1
+    assert "#reranker-on-restricted-networks" in err
+    assert not any(
+        isinstance(h, _DownloadHintHandler)
+        for h in logging.getLogger("huggingface_hub").handlers
+    )
+
+
 def test_api_key_env_resolved_and_sent(monkeypatch):
     monkeypatch.setenv("RAGX_TEST_KEY", "sk-test-123")
     cfg = _config({"embeddings": {"api_key_env": "RAGX_TEST_KEY"}})