ragx-cli 0.1.0__tar.gz

ragx_cli-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,34 @@
+name: Publish to PyPI
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+      - run: uv sync --group dev
+      - run: uv run pytest -q
+      - run: uv run ruff check src tests
+
+  publish:
+    needs: test
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v6
+      - name: Check tag matches package version
+        run: |
+          VERSION=$(uv version --short)
+          [ "v$VERSION" = "$GITHUB_REF_NAME" ] || {
+            echo "Tag $GITHUB_REF_NAME does not match pyproject version $VERSION" >&2
+            exit 1
+          }
+      - run: uv build
+      - run: uv publish --trusted-publishing always

ragx_cli-0.1.0/.gitignore

@@ -0,0 +1,9 @@
+.venv/
+__pycache__/
+*.pyc
+.pytest_cache/
+.ruff_cache/
+dist/
+.ragx/
+uv.lock
+.claude/worktrees/

ragx_cli-0.1.0/AGENTS.md

@@ -0,0 +1 @@
+@CLAUDE.md

ragx_cli-0.1.0/CLAUDE.md

@@ -0,0 +1,112 @@
+# CLAUDE.md — ragx
+
+## What this is
+
+`ragx` is a local CLI that indexes a corpus of files into a chunk-level **embedding similarity
+graph** and answers queries via vector search + **heat-propagation graph traversal** +
+**cross-encoder reranking**. Indexing is LLM-free (embeddings only); an LLM is optional at
+query time for multi-query/HyDE expansion. Built agent-first: single JSON doc on stdout
+(versioned schemas), logs on stderr, exit codes `0` results / `1` empty / `2` error,
+byte-exact source locations, `--explain` traces justifying every result.
+
+Design plan: `ragx-cli-plan.md` (phases, parameters, risks). Module build specs:
+`CONTRACTS.md` (Phase 0–1) and `CONTRACTS-PHASE23.md` (Phase 2–3) — historical but accurate
+API contracts for every core module. Deferred feature specs live in `docs/`.
+
+## Stack
+
+- Python 3.12, managed with **uv** (`uv sync --group dev`, `uv run pytest`). NOT Bun/TS —
+  deliberate exception, see plan §2.
+- typer (CLI), hnswlib (HNSW vector index, cosine), SQLite (chunks/edges/manifest, WAL),
+  httpx (+respx in tests), xxhash (incremental hashing), pathspec (gitignore globs).
+- Providers speak **OpenAI-compatible HTTP**. Local default: LM Studio at
+  `http://localhost:1234/v1`. `provider="ollama"` auto-switches to `:11434/v1`.
+  Env fallbacks: `OPENAI_BASE_URL` (only while base_url is still the default) and
+  `OPENAI_API_KEY` (only when `api_key_env` is unset); explicit config always wins.
+- Reranker is always local: sentence-transformers CrossEncoder `BAAI/bge-reranker-v2-m3`
+  (optional extra `ragx-cli[rerank]`). LM Studio has NO /v1/rerank endpoint — verified; don't try.
+
+## Layout (core/CLI split is the one architectural rule — MCP server later rides on it)
+
+```
+src/ragx/
+  core/            # all logic lives here, CLI-independent
+    config.py      #   .ragx/config.toml load/save, DEFAULTS dict, find_root/require_root
+    models.py      #   Chunk, ChunkDraft, FileRecord, Edge, ScoredChunk, QueryOutput
+    store.py       #   SQLite: files/chunks/edges/meta; replace_edges normalizes src<dst
+    vectors.py     #   hnswlib wrapper; soft-delete via JSON sidecar; get_vectors for graph
+    discovery.py   #   file walk, root-level .gitignore, binary/size/junk-dir filters, xxh64
+    chunking.py    #   markdown/code/recursive splitters; byte-exact slices; tiny-fragment merge
+    graph.py       #   knn_edges + affected_ids (incremental edge maintenance)
+    traversal.py   #   propagate_heat: max-aggregation, query floor, frontier cap, trace
+    expansion.py   #   one LLM call -> variants + HyDE; NEVER raises (degrades to no-op)
+    fusion.py      #   Reciprocal Rank Fusion
+    scoring.py     #   min-max normalize + alpha/beta/gamma combine (renormalizes w/o rerank)
+    indexer.py     #   run_index: discover->hash->chunk->embed->HNSW+store->kNN edges
+    query.py       #   run_query: expansion->fan-out->RRF->traversal->rerank->combine; JSON serializers
+    eval.py        #   recall@5/@10 + MRR over file-level ranking, injected query_fn
+  providers/       # base.py protocols (Embedder/Generator/Reranker); openai_compat.py;
+                   # st_reranker.py; registry.py factories (env-var + api_key_env resolution)
+  cli/             # thin shells only: app.py (init/status/config + registration),
+                   # pipeline.py (index/query), inspect_cmd.py, eval_cmd.py, output.py
+tests/             # 126 tests; mocked HTTP (respx), FakeEmbedder integration tests, no live network
+```
+
+## Flows
+
+**Index** (`ragx-cli index [--changed]`): discover files (include/exclude globs, root .gitignore,
+skip node_modules/.venv/hidden/binaries/>2MB) → xxhash diff (`--changed` = incremental, default
+= full rebuild) → chunk (~800 tok target, chunks are exact byte slices, sub-100-char fragments
+merge into neighbors) → embed batched → HNSW add → kNN edges per new chunk (k=8, cos ≥ 0.55)
++ recompute edge lists of pre-existing neighbors that appear in new lists.
+
+**Query** (`ragx-cli query "..."`): optional expansion (1 LLM call, strict-JSON parse, graceful
+no-op on failure) → embed all variants, HNSW top-20 each → RRF merge = seeds → heat propagation
+(2 hops, decay 0.5, heat = max not sum, neighbor admitted only if cos(query) ≥ 0.35 floor,
+frontier ≤ 150) → cross-encoder rerank of top-100 shortlist → final =
+0.6·rerank + 0.25·heat + 0.15·vector (each min-max normalized; weights renormalize when a
+stage is off). `--no-expand/--no-graph/--no-rerank` skip stages; `--files-only` ranks files
+by sum of top-3 chunk scores; `--explain` attaches seed→parent→edge_weight→hop traces.
+
+## State & benchmarks (as of 2026-07-10)
+
+Phases 0–3 of the plan are DONE and validated. Eval corpus: Albert's wiki
+(`~/projects/alber70g/notes/wiki`, .md only, 636 files → 1,323 chunks → 5,246 edges); labeled
+queries at `<corpus>/.ragx/queries.jsonl` (18, EN+NL). Results (recall@5 / recall@10 / MRR):
+baseline .833/.833/.593 · graph-only .778/.833/.522 · graph+rerank .759/**.889**/.568 ·
+full .759/**.889**/**.613**. Key finding: graph-only HURTS precision (near-duplicate neighbors
+displace direct hits — parameter sweeps plateau below baseline MRR; corpus property, not a
+tuning miss), but graph+rerank together deliver the recall win — one Dutch query's target is
+unreachable by vector search or rerank-alone, surfaced only via a hop-1 edge then reranked
+19→4. Ship graph+rerank together; `--no-graph --no-rerank` is the explicit fast mode.
+
+## Gotchas
+
+- Ornith (`mlx-community/Ornith-1.0-35B-3bit`, Albert's required expansion LLM) is a
+  *reasoning* model: answers land in `content` only after thinking; expansion uses
+  max_tokens=4096 and takes ~40 s/call. Don't lower it.
+- Changing `embeddings.model` invalidates the index; run_query fails loud on manifest
+  mismatch; full `ragx-cli index` rebuilds.
+- Chunk ids are SQLite AUTOINCREMENT rowids AND hnswlib labels — never reused after delete.
+  hnswlib can't enumerate deletions, hence the `vectors.hnsw.meta.json` sidecar.
+- `Config.set` rejects keys not in `DEFAULTS` — add new config keys to DEFAULTS first.
+- Tests must not hit the network; one provider test self-skips when sentence-transformers
+  is installed. Pyright import errors in editors = interpreter not set to `.venv`.
+- When running ragx against the wiki corpus: `cd` there, then
+  `uv run --project /Users/albert/projects/alber70g/ragx ragx-cli …` — and NEVER `git add`/commit
+  from that directory (it's Albert's personal notes repo; `.ragx/` is gitignored there).
+
+## What's next / deferred
+
+- Phase 4 (plan §5): Leiden communities over the edge list, `query --global`.
+- Phase 5: MCP server as a second thin shell over `ragx.core` (split already enforced).
+- Temporal weighting: full decided spec in `docs/feature-temporal-weighting.md` — opt-in
+  `--since/--until/--temporal recent|oldest`, date cascade filename/frontmatter → git → mtime,
+  schema v2 migration. Build only after it can be measured against the eval baseline.
+- README packaging claims (`uvx ragx-cli`; PyPI name ragx-cli, plain `ragx` is name-blocked) are aspirational — not yet published to PyPI.
+
+## Dev loop
+
+`uv sync --group dev --extra rerank` · `uv run pytest -q` (126 pass, ~5 s) ·
+`uv run ruff check src tests` · file soft cap ~150 lines · expected failures raise `RagxError`
+(CLI maps to exit 2). Live smoke: LM Studio must be running with the configured embedding model.

ragx_cli-0.1.0/CONTRACTS-PHASE23.md

@@ -0,0 +1,166 @@
+# ragx module contracts — Phase 2–3 build
+
+Same ground rules as CONTRACTS.md (read its "Global rules" and "Report protocol" — they apply
+verbatim). Own ONLY your listed files; never modify shared/core files or other modules' files.
+Shared APIs you may rely on (already implemented — read them first):
+`core/models.py`, `core/errors.py`, `core/config.py`, `core/store.py` (esp. `neighbors`,
+`replace_edges`), `core/vectors.py` (esp. `search`, `get_vectors`), `core/query.py`
+(`QueryOptions`, `QueryOutput`, `run_query`, `to_files_json`), `providers/base.py`,
+`cli/output.py`. NOTE: `cli/app.py` wiring is integration-owned — CLI modules expose
+`register(app)` and their tests build a fresh `typer.Typer()` and call `register` themselves.
+
+---
+
+## Module F — kNN edge construction
+
+**Files:** `src/ragx/core/graph.py`, `tests/test_graph.py`
+
+```python
+def knn_edges(index: VectorIndex, ids: Sequence[int], k: int, min_sim: float) -> dict[int, list[tuple[int, float]]]:
+    # for each id: search its own stored vector (index.get_vectors) for k+1 hits,
+    # drop self, keep sim >= min_sim, cap at k. Returns id -> [(neighbor_id, sim), ...] sim-desc.
+def affected_ids(new_edges: dict[int, list[tuple[int, float]]]) -> set[int]:
+    # every neighbor id referenced in new_edges values, minus new_edges' own keys —
+    # the incremental-maintenance set whose edge lists the indexer must recompute.
+```
+
+Pure functions over VectorIndex; no Store access. Batch `get_vectors` in one call.
+Tests: constructed unit vectors with known cosines — self excluded, min_sim filter, k cap,
+sim-desc order, affected_ids correctness, empty ids -> {}.
+
+---
+
+## Module G — heat-propagation traversal
+
+**Files:** `src/ragx/core/traversal.py`, `tests/test_traversal.py`
+
+```python
+@dataclass
+class TraversalResult:
+    heat: dict[int, float]   # node -> final heat
+    trace: dict[int, dict]   # node -> {"seed": int, "parent": int | None, "edge_weight": float, "hop": int}
+
+def propagate_heat(
+    seeds: dict[int, float],
+    neighbors_fn: Callable[[int], list[tuple[int, float]]],
+    query_sim_fn: Callable[[Sequence[int]], dict[int, float]],   # BATCH: ids -> cosine vs query
+    *, hops: int = 2, decay: float = 0.5, query_floor: float = 0.35, max_frontier: int = 150,
+) -> TraversalResult:
+```
+
+Semantics (exact):
+- Init: every seed gets heat = its seed score, trace {"seed": itself, "parent": None,
+  "edge_weight": 0.0, "hop": 0}. Seeds are exempt from the query floor.
+- For hop h = 1..hops: frontier = nodes whose heat was set/improved in the previous hop
+  (hop 1: the seeds). For each frontier node u and each (v, w) from neighbors_fn(u):
+  contribution = heat[u] * w * decay. A non-seed v is admissible only if its query similarity
+  >= query_floor — batch query_sim_fn over all first-seen candidates of the hop, cache results.
+  Aggregation is MAX, not sum: heat[v] = max(existing, contribution); on improvement, v's trace
+  becomes {"seed": trace[u]["seed"], "parent": u, "edge_weight": w, "hop": h}.
+- Frontier cap: after each hop keep only the top `max_frontier` improved nodes by heat as the
+  next frontier (others keep their heat but don't propagate). Ties break by lower node id.
+- A seed's heat never decreases below its seed score. hops=0 -> seeds only.
+- Deterministic for identical inputs; no randomness.
+
+Tests (hand-built adjacency dicts, no Store): max-not-sum at a hub node; per-hop decay values
+exact (seed 1.0, w 0.8 -> hop1 0.4, hop2 chain 0.16); query-floor exclusion (and that an excluded
+node also doesn't relay); frontier cap; trace parent-chain always reaches a seed; hops=0.
+
+---
+
+## Module H — expansion, RRF fusion, score combination
+
+**Files:** `src/ragx/core/expansion.py`, `src/ragx/core/fusion.py`, `src/ragx/core/scoring.py`,
+`tests/test_expansion.py`, `tests/test_fusion.py`
+
+```python
+# expansion.py
+@dataclass
+class Expansion:
+    variants: list[str]   # reformulations/sub-queries, excludes the original
+    hyde: str | None
+
+def expand_query(gen: Generator, query: str, *, variants: int = 3, hyde: bool = True) -> Expansion:
+    # ONE gen.generate() call. Prompt demands strict JSON: {"queries": [...], "hyde": "..."}
+    # (omit hyde from the prompt when hyde=False). Parse defensively: strip markdown fences,
+    # take first '{' .. last '}'. On ANY failure (generator exception, bad JSON, wrong shape):
+    # log a warning to stderr and return Expansion([], None) — NEVER raise. Clamp to `variants`
+    # items; drop empty strings and case-insensitive duplicates of the original query.
+
+# fusion.py
+def rrf(rankings: Sequence[Sequence[int]], k: int = 60) -> dict[int, float]:
+    # Reciprocal Rank Fusion: score(d) = sum over rankings containing d of 1/(k + rank), rank 1-based.
+
+# scoring.py
+def normalize(scores: dict[int, float]) -> dict[int, float]:
+    # min-max to [0,1]; empty -> {}; all-equal -> all 1.0
+def combine(candidates: Iterable[int], vector: dict[int, float], heat: dict[int, float],
+            rerank: dict[int, float] | None, *, alpha: float, beta: float, gamma: float) -> dict[int, float]:
+    # missing candidate scores are 0.0 raw; each component min-max normalized over candidates first.
+    # with rerank:  alpha*r + beta*h + gamma*v
+    # without rerank: weights renormalized -> (beta/(beta+gamma))*h + (gamma/(beta+gamma))*v
+```
+
+Tests: expansion with a mock Generator returning valid JSON / fenced JSON / garbage / raising;
+rrf hand-computed values and stable ordering; normalize edge cases; combine with and without
+rerank (weight renormalization exact).
+
+---
+
+## Module I — eval harness
+
+**Files:** `src/ragx/core/eval.py`, `src/ragx/cli/eval_cmd.py`, `tests/test_eval.py`
+
+`queries.jsonl`: one JSON object per line: `{"query": str, "relevant_files": [str, ...]}`
+(paths relative to corpus root). Blank lines skipped; malformed line -> RagxError naming the line number.
+
+```python
+# eval.py — decoupled from the pipeline via an injected callable
+QueryFn = Callable[[str, QueryOptions], QueryOutput]
+
+def load_queries(path: Path) -> list[dict]
+def evaluate(queries: list[dict], configs: list[tuple[str, QueryOptions]], query_fn: QueryFn,
+             *, top: int = 10) -> dict:
+    # For each config and query: out = query_fn(query, opts); rank FILES via
+    # ragx.core.query.to_files_json(out)["files"] order. Metrics per config, averaged over queries:
+    #   recall_at_5, recall_at_10  (fraction of relevant_files present in top k files)
+    #   mrr                        (reciprocal rank of the FIRST relevant file, 0 if absent)
+    # Returns {"schema": "ragx.eval.v1", "top": top, "query_count": n,
+    #          "configs": [{"name", "recall_at_5", "recall_at_10", "mrr"}, ...]}
+    # plus per-query detail under "queries": [{"query", "per_config": {name: {"hit_ranks": [...]}}}]
+
+# eval_cmd.py
+def register(app: typer.Typer) -> None: ...
+# ragx eval QUERIES_FILE [--json] [--top N] [--configs csv]
+# built-in configs: baseline (expand/graph/rerank all off), graph (only graph on),
+#                   full (all on). Default: run all three.
+# Constructs QueryOptions accordingly and query_fn from run_query + make_embedder(cfg) —
+# keep this thin; integration may extend it. Human output: aligned table of config x metrics.
+# Exit codes: 0 ok, 2 on missing/malformed file.
+```
+
+Tests: metric math against a fake query_fn with canned QueryOutput fixtures (verify recall@5/10
+and MRR by hand-computed values); load_queries error cases; CLI registered on a fresh Typer with
+a monkeypatched evaluate/query_fn (no embeddings, no network).
+
+---
+
+## Module J — inspect commands
+
+**Files:** `src/ragx/cli/inspect_cmd.py`, `tests/test_inspect.py`
+
+```python
+def register(app: typer.Typer) -> None:   # app.add_typer(inspect_app, name="inspect")
+# ragx inspect chunk ID [--json]      -> chunk text, file, line/byte range + its edges
+#                                        (neighbor id, weight, neighbor's file path)
+# ragx inspect file PATH [--json]     -> file record + its chunks (id, line range, first 80 chars)
+# ragx inspect neighbors ID [--json]  -> store.neighbors(ID) enriched with each neighbor's file/lines
+```
+
+All read-only over Store (find root via require_root; no embeddings, no vectors file needed).
+JSON schemas: `ragx.inspect.chunk.v1`, `ragx.inspect.file.v1`, `ragx.inspect.neighbors.v1`.
+Exit codes: unknown chunk id or file path -> stderr message, exit 2; entity exists but has no
+edges/chunks -> exit 1 (success-but-empty), matching the CLI conventions.
+Tests: CliRunner on a fresh Typer + `register`; build a real Store in tmp_path with a few files/
+chunks/edges directly (no embedding provider); cover all three subcommands, --json shape,
+exit codes 0/1/2.

ragx_cli-0.1.0/CONTRACTS.md

@@ -0,0 +1,229 @@
+# ragx module contracts (Phase 0–1 build)
+
+Ground truth for the module-parallel build. Each module owns exactly the files listed for it.
+**Do not create or edit files outside your module** — shared types live in `core/models.py`,
+provider protocols in `providers/base.py`, config in `core/config.py`, errors in `core/errors.py`
+(all already written; read them first, import from them, never modify them).
+
+Global rules:
+
+- Python 3.12, `uv sync --group dev` to set up, `uv run pytest tests/<your tests> -q` must pass,
+  `uv run ruff check <your files>` must be clean.
+- Soft cap ~150 lines per file; split if you blow past it.
+- No live network in tests (mock HTTP with `respx`; models/backends must not be required).
+- Raise `RagxError` (or a subclass) for expected failures; let bugs raise naturally.
+
+---
+
+## Module A — SQLite store
+
+**Files:** `src/ragx/core/store.py`, `tests/test_store.py`
+
+Single-file SQLite store for files, chunks, edges, and a key-value manifest.
+
+```python
+class Store:
+    def __init__(self, path: Path): ...   # open/create, apply schema, WAL mode, foreign_keys ON
+    def close(self) -> None: ...           # also __enter__/__exit__
+    # manifest (embedding model, dim, schema info)
+    def get_meta(self, key: str) -> str | None: ...
+    def set_meta(self, key: str, value: str) -> None: ...
+    # files
+    def get_file_hashes(self) -> dict[str, str]: ...      # path -> content_hash
+    def upsert_file(self, rec: FileRecord) -> None: ...
+    def delete_file(self, path: str) -> list[int]: ...    # cascades chunks + incident edges; returns removed chunk ids
+    def file_count(self) -> int: ...
+    # chunks
+    def insert_chunks(self, file_path: str, drafts: Sequence[ChunkDraft]) -> list[int]: ...
+    def get_chunks(self, ids: Sequence[int]) -> list[Chunk]: ...   # input order preserved; unknown ids skipped
+    def chunk_ids_for_file(self, path: str) -> list[int]: ...
+    def all_chunk_ids(self) -> list[int]: ...
+    def chunk_count(self) -> int: ...
+    # edges (undirected; store normalized src < dst)
+    def replace_edges(self, src: int, neighbors: Sequence[tuple[int, float]]) -> None: ...
+    def neighbors(self, chunk_id: int) -> list[tuple[int, float]]: ...  # both directions, weight desc
+    def delete_edges_incident(self, ids: Sequence[int]) -> None: ...
+    def edge_count(self) -> int: ...
+```
+
+Schema v1 (set `PRAGMA user_version = 1`):
+
+```sql
+CREATE TABLE meta(key TEXT PRIMARY KEY, value TEXT NOT NULL);
+CREATE TABLE files(path TEXT PRIMARY KEY, content_hash TEXT NOT NULL,
+                   mtime REAL NOT NULL, chunk_count INTEGER NOT NULL);
+CREATE TABLE chunks(id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    file_path TEXT NOT NULL REFERENCES files(path) ON DELETE CASCADE,
+                    text TEXT NOT NULL, byte_start INTEGER NOT NULL, byte_end INTEGER NOT NULL,
+                    line_start INTEGER NOT NULL, line_end INTEGER NOT NULL);
+CREATE INDEX chunks_file ON chunks(file_path);
+CREATE TABLE edges(src INTEGER NOT NULL, dst INTEGER NOT NULL, weight REAL NOT NULL,
+                   PRIMARY KEY(src, dst)) WITHOUT ROWID;
+CREATE INDEX edges_dst ON edges(dst);
+```
+
+`AUTOINCREMENT` is deliberate: chunk ids are HNSW labels and must never be reused after deletion.
+`insert_chunks` requires the file row to exist first (upsert_file before insert_chunks).
+
+Tests must cover: roundtrip of every entity; delete_file cascade removes chunks and edges touching
+them (both src and dst side); replace_edges normalization + dedup (inserting (5,3,w) then querying
+neighbors(3) and neighbors(5) both work); id monotonicity after delete + reinsert; meta roundtrip.
+
+---
+
+## Module B — Providers
+
+**Files:** `src/ragx/providers/openai_compat.py`, `src/ragx/providers/st_reranker.py`,
+`src/ragx/providers/registry.py`, `tests/test_providers.py`
+
+```python
+# openai_compat.py — used for LM Studio (localhost:1234/v1), Ollama's /v1, and OpenAI itself
+class OpenAICompatEmbedder:   # implements providers.base.Embedder
+    def __init__(self, base_url: str, model: str, doc_prefix: str = "", query_prefix: str = "",
+                 api_key: str | None = None, batch_size: int = 32, timeout: float = 120.0): ...
+    # POST {base_url}/embeddings with {"model", "input": [prefixed texts]}
+    # batches of batch_size; 3 retries with exponential backoff on 5xx/connect errors
+    # dimension(): lazily embeds one probe string, caches the length
+    # empty input list -> [] without any HTTP call
+
+class OpenAICompatGenerator:  # implements providers.base.Generator
+    def __init__(self, base_url: str, model: str, api_key: str | None = None,
+                 timeout: float = 300.0): ...
+    # POST {base_url}/chat/completions, messages=[system, user], temperature=0.3
+    # returns choices[0].message.content
+
+# st_reranker.py — lazy `import sentence_transformers` inside __init__;
+# ImportError -> RagxError("... install with: uv pip install 'ragx[rerank]'")
+class STReranker:             # implements providers.base.Reranker
+    def __init__(self, model: str): ...
+    def score(self, query: str, texts: Sequence[str]) -> list[float]: ...  # CrossEncoder.predict
+
+# registry.py — factories reading Config (see core/config.py DEFAULTS for keys)
+def make_embedder(cfg: Config) -> Embedder: ...
+    # embeddings.provider: "openai" -> OpenAICompatEmbedder with [embeddings] settings
+    #                      "ollama" -> same class, base_url defaulting to http://localhost:11434/v1
+    # unknown provider -> RagxError
+def make_generator(cfg: Config) -> Generator | None: ...  # None when expansion.enabled is false
+def make_reranker(cfg: Config) -> Reranker | None: ...
+    # None when rerank.enabled is false; on RagxError from missing extra: log warning to stderr, return None
+```
+
+Tests: respx-mocked embeddings (prefix application, batching — e.g. batch_size=2 with 5 inputs →
+3 requests; order preserved across batches), retry on a 500 then success, dimension() caching
+(one probe call), generator payload shape, registry dispatch incl. disabled → None. No real network.
+
+---
+
+## Module C — Discovery + chunking
+
+**Files:** `src/ragx/core/discovery.py`, `src/ragx/core/chunking.py`,
+`tests/test_discovery.py`, `tests/test_chunking.py`
+
+```python
+# discovery.py
+def discover_files(root: Path, include: list[str], exclude: list[str],
+                   respect_gitignore: bool = True) -> list[str]: ...
+    # sorted relative POSIX paths; include/exclude are gitwildmatch globs (use pathspec)
+    # always skipped: .ragx/, .git/, hidden dirs, binaries (NUL byte in first 8KB), files > 2MB
+    # gitignore: root-level .gitignore only (MVP; document the limitation)
+def hash_file(path: Path) -> str: ...   # xxhash.xxh64 hexdigest of file bytes
+
+# chunking.py
+def chunk_text(text: str, path: str, size_tokens: int = 800, overlap: float = 0.15) -> list[ChunkDraft]: ...
+```
+
+Chunking rules:
+
+- Token estimate: 1 token ≈ 4 chars → target = size_tokens*4 chars, hard max 1.5× target.
+- Dispatch on extension: `.md`/`.markdown` → markdown splitter; code
+  (`.py .ts .js .tsx .jsx .go .rs .java .rb`) → code splitter; anything else → recursive splitter.
+- Markdown: split at heading lines (`^#{1,6} `), heading stays with its section; adjacent small
+  sections merge until ≈target; oversized sections fall through to the recursive splitter.
+- Code: split at top-level `def`/`class`/`function`/`fn`/`func` boundaries (regex, no tree-sitter
+  in MVP); oversized pieces fall through to the recursive splitter.
+- Recursive: split by separators ["\n\n", "\n", ". ", " "] to fit max, with `overlap` fraction of
+  the previous chunk's tail prepended as range overlap.
+- **Invariant (test this):** every draft's `text` == `text_bytes[byte_start:byte_end].decode("utf-8")`
+  where text_bytes = the original file's utf-8 bytes — chunks are exact slices of the source
+  (overlap = overlapping ranges, never synthesized text). line_start/line_end are 1-based inclusive.
+- Empty/whitespace-only files → []. Never emit empty-text drafts.
+
+Tests: the slice invariant on multi-byte (emoji/accented) content; markdown heading split + small-
+section merging; oversized-section fallback; overlap ranges actually overlap; discovery honors
+include/exclude/.gitignore, skips binaries and .ragx.
+
+---
+
+## Module D — HNSW vector index
+
+**Files:** `src/ragx/core/vectors.py`, `tests/test_vectors.py`
+
+```python
+class VectorIndex:
+    @classmethod
+    def create(cls, path: Path, dim: int, capacity: int = 2048) -> VectorIndex: ...
+    @classmethod
+    def load(cls, path: Path, dim: int) -> VectorIndex: ...   # missing file -> RagxError
+    def add(self, ids: Sequence[int], vectors: Sequence[Sequence[float]]) -> None: ...
+    def search(self, vector: Sequence[float], k: int) -> list[tuple[int, float]]: ...
+    def mark_deleted(self, ids: Sequence[int]) -> None: ...
+    def save(self) -> None: ...
+    @property
+    def count(self) -> int: ...   # active (non-deleted) element count
+```
+
+- hnswlib `space="cosine"`, M=16, ef_construction=200, query-time ef = max(100, 2k).
+- `search` returns (id, cosine_similarity) sorted desc; similarity = 1 − hnswlib distance,
+  clamped to [0,1]; k silently clamped to active count; empty index → [].
+- `add` auto-resizes (double capacity) when full. ids are arbitrary non-reused ints (SQLite rowids).
+- Deleted elements never appear in results (hnswlib `mark_deleted`); deleting unknown ids is a no-op.
+- save/load roundtrip preserves search results and deletions.
+
+Tests: nearest-neighbor sanity on constructed vectors (a query nearest to its own cluster),
+resize past initial capacity, delete-then-search exclusion, save/load roundtrip, empty-index search.
+
+---
+
+## Module E — CLI shell
+
+**Files:** `src/ragx/cli/app.py`, `src/ragx/cli/output.py`, `tests/test_cli.py`
+
+typer app exposing `init`, `status`, `config`. (`index`/`query` are wired in a later integration
+pass — the app must be importable and testable without them.)
+
+Conventions (these ARE the product for agent callers — be exact):
+
+- `--json` on `status`: emit exactly one JSON document to stdout, nothing else on stdout.
+- All logging/diagnostics → stderr (configure `logging` to stderr in `main()`).
+- Exit codes: 0 = success with results, 1 = success but empty result, 2 = error.
+  Map `RagxError` → stderr message + exit 2 (no traceback); unexpected exceptions may traceback.
+
+```python
+# output.py
+def emit_json(doc: dict) -> None: ...           # json.dumps to stdout with trailing newline
+def fail(msg: str, code: int = 2) -> NoReturn: ...
+
+# app.py
+app = typer.Typer(...)
+# ragx init [path]      -> write_default_config at path (default cwd); error (exit 2) if .ragx exists;
+#                          prints created config path
+# ragx status [--json]  -> root path, embedding model/provider, counts (files/chunks/edges) read via
+#                          Store if index.db exists else zeros, schema "ragx.status.v1" in JSON mode
+# ragx config get KEY / ragx config set KEY VALUE  -> Config.get / Config.set + save
+def main() -> None: ...   # entry point declared in pyproject [project.scripts]
+```
+
+Module E may import Store (module A) **only** inside `status` behind a
+`try/except ImportError` fallback to zeros, so the module builds independently; the import will
+resolve after integration. Tests: use `typer.testing.CliRunner` + tmp_path; cover init (fresh +
+already-initialized), config get/set roundtrip persisted to disk, status --json shape + stdout
+purity (stdout parses as JSON), exit codes incl. `config get bogus.key` → 2.
+
+---
+
+## Report protocol (all modules)
+
+When done: `git add -A && git commit -m "<module>: <summary>"` in your worktree, then report:
+branch (`git rev-parse --abbrev-ref HEAD`), commit SHA, pytest pass/fail counts, ruff status,
+and any deviations from this contract. On any blocker: STOP — report the exact error output and
+what you attempted instead of improvising around the contract.

ragx_cli-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Albert Groothedde
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.