ragx-cli 0.7.0__tar.gz → 0.9.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/CHANGELOG.md +57 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/CLAUDE.md +56 -25
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/PKG-INFO +118 -34
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/README.md +117 -33
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/pyproject.toml +1 -1
- ragx_cli-0.9.0/research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md +111 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/skills/ragx-cli/SKILL.md +19 -12
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/app.py +10 -1
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/eval_cmd.py +2 -4
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/pipeline.py +44 -10
- ragx_cli-0.9.0/src/ragx/cli/web_cmd.py +47 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/config.py +9 -4
- ragx_cli-0.9.0/src/ragx/core/context.py +91 -0
- ragx_cli-0.9.0/src/ragx/core/corpus_globs.py +29 -0
- ragx_cli-0.9.0/src/ragx/core/graph_export.py +70 -0
- ragx_cli-0.9.0/src/ragx/core/grouping.py +31 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/indexer.py +2 -1
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/models.py +12 -2
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/query.py +59 -40
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/store.py +11 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/traversal.py +22 -0
- ragx_cli-0.9.0/src/ragx/web/__init__.py +2 -0
- ragx_cli-0.9.0/src/ragx/web/handlers.py +152 -0
- ragx_cli-0.9.0/src/ragx/web/query_runner.py +102 -0
- ragx_cli-0.9.0/src/ragx/web/server.py +62 -0
- ragx_cli-0.9.0/src/ragx/web/static/index.html +64 -0
- ragx_cli-0.9.0/src/ragx/web/static/js/colors.js +64 -0
- ragx_cli-0.9.0/src/ragx/web/static/js/graph.js +178 -0
- ragx_cli-0.9.0/src/ragx/web/static/js/layout.js +164 -0
- ragx_cli-0.9.0/src/ragx/web/static/js/main.js +194 -0
- ragx_cli-0.9.0/src/ragx/web/static/js/query.js +70 -0
- ragx_cli-0.9.0/src/ragx/web/static/js/render.js +105 -0
- ragx_cli-0.9.0/src/ragx/web/static/js/util.js +73 -0
- ragx_cli-0.9.0/src/ragx/web/static/styles.css +155 -0
- ragx_cli-0.9.0/src/ragx/web/static/vendor/LICENSE-graphology-library.txt +21 -0
- ragx_cli-0.9.0/src/ragx/web/static/vendor/LICENSE-graphology.txt +21 -0
- ragx_cli-0.9.0/src/ragx/web/static/vendor/LICENSE-sigma.txt +12 -0
- ragx_cli-0.9.0/src/ragx/web/static/vendor/graphology-library.min.js +2 -0
- ragx_cli-0.9.0/src/ragx/web/static/vendor/graphology.umd.min.js +2 -0
- ragx_cli-0.9.0/src/ragx/web/static/vendor/sigma.min.js +4185 -0
- ragx_cli-0.9.0/src/ragx/web/static_files.py +32 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_cli.py +7 -1
- ragx_cli-0.9.0/tests/test_context.py +109 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_discovery.py +22 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_eval.py +3 -1
- ragx_cli-0.9.0/tests/test_graph_export.py +82 -0
- ragx_cli-0.9.0/tests/test_grouping.py +55 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_integration.py +77 -12
- ragx_cli-0.9.0/tests/test_query_cmd.py +81 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_traversal.py +39 -1
- ragx_cli-0.9.0/tests/test_web_cmd.py +84 -0
- ragx_cli-0.9.0/tests/test_web_server.py +164 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/SKILL.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/api-cheatsheet.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/collection-management/python.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/collection-management/typescript.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/configuration.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/data-model.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/data-operations/python.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/data-operations/typescript.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/embedding.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/hybrid-search/python.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/hybrid-search/typescript.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/multimodal-search/python.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/multimodal-search/typescript.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/quick-start/python.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/quick-start/typescript.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/rag-system/python.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/rag-system/typescript.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/reranker.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/troubleshooting.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/vector-search/python.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.agents/skills/zvec/vector-search/typescript.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.github/workflows/publish.yml +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/.gitignore +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/AGENTS.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/CONTRACTS-PHASE23.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/CONTRACTS.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/LICENSE +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/docs/feature-temporal-weighting.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/docs/proposal-models-command-interactive-flow.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/ragx-cli-plan.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/bge-m3-colbert-multi-vector-late-interaction-for-ragx-rerank-alternative-and-token-level-graph-edges.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/bge-m3-dense-embeddings-as-ragx-provider-multilingual-quality-and-threshold-calibration.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/bge-m3-dense-q8-vs-nomic-q4-benchmark-2026-07-12-worktree-eval-results.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/bge-m3-sparse-lexical-weights-hybrid-retrieval-leg-for-ragx-vs-sqlite-fts5-bm25.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/bge-m3-subchunk-edge-ablation-2026-07-13-ties-again-and-beta-zero-shortlist-mechanism.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/fine-grained-sub-chunk-edges-with-coarse-chunk-nodes-multi-granularity-graph-rag-literature-validation.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/lm-studio-model-download-api-and-curated-embedding-reranker-catalog-for-ragx-models-command.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/research/subchunk-edge-ablation-2026-07-12-two-round-experiment-protocol-results-and-verdict.md +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/skills-lock.json +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/__init__.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/__init__.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/detect.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/doctor_cmd.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/init_prompt.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/inspect_cmd.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/models_cmd.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/models_ui.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/cli/output.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/__init__.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/catalog.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/chunking.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/communities.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/discovery.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/doctor.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/errors.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/eval.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/expansion.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/fusion.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/graph.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/lmstudio.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/scoring.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/core/vectors.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/__init__.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/base.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/llama_embedder.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/llama_process.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/llama_server.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/lmstudio_process.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/openai_compat.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/registry.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/src/ragx/providers/st_reranker.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/__init__.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/conftest.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_chunking.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_communities.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_config_rc.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_doctor.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_expansion.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_fusion.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_graph.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_inspect.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_llama_server.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_lmstudio_autostart.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_models.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_providers.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_store.py +0 -0
- {ragx_cli-0.7.0 → ragx_cli-0.9.0}/tests/test_vectors.py +0 -0
|
@@ -4,6 +4,63 @@ All notable changes to `ragx-cli` are documented here. The format follows
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
|
|
5
5
|
[Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [0.9.0] — 2026-10-21
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `ragx-cli web [--host] [--port] [--open/--no-open]` — serves an interactive graph view
|
|
12
|
+
(nodes/edges/communities) plus a query box over plain `http.server`. Requires an index built
|
|
13
|
+
with `graph.edge_source = "subchunk"`.
|
|
14
|
+
- Grep-style `-B`/`-A`/`-C` context lines on `ragx-cli query` results (`query.context_lines`,
|
|
15
|
+
default 5), and `explain.path` (the seed→parent chain) in `--explain` output.
|
|
16
|
+
- `ragx.query.v3` schema (breaking): adds the `context` and `explain.path` fields to each result
|
|
17
|
+
chunk.
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
|
|
21
|
+
- **`graph.edge_source` defaults to `"subchunk"` (was `"chunk"`); `graph.subchunk_size_tokens`
|
|
22
|
+
defaults to 256 (was 128). Breaking: an index built under the old defaults mismatches the
|
|
23
|
+
manifest — `ragx-cli index --full` rebuilds it, or pin `graph.edge_source = "chunk"` /
|
|
24
|
+
`graph.subchunk_size_tokens = 128` in `ragx.toml` to keep it.** A corpus whose `ragx.toml` was
|
|
25
|
+
written by `ragx-cli init` already pins both keys explicitly and is unaffected; only a
|
|
26
|
+
hand-written `ragx.toml` missing them picks up the new default. `ragx-cli query` is unaffected
|
|
27
|
+
either way — it doesn't read sub-chunk data. Measured tie with `"chunk"` on the scoped notes eval corpus;
|
|
28
|
+
256 chosen as the cheapest size that ties:
|
|
29
|
+
`research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md`.
|
|
30
|
+
|
|
31
|
+
## [0.8.0] — 2026-10-20
|
|
32
|
+
|
|
33
|
+
### Changed
|
|
34
|
+
|
|
35
|
+
- **`query` defaults to vector search + graph traversal.** Query expansion and
|
|
36
|
+
cross-encoder rerank are now opt-in via `--expand` and `--rerank` (`QueryOptions`
|
|
37
|
+
defaults `expand=False`, `rerank=False`). A default query makes no LLM call and loads
|
|
38
|
+
no reranker. `--no-graph` is unchanged. `--no-expand` and `--no-rerank` still parse
|
|
39
|
+
as deprecated no-ops with a stderr warning. For the previous behaviour pass
|
|
40
|
+
`--expand --rerank`.
|
|
41
|
+
- **Default corpus globs index source code and text docs only.** `corpus.include` lists
|
|
42
|
+
~70 source extensions, `Dockerfile`, `Makefile`, and `*.md`/`*.mdx`/`*.rst`/`*.txt`/
|
|
43
|
+
`*.adoc`/`*.org`; `corpus.exclude` skips lockfiles, `requirements*.txt`,
|
|
44
|
+
`constraints*.txt`, licenses, minified files, source maps, and `dist/`, `build/`,
|
|
45
|
+
`target/`, `coverage/`, `vendor/`. Applies to new `ragx-cli init` configs; an existing
|
|
46
|
+
`ragx.toml` keeps its lists.
|
|
47
|
+
|
|
48
|
+
- **Query results are one entry per file (`ragx.query.v2`, breaking).** A file whose
|
|
49
|
+
chunks would appear several times in the ranked list becomes one entry with up to 3
|
|
50
|
+
sub-entries; its score is their sum, so it ranks higher. Only chunks within the visible
|
|
51
|
+
window (ranked chunks until `--top` distinct files are seen) count. JSON results are
|
|
52
|
+
`{file, score, chunks: [...]}`; chunks no longer repeat `file`. `--files-only`
|
|
53
|
+
(`ragx.files.v1`) keeps its shape and now returns the same entries. On the notes eval
|
|
54
|
+
corpus: baseline MRR .646 → .665, graph config r@5 .808 → .923 and MRR .432 → .653.
|
|
55
|
+
- **`--top` counts result entries (files), default 20** (`query.top`, was 8 chunks).
|
|
56
|
+
Existing `ragx.toml` files keep their own `query.top`. `ragx-cli eval` no longer
|
|
57
|
+
over-fetches `top × 3` chunks.
|
|
58
|
+
|
|
59
|
+
### Added
|
|
60
|
+
|
|
61
|
+
- `ragx-cli init` prints a note that an AI agent can tailor `corpus.include`/`exclude`
|
|
62
|
+
to the project and add a CLAUDE.md/AGENTS.md section on using `ragx-cli query`.
|
|
63
|
+
|
|
7
64
|
## [0.7.0] — 2026-09-17
|
|
8
65
|
|
|
9
66
|
### Added
|
|
@@ -73,6 +73,8 @@ src/ragx/
|
|
|
73
73
|
catalog.py # curated embedding/reranker catalog (tiers, prefixes) + spec detect
|
|
74
74
|
lmstudio.py # `lms` CLI wrapper: find/download (`lms get --yes`)/`lms ls --json`;
|
|
75
75
|
# LM Studio can't download plain safetensors — rerankers stay on HF
|
|
76
|
+
graph_export.py # build_graph: ragx.graph.v1 payload for `web`; fails loud unless the
|
|
77
|
+
# index's stored edge_source is "subchunk" (see Flows)
|
|
76
78
|
providers/ # base.py protocols (Embedder/Generator/Reranker); openai_compat.py;
|
|
77
79
|
# st_reranker.py; lmstudio_process.py (LM Studio autostart);
|
|
78
80
|
# llama_process.py (managed llama-server lifecycle) +
|
|
@@ -80,44 +82,71 @@ src/ragx/
|
|
|
80
82
|
# registry.py factories (env-var + api_key_env resolution)
|
|
81
83
|
cli/ # thin shells only: app.py (init/status/config + registration),
|
|
82
84
|
# pipeline.py (index/query), inspect_cmd.py, eval_cmd.py, models_cmd.py
|
|
83
|
-
# (recommend/download/configure models), doctor_cmd.py, output.py
|
|
84
|
-
|
|
85
|
+
# (recommend/download/configure models), doctor_cmd.py, web_cmd.py, output.py
|
|
86
|
+
web/ # second thin shell over ragx.core (MCP server is the planned third):
|
|
87
|
+
# server.py (build_server: ThreadingHTTPServer + cached graph bytes),
|
|
88
|
+
# handlers.py (routing), query_runner.py (lazy providers, lock-serialized
|
|
89
|
+
# queries), static_files.py (path-traversal-safe static serving);
|
|
90
|
+
# static/ (frontend-owned: index.html, JS/CSS, vendored sigma.js)
|
|
91
|
+
tests/ # 295 tests; mocked HTTP (respx), FakeEmbedder integration tests, no live network
|
|
85
92
|
```
|
|
86
93
|
|
|
87
94
|
## Flows
|
|
88
95
|
|
|
89
|
-
**Index** (`ragx-cli index
|
|
90
|
-
|
|
91
|
-
|
|
96
|
+
**Index** (`ragx-cli index`): discover files (include/exclude globs — defaults in
|
|
97
|
+
`core/corpus_globs.py`: source + text docs, minus lockfiles/requirements/build output — root .gitignore,
|
|
98
|
+
skip node_modules/.venv/hidden/binaries/>2MB) → xxhash diff (incremental by default since 0.8.0;
|
|
99
|
+
`--full` rebuilds everything; `--changed` is a deprecated no-op alias for the default) → chunk
|
|
100
|
+
(~800 tok target, chunks are exact byte slices, sub-100-char fragments
|
|
92
101
|
merge into neighbors) → embed batched → HNSW add → kNN edges per new chunk (k=8, cos ≥ 0.55)
|
|
93
102
|
+ recompute edge lists of pre-existing neighbors that appear in new lists.
|
|
94
|
-
|
|
95
|
-
|
|
103
|
+
`graph.edge_source="subchunk"` (default since 2026-10-21; `"chunk"` is the opt-out): chunks
|
|
104
|
+
additionally split into ~`subchunk_size_tokens`-tok (default 256) sentence-aligned sub-chunks,
|
|
105
|
+
embedded separately (float32 blobs in the SQLite
|
|
96
106
|
`subchunks` table, cascade-deleted with their chunk; NOT in the query-time HNSW); edge weight =
|
|
97
107
|
max cos over sub-chunk pairs; edges with whole-chunk cos ≥ `graph.near_dup_sim` (0.9) dropped.
|
|
98
108
|
Query side untouched. In subchunk mode `graph.k` means links per SUB-chunk (no per-chunk cap —
|
|
99
|
-
edge budget scales with concept count; graph gets ~
|
|
100
|
-
`edge_source`/`subchunk_size_tokens` like the embedding model —
|
|
101
|
-
loud
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
109
|
+
edge budget scales with concept count; graph gets ~1.4x denser at the 256-tok default). The
|
|
110
|
+
manifest guards `edge_source`/`subchunk_size_tokens` like the embedding model — an incremental
|
|
111
|
+
`index` after a flip fails loud, naming the old/new value; `index --full` rebuilds, or pin the
|
|
112
|
+
old value in `ragx.toml`. Rationale + literature: `research/fine-grained-sub-chunk-edges-*.md`.
|
|
113
|
+
MEASURED 2026-07-12 (`.lab` experiments #10–#11): on the scoped notes eval corpus the dense
|
|
114
|
+
concept graph at hops2 TIES the tuned chunk-graph baseline (r@10 .923 / MRR .718 vs .717) at
|
|
115
|
+
~4.5x index embed cost (128-tok sub-chunks) — no win because the corpus's short single-topic
|
|
116
|
+
chunks have no dilution headroom (near-dup guard pruned ~3 edges). hops3 + uncapped budget
|
|
117
|
+
floods RERANK_CAP (candidates 643). Verdict at the time: default stays `edge_source="chunk"`;
|
|
118
|
+
subchunk opt-in, drop traversal to hops=2 if used. RE-MEASURED 2026-07-13 under BGE-M3: same tie
|
|
119
|
+
(rerank MRR .7548 vs .755) at 2x index time / 2.24x edges (128-tok) — verdict stands across
|
|
120
|
+
embedding models. That ablation also found that with tuned β=0 the pre-rerank shortlist beyond
|
|
121
|
+
the RRF seeds is filled by an ascending-chunk-id tie-break (graph-only candidates all have
|
|
122
|
+
vector=0), so graph/hops changes barely reach the reranker — latent design smell, see
|
|
112
123
|
`research/bge-m3-subchunk-edge-ablation-2026-07-13-ties-again-and-beta-zero-shortlist-mechanism.md`.
|
|
124
|
+
RE-VERDICT 2026-10-21 (`research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md`):
|
|
125
|
+
sweep of 64/96/128/192/256-tok sub-chunks ties `edge_source="chunk"` at every size on a second
|
|
126
|
+
eval corpus — size choice is driven by cost, not quality. Default flipped to
|
|
127
|
+
`edge_source="subchunk"`, `subchunk_size_tokens=256` (cheapest tying size: fewest edges,
|
|
128
|
+
~2x index time vs ~2.8–4.7x at 128/64-tok).
|
|
113
129
|
|
|
114
130
|
**Query** (`ragx-cli query "..."`): optional expansion (1 LLM call, strict-JSON parse, graceful
|
|
115
131
|
no-op on failure) → embed all variants, HNSW top-20 each → RRF merge = seeds → heat propagation
|
|
116
132
|
(2 hops, decay 0.5, heat = max not sum, neighbor admitted only if cos(query) ≥ 0.35 floor,
|
|
117
133
|
frontier ≤ 150) → cross-encoder rerank of top-100 shortlist → final =
|
|
118
134
|
0.6·rerank + 0.25·heat + 0.15·vector (each min-max normalized; weights renormalize when a
|
|
119
|
-
stage is off).
|
|
120
|
-
|
|
135
|
+
stage is off). Since 0.8.0 expansion and rerank are OPT-IN (`--expand`/`--rerank`;
|
|
136
|
+
`QueryOptions` defaults expand=False, rerank=False; `--no-expand/--no-rerank` are hidden
|
|
137
|
+
deprecated no-ops); graph stays on, `--no-graph` skips it. Output is folded per file
|
|
138
|
+
(`core/grouping.py`, schema `ragx.query.v3`): walk ranked chunks until `top` (default 20)
|
|
139
|
+
distinct files are seen, each file keeps its ≤3 chunks inside that window, entry score = their
|
|
140
|
+
sum. The window is load-bearing — summing ALL candidates let weak graph-neighbour chunks pad
|
|
141
|
+
files (graph MRR .43 → .38; windowed .65). `--files-only` = same entries, no text; `--explain` attaches seed→parent→edge_weight→hop traces.
|
|
142
|
+
Each result chunk also carries grep-style `-B`/`-A`/`-C` context lines (`core/context.py`,
|
|
143
|
+
`query.context_lines=5` default, counts non-empty lines only); `--files-only` skips the file
|
|
144
|
+
reads since no text is shown.
|
|
145
|
+
|
|
146
|
+
**Web** (`ragx-cli web`): builds the `ragx.graph.v1` payload once at startup (`graph_export.py`
|
|
147
|
+
— requires the index's stored `edge_source` to be `"subchunk"`, fails loud otherwise) and serves
|
|
148
|
+
it plus `/api/chunk/{id}` and `/api/query` (same pipeline as `query`, providers built lazily and
|
|
149
|
+
cached, one query at a time behind a lock) over stdlib `http.server`.
|
|
121
150
|
|
|
122
151
|
## State & benchmarks (as of 2026-07-10)
|
|
123
152
|
|
|
@@ -129,7 +158,8 @@ full .759/**.889**/**.613**. Key finding: graph-only HURTS precision (near-dupli
|
|
|
129
158
|
displace direct hits — parameter sweeps plateau below baseline MRR; corpus property, not a
|
|
130
159
|
tuning miss), but graph+rerank together deliver the recall win — one Dutch query's target is
|
|
131
160
|
unreachable by vector search or rerank-alone, surfaced only via a hop-1 edge then reranked
|
|
132
|
-
19→4.
|
|
161
|
+
19→4. Graph+rerank belong together for quality (`--rerank`); the 0.8.0 default (graph, no
|
|
162
|
+
rerank) is the measured "graph-only" row — chosen for speed, not quality.
|
|
133
163
|
|
|
134
164
|
UPDATE 2026-07-13: production embedding model is now `text-embedding-bge-m3` (Q8 GGUF, empty
|
|
135
165
|
doc/query prefixes) — notes-repo corpus config + index flipped in place from the worktree-built
|
|
@@ -162,7 +192,8 @@ mostly on Dutch/multilingual queries. Details: `research/bge-m3-dense-q8-vs-nomi
|
|
|
162
192
|
- Phase 4 (plan §5): Leiden community creation is DONE (index-time, `communities.py`,
|
|
163
193
|
read-only via `status`/`inspect communities`/`inspect community`). Remaining: community
|
|
164
194
|
labels, `query --global` for corpus-level questions.
|
|
165
|
-
- Phase 5: MCP server as a
|
|
195
|
+
- Phase 5: MCP server as a third thin shell over `ragx.core` (split already enforced; `web/`
|
|
196
|
+
is the second, see Layout/Flows).
|
|
166
197
|
- Temporal weighting: full decided spec in `docs/feature-temporal-weighting.md` — opt-in
|
|
167
198
|
`--since/--until/--temporal recent|oldest`, date cascade filename/frontmatter → git → mtime,
|
|
168
199
|
schema v2 migration. Build only after it can be measured against the eval baseline.
|
|
@@ -170,7 +201,7 @@ mostly on Dutch/multilingual queries. Details: `research/bge-m3-dense-q8-vs-nomi
|
|
|
170
201
|
|
|
171
202
|
## Dev loop
|
|
172
203
|
|
|
173
|
-
`uv sync --group dev --extra rerank` · `uv run pytest -q` (
|
|
204
|
+
`uv sync --group dev --extra rerank` · `uv run pytest -q` (295 pass, ~7 s) ·
|
|
174
205
|
`uv run ruff check src tests` · file soft cap ~150 lines · expected failures raise `RagxError`
|
|
175
206
|
(CLI maps to exit 2). Live smoke: LM Studio must be running with the configured embedding model.
|
|
176
207
|
Changes that impact usage (CLI flags, config keys/precedence, output schemas, install steps)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: ragx-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.9.0
|
|
4
4
|
Summary: Similarity-graph RAG CLI: index a corpus into a chunk-level embedding graph and query it
|
|
5
5
|
Project-URL: Repository, https://github.com/alber70g/ragx-cli
|
|
6
6
|
Project-URL: Issues, https://github.com/alber70g/ragx-cli/issues
|
|
@@ -89,21 +89,26 @@ embedded, and stored in an HNSW index. The similarity graph then falls out almos
|
|
|
89
89
|
one kNN pass over the vectors that are already in memory — each chunk gets edges to its top-k
|
|
90
90
|
nearest neighbors above a similarity floor.
|
|
91
91
|
|
|
92
|
-
|
|
92
|
+
`graph.edge_source = "subchunk"` (default) derives edge weights from sentence-aligned
|
|
93
93
|
**sub-chunks** instead of whole-chunk cosine — each chunk is split into ~`subchunk_size_tokens`
|
|
94
|
-
windows, embedded separately (stored in SQLite, never in the query-time HNSW), and
|
|
95
|
-
weight between two chunks becomes the *max* similarity over their sub-chunk pairs, so a
|
|
96
|
-
mixing several concepts gets one sharp edge per concept instead of a diluted average. Edges
|
|
94
|
+
(256) token windows, embedded separately (stored in SQLite, never in the query-time HNSW), and
|
|
95
|
+
the edge weight between two chunks becomes the *max* similarity over their sub-chunk pairs, so a
|
|
96
|
+
chunk mixing several concepts gets one sharp edge per concept instead of a diluted average. Edges
|
|
97
97
|
between near-duplicate chunks (whole-chunk cosine ≥ `near_dup_sim`) are dropped — those are the
|
|
98
|
-
measured precision-killers. Costs
|
|
99
|
-
units, traversal, and query flow are unchanged. Switching `edge_source` (or
|
|
100
|
-
requires `index --full`; incremental runs fail loud on the mismatch
|
|
98
|
+
measured precision-killers. Costs ~2× more embedding calls at index time than `edge_source =
|
|
99
|
+
"chunk"`; retrieval units, traversal, and query flow are unchanged. Switching `edge_source` (or
|
|
100
|
+
the sub-chunk size) requires `index --full`; incremental runs fail loud on the mismatch, naming
|
|
101
|
+
the old/new value and the fix. Literature grounding:
|
|
101
102
|
`research/fine-grained-sub-chunk-edges-with-coarse-chunk-nodes-multi-granularity-graph-rag-literature-validation.md`.
|
|
102
103
|
In this mode `k` counts links per *sub-chunk* with no per-chunk cap, so the graph is denser and
|
|
103
|
-
shallower traversal suffices
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
104
|
+
shallower traversal suffices (the default `hops=2` already matches this). Quality ties
|
|
105
|
+
`edge_source="chunk"` exactly at every sub-chunk size tried, 64–256 tokens (2026-07 and
|
|
106
|
+
2026-10-21 sweeps); the corpora measured so far have short, single-topic chunks with no
|
|
107
|
+
concept-dilution headroom to exploit. `256` is the default because it's the cheapest size that
|
|
108
|
+
ties — fewest edges, ~2× index time vs ~2.8–4.7× at 128/64 tokens. Set `graph.edge_source =
|
|
109
|
+
"chunk"` back if a corpus's long, genuinely multi-concept chunks make the extra edges worth the
|
|
110
|
+
cost — no corpus has measured a win yet. Sweep data:
|
|
111
|
+
`research/subchunk-size-sweep-2026-10-21-default-edge-source-subchunk-size-choice.md`.
|
|
107
112
|
|
|
108
113
|
After edge construction, a Leiden partition (`graspologic-native`, seeded, deterministic) is
|
|
109
114
|
computed over the whole edge list every index run and stored read-only — see
|
|
@@ -124,26 +129,54 @@ flowchart TD
|
|
|
124
129
|
re-embeds only new/modified files, drops deleted ones, and repairs only the edge lists those
|
|
125
130
|
chunks touch. Content hashes make `touch`-ed but unchanged files free, and include/exclude glob
|
|
126
131
|
changes converge to the same file set a rebuild would produce. `--full` rebuilds from scratch;
|
|
127
|
-
changing `embeddings.model`, `chunking.*`, or `graph.edge_source` requires
|
|
128
|
-
fail loud on the manifest mismatch
|
|
129
|
-
|
|
132
|
+
changing `embeddings.model`, `chunking.*`, or `graph.edge_source`/`subchunk_size_tokens` requires
|
|
133
|
+
it — incremental runs fail loud on the manifest mismatch, naming the changed key and its old/new
|
|
134
|
+
value. `ragx-cli status` reports **drift** (files new/changed/deleted since the last index) so
|
|
135
|
+
you can see when a re-index is due without running one.
|
|
136
|
+
|
|
137
|
+
Upgrading from an older `ragx-cli` after a default change (e.g. the `graph.edge_source` flip to
|
|
138
|
+
`"subchunk"`): `ragx-cli init` always writes every `[graph]` key explicitly into `ragx.toml`
|
|
139
|
+
(`Config.save` bakes the full default set, not just overrides), so an init-created corpus already
|
|
140
|
+
pins its `edge_source`/`subchunk_size_tokens` and is unaffected by a later default change. Only a
|
|
141
|
+
hand-written `ragx.toml` missing those keys picks up the new default; its next incremental
|
|
142
|
+
`index` then fails loud — `index --full` rebuilds, or pin the old value back in `ragx.toml`.
|
|
130
143
|
|
|
131
144
|
### Querying
|
|
132
145
|
|
|
133
|
-
|
|
134
|
-
|
|
146
|
+
A default query runs vector search + graph traversal: no LLM call, no cross-encoder.
|
|
147
|
+
Expansion (`--expand`) and rerank (`--rerank`) are opt-in; `--no-graph` skips traversal,
|
|
148
|
+
which leaves pure vector search. Callers trade latency for quality per query.
|
|
149
|
+
|
|
150
|
+
Every result chunk also comes with grep-style `-B`/`-A`/`-C` context lines, on by default
|
|
151
|
+
(`query.context_lines=5` on each side). `-B/--before N` and `-A/--after N` set each side
|
|
152
|
+
independently; `-C/--context N` sets both; explicit `-B`/`-A` win over `-C`, which wins over
|
|
153
|
+
the default. Counting rule: `N` counts **non-empty** lines walking outward from the chunk —
|
|
154
|
+
blank lines don't count toward `N` but are kept in the returned span if they fall between the
|
|
155
|
+
chunk and the Nth non-empty line; the span stops exactly at that line, so blank lines beyond it
|
|
156
|
+
are excluded. `N=0` disables that side; fewer than `N` non-empty lines in the file returns what
|
|
157
|
+
exists; none returns `null`. Context is read fresh from disk at query time, never truncated by
|
|
158
|
+
`query.max_chunk_chars`, and `--files-only` skips it (no file reads, since the text isn't shown).
|
|
159
|
+
|
|
160
|
+
Results are **one entry per file**. The pipeline ranks chunks, walks that list until it has
|
|
161
|
+
seen `--top` distinct files (default `query.top = 20`), and folds each file's chunks inside
|
|
162
|
+
that window into one entry: up to 3 sub-entries, entry score = their sum. A file that would
|
|
163
|
+
appear several times in a flat list therefore ranks higher. Chunks below the window never
|
|
164
|
+
count, so graph traversal's weak neighbour chunks cannot pad a file (measured on the notes
|
|
165
|
+
eval corpus: graph-config MRR .43 flat-list, .38 summing all chunks, .65 windowed).
|
|
135
166
|
|
|
136
167
|
```mermaid
|
|
137
168
|
flowchart TD
|
|
138
169
|
Q[query] --> X["1 · expansion (optional, one LLM call)<br/>2–4 reformulations + HyDE passage"]
|
|
139
170
|
X --> S["2 · fan-out vector search<br/>top-20 per variant"]
|
|
140
|
-
Q -. "--
|
|
171
|
+
Q -. "default (no --expand)" .-> S
|
|
141
172
|
S --> R["3 · Reciprocal Rank Fusion<br/>merged seed set"]
|
|
142
173
|
R --> H["4 · heat propagation over the graph<br/>2 hops · decay 0.5 · max-aggregation<br/>query-similarity floor · frontier cap"]
|
|
143
174
|
R -. "--no-graph" .-> K
|
|
144
|
-
H --> K["5 · cross-encoder rerank<br/>(query, chunk) pairs, capped shortlist"]
|
|
175
|
+
H --> K["5 · cross-encoder rerank (--rerank)<br/>(query, chunk) pairs, capped shortlist"]
|
|
176
|
+
H -. "default (no --rerank)" .-> F
|
|
145
177
|
K --> F["6 · combined score<br/>α·rerank + β·heat + γ·vector"]
|
|
146
|
-
F -->
|
|
178
|
+
F --> G["7 · fold per file<br/>a file's best ≤3 chunks → one entry, score = their sum"]
|
|
179
|
+
G --> O["top 20 entries (or paths via --files-only)<br/>+ --explain traversal trace"]
|
|
147
180
|
```
|
|
148
181
|
|
|
149
182
|
**Heat propagation** is what sets ragx-cli apart from plain RAG: seed chunks (from vector search)
|
|
@@ -163,7 +196,10 @@ flowchart LR
|
|
|
163
196
|
```
|
|
164
197
|
|
|
165
198
|
Because every admitted chunk records which seed and edge produced it, `--explain` can print the
|
|
166
|
-
full justification: *seed → edge(weight) → chunk*, per result.
|
|
199
|
+
full justification: *seed → edge(weight) → chunk*, per result. The explain object also carries
|
|
200
|
+
`path`: the seed-first chain of chunk ids along the traversal's own parent links
|
|
201
|
+
(`[seed, ..., chunk_id]`), so a UI can highlight the exact path the traversal took instead of
|
|
202
|
+
guessing a shortest path through the graph.
|
|
167
203
|
|
|
168
204
|
---
|
|
169
205
|
|
|
@@ -210,8 +246,10 @@ output for that result shows the exact seed → edge → chunk path.
|
|
|
210
246
|
**Honest caveat:** graph traversal *alone* hurts precision on this corpus (MRR 0.593 → 0.522) —
|
|
211
247
|
near-duplicate neighbors (e.g. adjacent meeting notes) displace weaker direct hits. A parameter
|
|
212
248
|
sweep over decay/floor/weights plateaued below baseline MRR, so this is a property of
|
|
213
|
-
similarity-only edges, not a tuning miss. Conclusion
|
|
214
|
-
|
|
249
|
+
similarity-only edges, not a tuning miss. Conclusion: **graph and rerank belong together**.
|
|
250
|
+
Since 0.8.0 the CLI default is the fast path (vector + graph, no rerank), which is the
|
|
251
|
+
"graph-only" row above; pass `--rerank` for the measured best quality, or `--no-graph` for
|
|
252
|
+
pure vector search.
|
|
215
253
|
|
|
216
254
|
The parameter study below explains the mechanism behind this caveat — and shows that once the
|
|
217
255
|
scoring weights stop putting heat into the final score, the graph win gets much bigger.
|
|
@@ -430,15 +468,53 @@ section with `ragx-cli config set embeddings.autostart false` (or machine-wide v
|
|
|
430
468
|
`ragx-cli models` still only downloads models and writes `ragx.toml`; it never launches
|
|
431
469
|
anything. If a run dies with "connection refused", `doctor` names the stage and the fix.
|
|
432
470
|
|
|
471
|
+
## Browsing the graph: `ragx-cli web`
|
|
472
|
+
|
|
473
|
+
`ragx-cli web` serves an interactive view of the chunk graph plus a query box, over plain
|
|
474
|
+
`http.server` (no web framework, no build step for the backend):
|
|
475
|
+
|
|
476
|
+
```
|
|
477
|
+
ragx-cli web [--host 127.0.0.1] [--port 8765] [--open/--no-open]
|
|
478
|
+
```
|
|
479
|
+
|
|
480
|
+
Requires an index built with `graph.edge_source = "subchunk"` (the default) — the graph view
|
|
481
|
+
renders stored edge weights as sub-chunk similarity, and a `"chunk"`-mode index's whole-chunk
|
|
482
|
+
cosine weights would mislabel what the edges mean. Fails loud (exit 2) naming the mismatch, or
|
|
483
|
+
when no index exists yet. `--open` (default) launches the browser; Ctrl-C shuts the server down
|
|
484
|
+
cleanly.
|
|
485
|
+
|
|
486
|
+
Endpoints (JSON; errors are `{"error": "<message>"}`):
|
|
487
|
+
|
|
488
|
+
- `GET /` and `GET /static/<path>` — the viewer's static assets.
|
|
489
|
+
- `GET /api/graph` — `ragx.graph.v1`: `meta` (root, edge_source, subchunk_size_tokens,
|
|
490
|
+
embedding_model, nodes/edges/communities counts), `nodes` (id, file, line_start/line_end,
|
|
491
|
+
label, community, degree), `edges` (`[src, dst, weight]`). Built once at startup, cached,
|
|
492
|
+
gzip-encoded when the request sends `Accept-Encoding: gzip`.
|
|
493
|
+
- `GET /api/chunk/{id}` — a chunk's file/lines/text/community plus its neighbors
|
|
494
|
+
(`{chunk_id, weight}`, weight descending). 404 for an unknown id.
|
|
495
|
+
- `POST /api/query` — body `{"text", "top", "expand", "graph", "rerank", "hops", "explain",
|
|
496
|
+
"before", "after"}` (only `text` required; the rest default like `ragx-cli query`). Returns
|
|
497
|
+
`ragx.query.v3` (same shape as `--json` query output) plus `elapsed_ms`. 400 on a malformed
|
|
498
|
+
body, 502 when the query pipeline raises (backend down, manifest mismatch).
|
|
499
|
+
|
|
500
|
+
Providers (embedder, reranker, expansion LLM) are built lazily on first use of their stage and
|
|
501
|
+
cached for the server's lifetime; queries run one at a time behind a lock.
|
|
502
|
+
|
|
433
503
|
## Agent-first conventions
|
|
434
504
|
|
|
435
|
-
- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.
|
|
505
|
+
- `--json` emits exactly one JSON document on stdout (versioned schemas: `ragx.query.v3`,
|
|
436
506
|
`ragx.files.v1`, `ragx.status.v1`, `ragx.doctor.v1`, `ragx.eval.v1`, `ragx.inspect.*.v1`); logs go
|
|
437
507
|
to stderr.
|
|
438
508
|
- Exit codes: `0` results, `1` success-but-empty, `2` error.
|
|
439
|
-
-
|
|
440
|
-
|
|
441
|
-
|
|
509
|
+
- `ragx.query.v3` results are one entry per file: `{file, score, chunks: [...]}`. Each chunk
|
|
510
|
+
carries `chunk_id`, `line_start/line_end`, `byte_start/byte_end`, `score`, `breakdown`, `text`,
|
|
511
|
+
`truncated` — agents jump to the exact source location and read the full text themselves
|
|
512
|
+
(JSON chunk text is truncated). v1 (one flat entry per chunk) is gone since 0.8.0.
|
|
513
|
+
- Each chunk also carries `context: {before, after}` (grep-style `-B`/`-A` lines, default 5
|
|
514
|
+
each; see [Querying](#querying)) — each side is `{line_start, line_end, text}` or `null`.
|
|
515
|
+
The key is omitted entirely when both `--before`/`-B` and `--after`/`-A` are 0.
|
|
516
|
+
- `--files-only` returns the same entries without text (`file`, `score`, `chunk_ids`) — the
|
|
517
|
+
mode coding agents use most.
|
|
442
518
|
- `ragx-cli status` includes a `drift` object (`new`/`changed`/`deleted` file counts vs the index) —
|
|
443
519
|
agents check it to decide whether to run `ragx-cli index` before querying.
|
|
444
520
|
- `ragx-cli doctor --json` verifies the providers answer before a batch of queries (`ok: false`
|
|
@@ -458,10 +534,11 @@ This repo has a ragx-cli index (`.ragx/`). Prefer it over grep for "where is X d
|
|
|
458
534
|
questions; fall back to grep for exact identifiers.
|
|
459
535
|
|
|
460
536
|
- Find relevant files: `ragx-cli query "<natural-language question>" --json --files-only`
|
|
461
|
-
- Get chunks with exact locations: `ragx-cli query "..." --json
|
|
462
|
-
`
|
|
463
|
-
for full context.
|
|
464
|
-
-
|
|
537
|
+
- Get chunks with exact locations: `ragx-cli query "..." --json` — one result per `file`
|
|
538
|
+
(default 20, `--top N`), each with up to 3 `chunks` carrying `line_start/line_end`; the JSON
|
|
539
|
+
`text` is truncated, so read the file yourself for full context.
|
|
540
|
+
- The default query makes no LLM call and loads no cross-encoder. For higher precision add
|
|
541
|
+
`--rerank`; add `--expand` for an LLM query-expansion pass (one extra LLM call).
|
|
465
542
|
- After adding or editing files: `ragx-cli index` (incremental and cheap — hash-based).
|
|
466
543
|
`ragx-cli status --json` shows `drift` counts if you want to check staleness first.
|
|
467
544
|
- stdout is exactly one JSON document; logs are on stderr.
|
|
@@ -611,12 +688,17 @@ non-thinking model is the better expansion choice; detection is a best-effort na
|
|
|
611
688
|
heuristic, the APIs expose no capability flag), and corpus include/exclude
|
|
612
689
|
(gitignore-style globs, comma-separated). Every prompt is pre-filled with the defaults
|
|
613
690
|
below — Enter accepts them all. With piped stdin (agents), `--yes`, or
|
|
614
|
-
`--no-interactive`, it writes the defaults unchanged, exactly as before.
|
|
691
|
+
`--no-interactive`, it writes the defaults unchanged, exactly as before. After writing the
|
|
692
|
+
config, `init` prints a note: an AI agent can tailor `corpus.include`/`exclude` to the
|
|
693
|
+
project, and add a CLAUDE.md/AGENTS.md section on searching it with `ragx-cli query`
|
|
694
|
+
(see [Using ragx-cli from a coding agent](#using-ragx-cli-from-a-coding-agent-claudemd--agentsmd)).
|
|
695
|
+
Key defaults:
|
|
615
696
|
|
|
616
697
|
| section | defaults |
|
|
617
698
|
|---|---|
|
|
699
|
+
| `[corpus]` | `include` = source code (`*.py`, `*.ts`, `*.go`, `*.rs`, … ~70 extensions, plus `Dockerfile`/`Makefile`) and text docs (`*.md`, `*.mdx`, `*.rst`, `*.txt`, `*.adoc`, `*.org`); `exclude` = lockfiles (`*.lock`, `package-lock.json`, `pnpm-lock.yaml`, `go.sum`, …), `requirements*.txt`/`constraints*.txt`, `LICENSE*`/`COPYING*`, `*.min.js`/`*.min.css`/`*.map`, and `dist/`, `build/`, `target/`, `coverage/`, `vendor/`; `respect_gitignore=true`. Full lists: `src/ragx/core/corpus_globs.py`. Existing `ragx.toml` files keep their own lists |
|
|
618
700
|
| `[chunking]` | `size_tokens=800`, `overlap=0.15` |
|
|
619
|
-
| `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="
|
|
701
|
+
| `[graph]` | `k=8`, `min_edge_sim=0.55`, `edge_source="subchunk"`, `subchunk_size_tokens=256`, `near_dup_sim=0.9` |
|
|
620
702
|
| `[traversal]` | `hops=2`, `decay=0.5`, `query_floor=0.35`, `max_frontier=150` |
|
|
621
703
|
| `[communities]` | `resolution=1.0`, `seed=42` — recomputed every index run; changing these never invalidates the index |
|
|
622
704
|
| `[fusion]` | `rrf_k=60`, `per_query_top=20` |
|
|
@@ -624,6 +706,7 @@ below — Enter accepts them all. With piped stdin (agents), `--yes`, or
|
|
|
624
706
|
| `[embeddings]` | `provider="openai"`, `base_url="http://localhost:1234/v1"`, prefixes for nomic-style models, `api_key_env=""`, `autostart=true` (start LM Studio + load the model when the URL is local) |
|
|
625
707
|
| `[expansion]` | optional LLM for multi-query/HyDE; reasoning models supported (4096-token budget); `api_key_env=""`, `autostart=true` |
|
|
626
708
|
| `[rerank]` | `BAAI/bge-reranker-v2-m3` via sentence-transformers (`uv tool install 'ragx-cli[rerank]'`) |
|
|
709
|
+
| `[query]` | `top=20`, `max_chunk_chars=1200`, `context_lines=5` (default for `-B`/`-A`/`-C`) |
|
|
627
710
|
|
|
628
711
|
## Features & roadmap
|
|
629
712
|
|
|
@@ -638,9 +721,10 @@ unchecked ones are next up. Release history: [CHANGELOG.md](CHANGELOG.md).
|
|
|
638
721
|
- [x] **Interactive `init`**: LM Studio/Ollama server + model detection, guided embeddings/expansion/corpus setup (`--yes` for defaults)
|
|
639
722
|
- [x] **Committable config**: `ragx.toml` at the corpus root (0.3.0; was `.ragx/config.toml`), index data stays gitignored in `.ragx/`
|
|
640
723
|
- [x] **Offline-friendly rerank**: graceful degradation + pre-seeding docs when huggingface.co is unreachable
|
|
641
|
-
- [x] **Sub-chunk edges** (
|
|
724
|
+
- [x] **Sub-chunk edges** (default since the 2026-10-21 default flip; `graph.edge_source="subchunk"`): edge weight = max cosine over sentence-aligned sub-chunk pairs, for corpora with long multi-concept chunks; set `graph.edge_source="chunk"` for the whole-chunk-cosine alternative
|
|
642
725
|
- [x] **Incremental by default** (0.4.0): `index` hash-diffs, `--full` rebuilds, `status` reports corpus drift, chunking params are manifest-guarded
|
|
643
726
|
- [x] **`ragx-cli models`**: curated embedding/reranker recommendation (quality tier + RAM detection), downloads via LM Studio's `lms get`, serving via llama-server or LM Studio
|
|
727
|
+
- [x] **`ragx-cli web`**: interactive sub-chunk graph viewer + query UI over stdlib `http.server`
|
|
644
728
|
- [ ] **Community labels**: name the Leiden communities so they're browsable without reading member chunks
|
|
645
729
|
- [ ] **query --global** for corpus-level questions (answer from community summaries, not individual chunks)
|
|
646
730
|
- [ ] **MCP server**: a second thin shell over `ragx.core` (the core/CLI split it needs is already enforced)
|