rag-your-code 0.7.0__tar.gz → 1.0.0__tar.gz

{rag_your_code-0.7.0/src/rag_your_code.egg-info → rag_your_code-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rag-your-code
-Version: 0.7.0
+Version: 1.0.0
 Summary: A local, explainable RAG index for codebases and coding agents
 Author: rag-your-code contributors
 License-Expression: MIT
@@ -226,9 +226,9 @@ and left Chinese retrieval unchanged.
 
 ### Measured on this repository
 
-This project describes its own implementation: every unit under `src/` carries
-an agent-written bilingual description, committed to the repo, and 68 of them
-have been promoted into the source as doc comments.
+This project describes its own implementation: all 163 units under `src/` carry
+an agent-written bilingual description, committed to the repo, and 155 of those
+163 declarations also carry the author's own documentation in the source.
 
 Seventy natural-language questions about this codebase, in English and
 Chinese, each listing every unit that genuinely answers it
@@ -236,13 +236,13 @@ Chinese, each listing every unit that genuinely answers it
 
 | | generated descriptions | agent-written |
 |---|---|---|
-| hit@1 | 0.271 | **0.500** |
-| hit@3 | 0.486 | **0.800** |
-| MRR | 0.367 | **0.631** |
-| answered with no shared word at all | 12.9% | **0%** |
+| hit@1 | 0.314 | **0.500** |
+| hit@3 | 0.457 | **0.729** |
+| MRR | 0.379 | **0.583** |
+| questions declined for want of evidence | 13.0% | **4.3%** |
 
-Roughly double the first-place accuracy. Fourteen questions still fail, which
-is what makes the set usable for measuring the next change;
+Half again the first-place accuracy. Nineteen questions still fail, which is
+what makes the set usable for measuring the next change;
 `tests/test_repo_queries.py` asserts that some question always does, and that
 the written column beats the generated one.
 
@@ -283,6 +283,77 @@ the code it tests, because it repeats that code's vocabulary and adds its own.
 Matching stays lexical. It is LLM-authored keyword expansion, and its reach is
 bounded by how many ways of saying the thing the agent thought to write down.
 
+### The question a ranking cannot answer
+
+Both rulers above ask questions that *have* an answer, so both can only score
+whether it was found. Neither can see the opposite failure. A ranking always
+produces a least-bad unit and hands it back with a score and a rank that read
+exactly like an answer — and it does that whether or not the repository
+contains anything relevant at all.
+
+So there is a third ruler: thirty questions about subjects neither repository
+implements, where the only correct reply is nothing
+([`benchmarks/absent_queries.json`](benchmarks/absent_queries.json)). Before
+1.0.0 it scored **zero**. All thirty answered, on both repositories, in both
+languages:
+
+| asked of a repository with no such code | answered with | on the evidence of |
+|---|---|---|
+| `where are CUDA kernels dispatched to the device` | a test about word counting | `are` `the` `to` `where` |
+| `准入控制为什么会拒绝没有资源限额的容器组` | the UTF-8 console setup | `拒绝` `控制` `没有` |
+| `how is the OAuth refresh token rotated` | a description-store method | `before` `is` `refresh` `the` |
+
+Not a Chinese problem and not a ranking problem — a missing question. Nothing
+in the pipeline asked *is any of this evidence*; it only asked which ranks
+highest. Retrieval now asks both, and returns nothing when the answer to the
+first is no:
+
+| | before | now |
+|---|---|---|
+| unanswerable questions correctly met with silence, this repository | 0.000 | **0.733** |
+| the same, on the foreign repository | 0.000 | **0.800** |
+| results resting on no lexical evidence at all, all three rulers | 0.029 – 0.129 | **0.000** |
+| hit@1 / hit@3 / MRR on the foreign ruler | 0.257 / 0.400 / 0.314 | **unchanged** |
+| hit@1 / hit@3 / MRR on this repository | 0.471 / 0.686 / 0.557 | **unchanged** |
+
+The bar is the share of a question's **discriminating** words that occur in
+the index — words the repository uses everywhere are dropped from both sides
+of that fraction, which is the part that does the work. Half of `where are
+CUDA kernels dispatched to the device` matches, and it looks like evidence
+until you notice which half. Counting only words that distinguish silenced 18
+of 30 unanswerable English questions that no plain coverage threshold reached
+at all, at identical cost in real answers — 97 of 98 either way.
+
+It is a ratio inside the query rather than a threshold on a score, because a
+score threshold is tied to whatever scale the ranking currently produces —
+this project has already had one of those stop meaning anything the moment
+BM25F changed the scale. `search.min_coverage` sets it; `0` restores the old
+behaviour exactly.
+
+**What it costs:** one question of the 158 measured. `控制台编码不是 UTF-8
+会怎么样` was reaching `_use_utf8_streams`, and the only words in it this
+repository contains are `utf` and `8`, both of which it uses everywhere. The
+gate says too little of that question is distinctive, which is defensible, and
+it was getting the right answer on a coincidence.
+
+**What it does not fix:** an English question whose words genuinely occur here
+in another sense. `how is the OAuth refresh token rotated` matches `refresh`
+because this repository refreshes *indexes*, and no threshold separates those.
+Six of fifteen English absent questions still get answered for that reason.
+That is the case a real embedding model exists for, and it is measurable now
+that the ruler exists.
+
+An empty answer says which kind of empty it is, because each is recovered by a
+different move:
+
+```json
+{"results": [],
+ "diagnosis": {"reason": "only_ubiquitous_terms_matched",
+               "matched_terms": [], "ubiquitous_terms": ["the", "to"],
+               "coverage": 0.0, "min_coverage": 0.4,
+               "hint": "The only words that matched are ones this repository uses throughout ..."}}
+```
+
 Descriptions live in `rag-your-code.descriptions.json` at the repository root
 and are meant to be committed, so one person's pass benefits everyone who
 clones. Each is keyed by unit id **and a digest of the unit's source**: when
@@ -332,23 +403,28 @@ per-release counts are in [CHANGELOG.md](CHANGELOG.md).
 
 ## Configuration
 
-Twelve settings in `rag-your-code.toml` at the repository root:
+21 settings in `rag-your-code.toml` at the repository root:
 
 ```bash
 rag-your-code config init                     # a commented file, all defaults
 rag-your-code config list                     # effective values and their source
 rag-your-code config set index.ignore '["vendor", "generated"]'
-rag-your-code config set search.vector_weight 0.25
+rag-your-code config set search.min_coverage 0.25
 ```
 
 | section | settings |
 |---|---|
 | `[index]` | `ignore`, `suffixes`, `max_file_bytes` |
-| `[embedding]` | `dimensions` |
-| `[search]` | `vector_weight`, `limit`, `max_chars` |
+| `[embedding]` | `dimensions`, `provider`, `endpoint`, `model`, `api_key_env`, `batch`, `timeout`, `retries` |
+| `[search]` | `min_coverage`, `vector_weight`, `vector_recall`, `limit`, `max_chars` |
 | `[agent]` | `max_open_bytes`, `max_open_chars` |
 | `[describe]` | `languages`, `batch`, `max_chars` |
 
+This table is asserted against the settings table in `config.py`, in both
+directions, by `tests/test_metadata.py` — it had already fallen nine settings
+behind by 1.0.0, and a section listing three quarters of what exists is worse
+than none, because it reads as complete.
+
 Resolution is CLI flag > file > built-in default. There is no environment
 layer: an index is an artifact of a repository, not of a shell.
 
@@ -358,9 +434,10 @@ silently dropped is indistinguishable from one that had no effect.
 suffix it cannot read is walked, parsed to nothing, and reported as a clean
 index of zero units.
 
-The four settings under `[index]` and `[embedding]` decide what an index
-*contains*, so a digest of them is stored in the index and a change forces a
-full rebuild. The rest take effect immediately and invalidate nothing.
+The settings under `[index]` and `[embedding]` that decide what an index
+*contains* — including which provider and model computed its vectors — have a
+digest stored in the index, and changing one forces a full rebuild. The rest
+take effect immediately and invalidate nothing.
 
 ## Agent protocol
 
@@ -407,14 +484,66 @@ it stopped.
 Nothing authored lives under `.rag-your-code/` — that directory is what people
 delete to clear the cache.
 
-## Not here yet
+## Bringing your own model
+
+Everything above works with no model at all. If you would rather have real
+semantics, point the index at any OpenAI-compatible embeddings endpoint —
+which includes a model server on your own machine:
+
+```toml
+# rag-your-code.toml
+[embedding]
+provider   = "openai-compatible"
+endpoint   = "http://localhost:11434/v1/embeddings"   # ollama, LM Studio, vLLM…
+model      = "nomic-embed-text"
+dimensions = 768                                      # must match the model
+```
+
+A hosted service is the same three lines with an `https://` endpoint, plus the
+name of the environment variable holding your key:
+
+```toml
+api_key_env = "OPENAI_API_KEY"    # the NAME of the variable, never the key
+```
 
-Provider-backed embeddings, Tree-sitter parsing, and a SQLite/ANN storage layer
-for repositories past the measured JSON envelope. Agent-authored descriptions
-are deliberately the cheaper answer to the same problem provider embeddings
-solve: they keep the zero-dependency, offline, reproducible-index properties,
-and produce text a human can read and correct rather than opaque floats. See
-[docs/ROADMAP.md](docs/ROADMAP.md).
+**The key is never a setting.** `rag-your-code.toml` is meant to be committed
+so everyone who clones can see what shaped the index; a credential is the one
+value with the opposite requirement, so the file only ever names the variable
+it lives in. Sending a key over plain `http://` to anything but your own
+machine is refused rather than warned about.
+
+Three things follow from turning this on, and it is worth knowing all three
+before you do:
+
+- **Your source leaves the machine**, unless the endpoint is local. That is
+  the whole reason the local case is written first here.
+- **Similarity may now find things, not just order them.** With the local
+  hash a cosine shortlist is measurably noise, so it is confined to
+  re-ranking. A real model earns the right to add candidates the words never
+  reached, which is the one gap no amount of ranking closes:
+  `search.vector_recall` sets how many. Lexical evidence still dominates — a
+  unit found by similarity alone scores at most `search.vector_weight`.
+- **A failure stops the build.** Falling back to the local hash would leave an
+  index whose vectors come from two incompatible spaces, and ranking would act
+  on the meaningless cosine between them with full confidence.
+
+Switching provider, model or width discards the old vectors and rebuilds, so
+an index can never be a mixture. An incremental run over unchanged files makes
+no request at all.
+
+**What is not measured:** whether this helps *your* repository, and by how
+much. No number here is from a real model — this project has no key, and a
+figure produced by a stub would be fiction. The instrument ships instead:
+point `benchmarks/repo_queries.py --index` at your own index and grade it. You
+will probably also want a higher `search.vector_weight` than the 0.15 tuned
+for a hash that carries no meaning.
+
+## Still not here
+
+Tree-sitter parsing, and a SQLite/ANN storage layer for repositories past the
+measured JSON envelope. Note also that `search.vector_recall` scans every
+unit's vector on every query, which is fine at the measured envelope and is
+the thing an ANN index would replace. See [docs/ROADMAP.md](docs/ROADMAP.md).
 
 ## Development
 

{rag_your_code-0.7.0 → rag_your_code-1.0.0}/README.md

@@ -199,9 +199,9 @@ and left Chinese retrieval unchanged.
 
 ### Measured on this repository
 
-This project describes its own implementation: every unit under `src/` carries
-an agent-written bilingual description, committed to the repo, and 68 of them
-have been promoted into the source as doc comments.
+This project describes its own implementation: all 163 units under `src/` carry
+an agent-written bilingual description, committed to the repo, and 155 of those
+163 declarations also carry the author's own documentation in the source.
 
 Seventy natural-language questions about this codebase, in English and
 Chinese, each listing every unit that genuinely answers it
@@ -209,13 +209,13 @@ Chinese, each listing every unit that genuinely answers it
 
 | | generated descriptions | agent-written |
 |---|---|---|
-| hit@1 | 0.271 | **0.500** |
-| hit@3 | 0.486 | **0.800** |
-| MRR | 0.367 | **0.631** |
-| answered with no shared word at all | 12.9% | **0%** |
+| hit@1 | 0.314 | **0.500** |
+| hit@3 | 0.457 | **0.729** |
+| MRR | 0.379 | **0.583** |
+| questions declined for want of evidence | 13.0% | **4.3%** |
 
-Roughly double the first-place accuracy. Fourteen questions still fail, which
-is what makes the set usable for measuring the next change;
+Half again the first-place accuracy. Nineteen questions still fail, which is
+what makes the set usable for measuring the next change;
 `tests/test_repo_queries.py` asserts that some question always does, and that
 the written column beats the generated one.
 
@@ -256,6 +256,77 @@ the code it tests, because it repeats that code's vocabulary and adds its own.
 Matching stays lexical. It is LLM-authored keyword expansion, and its reach is
 bounded by how many ways of saying the thing the agent thought to write down.
 
+### The question a ranking cannot answer
+
+Both rulers above ask questions that *have* an answer, so both can only score
+whether it was found. Neither can see the opposite failure. A ranking always
+produces a least-bad unit and hands it back with a score and a rank that read
+exactly like an answer — and it does that whether or not the repository
+contains anything relevant at all.
+
+So there is a third ruler: thirty questions about subjects neither repository
+implements, where the only correct reply is nothing
+([`benchmarks/absent_queries.json`](benchmarks/absent_queries.json)). Before
+1.0.0 it scored **zero**. All thirty answered, on both repositories, in both
+languages:
+
+| asked of a repository with no such code | answered with | on the evidence of |
+|---|---|---|
+| `where are CUDA kernels dispatched to the device` | a test about word counting | `are` `the` `to` `where` |
+| `准入控制为什么会拒绝没有资源限额的容器组` | the UTF-8 console setup | `拒绝` `控制` `没有` |
+| `how is the OAuth refresh token rotated` | a description-store method | `before` `is` `refresh` `the` |
+
+Not a Chinese problem and not a ranking problem — a missing question. Nothing
+in the pipeline asked *is any of this evidence*; it only asked which ranks
+highest. Retrieval now asks both, and returns nothing when the answer to the
+first is no:
+
+| | before | now |
+|---|---|---|
+| unanswerable questions correctly met with silence, this repository | 0.000 | **0.733** |
+| the same, on the foreign repository | 0.000 | **0.800** |
+| results resting on no lexical evidence at all, all three rulers | 0.029 – 0.129 | **0.000** |
+| hit@1 / hit@3 / MRR on the foreign ruler | 0.257 / 0.400 / 0.314 | **unchanged** |
+| hit@1 / hit@3 / MRR on this repository | 0.471 / 0.686 / 0.557 | **unchanged** |
+
+The bar is the share of a question's **discriminating** words that occur in
+the index — words the repository uses everywhere are dropped from both sides
+of that fraction, which is the part that does the work. Half of `where are
+CUDA kernels dispatched to the device` matches, and it looks like evidence
+until you notice which half. Counting only words that distinguish silenced 18
+of 30 unanswerable English questions that no plain coverage threshold reached
+at all, at identical cost in real answers — 97 of 98 either way.
+
+It is a ratio inside the query rather than a threshold on a score, because a
+score threshold is tied to whatever scale the ranking currently produces —
+this project has already had one of those stop meaning anything the moment
+BM25F changed the scale. `search.min_coverage` sets it; `0` restores the old
+behaviour exactly.
+
+**What it costs:** one question of the 158 measured. `控制台编码不是 UTF-8
+会怎么样` was reaching `_use_utf8_streams`, and the only words in it this
+repository contains are `utf` and `8`, both of which it uses everywhere. The
+gate says too little of that question is distinctive, which is defensible, and
+it was getting the right answer on a coincidence.
+
+**What it does not fix:** an English question whose words genuinely occur here
+in another sense. `how is the OAuth refresh token rotated` matches `refresh`
+because this repository refreshes *indexes*, and no threshold separates those.
+Six of fifteen English absent questions still get answered for that reason.
+That is the case a real embedding model exists for, and it is measurable now
+that the ruler exists.
+
+An empty answer says which kind of empty it is, because each is recovered by a
+different move:
+
+```json
+{"results": [],
+ "diagnosis": {"reason": "only_ubiquitous_terms_matched",
+               "matched_terms": [], "ubiquitous_terms": ["the", "to"],
+               "coverage": 0.0, "min_coverage": 0.4,
+               "hint": "The only words that matched are ones this repository uses throughout ..."}}
+```
+
 Descriptions live in `rag-your-code.descriptions.json` at the repository root
 and are meant to be committed, so one person's pass benefits everyone who
 clones. Each is keyed by unit id **and a digest of the unit's source**: when
@@ -305,23 +376,28 @@ per-release counts are in [CHANGELOG.md](CHANGELOG.md).
 
 ## Configuration
 
-Twelve settings in `rag-your-code.toml` at the repository root:
+21 settings in `rag-your-code.toml` at the repository root:
 
 ```bash
 rag-your-code config init                     # a commented file, all defaults
 rag-your-code config list                     # effective values and their source
 rag-your-code config set index.ignore '["vendor", "generated"]'
-rag-your-code config set search.vector_weight 0.25
+rag-your-code config set search.min_coverage 0.25
 ```
 
 | section | settings |
 |---|---|
 | `[index]` | `ignore`, `suffixes`, `max_file_bytes` |
-| `[embedding]` | `dimensions` |
-| `[search]` | `vector_weight`, `limit`, `max_chars` |
+| `[embedding]` | `dimensions`, `provider`, `endpoint`, `model`, `api_key_env`, `batch`, `timeout`, `retries` |
+| `[search]` | `min_coverage`, `vector_weight`, `vector_recall`, `limit`, `max_chars` |
 | `[agent]` | `max_open_bytes`, `max_open_chars` |
 | `[describe]` | `languages`, `batch`, `max_chars` |
 
+This table is asserted against the settings table in `config.py`, in both
+directions, by `tests/test_metadata.py` — it had already fallen nine settings
+behind by 1.0.0, and a section listing three quarters of what exists is worse
+than none, because it reads as complete.
+
 Resolution is CLI flag > file > built-in default. There is no environment
 layer: an index is an artifact of a repository, not of a shell.
 
@@ -331,9 +407,10 @@ silently dropped is indistinguishable from one that had no effect.
 suffix it cannot read is walked, parsed to nothing, and reported as a clean
 index of zero units.
 
-The four settings under `[index]` and `[embedding]` decide what an index
-*contains*, so a digest of them is stored in the index and a change forces a
-full rebuild. The rest take effect immediately and invalidate nothing.
+The settings under `[index]` and `[embedding]` that decide what an index
+*contains* — including which provider and model computed its vectors — have a
+digest stored in the index, and changing one forces a full rebuild. The rest
+take effect immediately and invalidate nothing.
 
 ## Agent protocol
 
@@ -380,14 +457,66 @@ it stopped.
 Nothing authored lives under `.rag-your-code/` — that directory is what people
 delete to clear the cache.
 
-## Not here yet
+## Bringing your own model
+
+Everything above works with no model at all. If you would rather have real
+semantics, point the index at any OpenAI-compatible embeddings endpoint —
+which includes a model server on your own machine:
+
+```toml
+# rag-your-code.toml
+[embedding]
+provider   = "openai-compatible"
+endpoint   = "http://localhost:11434/v1/embeddings"   # ollama, LM Studio, vLLM…
+model      = "nomic-embed-text"
+dimensions = 768                                      # must match the model
+```
+
+A hosted service is the same three lines with an `https://` endpoint, plus the
+name of the environment variable holding your key:
+
+```toml
+api_key_env = "OPENAI_API_KEY"    # the NAME of the variable, never the key
+```
 
-Provider-backed embeddings, Tree-sitter parsing, and a SQLite/ANN storage layer
-for repositories past the measured JSON envelope. Agent-authored descriptions
-are deliberately the cheaper answer to the same problem provider embeddings
-solve: they keep the zero-dependency, offline, reproducible-index properties,
-and produce text a human can read and correct rather than opaque floats. See
-[docs/ROADMAP.md](docs/ROADMAP.md).
+**The key is never a setting.** `rag-your-code.toml` is meant to be committed
+so everyone who clones can see what shaped the index; a credential is the one
+value with the opposite requirement, so the file only ever names the variable
+it lives in. Sending a key over plain `http://` to anything but your own
+machine is refused rather than warned about.
+
+Three things follow from turning this on, and it is worth knowing all three
+before you do:
+
+- **Your source leaves the machine**, unless the endpoint is local. That is
+  the whole reason the local case is written first here.
+- **Similarity may now find things, not just order them.** With the local
+  hash a cosine shortlist is measurably noise, so it is confined to
+  re-ranking. A real model earns the right to add candidates the words never
+  reached, which is the one gap no amount of ranking closes:
+  `search.vector_recall` sets how many. Lexical evidence still dominates — a
+  unit found by similarity alone scores at most `search.vector_weight`.
+- **A failure stops the build.** Falling back to the local hash would leave an
+  index whose vectors come from two incompatible spaces, and ranking would act
+  on the meaningless cosine between them with full confidence.
+
+Switching provider, model or width discards the old vectors and rebuilds, so
+an index can never be a mixture. An incremental run over unchanged files makes
+no request at all.
+
+**What is not measured:** whether this helps *your* repository, and by how
+much. No number here is from a real model — this project has no key, and a
+figure produced by a stub would be fiction. The instrument ships instead:
+point `benchmarks/repo_queries.py --index` at your own index and grade it. You
+will probably also want a higher `search.vector_weight` than the 0.15 tuned
+for a hash that carries no meaning.
+
+## Still not here
+
+Tree-sitter parsing, and a SQLite/ANN storage layer for repositories past the
+measured JSON envelope. Note also that `search.vector_recall` scans every
+unit's vector on every query, which is fine at the measured envelope and is
+the thing an ANN index would replace. See [docs/ROADMAP.md](docs/ROADMAP.md).
 
 ## Development
 

{rag_your_code-0.7.0 → rag_your_code-1.0.0}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "rag-your-code"
-version = "0.7.0"
+version = "1.0.0"
 description = "A local, explainable RAG index for codebases and coding agents"
 readme = "README.md"
 requires-python = ">=3.10"