rag-your-code 1.5.0__tar.gz → 1.5.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {rag_your_code-1.5.0/src/rag_your_code.egg-info → rag_your_code-1.5.2}/PKG-INFO +45 -49
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/README.md +44 -48
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/pyproject.toml +1 -1
- {rag_your_code-1.5.0 → rag_your_code-1.5.2/src/rag_your_code.egg-info}/PKG-INFO +45 -49
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/__init__.py +1 -1
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/LICENSE +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/setup.cfg +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/rag_your_code.egg-info/SOURCES.txt +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/rag_your_code.egg-info/dependency_links.txt +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/rag_your_code.egg-info/entry_points.txt +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/rag_your_code.egg-info/requires.txt +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/rag_your_code.egg-info/top_level.txt +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/agentic.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/annotate.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/cli.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/config.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/descriptions.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/document.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/embeddings.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/graph.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/indexer.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/models.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/parser.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/providers.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/py.typed +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/search.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/src/ragyourcode/workflow.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_absent_queries.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_agent_protocol.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_agentic.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_config.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_descriptions.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_diagrams.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_doc_comments.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_document.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_e2e_cli.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_evidence.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_golden.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_graph_incremental.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_language_fixtures.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_large_repo.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_local_model.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_metadata.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_multilanguage.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_parser_edges.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_providers.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_ragyourcode.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_ranking.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_repo_queries.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_resilience.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_retrieval_correctness.py +0 -0
- {rag_your_code-1.5.0 → rag_your_code-1.5.2}/tests/test_workflow.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: rag-your-code
|
|
3
|
-
Version: 1.5.
|
|
3
|
+
Version: 1.5.2
|
|
4
4
|
Summary: A local, explainable RAG index for codebases and coding agents
|
|
5
5
|
Author: rag-your-code contributors
|
|
6
6
|
License-Expression: MIT
|
|
@@ -74,7 +74,7 @@ whether or not the repository holds anything relevant.
|
|
|
74
74
|
| **Refuse** | Two evidence tests decide whether *any* result is an answer; when neither is met, retrieval returns nothing plus a machine-readable diagnosis. |
|
|
75
75
|
| **Expand** | Optional bounded walk over `calls` / `imports` / `contains`, each hop carrying its edge path as evidence. |
|
|
76
76
|
| **Describe** | Your agent writes the vocabulary the source never contained, stored in a committed sidecar or promoted into the code as a reviewable diff. |
|
|
77
|
-
| **Serve** | A CLI,
|
|
77
|
+
| **Serve** | A CLI, plus a JSON-lines protocol for a long-lived agent subprocess. |
|
|
78
78
|
|
|
79
79
|
**Scope.** Retrieval over source declarations — not a code-understanding model,
|
|
80
80
|
not a generation step, not an IDE index. Questions are answered in vocabulary
|
|
@@ -123,8 +123,8 @@ stopped existing the moment BM25F changed that scale.
|
|
|
123
123
|
### 3.2 · The vector was carrying nothing, and here is why
|
|
124
124
|
|
|
125
125
|
The default embedder is a signed feature hash. Ablating it entirely moves the
|
|
126
|
-
|
|
127
|
-
occupy **72.1%** of the index.
|
|
126
|
+
four positive rulers by **at most two questions, and in both directions**, while
|
|
127
|
+
the vectors occupy **72.1%** of the index. Known since 0.6.0 and unexplained.
|
|
128
128
|
The explanation, measured here:
|
|
129
129
|
|
|
130
130
|
- **Not saturation.** Median 56 distinct tokens per unit into 384 buckets, 0.4%
|
|
@@ -299,8 +299,8 @@ repository had grown by ninety units.
|
|
|
299
299
|
| **C** this repo, agent-written descriptions | the warmest case supported | 70 | 0.429 | 0.600 | 0.498 |
|
|
300
300
|
|
|
301
301
|
The corpora, without which none of the above is reproducible — **E** 602 units,
|
|
302
|
-
`3eabaa705477`; **A** 1,572 units, `5fd51169eacc`; **B**
|
|
303
|
-
`
|
|
302
|
+
`3eabaa705477`; **A** 1,572 units, `5fd51169eacc`; **B** 604 units,
|
|
303
|
+
`81e47eb0a50c`; **C** 604 units, `f84556ba7881`. Both foreign subjects are
|
|
304
304
|
carried in this repository at pinned tags, under
|
|
305
305
|
[`benchmarks/corpus/`](benchmarks/corpus/), and CI runs both as ordinary jobs.
|
|
306
306
|
|
|
@@ -325,12 +325,12 @@ Silence is lower on both foreign corpora than on this one, and the cause is a
|
|
|
325
325
|
limit of the design rather than a defect. A word counts as evidence unless it
|
|
326
326
|
occurs in more than 5% of units — a stopword list derived from the corpus, so
|
|
327
327
|
that it needs no list and works in any language. Here `how`, `when`, `does` and
|
|
328
|
-
`are` are everywhere, because
|
|
328
|
+
`are` are everywhere, because 317 units carry written English prose. Across a
|
|
329
329
|
corpus of short, tersely documented declarations they occur in 1–5% of them and
|
|
330
330
|
start counting as evidence: on cobra, three English questions get through on
|
|
331
331
|
sets like `[a, is, the, how, after]`.
|
|
332
332
|
|
|
333
|
-
**What each bar costs and buys** —
|
|
333
|
+
**What each bar costs and buys** — corpora stamped above, gate varied alone:
|
|
334
334
|
|
|
335
335
|
| gate | A | B | C | E | silence own / Flask / cobra |
|
|
336
336
|
|---|---|---|---|---|---|
|
|
@@ -340,9 +340,8 @@ sets like `[a, is, the, how, after]`.
|
|
|
340
340
|
| **both (1.1.0)** | **0.200/0.286/0.238** | **0.314/0.471/0.381** | 0.429/0.600/0.498 | 0.075/0.150/0.108 | **0.967 / 0.833 / 0.900** |
|
|
341
341
|
|
|
342
342
|
Ruler A is **unmoved by either bar**; B loses one hit@3 question to either bar
|
|
343
|
-
alone and nothing further when both apply. The rest
|
|
344
|
-
|
|
345
|
-
two of forty on the Go one.
|
|
343
|
+
alone and nothing further when both apply. The rest is four of seventy at hit@1
|
|
344
|
+
on the warmest ruler and six at hit@3, plus one and two of forty on the Go one.
|
|
346
345
|
|
|
347
346
|
**Both bars together give the best silence on all three corpora.** Through
|
|
348
347
|
1.3.0 this section said concentration subsumes coverage; on Flask it does not
|
|
@@ -357,28 +356,24 @@ hit@1, B to 0.214 and C to 0.329. **0.28 was chosen before either foreign
|
|
|
357
356
|
corpus existed and survived meeting both**, which is the only kind of evidence
|
|
358
357
|
a default can have.
|
|
359
358
|
|
|
360
|
-
**Latency** — warm corpus,
|
|
359
|
+
**Latency** — warm corpus, 604 units `f84556ba7881`, one
|
|
361
360
|
`python -m benchmarks.query_latency` (5 repeats × 420 samples), idle machine:
|
|
362
361
|
|
|
363
362
|
| | | across the repeats |
|
|
364
363
|
|---|---|---|
|
|
365
|
-
| query, median | **0.
|
|
366
|
-
| query, p95 |
|
|
367
|
-
| refusing an unanswerable query | **0.016 ms** | 0.015 – 0.
|
|
368
|
-
| refusal cheaper than answering by | **~
|
|
369
|
-
|
|
370
|
-
*Idle* is load-bearing
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
script that was never committed; both sit inside that band, which is the point:
|
|
379
|
-
they were unfalsifiable rather than wrong. Refusal is cheap structurally rather
|
|
380
|
-
than by tuning — an unanswerable query touches only the posting lists of its
|
|
381
|
-
own distinctive words and never reaches ranking.
|
|
364
|
+
| query, median | **0.73 ms** | 0.70 – 0.78 |
|
|
365
|
+
| query, p95 | 1.30 ms | 1.17 – 1.39 |
|
|
366
|
+
| refusing an unanswerable query | **0.016 ms** | 0.015 – 0.018 |
|
|
367
|
+
| refusal cheaper than answering by | **~44×** | 40 – 47 |
|
|
368
|
+
|
|
369
|
+
*Idle* is load-bearing, and nothing in the report can see whether it was: the
|
|
370
|
+
same corpus at the same commit measures about twice this median while another
|
|
371
|
+
job is running. Two significant figures and a spread, because across releases
|
|
372
|
+
that spread has been wider than any change the code has made to this number —
|
|
373
|
+
which is why the row ships with its range and its corpus stamp rather than to
|
|
374
|
+
three figures. Refusal is cheap structurally, not by tuning: an unanswerable
|
|
375
|
+
query touches only its distinctive words' posting lists and never reaches
|
|
376
|
+
ranking.
|
|
382
377
|
|
|
383
378
|
**Scale**, synthetic 10,000-unit repository (500 files), re-measured in 1.5.0
|
|
384
379
|
— the previous row of figures was optimistic by more than noise:
|
|
@@ -408,7 +403,7 @@ it again.
|
|
|
408
403
|
Directional local measurements, not service levels — but each is a command
|
|
409
404
|
rather than a memory, which two of them were not before. Each
|
|
410
405
|
prints the corpus fingerprint beside its score; quote both or neither.
|
|
411
|
-
[`benchmarks/README.md`](benchmarks/README.md) lists the
|
|
406
|
+
[`benchmarks/README.md`](benchmarks/README.md) lists the seven scripts and what
|
|
412
407
|
each is for, and the corpus one of them grades is now carried here too.
|
|
413
408
|
|
|
414
409
|
## 7 · `rag-your-code search` vs a Grep loop
|
|
@@ -437,12 +432,12 @@ payload: this side hands back a fifth to a half of the text, ranked and spanned.
|
|
|
437
432
|
|
|
438
433
|
**Once the vocabulary exists, it is not close.**
|
|
439
434
|
|
|
440
|
-
| this repository · 70 questions ·
|
|
435
|
+
| this repository · 70 questions · 604 units `f84556ba7881` · 317 described | Grep loop | rag-your-code |
|
|
441
436
|
|---|---|---|
|
|
442
437
|
| right file first | 22.9% | **58.6%** |
|
|
443
|
-
| right file in top 3 |
|
|
444
|
-
| lines it hands back, all questions | 12,
|
|
445
|
-
| characters returned, all questions | 1,
|
|
438
|
+
| right file in top 3 | 52.9% | **78.6%** |
|
|
439
|
+
| lines it hands back, all questions | 12,540 | — |
|
|
440
|
+
| characters returned, all questions | 1,190,816 | **611,859** |
|
|
446
441
|
| questions it answers | **61** | 60 |
|
|
447
442
|
|
|
448
443
|
That is section 3.3's argument measured rather than asserted, and it is the one
|
|
@@ -718,24 +713,25 @@ questions: it is the corpus's limit, not this tool's.
|
|
|
718
713
|
measured on every ruler that then existed: it improves both own-repository
|
|
719
714
|
rulers and costs the foreign one 3 of 35 hit@3, so it was rejected.
|
|
720
715
|
|
|
721
|
-
**A test declaration sometimes outranks real code** — 9 of
|
|
722
|
-
|
|
723
|
-
none of them
|
|
724
|
-
code it *tests*, is wrong: five of the
|
|
725
|
-
prose. A callee-before-caller rerank fires
|
|
726
|
-
field weight moves nothing because an
|
|
716
|
+
**A test declaration sometimes outranks real code** — 9 of 215 questions across
|
|
717
|
+
all four positive rulers (`benchmarks/displacement.py`), a test at rank 1
|
|
718
|
+
displacing an accepted answer at rank 2–3, **none of them foreign**. The long-standing
|
|
719
|
+
explanation, that a test outranks the code it *tests*, is wrong: five of the
|
|
720
|
+
nine are unrelated tests winning on prose. A callee-before-caller rerank fires
|
|
721
|
+
on zero questions, and the `name` field weight moves nothing because an
|
|
722
|
+
underscored test name is one token.
|
|
727
723
|
|
|
728
724
|
**Describing a declaration that already has a good docstring loses ground.** An
|
|
729
725
|
authored description *replaces* the generated sentence, which is the only route
|
|
730
|
-
by which the author's
|
|
731
|
-
writing one demotes
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
**The vectors are 72.1% of the index and earn
|
|
737
|
-
embedder — 74.8%
|
|
738
|
-
|
|
726
|
+
by which the author's docstring reaches the weight-3 description field — so
|
|
727
|
+
writing one demotes it to the weight-1 body. On `parser.py::_generic_units` a
|
|
728
|
+
long description cost one graded question and a short one cost three;
|
|
729
|
+
appending the docstring to every description instead cost the 1.5.0 corpus
|
|
730
|
+
0.443 → 0.414 hit@1. `describe.skip` records the decision.
|
|
731
|
+
|
|
732
|
+
**The vectors are 72.1% of the index and earn at most two questions** on any
|
|
733
|
+
ruler, in either direction, under the default embedder — 74.8% Flask, 79.7%
|
|
734
|
+
cobra. Kept: that same storage is what an optional model needs.
|
|
739
735
|
|
|
740
736
|
**`search.vector_recall` scans every vector per query** — under a semantic
|
|
741
737
|
embedder. The default hash never widens at all. Affordable at the measured
|
|
@@ -44,7 +44,7 @@ whether or not the repository holds anything relevant.
|
|
|
44
44
|
| **Refuse** | Two evidence tests decide whether *any* result is an answer; when neither is met, retrieval returns nothing plus a machine-readable diagnosis. |
|
|
45
45
|
| **Expand** | Optional bounded walk over `calls` / `imports` / `contains`, each hop carrying its edge path as evidence. |
|
|
46
46
|
| **Describe** | Your agent writes the vocabulary the source never contained, stored in a committed sidecar or promoted into the code as a reviewable diff. |
|
|
47
|
-
| **Serve** | A CLI,
|
|
47
|
+
| **Serve** | A CLI, plus a JSON-lines protocol for a long-lived agent subprocess. |
|
|
48
48
|
|
|
49
49
|
**Scope.** Retrieval over source declarations — not a code-understanding model,
|
|
50
50
|
not a generation step, not an IDE index. Questions are answered in vocabulary
|
|
@@ -93,8 +93,8 @@ stopped existing the moment BM25F changed that scale.
|
|
|
93
93
|
### 3.2 · The vector was carrying nothing, and here is why
|
|
94
94
|
|
|
95
95
|
The default embedder is a signed feature hash. Ablating it entirely moves the
|
|
96
|
-
|
|
97
|
-
occupy **72.1%** of the index.
|
|
96
|
+
four positive rulers by **at most two questions, and in both directions**, while
|
|
97
|
+
the vectors occupy **72.1%** of the index. Known since 0.6.0 and unexplained.
|
|
98
98
|
The explanation, measured here:
|
|
99
99
|
|
|
100
100
|
- **Not saturation.** Median 56 distinct tokens per unit into 384 buckets, 0.4%
|
|
@@ -269,8 +269,8 @@ repository had grown by ninety units.
|
|
|
269
269
|
| **C** this repo, agent-written descriptions | the warmest case supported | 70 | 0.429 | 0.600 | 0.498 |
|
|
270
270
|
|
|
271
271
|
The corpora, without which none of the above is reproducible — **E** 602 units,
|
|
272
|
-
`3eabaa705477`; **A** 1,572 units, `5fd51169eacc`; **B**
|
|
273
|
-
`
|
|
272
|
+
`3eabaa705477`; **A** 1,572 units, `5fd51169eacc`; **B** 604 units,
|
|
273
|
+
`81e47eb0a50c`; **C** 604 units, `f84556ba7881`. Both foreign subjects are
|
|
274
274
|
carried in this repository at pinned tags, under
|
|
275
275
|
[`benchmarks/corpus/`](benchmarks/corpus/), and CI runs both as ordinary jobs.
|
|
276
276
|
|
|
@@ -295,12 +295,12 @@ Silence is lower on both foreign corpora than on this one, and the cause is a
|
|
|
295
295
|
limit of the design rather than a defect. A word counts as evidence unless it
|
|
296
296
|
occurs in more than 5% of units — a stopword list derived from the corpus, so
|
|
297
297
|
that it needs no list and works in any language. Here `how`, `when`, `does` and
|
|
298
|
-
`are` are everywhere, because
|
|
298
|
+
`are` are everywhere, because 317 units carry written English prose. Across a
|
|
299
299
|
corpus of short, tersely documented declarations they occur in 1–5% of them and
|
|
300
300
|
start counting as evidence: on cobra, three English questions get through on
|
|
301
301
|
sets like `[a, is, the, how, after]`.
|
|
302
302
|
|
|
303
|
-
**What each bar costs and buys** —
|
|
303
|
+
**What each bar costs and buys** — corpora stamped above, gate varied alone:
|
|
304
304
|
|
|
305
305
|
| gate | A | B | C | E | silence own / Flask / cobra |
|
|
306
306
|
|---|---|---|---|---|---|
|
|
@@ -310,9 +310,8 @@ sets like `[a, is, the, how, after]`.
|
|
|
310
310
|
| **both (1.1.0)** | **0.200/0.286/0.238** | **0.314/0.471/0.381** | 0.429/0.600/0.498 | 0.075/0.150/0.108 | **0.967 / 0.833 / 0.900** |
|
|
311
311
|
|
|
312
312
|
Ruler A is **unmoved by either bar**; B loses one hit@3 question to either bar
|
|
313
|
-
alone and nothing further when both apply. The rest
|
|
314
|
-
|
|
315
|
-
two of forty on the Go one.
|
|
313
|
+
alone and nothing further when both apply. The rest is four of seventy at hit@1
|
|
314
|
+
on the warmest ruler and six at hit@3, plus one and two of forty on the Go one.
|
|
316
315
|
|
|
317
316
|
**Both bars together give the best silence on all three corpora.** Through
|
|
318
317
|
1.3.0 this section said concentration subsumes coverage; on Flask it does not
|
|
@@ -327,28 +326,24 @@ hit@1, B to 0.214 and C to 0.329. **0.28 was chosen before either foreign
|
|
|
327
326
|
corpus existed and survived meeting both**, which is the only kind of evidence
|
|
328
327
|
a default can have.
|
|
329
328
|
|
|
330
|
-
**Latency** — warm corpus,
|
|
329
|
+
**Latency** — warm corpus, 604 units `f84556ba7881`, one
|
|
331
330
|
`python -m benchmarks.query_latency` (5 repeats × 420 samples), idle machine:
|
|
332
331
|
|
|
333
332
|
| | | across the repeats |
|
|
334
333
|
|---|---|---|
|
|
335
|
-
| query, median | **0.
|
|
336
|
-
| query, p95 |
|
|
337
|
-
| refusing an unanswerable query | **0.016 ms** | 0.015 – 0.
|
|
338
|
-
| refusal cheaper than answering by | **~
|
|
339
|
-
|
|
340
|
-
*Idle* is load-bearing
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
script that was never committed; both sit inside that band, which is the point:
|
|
349
|
-
they were unfalsifiable rather than wrong. Refusal is cheap structurally rather
|
|
350
|
-
than by tuning — an unanswerable query touches only the posting lists of its
|
|
351
|
-
own distinctive words and never reaches ranking.
|
|
334
|
+
| query, median | **0.73 ms** | 0.70 – 0.78 |
|
|
335
|
+
| query, p95 | 1.30 ms | 1.17 – 1.39 |
|
|
336
|
+
| refusing an unanswerable query | **0.016 ms** | 0.015 – 0.018 |
|
|
337
|
+
| refusal cheaper than answering by | **~44×** | 40 – 47 |
|
|
338
|
+
|
|
339
|
+
*Idle* is load-bearing, and nothing in the report can see whether it was: the
|
|
340
|
+
same corpus at the same commit measures about twice this median while another
|
|
341
|
+
job is running. Two significant figures and a spread, because across releases
|
|
342
|
+
that spread has been wider than any change the code has made to this number —
|
|
343
|
+
which is why the row ships with its range and its corpus stamp rather than to
|
|
344
|
+
three figures. Refusal is cheap structurally, not by tuning: an unanswerable
|
|
345
|
+
query touches only its distinctive words' posting lists and never reaches
|
|
346
|
+
ranking.
|
|
352
347
|
|
|
353
348
|
**Scale**, synthetic 10,000-unit repository (500 files), re-measured in 1.5.0
|
|
354
349
|
— the previous row of figures was optimistic by more than noise:
|
|
@@ -378,7 +373,7 @@ it again.
|
|
|
378
373
|
Directional local measurements, not service levels — but each is a command
|
|
379
374
|
rather than a memory, which two of them were not before. Each
|
|
380
375
|
prints the corpus fingerprint beside its score; quote both or neither.
|
|
381
|
-
[`benchmarks/README.md`](benchmarks/README.md) lists the
|
|
376
|
+
[`benchmarks/README.md`](benchmarks/README.md) lists the seven scripts and what
|
|
382
377
|
each is for, and the corpus one of them grades is now carried here too.
|
|
383
378
|
|
|
384
379
|
## 7 · `rag-your-code search` vs a Grep loop
|
|
@@ -407,12 +402,12 @@ payload: this side hands back a fifth to a half of the text, ranked and spanned.
|
|
|
407
402
|
|
|
408
403
|
**Once the vocabulary exists, it is not close.**
|
|
409
404
|
|
|
410
|
-
| this repository · 70 questions ·
|
|
405
|
+
| this repository · 70 questions · 604 units `f84556ba7881` · 317 described | Grep loop | rag-your-code |
|
|
411
406
|
|---|---|---|
|
|
412
407
|
| right file first | 22.9% | **58.6%** |
|
|
413
|
-
| right file in top 3 |
|
|
414
|
-
| lines it hands back, all questions | 12,
|
|
415
|
-
| characters returned, all questions | 1,
|
|
408
|
+
| right file in top 3 | 52.9% | **78.6%** |
|
|
409
|
+
| lines it hands back, all questions | 12,540 | — |
|
|
410
|
+
| characters returned, all questions | 1,190,816 | **611,859** |
|
|
416
411
|
| questions it answers | **61** | 60 |
|
|
417
412
|
|
|
418
413
|
That is section 3.3's argument measured rather than asserted, and it is the one
|
|
@@ -688,24 +683,25 @@ questions: it is the corpus's limit, not this tool's.
|
|
|
688
683
|
measured on every ruler that then existed: it improves both own-repository
|
|
689
684
|
rulers and costs the foreign one 3 of 35 hit@3, so it was rejected.
|
|
690
685
|
|
|
691
|
-
**A test declaration sometimes outranks real code** — 9 of
|
|
692
|
-
|
|
693
|
-
none of them
|
|
694
|
-
code it *tests*, is wrong: five of the
|
|
695
|
-
prose. A callee-before-caller rerank fires
|
|
696
|
-
field weight moves nothing because an
|
|
686
|
+
**A test declaration sometimes outranks real code** — 9 of 215 questions across
|
|
687
|
+
all four positive rulers (`benchmarks/displacement.py`), a test at rank 1
|
|
688
|
+
displacing an accepted answer at rank 2–3, **none of them foreign**. The long-standing
|
|
689
|
+
explanation, that a test outranks the code it *tests*, is wrong: five of the
|
|
690
|
+
nine are unrelated tests winning on prose. A callee-before-caller rerank fires
|
|
691
|
+
on zero questions, and the `name` field weight moves nothing because an
|
|
692
|
+
underscored test name is one token.
|
|
697
693
|
|
|
698
694
|
**Describing a declaration that already has a good docstring loses ground.** An
|
|
699
695
|
authored description *replaces* the generated sentence, which is the only route
|
|
700
|
-
by which the author's
|
|
701
|
-
writing one demotes
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
**The vectors are 72.1% of the index and earn
|
|
707
|
-
embedder — 74.8%
|
|
708
|
-
|
|
696
|
+
by which the author's docstring reaches the weight-3 description field — so
|
|
697
|
+
writing one demotes it to the weight-1 body. On `parser.py::_generic_units` a
|
|
698
|
+
long description cost one graded question and a short one cost three;
|
|
699
|
+
appending the docstring to every description instead cost the 1.5.0 corpus
|
|
700
|
+
0.443 → 0.414 hit@1. `describe.skip` records the decision.
|
|
701
|
+
|
|
702
|
+
**The vectors are 72.1% of the index and earn at most two questions** on any
|
|
703
|
+
ruler, in either direction, under the default embedder — 74.8% Flask, 79.7%
|
|
704
|
+
cobra. Kept: that same storage is what an optional model needs.
|
|
709
705
|
|
|
710
706
|
**`search.vector_recall` scans every vector per query** — under a semantic
|
|
711
707
|
embedder. The default hash never widens at all. Affordable at the measured
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: rag-your-code
|
|
3
|
-
Version: 1.5.
|
|
3
|
+
Version: 1.5.2
|
|
4
4
|
Summary: A local, explainable RAG index for codebases and coding agents
|
|
5
5
|
Author: rag-your-code contributors
|
|
6
6
|
License-Expression: MIT
|
|
@@ -74,7 +74,7 @@ whether or not the repository holds anything relevant.
|
|
|
74
74
|
| **Refuse** | Two evidence tests decide whether *any* result is an answer; when neither is met, retrieval returns nothing plus a machine-readable diagnosis. |
|
|
75
75
|
| **Expand** | Optional bounded walk over `calls` / `imports` / `contains`, each hop carrying its edge path as evidence. |
|
|
76
76
|
| **Describe** | Your agent writes the vocabulary the source never contained, stored in a committed sidecar or promoted into the code as a reviewable diff. |
|
|
77
|
-
| **Serve** | A CLI,
|
|
77
|
+
| **Serve** | A CLI, plus a JSON-lines protocol for a long-lived agent subprocess. |
|
|
78
78
|
|
|
79
79
|
**Scope.** Retrieval over source declarations — not a code-understanding model,
|
|
80
80
|
not a generation step, not an IDE index. Questions are answered in vocabulary
|
|
@@ -123,8 +123,8 @@ stopped existing the moment BM25F changed that scale.
|
|
|
123
123
|
### 3.2 · The vector was carrying nothing, and here is why
|
|
124
124
|
|
|
125
125
|
The default embedder is a signed feature hash. Ablating it entirely moves the
|
|
126
|
-
|
|
127
|
-
occupy **72.1%** of the index.
|
|
126
|
+
four positive rulers by **at most two questions, and in both directions**, while
|
|
127
|
+
the vectors occupy **72.1%** of the index. Known since 0.6.0 and unexplained.
|
|
128
128
|
The explanation, measured here:
|
|
129
129
|
|
|
130
130
|
- **Not saturation.** Median 56 distinct tokens per unit into 384 buckets, 0.4%
|
|
@@ -299,8 +299,8 @@ repository had grown by ninety units.
|
|
|
299
299
|
| **C** this repo, agent-written descriptions | the warmest case supported | 70 | 0.429 | 0.600 | 0.498 |
|
|
300
300
|
|
|
301
301
|
The corpora, without which none of the above is reproducible — **E** 602 units,
|
|
302
|
-
`3eabaa705477`; **A** 1,572 units, `5fd51169eacc`; **B**
|
|
303
|
-
`
|
|
302
|
+
`3eabaa705477`; **A** 1,572 units, `5fd51169eacc`; **B** 604 units,
|
|
303
|
+
`81e47eb0a50c`; **C** 604 units, `f84556ba7881`. Both foreign subjects are
|
|
304
304
|
carried in this repository at pinned tags, under
|
|
305
305
|
[`benchmarks/corpus/`](benchmarks/corpus/), and CI runs both as ordinary jobs.
|
|
306
306
|
|
|
@@ -325,12 +325,12 @@ Silence is lower on both foreign corpora than on this one, and the cause is a
|
|
|
325
325
|
limit of the design rather than a defect. A word counts as evidence unless it
|
|
326
326
|
occurs in more than 5% of units — a stopword list derived from the corpus, so
|
|
327
327
|
that it needs no list and works in any language. Here `how`, `when`, `does` and
|
|
328
|
-
`are` are everywhere, because
|
|
328
|
+
`are` are everywhere, because 317 units carry written English prose. Across a
|
|
329
329
|
corpus of short, tersely documented declarations they occur in 1–5% of them and
|
|
330
330
|
start counting as evidence: on cobra, three English questions get through on
|
|
331
331
|
sets like `[a, is, the, how, after]`.
|
|
332
332
|
|
|
333
|
-
**What each bar costs and buys** —
|
|
333
|
+
**What each bar costs and buys** — corpora stamped above, gate varied alone:
|
|
334
334
|
|
|
335
335
|
| gate | A | B | C | E | silence own / Flask / cobra |
|
|
336
336
|
|---|---|---|---|---|---|
|
|
@@ -340,9 +340,8 @@ sets like `[a, is, the, how, after]`.
|
|
|
340
340
|
| **both (1.1.0)** | **0.200/0.286/0.238** | **0.314/0.471/0.381** | 0.429/0.600/0.498 | 0.075/0.150/0.108 | **0.967 / 0.833 / 0.900** |
|
|
341
341
|
|
|
342
342
|
Ruler A is **unmoved by either bar**; B loses one hit@3 question to either bar
|
|
343
|
-
alone and nothing further when both apply. The rest
|
|
344
|
-
|
|
345
|
-
two of forty on the Go one.
|
|
343
|
+
alone and nothing further when both apply. The rest is four of seventy at hit@1
|
|
344
|
+
on the warmest ruler and six at hit@3, plus one and two of forty on the Go one.
|
|
346
345
|
|
|
347
346
|
**Both bars together give the best silence on all three corpora.** Through
|
|
348
347
|
1.3.0 this section said concentration subsumes coverage; on Flask it does not
|
|
@@ -357,28 +356,24 @@ hit@1, B to 0.214 and C to 0.329. **0.28 was chosen before either foreign
|
|
|
357
356
|
corpus existed and survived meeting both**, which is the only kind of evidence
|
|
358
357
|
a default can have.
|
|
359
358
|
|
|
360
|
-
**Latency** — warm corpus,
|
|
359
|
+
**Latency** — warm corpus, 604 units `f84556ba7881`, one
|
|
361
360
|
`python -m benchmarks.query_latency` (5 repeats × 420 samples), idle machine:
|
|
362
361
|
|
|
363
362
|
| | | across the repeats |
|
|
364
363
|
|---|---|---|
|
|
365
|
-
| query, median | **0.
|
|
366
|
-
| query, p95 |
|
|
367
|
-
| refusing an unanswerable query | **0.016 ms** | 0.015 – 0.
|
|
368
|
-
| refusal cheaper than answering by | **~
|
|
369
|
-
|
|
370
|
-
*Idle* is load-bearing
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
script that was never committed; both sit inside that band, which is the point:
|
|
379
|
-
they were unfalsifiable rather than wrong. Refusal is cheap structurally rather
|
|
380
|
-
than by tuning — an unanswerable query touches only the posting lists of its
|
|
381
|
-
own distinctive words and never reaches ranking.
|
|
364
|
+
| query, median | **0.73 ms** | 0.70 – 0.78 |
|
|
365
|
+
| query, p95 | 1.30 ms | 1.17 – 1.39 |
|
|
366
|
+
| refusing an unanswerable query | **0.016 ms** | 0.015 – 0.018 |
|
|
367
|
+
| refusal cheaper than answering by | **~44×** | 40 – 47 |
|
|
368
|
+
|
|
369
|
+
*Idle* is load-bearing, and nothing in the report can see whether it was: the
|
|
370
|
+
same corpus at the same commit measures about twice this median while another
|
|
371
|
+
job is running. Two significant figures and a spread, because across releases
|
|
372
|
+
that spread has been wider than any change the code has made to this number —
|
|
373
|
+
which is why the row ships with its range and its corpus stamp rather than to
|
|
374
|
+
three figures. Refusal is cheap structurally, not by tuning: an unanswerable
|
|
375
|
+
query touches only its distinctive words' posting lists and never reaches
|
|
376
|
+
ranking.
|
|
382
377
|
|
|
383
378
|
**Scale**, synthetic 10,000-unit repository (500 files), re-measured in 1.5.0
|
|
384
379
|
— the previous row of figures was optimistic by more than noise:
|
|
@@ -408,7 +403,7 @@ it again.
|
|
|
408
403
|
Directional local measurements, not service levels — but each is a command
|
|
409
404
|
rather than a memory, which two of them were not before. Each
|
|
410
405
|
prints the corpus fingerprint beside its score; quote both or neither.
|
|
411
|
-
[`benchmarks/README.md`](benchmarks/README.md) lists the
|
|
406
|
+
[`benchmarks/README.md`](benchmarks/README.md) lists the seven scripts and what
|
|
412
407
|
each is for, and the corpus one of them grades is now carried here too.
|
|
413
408
|
|
|
414
409
|
## 7 · `rag-your-code search` vs a Grep loop
|
|
@@ -437,12 +432,12 @@ payload: this side hands back a fifth to a half of the text, ranked and spanned.
|
|
|
437
432
|
|
|
438
433
|
**Once the vocabulary exists, it is not close.**
|
|
439
434
|
|
|
440
|
-
| this repository · 70 questions ·
|
|
435
|
+
| this repository · 70 questions · 604 units `f84556ba7881` · 317 described | Grep loop | rag-your-code |
|
|
441
436
|
|---|---|---|
|
|
442
437
|
| right file first | 22.9% | **58.6%** |
|
|
443
|
-
| right file in top 3 |
|
|
444
|
-
| lines it hands back, all questions | 12,
|
|
445
|
-
| characters returned, all questions | 1,
|
|
438
|
+
| right file in top 3 | 52.9% | **78.6%** |
|
|
439
|
+
| lines it hands back, all questions | 12,540 | — |
|
|
440
|
+
| characters returned, all questions | 1,190,816 | **611,859** |
|
|
446
441
|
| questions it answers | **61** | 60 |
|
|
447
442
|
|
|
448
443
|
That is section 3.3's argument measured rather than asserted, and it is the one
|
|
@@ -718,24 +713,25 @@ questions: it is the corpus's limit, not this tool's.
|
|
|
718
713
|
measured on every ruler that then existed: it improves both own-repository
|
|
719
714
|
rulers and costs the foreign one 3 of 35 hit@3, so it was rejected.
|
|
720
715
|
|
|
721
|
-
**A test declaration sometimes outranks real code** — 9 of
|
|
722
|
-
|
|
723
|
-
none of them
|
|
724
|
-
code it *tests*, is wrong: five of the
|
|
725
|
-
prose. A callee-before-caller rerank fires
|
|
726
|
-
field weight moves nothing because an
|
|
716
|
+
**A test declaration sometimes outranks real code** — 9 of 215 questions across
|
|
717
|
+
all four positive rulers (`benchmarks/displacement.py`), a test at rank 1
|
|
718
|
+
displacing an accepted answer at rank 2–3, **none of them foreign**. The long-standing
|
|
719
|
+
explanation, that a test outranks the code it *tests*, is wrong: five of the
|
|
720
|
+
nine are unrelated tests winning on prose. A callee-before-caller rerank fires
|
|
721
|
+
on zero questions, and the `name` field weight moves nothing because an
|
|
722
|
+
underscored test name is one token.
|
|
727
723
|
|
|
728
724
|
**Describing a declaration that already has a good docstring loses ground.** An
|
|
729
725
|
authored description *replaces* the generated sentence, which is the only route
|
|
730
|
-
by which the author's
|
|
731
|
-
writing one demotes
|
|
732
|
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
**The vectors are 72.1% of the index and earn
|
|
737
|
-
embedder — 74.8%
|
|
738
|
-
|
|
726
|
+
by which the author's docstring reaches the weight-3 description field — so
|
|
727
|
+
writing one demotes it to the weight-1 body. On `parser.py::_generic_units` a
|
|
728
|
+
long description cost one graded question and a short one cost three;
|
|
729
|
+
appending the docstring to every description instead cost the 1.5.0 corpus
|
|
730
|
+
0.443 → 0.414 hit@1. `describe.skip` records the decision.
|
|
731
|
+
|
|
732
|
+
**The vectors are 72.1% of the index and earn at most two questions** on any
|
|
733
|
+
ruler, in either direction, under the default embedder — 74.8% Flask, 79.7%
|
|
734
|
+
cobra. Kept: that same storage is what an optional model needs.
|
|
739
735
|
|
|
740
736
|
**`search.vector_recall` scans every vector per query** — under a semantic
|
|
741
737
|
embedder. The default hash never widens at all. Affordable at the measured
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|