rag-your-code 0.4.1__tar.gz → 0.4.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- rag_your_code-0.4.2/PKG-INFO +330 -0
- rag_your_code-0.4.2/README.md +303 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/pyproject.toml +1 -1
- rag_your_code-0.4.2/src/rag_your_code.egg-info/PKG-INFO +330 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/__init__.py +1 -1
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_metadata.py +30 -0
- rag_your_code-0.4.1/PKG-INFO +0 -237
- rag_your_code-0.4.1/README.md +0 -210
- rag_your_code-0.4.1/src/rag_your_code.egg-info/PKG-INFO +0 -237
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/LICENSE +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/setup.cfg +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/rag_your_code.egg-info/SOURCES.txt +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/rag_your_code.egg-info/dependency_links.txt +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/rag_your_code.egg-info/entry_points.txt +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/rag_your_code.egg-info/requires.txt +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/rag_your_code.egg-info/top_level.txt +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/agentic.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/annotate.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/cli.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/config.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/descriptions.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/embeddings.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/graph.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/indexer.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/models.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/parser.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/py.typed +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/src/ragyourcode/search.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_agent_protocol.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_agentic.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_config.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_descriptions.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_e2e_cli.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_golden.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_graph_incremental.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_language_fixtures.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_large_repo.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_multilanguage.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_parser_edges.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_ragyourcode.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_resilience.py +0 -0
- {rag_your_code-0.4.1 → rag_your_code-0.4.2}/tests/test_retrieval_correctness.py +0 -0
|
@@ -0,0 +1,330 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: rag-your-code
|
|
3
|
+
Version: 0.4.2
|
|
4
|
+
Summary: A local, explainable RAG index for codebases and coding agents
|
|
5
|
+
Author: rag-your-code contributors
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Keywords: rag,code-search,retrieval,indexing,graphrag,offline,explainable,agent
|
|
8
|
+
Classifier: Development Status :: 4 - Beta
|
|
9
|
+
Classifier: Environment :: Console
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
17
|
+
Classifier: Topic :: Software Development :: Libraries
|
|
18
|
+
Classifier: Topic :: Text Processing :: Indexing
|
|
19
|
+
Classifier: Typing :: Typed
|
|
20
|
+
Requires-Python: >=3.10
|
|
21
|
+
Description-Content-Type: text/markdown
|
|
22
|
+
License-File: LICENSE
|
|
23
|
+
Provides-Extra: dev
|
|
24
|
+
Requires-Dist: pytest>=7; extra == "dev"
|
|
25
|
+
Requires-Dist: tomli>=2.0; python_version < "3.11" and extra == "dev"
|
|
26
|
+
Dynamic: license-file
|
|
27
|
+
|
|
28
|
+
# RAG Your Code
|
|
29
|
+
|
|
30
|
+
**A local code-retrieval index for coding agents.** Ask a question in plain
|
|
31
|
+
language, get back the functions that answer it — each with its file, its exact
|
|
32
|
+
line range, the words that matched, and its source.
|
|
33
|
+
|
|
34
|
+
[](https://pypi.org/project/rag-your-code/)
|
|
35
|
+
[](LICENSE)
|
|
36
|
+
[](pyproject.toml)
|
|
37
|
+
|
|
38
|
+
No network calls. No runtime dependencies. No model. It is built to run over a
|
|
39
|
+
private repository on a machine with the network switched off, and to produce
|
|
40
|
+
an index a human can read.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## The problem it solves
|
|
45
|
+
|
|
46
|
+
An agent that needs to find something in an unfamiliar codebase has two bad
|
|
47
|
+
options. It can grep — fast, but it only finds the string you already guessed.
|
|
48
|
+
Or it can read whole files into context — thorough, but a handful of them
|
|
49
|
+
exhausts the budget and most of what it read was irrelevant.
|
|
50
|
+
|
|
51
|
+
This sits in between. It indexes every function, method and class once, then
|
|
52
|
+
answers a question with the eight units most likely to be relevant, at roughly
|
|
53
|
+
a hundred lines instead of ten thousand. Every result carries its provenance,
|
|
54
|
+
so the agent can open the real code before it edits anything, and you can see
|
|
55
|
+
why each one came back.
|
|
56
|
+
|
|
57
|
+
It is the **R** in RAG. There is no generation here — your agent is the G.
|
|
58
|
+
|
|
59
|
+
## Install
|
|
60
|
+
|
|
61
|
+
**As a Claude Code plugin** (this is the primary way to use it):
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
/plugin marketplace add skymanbp/rag-your-code
|
|
65
|
+
/plugin install rag-your-code@rag-your-code
|
|
66
|
+
/reload-plugins
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The plugin is one skill and nothing else — no hooks, no agents, no MCP server.
|
|
70
|
+
Measured with `claude plugin details`: **~39 tokens added to every session**,
|
|
71
|
+
and ~1.4k only when the skill actually fires. The skill installs the Python
|
|
72
|
+
package itself on first use.
|
|
73
|
+
|
|
74
|
+
**Or as a plain CLI:**
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
pip install rag-your-code
|
|
78
|
+
|
|
79
|
+
rag-your-code index .
|
|
80
|
+
rag-your-code search "where are HTTP retries handled" --json
|
|
81
|
+
rag-your-code search "what calls the retry handler" --graph --hops 1 --json
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
The index is written under `.rag-your-code/`; your source files are never
|
|
85
|
+
modified. Later `index` runs reuse unchanged files. For a large repository,
|
|
86
|
+
prefer `rag-your-code index . --compact`.
|
|
87
|
+
|
|
88
|
+
## How it works
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
your repository
|
|
92
|
+
→ walk source files (configurable ignores, suffixes, size cap)
|
|
93
|
+
→ parse declarations Python via its own AST; 14 other languages via a
|
|
94
|
+
line scanner + per-language rule table
|
|
95
|
+
→ one CodeUnit each id, signature, exact line range, source, calls,
|
|
96
|
+
imports, a stable serial number, a description
|
|
97
|
+
→ embed description + source into a deterministic local vector
|
|
98
|
+
→ inverted word index + hybrid ranking
|
|
99
|
+
→ optional graph expansion over calls / imports / contains
|
|
100
|
+
→ results, or a JSON-lines protocol for an agent subprocess
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
**Parsing.** Python goes through the standard-library syntax tree, so nesting,
|
|
104
|
+
qualified names, call lists and line ranges are exact. Every other language
|
|
105
|
+
goes through three separated layers: a scanner that reads one line at a time,
|
|
106
|
+
a rule table per language, and a span closer that follows brace depth, Ruby's
|
|
107
|
+
`end`, or the next declaration. Because a pattern never sees a second line, a
|
|
108
|
+
reported line number *is* the scanner's loop index and cannot drift, and a
|
|
109
|
+
declaration cannot swallow the ones after it.
|
|
110
|
+
|
|
111
|
+
Fifteen languages: Python, JavaScript, TypeScript, Go, Rust, Java, Kotlin,
|
|
112
|
+
Scala, C#, C, C++, Ruby, PHP, Swift, shell.
|
|
113
|
+
|
|
114
|
+
**Graph.** `calls`, `imports` and `contains` edges, each conservative: an
|
|
115
|
+
unresolved or ambiguous reference produces no edge rather than a guessed one,
|
|
116
|
+
and every expanded result carries the exact edge path as evidence.
|
|
117
|
+
|
|
118
|
+
## What the embedding does — and what it does not
|
|
119
|
+
|
|
120
|
+
This matters more than any feature list, so it is here rather than in a
|
|
121
|
+
footnote.
|
|
122
|
+
|
|
123
|
+
The embedder is a **signed feature hash**: it hashes words into 384 buckets.
|
|
124
|
+
Cosine similarity over those vectors is therefore a normalised measure of
|
|
125
|
+
*shared words*, and it carries no semantics whatsoever:
|
|
126
|
+
|
|
127
|
+
| pair | cosine |
|
|
128
|
+
|---|---|
|
|
129
|
+
| `retry failed card charge` vs itself | 1.0000 |
|
|
130
|
+
| `sum two numbers` vs `add a pair of integers` | **0.0000** |
|
|
131
|
+
| `计算两个数的和` vs `sum two numbers` | **0.0000** |
|
|
132
|
+
| `sum two numbers` vs `delete the user database table` | 0.0000 |
|
|
133
|
+
|
|
134
|
+
A trained embedding model scores row 2 at around 0.8. Here a synonym pair and
|
|
135
|
+
an unrelated pair are indistinguishable, because no shared word is no shared
|
|
136
|
+
word either way.
|
|
137
|
+
|
|
138
|
+
Retrieval works regardless, because **identifiers and docstrings are already
|
|
139
|
+
natural language** — `retry_charge` contains *retry* and *charge*. But it
|
|
140
|
+
reaches only concepts somebody wrote down. Two things close the rest of the
|
|
141
|
+
gap, and neither is a model:
|
|
142
|
+
|
|
143
|
+
- **Your agent rewrites the query.** It has the conversation; turning
|
|
144
|
+
"重试扣款" into `retry charge payment gateway` costs it nothing.
|
|
145
|
+
- **Your agent writes the descriptions**, which puts the missing vocabulary
|
|
146
|
+
into the index once instead of into every query.
|
|
147
|
+
|
|
148
|
+
## Agent-authored descriptions
|
|
149
|
+
|
|
150
|
+
Every unit carries a description, and that description is indexed. By default
|
|
151
|
+
it is generated without a model: the identifier humanised, the parameters and
|
|
152
|
+
callees listed, the docstring appended. It introduces no vocabulary the source
|
|
153
|
+
did not already have — which is exactly why retrieval cannot reach a concept
|
|
154
|
+
nobody wrote down.
|
|
155
|
+
|
|
156
|
+
The agent already reading the index can supply those words:
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
rag-your-code describe status # coverage, and what is pending
|
|
160
|
+
rag-your-code describe export --limit 20 # a batch, with source and a brief
|
|
161
|
+
rag-your-code describe import written.json # store what the agent wrote
|
|
162
|
+
rag-your-code index . # apply it
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
or, in the protocol, `describe_pending` and `describe_put` — which take effect
|
|
166
|
+
in the same session, with no refresh.
|
|
167
|
+
|
|
168
|
+
### Measured on this repository
|
|
169
|
+
|
|
170
|
+
This project describes its own implementation: all 120 units under `src/`
|
|
171
|
+
carry an agent-written bilingual description, committed to the repo. Ten
|
|
172
|
+
natural-language questions about the codebase, before and after:
|
|
173
|
+
|
|
174
|
+
| | before | after |
|
|
175
|
+
|---|---|---|
|
|
176
|
+
| top result in the expected file | 2 / 8 | **6 / 8** |
|
|
177
|
+
| Chinese-language queries | 0 / 4 | **3 / 4** |
|
|
178
|
+
| queries with **no shared word at all** (pure fallback) | 4 | **0** |
|
|
179
|
+
|
|
180
|
+
Two still miss, and both are worth stating. One query says `catastrophic
|
|
181
|
+
backtracking` where the description says `backtracks catastrophically`: there
|
|
182
|
+
is no stemming, so those share no word — exactly the limit documented above.
|
|
183
|
+
The other returned a unit that answers the question from a different file than
|
|
184
|
+
predicted, so the expectation was wrong rather than the retrieval.
|
|
185
|
+
|
|
186
|
+
**What this is:** it moves the semantic work from query time to index time.
|
|
187
|
+
Matching stays lexical. It is LLM-authored keyword expansion, and its reach is
|
|
188
|
+
bounded by how many ways of saying the thing the agent thought to write down.
|
|
189
|
+
|
|
190
|
+
Descriptions live in `rag-your-code.descriptions.json` at the repository root
|
|
191
|
+
and are meant to be committed, so one person's pass benefits everyone who
|
|
192
|
+
clones. Each is keyed by unit id **and a digest of the unit's source**: when
|
|
193
|
+
the code changes, the description is not applied, the unit returns to the
|
|
194
|
+
pending queue, and retrieval falls back to the generated sentence. A
|
|
195
|
+
description that outlived its code would be a confident wrong answer, which is
|
|
196
|
+
the one thing this index is built not to give. When code merely *moves* — an
|
|
197
|
+
import added above it — the description follows it by digest.
|
|
198
|
+
|
|
199
|
+
## Measured
|
|
200
|
+
|
|
201
|
+
**Parsing**, against source-controlled fixtures in `tests/fixtures/languages/`
|
|
202
|
+
(15 fixture files, 96 expected units, 237 negative cases, 89 constructs the
|
|
203
|
+
spec deliberately excludes):
|
|
204
|
+
|
|
205
|
+
| | |
|
|
206
|
+
|---|---|
|
|
207
|
+
| core declarations found | **91 / 91** |
|
|
208
|
+
| with the correct `start_line` | **91 / 91** |
|
|
209
|
+
| with a usable signature | **91 / 91** |
|
|
210
|
+
| units that do not exist | **0** |
|
|
211
|
+
|
|
212
|
+
A 441-byte JavaScript file that once took **12.6 s** to parse now takes
|
|
213
|
+
**0.36 ms**, and 10 KB takes 2.1 ms — growth is linear again.
|
|
214
|
+
|
|
215
|
+
**Scale**, on a synthetic 10,000-unit repository (500 files):
|
|
216
|
+
|
|
217
|
+
| | |
|
|
218
|
+
|---|---|
|
|
219
|
+
| full build | 1.84 s |
|
|
220
|
+
| incremental rebuild after one file changes | 0.207 s (**8.9x**) |
|
|
221
|
+
| compact storage vs readable JSON | 35.6% |
|
|
222
|
+
| index load, in a fresh process | 45.4 ms |
|
|
223
|
+
| inverted index build | 117.7 ms |
|
|
224
|
+
| resident memory | 58.7 MiB |
|
|
225
|
+
| query, mean of 200 warmed samples | 3.90 ms |
|
|
226
|
+
|
|
227
|
+
Directional local measurements, not service levels; the archived run is
|
|
228
|
+
[`large-benchmark-result.json`](large-benchmark-result.json).
|
|
229
|
+
|
|
230
|
+
**Suite:** Python 3.10 – 3.13 on Linux and Windows, plus a job that installs
|
|
231
|
+
the built wheel into a clean environment and runs every command the
|
|
232
|
+
documentation prescribes, and another that runs the skill's own install line
|
|
233
|
+
verbatim. 208 tests as of 0.4.2 — the count is version-stamped rather than
|
|
234
|
+
maintained, because a bare figure in a living document is a claim that rots;
|
|
235
|
+
per-release counts are in [CHANGELOG.md](CHANGELOG.md).
|
|
236
|
+
|
|
237
|
+
## Configuration
|
|
238
|
+
|
|
239
|
+
Twelve settings in `rag-your-code.toml` at the repository root:
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
rag-your-code config init # a commented file, all defaults
|
|
243
|
+
rag-your-code config list # effective values and their source
|
|
244
|
+
rag-your-code config set index.ignore '["vendor", "generated"]'
|
|
245
|
+
rag-your-code config set search.vector_weight 0.25
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
| section | settings |
|
|
249
|
+
|---|---|
|
|
250
|
+
| `[index]` | `ignore`, `suffixes`, `max_file_bytes` |
|
|
251
|
+
| `[embedding]` | `dimensions` |
|
|
252
|
+
| `[search]` | `vector_weight`, `limit`, `max_chars` |
|
|
253
|
+
| `[agent]` | `max_open_bytes`, `max_open_chars` |
|
|
254
|
+
| `[describe]` | `languages`, `batch`, `max_chars` |
|
|
255
|
+
|
|
256
|
+
Resolution is CLI flag > file > built-in default. There is no environment
|
|
257
|
+
layer: an index is an artifact of a repository, not of a shell.
|
|
258
|
+
|
|
259
|
+
An unknown key or an out-of-range value is an error, not a shrug — a setting
|
|
260
|
+
silently dropped is indistinguishable from one that had no effect.
|
|
261
|
+
`index.suffixes` may only name suffixes the parser has rules for, because a
|
|
262
|
+
suffix it cannot read is walked, parsed to nothing, and reported as a clean
|
|
263
|
+
index of zero units.
|
|
264
|
+
|
|
265
|
+
The four settings under `[index]` and `[embedding]` decide what an index
|
|
266
|
+
*contains*, so a digest of them is stored in the index and a change forces a
|
|
267
|
+
full rebuild. The rest take effect immediately and invalidate nothing.
|
|
268
|
+
|
|
269
|
+
## Agent protocol
|
|
270
|
+
|
|
271
|
+
`rag-your-code agent --root PATH` reads one JSON request per line and writes
|
|
272
|
+
one reply per line:
|
|
273
|
+
|
|
274
|
+
```json
|
|
275
|
+
{"action":"search","query":"database transaction rollback","limit":5}
|
|
276
|
+
{"action":"research","query":"trace payment retry behavior","max_steps":2}
|
|
277
|
+
{"action":"neighbors","id":"payments.py:4:retry_charge","hops":1}
|
|
278
|
+
{"action":"open","path":"payments.py","start_line":1,"end_line":80}
|
|
279
|
+
{"action":"describe_pending","limit":20}
|
|
280
|
+
{"action":"describe_put","descriptions":[{"id":"payments.py:4:retry_charge","text":"..."}]}
|
|
281
|
+
{"action":"refresh"}
|
|
282
|
+
{"action":"stats"}
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
**No single request can end the session.** Numeric fields saturate at their
|
|
286
|
+
bounds, `open` is bounded in both lines and bytes, and anything unanticipated
|
|
287
|
+
is reported in-band with its exception type. Streams are pinned to UTF-8
|
|
288
|
+
rather than following the console codepage.
|
|
289
|
+
|
|
290
|
+
`research` is a deliberately bounded two-step controller: retrieve, then at
|
|
291
|
+
most one graph expansion when confidence is low, reporting each step and why
|
|
292
|
+
it stopped.
|
|
293
|
+
|
|
294
|
+
## What lives where
|
|
295
|
+
|
|
296
|
+
| path | authored or generated | commit it? |
|
|
297
|
+
|---|---|---|
|
|
298
|
+
| `rag-your-code.toml` | authored | yes |
|
|
299
|
+
| `rag-your-code.descriptions.json` | authored by your agent | yes |
|
|
300
|
+
| `.rag-your-code/` (index, vectors, annotations) | generated | no |
|
|
301
|
+
|
|
302
|
+
Nothing authored lives under `.rag-your-code/` — that directory is what people
|
|
303
|
+
delete to clear the cache.
|
|
304
|
+
|
|
305
|
+
## Not here yet
|
|
306
|
+
|
|
307
|
+
Provider-backed embeddings, Tree-sitter parsing, and a SQLite/ANN storage layer
|
|
308
|
+
for repositories past the measured JSON envelope. Agent-authored descriptions
|
|
309
|
+
are deliberately the cheaper answer to the same problem provider embeddings
|
|
310
|
+
solve: they keep the zero-dependency, offline, reproducible-index properties,
|
|
311
|
+
and produce text a human can read and correct rather than opaque floats. See
|
|
312
|
+
[docs/ROADMAP.md](docs/ROADMAP.md).
|
|
313
|
+
|
|
314
|
+
## Development
|
|
315
|
+
|
|
316
|
+
```bash
|
|
317
|
+
python -m pip install -e ".[dev]"
|
|
318
|
+
pytest -q
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
No runtime dependencies; `pytest` and, below Python 3.11, `tomli` come from the
|
|
322
|
+
`dev` extra.
|
|
323
|
+
|
|
324
|
+
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — how each stage works and why
|
|
325
|
+
- [docs/TESTING.md](docs/TESTING.md) — what the suites are protecting
|
|
326
|
+
- [docs/ROADMAP.md](docs/ROADMAP.md) — what shipped, what is still open
|
|
327
|
+
- [CONTRIBUTING.md](CONTRIBUTING.md) — ground rules, and how to add a language
|
|
328
|
+
- [CHANGELOG.md](CHANGELOG.md) — every release, with its measurements
|
|
329
|
+
|
|
330
|
+
MIT licensed.
|
|
@@ -0,0 +1,303 @@
|
|
|
1
|
+
# RAG Your Code
|
|
2
|
+
|
|
3
|
+
**A local code-retrieval index for coding agents.** Ask a question in plain
|
|
4
|
+
language, get back the functions that answer it — each with its file, its exact
|
|
5
|
+
line range, the words that matched, and its source.
|
|
6
|
+
|
|
7
|
+
[](https://pypi.org/project/rag-your-code/)
|
|
8
|
+
[](LICENSE)
|
|
9
|
+
[](pyproject.toml)
|
|
10
|
+
|
|
11
|
+
No network calls. No runtime dependencies. No model. It is built to run over a
|
|
12
|
+
private repository on a machine with the network switched off, and to produce
|
|
13
|
+
an index a human can read.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## The problem it solves
|
|
18
|
+
|
|
19
|
+
An agent that needs to find something in an unfamiliar codebase has two bad
|
|
20
|
+
options. It can grep — fast, but it only finds the string you already guessed.
|
|
21
|
+
Or it can read whole files into context — thorough, but a handful of them
|
|
22
|
+
exhausts the budget and most of what it read was irrelevant.
|
|
23
|
+
|
|
24
|
+
This sits in between. It indexes every function, method and class once, then
|
|
25
|
+
answers a question with the eight units most likely to be relevant, at roughly
|
|
26
|
+
a hundred lines instead of ten thousand. Every result carries its provenance,
|
|
27
|
+
so the agent can open the real code before it edits anything, and you can see
|
|
28
|
+
why each one came back.
|
|
29
|
+
|
|
30
|
+
It is the **R** in RAG. There is no generation here — your agent is the G.
|
|
31
|
+
|
|
32
|
+
## Install
|
|
33
|
+
|
|
34
|
+
**As a Claude Code plugin** (this is the primary way to use it):
|
|
35
|
+
|
|
36
|
+
```
|
|
37
|
+
/plugin marketplace add skymanbp/rag-your-code
|
|
38
|
+
/plugin install rag-your-code@rag-your-code
|
|
39
|
+
/reload-plugins
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
The plugin is one skill and nothing else — no hooks, no agents, no MCP server.
|
|
43
|
+
Measured with `claude plugin details`: **~39 tokens added to every session**,
|
|
44
|
+
and ~1.4k only when the skill actually fires. The skill installs the Python
|
|
45
|
+
package itself on first use.
|
|
46
|
+
|
|
47
|
+
**Or as a plain CLI:**
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
pip install rag-your-code
|
|
51
|
+
|
|
52
|
+
rag-your-code index .
|
|
53
|
+
rag-your-code search "where are HTTP retries handled" --json
|
|
54
|
+
rag-your-code search "what calls the retry handler" --graph --hops 1 --json
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
The index is written under `.rag-your-code/`; your source files are never
|
|
58
|
+
modified. Later `index` runs reuse unchanged files. For a large repository,
|
|
59
|
+
prefer `rag-your-code index . --compact`.
|
|
60
|
+
|
|
61
|
+
## How it works
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
your repository
|
|
65
|
+
→ walk source files (configurable ignores, suffixes, size cap)
|
|
66
|
+
→ parse declarations Python via its own AST; 14 other languages via a
|
|
67
|
+
line scanner + per-language rule table
|
|
68
|
+
→ one CodeUnit each id, signature, exact line range, source, calls,
|
|
69
|
+
imports, a stable serial number, a description
|
|
70
|
+
→ embed description + source into a deterministic local vector
|
|
71
|
+
→ inverted word index + hybrid ranking
|
|
72
|
+
→ optional graph expansion over calls / imports / contains
|
|
73
|
+
→ results, or a JSON-lines protocol for an agent subprocess
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
**Parsing.** Python goes through the standard-library syntax tree, so nesting,
|
|
77
|
+
qualified names, call lists and line ranges are exact. Every other language
|
|
78
|
+
goes through three separated layers: a scanner that reads one line at a time,
|
|
79
|
+
a rule table per language, and a span closer that follows brace depth, Ruby's
|
|
80
|
+
`end`, or the next declaration. Because a pattern never sees a second line, a
|
|
81
|
+
reported line number *is* the scanner's loop index and cannot drift, and a
|
|
82
|
+
declaration cannot swallow the ones after it.
|
|
83
|
+
|
|
84
|
+
Fifteen languages: Python, JavaScript, TypeScript, Go, Rust, Java, Kotlin,
|
|
85
|
+
Scala, C#, C, C++, Ruby, PHP, Swift, shell.
|
|
86
|
+
|
|
87
|
+
**Graph.** `calls`, `imports` and `contains` edges, each conservative: an
|
|
88
|
+
unresolved or ambiguous reference produces no edge rather than a guessed one,
|
|
89
|
+
and every expanded result carries the exact edge path as evidence.
|
|
90
|
+
|
|
91
|
+
## What the embedding does — and what it does not
|
|
92
|
+
|
|
93
|
+
This matters more than any feature list, so it is here rather than in a
|
|
94
|
+
footnote.
|
|
95
|
+
|
|
96
|
+
The embedder is a **signed feature hash**: it hashes words into 384 buckets.
|
|
97
|
+
Cosine similarity over those vectors is therefore a normalised measure of
|
|
98
|
+
*shared words*, and it carries no semantics whatsoever:
|
|
99
|
+
|
|
100
|
+
| pair | cosine |
|
|
101
|
+
|---|---|
|
|
102
|
+
| `retry failed card charge` vs itself | 1.0000 |
|
|
103
|
+
| `sum two numbers` vs `add a pair of integers` | **0.0000** |
|
|
104
|
+
| `计算两个数的和` vs `sum two numbers` | **0.0000** |
|
|
105
|
+
| `sum two numbers` vs `delete the user database table` | 0.0000 |
|
|
106
|
+
|
|
107
|
+
A trained embedding model scores row 2 at around 0.8. Here a synonym pair and
|
|
108
|
+
an unrelated pair are indistinguishable, because no shared word is no shared
|
|
109
|
+
word either way.
|
|
110
|
+
|
|
111
|
+
Retrieval works regardless, because **identifiers and docstrings are already
|
|
112
|
+
natural language** — `retry_charge` contains *retry* and *charge*. But it
|
|
113
|
+
reaches only concepts somebody wrote down. Two things close the rest of the
|
|
114
|
+
gap, and neither is a model:
|
|
115
|
+
|
|
116
|
+
- **Your agent rewrites the query.** It has the conversation; turning
|
|
117
|
+
"重试扣款" into `retry charge payment gateway` costs it nothing.
|
|
118
|
+
- **Your agent writes the descriptions**, which puts the missing vocabulary
|
|
119
|
+
into the index once instead of into every query.
|
|
120
|
+
|
|
121
|
+
## Agent-authored descriptions
|
|
122
|
+
|
|
123
|
+
Every unit carries a description, and that description is indexed. By default
|
|
124
|
+
it is generated without a model: the identifier humanised, the parameters and
|
|
125
|
+
callees listed, the docstring appended. It introduces no vocabulary the source
|
|
126
|
+
did not already have — which is exactly why retrieval cannot reach a concept
|
|
127
|
+
nobody wrote down.
|
|
128
|
+
|
|
129
|
+
The agent already reading the index can supply those words:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
rag-your-code describe status # coverage, and what is pending
|
|
133
|
+
rag-your-code describe export --limit 20 # a batch, with source and a brief
|
|
134
|
+
rag-your-code describe import written.json # store what the agent wrote
|
|
135
|
+
rag-your-code index . # apply it
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
or, in the protocol, `describe_pending` and `describe_put` — which take effect
|
|
139
|
+
in the same session, with no refresh.
|
|
140
|
+
|
|
141
|
+
### Measured on this repository
|
|
142
|
+
|
|
143
|
+
This project describes its own implementation: all 120 units under `src/`
|
|
144
|
+
carry an agent-written bilingual description, committed to the repo. Ten
|
|
145
|
+
natural-language questions about the codebase, before and after:
|
|
146
|
+
|
|
147
|
+
| | before | after |
|
|
148
|
+
|---|---|---|
|
|
149
|
+
| top result in the expected file | 2 / 8 | **6 / 8** |
|
|
150
|
+
| Chinese-language queries | 0 / 4 | **3 / 4** |
|
|
151
|
+
| queries with **no shared word at all** (pure fallback) | 4 | **0** |
|
|
152
|
+
|
|
153
|
+
Two still miss, and both are worth stating. One query says `catastrophic
|
|
154
|
+
backtracking` where the description says `backtracks catastrophically`: there
|
|
155
|
+
is no stemming, so those share no word — exactly the limit documented above.
|
|
156
|
+
The other returned a unit that answers the question from a different file than
|
|
157
|
+
predicted, so the expectation was wrong rather than the retrieval.
|
|
158
|
+
|
|
159
|
+
**What this is:** it moves the semantic work from query time to index time.
|
|
160
|
+
Matching stays lexical. It is LLM-authored keyword expansion, and its reach is
|
|
161
|
+
bounded by how many ways of saying the thing the agent thought to write down.
|
|
162
|
+
|
|
163
|
+
Descriptions live in `rag-your-code.descriptions.json` at the repository root
|
|
164
|
+
and are meant to be committed, so one person's pass benefits everyone who
|
|
165
|
+
clones. Each is keyed by unit id **and a digest of the unit's source**: when
|
|
166
|
+
the code changes, the description is not applied, the unit returns to the
|
|
167
|
+
pending queue, and retrieval falls back to the generated sentence. A
|
|
168
|
+
description that outlived its code would be a confident wrong answer, which is
|
|
169
|
+
the one thing this index is built not to give. When code merely *moves* — an
|
|
170
|
+
import added above it — the description follows it by digest.
|
|
171
|
+
|
|
172
|
+
## Measured
|
|
173
|
+
|
|
174
|
+
**Parsing**, against source-controlled fixtures in `tests/fixtures/languages/`
|
|
175
|
+
(15 fixture files, 96 expected units, 237 negative cases, 89 constructs the
|
|
176
|
+
spec deliberately excludes):
|
|
177
|
+
|
|
178
|
+
| | |
|
|
179
|
+
|---|---|
|
|
180
|
+
| core declarations found | **91 / 91** |
|
|
181
|
+
| with the correct `start_line` | **91 / 91** |
|
|
182
|
+
| with a usable signature | **91 / 91** |
|
|
183
|
+
| units that do not exist | **0** |
|
|
184
|
+
|
|
185
|
+
A 441-byte JavaScript file that once took **12.6 s** to parse now takes
|
|
186
|
+
**0.36 ms**, and 10 KB takes 2.1 ms — growth is linear again.
|
|
187
|
+
|
|
188
|
+
**Scale**, on a synthetic 10,000-unit repository (500 files):
|
|
189
|
+
|
|
190
|
+
| | |
|
|
191
|
+
|---|---|
|
|
192
|
+
| full build | 1.84 s |
|
|
193
|
+
| incremental rebuild after one file changes | 0.207 s (**8.9x**) |
|
|
194
|
+
| compact storage vs readable JSON | 35.6% |
|
|
195
|
+
| index load, in a fresh process | 45.4 ms |
|
|
196
|
+
| inverted index build | 117.7 ms |
|
|
197
|
+
| resident memory | 58.7 MiB |
|
|
198
|
+
| query, mean of 200 warmed samples | 3.90 ms |
|
|
199
|
+
|
|
200
|
+
Directional local measurements, not service levels; the archived run is
|
|
201
|
+
[`large-benchmark-result.json`](large-benchmark-result.json).
|
|
202
|
+
|
|
203
|
+
**Suite:** Python 3.10 – 3.13 on Linux and Windows, plus a job that installs
|
|
204
|
+
the built wheel into a clean environment and runs every command the
|
|
205
|
+
documentation prescribes, and another that runs the skill's own install line
|
|
206
|
+
verbatim. 208 tests as of 0.4.2 — the count is version-stamped rather than
|
|
207
|
+
maintained, because a bare figure in a living document is a claim that rots;
|
|
208
|
+
per-release counts are in [CHANGELOG.md](CHANGELOG.md).
|
|
209
|
+
|
|
210
|
+
## Configuration
|
|
211
|
+
|
|
212
|
+
Twelve settings in `rag-your-code.toml` at the repository root:
|
|
213
|
+
|
|
214
|
+
```bash
|
|
215
|
+
rag-your-code config init # a commented file, all defaults
|
|
216
|
+
rag-your-code config list # effective values and their source
|
|
217
|
+
rag-your-code config set index.ignore '["vendor", "generated"]'
|
|
218
|
+
rag-your-code config set search.vector_weight 0.25
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
| section | settings |
|
|
222
|
+
|---|---|
|
|
223
|
+
| `[index]` | `ignore`, `suffixes`, `max_file_bytes` |
|
|
224
|
+
| `[embedding]` | `dimensions` |
|
|
225
|
+
| `[search]` | `vector_weight`, `limit`, `max_chars` |
|
|
226
|
+
| `[agent]` | `max_open_bytes`, `max_open_chars` |
|
|
227
|
+
| `[describe]` | `languages`, `batch`, `max_chars` |
|
|
228
|
+
|
|
229
|
+
Resolution is CLI flag > file > built-in default. There is no environment
|
|
230
|
+
layer: an index is an artifact of a repository, not of a shell.
|
|
231
|
+
|
|
232
|
+
An unknown key or an out-of-range value is an error, not a shrug — a setting
|
|
233
|
+
silently dropped is indistinguishable from one that had no effect.
|
|
234
|
+
`index.suffixes` may only name suffixes the parser has rules for, because a
|
|
235
|
+
suffix it cannot read is walked, parsed to nothing, and reported as a clean
|
|
236
|
+
index of zero units.
|
|
237
|
+
|
|
238
|
+
The four settings under `[index]` and `[embedding]` decide what an index
|
|
239
|
+
*contains*, so a digest of them is stored in the index and a change forces a
|
|
240
|
+
full rebuild. The rest take effect immediately and invalidate nothing.
|
|
241
|
+
|
|
242
|
+
## Agent protocol
|
|
243
|
+
|
|
244
|
+
`rag-your-code agent --root PATH` reads one JSON request per line and writes
|
|
245
|
+
one reply per line:
|
|
246
|
+
|
|
247
|
+
```json
|
|
248
|
+
{"action":"search","query":"database transaction rollback","limit":5}
|
|
249
|
+
{"action":"research","query":"trace payment retry behavior","max_steps":2}
|
|
250
|
+
{"action":"neighbors","id":"payments.py:4:retry_charge","hops":1}
|
|
251
|
+
{"action":"open","path":"payments.py","start_line":1,"end_line":80}
|
|
252
|
+
{"action":"describe_pending","limit":20}
|
|
253
|
+
{"action":"describe_put","descriptions":[{"id":"payments.py:4:retry_charge","text":"..."}]}
|
|
254
|
+
{"action":"refresh"}
|
|
255
|
+
{"action":"stats"}
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
**No single request can end the session.** Numeric fields saturate at their
|
|
259
|
+
bounds, `open` is bounded in both lines and bytes, and anything unanticipated
|
|
260
|
+
is reported in-band with its exception type. Streams are pinned to UTF-8
|
|
261
|
+
rather than following the console codepage.
|
|
262
|
+
|
|
263
|
+
`research` is a deliberately bounded two-step controller: retrieve, then at
|
|
264
|
+
most one graph expansion when confidence is low, reporting each step and why
|
|
265
|
+
it stopped.
|
|
266
|
+
|
|
267
|
+
## What lives where
|
|
268
|
+
|
|
269
|
+
| path | authored or generated | commit it? |
|
|
270
|
+
|---|---|---|
|
|
271
|
+
| `rag-your-code.toml` | authored | yes |
|
|
272
|
+
| `rag-your-code.descriptions.json` | authored by your agent | yes |
|
|
273
|
+
| `.rag-your-code/` (index, vectors, annotations) | generated | no |
|
|
274
|
+
|
|
275
|
+
Nothing authored lives under `.rag-your-code/` — that directory is what people
|
|
276
|
+
delete to clear the cache.
|
|
277
|
+
|
|
278
|
+
## Not here yet
|
|
279
|
+
|
|
280
|
+
Provider-backed embeddings, Tree-sitter parsing, and a SQLite/ANN storage layer
|
|
281
|
+
for repositories past the measured JSON envelope. Agent-authored descriptions
|
|
282
|
+
are deliberately the cheaper answer to the same problem provider embeddings
|
|
283
|
+
solve: they keep the zero-dependency, offline, reproducible-index properties,
|
|
284
|
+
and produce text a human can read and correct rather than opaque floats. See
|
|
285
|
+
[docs/ROADMAP.md](docs/ROADMAP.md).
|
|
286
|
+
|
|
287
|
+
## Development
|
|
288
|
+
|
|
289
|
+
```bash
|
|
290
|
+
python -m pip install -e ".[dev]"
|
|
291
|
+
pytest -q
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
No runtime dependencies; `pytest` and, below Python 3.11, `tomli` come from the
|
|
295
|
+
`dev` extra.
|
|
296
|
+
|
|
297
|
+
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — how each stage works and why
|
|
298
|
+
- [docs/TESTING.md](docs/TESTING.md) — what the suites are protecting
|
|
299
|
+
- [docs/ROADMAP.md](docs/ROADMAP.md) — what shipped, what is still open
|
|
300
|
+
- [CONTRIBUTING.md](CONTRIBUTING.md) — ground rules, and how to add a language
|
|
301
|
+
- [CHANGELOG.md](CHANGELOG.md) — every release, with its measurements
|
|
302
|
+
|
|
303
|
+
MIT licensed.
|