hitgate 0.1.0__tar.gz

hitgate-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Lucas Santana
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

hitgate-0.1.0/PKG-INFO

@@ -0,0 +1,256 @@
+Metadata-Version: 2.4
+Name: hitgate
+Version: 0.1.0
+Summary: A label-free retrieval-quality regression gate: measure any retriever's ranking (Hit@K/MRR) on your own corpus, no labels, gated in CI like a test.
+Author: Lucas Santana
+License: MIT License
+        
+        Copyright (c) 2026 Lucas Santana
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Homepage, https://github.com/LucasSantana-Dev/evidence-first-rag
+Project-URL: Repository, https://github.com/LucasSantana-Dev/evidence-first-rag
+Project-URL: Methodology, https://github.com/LucasSantana-Dev/evidence-first-rag/blob/main/docs/METHODOLOGY.md
+Keywords: rag,retrieval,evaluation,hit@k,mrr,regression-gate,ci,bm25,reranking,search
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: hybrid
+Requires-Dist: sentence-transformers>=2.2; extra == "hybrid"
+Requires-Dist: rank-bm25>=0.2; extra == "hybrid"
+Requires-Dist: numpy>=1.24; extra == "hybrid"
+Dynamic: license-file
+
+# hitgate
+
+[![eval-gate (advisory)](https://github.com/LucasSantana-Dev/hitgate/actions/workflows/eval.yml/badge.svg)](https://github.com/LucasSantana-Dev/hitgate/actions/workflows/eval.yml)
+
+**`pip install hitgate`** installs the harness (dependency-free; measures *any* retriever via `--retriever`).
+**`pip install "hitgate[hybrid]"`** adds the bundled hybrid retriever used in the demo below.
+
+> **A pytest-style regression gate for retrieval quality** — plus the small hybrid
+> retriever it was built to measure. Point it at *your* retriever and find out whether a
+> change helped or hurt, when you have **no labeled data and no users to A/B against**.
+
+**Status:** working · stable · single-author personal tooling, published for the
+*methodology*. The adoptable part is the **harness**: a label-free, regression-gated quality
+check for any retriever (`--retriever module:callable`). The bundled hybrid engine is just
+the thing it measures.
+
+---
+
+## Why this exists
+
+Building RAG is easy; *knowing whether a change made it better or worse* is the hard
+part. With a small corpus and a single user you have none of the production crutches —
+no click logs, no A/B traffic, no annotation budget. This repo is one answer: treat
+retrieval quality as a **measurable, regression-gated property**, like a test suite,
+and be ruthlessly honest about what the numbers do and don't prove.
+
+## Quickstart (reproducible in ~10 seconds)
+
+```bash
+pip install -e ".[hybrid]"   # harness core is dependency-free; [hybrid] adds the bundled retriever
+
+# Index this repo into a local ./.rag-index/ (the tool indexes itself)
+RAG_SOURCE_ROOTS="$PWD" python -m ragcore.build
+
+# Ask it something
+RAG_SOURCE_ROOTS="$PWD" python -m ragcore.query --scope code "how does the reranker fall back"
+
+# Run the eval gate (bundled retriever)
+RAG_RERANK_AUTO=off python -m hitgate.run --label demo
+
+# ...or point the SAME gate at YOUR retriever — any callable (query, top, scope) -> [{"path": ...}]
+python -m hitgate.run --retriever hitgate.example_external_retriever:retrieve --label mine
+```
+
+That eval indexes the repo's own source and scores 50 golden cases against it — so
+**you can reproduce the number below yourself**, no private data required.
+
+## Results
+
+### Self-indexed demo (reproducible)
+
+| Metric | Value |
+|---|---|
+| **Hit@5** (code scope, pure hybrid) — *the regression-gated headline* | **1.0** |
+| Hit@1 | 0.663 |
+| MRR | 0.800 |
+| Corpus | this repo, self-indexed · 101 cases |
+
+67 of 101 cases hit at rank 1; the misses are left in on purpose. Inflating a benchmark by
+quietly dropping the cases it fails is the first thing this project refuses to do — see
+[DECISIONS.md](./DECISIONS.md); measured before/after deltas are in
+[CHANGELOG.md](./CHANGELOG.md). An honest ablation — where **BM25-only wins Hit@1**
+(0.522) while **hybrid wins Hit@5** (1.0) — is walked through in
+[docs/METHODOLOGY.md](./docs/METHODOLOGY.md).
+
+Because the demo indexes **this repo itself**, the corpus grows as the repo does, so
+Hit@1 and MRR drift over time — adding a file can demote a borderline case. That's why
+**Hit@5 is the number under regression gate** (`hitgate/check.sh`, ±5pp). The drift is the
+honest behavior of a self-indexing benchmark, not noise swept under a frozen number.
+
+### External corpus benchmarks
+
+The same retriever — zero tuning, same `hitgate/run.py` pipeline — measured against 7 other
+codebases with no corpus-specific configuration:
+
+| Corpus | Language | n | Hit@5 | Hit@1 | MRR |
+|---|---|---|---|---|---|
+| FastAPI v0.115 | Python | 25 | **1.0** | 0.64 | 0.79 |
+| forge-space / mcp-gateway | TypeScript | 20 | **1.0** | 0.70 | 0.821 |
+| portfolio / src | React/TS | 15 | **1.0** | 0.60 | 0.778 |
+| ai-dev-toolkit / packages/core | Python + TS | 20 | **1.0** | 0.85 | 0.925 |
+| homelab / homelab\_manager | Python | 20 | 0.950 | 0.85 | 0.900 |
+| Lucky / packages/backend | TypeScript | 21 | 0.905 | 0.71 | 0.810 |
+| Criativaria / web-app | Next.js/TS | 27 | 0.741 | 0.59 | 0.660 |
+
+Hit@5=1.0 on four of seven corpora. The two lowest-performing corpora have structural
+causes: Lucky has one Category B drift miss (Prometheus registry vs middleware, identical
+vocabulary); Criativaria is a homogeneous Next.js component library where sibling components
+are lexically indistinguishable — a genuine retrieval ceiling, not a tuning problem.
+
+The finding that matters: **corpus module clarity predicts Hit@1 better than language or
+size.** Clean functional boundaries (homelab, ADT) → 0.85. Same-layer UI components
+(portfolio, Criativaria) → 0.59–0.60. Python vs TypeScript is not the variable.
+
+Full methodology, miss taxonomy, and reproduce commands: [docs/METHODOLOGY.md](./docs/METHODOLOGY.md).
+
+## How it works
+
+- **Hybrid retrieval** — dense embeddings (`intfloat/multilingual-e5-small`) + lexical
+  BM25, fused with Reciprocal Rank Fusion. A code-aware tokenizer splits identifiers
+  into `camelCase`/`snake_case` subtokens so "get user profile" matches `getUserProfile`.
+- **Selective reranking** (optional) — a cross-encoder reranker that, when enabled, is
+  scoped to code-scope queries only (it was measured to *help* code and *regress*
+  prose), with graceful fallback to the fused ranking if the model isn't present.
+- **Language-aware chunking** — Python by AST symbol, TS/JS/Shell by regex, with a
+  word-count fallback.
+- **Config by env var** — zero-setup defaults (`RAG_*`); see [`ragcore/config.py`](./ragcore/config.py).
+- **Eval (the point)** — `hitgate/run.py` reports Hit@K/MRR for *any* retriever via
+  `--retriever`; `hitgate/check.sh` gates a run against a frozen baseline (±5pp).
+- **Golden set generator** — `hitgate/generate.py` bootstraps candidate cases from your corpus
+  structure (docstrings, symbol names) with zero dependencies. LLM paraphrase generation is
+  opt-in via `--llm`. Output feeds directly into `hitgate/run.py --dataset`.
+
+## Use it on your own retriever
+
+The harness doesn't care whose retriever it's measuring. A retriever is any callable:
+
+```python
+retrieve(query: str, top: int, scope: str | None) -> Sequence[Mapping]
+# results ranked best-first; each a mapping with at least "path" (optionally "start_line")
+```
+
+Point the gate at yours with `--retriever module.path:callable`:
+
+```bash
+python -m hitgate.run --retriever mypkg.myretriever:retrieve --label mine
+```
+
+A runnable, dependency-free example — a deliberately dumb keyword matcher — is in
+[`hitgate/example_external_retriever.py`](./hitgate/example_external_retriever.py). Ecosystem
+wrappers (LangChain / LlamaIndex) live under [`adapters/`](./adapters/README.md). Bring your
+own retriever and corpus; keep the measurement discipline.
+
+### Bring your own corpus — 4-step quickstart
+
+**0. Bootstrap candidate cases from your corpus (optional):**
+```bash
+RAG_SOURCE_ROOTS="/path/to/your/corpus" python -m hitgate.generate \
+    --output hitgate/candidates.jsonl \
+    --min-confidence medium
+
+# LLM-enhanced (identifier + paraphrase per chunk, no extra package needed):
+OPENAI_API_KEY=sk-... RAG_SOURCE_ROOTS="/path/to/your/corpus" \
+    python -m hitgate.generate --llm --output hitgate/candidates.jsonl
+```
+Review and curate `hitgate/candidates.jsonl` — delete cases where the query is too vague
+or the expected file is wrong — then use it as your golden set below.
+
+**1. Write golden cases** — each is a JSON object with three fields:
+```jsonl
+{"query": "what handles pagination in the API", "expect_path_contains": "api/pagination.py", "expect_scope": "code"}
+{"query": "where are rate limits configured",    "expect_path_contains": "config/limits.yaml", "expect_scope": "code"}
+```
+`expect_path_contains` is a substring of the expected result's path (file name is usually enough).
+Aim for 20–50 cases across a mix of identifier lookups and paraphrase queries. Save as any `.jsonl`.
+
+**2. Run your retriever against the cases:**
+```bash
+python -m hitgate.run \
+    --retriever mypkg.myretriever:retrieve \
+    --dataset   my_golden.jsonl \
+    --label     baseline-v1
+# writes hitgate/baseline-v1.json with hit@1/hit@3/hit@5/mrr + per_case breakdown
+```
+
+**3. Freeze the baseline:**
+```bash
+cp hitgate/baseline-v1.json hitgate/baseline.my-project.json
+# edit _note to record conditions: corpus, model, date
+```
+
+**4. Gate future runs with check.sh:**
+```bash
+# hitgate/check.sh already reads BASELINE_FILE env var
+BASELINE_FILE=hitgate/baseline.my-project.json \
+RAG_SOURCE_ROOTS="/path/to/your/corpus" \
+python -m hitgate.run --retriever mypkg.myretriever:retrieve --dataset my_golden.jsonl --label ci
+bash hitgate/check.sh hitgate/ci.json hitgate/baseline.my-project.json
+# exits 1 if any metric regresses by more than 5pp
+```
+
+To diff two runs case-by-case: `python -m hitgate.diff hitgate/baseline-v1.json hitgate/ci.json`.
+
+## What to adopt (and what to skip)
+
+**Adopt the harness.** The reusable thing here is `hitgate/` — the label-free, regression-gated
+quality check and the `--retriever` interface. The bundled hybrid engine is a reference
+implementation, not the product. What this is **not**:
+
+- **Not a framework or a hosted service** — no plugin marketplace, no SaaS. Fork the harness;
+  the retriever is swappable by design.
+- **Not state-of-the-art retrieval research** — a pragmatic single-user system that
+  knows its own ceiling and stops there.
+- **Not a maintained project** — a solo operator's personal tool, shared for the
+  methodology. Issues and PRs are welcome but may not be triaged; expect best-effort,
+  no SLA. The eval workflow is an *advisory* gate (it proves the numbers reproduce), not
+  a support promise.
+
+Other conventional repo furniture — `CONTRIBUTING`, issue templates, a badge wall — is
+**deliberately** omitted, not unfinished. [DECISIONS.md](./DECISIONS.md) records what's
+left out on purpose and the trigger that would reopen each.
+
+## Extending
+
+The core indexes code + docs + commits and nothing else, on purpose. Tool-specific
+sources (assistant transcripts, code-graphs, other memory stores) plug in as opt-in
+adapters — see [`adapters/README.md`](./adapters/README.md).
+
+## Where this could go
+
+Candidate experiments — each gated on a measured win, none promised — are written up in
+[ROADMAP.md](./ROADMAP.md). They're directions, not commitments.
+
+## License
+
+MIT — see [LICENSE](./LICENSE). Use the methodology freely.

hitgate-0.1.0/README.md

@@ -0,0 +1,216 @@
+# hitgate
+
+[![eval-gate (advisory)](https://github.com/LucasSantana-Dev/hitgate/actions/workflows/eval.yml/badge.svg)](https://github.com/LucasSantana-Dev/hitgate/actions/workflows/eval.yml)
+
+**`pip install hitgate`** installs the harness (dependency-free; measures *any* retriever via `--retriever`).
+**`pip install "hitgate[hybrid]"`** adds the bundled hybrid retriever used in the demo below.
+
+> **A pytest-style regression gate for retrieval quality** — plus the small hybrid
+> retriever it was built to measure. Point it at *your* retriever and find out whether a
+> change helped or hurt, when you have **no labeled data and no users to A/B against**.
+
+**Status:** working · stable · single-author personal tooling, published for the
+*methodology*. The adoptable part is the **harness**: a label-free, regression-gated quality
+check for any retriever (`--retriever module:callable`). The bundled hybrid engine is just
+the thing it measures.
+
+---
+
+## Why this exists
+
+Building RAG is easy; *knowing whether a change made it better or worse* is the hard
+part. With a small corpus and a single user you have none of the production crutches —
+no click logs, no A/B traffic, no annotation budget. This repo is one answer: treat
+retrieval quality as a **measurable, regression-gated property**, like a test suite,
+and be ruthlessly honest about what the numbers do and don't prove.
+
+## Quickstart (reproducible in ~10 seconds)
+
+```bash
+pip install -e ".[hybrid]"   # harness core is dependency-free; [hybrid] adds the bundled retriever
+
+# Index this repo into a local ./.rag-index/ (the tool indexes itself)
+RAG_SOURCE_ROOTS="$PWD" python -m ragcore.build
+
+# Ask it something
+RAG_SOURCE_ROOTS="$PWD" python -m ragcore.query --scope code "how does the reranker fall back"
+
+# Run the eval gate (bundled retriever)
+RAG_RERANK_AUTO=off python -m hitgate.run --label demo
+
+# ...or point the SAME gate at YOUR retriever — any callable (query, top, scope) -> [{"path": ...}]
+python -m hitgate.run --retriever hitgate.example_external_retriever:retrieve --label mine
+```
+
+That eval indexes the repo's own source and scores 50 golden cases against it — so
+**you can reproduce the number below yourself**, no private data required.
+
+## Results
+
+### Self-indexed demo (reproducible)
+
+| Metric | Value |
+|---|---|
+| **Hit@5** (code scope, pure hybrid) — *the regression-gated headline* | **1.0** |
+| Hit@1 | 0.663 |
+| MRR | 0.800 |
+| Corpus | this repo, self-indexed · 101 cases |
+
+67 of 101 cases hit at rank 1; the misses are left in on purpose. Inflating a benchmark by
+quietly dropping the cases it fails is the first thing this project refuses to do — see
+[DECISIONS.md](./DECISIONS.md); measured before/after deltas are in
+[CHANGELOG.md](./CHANGELOG.md). An honest ablation — where **BM25-only wins Hit@1**
+(0.522) while **hybrid wins Hit@5** (1.0) — is walked through in
+[docs/METHODOLOGY.md](./docs/METHODOLOGY.md).
+
+Because the demo indexes **this repo itself**, the corpus grows as the repo does, so
+Hit@1 and MRR drift over time — adding a file can demote a borderline case. That's why
+**Hit@5 is the number under regression gate** (`hitgate/check.sh`, ±5pp). The drift is the
+honest behavior of a self-indexing benchmark, not noise swept under a frozen number.
+
+### External corpus benchmarks
+
+The same retriever — zero tuning, same `hitgate/run.py` pipeline — measured against 7 other
+codebases with no corpus-specific configuration:
+
+| Corpus | Language | n | Hit@5 | Hit@1 | MRR |
+|---|---|---|---|---|---|
+| FastAPI v0.115 | Python | 25 | **1.0** | 0.64 | 0.79 |
+| forge-space / mcp-gateway | TypeScript | 20 | **1.0** | 0.70 | 0.821 |
+| portfolio / src | React/TS | 15 | **1.0** | 0.60 | 0.778 |
+| ai-dev-toolkit / packages/core | Python + TS | 20 | **1.0** | 0.85 | 0.925 |
+| homelab / homelab\_manager | Python | 20 | 0.950 | 0.85 | 0.900 |
+| Lucky / packages/backend | TypeScript | 21 | 0.905 | 0.71 | 0.810 |
+| Criativaria / web-app | Next.js/TS | 27 | 0.741 | 0.59 | 0.660 |
+
+Hit@5=1.0 on four of seven corpora. The two lowest-performing corpora have structural
+causes: Lucky has one Category B drift miss (Prometheus registry vs middleware, identical
+vocabulary); Criativaria is a homogeneous Next.js component library where sibling components
+are lexically indistinguishable — a genuine retrieval ceiling, not a tuning problem.
+
+The finding that matters: **corpus module clarity predicts Hit@1 better than language or
+size.** Clean functional boundaries (homelab, ADT) → 0.85. Same-layer UI components
+(portfolio, Criativaria) → 0.59–0.60. Python vs TypeScript is not the variable.
+
+Full methodology, miss taxonomy, and reproduce commands: [docs/METHODOLOGY.md](./docs/METHODOLOGY.md).
+
+## How it works
+
+- **Hybrid retrieval** — dense embeddings (`intfloat/multilingual-e5-small`) + lexical
+  BM25, fused with Reciprocal Rank Fusion. A code-aware tokenizer splits identifiers
+  into `camelCase`/`snake_case` subtokens so "get user profile" matches `getUserProfile`.
+- **Selective reranking** (optional) — a cross-encoder reranker that, when enabled, is
+  scoped to code-scope queries only (it was measured to *help* code and *regress*
+  prose), with graceful fallback to the fused ranking if the model isn't present.
+- **Language-aware chunking** — Python by AST symbol, TS/JS/Shell by regex, with a
+  word-count fallback.
+- **Config by env var** — zero-setup defaults (`RAG_*`); see [`ragcore/config.py`](./ragcore/config.py).
+- **Eval (the point)** — `hitgate/run.py` reports Hit@K/MRR for *any* retriever via
+  `--retriever`; `hitgate/check.sh` gates a run against a frozen baseline (±5pp).
+- **Golden set generator** — `hitgate/generate.py` bootstraps candidate cases from your corpus
+  structure (docstrings, symbol names) with zero dependencies. LLM paraphrase generation is
+  opt-in via `--llm`. Output feeds directly into `hitgate/run.py --dataset`.
+
+## Use it on your own retriever
+
+The harness doesn't care whose retriever it's measuring. A retriever is any callable:
+
+```python
+retrieve(query: str, top: int, scope: str | None) -> Sequence[Mapping]
+# results ranked best-first; each a mapping with at least "path" (optionally "start_line")
+```
+
+Point the gate at yours with `--retriever module.path:callable`:
+
+```bash
+python -m hitgate.run --retriever mypkg.myretriever:retrieve --label mine
+```
+
+A runnable, dependency-free example — a deliberately dumb keyword matcher — is in
+[`hitgate/example_external_retriever.py`](./hitgate/example_external_retriever.py). Ecosystem
+wrappers (LangChain / LlamaIndex) live under [`adapters/`](./adapters/README.md). Bring your
+own retriever and corpus; keep the measurement discipline.
+
+### Bring your own corpus — 4-step quickstart
+
+**0. Bootstrap candidate cases from your corpus (optional):**
+```bash
+RAG_SOURCE_ROOTS="/path/to/your/corpus" python -m hitgate.generate \
+    --output hitgate/candidates.jsonl \
+    --min-confidence medium
+
+# LLM-enhanced (identifier + paraphrase per chunk, no extra package needed):
+OPENAI_API_KEY=sk-... RAG_SOURCE_ROOTS="/path/to/your/corpus" \
+    python -m hitgate.generate --llm --output hitgate/candidates.jsonl
+```
+Review and curate `hitgate/candidates.jsonl` — delete cases where the query is too vague
+or the expected file is wrong — then use it as your golden set below.
+
+**1. Write golden cases** — each is a JSON object with three fields:
+```jsonl
+{"query": "what handles pagination in the API", "expect_path_contains": "api/pagination.py", "expect_scope": "code"}
+{"query": "where are rate limits configured",    "expect_path_contains": "config/limits.yaml", "expect_scope": "code"}
+```
+`expect_path_contains` is a substring of the expected result's path (file name is usually enough).
+Aim for 20–50 cases across a mix of identifier lookups and paraphrase queries. Save as any `.jsonl`.
+
+**2. Run your retriever against the cases:**
+```bash
+python -m hitgate.run \
+    --retriever mypkg.myretriever:retrieve \
+    --dataset   my_golden.jsonl \
+    --label     baseline-v1
+# writes hitgate/baseline-v1.json with hit@1/hit@3/hit@5/mrr + per_case breakdown
+```
+
+**3. Freeze the baseline:**
+```bash
+cp hitgate/baseline-v1.json hitgate/baseline.my-project.json
+# edit _note to record conditions: corpus, model, date
+```
+
+**4. Gate future runs with check.sh:**
+```bash
+# hitgate/check.sh already reads BASELINE_FILE env var
+BASELINE_FILE=hitgate/baseline.my-project.json \
+RAG_SOURCE_ROOTS="/path/to/your/corpus" \
+python -m hitgate.run --retriever mypkg.myretriever:retrieve --dataset my_golden.jsonl --label ci
+bash hitgate/check.sh hitgate/ci.json hitgate/baseline.my-project.json
+# exits 1 if any metric regresses by more than 5pp
+```
+
+To diff two runs case-by-case: `python -m hitgate.diff hitgate/baseline-v1.json hitgate/ci.json`.
+
+## What to adopt (and what to skip)
+
+**Adopt the harness.** The reusable thing here is `hitgate/` — the label-free, regression-gated
+quality check and the `--retriever` interface. The bundled hybrid engine is a reference
+implementation, not the product. What this is **not**:
+
+- **Not a framework or a hosted service** — no plugin marketplace, no SaaS. Fork the harness;
+  the retriever is swappable by design.
+- **Not state-of-the-art retrieval research** — a pragmatic single-user system that
+  knows its own ceiling and stops there.
+- **Not a maintained project** — a solo operator's personal tool, shared for the
+  methodology. Issues and PRs are welcome but may not be triaged; expect best-effort,
+  no SLA. The eval workflow is an *advisory* gate (it proves the numbers reproduce), not
+  a support promise.
+
+Other conventional repo furniture — `CONTRIBUTING`, issue templates, a badge wall — is
+**deliberately** omitted, not unfinished. [DECISIONS.md](./DECISIONS.md) records what's
+left out on purpose and the trigger that would reopen each.
+
+## Extending
+
+The core indexes code + docs + commits and nothing else, on purpose. Tool-specific
+sources (assistant transcripts, code-graphs, other memory stores) plug in as opt-in
+adapters — see [`adapters/README.md`](./adapters/README.md).
+
+## Where this could go
+
+Candidate experiments — each gated on a measured win, none promised — are written up in
+[ROADMAP.md](./ROADMAP.md). They're directions, not commitments.
+
+## License
+
+MIT — see [LICENSE](./LICENSE). Use the methodology freely.

hitgate-0.1.0/hitgate/__init__.py

@@ -0,0 +1,5 @@
+"""hitgate — a label-free, regression-gated retrieval-evaluation harness.
+
+The harness core is dependency-free; the bundled hybrid retriever it can measure
+lives in `ragcore` and installs with the optional `[hybrid]` extra.
+"""

hitgate-0.1.0/hitgate/audit_contamination.py

@@ -0,0 +1,146 @@
+#!/usr/bin/env python3
+"""hitgate/audit_contamination.py — find un-winnable cases in an eval set.
+
+The most insidious way a retrieval benchmark lies is *contamination*: a "golden"
+case whose expected answer isn't in the indexed corpus at all. Such a case can
+only ever miss, so it caps the score with a constant penalty that looks like a
+quality floor — and every decision made on that number inherits the lie. (This is
+the audit that moved this project's own baseline ~8pp; see DECISIONS.md.)
+
+This script makes that audit reusable. Point it at any eval set (same schema as
+hitgate/golden.demo.jsonl) and an index, and it classifies every case:
+
+  ok             — the expected path is indexed within the case's declared scope
+  scope-mismatch — the path is indexed, but only OUTSIDE the declared scope
+  CONTAMINATED   — the expected path is not in the corpus at all → un-winnable
+
+Exit 0 if no contamination, 1 if any case is un-winnable (so it can gate a build).
+scope-mismatch is reported as a warning, not a failure.
+
+Usage:
+  RAG_SOURCE_ROOTS="$PWD" python -m ragcore.build          # build the index first
+  RAG_SOURCE_ROOTS="$PWD" python -m hitgate.audit_contamination
+  python -m hitgate.audit_contamination --dataset path/to/your.jsonl
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sqlite3
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+from ragcore.config import DB  # honors RAG_INDEX_DIR
+
+DEFAULT_DATASET = ROOT / "hitgate" / "golden.demo.jsonl"
+
+
+def _expected_substrings(case: dict) -> list[str]:
+    """Non-empty expected path substrings for a case (empties dropped — '' matches every path)."""
+    raw = case["expect_path_contains"]
+    raw = raw if isinstance(raw, list) else [raw]
+    return [e for e in raw if isinstance(e, str) and e.strip()]
+
+
+def load_cases(path: Path) -> list[dict]:
+    if not path.exists():
+        sys.exit(f"dataset not found: {path}")
+    cases = []
+    for i, line in enumerate(path.read_text().splitlines(), 1):
+        if not line.strip():
+            continue
+        try:
+            case = json.loads(line)
+        except json.JSONDecodeError as e:
+            sys.exit(f"{path}:{i}: invalid JSON — {e}")
+        if "expect_path_contains" not in case:  # tolerate other schemas by skipping
+            continue
+        if not _expected_substrings(case):
+            sys.exit(
+                f"{path}:{i}: case {case.get('query', '?')!r} has empty expect_path_contains "
+                f"— a malformed eval case (an empty substring matches every path). Fix the eval set."
+            )
+        cases.append(case)
+    return cases
+
+
+def load_corpus(db: Path) -> list[tuple[str, str]]:
+    """(source_type, path) for every indexed chunk."""
+    if not db.exists():
+        sys.exit(f"no index at {db} — run ragcore/build.py first")
+    conn = sqlite3.connect(db)
+    try:
+        return conn.execute("SELECT source_type, path FROM chunks").fetchall()
+    finally:
+        conn.close()
+
+
+def classify(case: dict, corpus: list[tuple[str, str]]) -> str:
+    expected = _expected_substrings(case)  # already validated non-empty in load_cases
+    scope = case.get("expect_scope")
+    scopes = scope if isinstance(scope, list) else ([scope] if scope else [])
+
+    def path_matches(p: str) -> bool:
+        return any(e in p for e in expected)
+
+    anywhere = [(st, p) for st, p in corpus if path_matches(p)]
+    if not anywhere:
+        return "CONTAMINATED"
+    if scopes and not any(st in scopes for st, _ in anywhere):
+        return "scope-mismatch"
+    return "ok"
+
+
+def resolve_dataset(arg: str) -> Path:
+    """Resolve --dataset robustly: absolute as-is; otherwise try cwd-relative (standard
+    CLI behavior) and then repo-root-relative, so the tool works whether you run it from
+    inside the repo or from elsewhere. Errors clearly, naming both paths tried."""
+    p = Path(arg)
+    if p.is_absolute():
+        return p
+    candidates = [Path.cwd() / arg, ROOT / arg]
+    for candidate in candidates:
+        if candidate.exists():
+            return candidate
+    sys.exit("dataset not found — tried " + " and ".join(str(c) for c in candidates))
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--dataset", default=str(DEFAULT_DATASET),
+                    help="eval jsonl to audit (absolute, or relative to cwd or repo root)")
+    args = ap.parse_args()
+
+    dataset = resolve_dataset(args.dataset)
+    cases = load_cases(dataset)
+    if not cases:
+        sys.exit(f"no usable cases in {dataset}")
+    corpus = load_corpus(DB)
+
+    verdicts = {"ok": [], "scope-mismatch": [], "CONTAMINATED": []}
+    for case in cases:
+        verdicts[classify(case, corpus)].append(case)
+
+    n = len(cases)
+    print(f"contamination audit: {n} cases vs {len(corpus)} indexed chunks ({DB})")
+    print(f"  ok:             {len(verdicts['ok'])}")
+    print(f"  scope-mismatch: {len(verdicts['scope-mismatch'])}")
+    print(f"  CONTAMINATED:   {len(verdicts['CONTAMINATED'])}")
+
+    for case in verdicts["scope-mismatch"]:
+        print(f"\n  ⚠ scope-mismatch: {case['query'][:70]!r}")
+        print(f"      expects {case['expect_path_contains']} in scope={case.get('expect_scope')}, found only out of scope")
+    for case in verdicts["CONTAMINATED"]:
+        print(f"\n  ✗ CONTAMINATED: {case['query'][:70]!r}")
+        print(f"      expects {case['expect_path_contains']} — not in the corpus; this case is un-winnable")
+
+    if verdicts["CONTAMINATED"]:
+        print(f"\n{len(verdicts['CONTAMINATED'])} un-winnable case(s) — remove them or fix the corpus before trusting the score.")
+        return 1
+    print("\n✓ no contamination — every case's answer is in the corpus.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())