reponeedle 1.1.0__tar.gz

reponeedle-1.1.0/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2026 RepoNeedle Authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

reponeedle-1.1.0/PKG-INFO

@@ -0,0 +1,406 @@
+Metadata-Version: 2.4
+Name: reponeedle
+Version: 1.1.0
+Summary: Token-Budget-Aware, Workflow-Adaptive Repository Context Retrieval for AI Coding Agents
+Author: RepoNeedle Contributors
+License-Expression: MIT
+Keywords: code-retrieval,ai-agents,context-selection,token-budget,repository-retrieval,llm
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: scipy>=1.10.0
+Requires-Dist: networkx>=3.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: pathspec>=0.11.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: tomli>=2.0.1; python_version < "3.11"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Provides-Extra: api
+Requires-Dist: fastapi<1.0,>=0.115; extra == "api"
+Requires-Dist: httpx>=0.24; extra == "api"
+Requires-Dist: uvicorn[standard]<1.0,>=0.30; extra == "api"
+Provides-Extra: hf
+Requires-Dist: huggingface_hub>=0.20.0; extra == "hf"
+Requires-Dist: gradio>=4.0.0; extra == "hf"
+Dynamic: license-file
+
+# 🪡 RepoNeedle
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+[![CI](https://github.com/krishika08/RepoNeedle/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/krishika08/RepoNeedle/actions/workflows/ci.yml)
+[![Hugging Face Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-RepoNeedle--Bench-blue)](https://huggingface.co/datasets/RepoNeedle/RepoNeedle-Bench)
+[![Hugging Face Space](https://img.shields.io/badge/%F0%9F%A4%97%20Space-RepoNeedle--Playground-indigo)](https://huggingface.co/spaces/RepoNeedle/RepoNeedle-Playground)
+[![Hugging Face Space](https://img.shields.io/badge/%F0%9F%A4%97%20Space-RepoNeedle--Playground-indigo)](https://huggingface.co/spaces/Krishikax/RepoNeedle-Playground)
+
+[![Buy Me a Coffee](https://img.shields.io/badge/Buy%20Me%20a%20Coffee-Support%20RepoNeedle-FFDD00?logo=buymeacoffee&logoColor=000000)](https://buymeacoffee.com/krishika.x)
+
+**RepoNeedle** is an open-source, token-budget-aware repository context retrieval system designed to help AI coding agents discover the most critical files for a coding task under strict context-token constraints.
+
+---
+
+## ⚡ The Upstream Context Problem in AI Coding
+
+Autonomous coding agents (e.g., Claude Code, Aider, OpenHands, Cline) face a severe upstream bottleneck: **Context Acquisition**.
+
+When an agent tackles an issue, bug report, or test failure, passing the entire repository into the prompt is cost-prohibitive or exceeds context windows. Existing approaches face persistent failure modes:
+- **Dense Embedding Models**: Rank files without considering file size (tokens), repository topology (call/import graphs), or task intent. They frequently retrieve large, irrelevant files that exhaust the context window.
+- **RepoMap / PageRank Centrality**: Heavily prioritizes architectural hubs, but is completely blind to test files (which are leaf nodes with zero in-degree in import graphs) and cannot route specific error frames.
+- **Context Bloat & Token Waste**: Studies like ContextBench demonstrate that coding agents pull massive amounts of irrelevant noise into context, degrading reasoning quality via "lost-in-the-middle" effects.
+- **Selective Hallucination**: When a query cannot be answered within the repository (e.g. out-of-scope or counterfactual tasks), standard retrievers still force top-$k$ hallucinated files into context instead of abstaining.
+
+RepoNeedle solves this upstream bottleneck by combining:
+1. **Multi-Signal Hybrid Scoring**: Field-weighted BM25 + Subword Semantic Vectors + AST Repository Graph Centrality.
+2. **Workflow-Adaptive Routing**: Automatic classification of query intent (`trace2code`, `code2test`, `edit2ripple`, `comment2context`, `selective`) with dynamic weight tuning.
+3. **Inverted Test-to-Target Mapping**: Dependency graph inversion and naming convention heuristics to locate regression test suites.
+4. **Calibrated Selective Abstention**: Peak-to-noise ($Z$-score) and repository symbol overlap gating to abstain from returning noise on counterfactual tasks.
+5. **CUD-Pack (Capacity-Sensitive Utility-Density Packing)**: Adaptive knapsack selection engine providing strict zero downstream overflows ($C_{\text{actual}} \le B_{\text{configured}}$) according to RepoNeedle's regex code token model across dual modes: **Compact Mode** ($\le 4\text{k}$ tokens, extracting high-density implementation spans and structural skeletons) and **Complete Context Mode** ($\ge 8\text{k}$ tokens, preserving whole source files).
+6. **Change-Impact Ranking Engine (V1.1 Opt-in)**: Seed-relative relationship scoring (evaluating static imports, symbol definitions, call sites, build manifests (`setup.py`, `setup.cfg`, `pyproject.toml`, `cargo.toml`, `package.json`, `go.mod`, `pom.xml`), test-stems, and directory proximity) designed for bug fixing and refactoring tasks.
+
+---
+
+## 📊 Benchmark Validation
+
+To maintain strict scientific integrity, **Official External Benchmark Results** are reported separately from **Independent Research Evaluations**. Dataset sizes, query counts ($N$), candidate pool properties, and evaluation methodologies are explicitly documented for every measurement.
+
+---
+
+### Part A: Full Official Agent Retrieval Bench (ARB v2) Evaluation ($N=345$ Positive Samples)
+
+Evaluated directly using the official `agent_retrieval_bench` evaluation package (`agent_retrieval_bench.baseline.sample_metrics`, v0.2.1, commit `9e4d68937f29`) on the complete **Agent Retrieval Bench v2** release ($N=345$ positive retrieval samples across 25 open-source repositories and 218 frozen repository snapshots):
+
+| System / Ranker | Overall MRR | Recall@5 | Recall@10 | Recall@20 | BCY@4k | BCY@8k | BCY@16k | BCY@32k |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **RepoNeedle (Full)** | **0.2287** | **0.3011** | **0.4110** | **0.4279** | **0.2436** | **0.3450** | **0.4192** | **0.4279** |
+| **Official RepoMap** | 0.2140 | 0.3185 | 0.4786 | **0.6306** | 0.2024 | **0.3788** | **0.5235** | **0.6166** |
+| **Official Lexical** | 0.1574 | 0.2292 | 0.3548 | 0.4940 | 0.1488 | 0.2650 | 0.3886 | 0.4882 |
+| **Official BM25** | 0.1520 | 0.1899 | 0.3034 | 0.4452 | 0.1220 | 0.2051 | 0.3184 | 0.4365 |
+| *Qwen3-Embedding-4B (Ext Ref)* | *0.2379* | *—* | *—* | *0.6306* | *—* | *0.3409* | *—* | *—* |
+| *Qwen3-Embedding-8B (Ext Ref)* | *0.2336* | *—* | *—* | *0.7029* | *—* | *0.3732* | *—* | *—* |
+| *pplx-embed-v1-4b (Ext Ref)* | *0.2267* | *—* | *—* | *0.6072* | *—* | *0.3549* | *—* | *—* |
+| *jina-code-0.5b (Ext Ref)* | *0.1914* | *—* | *—* | *0.4823* | *—* | *0.2783* | *—* | *—* |
+
+#### Matched Paired Statistical Tests against Baselines ($N=345$ Identical Queries)
+
+Evaluated with **10,000 bootstrap resamples** (repository-clustered bootstrap across 25 repositories):
+
+| Paired Comparison | Mean Diff (Δ) | Std Error | Sample 95% CI | Repo-Clustered 95% CI | Paired $t$-test | Wilcoxon $W$ ($p$-value) | Wins / Ties / Losses | Statistically Significant |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **RepoNeedle vs BM25** | **+0.0767** (+50.5%) | 0.0161 | [+0.0458, +0.1085] | **[+0.0202, +0.1129]** | $t = 4.776$ ($p = 2.65\times 10^{-6}$) | $W = 22384.0$ ($p = 0.0096$) | 148 / 18 / 179 | **YES ($p < 0.01$)** |
+| **RepoNeedle vs RepoMap** | **+0.0148** (+6.9%) | 0.0179 | [-0.0198, +0.0497] | **[-0.0466, +0.0619]** | $t = 0.827$ ($p = 0.4089$) | $W = 23821.5$ ($p = 0.3032$) | 122 / 26 / 197 | **NO (Parity in MRR)** |
+| **RepoNeedle vs Lexical** | **+0.0713** (+45.3%) | 0.0157 | [+0.0409, +0.1031] | **[+0.0307, +0.1029]** | $t = 4.529$ ($p = 8.17\times 10^{-6}$) | $W = 22521.5$ ($p = 0.1215$) | 133 / 29 / 183 | **YES ($p < 0.001$)** |
+
+> [!IMPORTANT]
+> **Scientific Disclosure on RepoMap Comparison**: RepoNeedle does **NOT** achieve statistically significant overall superiority over RepoMap on MRR ($p = 0.4089$, repository-clustered 95% CI crosses zero: `[-0.0466, +0.0619]`). Furthermore, **RepoMap achieves substantially higher deep recall** (Recall@20: `0.6306` vs `0.4279`, a +47.4% relative advantage for RepoMap) and higher BCY@8k (`0.3788` vs `0.3450`, +9.8% relative advantage). RepoNeedle demonstrates a focused advantage under strict low-token budgets (BCY@4k: `0.2436` vs `0.2024`, +20.4% relative advantage) and specialized regression test discovery (`code2test`).
+
+#### Per-Task Track Breakdown ($N=345$ Positive Samples)
+
+| Task Track | Sample Count ($N$) | Metric | Official BM25 | Official Lexical | Official RepoMap | RepoNeedle | Relative Gain vs BM25 | Relative Gain vs RepoMap |
+| :--- | :---: | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
+| **`code2test`** | $N=106$ | **MRR** | 0.1057 | 0.0663 | 0.1942 | **0.2660** | **+151.7%** | **+37.0%** |
+| | | Recall@5 | 0.1305 | 0.0676 | 0.2645 | **0.3918** | +200.2% | +48.1% |
+| | | Recall@20 | 0.3066 | 0.2469 | 0.5572 | **0.5931** | +93.4% | +6.4% |
+| | | **BCY@8k** | 0.1541 | 0.0928 | 0.3462 | **0.4814** | **+212.4%** | **+39.1%** |
+| **`comment2context`** | $N=80$ | **MRR** | **0.1974** | 0.1530 | 0.1532 | 0.1708 | **-13.5%** | +11.5% |
+| | | Recall@5 | **0.2146** | 0.1667 | 0.1750 | 0.1646 | -23.3% | -5.9% |
+| | | Recall@20 | **0.5292** | 0.4979 | 0.4688 | 0.2083 | -60.6% | -55.6% |
+| | | **BCY@8k** | 0.2604 | 0.2167 | **0.2792** | 0.1896 | -27.2% | -32.1% |
+| **`trace2code`** | $N=101$ | **MRR** | 0.1638 | 0.2075 | **0.2723** | 0.2337 | +42.7% | **-14.2%** |
+| | | Recall@5 | 0.2228 | 0.3432 | **0.4604** | 0.2855 | +28.1% | -38.0% |
+| | | Recall@20 | 0.4934 | 0.6964 | **0.8465** | 0.4092 | -17.1% | -51.7% |
+| | | **BCY@8k** | 0.1931 | 0.3795 | **0.5066** | 0.2954 | +53.0% | -41.7% |
+| **`edit2ripple`** | $N=58$ | **MRR** | 0.1536 | **0.2428** | 0.2323 | 0.2320 | +51.0% | -0.1% |
+| | | Recall@5 | 0.2069 | **0.4124** | 0.3678 | 0.3506 | +69.5% | -4.7% |
+| | | Recall@20 | 0.4986 | 0.5876 | **0.6121** | 0.4612 | -7.5% | -24.7% |
+| | | **BCY@8k** | 0.2428 | **0.4468** | 0.3534 | 0.3966 | +63.3% | +12.2% |
+
+> [!IMPORTANT]
+> **Methodological Disclosure on `code2test` (+151.7%) and Workflow Routing**:
+> - **Task-Conditioned Evaluation**: In the official ARB evaluation above, benchmark task labels (`code2test`, `trace2code`, etc.) were supplied directly to the `WorkflowRouter` (`task_type = sample.get("task_type")`), dynamically activating task-specific fusion weights and test-graph linkages.
+> - **Not Default AUTO-Mode Performance**: This measures **task-conditioned retrieval capacity**, not unassisted automatic classification. In unassisted production deployment (`TaskType.AUTO`), the engine relies on query text regex heuristics. If a natural-language query lacks explicit test keywords, automatic intent classification may default to `comment2context`, reducing the observed gain toward baseline BM25 performance (`0.1057`).
+> - **Candidate Set Integrity**: The task conditioning dynamically modulates ranking weights and structural graph boosts; candidate file pools themselves are never filtered by oracle labels.
+
+---
+
+### Part B: Official ARB Selective Retrieval Evaluation ($N=82$ Abstention Samples)
+
+Evaluated on the complete official Agent Retrieval Bench Selective Retrieval split (`v2_selective_mixed`, $N=82$ total queries across 71 repository snapshots):
+
+| Evaluation Track | Sample Count ($N$) | Correctly Abstained | False Positives (Returned Noise) | Abstention Accuracy |
+| :--- | :---: | :---: | :---: | :---: |
+| **Natural No-Gold Issues** | 50 | 0 | 50 (100.0%) | **0.0%** |
+| **Counterfactual Wrong-Repo** | 32 | 0 | 32 (100.0%) | **0.0%** |
+| **Combined Selective Benchmark** | **82** | **0** | **82 (100.0%)** | **0.0%** |
+
+> [!CAUTION]
+> **Heuristic Abstention Failure Analysis**: In production benchmark conditions, RepoNeedle's heuristic confidence gating ($Z$-score + repository symbol overlap) achieved **0% abstention accuracy (0/82)**. 
+> - *Root Cause*: Heuristic confidence scores ranged between `0.64` and `0.86`, far exceeding the default threshold (`0.12` / `0.38`). Natural GitHub issue descriptions filed against a repository inherently contain valid repository vocabulary. Counterfactual queries similarly contain ubiquitous software engineering terminology (`error`, `parse`, `client`, `handler`).
+> - *Recommendation*: Heuristic score thresholds are insufficient for reliable abstention in large codebases. Selective retrieval requires a dedicated, trained neural discriminator rather than surface lexical statistics.
+
+---
+
+### Part C: Component Ablations on Full ARB v2 ($N=345$)
+
+Ablation evaluation conducted on the complete $N=345$ positive ARB v2 benchmark:
+
+| Configuration | Overall MRR | Drop vs Full | Recall@5 | Recall@20 | Description |
+| :--- | :---: | :---: | :---: | :---: | :--- |
+| **Full RepoNeedle** | **0.2287** | **0.00%** | **0.3011** | 0.4279 | All components active (Router + Graph + BM25 + Skeletons) |
+| *w/o WorkflowRouter* | 0.1642 | **-28.20%** | — | — | Static uniform weights across all task categories |
+| *w/o RepoGraph* | 0.2287 | 0.00% | — | — | Graph centrality and test-to-target links zeroed |
+| **Official RepoMap Baseline** | 0.2140 | -6.45% | 0.3185 | **0.6306** | Standalone PageRank graph centrality ranker |
+| **Official BM25 Baseline** | 0.1520 | -33.53% | 0.1899 | 0.4452 | Standalone field BM25 ranker |
+
+---
+
+### Part D: Multi-Scale Token Payload Reduction & Measured Runtime
+
+#### 1. Actual Content Payload Reduction Across Multiple Languages & File Sizes
+
+Measured using exact BPE code subword regex (`re.compile(r"[A-Za-z0-9_]+|[^\s\w]|\s+")`, calibrated to OpenAI cl100k / LLaMA / Qwen tokenizers):
+
+| Corpus File & Language | Lines | Full BPE Tokens | Skeleton BPE Tokens | Actual BPE Token Savings | Character Savings | Architectural Symbols Preserved |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
+| **Python (Short)** | 50L | 280 | 280 | 0.0% *(fits budget)* | 0.0% | **YES** (Complete) |
+| **Python (Medium Adapter)** | 250L | 2,060 | 305 | **85.2% reduction** | 85.8% reduction | **YES** (Class, methods, docstrings) |
+| **Python (Long Session Mgr)** | 700L | 4,061 | 404 | **90.1% reduction** | 90.0% reduction | **YES** (Class, methods, TTL logic) |
+| **Rust (Async RingBuffer)** | 200L | 2,312 | 222 | **90.4% reduction** | 91.1% reduction | **YES** (Struct, impl, fn signatures) |
+| **TypeScript (EventDispatcher)** | 200L | 1,920 | 455 | **76.3% reduction** | 74.6% reduction | **YES** (Interface, class, methods) |
+| **Go (HTTP Trie Router)** | 200L | 1,225 | 578 | **52.8% reduction** | 44.2% reduction | **YES** (Types, struct, methods) |
+
+#### 2. Measured Benchmark Latency Distribution (Commodity CPU, Windows 11 AMD64)
+
+Empirically measured across the full Agent Retrieval Bench evaluation suite:
+
+| Component / Ranker | Mean Latency | Median ($p_{50}$) | 95th Percentile ($p_{95}$) | 99th Percentile ($p_{99}$) | Method Notes |
+| :--- | :---: | :---: | :---: | :---: | :--- |
+| **RepoNeedle Query Latency** | `317.7 ms` | `142.4 ms` | `827.2 ms` | `997.6 ms` | Multi-signal scoring + knapsack packing |
+| **Official BM25 Query Latency** | `994.4 ms` | `1,158.3 ms` | `1,246.0 ms` | `1,305.7 ms` | Full inverted index posting scan |
+| **Official RepoMap Query Latency**| `157.5 ms` | `192.1 ms` | `239.8 ms` | `250.1 ms` | Graph PageRank vector lookup |
+| **Official Lexical Query Latency**| `884.1 ms` | `1,005.2 ms` | `1,150.3 ms` | `1,156.6 ms` | BM25 + subword character bigram matching |
+| **RepoNeedle Index Build** | `8,622.9 ms` | `10,471.2 ms` | `11,484.8 ms` | `11,551.5 ms` | One-time AST + dependency graph per repo |
+
+---
+
+### Part E: System Architecture & Deployment Status
+
+- **CPU Native Architecture**: RepoNeedle runs entirely in Python/NumPy/SciPy/NetworkX without requiring CUDA GPUs or heavy deep learning runtime dependencies.
+- **Release Status**: **V1.1.0 source integrated on main; formal release publication pending.** Features default `legacy` ranker with backwards-compatible contract preservation, and opt-in `change_impact` ranking engine for seed-relative relationship scoring.
+
+---
+
+### Part F: Independent R6 Change-Impact Benchmark ($N=180$ Queries across 18 Repositories)
+
+To evaluate retrieval generalizability beyond the original 25 ARB repositories, an independent benchmark cohort was constructed across **18 unseen open-source repositories** comprising 180 retrieval queries.
+
+Statistical inference was conducted using **repository-clustered statistical models** across independent repository clusters:
+
+| Partition | Queries ($N$) | Repositories | Baseline MRR | Champion MRR | Delta (Δ) | Repository-Clustered Inference (95% CI) | Statistical Significance |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **Training** | $N=90$ | 9 repos | 0.6213 | 0.6546 | +0.0333 | — | Training fit |
+| **Validation** | $N=40$ | 4 repos | 0.6472 | 0.6708 | +0.0236 | $p = 0.5073$, 95% CI [0.0000, +0.0472] | Not Statistically Significant |
+| **Held-Out** | $N=50$ | 5 repos | 0.5215 | 0.5467 | +0.0252 | $p = 0.5025$, 95% CI [0.0000, +0.0629] | Not Statistically Significant |
+| **Pooled Cohort** | $N=180$ | 18 repos | 0.5994 | **0.6282** | **+0.0288** | $p = 0.0078$, 95% CI [+0.0082, +0.0524] | **Statistically Significant ($p < 0.01$)** |
+
+> [!NOTE]
+> **Methodological Context & Scope of R6 Results**:
+> - **Pre-Filtered Candidate Pools**: Queries were evaluated against pre-filtered candidate pools averaging approximately **53.79 files per query**. Within these candidate pools, Candidate-Pool Recall@20 is 100%.
+> - **Whole-Repository Generalization**: Whole-repository Recall@20 of 60–75% represents a projected estimate under full repository indexing, not an empirically measured whole-codebase scan.
+> - **Statistical Power & Default Ranker**: While the pooled 18-repository cohort achieves statistically significant improvement under repository-clustered inference ($p = 0.0078$), the smaller isolated validation ($N=40$, $p = 0.5073$) and held-out ($N=50$, $p = 0.5025$) subsets are statistically underpowered. For this reason, the Change-Impact engine is provided as an **opt-in capability** rather than overriding the default `legacy` ranker.
+> - **Aspirational Research Target**: The long-term research objective of $\text{MRR} \ge 0.80$ is an aspirational research milestone, not an achieved production metric.
+
+---
+
+## 🚀 Quickstart & Installation
+
+RepoNeedle is self-contained and requires standard Python dependencies (`numpy`, `scipy`, `networkx`, `pydantic`, `pathspec`, `rich`, and `tomli` on Python < 3.11).
+
+### 1. Installation
+
+```bash
+# Clone the verified repository
+git clone https://github.com/krishika08/RepoNeedle.git
+cd RepoNeedle
+
+# Install package in editable mode
+pip install -e .
+
+# Or install with development and test dependencies
+pip install -e ".[dev]"
+```
+
+> [!TIP]
+> On Python 3.10, `tomli>=2.0.1` is automatically installed via conditional dependency. On Python 3.11 and 3.12, native standard library `tomllib` is used with zero extra dependencies.
+
+---
+
+### 2. Python API
+
+#### Standard Retrieval (Default Legacy Ranker):
+```python
+from reponeedle import RepoNeedleEngine, BudgetConfig
+
+# 1. Initialize engine on a local codebase (uses default 'legacy' ranker)
+engine = RepoNeedleEngine.from_directory(
+    repo_dir="./my-project",
+    budget_config=BudgetConfig(max_tokens=4000, allow_skeletons=True, mode="auto"),
+)
+
+# 2. Retrieve context for a task (auto-resolves to compact mode for <= 4k)
+result = engine.retrieve(
+    query="Add support for blueprint url_prefix inheritance in route registration",
+    budget_tokens=4000,
+)
+
+if result.abstained:
+    print(f"Abstained: {result.explanation}")
+else:
+    print(f"Retrieved {len(result.selected_files)} files ({result.total_tokens} / {result.budget_tokens} tokens):")
+    for f in result.selected_files:
+        print(f"[{f.tier.value.upper()}] {f.path} (score: {f.score:.3f}, tokens: {f.token_count})")
+
+    # Access the exact fenced-markdown serialized prompt ready for LLM consumption:
+    llm_prompt_context = result.rendered_context
+```
+
+#### Opt-In: Seed-Based Change-Impact Retrieval:
+```python
+from reponeedle import RepoNeedleEngine, Query
+
+# Initialize engine with Change-Impact ranking
+engine = RepoNeedleEngine.from_directory(
+    repo_dir="./my-project",
+    ranker="change_impact",
+)
+
+# Populate anchor_files with the seed path to activate relationship scoring
+query = Query(
+    text="Fix authentication session expiration",
+    anchor_files=["auth/session.py"],
+    budget_tokens=4000,
+)
+
+result = engine.retrieve(query)
+
+for f in result.selected_files:
+    print(f"[{f.tier.value.upper()}] {f.path} (score: {f.score:.3f}, tokens: {f.token_count})")
+    if f.explanation:
+        print(f"       Attribution: {f.explanation}")
+```
+
+> [!NOTE]
+> **Behavior of Plain-String (Seedless) Change-Impact Queries**:
+> The Change-Impact ranking engine reads seed file paths from `query.anchor_files`. When `engine.retrieve()` is called with a plain string query (or without `anchor_files`):
+> - **Lexical and semantic scoring remain active**: Candidate files continue to be retrieved and scored using combined BM25 lexical matching and subword semantic retrieval.
+> - **Seed suppression is inactive without `anchor_files`**: Seed File Suppression (SFS) is not triggered since there are no designated seed files to down-weight.
+> - **Seed-relative relationship boosts are inactive without `anchor_files`**: Relationship scoring across dependencies (static imports, symbols, calls, build manifests, test-stems, and directory proximity) is skipped.
+> - **Test Intent Prior adjustments remain available according to detected query intent**: Test files receive boosts or penalties (`tip_multiplier_test` / `tip_multiplier_non_test`) based on detected test intent in the query text.
+> 
+> To activate seed-relative relationship boosts and seed suppression, pass seed paths via `Query(text=..., anchor_files=['path/to/seed.py'])`.
+
+---
+
+### 3. Command Line Interface (CLI)
+
+```bash
+# Default 'legacy' retrieval under a 4000 token limit
+reponeedle "Fix AttributeError in SecureCookieSessionInterface cookie saving" --repo ./flask --budget 4000
+
+# Opt-in Change-Impact retrieval (applies Test Intent Prior scoring)
+reponeedle "fix token expiration in auth/session.py" --repo ./my-project --budget 4000 --ranker change_impact
+
+# Task-conditioned retrieval for finding regression tests
+reponeedle "test suite for HTTPBasicAuth" --repo ./requests --task code2test --budget 2000
+
+# Serialize packaged markdown directly to a file
+reponeedle "blueprints and route registration" --repo ./flask --budget 4000 --output-file prompt.md
+```
+
+> [!NOTE]
+> **CLI Seed-File Limitation**: The current CLI accepts query text as a positional string and does not currently expose a seed-file argument. Standard CLI queries run with `--ranker change_impact` execute as seedless queries: lexical and semantic scoring remain active, and Test Intent Prior adjustments remain available according to detected query intent, while seed suppression and seed-relative relationship boosts are inactive without `anchor_files`. Full seed-based relationship scoring is available via the Python API (`Query(anchor_files=[...])`). CLI seed-file support is planned as a separate future enhancement.
+
+---
+
+## 🧠 CUD-Pack Architecture (Capacity-Sensitive Utility-Density Packing)
+
+RepoNeedle integrates the **CUD-Pack** context optimization engine, replacing heuristic packing with capacity-elastic utility-density optimization:
+
+### 1. Dual Operational Modes
+- **Compact Mode ($\le 4\text{k}$ tokens, $\alpha = 0.65$)**: Prioritizes focal implementation spans (`FOCUS_SPAN`) and structural skeletons (`SKELETON`). Prevents single distractor files from consuming the budget and displacing multi-target dependencies.
+- **Complete Context Mode ($\ge 8\text{k}$ tokens, $\alpha = 0.25$)**: Prioritizes whole source files (`FULL`), selectively utilizing structural outlines for oversized files ($> 500$ tokens) when capacity is tight.
+
+### 2. Verified Token Budget Invariants
+- **Deterministic Budget Enforcement**: Evaluated across tested nominal budgets (30, 50, 100, 500, 2000, 4000 tokens), CUD-Pack strictly enforces $C_{\text{actual}} \le B_{\text{configured}}$ with zero downstream token overflows according to RepoNeedle's deterministic regex token-counting model (`regex_code_tokenizer_v1`).
+- **Overhead Accounting**: Pre-computes file header fences (`# File: {path}\n```{ext}\n...\n```\n\n`) deterministically before admitting candidates into the knapsack.
+- **Representation Fallback Hierarchy**: Monotonically degrades candidate representations (`FULL` $\to$ `FOCUS_SPAN` $\to$ `SKELETON`) when full files exceed available token capacity.
+
+> [!WARNING]
+> **Tokenizer Boundary Disclosure**: Budget non-exceedance is enforced strictly according to RepoNeedle's internal regex token model (`regex_code_tokenizer_v1`). This does not imply exact token-count equivalence with arbitrary third-party LLM tokenizers (e.g. OpenAI tiktoken cl100k, LLaMA sentencepiece, or Qwen) or guarantee downstream token behavior for every external provider.
+
+---
+
+## 🔬 Reproducing Benchmark Results
+
+Run the complete deterministic test suite and benchmarks:
+
+```bash
+# 1. Run automated test suite (271 passed tests locally, 245 in clean CI checkouts)
+pytest -q -rs
+
+# 2. Run component ablation study with 95% Bootstrap CI
+python benchmarks/run_ablations.py
+
+# 3. Verify actual BPE token payload reduction and latency
+python scripts/verify_token_savings_and_latency.py
+
+# 4. Build distribution packages
+python -m build --no-isolation
+```
+
+---
+
+## 📦 Hugging Face Release Artifacts
+
+RepoNeedle distributes its validated assets primarily via Hugging Face:
+
+1. **RepoNeedle-Bench Dataset**: [`hf_artifacts/dataset/`](hf_artifacts/dataset/README.md)
+   - Permissively licensed JSONL datasets (`train.jsonl`, `test.jsonl`), schema, and provenance documentation.
+2. **RepoNeedle Playground Space**: [`hf_artifacts/space/`](hf_artifacts/space/README.md)
+   - Interactive Gradio demonstration allowing side-by-side comparison of RepoNeedle against classical BM25 and RepoMap PageRank.
+3. **Automated Publishing Script**: [`scripts/prepare_hf_release.py`](scripts/prepare_hf_release.py)
+   - Verified local release package ready to publish with `python scripts/prepare_hf_release.py --token <HF_TOKEN>`.
+
+---
+
+## 🛡️ License & Data Provenance
+
+- **Source Code**: [MIT License](LICENSE)
+- **RepoNeedle-Bench**: [MIT License](hf_artifacts/dataset/README.md)
+- **Data Provenance**: Sourced exclusively from permissively licensed open-source projects (BSD-3-Clause, Apache-2.0, MIT). **Zero CC BY-NC or copyleft share-alike data is used.** Full manifest available in [`data_provenance/PROVENANCE.md`](data_provenance/PROVENANCE.md).
+- **Upstream Notices**: Agent Retrieval Bench is Copyright (c) 2026 Agent Retrieval Bench contributors (MIT License). All upstream open-source code snapshots retain their original licenses and copyright notices.
+
+---
+
+## 📚 Citation
+
+```bibtex
+@misc{reponeedle2026,
+  author = {RepoNeedle Contributors},
+  title = {RepoNeedle: Token-Budget-Aware, Workflow-Adaptive Repository Context Retrieval for AI Coding Agents},
+  year = {2026},
+  publisher = {GitHub},
+  howpublished = {\url{https://github.com/krishika08/RepoNeedle}}
+}
+```