evret 0.0.1b0__tar.gz

evret-0.0.1b0/LICENSE

@@ -0,0 +1,20 @@
+MIT License
+
+Copyright (c) 2026 Kaivid Labs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

evret-0.0.1b0/PKG-INFO

@@ -0,0 +1,363 @@
+Metadata-Version: 2.4
+Name: evret
+Version: 0.0.1b0
+Summary: A focused retriever evaluation framework with standard ranking metrics.
+Author: Evret Contributors
+License: MIT
+Project-URL: Homepage, https://github.com/example/evret
+Project-URL: Repository, https://github.com/example/evret
+Keywords: retrieval,rag,evaluation,metrics,information-retrieval
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastembed>=0.8.0
+Requires-Dist: qdrant-client>=1.17.1
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0.0; extra == "dev"
+Provides-Extra: qdrant
+Requires-Dist: qdrant-client>=1.9.0; extra == "qdrant"
+Provides-Extra: chroma
+Requires-Dist: chromadb>=0.5.0; extra == "chroma"
+Provides-Extra: weaviate
+Requires-Dist: weaviate-client>=4.16.0; extra == "weaviate"
+Provides-Extra: milvus
+Requires-Dist: pymilvus>=2.4.0; extra == "milvus"
+Provides-Extra: langchain
+Requires-Dist: langchain-core>=0.2.0; extra == "langchain"
+Provides-Extra: llamaindex
+Requires-Dist: llama-index-core>=0.10.0; extra == "llamaindex"
+Provides-Extra: all
+Requires-Dist: qdrant-client>=1.9.0; extra == "all"
+Requires-Dist: chromadb>=0.5.0; extra == "all"
+Requires-Dist: weaviate-client>=4.16.0; extra == "all"
+Requires-Dist: pymilvus>=2.4.0; extra == "all"
+Requires-Dist: langchain-core>=0.2.0; extra == "all"
+Requires-Dist: llama-index-core>=0.10.0; extra == "all"
+Provides-Extra: integration
+Requires-Dist: pytest>=8.0.0; extra == "integration"
+Requires-Dist: qdrant-client>=1.9.0; extra == "integration"
+Requires-Dist: chromadb>=0.5.0; extra == "integration"
+Requires-Dist: weaviate-client>=4.16.0; extra == "integration"
+Requires-Dist: pymilvus>=2.4.0; extra == "integration"
+Requires-Dist: langchain-core>=0.2.0; extra == "integration"
+Requires-Dist: llama-index-core>=0.10.0; extra == "integration"
+Requires-Dist: testcontainers>=4.8.0; extra == "integration"
+Dynamic: license-file
+
+<div align="center">
+
+# 🎯 Evret
+
+**A focused, lightweight retriever evaluation framework with standard Information Retrieval metrics**
+
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![PyPI version](https://badge.fury.io/py/evret.svg)](https://badge.fury.io/py/evret)
+[![Tests](https://img.shields.io/badge/tests-passing-brightgreen.svg)](https://github.com/example/evret/actions)
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)
+
+Evret brings standard Information Retrieval metrics to your RAG and search systems. Evaluate retrievers with Hit Rate, Recall, Precision, MRR, NDCG, and Average Precision in just a few lines of code. Built for simplicity, extensibility, and seamless integration with vector databases and AI frameworks.
+
+</div>
+
+---
+
+## 🌟 Overview
+
+**Evret** is a modern Python framework designed for evaluating retrieval systems in RAG (Retrieval-Augmented Generation) pipelines and search applications. It provides:
+
+- **Standard IR Metrics**: Hit Rate, Recall, Precision, MRR, NDCG, and Average Precision
+- **Vector Database Support**: Native adapters for Qdrant and other vector databases
+- **Framework Integration**: Seamless adapters for LangChain
+- **Production-Ready**: Type-safe, well-tested, and optimized for real-world use cases
+
+Whether you're building a semantic search engine, evaluating RAG systems, or benchmarking retrieval models, Evret gives you the tools to measure what matters.
+
+---
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+pip install evret
+```
+
+For optional integrations:
+
+```bash
+# Install all optional integrations
+pip install evret[all]
+```
+
+### 5-Minute Evaluation
+
+```python
+from evret import EvaluationDataset, Evaluator, HitRate, MRR, NDCG
+# Load your evaluation dataset
+dataset = EvaluationDataset.from_json("eval_data.json")
+
+# Evaluate your retriever
+evaluator = Evaluator(
+    retriever=my_retriever,
+    metrics=[HitRate(k=5), MRR(k=10), NDCG(k=10)]
+)
+
+results = evaluator.evaluate(dataset)
+print(results.summary())
+
+# Export results
+results.to_json("results.json")
+results.to_csv("results.csv")
+```
+
+### Minimal Metrics Example
+
+```python
+from evret import HitRate, Recall, Precision, MRR, NDCG, AveragePrecision
+
+retrieved = [
+    ["doc_1", "doc_5", "doc_2", "doc_9"],
+    ["doc_8", "doc_7", "doc_6", "doc_3"],
+]
+relevant = [
+    {"doc_1", "doc_2"},
+    {"doc_3"},
+]
+
+metrics = [HitRate(k=3), Recall(k=3), Precision(k=3), MRR(k=3), NDCG(k=3)]
+
+for metric in metrics:
+    score = metric.score(
+        retrieved_by_query=retrieved,
+        relevant_by_query=relevant,
+    )
+    print(f"{metric.name}: {score:.4f}")
+```
+
+---
+
+## 🛠 Local Development Setup
+
+Use `uv` for local development:
+
+```bash
+uv venv
+source .venv/bin/activate
+uv pip install -e .
+```
+
+Install all optional integrations:
+
+```bash
+uv pip install -e ".[all]"
+```
+
+Run tests:
+
+```bash
+pytest
+```
+
+---
+
+## 📚 Documentation
+
+Evret docs use MkDocs with Material theme.
+
+Install docs dependencies:
+
+```bash
+uv pip install -e ".[docs]"
+```
+
+Run docs locally:
+
+```bash
+mkdocs serve
+```
+
+Build static docs:
+
+```bash
+mkdocs build
+```
+
+---
+
+## 📊 Metrics
+
+Evret supports all standard Information Retrieval metrics:
+
+| Metric | Description | Use Case |
+|--------|-------------|----------|
+| **Hit Rate@k** | % of queries with at least one relevant doc in top-k | Binary relevance, recall-focused |
+| **Recall@k** | % of relevant docs found in top-k | Comprehensive retrieval |
+| **Precision@k** | % of top-k results that are relevant | Precision-focused systems |
+| **MRR@k** | Mean Reciprocal Rank of first relevant doc | Single-answer retrieval |
+| **NDCG@k** | Normalized Discounted Cumulative Gain | Rank-aware binary relevance quality |
+| **Average Precision@k** | Area under precision-recall curve | Overall ranking quality |
+
+### Usage Example
+
+```python
+from evret import HitRate, Recall, Precision, MRR, NDCG, AveragePrecision
+
+metrics = [
+    HitRate(k=5),
+    Recall(k=10),
+    Precision(k=5),
+    MRR(k=10),
+    NDCG(k=10),
+    AveragePrecision(k=10),
+]
+
+# Score a single query
+retrieved = ["doc_1", "doc_5", "doc_2"]
+relevant = {"doc_1", "doc_2"}
+
+for metric in metrics:
+    score = metric.score(
+        retrieved_by_query=[retrieved],
+        relevant_by_query=[relevant],
+    )
+    print(f"{metric.name}: {score:.4f}")
+```
+
+---
+
+## 🔌 Integrations
+
+### Qdrant Vector Database
+
+```python
+from evret.retrievers import QdrantRetriever
+
+retriever = QdrantRetriever(
+    collection_name="docs",
+    query_encoder=embed_query,
+    url="http://localhost:6333",
+    id_field="doc_id",
+)
+```
+
+### LangChain Integration
+
+```python
+from evret.integrations import LangChainRetrieverAdapter
+
+# Wrap any Evret retriever for use in LangChain
+lc_retriever = LangChainRetrieverAdapter(evret_retriever=retriever, k=5)
+docs = lc_retriever.invoke("what is RAG?")
+```
+
+---
+
+## 📁 Examples
+
+### Basic Evaluation Pipeline
+
+```python
+from evret import EvaluationDataset, Evaluator, HitRate, MRR
+
+dataset = EvaluationDataset.from_json("eval_data.json")
+evaluator = Evaluator(retriever=my_retriever, metrics=[HitRate(k=5), MRR(k=10)])
+results = evaluator.evaluate(dataset)
+
+results.to_json("results.json")
+results.to_csv("results.csv")
+print(results.summary())
+```
+
+### Custom Retriever
+
+```python
+from evret.retrievers import BaseRetriever
+from evret import RetrievalResult
+
+class MyCustomRetriever(BaseRetriever):
+    def retrieve(self, query: str, k: int) -> list[RetrievalResult]:
+        self._validate_k(k)
+        # Your retrieval logic here
+        return [
+            RetrievalResult(doc_id="doc_1", score=0.95, metadata={"text": "..."}),
+            RetrievalResult(doc_id="doc_2", score=0.87),
+        ]
+```
+
+### Run Examples Locally
+
+```bash
+# Basic evaluation example
+python examples/evaluate_retriever.py
+
+# Jupyter notebook quickstart
+jupyter notebook examples/langchain_rag_evaluation.ipynb
+```
+
+---
+
+## 🧪 Testing
+
+### Run Unit Tests
+
+```bash
+pytest
+```
+
+### Run Integration Tests
+
+Integration tests require Docker to be running:
+
+```bash
+# Start Docker Desktop/daemon first, then run:
+EVRET_RUN_INTEGRATION=1 pytest -m integration
+```
+
+This will spin up Docker containers for Qdrant and Chroma to run end-to-end tests.
+
+---
+
+## 📄 License
+
+MIT License - see [LICENSE](LICENSE) for details.
+
+---
+
+## 🤝 Contributing
+
+Contributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.
+
+---
+
+## 📚 Citation
+
+If you use Evret in your research, please cite:
+
+```bibtex
+@software{evret2026,
+  title={Evret: A Focused Retriever Evaluation Framework},
+  author={lucifertrj},
+  year={2026},
+}
+```
+
+---
+
+<div align="center">
+
+**Built with ❤️ for the RAG and IR community**
+
+[GitHub](https://github.com/example/evret) • [Issues](https://github.com/example/evret/issues) • [Discussions](https://github.com/example/evret/discussions)
+
+</div>

evret-0.0.1b0/README.md

@@ -0,0 +1,308 @@
+<div align="center">
+
+# 🎯 Evret
+
+**A focused, lightweight retriever evaluation framework with standard Information Retrieval metrics**
+
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![PyPI version](https://badge.fury.io/py/evret.svg)](https://badge.fury.io/py/evret)
+[![Tests](https://img.shields.io/badge/tests-passing-brightgreen.svg)](https://github.com/example/evret/actions)
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)
+
+Evret brings standard Information Retrieval metrics to your RAG and search systems. Evaluate retrievers with Hit Rate, Recall, Precision, MRR, NDCG, and Average Precision in just a few lines of code. Built for simplicity, extensibility, and seamless integration with vector databases and AI frameworks.
+
+</div>
+
+---
+
+## 🌟 Overview
+
+**Evret** is a modern Python framework designed for evaluating retrieval systems in RAG (Retrieval-Augmented Generation) pipelines and search applications. It provides:
+
+- **Standard IR Metrics**: Hit Rate, Recall, Precision, MRR, NDCG, and Average Precision
+- **Vector Database Support**: Native adapters for Qdrant and other vector databases
+- **Framework Integration**: Seamless adapters for LangChain
+- **Production-Ready**: Type-safe, well-tested, and optimized for real-world use cases
+
+Whether you're building a semantic search engine, evaluating RAG systems, or benchmarking retrieval models, Evret gives you the tools to measure what matters.
+
+---
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+pip install evret
+```
+
+For optional integrations:
+
+```bash
+# Install all optional integrations
+pip install evret[all]
+```
+
+### 5-Minute Evaluation
+
+```python
+from evret import EvaluationDataset, Evaluator, HitRate, MRR, NDCG
+# Load your evaluation dataset
+dataset = EvaluationDataset.from_json("eval_data.json")
+
+# Evaluate your retriever
+evaluator = Evaluator(
+    retriever=my_retriever,
+    metrics=[HitRate(k=5), MRR(k=10), NDCG(k=10)]
+)
+
+results = evaluator.evaluate(dataset)
+print(results.summary())
+
+# Export results
+results.to_json("results.json")
+results.to_csv("results.csv")
+```
+
+### Minimal Metrics Example
+
+```python
+from evret import HitRate, Recall, Precision, MRR, NDCG, AveragePrecision
+
+retrieved = [
+    ["doc_1", "doc_5", "doc_2", "doc_9"],
+    ["doc_8", "doc_7", "doc_6", "doc_3"],
+]
+relevant = [
+    {"doc_1", "doc_2"},
+    {"doc_3"},
+]
+
+metrics = [HitRate(k=3), Recall(k=3), Precision(k=3), MRR(k=3), NDCG(k=3)]
+
+for metric in metrics:
+    score = metric.score(
+        retrieved_by_query=retrieved,
+        relevant_by_query=relevant,
+    )
+    print(f"{metric.name}: {score:.4f}")
+```
+
+---
+
+## 🛠 Local Development Setup
+
+Use `uv` for local development:
+
+```bash
+uv venv
+source .venv/bin/activate
+uv pip install -e .
+```
+
+Install all optional integrations:
+
+```bash
+uv pip install -e ".[all]"
+```
+
+Run tests:
+
+```bash
+pytest
+```
+
+---
+
+## 📚 Documentation
+
+Evret docs use MkDocs with Material theme.
+
+Install docs dependencies:
+
+```bash
+uv pip install -e ".[docs]"
+```
+
+Run docs locally:
+
+```bash
+mkdocs serve
+```
+
+Build static docs:
+
+```bash
+mkdocs build
+```
+
+---
+
+## 📊 Metrics
+
+Evret supports all standard Information Retrieval metrics:
+
+| Metric | Description | Use Case |
+|--------|-------------|----------|
+| **Hit Rate@k** | % of queries with at least one relevant doc in top-k | Binary relevance, recall-focused |
+| **Recall@k** | % of relevant docs found in top-k | Comprehensive retrieval |
+| **Precision@k** | % of top-k results that are relevant | Precision-focused systems |
+| **MRR@k** | Mean Reciprocal Rank of first relevant doc | Single-answer retrieval |
+| **NDCG@k** | Normalized Discounted Cumulative Gain | Rank-aware binary relevance quality |
+| **Average Precision@k** | Area under precision-recall curve | Overall ranking quality |
+
+### Usage Example
+
+```python
+from evret import HitRate, Recall, Precision, MRR, NDCG, AveragePrecision
+
+metrics = [
+    HitRate(k=5),
+    Recall(k=10),
+    Precision(k=5),
+    MRR(k=10),
+    NDCG(k=10),
+    AveragePrecision(k=10),
+]
+
+# Score a single query
+retrieved = ["doc_1", "doc_5", "doc_2"]
+relevant = {"doc_1", "doc_2"}
+
+for metric in metrics:
+    score = metric.score(
+        retrieved_by_query=[retrieved],
+        relevant_by_query=[relevant],
+    )
+    print(f"{metric.name}: {score:.4f}")
+```
+
+---
+
+## 🔌 Integrations
+
+### Qdrant Vector Database
+
+```python
+from evret.retrievers import QdrantRetriever
+
+retriever = QdrantRetriever(
+    collection_name="docs",
+    query_encoder=embed_query,
+    url="http://localhost:6333",
+    id_field="doc_id",
+)
+```
+
+### LangChain Integration
+
+```python
+from evret.integrations import LangChainRetrieverAdapter
+
+# Wrap any Evret retriever for use in LangChain
+lc_retriever = LangChainRetrieverAdapter(evret_retriever=retriever, k=5)
+docs = lc_retriever.invoke("what is RAG?")
+```
+
+---
+
+## 📁 Examples
+
+### Basic Evaluation Pipeline
+
+```python
+from evret import EvaluationDataset, Evaluator, HitRate, MRR
+
+dataset = EvaluationDataset.from_json("eval_data.json")
+evaluator = Evaluator(retriever=my_retriever, metrics=[HitRate(k=5), MRR(k=10)])
+results = evaluator.evaluate(dataset)
+
+results.to_json("results.json")
+results.to_csv("results.csv")
+print(results.summary())
+```
+
+### Custom Retriever
+
+```python
+from evret.retrievers import BaseRetriever
+from evret import RetrievalResult
+
+class MyCustomRetriever(BaseRetriever):
+    def retrieve(self, query: str, k: int) -> list[RetrievalResult]:
+        self._validate_k(k)
+        # Your retrieval logic here
+        return [
+            RetrievalResult(doc_id="doc_1", score=0.95, metadata={"text": "..."}),
+            RetrievalResult(doc_id="doc_2", score=0.87),
+        ]
+```
+
+### Run Examples Locally
+
+```bash
+# Basic evaluation example
+python examples/evaluate_retriever.py
+
+# Jupyter notebook quickstart
+jupyter notebook examples/langchain_rag_evaluation.ipynb
+```
+
+---
+
+## 🧪 Testing
+
+### Run Unit Tests
+
+```bash
+pytest
+```
+
+### Run Integration Tests
+
+Integration tests require Docker to be running:
+
+```bash
+# Start Docker Desktop/daemon first, then run:
+EVRET_RUN_INTEGRATION=1 pytest -m integration
+```
+
+This will spin up Docker containers for Qdrant and Chroma to run end-to-end tests.
+
+---
+
+## 📄 License
+
+MIT License - see [LICENSE](LICENSE) for details.
+
+---
+
+## 🤝 Contributing
+
+Contributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.
+
+---
+
+## 📚 Citation
+
+If you use Evret in your research, please cite:
+
+```bibtex
+@software{evret2026,
+  title={Evret: A Focused Retriever Evaluation Framework},
+  author={lucifertrj},
+  year={2026},
+}
+```
+
+---
+
+<div align="center">
+
+**Built with ❤️ for the RAG and IR community**
+
+[GitHub](https://github.com/example/evret) • [Issues](https://github.com/example/evret/issues) • [Discussions](https://github.com/example/evret/discussions)
+
+</div>