ragcheck-cli 0.2.0__py3-none-any.whl

ragcheck_cli-0.2.0.dist-info/METADATA

@@ -0,0 +1,193 @@
+Metadata-Version: 2.4
+Name: ragcheck-cli
+Version: 0.2.0
+Summary: Lighthouse for RAG systems — diagnose and fix your retrieval pipeline
+Project-URL: Homepage, https://github.com/pranay7863/ragcheck
+Project-URL: Documentation, https://github.com/pranay7863/ragcheck/blob/main/README.md
+Project-URL: Repository, https://github.com/pranay7863/ragcheck
+Project-URL: Issues, https://github.com/pranay7863/ragcheck/issues
+Author-email: Pranay Mane <pranaymane78@gmail.com>
+License: MIT
+License-File: LICENSE
+Keywords: ai,chunking,diagnostics,evaluation,llm,rag,retrieval
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Requires-Dist: chromadb>=0.4.0
+Requires-Dist: jinja2>=3.1.0
+Requires-Dist: litellm>=1.0.0
+Requires-Dist: nltk>=3.9.0
+Requires-Dist: pydantic>=2.5.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: sentence-transformers>=2.2.0
+Requires-Dist: transformers>=4.30.0
+Requires-Dist: typer>=0.12.0
+Provides-Extra: export
+Requires-Dist: playwright>=1.40.0; extra == 'export'
+Provides-Extra: pdf
+Requires-Dist: pypdf2>=3.0.0; extra == 'pdf'
+Provides-Extra: ragas
+Requires-Dist: ragas<0.5.0,>=0.4.0; extra == 'ragas'
+Description-Content-Type: text/markdown
+
+# ragcheck - Lighthouse for RAG Systems
+
+[![PyPI version](https://badge.fury.io/py/ragcheck.svg)](https://badge.fury.io/py/ragcheck)
+[![Python](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+> One command to diagnose your RAG pipeline and get actionable fixes.
+
+```bash
+pip install ragcheck
+ragcheck init
+ragcheck run --docs ./data --query "What is Article 370?"
+```
+
+## What is ragcheck?
+
+**ragcheck** is a lightweight, one-command diagnostic CLI that generates a beautiful, shareable HTML report analyzing why your RAG system fails and how to fix it.
+
+Think of it as **Lighthouse for RAG systems** — just like Lighthouse audits web pages, ragcheck audits your retrieval pipeline.
+
+## Features
+
+- **Auto-Generated Test Suite** - 50 synthetic questions from your documents
+- **Chunk Visualizer** - See exactly where your chunking breaks
+- **Retrieval Heatmap** - Identify dead chunks and dominant chunks
+- **Failure Classification** - Know WHY your RAG fails, not just THAT it fails
+- **Actionable Recommendations** - Specific fixes with predicted impact
+- **CI/CD Integration** - Fail builds when RAG quality regresses
+
+## Quick Start
+
+### Installation
+
+```bash
+pip install ragcheck
+```
+
+Or with [uv](https://github.com/astral-sh/uv):
+
+```bash
+uv tool install ragcheck
+```
+
+### Initialize
+
+```bash
+ragcheck init
+```
+
+Creates a `ragcheck.yaml` config file in your project.
+
+### Run Analysis
+
+```bash
+ragcheck run --docs ./data --query "Your test query"
+```
+
+Generates `ragcheck_report.html` with:
+- Scorecards (retrieval accuracy, faithfulness)
+- Chunk boundary visualization
+- Retrieval heatmap
+- Failure mode classification
+- Before/after score predictions
+
+### CI Mode
+
+```bash
+ragcheck run --docs ./data --ci --min-score 0.80
+```
+
+Returns exit code 0/1. Use in GitHub Actions to fail builds on quality regression.
+
+## Example Report
+
+![ragcheck report](https://raw.githubusercontent.com/yourusername/ragcheck/main/docs/report-screenshot.png)
+
+## Architecture
+
+```
+ragcheck CLI
+    ├── Chunk Analyzer (6 strategies + benchmark)
+    ├── Retriever Tester (auto-QA + dense retrieval)
+    ├── Failure Classifier (4 failure modes)
+    ├── Recommendation Engine (decision tree)
+    └── Report Engine (Jinja2 + CSS/HTML HTML)
+```
+
+## Tech Stack
+
+| Component | Tool |
+|-----------|------|
+| CLI | Typer + Rich |
+| Config | Pydantic |
+| Embeddings | sentence-transformers |
+| Vector DB | ChromaDB |
+| LLM Interface | LiteLLM |
+| Reports | Jinja2 + CSS/HTML |
+
+## Configuration
+
+`ragcheck.yaml`:
+
+```yaml
+project_name: ragcheck
+docs_path: ./data
+chunking:
+  strategy: recursive
+  chunk_size: 512
+  chunk_overlap: 128
+llm:
+  provider: openai
+  model: gpt-3.5-turbo
+retrieval:
+  top_k: 5
+  similarity_threshold: 0.7
+report:
+  format: html
+  include_heatmap: true
+```
+
+## Development
+
+```bash
+git clone https://github.com/pranay7863/ragcheck.git
+cd ragcheck
+uv sync
+uv run pytest
+uv run ruff check .
+uv run mypy ragcheck/
+```
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md)
+
+## License
+
+MIT — see [LICENSE](LICENSE)
+
+## Roadmap
+
+- [x] v0.2.0 — Offline reports, NLI faithfulness, scaled auto-QA, chunk viz
+- [ ] v0.3.0 — More vector DBs (Pinecone, Weaviate)
+- [ ] v0.3.0 — SaaS API for teams
+- [ ] v0.4.0 — Enterprise features (SSO, audit logs)
+
+## Support
+
+- [GitHub Issues](https://github.com/pranay7863/ragcheck/issues)
+- Twitter: [@ypranay53](https://twitter.com/pranay53)
+
+---
+
+**Built with discipline.** Read the [blueprint](docs/ARCHITECTURE.md) that started it all.

ragcheck_cli-0.2.0.dist-info/RECORD

@@ -0,0 +1,5 @@
+ragcheck_cli-0.2.0.dist-info/METADATA,sha256=JPpmFymaWImVpuUKi0rLZREHIXT2nr3N-Ox9eiz4SlM,5234
+ragcheck_cli-0.2.0.dist-info/WHEEL,sha256=mffPy8wBnZQn2VnJUU5jE99KsxaSfiyMHV9Yt0aLVxs,87
+ragcheck_cli-0.2.0.dist-info/entry_points.txt,sha256=QqpErtZhq0YrjKK5Zb5F1mo5RXGujeMxooiT0W6DXMc,46
+ragcheck_cli-0.2.0.dist-info/licenses/LICENSE,sha256=nvPUPz4ZTWla9Kj_H9V9STjmz39v9DiIAuGuXxjAKSw,1068
+ragcheck_cli-0.2.0.dist-info/RECORD,,

ragcheck_cli-0.2.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.30.1
+Root-Is-Purelib: true
+Tag: py3-none-any

ragcheck_cli-0.2.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+ragcheck = ragcheck.cli:app

ragcheck_cli-0.2.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Pranay Mane
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.