socratic-rag 0.1.0__tar.gz

socratic_rag-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Socratic RAG Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

socratic_rag-0.1.0/PKG-INFO

@@ -0,0 +1,416 @@
+Metadata-Version: 2.4
+Name: socratic-rag
+Version: 0.1.0
+Summary: Production-grade Retrieval-Augmented Generation package
+Author-email: Socratic RAG <info@socratic-rag.dev>
+License: MIT
+Project-URL: Homepage, https://github.com/Nireus79/Socratic-rag
+Project-URL: Documentation, https://github.com/Nireus79/Socratic-rag/tree/main/docs
+Project-URL: Repository, https://github.com/Nireus79/Socratic-rag.git
+Project-URL: Issues, https://github.com/Nireus79/Socratic-rag/issues
+Keywords: rag,retrieval-augmented-generation,nlp,embeddings,vector-database
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20.0
+Requires-Dist: sentence-transformers>=2.0.0
+Provides-Extra: chromadb
+Requires-Dist: chromadb>=0.4.0; extra == "chromadb"
+Provides-Extra: qdrant
+Requires-Dist: qdrant-client>=1.0.0; extra == "qdrant"
+Provides-Extra: faiss
+Requires-Dist: faiss-cpu>=1.7.0; extra == "faiss"
+Provides-Extra: pinecone
+Requires-Dist: pinecone-client>=2.0.0; extra == "pinecone"
+Provides-Extra: pdf
+Requires-Dist: PyPDF2>=3.0.0; extra == "pdf"
+Provides-Extra: markdown
+Requires-Dist: markdown>=3.0.0; extra == "markdown"
+Provides-Extra: openclaw
+Provides-Extra: langchain
+Requires-Dist: langchain>=0.1.0; extra == "langchain"
+Provides-Extra: nexus
+Requires-Dist: socrates-nexus>=0.3.0; extra == "nexus"
+Provides-Extra: all
+Requires-Dist: chromadb>=0.4.0; extra == "all"
+Requires-Dist: qdrant-client>=1.0.0; extra == "all"
+Requires-Dist: faiss-cpu>=1.7.0; extra == "all"
+Requires-Dist: PyPDF2>=3.0.0; extra == "all"
+Requires-Dist: markdown>=3.0.0; extra == "all"
+Requires-Dist: langchain>=0.1.0; extra == "all"
+Requires-Dist: socrates-nexus>=0.3.0; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Requires-Dist: black>=23.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0; extra == "dev"
+Dynamic: license-file
+
+# Socratic RAG
+
+Production-grade Retrieval-Augmented Generation (RAG) package for Python.
+
+## Features
+
+- **Multiple Vector Databases**: ChromaDB, Qdrant, FAISS, Pinecone (with extensible provider pattern)
+- **Flexible Embedding Providers**: Sentence Transformers, OpenAI (via Socrates Nexus)
+- **Smart Chunking**: Fixed-size, semantic, and recursive chunking strategies
+- **Document Processing**: Text, PDF, Markdown, and code files
+- **Framework Integrations**: Openclaw skills and LangChain components
+- **Async Support**: Full async/await interface for non-blocking operations
+- **Production Ready**: 70%+ test coverage, type hints, comprehensive documentation
+
+## Installation
+
+### Basic Installation
+
+```bash
+pip install socratic-rag
+```
+
+### With Optional Dependencies
+
+```bash
+# All features
+pip install socratic-rag[all]
+
+# Specific vector stores
+pip install socratic-rag[chromadb,qdrant,faiss]
+
+# Document processing
+pip install socratic-rag[pdf,markdown]
+
+# Integrations
+pip install socratic-rag[langchain,openclaw,nexus]
+
+# Development
+pip install socratic-rag[dev]
+```
+
+## Quick Start
+
+### Basic Usage
+
+```python
+from socratic_rag import RAGClient
+
+# Initialize client
+client = RAGClient()
+
+# Add documents
+doc_id = client.add_document(
+    content="Python is a programming language created by Guido van Rossum.",
+    source="python_facts.txt"
+)
+
+# Search
+results = client.search("What is Python?", top_k=5)
+for result in results:
+    print(f"Score: {result.score:.2f}")
+    print(f"Text: {result.chunk.text}\n")
+
+# Retrieve formatted context for LLM
+context = client.retrieve_context("What is Python?")
+print(context)
+```
+
+### Custom Configuration
+
+```python
+from socratic_rag import RAGClient, RAGConfig
+
+config = RAGConfig(
+    vector_store="chromadb",
+    embedder="sentence-transformers",
+    chunking_strategy="fixed",
+    chunk_size=512,
+    chunk_overlap=50,
+    top_k=5,
+)
+
+client = RAGClient(config)
+```
+
+### Async Usage
+
+```python
+import asyncio
+from socratic_rag import AsyncRAGClient
+
+async def main():
+    client = AsyncRAGClient()
+
+    # Add documents asynchronously
+    doc_id = await client.add_document(
+        content="Document content",
+        source="source.txt"
+    )
+
+    # Search asynchronously
+    results = await client.search("query")
+
+    # Retrieve context asynchronously
+    context = await client.retrieve_context("query")
+
+asyncio.run(main())
+```
+
+## Architecture
+
+### Provider Pattern
+
+Socratic RAG uses an extensible provider pattern for easy integration of new components:
+
+```
+RAGClient
+├── Embedder Provider (sentence-transformers, OpenAI, etc.)
+├── Chunker Provider (fixed, semantic, recursive)
+└── Vector Store Provider (ChromaDB, Qdrant, FAISS, Pinecone)
+```
+
+### Core Models
+
+- **Document**: Raw document with metadata
+- **Chunk**: Text chunk with position and metadata
+- **SearchResult**: Search result with relevance score
+- **RAGConfig**: Configuration object
+
+## Examples
+
+See the `examples/` directory for:
+
+1. `01_basic_rag.py` - Basic RAG workflow
+2. `02_qdrant_rag.py` - Using Qdrant vector store
+3. `03_faiss_rag.py` - Using FAISS vector store
+4. `04_document_processing.py` - Document processing
+5. `05_openclaw_integration.py` - Openclaw skill integration
+6. `06_langchain_integration.py` - LangChain integration
+7. `07_llm_powered_rag.py` - RAG with LLM (using Socrates Nexus)
+
+## Testing
+
+Run the test suite:
+
+```bash
+pytest tests/ -v --cov=socratic_rag
+```
+
+Specific test categories:
+
+```bash
+# Unit tests only
+pytest tests/ -m unit
+
+# Integration tests only
+pytest tests/ -m integration
+
+# Exclude slow tests
+pytest tests/ -m "not slow"
+```
+
+## Vector Store Providers
+
+### ChromaDB (Default)
+
+```python
+from socratic_rag import RAGClient, RAGConfig
+
+config = RAGConfig(vector_store="chromadb")
+client = RAGClient(config)
+```
+
+### Qdrant
+
+```python
+config = RAGConfig(vector_store="qdrant")
+client = RAGClient(config)
+```
+
+### FAISS
+
+```python
+config = RAGConfig(vector_store="faiss")
+client = RAGClient(config)
+```
+
+## Embedding Providers
+
+### Sentence Transformers (Default)
+
+```python
+config = RAGConfig(embedder="sentence-transformers")
+client = RAGClient(config)
+```
+
+### OpenAI (via Socrates Nexus)
+
+```python
+config = RAGConfig(embedder="openai")
+client = RAGClient(config)
+```
+
+## Document Processing
+
+### Text Files
+
+```python
+from socratic_rag.processors import TextProcessor
+
+processor = TextProcessor()
+documents = processor.process("path/to/file.txt")
+```
+
+### PDF Files
+
+```python
+from socratic_rag.processors import PDFProcessor
+
+processor = PDFProcessor()
+documents = processor.process("path/to/file.pdf")
+```
+
+### Markdown Files
+
+```python
+from socratic_rag.processors import MarkdownProcessor
+
+processor = MarkdownProcessor()
+documents = processor.process("path/to/file.md")
+```
+
+## Framework Integrations
+
+### Openclaw
+
+```python
+from socratic_rag.integrations.openclaw import SocraticRAGSkill
+
+skill = SocraticRAGSkill(vector_store="chromadb")
+skill.add_document("content", "source.txt")
+results = skill.search("query")
+```
+
+### LangChain
+
+```python
+from socratic_rag.integrations.langchain import SocraticRAGRetriever
+from langchain.chat_models import ChatAnthropic
+from langchain.chains import RetrievalQA
+
+retriever = SocraticRAGRetriever(client=rag_client, top_k=5)
+llm = ChatAnthropic(model="claude-sonnet")
+qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)
+
+answer = qa.run("What is Python?")
+```
+
+## API Reference
+
+### RAGClient
+
+#### Methods
+
+- `add_document(content, source, metadata=None) -> str` - Add document to knowledge base
+- `search(query, top_k=None, filters=None) -> List[SearchResult]` - Search for relevant documents
+- `retrieve_context(query, top_k=None) -> str` - Retrieve formatted context for LLM
+- `clear() -> bool` - Clear all documents
+
+### AsyncRAGClient
+
+Same methods as RAGClient but with async/await.
+
+## Configuration Options
+
+```python
+RAGConfig(
+    vector_store="chromadb",           # Vector store provider
+    embedder="sentence-transformers",  # Embedding provider
+    chunking_strategy="fixed",         # Chunking strategy
+    chunk_size=512,                    # Characters per chunk
+    chunk_overlap=50,                  # Overlap between chunks
+    top_k=5,                           # Default number of results
+    embedding_cache=True,              # Cache embeddings
+    cache_ttl=3600,                    # Cache TTL in seconds
+    collection_name="socratic_rag",    # Collection name
+)
+```
+
+## Exceptions
+
+- `SocraticRAGError` - Base exception
+- `ConfigurationError` - Configuration validation error
+- `VectorStoreError` - Vector store operation error
+- `EmbeddingError` - Embedding operation error
+- `ChunkingError` - Chunking operation error
+- `ProcessorError` - Document processing error
+- `DocumentNotFoundError` - Document not found
+- `ProviderNotFoundError` - Provider not found
+
+## Performance Tips
+
+1. **Use appropriate chunk size**: Smaller chunks (256-512) for dense retrieval, larger (1024+) for sparse
+2. **Set overlap wisely**: 10-15% overlap usually works well
+3. **Cache embeddings**: Enable embedding cache for repeated queries
+4. **Batch operations**: Use `embed_batch()` for multiple embeddings
+5. **Choose right embedder**: Sentence Transformers for local, OpenAI for production scale
+
+## Contributing
+
+Contributions are welcome! Please see `CONTRIBUTING.md` for guidelines.
+
+## License
+
+MIT License - see `LICENSE` file for details
+
+## Citation
+
+If you use Socratic RAG in your research, please cite:
+
+```bibtex
+@software{socratic_rag_2024,
+  title={Socratic RAG: Production-grade Retrieval-Augmented Generation},
+  author={Your Name},
+  year={2024},
+  url={https://github.com/Nireus79/Socratic-rag}
+}
+```
+
+## Roadmap
+
+### v0.1.0 (Current)
+- ✅ Core RAG functionality
+- ✅ ChromaDB vector store
+- ✅ Sentence Transformers embeddings
+- ✅ Fixed-size chunking
+- ✅ Openclaw integration
+- ✅ LangChain integration
+
+### v0.2.0
+- Qdrant and FAISS vector stores
+- Semantic chunking
+- Pinecone cloud provider
+- Vision model support
+
+### v0.3.0
+- Hybrid search (vector + keyword)
+- Re-ranking with cross-encoders
+- Multi-language support
+
+## Support
+
+For issues, questions, or suggestions:
+- GitHub Issues: https://github.com/Nireus79/Socratic-rag/issues
+- Discussions: https://github.com/Nireus79/Socratic-rag/discussions