haiku.rag 0.1.0__py3-none-any.whl

haiku_rag-0.1.0.dist-info/METADATA

@@ -0,0 +1,195 @@
+Metadata-Version: 2.4
+Name: haiku.rag
+Version: 0.1.0
+Summary: Retrieval Augmented Generation (RAG) with SQLite
+Author-email: Yiorgis Gozadinos <ggozadinos@gmail.com>
+License: MIT
+License-File: LICENSE
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows :: Windows 10
+Classifier: Operating System :: Microsoft :: Windows :: Windows 11
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: fastmcp>=2.8.1
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: markitdown[audio-transcription,docx,pdf,pptx,xlsx]>=0.1.2
+Requires-Dist: ollama>=0.5.1
+Requires-Dist: pydantic>=2.11.7
+Requires-Dist: python-dotenv>=1.1.0
+Requires-Dist: rich>=14.0.0
+Requires-Dist: sqlite-vec>=0.1.6
+Requires-Dist: tiktoken>=0.9.0
+Requires-Dist: typer>=0.16.0
+Requires-Dist: watchfiles>=1.1.0
+Provides-Extra: voyageai
+Requires-Dist: voyageai>=0.3.2; extra == 'voyageai'
+Description-Content-Type: text/markdown
+
+# Haiku SQLite RAG
+
+A SQLite-based Retrieval-Augmented Generation (RAG) system built for efficient document storage, chunking, and hybrid search capabilities.
+
+## Features
+- **Local SQLite**: No need to run additional servers
+- **Support for various embedding providers**: You can use Ollama, VoyageAI or add your own
+- **Hybrid Search**: Vector search using `sqlite-vec` combined with full-text search `FTS5`, using Reciprocal Rank Fusion
+- **Multi-format Support**: Parse 40+ file formats including PDF, DOCX, HTML, Markdown, audio and more. Or add a url!
+
+## Installation
+
+```bash
+uv pip install haiku.rag
+```
+
+By default Ollama (with the `mxbai-embed-large` model) is used for the embeddings.
+For other providers use:
+
+- **VoyageAI**: `uv pip install haiku.rag --extra voyageai`
+
+## Configuration
+
+If you want to use an alternative embeddings provider (Ollama being the default) you will need to set the provider details through environment variables:
+
+By default:
+
+```bash
+EMBEDDING_PROVIDER="ollama"
+EMBEDDING_MODEL="mxbai-embed-large" # or any other model
+EMBEDDING_VECTOR_DIM=1024
+```
+
+For VoyageAI:
+```bash
+EMBEDDING_PROVIDER="voyageai"
+EMBEDDING_MODEL="voyage-3.5" # or any other model
+EMBEDDING_VECTOR_DIM=1024
+```
+
+## Command Line Interface
+
+`haiku.rag` includes a CLI application for managing documents and performing searches from the command line:
+
+### Available Commands
+
+```bash
+# List all documents
+haiku-rag list
+
+# Add document from text
+haiku-rag add "Your document content here"
+
+# Add document from file or URL
+haiku-rag add-src /path/to/document.pdf
+haiku-rag add-src https://example.com/article.html
+
+# Get and display a specific document
+haiku-rag get 1
+
+# Delete a document by ID
+haiku-rag delete 1
+
+# Search documents
+haiku-rag search "machine learning"
+
+# Search with custom options
+haiku-rag search "python programming" --limit 10 --k 100
+
+# Start MCP server (default HTTP transport)
+haiku-rag serve # --stdio for stdio transport or --sse for SSE transport
+```
+
+All commands support the `--db` option to specify a custom database path. Run
+```bash
+haiku-rag command -h
+```
+to see additional parameters for a command.
+
+## MCP Server
+
+`haiku.rag` includes a Model Context Protocol (MCP) server that exposes RAG functionality as tools for AI assistants like Claude Desktop. The MCP server provides the following tools:
+
+- `add_document_from_file` - Add documents from local file paths
+- `add_document_from_url` - Add documents from URLs
+- `add_document_from_text` - Add documents from raw text content
+- `search_documents` - Search documents using hybrid search
+- `get_document` - Retrieve specific documents by ID
+- `list_documents` - List all documents with pagination
+- `delete_document` - Delete documents by ID
+
+You can start the server (using Streamble HTTP, stdio or SSE transports) with:
+
+```bash
+# Start with default HTTP transport
+haiku-rag serve # --stdio for stdio transport or --sse for SSE transport
+```
+
+## Using `haiku.rag` from python
+
+### Managing documents
+
+```python
+from pathlib import Path
+from haiku.rag.client import HaikuRAG
+
+# Use as async context manager (recommended)
+async with HaikuRAG("path/to/database.db") as client:
+    # Create document from text
+    doc = await client.create_document(
+        content="Your document content here",
+        uri="doc://example",
+        metadata={"source": "manual", "topic": "example"}
+    )
+
+    # Create document from file (auto-parses content)
+    doc = await client.create_document_from_source("path/to/document.pdf")
+
+    # Create document from URL
+    doc = await client.create_document_from_source("https://example.com/article.html")
+
+    # Retrieve documents
+    doc = await client.get_document_by_id(1)
+    doc = await client.get_document_by_uri("file:///path/to/document.pdf")
+
+    # List all documents with pagination
+    docs = await client.list_documents(limit=10, offset=0)
+
+    # Update document content
+    doc.content = "Updated content"
+    await client.update_document(doc)
+
+    # Delete document
+    await client.delete_document(doc.id)
+
+    # Search documents using hybrid search (vector + full-text)
+    results = await client.search("machine learning algorithms", limit=5)
+    for chunk, score in results:
+        print(f"Score: {score:.3f}")
+        print(f"Content: {chunk.content}")
+        print(f"Document ID: {chunk.document_id}")
+        print("---")
+```
+
+## Searching documents
+
+```python
+async with HaikuRAG("database.db") as client:
+
+    results = await client.search(
+        query="machine learning",
+        limit=5,  # Maximum results to return, defaults to 5
+        k=60      # RRF parameter for reciprocal rank fusion, defaults to 60
+    )
+
+    # Process results
+    for chunk, relevance_score in results:
+        print(f"Relevance: {relevance_score:.3f}")
+        print(f"Content: {chunk.content}")
+        print(f"From document: {chunk.document_id}")
+```

haiku_rag-0.1.0.dist-info/RECORD

@@ -0,0 +1,27 @@
+haiku/rag/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+haiku/rag/app.py,sha256=jJb5THgH3nbh2K8uiYsMVlkqVbSkIGEyxPMISM3epMA,4546
+haiku/rag/chunker.py,sha256=lSSPWgNAe7gNZL_yNLmDtqxJix4YclOiG7gbARcEpV8,1871
+haiku/rag/cli.py,sha256=XOxl7H86La7fB4DvsEJxtNuSfZgOtwqQDmECSaxv4sY,4020
+haiku/rag/client.py,sha256=H5zE-HO8Asxo-_vEcnxFqvQixdiTFTqvNH8EkH7Xo4E,9713
+haiku/rag/config.py,sha256=GxpfUwsQmfzQcknIAPEET_Qu-0WFYtPkHrV3arvNdxM,596
+haiku/rag/mcp.py,sha256=tMN6fNX7ZtAER1R6DL1GkC9HZozTC4HzuQs199p7icI,4551
+haiku/rag/reader.py,sha256=S7-Z72pDvSHedvgt4-RkTOwZadG88Oed9keJ69SVITk,962
+haiku/rag/utils.py,sha256=6xVM6z2OmhzB4FEDlPbMsr_ZBBmCbMQb83nP6E2UdxY,629
+haiku/rag/embeddings/__init__.py,sha256=jOamqhoeFX9J-ThwvVyHGd2s8jqJzA8B6J4sxHGZ39o,1007
+haiku/rag/embeddings/base.py,sha256=PTAWKTU-Q-hXIhbRK1o6pIdpaW7DFdzJXQ0Nzc6VI-w,379
+haiku/rag/embeddings/ollama.py,sha256=i_w7hbh-_ukysco274fLkQuFRgaFq0zIwIs8CNmRcLE,440
+haiku/rag/embeddings/voyageai.py,sha256=MPioqQ0duzjglqvnN_8ftVq11fvBrcpV03p9MMLwflM,533
+haiku/rag/store/__init__.py,sha256=hq0W0DAC7ysqhWSP2M2uHX8cbG6kbr-sWHxhq6qQcY0,103
+haiku/rag/store/engine.py,sha256=BeYZRZ08zaYeeu375ysnAL3tGz4roA3GzP7WRNwznCo,2603
+haiku/rag/store/models/__init__.py,sha256=s0E72zneGlowvZrFWaNxHYjOAUjgWdLxzdYsnvNRVlY,88
+haiku/rag/store/models/chunk.py,sha256=D-fLHXtItXXyClj_KaE1OV-QQ-urDGS7lTE-qv2VHjw,223
+haiku/rag/store/models/document.py,sha256=TVXVY-nQs-1vCORQEs9rA7zOtndeGC4dgCoujLAS054,396
+haiku/rag/store/repositories/__init__.py,sha256=uIBhxjQh-4o3O-ck8b7BQ58qXQTuJdPvrDIHVhY5T1A,263
+haiku/rag/store/repositories/base.py,sha256=cm3VyQXhtxvRfk1uJHpA0fDSxMpYN-mjQmRiDiLsQ68,1008
+haiku/rag/store/repositories/chunk.py,sha256=6zABVlb5zbMQ4s50z9qb53ieHYaiv4CjgxpbsXxs814,14639
+haiku/rag/store/repositories/document.py,sha256=xpWOpjHFbhVwNJ1gpusEKNY6l_Qyibg9y_bdHCwcfpk,7133
+haiku_rag-0.1.0.dist-info/METADATA,sha256=kDmX6IcmvyL8ss4Go30_UDaSBA4TTzpkp6unzcDOgnM,6141
+haiku_rag-0.1.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+haiku_rag-0.1.0.dist-info/entry_points.txt,sha256=G1U3nAkNd5YDYd4v0tuYFbriz0i-JheCsFuT9kIoGCI,48
+haiku_rag-0.1.0.dist-info/licenses/LICENSE,sha256=eXZrWjSk9PwYFNK9yUczl3oPl95Z4V9UXH7bPN46iPo,1065
+haiku_rag-0.1.0.dist-info/RECORD,,

haiku_rag-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.27.0
+Root-Is-Purelib: true
+Tag: py3-none-any

haiku_rag-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+haiku-rag = haiku.rag.cli:cli

haiku_rag-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2025 Yiorgis Gozadinos
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.