haiku.rag 0.10.0__tar.gz → 0.10.2__tar.gz

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: haiku.rag
-Version: 0.10.0
+Version: 0.10.2
 Summary: Agentic Retrieval Augmented Generation (RAG) with LanceDB
 Author-email: Yiorgis Gozadinos <ggozadinos@gmail.com>
 License: MIT
@@ -66,7 +66,8 @@ uv pip install haiku.rag
 
 # Add documents
 haiku-rag add "Your content here"
-haiku-rag add-src document.pdf
+haiku-rag add "Your content here" --meta author=alice --meta topic=notes
+haiku-rag add-src document.pdf --meta source=manual
 
 # Search
 haiku-rag search "query"

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/README.md

@@ -28,7 +28,8 @@ uv pip install haiku.rag
 
 # Add documents
 haiku-rag add "Your content here"
-haiku-rag add-src document.pdf
+haiku-rag add "Your content here" --meta author=alice --meta topic=notes
+haiku-rag add-src document.pdf --meta source=manual
 
 # Search
 haiku-rag search "query"

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/agents.md

@@ -13,7 +13,8 @@ The simple QA agent answers a single question using the knowledge base. It retri
 Key points:
 
 - Uses a single `search_documents` tool to fetch relevant chunks
-- Can be run with or without inline citations in the prompt
+- Can be run with or without inline citations in the prompt (citations prefer
+  document titles when present, otherwise URIs)
 - Returns a plain string answer
 
 Python usage:

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/cli.md

@@ -27,12 +27,22 @@ haiku-rag list
 From text:
 ```bash
 haiku-rag add "Your document content here"
+
+# Attach metadata (repeat --meta for multiple entries)
+haiku-rag add "Your document content here" --meta author=alice --meta topic=notes
 ```
 
 From file or URL:
 ```bash
 haiku-rag add-src /path/to/document.pdf
 haiku-rag add-src https://example.com/article.html
+
+# Optionally set a human‑readable title stored in the DB schema
+haiku-rag add-src /mnt/data/doc1.pdf --title "Q3 Financial Report"
+
+# Optionally attach metadata (repeat --meta). Values use JSON parsing if possible:
+# numbers, booleans, null, arrays/objects; otherwise kept as strings.
+haiku-rag add-src /mnt/data/doc1.pdf --meta source=manual --meta page_count=12 --meta published=true
 ```
 
 !!! note
@@ -83,6 +93,7 @@ haiku-rag ask "Who is the author of haiku.rag?" --cite
 ```
 
 The QA agent will search your documents for relevant information and provide a comprehensive answer. With `--cite`, responses include citations showing which documents were used.
+When available, citations use the document title; otherwise they fall back to the URI.
 
 ## Research
 
@@ -111,9 +122,6 @@ haiku-rag serve
 
 # stdio transport
 haiku-rag serve --stdio
-
-# SSE transport
-haiku-rag serve --sse
 ```
 
 ## Settings
@@ -125,6 +133,26 @@ haiku-rag settings
 
 ## Maintenance
 
+### Info (Read-only)
+
+Display database metadata without upgrading or modifying it:
+
+```bash
+haiku-rag info [--db /path/to/your.lancedb]
+```
+
+Shows:
+- path to the database
+- stored haiku.rag version (from settings)
+- embeddings provider/model and vector dimension
+- number of documents
+- table versions per table (documents, chunks)
+
+At the end, a separate “Versions” section lists runtime package versions:
+- haiku.rag
+- lancedb
+- docling
+
 ### Vacuum (Optimize and Cleanup)
 
 Reduce disk usage by optimizing and pruning old table versions across all tables:
@@ -142,6 +170,18 @@ when want to switch embeddings provider or model:
 haiku-rag rebuild
 ```
 
+### Download Models
+
+Download required runtime models:
+
+```bash
+haiku-rag download-models
+```
+
+This command:
+- Downloads Docling OCR/conversion models (no-op if already present).
+- Pulls Ollama models referenced in your configuration (embeddings, QA, research, rerank).
+
 ## Migration
 
 ### Migrate from SQLite to LanceDB

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/configuration.md

@@ -211,6 +211,16 @@ Authentication is handled through standard cloud provider credentials (AWS CLI,
 
 **Note:** Table optimization is automatically handled by LanceDB Cloud (`db://` URIs) and is disabled for better performance. For object storage backends (S3, Azure, GCS), optimization is still performed locally.
 
+#### Disable database auto-creation
+
+By default, haiku.rag creates the local LanceDB directory and required tables on first use. To prevent accidental database creation and fail fast if a database hasn’t been set up yet, set:
+
+```bash
+DISABLE_DB_AUTOCREATE=true
+```
+
+When enabled, for local paths, haiku.rag errors if the LanceDB directory does not exist, and it will not create parent directories.
+
 ### Document Processing
 
 ```bash

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/index.md

@@ -15,6 +15,7 @@
 - **Extended file format support**: Parse 40+ file formats including PDF, DOCX, HTML, Markdown, code files and more. Or add a URL!
 - **MCP server**: Exposes functionality as MCP tools
 - **CLI commands**: Access all functionality from your terminal
+  - Add sources from text, files, or URLs, optionally with a human‑readable title
 - **Python client**: Call `haiku.rag` from your own python applications
 
 ## Quick Start
@@ -42,6 +43,8 @@ async with HaikuRAG("database.lancedb") as client:
 Or use the CLI:
 ```bash
 haiku-rag add "Your document content"
+haiku-rag add "Your document content" --meta author=alice
+haiku-rag add-src /path/to/document.pdf --title "Q3 Financial Report" --meta source=manual
 haiku-rag search "query"
 haiku-rag ask "Who is the author of haiku.rag?"
 haiku-rag migrate old_database.sqlite  # Migrate from SQLite

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/installation.md

@@ -72,3 +72,13 @@ VLLM_RERANK_BASE_URL="http://localhost:8001"
 - Python 3.10+
 - Ollama (for default embeddings)
 - vLLM server (for vLLM provider)
+
+## Pre-download Models (Optional)
+
+You can prefetch all required runtime models before first use:
+
+```bash
+haiku-rag download-models
+```
+
+This will download Docling models and pull any Ollama models referenced by your current configuration.

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/mcp.md

@@ -19,7 +19,7 @@ The MCP server exposes `haiku.rag` as MCP tools for compatible MCP clients.
 
 ## Starting MCP Server
 
-The MCP server starts automatically with the serve command and supports Streamable HTTP, stdio and SSE transports:
+The MCP server starts automatically with the serve command and supports Streamable HTTP and stdio transports:
 
 ```bash
 # Default streamable HTTP transport
@@ -27,7 +27,4 @@ haiku-rag serve
 
 # stdio transport (for Claude Desktop)
 haiku-rag serve --stdio
-
-# SSE transport
-haiku-rag serve --sse
 ```

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/python.md

@@ -23,6 +23,7 @@ From text:
 doc = await client.create_document(
     content="Your document content here",
     uri="doc://example",
+    title="My Example Document",  # optional human‑readable title
     metadata={"source": "manual", "topic": "example"}
 )
 ```
@@ -54,12 +55,16 @@ doc = await client.create_document(
 
 From file:
 ```python
-doc = await client.create_document_from_source("path/to/document.pdf")
+doc = await client.create_document_from_source(
+    "path/to/document.pdf", title="Project Brief"
+)
 ```
 
 From URL:
 ```python
-doc = await client.create_document_from_source("https://example.com/article.html")
+doc = await client.create_document_from_source(
+    "https://example.com/article.html", title="Example Article"
+)
 ```
 
 ### Retrieving Documents
@@ -159,6 +164,7 @@ for chunk, relevance_score in results:
     print(f"Content: {chunk.content}")
     print(f"From document: {chunk.document_id}")
     print(f"Document URI: {chunk.document_uri}")
+    print(f"Document Title: {chunk.document_title}")  # when available
     print(f"Document metadata: {chunk.document_meta}")
 ```
 
@@ -201,7 +207,7 @@ answer = await client.ask("Who is the author of haiku.rag?", cite=True)
 print(answer)
 ```
 
-The QA agent will search your documents for relevant information and use the configured LLM to generate a comprehensive answer. With `cite=True`, responses include citations showing which documents were used as sources.
+The QA agent will search your documents for relevant information and use the configured LLM to generate a comprehensive answer. With `cite=True`, responses include citations showing which documents were used as sources. Citations prefer the document title when present, otherwise they use the URI.
 
 The QA provider and model can be configured via environment variables (see [Configuration](configuration.md)).
 

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/docs/server.md

@@ -11,7 +11,6 @@ haiku-rag serve
 Transport options:
 - Default - Streamable HTTP transport
 - `--stdio` - Standard input/output transport
-- `--sse` - Server-sent events transport
 
 ## File Monitoring
 

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/pyproject.toml

@@ -2,7 +2,7 @@
 
 name = "haiku.rag"
 description = "Agentic Retrieval Augmented Generation (RAG) with LanceDB"
-version = "0.10.0"
+version = "0.10.2"
 authors = [{ name = "Yiorgis Gozadinos", email = "ggozadinos@gmail.com" }]
 license = { text = "MIT" }
 readme = { file = "README.md", content-type = "text/markdown" }

{haiku_rag-0.10.0 → haiku_rag-0.10.2}/src/haiku/rag/app.py

@@ -1,4 +1,6 @@
 import asyncio
+import json
+from importlib.metadata import version as pkg_version
 from pathlib import Path
 
 from rich.console import Console
@@ -25,26 +27,141 @@ class HaikuRAGApp:
         self.db_path = db_path
         self.console = Console()
 
+    async def info(self):
+        """Display read-only information about the database without modifying it."""
+
+        import lancedb
+
+        # Basic: show path
+        self.console.print("[bold]haiku.rag database info[/bold]")
+        self.console.print(
+            f"  [repr.attrib_name]path[/repr.attrib_name]: {self.db_path}"
+        )
+
+        if not self.db_path.exists():
+            self.console.print("[red]Database path does not exist.[/red]")
+            return
+
+        # Connect without going through Store to avoid upgrades/validation writes
+        try:
+            db = lancedb.connect(self.db_path)
+            table_names = set(db.table_names())
+        except Exception as e:
+            self.console.print(f"[red]Failed to open database: {e}[/red]")
+            return
+
+        try:
+            ldb_version = pkg_version("lancedb")
+        except Exception:
+            ldb_version = "unknown"
+        try:
+            hr_version = pkg_version("haiku.rag")
+        except Exception:
+            hr_version = "unknown"
+        try:
+            docling_version = pkg_version("docling")
+        except Exception:
+            docling_version = "unknown"
+
+        # Read settings (if present) to find stored haiku.rag version and embedding config
+        stored_version = "unknown"
+        embed_provider: str | None = None
+        embed_model: str | None = None
+        vector_dim: int | None = None
+
+        if "settings" in table_names:
+            settings_tbl = db.open_table("settings")
+            arrow = settings_tbl.search().where("id = 'settings'").limit(1).to_arrow()
+            rows = arrow.to_pylist() if arrow is not None else []
+            if rows:
+                raw = rows[0].get("settings") or "{}"
+                data = json.loads(raw) if isinstance(raw, str) else (raw or {})
+                stored_version = str(data.get("version", stored_version))
+                embed_provider = data.get("EMBEDDINGS_PROVIDER")
+                embed_model = data.get("EMBEDDINGS_MODEL")
+                vector_dim = (
+                    int(data.get("EMBEDDINGS_VECTOR_DIM"))  # pyright: ignore[reportArgumentType]
+                    if data.get("EMBEDDINGS_VECTOR_DIM") is not None
+                    else None
+                )
+
+        num_docs = 0
+        if "documents" in table_names:
+            docs_tbl = db.open_table("documents")
+            num_docs = int(docs_tbl.count_rows())  # type: ignore[attr-defined]
+
+        # Table versions per table (direct API)
+        doc_versions = (
+            len(list(db.open_table("documents").list_versions()))
+            if "documents" in table_names
+            else 0
+        )
+        chunk_versions = (
+            len(list(db.open_table("chunks").list_versions()))
+            if "chunks" in table_names
+            else 0
+        )
+
+        self.console.print(
+            f"  [repr.attrib_name]haiku.rag version (db)[/repr.attrib_name]: {stored_version}"
+        )
+        if embed_provider or embed_model or vector_dim:
+            provider_part = embed_provider or "unknown"
+            model_part = embed_model or "unknown"
+            dim_part = f"{vector_dim}" if vector_dim is not None else "unknown"
+            self.console.print(
+                "  [repr.attrib_name]embeddings[/repr.attrib_name]: "
+                f"{provider_part}/{model_part} (dim: {dim_part})"
+            )
+        else:
+            self.console.print(
+                "  [repr.attrib_name]embeddings[/repr.attrib_name]: unknown"
+            )
+        self.console.print(
+            f"  [repr.attrib_name]documents[/repr.attrib_name]: {num_docs}"
+        )
+        self.console.print(
+            f"  [repr.attrib_name]versions (documents)[/repr.attrib_name]: {doc_versions}"
+        )
+        self.console.print(
+            f"  [repr.attrib_name]versions (chunks)[/repr.attrib_name]: {chunk_versions}"
+        )
+        self.console.rule()
+        self.console.print("[bold]Versions[/bold]")
+        self.console.print(
+            f"  [repr.attrib_name]haiku.rag[/repr.attrib_name]: {hr_version}"
+        )
+        self.console.print(
+            f"  [repr.attrib_name]lancedb[/repr.attrib_name]: {ldb_version}"
+        )
+        self.console.print(
+            f"  [repr.attrib_name]docling[/repr.attrib_name]: {docling_version}"
+        )
+
     async def list_documents(self):
         async with HaikuRAG(db_path=self.db_path) as self.client:
             documents = await self.client.list_documents()
             for doc in documents:
                 self._rich_print_document(doc, truncate=True)
 
-    async def add_document_from_text(self, text: str):
+    async def add_document_from_text(self, text: str, metadata: dict | None = None):
         async with HaikuRAG(db_path=self.db_path) as self.client:
-            doc = await self.client.create_document(text)
+            doc = await self.client.create_document(text, metadata=metadata)
             self._rich_print_document(doc, truncate=True)
             self.console.print(
-                f"[b]Document with id [cyan]{doc.id}[/cyan] added successfully.[/b]"
+                f"[bold green]Document {doc.id} added successfully.[/bold green]"
             )
 
-    async def add_document_from_source(self, source: str):
+    async def add_document_from_source(
+        self, source: str, title: str | None = None, metadata: dict | None = None
+    ):
         async with HaikuRAG(db_path=self.db_path) as self.client:
-            doc = await self.client.create_document_from_source(source)
+            doc = await self.client.create_document_from_source(
+                source, title=title, metadata=metadata
+            )
             self._rich_print_document(doc, truncate=True)
             self.console.print(
-                f"[b]Document with id [cyan]{doc.id}[/cyan] added successfully.[/b]"
+                f"[bold green]Document {doc.id} added successfully.[/bold green]"
             )
 
     async def get_document(self, doc_id: str):
@@ -59,7 +176,9 @@ class HaikuRAGApp:
         async with HaikuRAG(db_path=self.db_path) as self.client:
             deleted = await self.client.delete_document(doc_id)
             if deleted:
-                self.console.print(f"[b]Document {doc_id} deleted successfully.[/b]")
+                self.console.print(
+                    f"[bold green]Document {doc_id} deleted successfully.[/bold green]"
+                )
             else:
                 self.console.print(
                     f"[yellow]Document with id {doc_id} not found.[/yellow]"
@@ -69,7 +188,7 @@ class HaikuRAGApp:
         async with HaikuRAG(db_path=self.db_path) as self.client:
             results = await self.client.search(query, limit=limit)
             if not results:
-                self.console.print("[red]No results found.[/red]")
+                self.console.print("[yellow]No results found.[/yellow]")
                 return
             for chunk, score in results:
                 self._rich_print_search_result(chunk, score)
@@ -202,14 +321,16 @@ class HaikuRAGApp:
                     return
 
                 self.console.print(
-                    f"[b]Rebuilding database with {total_docs} documents...[/b]"
+                    f"[bold cyan]Rebuilding database with {total_docs} documents...[/bold cyan]"
                 )
                 with Progress() as progress:
                     task = progress.add_task("Rebuilding...", total=total_docs)
                     async for _ in client.rebuild_database():
                         progress.update(task, advance=1)
 
-                self.console.print("[b]Database rebuild completed successfully.[/b]")
+                self.console.print(
+                    "[bold green]Database rebuild completed successfully.[/bold green]"
+                )
             except Exception as e:
                 self.console.print(f"[red]Error rebuilding database: {e}[/red]")
 
@@ -218,7 +339,9 @@ class HaikuRAGApp:
         try:
             async with HaikuRAG(db_path=self.db_path, skip_validation=True) as client:
                 await client.vacuum()
-            self.console.print("[b]Vacuum completed successfully.[/b]")
+            self.console.print(
+                "[bold green]Vacuum completed successfully.[/bold green]"
+            )
         except Exception as e:
             self.console.print(f"[red]Error during vacuum: {e}[/red]")
 
@@ -240,7 +363,9 @@ class HaikuRAGApp:
             else:
                 display_value = field_value
 
-            self.console.print(f"  [cyan]{field_name}[/cyan]: {display_value}")
+            self.console.print(
+                f"  [repr.attrib_name]{field_name}[/repr.attrib_name]: {display_value}"
+            )
 
     def _rich_print_document(self, doc: Document, truncate: bool = False):
         """Format a document for display."""
@@ -252,8 +377,16 @@ class HaikuRAGApp:
             content = Markdown(content)
         else:
             content = Markdown(doc.content)
+        title_part = (
+            f" [repr.attrib_name]title[/repr.attrib_name]: {doc.title}"
+            if doc.title
+            else ""
+        )
         self.console.print(
-            f"[repr.attrib_name]id[/repr.attrib_name]: {doc.id} [repr.attrib_name]uri[/repr.attrib_name]: {doc.uri} [repr.attrib_name]meta[/repr.attrib_name]: {doc.metadata}"
+            f"[repr.attrib_name]id[/repr.attrib_name]: {doc.id} "
+            f"[repr.attrib_name]uri[/repr.attrib_name]: {doc.uri}"
+            + title_part
+            + f" [repr.attrib_name]meta[/repr.attrib_name]: {doc.metadata}"
         )
         self.console.print(
             f"[repr.attrib_name]created at[/repr.attrib_name]: {doc.created_at} [repr.attrib_name]updated at[/repr.attrib_name]: {doc.updated_at}"
@@ -272,6 +405,9 @@ class HaikuRAGApp:
         if chunk.document_uri:
             self.console.print("[repr.attrib_name]document uri[/repr.attrib_name]:")
             self.console.print(chunk.document_uri)
+        if chunk.document_title:
+            self.console.print("[repr.attrib_name]document title[/repr.attrib_name]:")
+            self.console.print(chunk.document_title)
         if chunk.document_meta:
             self.console.print("[repr.attrib_name]document meta[/repr.attrib_name]:")
             self.console.print(chunk.document_meta)
@@ -289,8 +425,6 @@ class HaikuRAGApp:
             try:
                 if transport == "stdio":
                     await server.run_stdio_async()
-                elif transport == "sse":
-                    await server.run_sse_async()
                 else:
                     await server.run_http_async(transport="streamable-http")
             except KeyboardInterrupt: