haiku.rag 0.10.0__tar.gz → 0.10.1__tar.gz

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: haiku.rag
-Version: 0.10.0
+Version: 0.10.1
 Summary: Agentic Retrieval Augmented Generation (RAG) with LanceDB
 Author-email: Yiorgis Gozadinos <ggozadinos@gmail.com>
 License: MIT

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/docs/agents.md

@@ -13,7 +13,8 @@ The simple QA agent answers a single question using the knowledge base. It retri
 Key points:
 
 - Uses a single `search_documents` tool to fetch relevant chunks
-- Can be run with or without inline citations in the prompt
+- Can be run with or without inline citations in the prompt (citations prefer
+  document titles when present, otherwise URIs)
 - Returns a plain string answer
 
 Python usage:

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/docs/cli.md

@@ -33,6 +33,9 @@ From file or URL:
 ```bash
 haiku-rag add-src /path/to/document.pdf
 haiku-rag add-src https://example.com/article.html
+
+# Optionally set a human‑readable title stored in the DB schema
+haiku-rag add-src /mnt/data/doc1.pdf --title "Q3 Financial Report"
 ```
 
 !!! note
@@ -83,6 +86,7 @@ haiku-rag ask "Who is the author of haiku.rag?" --cite
 ```
 
 The QA agent will search your documents for relevant information and provide a comprehensive answer. With `--cite`, responses include citations showing which documents were used.
+When available, citations use the document title; otherwise they fall back to the URI.
 
 ## Research
 
@@ -111,9 +115,6 @@ haiku-rag serve
 
 # stdio transport
 haiku-rag serve --stdio
-
-# SSE transport
-haiku-rag serve --sse
 ```
 
 ## Settings

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/docs/configuration.md

@@ -211,6 +211,16 @@ Authentication is handled through standard cloud provider credentials (AWS CLI,
 
 **Note:** Table optimization is automatically handled by LanceDB Cloud (`db://` URIs) and is disabled for better performance. For object storage backends (S3, Azure, GCS), optimization is still performed locally.
 
+#### Disable database auto-creation
+
+By default, haiku.rag creates the local LanceDB directory and required tables on first use. To prevent accidental database creation and fail fast if a database hasn’t been set up yet, set:
+
+```bash
+DISABLE_DB_AUTOCREATE=true
+```
+
+When enabled, for local paths, haiku.rag errors if the LanceDB directory does not exist, and it will not create parent directories.
+
 ### Document Processing
 
 ```bash

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/docs/index.md

@@ -15,6 +15,7 @@
 - **Extended file format support**: Parse 40+ file formats including PDF, DOCX, HTML, Markdown, code files and more. Or add a URL!
 - **MCP server**: Exposes functionality as MCP tools
 - **CLI commands**: Access all functionality from your terminal
+  - Add sources from text, files, or URLs, optionally with a human‑readable title
 - **Python client**: Call `haiku.rag` from your own python applications
 
 ## Quick Start
@@ -42,6 +43,7 @@ async with HaikuRAG("database.lancedb") as client:
 Or use the CLI:
 ```bash
 haiku-rag add "Your document content"
+haiku-rag add-src /path/to/document.pdf --title "Q3 Financial Report"
 haiku-rag search "query"
 haiku-rag ask "Who is the author of haiku.rag?"
 haiku-rag migrate old_database.sqlite  # Migrate from SQLite

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/docs/mcp.md

@@ -19,7 +19,7 @@ The MCP server exposes `haiku.rag` as MCP tools for compatible MCP clients.
 
 ## Starting MCP Server
 
-The MCP server starts automatically with the serve command and supports Streamable HTTP, stdio and SSE transports:
+The MCP server starts automatically with the serve command and supports Streamable HTTP and stdio transports:
 
 ```bash
 # Default streamable HTTP transport
@@ -27,7 +27,4 @@ haiku-rag serve
 
 # stdio transport (for Claude Desktop)
 haiku-rag serve --stdio
-
-# SSE transport
-haiku-rag serve --sse
 ```

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/docs/python.md

@@ -23,6 +23,7 @@ From text:
 doc = await client.create_document(
     content="Your document content here",
     uri="doc://example",
+    title="My Example Document",  # optional human‑readable title
     metadata={"source": "manual", "topic": "example"}
 )
 ```
@@ -54,12 +55,16 @@ doc = await client.create_document(
 
 From file:
 ```python
-doc = await client.create_document_from_source("path/to/document.pdf")
+doc = await client.create_document_from_source(
+    "path/to/document.pdf", title="Project Brief"
+)
 ```
 
 From URL:
 ```python
-doc = await client.create_document_from_source("https://example.com/article.html")
+doc = await client.create_document_from_source(
+    "https://example.com/article.html", title="Example Article"
+)
 ```
 
 ### Retrieving Documents
@@ -159,6 +164,7 @@ for chunk, relevance_score in results:
     print(f"Content: {chunk.content}")
     print(f"From document: {chunk.document_id}")
     print(f"Document URI: {chunk.document_uri}")
+    print(f"Document Title: {chunk.document_title}")  # when available
     print(f"Document metadata: {chunk.document_meta}")
 ```
 
@@ -201,7 +207,7 @@ answer = await client.ask("Who is the author of haiku.rag?", cite=True)
 print(answer)
 ```
 
-The QA agent will search your documents for relevant information and use the configured LLM to generate a comprehensive answer. With `cite=True`, responses include citations showing which documents were used as sources.
+The QA agent will search your documents for relevant information and use the configured LLM to generate a comprehensive answer. With `cite=True`, responses include citations showing which documents were used as sources. Citations prefer the document title when present, otherwise they use the URI.
 
 The QA provider and model can be configured via environment variables (see [Configuration](configuration.md)).
 

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/docs/server.md

@@ -11,7 +11,6 @@ haiku-rag serve
 Transport options:
 - Default - Streamable HTTP transport
 - `--stdio` - Standard input/output transport
-- `--sse` - Server-sent events transport
 
 ## File Monitoring
 

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/pyproject.toml

@@ -2,7 +2,7 @@
 
 name = "haiku.rag"
 description = "Agentic Retrieval Augmented Generation (RAG) with LanceDB"
-version = "0.10.0"
+version = "0.10.1"
 authors = [{ name = "Yiorgis Gozadinos", email = "ggozadinos@gmail.com" }]
 license = { text = "MIT" }
 readme = { file = "README.md", content-type = "text/markdown" }

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/src/haiku/rag/app.py

@@ -39,9 +39,9 @@ class HaikuRAGApp:
                 f"[b]Document with id [cyan]{doc.id}[/cyan] added successfully.[/b]"
             )
 
-    async def add_document_from_source(self, source: str):
+    async def add_document_from_source(self, source: str, title: str | None = None):
         async with HaikuRAG(db_path=self.db_path) as self.client:
-            doc = await self.client.create_document_from_source(source)
+            doc = await self.client.create_document_from_source(source, title=title)
             self._rich_print_document(doc, truncate=True)
             self.console.print(
                 f"[b]Document with id [cyan]{doc.id}[/cyan] added successfully.[/b]"
@@ -252,8 +252,16 @@ class HaikuRAGApp:
             content = Markdown(content)
         else:
             content = Markdown(doc.content)
+        title_part = (
+            f" [repr.attrib_name]title[/repr.attrib_name]: {doc.title}"
+            if doc.title
+            else ""
+        )
         self.console.print(
-            f"[repr.attrib_name]id[/repr.attrib_name]: {doc.id} [repr.attrib_name]uri[/repr.attrib_name]: {doc.uri} [repr.attrib_name]meta[/repr.attrib_name]: {doc.metadata}"
+            f"[repr.attrib_name]id[/repr.attrib_name]: {doc.id} "
+            f"[repr.attrib_name]uri[/repr.attrib_name]: {doc.uri}"
+            + title_part
+            + f" [repr.attrib_name]meta[/repr.attrib_name]: {doc.metadata}"
         )
         self.console.print(
             f"[repr.attrib_name]created at[/repr.attrib_name]: {doc.created_at} [repr.attrib_name]updated at[/repr.attrib_name]: {doc.updated_at}"
@@ -272,6 +280,9 @@ class HaikuRAGApp:
         if chunk.document_uri:
             self.console.print("[repr.attrib_name]document uri[/repr.attrib_name]:")
             self.console.print(chunk.document_uri)
+        if chunk.document_title:
+            self.console.print("[repr.attrib_name]document title[/repr.attrib_name]:")
+            self.console.print(chunk.document_title)
         if chunk.document_meta:
             self.console.print("[repr.attrib_name]document meta[/repr.attrib_name]:")
             self.console.print(chunk.document_meta)
@@ -289,8 +300,6 @@ class HaikuRAGApp:
             try:
                 if transport == "stdio":
                     await server.run_stdio_async()
-                elif transport == "sse":
-                    await server.run_sse_async()
                 else:
                     await server.run_http_async(transport="streamable-http")
             except KeyboardInterrupt:

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/src/haiku/rag/cli.py

@@ -3,28 +3,16 @@ import warnings
 from importlib.metadata import version
 from pathlib import Path
 
-import logfire
 import typer
-from rich.console import Console
 
-from haiku.rag.app import HaikuRAGApp
 from haiku.rag.config import Config
 from haiku.rag.logging import configure_cli_logging
-from haiku.rag.migration import migrate_sqlite_to_lancedb
 from haiku.rag.utils import is_up_to_date
 
-if Config.ENV == "development":
-    logfire.configure(send_to_logfire="if-token-present")
-    logfire.instrument_pydantic_ai()
-else:
-    warnings.filterwarnings("ignore")
-
 cli = typer.Typer(
     context_settings={"help_option_names": ["-h", "--help"]}, no_args_is_help=True
 )
 
-console = Console()
-
 
 def complete_document_ids(ctx: typer.Context, incomplete: str):
     """Autocomplete document IDs from the selected DB."""
@@ -89,16 +77,16 @@ async def check_version():
     """Check if haiku.rag is up to date and show warning if not."""
     up_to_date, current_version, latest_version = await is_up_to_date()
     if not up_to_date:
-        console.print(
-            f"[yellow]Warning: haiku.rag is outdated. Current: {current_version}, Latest: {latest_version}[/yellow]"
+        typer.echo(
+            f"Warning: haiku.rag is outdated. Current: {current_version}, Latest: {latest_version}",
         )
-        console.print("[yellow]Please update.[/yellow]")
+        typer.echo("Please update.")
 
 
 def version_callback(value: bool):
     if value:
         v = version("haiku.rag")
-        console.print(f"haiku.rag version {v}")
+        typer.echo(f"haiku.rag version {v}")
         raise typer.Exit()
 
 
@@ -113,10 +101,26 @@ def main(
     ),
 ):
     """haiku.rag CLI - Vector database RAG system"""
-    # Ensure only haiku.rag logs are emitted in CLI context
-    configure_cli_logging()
+    # Configure logging minimally for CLI context
+    if Config.ENV == "development":
+        # Lazy import logfire only in development
+        try:
+            import logfire  # type: ignore
+
+            logfire.configure(send_to_logfire="if-token-present")
+            logfire.instrument_pydantic_ai()
+        except Exception:
+            pass
+    else:
+        configure_cli_logging()
+        warnings.filterwarnings("ignore")
+
     # Run version check before any command
-    asyncio.run(check_version())
+    try:
+        asyncio.run(check_version())
+    except Exception:
+        # Do not block CLI on version check issues
+        pass
 
 
 @cli.command("list", help="List all stored documents")
@@ -127,6 +131,8 @@ def list_documents(
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.list_documents())
 
@@ -142,6 +148,8 @@ def add_document_text(
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.add_document_from_text(text=text))
 
@@ -152,14 +160,21 @@ def add_document_src(
         help="The file path or URL of the document to add",
         autocompletion=complete_local_paths,
     ),
+    title: str | None = typer.Option(
+        None,
+        "--title",
+        help="Optional human-readable title to store with the document",
+    ),
     db: Path = typer.Option(
         Config.DEFAULT_DATA_DIR / "haiku.rag.lancedb",
         "--db",
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
-    asyncio.run(app.add_document_from_source(source=source))
+    asyncio.run(app.add_document_from_source(source=source, title=title))
 
 
 @cli.command("get", help="Get and display a document by its ID")
@@ -174,6 +189,8 @@ def get_document(
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.get_document(doc_id=doc_id))
 
@@ -190,6 +207,8 @@ def delete_document(
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.delete_document(doc_id=doc_id))
 
@@ -215,6 +234,8 @@ def search(
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.search(query=query, limit=limit))
 
@@ -235,6 +256,8 @@ def ask(
         help="Include citations in the response",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.ask(question=question, cite=cite))
 
@@ -271,6 +294,8 @@ def research(
         help="Show verbose progress output",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(
         app.research(
@@ -285,6 +310,8 @@ def research(
 
 @cli.command("settings", help="Display current configuration settings")
 def settings():
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=Path())  # Don't need actual DB for settings
     app.show_settings()
 
@@ -300,6 +327,8 @@ def rebuild(
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.rebuild())
 
@@ -312,6 +341,8 @@ def vacuum(
         help="Path to the LanceDB database file",
     ),
 ):
+    from haiku.rag.app import HaikuRAGApp
+
     app = HaikuRAGApp(db_path=db)
     asyncio.run(app.vacuum())
 
@@ -330,24 +361,15 @@ def serve(
         "--stdio",
         help="Run MCP server on stdio Transport",
     ),
-    sse: bool = typer.Option(
-        False,
-        "--sse",
-        help="Run MCP server on SSE transport",
-    ),
 ) -> None:
     """Start the MCP server."""
-    if stdio and sse:
-        console.print("[red]Error: Cannot use both --stdio and --http options[/red]")
-        raise typer.Exit(1)
+    from haiku.rag.app import HaikuRAGApp
 
     app = HaikuRAGApp(db_path=db)
 
     transport = None
     if stdio:
         transport = "stdio"
-    elif sse:
-        transport = "sse"
 
     asyncio.run(app.serve(transport=transport))
 
@@ -361,6 +383,9 @@ def migrate(
     # Generate LanceDB path in same parent directory
     lancedb_path = sqlite_path.parent / (sqlite_path.stem + ".lancedb")
 
+    # Lazy import to avoid heavy deps on simple invocations
+    from haiku.rag.migration import migrate_sqlite_to_lancedb
+
     success = asyncio.run(migrate_sqlite_to_lancedb(sqlite_path, lancedb_path))
 
     if not success:

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/src/haiku/rag/client.py

@@ -33,8 +33,6 @@ class HaikuRAG:
             db_path: Path to the database file.
             skip_validation: Whether to skip configuration validation on database load.
         """
-        if not db_path.parent.exists():
-            Path.mkdir(db_path.parent, parents=True)
         self.store = Store(db_path, skip_validation=skip_validation)
         self.document_repository = DocumentRepository(self.store)
         self.chunk_repository = ChunkRepository(self.store)
@@ -52,6 +50,7 @@ class HaikuRAG:
         self,
         docling_document,
         uri: str | None = None,
+        title: str | None = None,
         metadata: dict | None = None,
         chunks: list[Chunk] | None = None,
     ) -> Document:
@@ -60,6 +59,7 @@ class HaikuRAG:
         document = Document(
             content=content,
             uri=uri,
+            title=title,
             metadata=metadata or {},
         )
         return await self.document_repository._create_with_docling(
@@ -70,6 +70,7 @@ class HaikuRAG:
         self,
         content: str,
         uri: str | None = None,
+        title: str | None = None,
         metadata: dict | None = None,
         chunks: list[Chunk] | None = None,
     ) -> Document:
@@ -90,6 +91,7 @@ class HaikuRAG:
         document = Document(
             content=content,
             uri=uri,
+            title=title,
             metadata=metadata or {},
         )
         return await self.document_repository._create_with_docling(
@@ -97,7 +99,7 @@ class HaikuRAG:
         )
 
     async def create_document_from_source(
-        self, source: str | Path, metadata: dict = {}
+        self, source: str | Path, title: str | None = None, metadata: dict | None = None
     ) -> Document:
         """Create or update a document from a file path or URL.
 
@@ -118,11 +120,16 @@ class HaikuRAG:
             httpx.RequestError: If URL request fails
         """
 
+        # Normalize metadata
+        metadata = metadata or {}
+
         # Check if it's a URL
         source_str = str(source)
         parsed_url = urlparse(source_str)
         if parsed_url.scheme in ("http", "https"):
-            return await self._create_or_update_document_from_url(source_str, metadata)
+            return await self._create_or_update_document_from_url(
+                source_str, title=title, metadata=metadata
+            )
         elif parsed_url.scheme == "file":
             # Handle file:// URI by converting to path
             source_path = Path(parsed_url.path)
@@ -138,37 +145,51 @@ class HaikuRAG:
         uri = source_path.absolute().as_uri()
         md5_hash = hashlib.md5(source_path.read_bytes()).hexdigest()
 
+        # Get content type from file extension (do before early return)
+        content_type, _ = mimetypes.guess_type(str(source_path))
+        if not content_type:
+            content_type = "application/octet-stream"
+        # Merge metadata with contentType and md5
+        metadata.update({"contentType": content_type, "md5": md5_hash})
+
         # Check if document already exists
         existing_doc = await self.get_document_by_uri(uri)
         if existing_doc and existing_doc.metadata.get("md5") == md5_hash:
-            # MD5 unchanged, return existing document
+            # MD5 unchanged; update title/metadata if provided
+            updated = False
+            if title is not None and title != existing_doc.title:
+                existing_doc.title = title
+                updated = True
+            if metadata:
+                existing_doc.metadata = {**(existing_doc.metadata or {}), **metadata}
+                updated = True
+            if updated:
+                return await self.document_repository.update(existing_doc)
             return existing_doc
 
+        # Parse file only when content changed or new document
         docling_document = FileReader.parse_file(source_path)
 
-        # Get content type from file extension
-        content_type, _ = mimetypes.guess_type(str(source_path))
-        if not content_type:
-            content_type = "application/octet-stream"
-
-        # Merge metadata with contentType and md5
-        metadata.update({"contentType": content_type, "md5": md5_hash})
-
         if existing_doc:
             # Update existing document
             existing_doc.content = docling_document.export_to_markdown()
             existing_doc.metadata = metadata
+            if title is not None:
+                existing_doc.title = title
             return await self.document_repository._update_with_docling(
                 existing_doc, docling_document
             )
         else:
             # Create new document using DoclingDocument
             return await self._create_document_with_docling(
-                docling_document=docling_document, uri=uri, metadata=metadata
+                docling_document=docling_document,
+                uri=uri,
+                title=title,
+                metadata=metadata,
             )
 
     async def _create_or_update_document_from_url(
-        self, url: str, metadata: dict = {}
+        self, url: str, title: str | None = None, metadata: dict | None = None
     ) -> Document:
         """Create or update a document from a URL by downloading and parsing the content.
 
@@ -188,20 +209,35 @@ class HaikuRAG:
             ValueError: If the content cannot be parsed
             httpx.RequestError: If URL request fails
         """
+        metadata = metadata or {}
+
         async with httpx.AsyncClient() as client:
             response = await client.get(url)
             response.raise_for_status()
 
             md5_hash = hashlib.md5(response.content).hexdigest()
 
+            # Get content type early (used for potential no-op update)
+            content_type = response.headers.get("content-type", "").lower()
+
             # Check if document already exists
             existing_doc = await self.get_document_by_uri(url)
             if existing_doc and existing_doc.metadata.get("md5") == md5_hash:
-                # MD5 unchanged, return existing document
+                # MD5 unchanged; update title/metadata if provided
+                updated = False
+                if title is not None and title != existing_doc.title:
+                    existing_doc.title = title
+                    updated = True
+                metadata.update({"contentType": content_type, "md5": md5_hash})
+                if metadata:
+                    existing_doc.metadata = {
+                        **(existing_doc.metadata or {}),
+                        **metadata,
+                    }
+                    updated = True
+                if updated:
+                    return await self.document_repository.update(existing_doc)
                 return existing_doc
-
-            # Get content type to determine file extension
-            content_type = response.headers.get("content-type", "").lower()
             file_extension = self._get_extension_from_content_type_or_url(
                 url, content_type
             )
@@ -228,12 +264,17 @@ class HaikuRAG:
             if existing_doc:
                 existing_doc.content = docling_document.export_to_markdown()
                 existing_doc.metadata = metadata
+                if title is not None:
+                    existing_doc.title = title
                 return await self.document_repository._update_with_docling(
                     existing_doc, docling_document
                 )
             else:
                 return await self._create_document_with_docling(
-                    docling_document=docling_document, uri=url, metadata=metadata
+                    docling_document=docling_document,
+                    uri=url,
+                    title=title,
+                    metadata=metadata,
                 )
 
     def _get_extension_from_content_type_or_url(
@@ -418,6 +459,7 @@ class HaikuRAG:
                     content="".join(combined_content_parts),
                     metadata=original_chunk.metadata,
                     document_uri=original_chunk.document_uri,
+                    document_title=original_chunk.document_title,
                     document_meta=original_chunk.document_meta,
                 )
 
@@ -524,7 +566,7 @@ class HaikuRAG:
 
                     # Try to re-create from source (this creates the document with chunks)
                     new_doc = await self.create_document_from_source(
-                        doc.uri, doc.metadata or {}
+                        source=doc.uri, metadata=doc.metadata or {}
                     )
 
                     assert new_doc.id is not None, "New document ID should not be None"

{haiku_rag-0.10.0 → haiku_rag-0.10.1}/src/haiku/rag/config.py

@@ -53,6 +53,10 @@ class AppConfig(BaseModel):
     ANTHROPIC_API_KEY: str = ""
     COHERE_API_KEY: str = ""
 
+    # If true, refuse to auto-create a new LanceDB database or tables
+    # and error out when the database does not already exist.
+    DISABLE_DB_AUTOCREATE: bool = False
+
     @field_validator("MONITOR_DIRECTORIES", mode="before")
     @classmethod
     def parse_monitor_directories(cls, v):