OpenRAG 0.2.5__tar.gz → 0.3.1__tar.gz

{openrag-0.2.5/src/openrag.egg-info → openrag-0.3.1}/PKG-INFO

@@ -1,7 +1,17 @@
 Metadata-Version: 2.4
 Name: openrag
-Version: 0.2.5
+Version: 0.3.1
 Summary: OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations.
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.13
 Description-Content-Type: text/markdown
 License-File: LICENSE
@@ -30,7 +40,7 @@ Dynamic: license-file
 
 <div align="center">
 
-<img src="./docs/static/img/openrag-logo-dog.svg" alt="OpenRAG Logo" width="120"/>
+<img src="./docs/static/img/openrag-logo-dog.svg" alt="" width="120"/>
 
 # OpenRAG
 
@@ -39,7 +49,7 @@ Dynamic: license-file
 </h3>
 
 <!-- Badges -->
-  
+
 [![Langflow](https://img.shields.io/badge/Langflow-1C1C1E?style=for-the-badge&logo=langflow)](https://github.com/langflow-ai/langflow)
 [![OpenSearch](https://img.shields.io/badge/OpenSearch-005EB8?style=for-the-badge&logo=opensearch&logoColor=white)](https://github.com/opensearch-project/OpenSearch)
 [![Docling](https://img.shields.io/badge/Docling-000000?style=for-the-badge)](https://github.com/docling-project/docling)
@@ -48,7 +58,7 @@ Dynamic: license-file
 [![GitHub stars](https://img.shields.io/github/stars/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/stargazers)
 [![GitHub forks](https://img.shields.io/github/forks/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/network/members)
 
-[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langflow-ai/openrag)
+[![Documentation](https://img.shields.io/badge/Documentation-773eff)](https://docs.openr.ag) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langflow-ai/openrag)
 
 </div>
 
@@ -56,9 +66,11 @@ Dynamic: license-file
 
 OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations.
 
-Users can upload, process, and query documents through a chat interface backed by large language models and semantic search capabilities. The system utilizes Langflow for document ingestion, retrieval workflows, and intelligent nudges, providing a seamless RAG experience. 
+Users can upload, process, and query documents through a chat interface backed by large language models and semantic search capabilities. The system utilizes Langflow for document ingestion, retrieval workflows, and intelligent nudges, providing a seamless RAG experience.
+
+Check out the [documentation](https://docs.openr.ag/) or get started with the [quickstart](https://docs.openr.ag/quickstart).
 
-Built with [Starlette](https://github.com/Kludex/starlette) and [Next.js](https://github.com/vercel/next.js). 
+Built with [FastAPI](https://fastapi.tiangolo.com/) and [Next.js](https://github.com/vercel/next.js). 
 Powered by [OpenSearch](https://github.com/opensearch-project/OpenSearch), [Langflow](https://github.com/langflow-ai/langflow), and [Docling](https://github.com/docling-project/docling).
 
 ---
@@ -189,7 +201,7 @@ The MCP server provides tools for RAG-enhanced chat, semantic search, and settin
 
 For developers who want to [contribute to OpenRAG](https://docs.openr.ag/support/contribute) or set up a development environment, see [CONTRIBUTING.md](CONTRIBUTING.md).
 
-## 🆘 Troubleshooting
+## 🛟 Troubleshooting
 
 For assistance with OpenRAG, see [Troubleshoot OpenRAG](https://docs.openr.ag/support/troubleshoot) and visit the [Discussions page](https://github.com/langflow-ai/openrag/discussions).
 

{openrag-0.2.5 → openrag-0.3.1}/README.md

@@ -1,6 +1,6 @@
 <div align="center">
 
-<img src="./docs/static/img/openrag-logo-dog.svg" alt="OpenRAG Logo" width="120"/>
+<img src="./docs/static/img/openrag-logo-dog.svg" alt="" width="120"/>
 
 # OpenRAG
 
@@ -9,7 +9,7 @@
 </h3>
 
 <!-- Badges -->
-  
+
 [![Langflow](https://img.shields.io/badge/Langflow-1C1C1E?style=for-the-badge&logo=langflow)](https://github.com/langflow-ai/langflow)
 [![OpenSearch](https://img.shields.io/badge/OpenSearch-005EB8?style=for-the-badge&logo=opensearch&logoColor=white)](https://github.com/opensearch-project/OpenSearch)
 [![Docling](https://img.shields.io/badge/Docling-000000?style=for-the-badge)](https://github.com/docling-project/docling)
@@ -18,7 +18,7 @@
 [![GitHub stars](https://img.shields.io/github/stars/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/stargazers)
 [![GitHub forks](https://img.shields.io/github/forks/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/network/members)
 
-[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langflow-ai/openrag)
+[![Documentation](https://img.shields.io/badge/Documentation-773eff)](https://docs.openr.ag) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langflow-ai/openrag)
 
 </div>
 
@@ -26,9 +26,11 @@
 
 OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations.
 
-Users can upload, process, and query documents through a chat interface backed by large language models and semantic search capabilities. The system utilizes Langflow for document ingestion, retrieval workflows, and intelligent nudges, providing a seamless RAG experience. 
+Users can upload, process, and query documents through a chat interface backed by large language models and semantic search capabilities. The system utilizes Langflow for document ingestion, retrieval workflows, and intelligent nudges, providing a seamless RAG experience.
+
+Check out the [documentation](https://docs.openr.ag/) or get started with the [quickstart](https://docs.openr.ag/quickstart).
 
-Built with [Starlette](https://github.com/Kludex/starlette) and [Next.js](https://github.com/vercel/next.js). 
+Built with [FastAPI](https://fastapi.tiangolo.com/) and [Next.js](https://github.com/vercel/next.js). 
 Powered by [OpenSearch](https://github.com/opensearch-project/OpenSearch), [Langflow](https://github.com/langflow-ai/langflow), and [Docling](https://github.com/docling-project/docling).
 
 ---
@@ -159,7 +161,7 @@ The MCP server provides tools for RAG-enhanced chat, semantic search, and settin
 
 For developers who want to [contribute to OpenRAG](https://docs.openr.ag/support/contribute) or set up a development environment, see [CONTRIBUTING.md](CONTRIBUTING.md).
 
-## 🆘 Troubleshooting
+## 🛟 Troubleshooting
 
 For assistance with OpenRAG, see [Troubleshoot OpenRAG](https://docs.openr.ag/support/troubleshoot) and visit the [Discussions page](https://github.com/langflow-ai/openrag/discussions).
 

{openrag-0.2.5 → openrag-0.3.1}/pyproject.toml

@@ -4,10 +4,22 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "openrag"
-version = "0.2.5"
+version = "0.3.1"
 description = "OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations."
 readme = "README.md"
 requires-python = ">=3.13"
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Environment :: Console",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: Apache Software License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
 dependencies = [
     "agentd>=0.2.2",
     "aiofiles>=24.1.0",

{openrag-0.2.5 → openrag-0.3.1}/src/agent.py

@@ -135,14 +135,13 @@ async def async_response_stream(
 
         full_response = ""
         chunk_count = 0
-        detected_tool_call = False  # Track if we've detected a tool call
+        detected_tool_call = False
         async for chunk in response:
             chunk_count += 1
             logger.debug(
                 "Stream chunk received", chunk_count=chunk_count, chunk=str(chunk)
             )
 
-            # Yield the raw event as JSON for the UI to process
             import json
 
             # Also extract text content for logging
@@ -186,7 +185,7 @@ async def async_response_stream(
                 if isinstance(chunk_data, dict):
                     # Check for any fields that might indicate tool usage
                     potential_tool_fields = {
-                        k: v for k, v in chunk_data.items() 
+                        k: v for k, v in chunk_data.items()
                         if any(keyword in str(k).lower() for keyword in ['tool', 'call', 'retrieval', 'function', 'result', 'output'])
                     }
                     if potential_tool_fields:
@@ -235,8 +234,8 @@ async def async_response_stream(
                                 "tool_name": "Retrieval",
                                 "status": "completed",
                                 "inputs": {"implicit": True, "backend_detected": True},
-                                "results": chunk_data.get('results') or chunk_data.get('outputs') or 
-                                         chunk_data.get('retrieved_documents') or 
+                                "results": chunk_data.get('results') or chunk_data.get('outputs') or
+                                         chunk_data.get('retrieved_documents') or
                                          chunk_data.get('retrieval_results') or []
                             }
                         }
@@ -625,8 +624,25 @@ async def async_langflow_chat(
             message_count=len(conversation_state["messages"]),
         )
 
+    # Extract sources from retrieval tool calls in the response
+    sources = []
+    if hasattr(response_obj, "output") and response_obj.output:
+        for output_item in response_obj.output:
+            item_type = getattr(output_item, "type", None)
+            if item_type in ("tool_call", "retrieval_call"):
+                for result in getattr(output_item, "results", None) or []:
+                    rd = result.model_dump() if hasattr(result, "model_dump") else (result if isinstance(result, dict) else {})
+                    if "text" in rd:
+                        sources.append({
+                            "filename": rd.get("filename", ""),
+                            "text": rd.get("text", ""),
+                            "score": rd.get("score", 0),
+                            "page": rd.get("page"),
+                            "mimetype": rd.get("mimetype"),
+                        })
+
     if not store_conversation:
-        return response_text, response_id
+        return response_text, response_id, sources
 
     # Store the conversation thread with its response_id
     if response_id:
@@ -662,7 +678,7 @@ async def async_langflow_chat(
     else:
         logger.warning("No response_id received from langflow, conversation not stored")
 
-    return response_text, response_id
+    return response_text, response_id, sources
 
 
 # Async langflow function with conversation storage (streaming)
@@ -698,47 +714,55 @@ async def async_langflow_chat_stream(
     response_id = None
     usage_data = None
     collected_chunks = []  # Store all chunks for function call data
+    error_occurred = False
 
-    async for chunk in async_stream(
-        langflow_client,
-        prompt,
-        flow_id,
-        extra_headers=extra_headers,
-        previous_response_id=previous_response_id,
-        log_prefix="langflow",
-    ):
-        # Extract text content to build full response for history
-        try:
-            import json
-
-            chunk_data = json.loads(chunk.decode("utf-8"))
-            collected_chunks.append(chunk_data)  # Collect all chunk data
-
-            if "delta" in chunk_data and "content" in chunk_data["delta"]:
-                full_response += chunk_data["delta"]["content"]
-            # Extract response_id from chunk
-            if "id" in chunk_data:
-                response_id = chunk_data["id"]
-            elif "response_id" in chunk_data:
-                response_id = chunk_data["response_id"]
-            # Capture usage from response.completed event
-            if chunk_data.get("type") == "response.completed":
-                response_obj = chunk_data.get("response", {})
-                usage_data = response_obj.get("usage")
-        except:
-            pass
-        yield chunk
+    try:
+        async for chunk in async_stream(
+            langflow_client,
+            prompt,
+            flow_id,
+            extra_headers=extra_headers,
+            previous_response_id=previous_response_id,
+            log_prefix="langflow",
+        ):
+            # Extract text content to build full response for history
+            try:
+                import json
+
+                chunk_data = json.loads(chunk.decode("utf-8"))
+                collected_chunks.append(chunk_data)  # Collect all chunk data
+
+                if "delta" in chunk_data and "content" in chunk_data["delta"]:
+                    full_response += chunk_data["delta"]["content"]
+                # Extract response_id from chunk
+                if "id" in chunk_data:
+                    response_id = chunk_data["id"]
+                elif "response_id" in chunk_data:
+                    response_id = chunk_data["response_id"]
+                
+                # Check for error status
+                if chunk_data.get("finish_reason") == "error" or chunk_data.get("status") == "failed":
+                    error_occurred = True
+                    logger.error("Error detected in Langflow stream chunk")
+                # Capture usage from response.completed event
+                if chunk_data.get("type") == "response.completed":
+                    response_obj = chunk_data.get("response", {})
+                    usage_data = response_obj.get("usage")
+            except:
+                pass
+            yield chunk
 
-    # Add the complete assistant response to message history with response_id, timestamp, and function call data
-    if full_response:
-        assistant_message = {
-            "role": "assistant",
-            "content": full_response,
-            "response_id": response_id,
-            "timestamp": datetime.now(),
-            "chunks": collected_chunks,  # Store complete chunk data for function calls
-        }
-        # Store usage data if available (from response.completed event)
+        # Add the complete assistant response to message history with response_id, timestamp, and function call data
+        if full_response:
+            assistant_message = {
+                "role": "assistant",
+                "content": full_response,
+                "response_id": response_id,
+                "timestamp": datetime.now(),
+                "chunks": collected_chunks,  # Store complete chunk data for function calls
+                "error": error_occurred,  # Mark if this was an error response
+            }
+            # Store usage data if available (from response.completed event)
         if usage_data:
             assistant_message["response_data"] = {"usage": usage_data}
         conversation_state["messages"].append(assistant_message)
@@ -760,6 +784,34 @@ async def async_langflow_chat_stream(
             logger.debug(
                 f"Stored langflow conversation thread for user {user_id} with response_id: {response_id}"
             )
+    except Exception as e:
+        # Log the error
+        logger.error(f"Error in langflow chat stream: {e}", exc_info=True)
+        error_occurred = True
+        
+        # Store error message in conversation history so it persists
+        error_message = {
+            "role": "assistant",
+            "content": f"Sorry, I encountered an error: {str(e)}",
+            "timestamp": datetime.now(),
+            "error": True,
+        }
+        conversation_state["messages"].append(error_message)
+        
+        # Try to store the conversation with error message
+        # Use a temporary response_id if we don't have one
+        if not response_id:
+            response_id = f"error_{user_id}_{int(datetime.now().timestamp())}"
+        
+        try:
+            conversation_state["last_activity"] = datetime.now()
+            await store_conversation_thread(user_id, response_id, conversation_state)
+            logger.debug(f"Stored conversation with error for user {user_id}")
+        except Exception as store_error:
+            logger.error(f"Failed to store error conversation: {store_error}")
+        
+        # Re-raise the exception so it propagates to the API layer
+        raise
 
 
 async def delete_user_conversation(user_id: str, response_id: str) -> bool:

{openrag-0.2.5 → openrag-0.3.1}/src/api/documents.py

@@ -13,6 +13,92 @@ class DeleteDocumentBody(BaseModel):
     filename: str
 
 
+async def delete_documents_by_filename_core(
+    filename: str,
+    session_manager,
+    user_id: str,
+    jwt_token: str | None,
+):
+    """Shared delete-by-filename logic for v1 and non-v1 endpoints."""
+    from config.settings import get_index_name
+    from utils.opensearch_queries import build_filename_delete_body
+
+    normalized_filename = (filename or "").strip()
+    if not normalized_filename:
+        return (
+            {
+                "success": False,
+                "deleted_chunks": 0,
+                "filename": normalized_filename,
+                "message": None,
+                "error": "Filename is required",
+            },
+            400,
+        )
+
+    try:
+        opensearch_client = session_manager.get_user_opensearch_client(
+            user_id, jwt_token
+        )
+        delete_query = build_filename_delete_body(normalized_filename)
+        result = await opensearch_client.delete_by_query(
+            index=get_index_name(),
+            body=delete_query,
+            conflicts="proceed",
+        )
+
+        deleted_count = result.get("deleted", 0)
+        logger.info(
+            f"Deleted {deleted_count} chunks for filename {normalized_filename}",
+            user_id=user_id,
+        )
+
+        if deleted_count == 0:
+            return (
+                {
+                    "success": False,
+                    "deleted_chunks": 0,
+                    "filename": normalized_filename,
+                    "message": None,
+                    "error": "No matching document chunks were deleted. The file may be missing or not deletable in the current user context.",
+                },
+                404,
+            )
+
+        return (
+            {
+                "success": True,
+                "deleted_chunks": deleted_count,
+                "filename": normalized_filename,
+                "message": f"All documents with filename '{normalized_filename}' deleted successfully",
+                "error": None,
+            },
+            200,
+        )
+    except Exception as e:
+        logger.error(
+            "Error deleting documents by filename",
+            filename=normalized_filename,
+            error=str(e),
+        )
+        error_str = str(e)
+        status_code = 403 if "AuthenticationException" in error_str else 500
+        return (
+            {
+                "success": False,
+                "deleted_chunks": 0,
+                "filename": normalized_filename,
+                "message": None,
+                "error": (
+                    "Access denied: insufficient permissions"
+                    if status_code == 403
+                    else "An internal error has occurred while deleting documents"
+                ),
+            },
+            status_code,
+        )
+
+
 async def _ensure_index_exists():
     """Create the OpenSearch index if it doesn't exist yet."""
     from main import init_index
@@ -70,43 +156,12 @@ async def delete_documents_by_filename(
     body: DeleteDocumentBody,
     session_manager=Depends(get_session_manager),
     user: User = Depends(get_current_user),
-):
+    ):
     """Delete all documents with a specific filename"""
-    from config.settings import get_index_name
-
-    jwt_token = user.jwt_token
-
-    try:
-        opensearch_client = session_manager.get_user_opensearch_client(
-            user.user_id, jwt_token
-        )
-
-        from utils.opensearch_queries import build_filename_delete_body
-
-        delete_query = build_filename_delete_body(body.filename)
-
-        logger.debug(f"Deleting documents with filename: {body.filename}")
-
-        result = await opensearch_client.delete_by_query(
-            index=get_index_name(),
-            body=delete_query,
-            conflicts="proceed"
-        )
-
-        deleted_count = result.get("deleted", 0)
-        logger.info(f"Deleted {deleted_count} chunks for filename {body.filename}", user_id=user.user_id)
-
-        return JSONResponse({
-            "success": True,
-            "deleted_chunks": deleted_count,
-            "filename": body.filename,
-            "message": f"All documents with filename '{body.filename}' deleted successfully"
-        }, status_code=200)
-
-    except Exception as e:
-        logger.error("Error deleting documents by filename", filename=body.filename, error=str(e))
-        error_str = str(e)
-        if "AuthenticationException" in error_str:
-            return JSONResponse({"error": "Access denied: insufficient permissions"}, status_code=403)
-        else:
-            return JSONResponse({"error": str(e)}, status_code=500)
+    payload, status_code =await delete_documents_by_filename_core(
+        filename=body.filename,
+        session_manager=session_manager,
+        user_id=user.user_id,
+        jwt_token=user.jwt_token,
+    )
+    return JSONResponse(payload, status_code=status_code)