OpenRAG 0.2.4__tar.gz → 0.2.5__tar.gz

openrag-0.2.5/PKG-INFO

@@ -0,0 +1,196 @@
+Metadata-Version: 2.4
+Name: openrag
+Version: 0.2.5
+Summary: OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations.
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: agentd>=0.2.2
+Requires-Dist: aiofiles>=24.1.0
+Requires-Dist: cryptography>=45.0.6
+Requires-Dist: google-api-python-client>=2.143.0
+Requires-Dist: google-auth-httplib2>=0.2.0
+Requires-Dist: google-auth-oauthlib>=1.2.0
+Requires-Dist: msal>=1.29.0
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: opensearch-py[async]>=3.0.0
+Requires-Dist: pyjwt>=2.8.0
+Requires-Dist: python-multipart>=0.0.20
+Requires-Dist: fastapi>=0.115.0
+Requires-Dist: uvicorn>=0.35.0
+Requires-Dist: boto3>=1.35.0
+Requires-Dist: psutil>=7.0.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: textual>=0.45.0
+Requires-Dist: python-dotenv>=1.0.0
+Requires-Dist: textual-fspicker>=0.6.0
+Requires-Dist: structlog>=25.4.0
+Requires-Dist: zxcvbn>=4.5.0
+Dynamic: license-file
+
+<div align="center">
+
+<img src="./docs/static/img/openrag-logo-dog.svg" alt="OpenRAG Logo" width="120"/>
+
+# OpenRAG
+
+<h3>
+  <em>Intelligent Agent-powered document search</em>
+</h3>
+
+<!-- Badges -->
+  
+[![Langflow](https://img.shields.io/badge/Langflow-1C1C1E?style=for-the-badge&logo=langflow)](https://github.com/langflow-ai/langflow)
+[![OpenSearch](https://img.shields.io/badge/OpenSearch-005EB8?style=for-the-badge&logo=opensearch&logoColor=white)](https://github.com/opensearch-project/OpenSearch)
+[![Docling](https://img.shields.io/badge/Docling-000000?style=for-the-badge)](https://github.com/docling-project/docling)
+
+[![YouTube Channel](https://img.shields.io/youtube/channel/subscribers/UCn2bInQrjdDYKEEmbpwblLQ?label=Subscribe&style=social)](https://www.youtube.com/@OpenRAG/)
+[![GitHub stars](https://img.shields.io/github/stars/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/stargazers)
+[![GitHub forks](https://img.shields.io/github/forks/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/network/members)
+
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langflow-ai/openrag)
+
+</div>
+
+---
+
+OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations.
+
+Users can upload, process, and query documents through a chat interface backed by large language models and semantic search capabilities. The system utilizes Langflow for document ingestion, retrieval workflows, and intelligent nudges, providing a seamless RAG experience. 
+
+Built with [Starlette](https://github.com/Kludex/starlette) and [Next.js](https://github.com/vercel/next.js). 
+Powered by [OpenSearch](https://github.com/opensearch-project/OpenSearch), [Langflow](https://github.com/langflow-ai/langflow), and [Docling](https://github.com/docling-project/docling).
+
+---
+
+<div align="center">
+  <img src="./docs/static/img/openrag_readme_downsized.gif" alt="OpenRAG Demo" width="100%"/>
+</div>
+
+## ✨ Highlight Features
+
+- **Pre-packaged & ready to run** - All core tools are hooked up and ready to go, just install and run
+- **Agentic RAG workflows** - Advanced orchestration with re-ranking and multi-agent coordination
+- **Document ingestion** - Handles messy, real-world data with intelligent parsing
+- **Drag-and-drop workflow builder** - Visual interface powered by Langflow for rapid iteration
+- **Modular enterprise add-ons** - Extend functionality when you need it
+- **Enterprise search at any scale** - Powered by OpenSearch for production-grade performance
+
+## 🔄 How OpenRAG Works
+
+OpenRAG follows a streamlined workflow to transform your documents into intelligent, searchable knowledge:
+
+<div align="center">
+  <img src="./docs/static/img/workflow-diagram.svg" alt="OpenRAG Workflow Diagram" width="800"/>
+</div>
+
+## 🚀 Install OpenRAG
+
+To get started with OpenRAG, see the installation guides in the OpenRAG documentation:
+
+* [Quickstart](https://docs.openr.ag/quickstart)
+* [Install the OpenRAG Python package](https://docs.openr.ag/install-options)
+* [Deploy self-managed services with Docker or Podman](https://docs.openr.ag/docker)
+
+## ✨ Quick Start Workflow
+
+<div align="center">
+
+<img src="./docs/static/img/uv_run_openrag.png" alt="Use uv run openrag to start" width="300"/>
+
+**1. Launch OpenRAG**
+
+↓
+
+<img src="./docs/static/img/add_knowledge_openrag.png" alt="Add files or folders as knowledge" width="300"/>
+
+**2. Add Knowledge**
+
+↓
+
+<img src="./docs/static/img/chat_openrag.png" alt="Start Chatting with your knowledge" width="700"/>
+
+**3. Start Chatting**
+
+</div>
+
+## 📦 SDKs
+
+Integrate OpenRAG into your applications with our official SDKs:
+
+### Python SDK
+```bash
+pip install openrag-sdk
+```
+
+**Quick Example:**
+```python
+import asyncio
+from openrag_sdk import OpenRAGClient
+
+
+async def main():
+    async with OpenRAGClient() as client:
+        response = await client.chat.create(message="What is RAG?")
+        print(response.response)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+📖 [Full Python SDK Documentation](https://pypi.org/project/openrag-sdk/)
+
+### TypeScript/JavaScript SDK
+```bash
+npm install openrag-sdk
+```
+
+**Quick Example:**
+```typescript
+import { OpenRAGClient } from "openrag-sdk";
+
+const client = new OpenRAGClient();
+const response = await client.chat.create({ message: "What is RAG?" });
+console.log(response.response);
+```
+
+📖 [Full TypeScript/JavaScript SDK Documentation](https://www.npmjs.com/package/openrag-sdk)
+
+## 🔌 Model Context Protocol (MCP)
+
+Connect AI assistants like Cursor and Claude Desktop to your OpenRAG knowledge base:
+
+```bash
+pip install openrag-mcp
+```
+
+**Quick Example (Cursor/Claude Desktop config):**
+```json
+{
+  "mcpServers": {
+    "openrag": {
+      "command": "uvx",
+      "args": ["openrag-mcp"],
+      "env": {
+        "OPENRAG_URL": "http://localhost:3000",
+        "OPENRAG_API_KEY": "your_api_key_here"
+      }
+    }
+  }
+}
+```
+
+The MCP server provides tools for RAG-enhanced chat, semantic search, and settings management.
+
+📖 [Full MCP Documentation](https://pypi.org/project/openrag-mcp/)
+
+## 🛠️ Development
+
+For developers who want to [contribute to OpenRAG](https://docs.openr.ag/support/contribute) or set up a development environment, see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## 🆘 Troubleshooting
+
+For assistance with OpenRAG, see [Troubleshoot OpenRAG](https://docs.openr.ag/support/troubleshoot) and visit the [Discussions page](https://github.com/langflow-ai/openrag/discussions).
+
+To report a bug or submit a feature request, visit the [Issues page](https://github.com/langflow-ai/openrag/issues).

openrag-0.2.5/README.md

@@ -0,0 +1,166 @@
+<div align="center">
+
+<img src="./docs/static/img/openrag-logo-dog.svg" alt="OpenRAG Logo" width="120"/>
+
+# OpenRAG
+
+<h3>
+  <em>Intelligent Agent-powered document search</em>
+</h3>
+
+<!-- Badges -->
+  
+[![Langflow](https://img.shields.io/badge/Langflow-1C1C1E?style=for-the-badge&logo=langflow)](https://github.com/langflow-ai/langflow)
+[![OpenSearch](https://img.shields.io/badge/OpenSearch-005EB8?style=for-the-badge&logo=opensearch&logoColor=white)](https://github.com/opensearch-project/OpenSearch)
+[![Docling](https://img.shields.io/badge/Docling-000000?style=for-the-badge)](https://github.com/docling-project/docling)
+
+[![YouTube Channel](https://img.shields.io/youtube/channel/subscribers/UCn2bInQrjdDYKEEmbpwblLQ?label=Subscribe&style=social)](https://www.youtube.com/@OpenRAG/)
+[![GitHub stars](https://img.shields.io/github/stars/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/stargazers)
+[![GitHub forks](https://img.shields.io/github/forks/langflow-ai/openrag?style=social)](https://github.com/langflow-ai/openrag/network/members)
+
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langflow-ai/openrag)
+
+</div>
+
+---
+
+OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations.
+
+Users can upload, process, and query documents through a chat interface backed by large language models and semantic search capabilities. The system utilizes Langflow for document ingestion, retrieval workflows, and intelligent nudges, providing a seamless RAG experience. 
+
+Built with [Starlette](https://github.com/Kludex/starlette) and [Next.js](https://github.com/vercel/next.js). 
+Powered by [OpenSearch](https://github.com/opensearch-project/OpenSearch), [Langflow](https://github.com/langflow-ai/langflow), and [Docling](https://github.com/docling-project/docling).
+
+---
+
+<div align="center">
+  <img src="./docs/static/img/openrag_readme_downsized.gif" alt="OpenRAG Demo" width="100%"/>
+</div>
+
+## ✨ Highlight Features
+
+- **Pre-packaged & ready to run** - All core tools are hooked up and ready to go, just install and run
+- **Agentic RAG workflows** - Advanced orchestration with re-ranking and multi-agent coordination
+- **Document ingestion** - Handles messy, real-world data with intelligent parsing
+- **Drag-and-drop workflow builder** - Visual interface powered by Langflow for rapid iteration
+- **Modular enterprise add-ons** - Extend functionality when you need it
+- **Enterprise search at any scale** - Powered by OpenSearch for production-grade performance
+
+## 🔄 How OpenRAG Works
+
+OpenRAG follows a streamlined workflow to transform your documents into intelligent, searchable knowledge:
+
+<div align="center">
+  <img src="./docs/static/img/workflow-diagram.svg" alt="OpenRAG Workflow Diagram" width="800"/>
+</div>
+
+## 🚀 Install OpenRAG
+
+To get started with OpenRAG, see the installation guides in the OpenRAG documentation:
+
+* [Quickstart](https://docs.openr.ag/quickstart)
+* [Install the OpenRAG Python package](https://docs.openr.ag/install-options)
+* [Deploy self-managed services with Docker or Podman](https://docs.openr.ag/docker)
+
+## ✨ Quick Start Workflow
+
+<div align="center">
+
+<img src="./docs/static/img/uv_run_openrag.png" alt="Use uv run openrag to start" width="300"/>
+
+**1. Launch OpenRAG**
+
+↓
+
+<img src="./docs/static/img/add_knowledge_openrag.png" alt="Add files or folders as knowledge" width="300"/>
+
+**2. Add Knowledge**
+
+↓
+
+<img src="./docs/static/img/chat_openrag.png" alt="Start Chatting with your knowledge" width="700"/>
+
+**3. Start Chatting**
+
+</div>
+
+## 📦 SDKs
+
+Integrate OpenRAG into your applications with our official SDKs:
+
+### Python SDK
+```bash
+pip install openrag-sdk
+```
+
+**Quick Example:**
+```python
+import asyncio
+from openrag_sdk import OpenRAGClient
+
+
+async def main():
+    async with OpenRAGClient() as client:
+        response = await client.chat.create(message="What is RAG?")
+        print(response.response)
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+📖 [Full Python SDK Documentation](https://pypi.org/project/openrag-sdk/)
+
+### TypeScript/JavaScript SDK
+```bash
+npm install openrag-sdk
+```
+
+**Quick Example:**
+```typescript
+import { OpenRAGClient } from "openrag-sdk";
+
+const client = new OpenRAGClient();
+const response = await client.chat.create({ message: "What is RAG?" });
+console.log(response.response);
+```
+
+📖 [Full TypeScript/JavaScript SDK Documentation](https://www.npmjs.com/package/openrag-sdk)
+
+## 🔌 Model Context Protocol (MCP)
+
+Connect AI assistants like Cursor and Claude Desktop to your OpenRAG knowledge base:
+
+```bash
+pip install openrag-mcp
+```
+
+**Quick Example (Cursor/Claude Desktop config):**
+```json
+{
+  "mcpServers": {
+    "openrag": {
+      "command": "uvx",
+      "args": ["openrag-mcp"],
+      "env": {
+        "OPENRAG_URL": "http://localhost:3000",
+        "OPENRAG_API_KEY": "your_api_key_here"
+      }
+    }
+  }
+}
+```
+
+The MCP server provides tools for RAG-enhanced chat, semantic search, and settings management.
+
+📖 [Full MCP Documentation](https://pypi.org/project/openrag-mcp/)
+
+## 🛠️ Development
+
+For developers who want to [contribute to OpenRAG](https://docs.openr.ag/support/contribute) or set up a development environment, see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## 🆘 Troubleshooting
+
+For assistance with OpenRAG, see [Troubleshoot OpenRAG](https://docs.openr.ag/support/troubleshoot) and visit the [Discussions page](https://github.com/langflow-ai/openrag/discussions).
+
+To report a bug or submit a feature request, visit the [Issues page](https://github.com/langflow-ai/openrag/issues).

{openrag-0.2.4 → openrag-0.2.5}/pyproject.toml

@@ -4,16 +4,14 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "openrag"
-version = "0.2.4"
-description = "Add your description here"
+version = "0.2.5"
+description = "OpenRAG is a comprehensive Retrieval-Augmented Generation platform that enables intelligent document search and AI-powered conversations."
 readme = "README.md"
 requires-python = ">=3.13"
 dependencies = [
     "agentd>=0.2.2",
     "aiofiles>=24.1.0",
     "cryptography>=45.0.6",
-    "docling[vlm]==2.41.0; sys_platform != 'darwin'",
-    "docling[ocrmac,vlm]==2.41.0; sys_platform == 'darwin'",
     "google-api-python-client>=2.143.0",
     "google-auth-httplib2>=0.2.0",
     "google-auth-oauthlib>=1.2.0",
@@ -22,8 +20,7 @@ dependencies = [
     "opensearch-py[async]>=3.0.0",
     "pyjwt>=2.8.0",
     "python-multipart>=0.0.20",
-    "starlette>=0.47.1",
-    "torch>=2.7.1",
+    "fastapi>=0.115.0",
     "uvicorn>=0.35.0",
     "boto3>=1.35.0",
     "psutil>=7.0.0",
@@ -32,9 +29,6 @@ dependencies = [
     "python-dotenv>=1.0.0",
     "textual-fspicker>=0.6.0",
     "structlog>=25.4.0",
-    "docling-serve==1.5.0",
-    "docling-core==2.48.1",
-    "easyocr>=1.7.1; sys_platform != 'darwin'",
     "zxcvbn>=4.5.0"
 ]
 
@@ -48,15 +42,3 @@ openrag = "tui.main:run_tui"
 package = true
 
 
-[tool.uv.sources]
-torch = [
-  { index = "pytorch-cu128", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
-]
-torchvision = [
-  { index = "pytorch-cu128", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
-]
-
-[[tool.uv.index]]
-name     = "pytorch-cu128"
-url      = "https://download.pytorch.org/whl/cu128"
-explicit = true

{openrag-0.2.4 → openrag-0.2.5}/src/agent.py

@@ -197,6 +197,18 @@ async def async_response_stream(
                             sample_data=str(potential_tool_fields)[:500]
                         )
 
+                # Detect response.completed event and log usage
+                if isinstance(chunk_data, dict) and chunk_data.get("type") == "response.completed":
+                    response_data = chunk_data.get("response", {})
+                    usage = response_data.get("usage")
+                    if usage:
+                        logger.info(
+                            "Stream usage data",
+                            input_tokens=usage.get("input_tokens"),
+                            output_tokens=usage.get("output_tokens"),
+                            total_tokens=usage.get("total_tokens"),
+                        )
+
                 # Middleware: Detect implicit tool calls and inject standardized events
                 # This helps Granite 3.3 8b and other models that don't emit standard markers
                 if isinstance(chunk_data, dict) and not detected_tool_call:
@@ -487,6 +499,7 @@ async def async_chat_stream(
 
     full_response = ""
     response_id = None
+    usage_data = None
     async for chunk in async_stream(
         async_client,
         prompt,
@@ -506,6 +519,10 @@ async def async_chat_stream(
                 response_id = chunk_data["id"]
             elif "response_id" in chunk_data:
                 response_id = chunk_data["response_id"]
+            # Capture usage from response.completed event
+            if chunk_data.get("type") == "response.completed":
+                response_obj = chunk_data.get("response", {})
+                usage_data = response_obj.get("usage")
         except:
             pass
         yield chunk
@@ -518,6 +535,9 @@ async def async_chat_stream(
             "response_id": response_id,
             "timestamp": datetime.now(),
         }
+        # Store usage data if available (from response.completed event)
+        if usage_data:
+            assistant_message["response_data"] = {"usage": usage_data}
         conversation_state["messages"].append(assistant_message)
 
         # Store the conversation thread with its response_id
@@ -676,6 +696,7 @@ async def async_langflow_chat_stream(
 
     full_response = ""
     response_id = None
+    usage_data = None
     collected_chunks = []  # Store all chunks for function call data
 
     async for chunk in async_stream(
@@ -700,6 +721,10 @@ async def async_langflow_chat_stream(
                 response_id = chunk_data["id"]
             elif "response_id" in chunk_data:
                 response_id = chunk_data["response_id"]
+            # Capture usage from response.completed event
+            if chunk_data.get("type") == "response.completed":
+                response_obj = chunk_data.get("response", {})
+                usage_data = response_obj.get("usage")
         except:
             pass
         yield chunk
@@ -713,6 +738,9 @@ async def async_langflow_chat_stream(
             "timestamp": datetime.now(),
             "chunks": collected_chunks,  # Store complete chunk data for function calls
         }
+        # Store usage data if available (from response.completed event)
+        if usage_data:
+            assistant_message["response_data"] = {"usage": usage_data}
         conversation_state["messages"].append(assistant_message)
 
         # Store the conversation thread with its response_id

{openrag-0.2.4 → openrag-0.2.5}/src/api/auth.py

@@ -1,22 +1,48 @@
-from starlette.requests import Request
-from starlette.responses import JSONResponse
+from typing import Optional
+
+from fastapi import Depends, Request
+from fastapi.responses import JSONResponse
 from utils.telemetry import TelemetryClient, Category, MessageId
 
+from dependencies import (
+    get_auth_service,
+    get_optional_user,
+    get_current_user,
+)
+from pydantic import BaseModel
+from session_manager import User
+
+
+class AuthInitBody(BaseModel):
+    connector_type: str
+    purpose: str = "data_source"
+    name: Optional[str] = None
+    redirect_uri: Optional[str] = None
+
+
+class AuthCallbackBody(BaseModel):
+    connection_id: str
+    authorization_code: str
+    state: str
 
-async def auth_init(request: Request, auth_service, session_manager):
+
+class TokenIntrospectBody(BaseModel):
+    token: str
+
+
+async def auth_init(
+    body: AuthInitBody,
+    request: Request,
+    auth_service=Depends(get_auth_service),
+    user: Optional[User] = Depends(get_optional_user),
+):
     """Initialize OAuth flow for authentication or data source connection"""
     try:
-        data = await request.json()
-        connector_type = data.get("connector_type")
-        purpose = data.get("purpose", "data_source")
-        connection_name = data.get("name", f"{connector_type}_{purpose}")
-        redirect_uri = data.get("redirect_uri")
-
-        user = getattr(request.state, "user", None)
+        connection_name = body.name or f"{body.connector_type}_{body.purpose}"
         user_id = user.user_id if user else None
 
         result = await auth_service.init_oauth(
-            connector_type, purpose, connection_name, redirect_uri, user_id
+            body.connector_type, body.purpose, connection_name, body.redirect_uri, user_id
         )
         return JSONResponse(result)
 
@@ -29,16 +55,15 @@ async def auth_init(request: Request, auth_service, session_manager):
         )
 
 
-async def auth_callback(request: Request, auth_service, session_manager):
+async def auth_callback(
+    body: AuthCallbackBody,
+    request: Request,
+    auth_service=Depends(get_auth_service),
+):
     """Handle OAuth callback - exchange authorization code for tokens"""
     try:
-        data = await request.json()
-        connection_id = data.get("connection_id")
-        authorization_code = data.get("authorization_code")
-        state = data.get("state")
-
         result = await auth_service.handle_oauth_callback(
-            connection_id, authorization_code, state, request
+            body.connection_id, body.authorization_code, body.state, request
         )
 
         await TelemetryClient.send_event(Category.AUTHENTICATION, MessageId.ORB_AUTH_OAUTH_CALLBACK)
@@ -69,13 +94,20 @@ async def auth_callback(request: Request, auth_service, session_manager):
         return JSONResponse({"error": f"Callback failed: {str(e)}"}, status_code=500)
 
 
-async def auth_me(request: Request, auth_service, session_manager):
+async def auth_me(
+    request: Request,
+    auth_service=Depends(get_auth_service),
+    user: Optional[User] = Depends(get_optional_user),
+):
     """Get current user information"""
     result = await auth_service.get_user_info(request)
     return JSONResponse(result)
 
 
-async def auth_logout(request: Request, auth_service, session_manager):
+async def auth_logout(
+    auth_service=Depends(get_auth_service),
+    user: User = Depends(get_current_user),
+):
     """Logout user by clearing auth cookie"""
     await TelemetryClient.send_event(Category.AUTHENTICATION, MessageId.ORB_AUTH_LOGOUT)
     response = JSONResponse(