local-deep-research 0.3.10__tar.gz → 0.3.12__tar.gz

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: local-deep-research
-Version: 0.3.10
+Version: 0.3.12
 Summary: AI-powered research assistant with deep, iterative analysis using LLMs and web searches
 Author-Email: LearningCircuit <185559241+LearningCircuit@users.noreply.github.com>, HashedViking <6432677+HashedViking@users.noreply.github.com>
 License: MIT License
@@ -106,7 +106,7 @@ Local Deep Research combines the power of large language models with intelligent
 
 ## ⚡ Quick Start
 
-### Option 1: Docker (Recommended)
+### Option 1: Docker (Quickstart no MAC/ARM)
 
 ```bash
 # Step 1: Pull and run SearXNG for optimal search results
@@ -115,29 +115,54 @@ docker run -d -p 8080:8080 --name searxng searxng/searxng
 
 # Step 2: Pull and run Local Deep Research (Please build your own docker on ARM)
 docker pull localdeepresearch/local-deep-research
-docker run -d -p 5000:5000 --name local-deep-research localdeepresearch/local-deep-research
+docker run -d -p 5000:5000 --network host --name local-deep-research localdeepresearch/local-deep-research
 
-# Optional 3a: For connecting to already installed local Ollama (https://ollama.com/download) or other local services
-# docker run -d -p 5000:5000 --network host --name local-deep-research localdeepresearch/local-deep-research
-
-# Optional 3b (recommended): Pull and run Ollama for local LLM capabilities
-# docker pull ollama/ollama
-# docker run -d -p 11434:11434 --name ollama ollama/ollama
-# docker exec -it ollama ollama pull gemma3:12b
-
-# Start containers - Required after each reboot (can be automated with this flag --restart unless-stopped in run)
+# Start containers - Required after each reboot (can be automated with this flag in run command --restart unless-stopped)
 docker start searxng
 docker start local-deep-research
-# docker start ollama
+
+```
+
+### Option 2: Docker Compose (Recommended)
+
+LDR uses Docker compose to bundle the web app and all it's dependencies so
+you can get up and running quickly.
+
+### Prerequisites
+
+- [Docker](https://docs.docker.com/engine/install/)
+- [Docker Compose](https://docs.docker.com/compose/install/)
+- `cookiecutter`: Run `pip install --user cookiecutter`
+
+Clone the repository:
+
+```bash
+git clone https://github.com/LearningCircuit/local-deep-research.git
+cd local-deep-research
+```
+
+### Configuring with Docker Compose
+
+In the LDR repository, run the following command
+to do generate the compose file:
+
+```bash
+cookiecutter cookiecutter-docker/
+```
+
+This will prompt you to answer a series of questions. Hit Enter repeatedly
+to accept the default values. It should generate a file in the repository called `docker-compose.default.yml`. To run LDR, use the following command:
+
+```bash
+docker compose -f docker-compose.default.yml up
 ```
 
 Then visit `http://127.0.0.1:5000` to start researching!
 
-> **Note**: If you need to connect to local services (like Ollama), add `--network host` to the command.
->
-> **Don't have Docker? It's installed in a few clicks: [Install Docker here](https://www.docker.com/get-started/)**
+See [here](https://github.com/LearningCircuit/local-deep-research/wiki/Installation#docker-installation-recommended) for more information about
+using Docker.
 
-### Option 2: Python Package (mostly for programmatic access)
+### Option 3: Python Package (mostly for programmatic access)
 
 ```bash
 # Install the package

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/README.md

@@ -35,7 +35,7 @@ Local Deep Research combines the power of large language models with intelligent
 
 ## ⚡ Quick Start
 
-### Option 1: Docker (Recommended)
+### Option 1: Docker (Quickstart no MAC/ARM)
 
 ```bash
 # Step 1: Pull and run SearXNG for optimal search results
@@ -44,29 +44,54 @@ docker run -d -p 8080:8080 --name searxng searxng/searxng
 
 # Step 2: Pull and run Local Deep Research (Please build your own docker on ARM)
 docker pull localdeepresearch/local-deep-research
-docker run -d -p 5000:5000 --name local-deep-research localdeepresearch/local-deep-research
+docker run -d -p 5000:5000 --network host --name local-deep-research localdeepresearch/local-deep-research
 
-# Optional 3a: For connecting to already installed local Ollama (https://ollama.com/download) or other local services
-# docker run -d -p 5000:5000 --network host --name local-deep-research localdeepresearch/local-deep-research
-
-# Optional 3b (recommended): Pull and run Ollama for local LLM capabilities
-# docker pull ollama/ollama
-# docker run -d -p 11434:11434 --name ollama ollama/ollama
-# docker exec -it ollama ollama pull gemma3:12b
-
-# Start containers - Required after each reboot (can be automated with this flag --restart unless-stopped in run)
+# Start containers - Required after each reboot (can be automated with this flag in run command --restart unless-stopped)
 docker start searxng
 docker start local-deep-research
-# docker start ollama
+
+```
+
+### Option 2: Docker Compose (Recommended)
+
+LDR uses Docker compose to bundle the web app and all it's dependencies so
+you can get up and running quickly.
+
+### Prerequisites
+
+- [Docker](https://docs.docker.com/engine/install/)
+- [Docker Compose](https://docs.docker.com/compose/install/)
+- `cookiecutter`: Run `pip install --user cookiecutter`
+
+Clone the repository:
+
+```bash
+git clone https://github.com/LearningCircuit/local-deep-research.git
+cd local-deep-research
+```
+
+### Configuring with Docker Compose
+
+In the LDR repository, run the following command
+to do generate the compose file:
+
+```bash
+cookiecutter cookiecutter-docker/
+```
+
+This will prompt you to answer a series of questions. Hit Enter repeatedly
+to accept the default values. It should generate a file in the repository called `docker-compose.default.yml`. To run LDR, use the following command:
+
+```bash
+docker compose -f docker-compose.default.yml up
 ```
 
 Then visit `http://127.0.0.1:5000` to start researching!
 
-> **Note**: If you need to connect to local services (like Ollama), add `--network host` to the command.
->
-> **Don't have Docker? It's installed in a few clicks: [Install Docker here](https://www.docker.com/get-started/)**
+See [here](https://github.com/LearningCircuit/local-deep-research/wiki/Installation#docker-installation-recommended) for more information about
+using Docker.
 
-### Option 2: Python Package (mostly for programmatic access)
+### Option 3: Python Package (mostly for programmatic access)
 
 ```bash
 # Install the package

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/pyproject.toml

@@ -57,7 +57,7 @@ dependencies = [
     "setuptools>=78.1.0",
     "flask-wtf>=1.2.2",
 ]
-version = "0.3.10"
+version = "0.3.12"
 
 [project.license]
 file = "LICENSE"
@@ -93,6 +93,14 @@ distribution = true
 source = "file"
 path = "src/local_deep_research/__version__.py"
 
+[[tool.pdm.source]]
+url = "https://download.pytorch.org/whl/cpu"
+name = "torch"
+include_packages = [
+    "torch",
+    "torch*",
+]
+
 [dependency-groups]
 dev = [
     "isort>=6.0.1",
@@ -100,4 +108,5 @@ dev = [
     "pre-commit>=4.2.0",
     "flake8>=7.1.2",
     "jupyter>=1.1.1",
+    "cookiecutter>=2.6.0",
 ]

local_deep_research-0.3.12/src/local_deep_research/__version__.py

@@ -0,0 +1 @@
+__version__ = "0.3.12"

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/config/llm_config.py

@@ -9,6 +9,7 @@ from langchain_openai import ChatOpenAI
 
 from ..utilities.db_utils import get_db_setting
 from ..utilities.search_utilities import remove_think_tags
+from ..utilities.url_utils import normalize_url
 
 # Setup logging
 logger = logging.getLogger(__name__)
@@ -141,7 +142,12 @@ def get_llm(model_name=None, temperature=None, provider=None, openai_endpoint_ur
     elif provider == "ollama":
         try:
             # Use the configurable Ollama base URL
-            base_url = get_db_setting("llm.ollama.url", "http://localhost:11434")
+            raw_base_url = get_db_setting("llm.ollama.url", "http://localhost:11434")
+            base_url = (
+                normalize_url(raw_base_url)
+                if raw_base_url
+                else "http://localhost:11434"
+            )
 
             # Check if Ollama is available before trying to use it
             if not is_ollama_available():
@@ -371,7 +377,10 @@ def is_ollama_available():
     try:
         import requests
 
-        base_url = get_db_setting("llm.ollama.url", "http://localhost:11434")
+        raw_base_url = get_db_setting("llm.ollama.url", "http://localhost:11434")
+        base_url = (
+            normalize_url(raw_base_url) if raw_base_url else "http://localhost:11434"
+        )
         logger.info(f"Checking Ollama availability at {base_url}/api/tags")
 
         try:

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/config/search_config.py

@@ -26,7 +26,7 @@ def get_search(search_tool=None, llm_instance=None):
     """
 
     # Use specified tool or default from settings
-    tool = search_tool or get_db_setting("search.tool", "auto")
+    tool = search_tool or get_db_setting("search.tool", "searxng")
     logger.info(f"Creating search engine with tool: {tool}")
 
     # Get LLM instance (use provided or get fresh one)

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/defaults/default_settings.json

@@ -733,7 +733,7 @@
         "step": null,
         "type": "SEARCH",
         "ui_element": "select",
-        "value": "auto",
+        "value": "searxng",
         "visible": true
     },
     "search.engine.web.arxiv.display_name": {
@@ -3029,16 +3029,29 @@
     },
     "search.engine.web.searxng.default_params.safe_search": {
         "category": "searxng",
-        "description": "Setting for searxng.default_params.safe_search",
+        "description": "Configure the safe search level",
         "editable": true,
         "max_value": null,
         "min_value": null,
         "name": "Safe Search",
-        "options": null,
+        "options": [
+            {
+                "label": "Off",
+                "value": "OFF"
+            },
+            {
+                "label": "Moderate",
+                "value": "MODERATE"
+            },
+            {
+                "label": "Strict",
+                "value": "STRICT"
+            }
+        ],
         "step": null,
         "type": "SEARCH",
-        "ui_element": "checkbox",
-        "value": 1,
+        "ui_element": "select",
+        "value": "OFF",
         "visible": true
     },
     "search.engine.web.searxng.full_search_class": {

local_deep_research-0.3.12/src/local_deep_research/utilities/url_utils.py

@@ -0,0 +1,57 @@
+"""URL utility functions for the local deep research application."""
+
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+def normalize_url(raw_url: str) -> str:
+    """
+    Normalize a URL to ensure it has a proper scheme and format.
+
+    Args:
+        raw_url: The raw URL string to normalize
+
+    Returns:
+        A properly formatted URL string
+
+    Examples:
+        >>> normalize_url("localhost:11434")
+        'http://localhost:11434'
+        >>> normalize_url("https://example.com:11434")
+        'https://example.com:11434'
+        >>> normalize_url("http:example.com")
+        'http://example.com'
+    """
+    if not raw_url:
+        raise ValueError("URL cannot be empty")
+
+    # Clean up the URL
+    raw_url = raw_url.strip()
+
+    # First check if the URL already has a proper scheme
+    if raw_url.startswith(("http://", "https://")):
+        return raw_url
+
+    # Handle case where URL is malformed like "http:hostname" (missing //)
+    if raw_url.startswith(("http:", "https:")) and not raw_url.startswith(
+        ("http://", "https://")
+    ):
+        scheme = raw_url.split(":", 1)[0]
+        rest = raw_url.split(":", 1)[1]
+        return f"{scheme}://{rest}"
+
+    # Handle URLs that start with //
+    if raw_url.startswith("//"):
+        # Remove the // and process
+        raw_url = raw_url[2:]
+
+    # At this point, we should have hostname:port or just hostname
+    # Determine if this is localhost or an external host
+    hostname = raw_url.split(":")[0].split("/")[0]
+    is_localhost = hostname in ("localhost", "127.0.0.1", "[::1]", "0.0.0.0")
+
+    # Use http for localhost, https for external hosts
+    scheme = "http" if is_localhost else "https"
+
+    return f"{scheme}://{raw_url}"

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/web/routes/api_routes.py

@@ -1,10 +1,10 @@
 import json
 import logging
-import os
 
 import requests
 from flask import Blueprint, current_app, jsonify, request
 
+from ...utilities.url_utils import normalize_url
 from ..models.database import get_db_connection
 from ..routes.research_routes import active_research, termination_flags
 from ..services.research_service import (
@@ -49,7 +49,7 @@ def api_start_research():
         research_settings = {
             "model_provider": "OLLAMA",  # Default
             "model": "llama2",  # Default
-            "search_engine": "auto",  # Default
+            "search_engine": "searxng",  # Default
         }
 
         cursor.execute(
@@ -255,10 +255,14 @@ def check_ollama_status():
                 {"running": True, "message": f"Using provider: {provider}, not Ollama"}
             )
 
-        # Get Ollama API URL
-        ollama_base_url = os.getenv(
-            "OLLAMA_BASE_URL",
-            llm_config.get("ollama_base_url", "http://localhost:11434"),
+        # Get Ollama API URL from LLM config
+        raw_ollama_base_url = llm_config.get(
+            "ollama_base_url", "http://localhost:11434"
+        )
+        ollama_base_url = (
+            normalize_url(raw_ollama_base_url)
+            if raw_ollama_base_url
+            else "http://localhost:11434"
         )
 
         logger.info(f"Checking Ollama status at: {ollama_base_url}")
@@ -380,9 +384,14 @@ def check_ollama_model():
         # Log which model we're checking for debugging
         logger.info(f"Checking availability of Ollama model: {model_name}")
 
-        ollama_base_url = os.getenv(
-            "OLLAMA_BASE_URL",
-            llm_config.get("ollama_base_url", "http://localhost:11434"),
+        # Get Ollama API URL from LLM config
+        raw_ollama_base_url = llm_config.get(
+            "ollama_base_url", "http://localhost:11434"
+        )
+        ollama_base_url = (
+            normalize_url(raw_ollama_base_url)
+            if raw_ollama_base_url
+            else "http://localhost:11434"
         )
 
         # Check if the model is available

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/web/routes/settings_routes.py

@@ -19,6 +19,7 @@ from flask_wtf.csrf import generate_csrf
 from sqlalchemy.orm import Session
 
 from ...utilities.db_utils import get_db_setting
+from ...utilities.url_utils import normalize_url
 from ..database.models import Setting, SettingType
 from ..services.settings_service import (
     create_or_update_setting,
@@ -667,7 +668,15 @@ def api_get_available_models():
             try:
                 current_app.logger.info("Attempting to connect to Ollama API")
 
-                base_url = get_db_setting("llm.ollama.url", "http://localhost:11434")
+                raw_base_url = get_db_setting(
+                    "llm.ollama.url", "http://localhost:11434"
+                )
+                base_url = (
+                    normalize_url(raw_base_url)
+                    if raw_base_url
+                    else "http://localhost:11434"
+                )
+
                 ollama_response = requests.get(f"{base_url}/api/tags", timeout=5)
 
                 current_app.logger.debug(
@@ -1269,11 +1278,12 @@ def fix_corrupted_settings():
 def check_ollama_status():
     """Check if Ollama is running and available"""
     try:
-        # Set a shorter timeout for the request
-        base_url = os.getenv(
-            "OLLAMA_BASE_URL",
-            "http://localhost:11434",
+        # Get Ollama URL from settings
+        raw_base_url = get_db_setting("llm.ollama.url", "http://localhost:11434")
+        base_url = (
+            normalize_url(raw_base_url) if raw_base_url else "http://localhost:11434"
         )
+
         response = requests.get(f"{base_url}/api/version", timeout=2.0)
 
         if response.status_code == 200:

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/web/services/research_service.py

@@ -530,7 +530,7 @@ def run_research_process(
                     report_path = os.path.join(
                         OUTPUT_DIR,
                         f"quick_summary_{safe_query}_"
-                        f"{datetime.now().isoformat()}.md",
+                        f"{int(datetime.now().timestamp())}.md",
                     )
 
                     # Send progress update for writing to file
@@ -643,7 +643,7 @@ def run_research_process(
             safe_query = safe_query.replace(" ", "_").lower()
             report_path = os.path.join(
                 OUTPUT_DIR,
-                f"detailed_report_{safe_query}_{datetime.now().isoformat()}.md",
+                f"detailed_report_{safe_query}_{int(datetime.now().timestamp())}.md",
             )
 
             with open(report_path, "w", encoding="utf-8") as f:

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/web_search_engines/engines/full_search.py

@@ -25,7 +25,7 @@ class FullSearchResults:
         max_results: int = 10,
         region: str = "wt-wt",
         time: str = "y",
-        safesearch: str = "Moderate",
+        safesearch: str | int = "Moderate",
     ):
         self.llm = llm
         self.output_format = output_format

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/web_search_engines/engines/search_engine_local.py

@@ -32,6 +32,7 @@ from langchain_text_splitters import RecursiveCharacterTextSplitter
 
 from ...config import search_config
 from ...utilities.db_utils import get_db_setting
+from ...utilities.url_utils import normalize_url
 from ..search_engine_base import BaseSearchEngine
 
 # Setup logging
@@ -169,12 +170,20 @@ class LocalEmbeddingManager:
             if self.embedding_model_type == "ollama":
                 # Use Ollama for embeddings
                 if not self.ollama_base_url:
-                    self.ollama_base_url = get_db_setting(
+                    raw_ollama_base_url = get_db_setting(
                         "llm.ollama.url", "http://localhost:11434"
                     )
+                    self.ollama_base_url = (
+                        normalize_url(raw_ollama_base_url)
+                        if raw_ollama_base_url
+                        else "http://localhost:11434"
+                    )
+                else:
+                    # Ensure scheme is present if ollama_base_url was passed in constructor
+                    self.ollama_base_url = normalize_url(self.ollama_base_url)
 
                 logger.info(
-                    f"Initializing Ollama embeddings with model {self.embedding_model}"
+                    f"Initializing Ollama embeddings with model {self.embedding_model} and base_url {self.ollama_base_url}"
                 )
                 return OllamaEmbeddings(
                     model=self.embedding_model, base_url=self.ollama_base_url
@@ -563,7 +572,7 @@ class LocalEmbeddingManager:
                         str(index_path),
                         self.embeddings,
                         allow_dangerous_deserialization=True,
-                        nomalize_L2=True,
+                        normalize_L2=True,
                     )
                 except Exception as e:
                     logger.error(f"Error loading index for {folder_path}: {e}")

{local_deep_research-0.3.10 → local_deep_research-0.3.12}/src/local_deep_research/web_search_engines/engines/search_engine_searxng.py

@@ -1,3 +1,4 @@
+import enum
 import logging
 import os
 import time
@@ -15,6 +16,17 @@ logging.basicConfig(level=logging.INFO)
 logger = logging.getLogger(__name__)
 
 
+@enum.unique
+class SafeSearchSetting(enum.IntEnum):
+    """
+    Acceptable settings for safe search.
+    """
+
+    OFF = 0
+    MODERATE = 1
+    STRICT = 2
+
+
 class SearXNGSearchEngine(BaseSearchEngine):
     """
     SearXNG search engine implementation that requires an instance URL provided via
@@ -29,7 +41,7 @@ class SearXNGSearchEngine(BaseSearchEngine):
         categories: Optional[List[str]] = None,
         engines: Optional[List[str]] = None,
         language: str = "en",
-        safe_search: int = 1,
+        safe_search: str = SafeSearchSetting.OFF.name,
         time_range: Optional[str] = None,
         delay_between_requests: float = 0.0,
         llm: Optional[BaseLLM] = None,
@@ -89,7 +101,14 @@ class SearXNGSearchEngine(BaseSearchEngine):
         self.categories = categories or ["general"]
         self.engines = engines
         self.language = language
-        self.safe_search = safe_search
+        try:
+            self.safe_search = SafeSearchSetting[safe_search]
+        except ValueError:
+            logger.error(
+                "'{}' is not a valid safe search setting. Disabling safe search",
+                safe_search,
+            )
+            self.safe_search = SafeSearchSetting.OFF
         self.time_range = time_range
 
         self.delay_between_requests = float(
@@ -114,11 +133,7 @@ class SearXNGSearchEngine(BaseSearchEngine):
                 max_results=max_results,
                 region="wt-wt",
                 time="y",
-                safesearch=(
-                    "Moderate"
-                    if safe_search == 1
-                    else "Off" if safe_search == 0 else "Strict"
-                ),
+                safesearch=self.safe_search.value,
             )
 
         self.last_request_time = 0
@@ -177,7 +192,7 @@ class SearXNGSearchEngine(BaseSearchEngine):
                 "language": self.language,
                 "format": "html",  # Use HTML format instead of JSON
                 "pageno": 1,
-                "safesearch": self.safe_search,
+                "safesearch": self.safe_search.value,
                 "count": self.max_results,
             }
 

local_deep_research-0.3.12/tests/test_url_utils.py

@@ -0,0 +1,55 @@
+"""Test URL utility functions."""
+
+import pytest
+
+from local_deep_research.utilities.url_utils import normalize_url
+
+
+class TestNormalizeUrl:
+    """Test cases for the normalize_url function."""
+
+    def test_localhost_without_scheme(self):
+        """Test that localhost addresses get http:// prefix."""
+        assert normalize_url("localhost:11434") == "http://localhost:11434"
+        assert normalize_url("127.0.0.1:11434") == "http://127.0.0.1:11434"
+        assert normalize_url("[::1]:11434") == "http://[::1]:11434"
+        assert normalize_url("0.0.0.0:11434") == "http://0.0.0.0:11434"
+
+    def test_external_host_without_scheme(self):
+        """Test that external hosts get https:// prefix."""
+        assert normalize_url("example.com:11434") == "https://example.com:11434"
+        assert normalize_url("api.example.com") == "https://api.example.com"
+
+    def test_malformed_url_with_scheme(self):
+        """Test correction of malformed URLs like 'http:hostname'."""
+        assert normalize_url("http:localhost:11434") == "http://localhost:11434"
+        assert normalize_url("https:example.com:11434") == "https://example.com:11434"
+
+    def test_well_formed_urls(self):
+        """Test that well-formed URLs are unchanged."""
+        assert normalize_url("http://localhost:11434") == "http://localhost:11434"
+        assert normalize_url("https://example.com:11434") == "https://example.com:11434"
+        assert (
+            normalize_url("http://192.168.1.100:11434") == "http://192.168.1.100:11434"
+        )
+
+    def test_urls_with_double_slash_prefix(self):
+        """Test URLs that start with //."""
+        assert normalize_url("//localhost:11434") == "http://localhost:11434"
+        assert normalize_url("//example.com:11434") == "https://example.com:11434"
+
+    def test_empty_or_none_url(self):
+        """Test handling of empty or None URLs."""
+        with pytest.raises(ValueError):
+            normalize_url("")
+        with pytest.raises(ValueError):
+            normalize_url(None)
+
+    def test_url_with_path(self):
+        """Test URLs with paths."""
+        assert normalize_url("localhost:11434/api") == "http://localhost:11434/api"
+        assert normalize_url("example.com/api/v1") == "https://example.com/api/v1"
+
+
+if __name__ == "__main__":
+    pytest.main([__file__, "-v"])