langroid 0.43.0__tar.gz → 0.44.0__tar.gz

{langroid-0.43.0 → langroid-0.44.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: langroid
-Version: 0.43.0
+Version: 0.44.0
 Summary: Harness LLMs with Multi-Agent Programming
 Author-email: Prasad Chalasani <pchalasani@gmail.com>
 License: MIT
@@ -237,9 +237,11 @@ This Multi-Agent paradigm is inspired by the
 
 `Langroid` is a fresh take on LLM app-development, where considerable thought has gone 
 into simplifying the developer experience; 
-it does not use `Langchain`, or any other LLM framework.
+it does not use `Langchain`, or any other LLM framework, 
+and works with [practically any LLM](https://langroid.github.io/langroid/tutorials/supported-models/).
 
-:fire: Read the (WIP) [overview of the langroid architecture](https://langroid.github.io/langroid/blog/2024/08/15/overview-of-langroids-multi-agent-architecture-prelim/) and a [quick tour of Langroid](https://langroid.github.io/langroid/tutorials/langroid-tour/)
+:fire: Read the (WIP) [overview of the langroid architecture](https://langroid.github.io/langroid/blog/2024/08/15/overview-of-langroids-multi-agent-architecture-prelim/), 
+ and a [quick tour of Langroid](https://langroid.github.io/langroid/tutorials/langroid-tour/).
 
 📢 Companies are using/adapting Langroid in **production**. Here is a quote:
 
@@ -327,6 +329,18 @@ teacher_task.run()
 <details>
 <summary> <b>Click to expand</b></summary>
 
+- **Feb 2025:**
+  - [0.43.0](https://github.com/langroid/langroid/releases/tag/0.43.0): `GeminiPdfParser` for parsing PDF using 
+    Gemini LLMs - Thanks @abab-dev.
+  - [0.42.0](https://github.com/langroid/langroid/releases/tag/0.42.0): `markitdown` parser for `pptx,xlsx,xls` files 
+    Thanks @abab-dev.
+  - [0.41.0](https://github.com/langroid/langroid/releases/tag/0.41.0): `pinecone` vector-db (Thanks @coretado), 
+    `Tavily` web-search (Thanks @Sozhan308), `Exa` web-search (Thanks @MuddyHope).
+  - [0.40.0](https://github.com/langroid/langroid/releases/tag/0.40.0): `pgvector` vector-db. Thanks @abab-dev.
+  - [0.39.0](https://github.com/langroid/langroid/releases/tag/0.39.0): `ChatAgentConfig.handle_llm_no_tool` for 
+    handling LLM "forgetting" to use a tool.
+  - [0.38.0](https://github.com/langroid/langroid/releases/tag/0.38.0): Gemini embeddings - Thanks @abab-dev)
+  - [0.37.0](https://github.com/langroid/langroid/releases/tag/0.37.0): New PDF Parsers: `docling`, `pymupdf4llm`
 - **Jan 2025:**
   - [0.36.0](https://github.com/langroid/langroid/releases/tag/0.36.0): Weaviate vector-db support (thanks @abab-dev).
   - [0.35.0](https://github.com/langroid/langroid/releases/tag/0.35.0): Capture/Stream reasoning content from 
@@ -591,7 +605,8 @@ section above)
   Agents with specific skills, wrap them in Tasks, and combine tasks in a flexible way.
 - **LLM Support**: Langroid supports OpenAI LLMs as well as LLMs from hundreds of 
 providers ([local/open](https://langroid.github.io/langroid/tutorials/local-llm-setup/) or [remote/commercial](https://langroid.github.io/langroid/tutorials/non-openai-llms/)) via proxy libraries and local model servers
-such as [ollama](https://github.com/ollama), [oobabooga](https://github.com/oobabooga/text-generation-webui), [LiteLLM](https://docs.litellm.ai/docs/providers) that in effect mimic the OpenAI API. 
+such as [ollama](https://github.com/ollama), [oobabooga](https://github.com/oobabooga/text-generation-webui), 
+  [LiteLLM](https://docs.litellm.ai/docs/providers) that in effect mimic the OpenAI API. See the [supported LLMs](https://langroid.github.io/langroid/tutorials/supported-models/). 
 - **Caching of LLM responses:** Langroid supports [Redis](https://redis.com/try-free/) and 
   [Momento](https://www.gomomento.com/) to cache LLM responses.
 - **Vector-stores**: [LanceDB](https://github.com/lancedb/lancedb), [Qdrant](https://qdrant.tech/), [Chroma](https://www.trychroma.com/) are currently supported.

{langroid-0.43.0 → langroid-0.44.0}/README.md

@@ -45,9 +45,11 @@ This Multi-Agent paradigm is inspired by the
 
 `Langroid` is a fresh take on LLM app-development, where considerable thought has gone 
 into simplifying the developer experience; 
-it does not use `Langchain`, or any other LLM framework.
+it does not use `Langchain`, or any other LLM framework, 
+and works with [practically any LLM](https://langroid.github.io/langroid/tutorials/supported-models/).
 
-:fire: Read the (WIP) [overview of the langroid architecture](https://langroid.github.io/langroid/blog/2024/08/15/overview-of-langroids-multi-agent-architecture-prelim/) and a [quick tour of Langroid](https://langroid.github.io/langroid/tutorials/langroid-tour/)
+:fire: Read the (WIP) [overview of the langroid architecture](https://langroid.github.io/langroid/blog/2024/08/15/overview-of-langroids-multi-agent-architecture-prelim/), 
+ and a [quick tour of Langroid](https://langroid.github.io/langroid/tutorials/langroid-tour/).
 
 📢 Companies are using/adapting Langroid in **production**. Here is a quote:
 
@@ -135,6 +137,18 @@ teacher_task.run()
 <details>
 <summary> <b>Click to expand</b></summary>
 
+- **Feb 2025:**
+  - [0.43.0](https://github.com/langroid/langroid/releases/tag/0.43.0): `GeminiPdfParser` for parsing PDF using 
+    Gemini LLMs - Thanks @abab-dev.
+  - [0.42.0](https://github.com/langroid/langroid/releases/tag/0.42.0): `markitdown` parser for `pptx,xlsx,xls` files 
+    Thanks @abab-dev.
+  - [0.41.0](https://github.com/langroid/langroid/releases/tag/0.41.0): `pinecone` vector-db (Thanks @coretado), 
+    `Tavily` web-search (Thanks @Sozhan308), `Exa` web-search (Thanks @MuddyHope).
+  - [0.40.0](https://github.com/langroid/langroid/releases/tag/0.40.0): `pgvector` vector-db. Thanks @abab-dev.
+  - [0.39.0](https://github.com/langroid/langroid/releases/tag/0.39.0): `ChatAgentConfig.handle_llm_no_tool` for 
+    handling LLM "forgetting" to use a tool.
+  - [0.38.0](https://github.com/langroid/langroid/releases/tag/0.38.0): Gemini embeddings - Thanks @abab-dev)
+  - [0.37.0](https://github.com/langroid/langroid/releases/tag/0.37.0): New PDF Parsers: `docling`, `pymupdf4llm`
 - **Jan 2025:**
   - [0.36.0](https://github.com/langroid/langroid/releases/tag/0.36.0): Weaviate vector-db support (thanks @abab-dev).
   - [0.35.0](https://github.com/langroid/langroid/releases/tag/0.35.0): Capture/Stream reasoning content from 
@@ -399,7 +413,8 @@ section above)
   Agents with specific skills, wrap them in Tasks, and combine tasks in a flexible way.
 - **LLM Support**: Langroid supports OpenAI LLMs as well as LLMs from hundreds of 
 providers ([local/open](https://langroid.github.io/langroid/tutorials/local-llm-setup/) or [remote/commercial](https://langroid.github.io/langroid/tutorials/non-openai-llms/)) via proxy libraries and local model servers
-such as [ollama](https://github.com/ollama), [oobabooga](https://github.com/oobabooga/text-generation-webui), [LiteLLM](https://docs.litellm.ai/docs/providers) that in effect mimic the OpenAI API. 
+such as [ollama](https://github.com/ollama), [oobabooga](https://github.com/oobabooga/text-generation-webui), 
+  [LiteLLM](https://docs.litellm.ai/docs/providers) that in effect mimic the OpenAI API. See the [supported LLMs](https://langroid.github.io/langroid/tutorials/supported-models/). 
 - **Caching of LLM responses:** Langroid supports [Redis](https://redis.com/try-free/) and 
   [Momento](https://www.gomomento.com/) to cache LLM responses.
 - **Vector-stores**: [LanceDB](https://github.com/lancedb/lancedb), [Qdrant](https://qdrant.tech/), [Chroma](https://www.trychroma.com/) are currently supported.

{langroid-0.43.0 → langroid-0.44.0}/langroid/agent/callbacks/chainlit.py

@@ -5,7 +5,16 @@ Callbacks for Chainlit integration.
 import json
 import logging
 import textwrap
-from typing import Any, Callable, Dict, List, Literal, Optional, no_type_check
+from typing import (
+    TYPE_CHECKING,
+    Any,
+    Callable,
+    Dict,
+    List,
+    Literal,
+    Optional,
+    no_type_check,
+)
 
 from langroid.exceptions import LangroidImportError
 from langroid.pydantic_v1 import BaseSettings
@@ -18,7 +27,8 @@ except ImportError:
 from chainlit import run_sync
 from chainlit.logger import logger
 
-import langroid as lr
+if TYPE_CHECKING:
+    from langroid import Agent, Task
 import langroid.language_models as lm
 from langroid.language_models import StreamEventType
 from langroid.utils.configuration import settings
@@ -222,11 +232,11 @@ class ChainlitAgentCallbacks:
     last_step: Optional[cl.Step] = None  # used to display sub-steps under this
     curr_step: Optional[cl.Step] = None  # used to update an initiated step
     stream: Optional[cl.Step] = None  # pushed into openai_gpt.py to stream tokens
-    parent_agent: Optional[lr.Agent] = None  # used to get parent id, for step nesting
+    parent_agent: Optional["Agent"] = None  # used to get parent id, for step nesting
 
     def __init__(
         self,
-        agent: lr.Agent,
+        agent: "Agent",
         config: ChainlitCallbackConfig = ChainlitCallbackConfig(),
     ):
         """Add callbacks to the agent, and save the initial message,
@@ -245,7 +255,7 @@ class ChainlitAgentCallbacks:
         agent.callbacks.show_error_message = self.show_error_message
         agent.callbacks.show_start_response = self.show_start_response
         self.config = config
-        self.agent: lr.Agent = agent
+        self.agent: "Agent" = agent
         if self.agent.llm is not None:
             # We don't want to suppress LLM output in async + streaming,
             # since we often use chainlit async callbacks to display LLM output
@@ -271,7 +281,7 @@ class ChainlitAgentCallbacks:
         )
         return last_step.id  # type: ignore
 
-    def set_parent_agent(self, parent: lr.Agent) -> None:
+    def set_parent_agent(self, parent: "Agent") -> None:
         self.parent_agent = parent
 
     def get_last_step(self) -> Optional[cl.Step]:
@@ -559,7 +569,7 @@ class ChainlitTaskCallbacks(ChainlitAgentCallbacks):
 
     def __init__(
         self,
-        task: lr.Task,
+        task: "Task",
         config: ChainlitCallbackConfig = ChainlitCallbackConfig(),
     ):
         """Inject callbacks recursively, ensuring msg is passed to the
@@ -573,7 +583,7 @@ class ChainlitTaskCallbacks(ChainlitAgentCallbacks):
 
     @classmethod
     def _inject_callbacks(
-        cls, task: lr.Task, config: ChainlitCallbackConfig = ChainlitCallbackConfig()
+        cls, task: "Task", config: ChainlitCallbackConfig = ChainlitCallbackConfig()
     ) -> None:
         # recursively apply ChainlitAgentCallbacks to agents of sub-tasks
         for t in task.sub_tasks:
@@ -581,7 +591,7 @@ class ChainlitTaskCallbacks(ChainlitAgentCallbacks):
             # ChainlitTaskCallbacks(t, config=config)
 
     def show_subtask_response(
-        self, task: lr.Task, content: str, is_tool: bool = False
+        self, task: "Task", content: str, is_tool: bool = False
     ) -> None:
         """Show sub-task response as a step, nested at the right level."""
 

{langroid-0.43.0 → langroid-0.44.0}/langroid/agent/chat_agent.py

@@ -1069,6 +1069,13 @@ class ChatAgent(Agent):
         was enabled, disables it for the tool, else triggers strict recovery.
         """
         self.tool_error = False
+        most_recent_sent_by_llm = (
+            len(self.message_history) > 0
+            and self.message_history[-1].role == Role.ASSISTANT
+        )
+        was_llm = most_recent_sent_by_llm or (
+            isinstance(msg, ChatDocument) and msg.metadata.sender == Entity.LLM
+        )
         try:
             tools = super().get_tool_messages(msg, all_tools)
         except ValidationError as ve:
@@ -1099,9 +1106,16 @@ class ChatAgent(Agent):
                     if isinstance(msg, ChatDocument):
                         self.tool_error = msg.metadata.sender == Entity.LLM
                     else:
-                        self.tool_error = True
+                        self.tool_error = most_recent_sent_by_llm
 
-            raise ve
+            if was_llm:
+                raise ve
+            else:
+                self.tool_error = False
+                return []
+
+        if not was_llm:
+            self.tool_error = False
 
         return tools
 

{langroid-0.43.0 → langroid-0.44.0}/langroid/agent/special/doc_chat_agent.py

@@ -14,6 +14,7 @@ pip install "langroid[hf-embeddings]"
 
 """
 
+import importlib
 import logging
 from collections import OrderedDict
 from functools import cache
@@ -82,14 +83,13 @@ about them, or summarize them into coherent answers.
 """
 
 CHUNK_ENRICHMENT_DELIMITER = "\n<##-##-##>\n"
-
-has_sentence_transformers = False
 try:
-    from sentence_transformers import SentenceTransformer  # noqa: F401
-
-    has_sentence_transformers = True
-except ImportError:
-    pass
+    # Check if  module exists in sys.path
+    spec = importlib.util.find_spec("sentence_transformers")
+    has_sentence_transformers = spec is not None
+except Exception as e:
+    logger.warning(f"Error checking sentence_transformers: {e}")
+    has_sentence_transformers = False
 
 
 hf_embed_config = SentenceTransformerEmbeddingsConfig(
@@ -236,6 +236,7 @@ class DocChatAgent(ChatAgent):
         self.chunked_docs: List[Document] = []
         self.chunked_docs_clean: List[Document] = []
         self.response: None | Document = None
+
         if len(config.doc_paths) > 0:
             self.ingest()
 

{langroid-0.43.0 → langroid-0.44.0}/langroid/parsing/document_parser.py

@@ -16,28 +16,11 @@ from dotenv import load_dotenv
 from langroid.exceptions import LangroidImportError
 from langroid.utils.object_registry import ObjectRegistry
 
-try:
+if TYPE_CHECKING:
+    import docling  # noqa
     import fitz
-except ImportError:
-    if not TYPE_CHECKING:
-        fitz = None
-try:
-    import pymupdf4llm
-except ImportError:
-    if not TYPE_CHECKING:
-        pymupdf4llm = None
-
-try:
-    import docling
-except ImportError:
-    if not TYPE_CHECKING:
-        docling = None
-
-try:
+    import pymupdf4llm  # noqa
     import pypdf
-except ImportError:
-    if not TYPE_CHECKING:
-        pypdf = None
 
 
 import requests
@@ -469,8 +452,10 @@ class FitzPDFParser(DocumentParser):
         Returns:
             Generator[fitz.Page]: Generator yielding each page.
         """
-        if fitz is None:
-            raise LangroidImportError("fitz", "pdf-parsers")
+        try:
+            import fitz
+        except ImportError:
+            LangroidImportError("fitz", "doc-chat")
         doc = fitz.open(stream=self.doc_bytes, filetype="pdf")
         for i, page in enumerate(doc):
             yield i, page
@@ -504,7 +489,10 @@ class PyMuPDF4LLMParser(DocumentParser):
         Returns:
             Generator[fitz.Page]: Generator yielding each page.
         """
-        if fitz is None:
+        try:
+            import pymupdf4llm  # noqa
+            import fitz
+        except ImportError:
             raise LangroidImportError(
                 "pymupdf4llm", ["pymupdf4llm", "all", "pdf-parsers", "doc-chat"]
             )
@@ -548,7 +536,9 @@ class DoclingParser(DocumentParser):
         Returns:
             Generator[docling.Page]: Generator yielding each page.
         """
-        if docling is None:
+        try:
+            import docling  # noqa
+        except ImportError:
             raise LangroidImportError(
                 "docling", ["docling", "pdf-parsers", "all", "doc-chat"]
             )
@@ -637,7 +627,9 @@ class PyPDFParser(DocumentParser):
         Returns:
             Generator[pypdf.pdf.PageObject]: Generator yielding each page.
         """
-        if pypdf is None:
+        try:
+            import pypdf
+        except ImportError:
             raise LangroidImportError("pypdf", "pdf-parsers")
         reader = pypdf.PdfReader(self.doc_bytes)
         for i, page in enumerate(reader.pages):

{langroid-0.43.0 → langroid-0.44.0}/langroid/parsing/repo_loader.py

@@ -7,14 +7,16 @@ import tempfile
 import time
 from collections import deque
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Tuple, Union
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple, Union
 from urllib.parse import urlparse
 
 from dotenv import load_dotenv
-from github import Github
-from github.ContentFile import ContentFile
-from github.Label import Label
-from github.Repository import Repository
+
+if TYPE_CHECKING:
+    from github import Github
+    from github.ContentFile import ContentFile
+    from github.Label import Label
+    from github.Repository import Repository
 
 from langroid.mytypes import DocMetaData, Document
 from langroid.parsing.document_parser import DocumentParser, DocumentType
@@ -24,7 +26,7 @@ from langroid.pydantic_v1 import BaseModel, BaseSettings, Field
 logger = logging.getLogger(__name__)
 
 
-def _get_decoded_content(content_file: ContentFile) -> str:
+def _get_decoded_content(content_file: "ContentFile") -> str:
     if content_file.encoding == "base64":
         return content_file.decoded_content.decode("utf-8") or ""
     elif content_file.encoding == "none":
@@ -54,7 +56,7 @@ class IssueData(BaseModel):
     text: str = Field(..., description="Text of issue, i.e. description body")
 
 
-def get_issue_size(labels: List[Label]) -> str | None:
+def get_issue_size(labels: List["Label"]) -> str | None:
     sizes = ["XS", "S", "M", "L", "XL", "XXL"]
     return next((label.name for label in labels if label.name in sizes), None)
 
@@ -117,6 +119,8 @@ class RepoLoader:
         self.config = config
         self.clone_path: Optional[str] = None
         self.log_file = ".logs/repo_loader/download_log.json"
+        self.repo: Optional["Repository"] = None  # Initialize repo as Optional
+
         os.makedirs(os.path.dirname(self.log_file), exist_ok=True)
         if not os.path.exists(self.log_file):
             with open(self.log_file, "w") as f:
@@ -127,20 +131,25 @@ class RepoLoader:
             logger.info(f"Repo Already downloaded in {log[self.url]}")
             self.clone_path = log[self.url]
 
+        # it's a core dependency, so we don't need to enclose in try/except
+        from github import Github  # Late import
+
+        load_dotenv()
+        # authenticated calls to github api have higher rate limit
+        token = os.getenv("GITHUB_ACCESS_TOKEN")
+
         if "github.com" in self.url:
             repo_name = self.url.split("github.com/")[1]
         else:
             repo_name = self.url
-        load_dotenv()
-        # authenticated calls to github api have higher rate limit
-        token = os.getenv("GITHUB_ACCESS_TOKEN")
+
         g = Github(token)
         self.repo = self._get_repo_with_retry(g, repo_name)
 
     @staticmethod
     def _get_repo_with_retry(
-        g: Github, repo_name: str, max_retries: int = 5
-    ) -> Repository:
+        g: "Github", repo_name: str, max_retries: int = 5
+    ) -> "Repository":
         """
         Get a repo from the GitHub API, retrying if the request fails,
         with exponential backoff.
@@ -173,6 +182,10 @@ class RepoLoader:
 
     def get_issues(self, k: int | None = 100) -> List[IssueData]:
         """Get up to k issues from the GitHub repo."""
+        if self.repo is None:
+            logger.warning("No repo found. Ensure the URL is correct.")
+            return []  # Return an empty list rather than raise an error in this case
+
         if k is None:
             issues = self.repo.get_issues(state="all")
         else:
@@ -224,7 +237,7 @@ class RepoLoader:
         """
         return file_type not in self.config.non_code_types
 
-    def _is_allowed(self, content: ContentFile) -> bool:
+    def _is_allowed(self, content: "ContentFile") -> bool:
         """
         Check if a file or directory content is allowed to be included.
 
@@ -301,6 +314,10 @@ class RepoLoader:
             Dict[str, Union[str, List[Dict]]]:
             A dictionary containing file and directory names, with file contents.
         """
+        if self.repo is None:
+            logger.warning("No repo found. Ensure the URL is correct.")
+            return {}  # Return an empty dict rather than raise an error in this case
+
         root_contents = self.repo.get_contents("")
         if not isinstance(root_contents, list):
             root_contents = [root_contents]
@@ -519,8 +536,7 @@ class RepoLoader:
                 which includes all depths.
             lines (int, optional): Number of lines to read from each file.
                 Defaults to None, which reads all lines.
-            doc_type (str|DocumentType, optional): The type of document to parse.
-
+            doc_type (str|DocumentType | None, optional): The type of document to parse.
         Returns:
             List[Document]: List of Document objects representing files.
 
@@ -584,6 +600,10 @@ class RepoLoader:
             list of Document objects, each has fields `content` and `metadata`,
             and `metadata` has fields `url`, `filename`, `extension`, `language`
         """
+        if self.repo is None:
+            logger.warning("No repo found. Ensure the URL is correct.")
+            return []  # Return an empty list rather than raise an error
+
         contents = self.repo.get_contents("")
         if not isinstance(contents, list):
             contents = [contents]

{langroid-0.43.0 → langroid-0.44.0}/langroid/parsing/search.py

@@ -10,9 +10,6 @@ import difflib
 import re
 from typing import List, Tuple
 
-from nltk.corpus import stopwords
-from nltk.stem import WordNetLemmatizer
-from nltk.tokenize import RegexpTokenizer
 from rank_bm25 import BM25Okapi
 from thefuzz import fuzz, process
 
@@ -120,6 +117,9 @@ def preprocess_text(text: str) -> str:
     # Ensure the NLTK resources are available
     for resource in ["tokenizers/punkt", "corpora/wordnet", "corpora/stopwords"]:
         download_nltk_resource(resource)
+    from nltk.corpus import stopwords
+    from nltk.stem import WordNetLemmatizer
+    from nltk.tokenize import RegexpTokenizer
 
     # Lowercase the text
     text = text.lower()

{langroid-0.43.0 → langroid-0.44.0}/langroid/parsing/url_loader.py

@@ -4,12 +4,6 @@ from tempfile import NamedTemporaryFile
 from typing import List, no_type_check
 
 import requests
-import trafilatura
-from trafilatura.downloads import (
-    add_to_compressed_dict,
-    buffered_downloads,
-    load_download_buffer,
-)
 
 from langroid.mytypes import DocMetaData, Document
 from langroid.parsing.document_parser import DocumentParser, ImagePdfParser
@@ -36,6 +30,13 @@ class URLLoader:
 
     @no_type_check
     def load(self) -> List[Document]:
+        import trafilatura
+        from trafilatura.downloads import (
+            add_to_compressed_dict,
+            buffered_downloads,
+            load_download_buffer,
+        )
+
         docs = []
         threads = 4
         # converted the input list to an internal format

{langroid-0.43.0 → langroid-0.44.0}/langroid/parsing/urls.py

@@ -11,7 +11,6 @@ import requests
 from bs4 import BeautifulSoup
 from rich import print
 from rich.prompt import Prompt
-from trafilatura.spider import focused_crawler
 
 from langroid.pydantic_v1 import BaseModel, HttpUrl, ValidationError, parse_obj_as
 
@@ -150,6 +149,8 @@ def crawl_url(url: str, max_urls: int = 1) -> List[str]:
     up to a maximum of `max_urls`.
     This has not been tested to work as intended. Ignore.
     """
+    from trafilatura.spider import focused_crawler
+
     if max_urls == 1:
         # no need to crawl, just return the original list
         return [url]

{langroid-0.43.0 → langroid-0.44.0}/langroid/parsing/utils.py

@@ -6,7 +6,6 @@ from functools import cache
 from itertools import islice
 from typing import Iterable, List, Sequence, TypeVar
 
-import nltk
 from faker import Faker
 
 from langroid.mytypes import Document
@@ -22,19 +21,19 @@ random.seed(43)
 logger = logging.getLogger(__name__)
 
 
-# Ensures the NLTK resource is available
-@cache
 def download_nltk_resource(resource: str) -> None:
-    try:
-        nltk.data.find(resource)
-    except LookupError:
-        model = resource.split("/")[-1]
-        nltk.download(model, quiet=True)
+    import nltk
+
+    @cache
+    def _download() -> None:
+        try:
+            nltk.data.find(resource)
+        except LookupError:
+            model = resource.split("/")[-1]
+            nltk.download(model, quiet=True)
 
+    _download()
 
-# Download punkt_tab resource at module import
-download_nltk_resource("tokenizers/punkt_tab")
-download_nltk_resource("corpora/gutenberg")
 
 T = TypeVar("T")
 
@@ -51,9 +50,12 @@ def batched(iterable: Iterable[T], n: int) -> Iterable[Sequence[T]]:
 
 def generate_random_sentences(k: int) -> str:
     # Load the sample text
-
+    import nltk
     from nltk.corpus import gutenberg
 
+    download_nltk_resource("corpora/gutenberg")
+    download_nltk_resource("tokenizers/punkt")
+
     text = gutenberg.raw("austen-emma.txt")
 
     # Split the text into sentences
@@ -155,6 +157,8 @@ def number_segments(s: str, granularity: int = 1) -> str:
         >>> number_segments("Hello world! How are you? Have a good day.")
         '<#1#> Hello world! <#2#> How are you? <#3#> Have a good day.'
     """
+    import nltk
+
     if granularity < 0:
         return "<#1#> " + s
     numbered_text = []

{langroid-0.43.0 → langroid-0.44.0}/langroid/vector_store/postgres.py

@@ -27,7 +27,6 @@ try:
     )
     from sqlalchemy.dialects.postgresql import JSONB
     from sqlalchemy.engine import Connection, Engine
-    from sqlalchemy.orm import sessionmaker
     from sqlalchemy.sql.expression import insert
 except ImportError:
     Engine = Any  # type: ignore
@@ -56,6 +55,11 @@ class PostgresDB(VectorStore):
         super().__init__(config)
         if not has_postgres:
             raise LangroidImportError("pgvector", "postgres")
+        try:
+            from sqlalchemy.orm import sessionmaker
+        except ImportError:
+            raise LangroidImportError("sqlalchemy", "postgres")
+
         self.config: PostgresDBConfig = config
         self.engine = self._create_engine()
         PostgresDB._create_vector_extension(self.engine)