langroid 0.1.132__tar.gz → 0.1.134__tar.gz

{langroid-0.1.132 → langroid-0.1.134}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: langroid
-Version: 0.1.132
+Version: 0.1.134
 Summary: Harness LLMs with Multi-Agent Programming
 License: MIT
 Author: Prasad Chalasani
@@ -100,7 +100,7 @@ Description-Content-Type: text/markdown
 [![Multi-Architecture DockerHub](https://github.com/langroid/langroid/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/langroid/langroid/actions/workflows/docker-publish.yml)
 
 [![Static Badge](https://img.shields.io/badge/Documentation-blue?link=https%3A%2F%2Flangroid.github.io%2Flangroid%2F&link=https%3A%2F%2Flangroid.github.io%2Flangroid%2F)](https://langroid.github.io/langroid)
-[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/langroid_quick_examples.ipynb)
+[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
 [![Discord](https://img.shields.io/badge/Discord-%235865F2.svg?style=flat&logo=discord&logoColor=white)](https://discord.gg/ZU36McDgDs)
 [![Substack](https://img.shields.io/badge/Substack-%23006f5c.svg?style=flat&logo=substack&logoColor=FF6719)](https://langroid.substack.com/p/langroid-harness-llms-with-multi-agent-programming)
 </div>
@@ -138,7 +138,7 @@ We welcome contributions -- See the [contributions](./CONTRIBUTING.md) document
 for ideas on what to contribute.
 
 
-Building LLM Applications? Prasad Chalasani is available for consulting
+Building LLM Applications? [Prasad Chalasani](https://www.linkedin.com/in/pchalasani/) is available for consulting
 (advisory/development): pchalasani at gmail dot com.
 
 Sponsorship is also accepted via [GitHub Sponsors](https://github.com/sponsors/langroid)
@@ -148,12 +148,12 @@ Sponsorship is also accepted via [GitHub Sponsors](https://github.com/sponsors/l
 # Quick glimpse of coding with Langroid
 This is just a teaser; there's much more, like function-calling/tools, 
 Multi-Agent Collaboration, Structured Information Extraction, DocChatAgent 
-(RAG), SQLChatAgent, etc. Scroll down or see docs for more.
-
-:fire: Just released! See this [Colab](https://colab.research.google.com/drive/190Tk7t4AdY1P9F_NlZ33-YEoGnHweQQ0) 
-for a walk-through of the new `OpenAIAssistant` class (with near-complete support for the new OpenAI Assistants API) 
-in a multi-agent setting.
+(RAG), SQLChatAgent, non-OpenAI local/remote LLMs, etc. Scroll down or see docs for more.
 
+:fire: Just released! Updated Langroid Quick-Start [Colab](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
+that builds up to a 2-agent chat example using the OpenAI ChatCompletion API. 
+See also this [version](https://colab.research.google.com/drive/190Tk7t4AdY1P9F_NlZ33-YEoGnHweQQ0) 
+that uses the OpenAI Assistants API instead.
 
 ```python
 from langroid.language_models import  OpenAIGPTConfig, OpenAIChatModel, OpenAIGPT
@@ -264,7 +264,7 @@ See [this test](tests/main/test_recipient_tool.py) for example usage.
   - **Example:** [Answer questions](examples/docqa/chat-search.py) using Google Search + vecdb-retrieval from URL contents. 
   - **0.1.39:** [`GoogleSearchTool`](langroid/agent/tools/google_search_tool.py) to enable Agents (their LLM) to do Google searches via function-calling/tools.
     See [this chat example](examples/basic/chat-search.py) for how easy it is to add this tool to an agent.
-  - **Colab notebook** to try the quick-start examples: [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/langroid_quick_examples.ipynb) 
+  - **Colab notebook** to try the quick-start examples: [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb) 
   - **0.1.37:** Added [`SQLChatAgent`](langroid/agent/special/sql_chat_agent.py) -- thanks to our latest contributor [Rithwik Babu](https://github.com/rithwikbabu)!
   - Multi-agent Example: [Autocorrect chat](examples/basic/autocorrect.py)
 - **July 2023:** 
@@ -515,7 +515,7 @@ for a detailed tutorial.
 
 Click to expand any of the code examples below.
 All of these can be run in a Colab notebook:
-[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/langroid_quick_examples.ipynb)
+[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
 
 <details>
 <summary> <b> Direct interaction with OpenAI LLM </b> </summary>

{langroid-0.1.132 → langroid-0.1.134}/README.md

@@ -11,7 +11,7 @@
 [![Multi-Architecture DockerHub](https://github.com/langroid/langroid/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/langroid/langroid/actions/workflows/docker-publish.yml)
 
 [![Static Badge](https://img.shields.io/badge/Documentation-blue?link=https%3A%2F%2Flangroid.github.io%2Flangroid%2F&link=https%3A%2F%2Flangroid.github.io%2Flangroid%2F)](https://langroid.github.io/langroid)
-[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/langroid_quick_examples.ipynb)
+[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
 [![Discord](https://img.shields.io/badge/Discord-%235865F2.svg?style=flat&logo=discord&logoColor=white)](https://discord.gg/ZU36McDgDs)
 [![Substack](https://img.shields.io/badge/Substack-%23006f5c.svg?style=flat&logo=substack&logoColor=FF6719)](https://langroid.substack.com/p/langroid-harness-llms-with-multi-agent-programming)
 </div>
@@ -49,7 +49,7 @@ We welcome contributions -- See the [contributions](./CONTRIBUTING.md) document
 for ideas on what to contribute.
 
 
-Building LLM Applications? Prasad Chalasani is available for consulting
+Building LLM Applications? [Prasad Chalasani](https://www.linkedin.com/in/pchalasani/) is available for consulting
 (advisory/development): pchalasani at gmail dot com.
 
 Sponsorship is also accepted via [GitHub Sponsors](https://github.com/sponsors/langroid)
@@ -59,12 +59,12 @@ Sponsorship is also accepted via [GitHub Sponsors](https://github.com/sponsors/l
 # Quick glimpse of coding with Langroid
 This is just a teaser; there's much more, like function-calling/tools, 
 Multi-Agent Collaboration, Structured Information Extraction, DocChatAgent 
-(RAG), SQLChatAgent, etc. Scroll down or see docs for more.
-
-:fire: Just released! See this [Colab](https://colab.research.google.com/drive/190Tk7t4AdY1P9F_NlZ33-YEoGnHweQQ0) 
-for a walk-through of the new `OpenAIAssistant` class (with near-complete support for the new OpenAI Assistants API) 
-in a multi-agent setting.
+(RAG), SQLChatAgent, non-OpenAI local/remote LLMs, etc. Scroll down or see docs for more.
 
+:fire: Just released! Updated Langroid Quick-Start [Colab](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
+that builds up to a 2-agent chat example using the OpenAI ChatCompletion API. 
+See also this [version](https://colab.research.google.com/drive/190Tk7t4AdY1P9F_NlZ33-YEoGnHweQQ0) 
+that uses the OpenAI Assistants API instead.
 
 ```python
 from langroid.language_models import  OpenAIGPTConfig, OpenAIChatModel, OpenAIGPT
@@ -175,7 +175,7 @@ See [this test](tests/main/test_recipient_tool.py) for example usage.
   - **Example:** [Answer questions](examples/docqa/chat-search.py) using Google Search + vecdb-retrieval from URL contents. 
   - **0.1.39:** [`GoogleSearchTool`](langroid/agent/tools/google_search_tool.py) to enable Agents (their LLM) to do Google searches via function-calling/tools.
     See [this chat example](examples/basic/chat-search.py) for how easy it is to add this tool to an agent.
-  - **Colab notebook** to try the quick-start examples: [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/langroid_quick_examples.ipynb) 
+  - **Colab notebook** to try the quick-start examples: [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb) 
   - **0.1.37:** Added [`SQLChatAgent`](langroid/agent/special/sql_chat_agent.py) -- thanks to our latest contributor [Rithwik Babu](https://github.com/rithwikbabu)!
   - Multi-agent Example: [Autocorrect chat](examples/basic/autocorrect.py)
 - **July 2023:** 
@@ -426,7 +426,7 @@ for a detailed tutorial.
 
 Click to expand any of the code examples below.
 All of these can be run in a Colab notebook:
-[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/langroid_quick_examples.ipynb)
+[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langroid/langroid/blob/main/examples/Langroid_quick_start.ipynb)
 
 <details>
 <summary> <b> Direct interaction with OpenAI LLM </b> </summary>

{langroid-0.1.132 → langroid-0.1.134}/langroid/agent/base.py

@@ -143,6 +143,9 @@ class Agent(ABC):
     def get_dialog(self) -> List[Tuple[str, str]]:
         return self.dialog
 
+    def clear_dialog(self) -> None:
+        self.dialog = []
+
     def _get_tool_list(
         self, message_class: Optional[Type[ToolMessage]] = None
     ) -> List[str]:

{langroid-0.1.132 → langroid-0.1.134}/langroid/agent/task.py

@@ -100,6 +100,7 @@ class Task:
         if isinstance(agent, ChatAgent) and len(agent.message_history) == 0 or restart:
             agent = cast(ChatAgent, agent)
             agent.clear_history(0)
+            agent.clear_dialog()
             # possibly change the system and user messages
             if system_message:
                 # we always have at least 1 task_message

{langroid-0.1.132 → langroid-0.1.134}/langroid/language_models/openai_gpt.py

@@ -233,13 +233,21 @@ class OpenAIGPT(LanguageModel):
                 # switch to fresh momento config if needed
                 config.cache_config = MomentoCacheConfig()
             self.cache = MomentoCache(config.cache_config)
-        else:
+        elif "redis" in settings.cache_type:
             if config.cache_config is None or isinstance(
                 config.cache_config, MomentoCacheConfig
             ):
                 # switch to fresh redis config if needed
-                config.cache_config = RedisCacheConfig()
+                config.cache_config = RedisCacheConfig(
+                    fake="fake" in settings.cache_type
+                )
             self.cache = RedisCache(config.cache_config)
+            config.cache_config.fake = "fake" in settings.cache_type
+        else:
+            raise ValueError(
+                f"Invalid cache type {settings.cache_type}. "
+                "Valid types are momento, redis, fakeredis"
+            )
 
         self.config._validate_litellm()
 

{langroid-0.1.132 → langroid-0.1.134}/langroid/parsing/parser.py

@@ -1,6 +1,6 @@
 import logging
 from enum import Enum
-from typing import List
+from typing import Dict, List
 
 import tiktoken
 from pydantic import BaseSettings
@@ -57,22 +57,34 @@ class Parser:
 
         # The original metadata.id (if any) is ignored since it will be same for all
         # chunks and is useless. We want a distinct id for each chunk.
+        orig_ids = [c.metadata.id for c in chunks]
         ids = [Document.hash_id(str(c)) for c in chunks]
 
+        # group the ids by orig_id
+        orig_id_to_ids: Dict[str, List[str]] = {}
+        for orig_id, id in zip(orig_ids, ids):
+            if orig_id not in orig_id_to_ids:
+                orig_id_to_ids[orig_id] = []  # type: ignore
+            orig_id_to_ids[orig_id].append(id)  # type: ignore
+
+        # now each orig_id maps to a sequence of ids within a single doc
+
         k = self.config.n_neighbor_ids
-        n = len(ids)
-        window_ids = [ids[max(0, i - k) : min(n, i + k + 1)] for i in range(n)]
-        for i, c in enumerate(chunks):
-            if c.content.strip() == "":
-                continue
-            c.metadata.window_ids = window_ids[i]
-            c.metadata.id = ids[i]
-            c.metadata.is_chunk = True
+        for orig, ids in orig_id_to_ids.items():
+            n = len(ids)
+            window_ids = [ids[max(0, i - k) : min(n, i + k + 1)] for i in range(n)]
+            for i, c in enumerate(chunks):
+                if c.content.strip() == "":
+                    continue
+                c.metadata.window_ids = window_ids[i]
+                c.metadata.id = ids[i]
+                c.metadata.is_chunk = True
 
     def split_simple(self, docs: List[Document]) -> List[Document]:
         if len(self.config.separators) == 0:
             raise ValueError("Must have at least one separator")
         final_docs = []
+
         for d in docs:
             if d.content.strip() == "":
                 continue
@@ -240,6 +252,11 @@ class Parser:
     def split(self, docs: List[Document]) -> List[Document]:
         if len(docs) == 0:
             return []
+        # create ids in metadata of docs if absent:
+        # we need this to distinguish docs later in add_window_ids
+        for d in docs:
+            if d.metadata.id is None:
+                d.metadata.id = d.id()
         # some docs are already splits, so don't split them further!
         chunked_docs = [d for d in docs if d.metadata.is_chunk]
         big_docs = [d for d in docs if not d.metadata.is_chunk]

{langroid-0.1.132 → langroid-0.1.134}/langroid/utils/configuration.py

@@ -1,7 +1,7 @@
 import copy
 import os
 from contextlib import contextmanager
-from typing import Iterator, List
+from typing import Iterator, List, Literal
 
 from dotenv import find_dotenv, load_dotenv
 from pydantic import BaseSettings
@@ -14,7 +14,7 @@ class Settings(BaseSettings):
     progress: bool = False  # show progress spinners/bars?
     stream: bool = True  # stream output?
     cache: bool = True  # use cache?
-    cache_type: str = "redis"  # cache type: "redis" or "momento"
+    cache_type: Literal["redis", "fakeredis", "momento"] = "redis"  # cache type
     interactive: bool = True  # interactive mode?
     gpt3_5: bool = True  # use GPT-3.5?
     nofunc: bool = False  # use model without function_call? (i.e. gpt-4)

{langroid-0.1.132 → langroid-0.1.134}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "langroid"
-version = "0.1.132"
+version = "0.1.134"
 description = "Harness LLMs with Multi-Agent Programming"
 authors = ["Prasad Chalasani <pchalasani@gmail.com>"]
 readme = "README.md"