chatlas 0.11.1__tar.gz → 0.13.0__tar.gz

{chatlas-0.11.1 → chatlas-0.13.0}/.vscode/settings.json

@@ -7,5 +7,12 @@
         },
         "editor.defaultFormatter": "charliermarsh.ruff",
     },
-    "flake8.args": ["--max-line-length=120"]
-}
+    "flake8.args": [
+      "--max-line-length=120"
+    ],
+    "python.testing.pytestArgs": [
+      "tests"
+    ],
+    "python.testing.unittestEnabled": false,
+    "python.testing.pytestEnabled": true
+}

{chatlas-0.11.1 → chatlas-0.13.0}/CHANGELOG.md

@@ -7,6 +7,38 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 -->
 
+## [0.13.0] - 2025-09-10
+
+### New features
+
+* Added support for submitting multiple chats in one batch. With batch submission, results can take up to 24 hours to complete, but in return you pay ~50% less than usual. For more, see the [reference](https://posit-dev.github.io/chatlas/reference/) for `batch_chat()`, `batch_chat_text()`, `batch_chat_structured()` and `batch_chat_completed()`. (#177)
+* The `Chat` class gains new `.chat_structured()` (and `.chat_structured_async()`) methods. These methods supersede the now deprecated `.extract_data()` (and `.extract_data_async()`). The only difference is that the new methods return a `BaseModel` instance (instead of a `dict()`), leading to a better type hinting/checking experience.  (#175)
+* The `.get_turns()` method gains a `tool_result_role` parameter. Set `tool_result_role="assistant"` to collect tool result content (plus the surrounding assistant turn contents) into a single assistant turn. This is convenient for display purposes and more generally if you want the tool calling loop to be contained in a single turn. (#179) 
+
+### Improvements
+
+* The `.app()` method now:
+    * Enables bookmarking by default (i.e., chat session survives page reload). (#179)
+    * Correctly renders pre-existing turns that contain tool calls. (#179)
+
+## [0.12.0] - 2025-09-08
+
+### Breaking changes
+
+* `ChatAuto()`'s first (optional) positional parameter has changed from `system_prompt` to `provider_model`, and `system_prompt` is now a keyword parameter. As a result, you may need to change `ChatAuto("[system prompt]")` -> `ChatAuto(system_prompt="[system prompt]")`. In addition, the `provider` and `model` keyword arguments are now deprecated, but continue to work with a warning, as are the previous `CHATLAS_CHAT_PROVIDER` and `CHATLAS_CHAT_MODEL` environment variables. (#159)
+
+### New features
+
+* `ChatAuto()`'s new `provider_model` takes both provider and model in a single string in the format `"{provider}/{model}"`, e.g. `"openai/gpt-5"`. If not provided, `ChatAuto()` looks for the `CHATLAS_CHAT_PROVIDER_MODEL` environment variable, defaulting to `"openai"` if neither are provided. Unlike previous versions of `ChatAuto()`, the environment variables are now used *only if function arguments are not provided*. In other words, if `provider_model` is given, the `CHATLAS_CHAT_PROVIDER_MODEL` environment variable is ignored. Similarly, `CHATLAS_CHAT_ARGS` are only used if no `kwargs` are provided. This improves interactive use cases, makes it easier to introduce application-specific environment variables, and puts more control in the hands of the developer. (#159)
+* The `.register_tool()` method now: 
+  * Accepts a `Tool` instance as input. This is primarily useful for binding things like `annotations` to the `Tool` in one place, and registering it in another. (#172)
+  * Supports function parameter names that start with an underscore. (#174)
+* The `ToolAnnotations` type gains an `extra` key field -- providing a place for providing additional information that other consumers of tool annotations (e.g., [shinychat](https://posit-dev.github.io/shinychat/)) may make use of.
+
+### Bug fixes
+
+* `ChatAuto()` now supports recently added providers such as `ChatCloudflare()`, `ChatDeepseek()`, `ChatHuggingFace()`, etc. (#159)
+
 ## [0.11.1] - 2025-08-29
 
 ### New features
@@ -22,7 +54,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 * `.register_tool(annotations=annotations)` drops support for `mcp.types.ToolAnnotations()` and instead expects a dictionary of the same info. (#164)
 
-
 ## [0.11.0] - 2025-08-26
 
 ### New features
@@ -42,7 +73,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### New features
 
-* Added `ChatCloudflare()` for chatting via [Cloudflare AI](https://developers.cloudflare.com/workers-ai/get-started/rest-api/). (#150) 
+* Added `ChatCloudflare()` for chatting via [Cloudflare AI](https://developers.cloudflare.com/workers-ai/get-started/rest-api/). (#150)
 * Added `ChatDeepSeek()` for chatting via [DeepSeek](https://www.deepseek.com/). (#147)
 * Added `ChatOpenRouter()` for chatting via [Open Router](https://openrouter.ai/). (#148)
 * Added `ChatHuggingFace()` for chatting via [Hugging Face](https://huggingface.co/). (#144)
@@ -78,7 +109,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### New features
 
-* `Chat` gains a handful of new methods: 
+* `Chat` gains a handful of new methods:
     * `.register_mcp_tools_http_stream_async()` and `.register_mcp_tools_stdio_async()`: for registering tools from a [MCP server](https://modelcontextprotocol.io/). (#39)
     * `.get_tools()` and `.set_tools()`: for fine-grained control over registered tools. (#39)
     * `.set_model_params()`: for setting common LLM parameters in a model-agnostic fashion. (#127)
@@ -87,7 +118,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 * Tool functions passed to `.register_tool()` can now `yield` numerous results. (#39)
 * A `ContentToolResultImage` content class was added for returning images from tools. It is currently only works with `ChatAnthropic`. (#39)
 * A `Tool` can now be constructed from a pre-existing tool schema (via a new `__init__` method). (#39)
-* The `Chat.app()` method gains a `host` parameter. (#122) 
+* The `Chat.app()` method gains a `host` parameter. (#122)
 * `ChatGithub()` now supports the more standard `GITHUB_TOKEN` environment variable for storing the API key. (#123)
 
 ### Changes
@@ -149,7 +180,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [0.7.1] - 2025-05-10
 
-* Added `openai` as a hard dependency, making installation easier for a wide range of use cases. (#91) 
+* Added `openai` as a hard dependency, making installation easier for a wide range of use cases. (#91)
 
 ## [0.7.0] - 2025-04-22
 
@@ -159,7 +190,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 * `.stream()` and `.stream_async()` gain a `content` argument. Set this to `"all"` to include `ContentToolResult`/`ContentToolRequest` objects in the stream. (#75)
 * `ContentToolResult`/`ContentToolRequest` are now exported to `chatlas` namespace. (#75)
 * `ContentToolResult`/`ContentToolRequest` gain a `.tagify()` method so they render sensibly in a Shiny app. (#75)
-* A tool can now return a `ContentToolResult`. This is useful for: 
+* A tool can now return a `ContentToolResult`. This is useful for:
     * Specifying the format used for sending the tool result to the chat model (`model_format`). (#87)
     * Custom rendering of the tool result (by overriding relevant methods in a subclass). (#75)
 * `Chat` gains a new `.current_display` property. When a `.chat()` or `.stream()` is currently active, this property returns an object with a `.echo()` method (to echo new content to the display). This is primarily useful for displaying custom content during a tool call. (#79)

{chatlas-0.11.1 → chatlas-0.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: chatlas
-Version: 0.11.1
+Version: 0.13.0
 Summary: A simple and consistent interface for chatting with LLMs
 Project-URL: Homepage, https://posit-dev.github.io/chatlas
 Project-URL: Documentation, https://posit-dev.github.io/chatlas
@@ -44,6 +44,7 @@ Requires-Dist: pillow; extra == 'dev'
 Requires-Dist: python-dotenv; extra == 'dev'
 Requires-Dist: ruff>=0.6.5; extra == 'dev'
 Requires-Dist: shiny; extra == 'dev'
+Requires-Dist: shinychat; extra == 'dev'
 Requires-Dist: snowflake-ml-python>=1.8.4; extra == 'dev'
 Requires-Dist: tenacity; extra == 'dev'
 Requires-Dist: tiktoken; extra == 'dev'

{chatlas-0.11.1 → chatlas-0.13.0}/chatlas/__init__.py

@@ -1,5 +1,11 @@
 from . import types
 from ._auto import ChatAuto
+from ._batch_chat import (
+    batch_chat,
+    batch_chat_completed,
+    batch_chat_structured,
+    batch_chat_text,
+)
 from ._chat import Chat
 from ._content import (
     ContentToolRequest,
@@ -36,6 +42,10 @@ except ImportError:  # pragma: no cover
     __version__ = "0.0.0"  # stub value for docs
 
 __all__ = (
+    "batch_chat",
+    "batch_chat_completed",
+    "batch_chat_structured",
+    "batch_chat_text",
     "ChatAnthropic",
     "ChatAuto",
     "ChatBedrockAnthropic",

chatlas-0.13.0/chatlas/_auto.py

@@ -0,0 +1,290 @@
+from __future__ import annotations
+
+import os
+import warnings
+from typing import Callable, Literal, Optional
+
+import orjson
+
+from ._chat import Chat
+from ._provider_anthropic import ChatAnthropic, ChatBedrockAnthropic
+from ._provider_cloudflare import ChatCloudflare
+from ._provider_databricks import ChatDatabricks
+from ._provider_deepseek import ChatDeepSeek
+from ._provider_github import ChatGithub
+from ._provider_google import ChatGoogle, ChatVertex
+from ._provider_groq import ChatGroq
+from ._provider_huggingface import ChatHuggingFace
+from ._provider_mistral import ChatMistral
+from ._provider_ollama import ChatOllama
+from ._provider_openai import ChatAzureOpenAI, ChatOpenAI
+from ._provider_openrouter import ChatOpenRouter
+from ._provider_perplexity import ChatPerplexity
+from ._provider_portkey import ChatPortkey
+from ._provider_snowflake import ChatSnowflake
+from ._utils import MISSING_TYPE as DEPRECATED_TYPE
+
+AutoProviders = Literal[
+    "anthropic",
+    "bedrock-anthropic",
+    "cloudflare",
+    "databricks",
+    "deep-seek",
+    "github",
+    "google",
+    "groq",
+    "hugging-face",
+    "mistral",
+    "ollama",
+    "openai",
+    "azure-openai",
+    "open-router",
+    "perplexity",
+    "portkey",
+    "snowflake",
+    "vertex",
+]
+
+_provider_chat_model_map: dict[AutoProviders, Callable[..., Chat]] = {
+    "anthropic": ChatAnthropic,
+    "bedrock-anthropic": ChatBedrockAnthropic,
+    "cloudflare": ChatCloudflare,
+    "databricks": ChatDatabricks,
+    "deep-seek": ChatDeepSeek,
+    "github": ChatGithub,
+    "google": ChatGoogle,
+    "groq": ChatGroq,
+    "hugging-face": ChatHuggingFace,
+    "mistral": ChatMistral,
+    "ollama": ChatOllama,
+    "openai": ChatOpenAI,
+    "azure-openai": ChatAzureOpenAI,
+    "open-router": ChatOpenRouter,
+    "perplexity": ChatPerplexity,
+    "portkey": ChatPortkey,
+    "snowflake": ChatSnowflake,
+    "vertex": ChatVertex,
+}
+
+DEPRECATED = DEPRECATED_TYPE()
+
+
+def ChatAuto(
+    provider_model: Optional[str] = None,
+    *,
+    system_prompt: Optional[str] = None,
+    provider: AutoProviders | DEPRECATED_TYPE = DEPRECATED,
+    model: str | DEPRECATED_TYPE = DEPRECATED,
+    **kwargs,
+) -> Chat:
+    """
+    Chat with any provider.
+
+    This is a generic interface to all the other `Chat*()` functions, allowing
+    you to pick the provider (and model) with a simple string.
+
+    Prerequisites
+    -------------
+
+    ::: {.callout-note}
+    ## API key
+
+    Follow the instructions for the specific provider to obtain an API key.
+    :::
+
+    ::: {.callout-note}
+    ## Python requirements
+
+    Follow the instructions for the specific provider to install the required
+    Python packages.
+    :::
+
+    Examples
+    --------
+
+    `ChatAuto()` makes it easy to switch between different chat providers and models.
+
+    ```python
+    import pandas as pd
+    from chatlas import ChatAuto
+
+    # Default provider (OpenAI) & model
+    chat = ChatAuto()
+    print(chat.provider.name)
+    print(chat.provider.model)
+
+    # Different provider (Anthropic) & default model
+    chat = ChatAuto("anthropic")
+
+    # List models available through the provider
+    models = chat.list_models()
+    print(pd.DataFrame(models))
+
+    # Choose specific provider/model (Claude Sonnet 4)
+    chat = ChatAuto("anthropic/claude-sonnet-4-0")
+    ```
+
+    The default provider/model can also be controlled through an environment variable:
+
+    ```bash
+    export CHATLAS_CHAT_PROVIDER_MODEL="anthropic/claude-sonnet-4-0"
+    ```
+
+    ```python
+    from chatlas import ChatAuto
+
+    chat = ChatAuto()
+    print(chat.provider.name)   # anthropic
+    print(chat.provider.model)  # claude-sonnet-4-0
+    ```
+
+    For application-specific configurations, consider defining your own environment variables:
+
+    ```bash
+    export MYAPP_PROVIDER_MODEL="google/gemini-2.5-flash"
+    ```
+
+    And passing them to `ChatAuto()` as an alternative way to configure the provider/model:
+
+    ```python
+    import os
+    from chatlas import ChatAuto
+
+    chat = ChatAuto(os.getenv("MYAPP_PROVIDER_MODEL"))
+    print(chat.provider.name)   # google
+    print(chat.provider.model)  # gemini-2.5-flash
+    ```
+
+    Parameters
+    ----------
+    provider_model
+        The name of the provider and model to use in the format
+        `"{provider}/{model}"`. Providers are strings formatted in kebab-case,
+        e.g. to use `ChatBedrockAnthropic` set `provider="bedrock-anthropic"`,
+        and models are the provider-specific model names, e.g.
+        `"claude-3-7-sonnet-20250219"`. The `/{model}` portion may also be
+        omitted, in which case, the default model for that provider will be
+        used.
+
+        If no value is provided, the `CHATLAS_CHAT_PROVIDER_MODEL` environment
+        variable will be consulted for a fallback value. If this variable is also
+        not set, a default value of `"openai"` is used.
+    system_prompt
+        A system prompt to set the behavior of the assistant.
+    provider
+        Deprecated; use `provider_model` instead.
+    model
+        Deprecated; use `provider_model` instead.
+    **kwargs
+        Additional keyword arguments to pass to the `Chat` constructor. See the
+        documentation for each provider for more details on the available
+        options.
+
+        These arguments can also be provided via the `CHATLAS_CHAT_ARGS`
+        environment variable as a JSON string. When any additional arguments are
+        provided to `ChatAuto()`, the env var is ignored.
+
+        Note that `system_prompt` and `turns` can't be set via environment variables.
+        They must be provided/set directly to/on `ChatAuto()`.
+
+    Note
+    ----
+    If you want to work with a specific provider, but don't know what models are
+    available (or the exact model name), use
+    `ChatAuto('provider_name').list_models()` to list available models. Another
+    option is to use the provider more directly (e.g., `ChatAnthropic()`). There,
+    the `model` parameter may have type hints for available models.
+
+    Returns
+    -------
+    Chat
+        A chat instance using the specified provider.
+
+    Raises
+    ------
+    ValueError
+        If no valid provider is specified either through parameters or
+        environment variables.
+    """
+    if provider is not DEPRECATED:
+        warn_deprecated_param("provider")
+
+    if model is not DEPRECATED:
+        if provider is DEPRECATED:
+            raise ValueError(
+                "The `model` parameter is deprecated and cannot be used without the `provider` parameter. "
+                "Use `provider_model` instead."
+            )
+        warn_deprecated_param("model")
+
+    if provider_model is None:
+        provider_model = os.environ.get("CHATLAS_CHAT_PROVIDER_MODEL")
+
+    # Backwards compatibility: construct from old env vars as a fallback
+    if provider_model is None:
+        env_provider = get_legacy_env_var("CHATLAS_CHAT_PROVIDER", provider)
+        env_model = get_legacy_env_var("CHATLAS_CHAT_MODEL", model)
+
+        if env_provider:
+            provider_model = env_provider
+            if env_model:
+                provider_model += f"/{env_model}"
+
+    # Fall back to OpenAI if nothing is specified
+    if provider_model is None:
+        provider_model = "openai"
+
+    if "/" in provider_model:
+        the_provider, the_model = provider_model.split("/", 1)
+    else:
+        the_provider, the_model = provider_model, None
+
+    if the_provider not in _provider_chat_model_map:
+        raise ValueError(
+            f"Provider name '{the_provider}' is not a known chatlas provider: "
+            f"{', '.join(_provider_chat_model_map.keys())}"
+        )
+
+    # `system_prompt`, `turns` and `model` always come from `ChatAuto()`
+    base_args = {
+        "system_prompt": system_prompt,
+        "turns": None,
+        "model": the_model,
+    }
+
+    # Environment kwargs, used only if no kwargs provided
+    env_kwargs = {}
+    if not kwargs:
+        env_kwargs = orjson.loads(os.environ.get("CHATLAS_CHAT_ARGS", "{}"))
+
+    final_kwargs = {**env_kwargs, **kwargs, **base_args}
+    final_kwargs = {k: v for k, v in final_kwargs.items() if v is not None}
+
+    return _provider_chat_model_map[the_provider](**final_kwargs)
+
+
+def get_legacy_env_var(
+    env_var_name: str,
+    default: str | DEPRECATED_TYPE,
+) -> str | None:
+    env_value = os.environ.get(env_var_name)
+    if env_value:
+        warnings.warn(
+            f"The '{env_var_name}' environment variable is deprecated. "
+            "Use 'CHATLAS_CHAT_PROVIDER_MODEL' instead.",
+            DeprecationWarning,
+            stacklevel=3,
+        )
+        return env_value
+    elif isinstance(default, DEPRECATED_TYPE):
+        return None
+    else:
+        return default
+
+
+def warn_deprecated_param(param_name: str, stacklevel: int = 3) -> None:
+    warnings.warn(
+        f"The '{param_name}' parameter is deprecated. Use 'provider_model' instead.",
+        DeprecationWarning,
+        stacklevel=stacklevel,
+    )

chatlas-0.13.0/chatlas/_batch_chat.py

@@ -0,0 +1,211 @@
+"""
+Batch chat processing for submitting multiple requests simultaneously.
+
+This module provides functionality for submitting multiple chat requests
+in batches to providers that support it (currently OpenAI and Anthropic).
+Batch processing can take up to 24 hours but offers significant cost savings
+(up to 50% less than regular requests).
+"""
+
+from __future__ import annotations
+
+import copy
+from pathlib import Path
+from typing import TypeVar, Union
+
+from pydantic import BaseModel
+
+from ._batch_job import BatchJob, ContentT
+from ._chat import Chat
+
+ChatT = TypeVar("ChatT", bound=Chat)
+BaseModelT = TypeVar("BaseModelT", bound=BaseModel)
+
+
+def batch_chat(
+    chat: ChatT,
+    prompts: list[ContentT] | list[list[ContentT]],
+    path: Union[str, Path],
+    wait: bool = True,
+) -> list[ChatT | None]:
+    """
+    Submit multiple chat requests in a batch.
+
+    This function allows you to submit multiple chat requests simultaneously
+    using provider batch APIs (currently OpenAI and Anthropic). Batch processing
+    can take up to 24 hours but offers significant cost savings.
+
+    Parameters
+    ----------
+    chat
+        Chat instance to use for the batch
+    prompts
+        List of prompts to process. Each can be a string or list of strings.
+    path
+        Path to file (with .json extension) to store batch state
+    wait
+        If True, wait for batch to complete. If False, return None if incomplete.
+
+    Returns
+    -------
+    List of Chat objects (one per prompt) if complete, None if wait=False and incomplete.
+    Individual Chat objects may be None if their request failed.
+
+    Example
+    -------
+
+    ```python
+    from chatlas import ChatOpenAI
+
+    chat = ChatOpenAI()
+    prompts = [
+        "What's the capital of France?",
+        "What's the capital of Germany?",
+        "What's the capital of Italy?",
+    ]
+
+    chats = batch_chat(chat, prompts, "capitals.json")
+    for i, result_chat in enumerate(chats):
+        if result_chat:
+            print(f"Prompt {i + 1}: {result_chat.get_last_turn().text}")
+    ```
+    """
+    job = BatchJob(chat, prompts, path, wait=wait)
+    job.step_until_done()
+
+    chats = []
+    assistant_turns = job.result_turns()
+    for user, assistant in zip(job.user_turns, assistant_turns):
+        if assistant is not None:
+            new_chat = copy.deepcopy(chat)
+            new_chat.add_turn(user)
+            new_chat.add_turn(assistant)
+            chats.append(new_chat)
+        else:
+            chats.append(None)
+
+    return chats
+
+
+def batch_chat_text(
+    chat: Chat,
+    prompts: list[ContentT] | list[list[ContentT]],
+    path: Union[str, Path],
+    wait: bool = True,
+) -> list[str | None]:
+    """
+    Submit multiple chat requests in a batch and return text responses.
+
+    This is a convenience function that returns just the text of the responses
+    rather than full Chat objects.
+
+    Parameters
+    ----------
+    chat
+        Chat instance to use for the batch
+    prompts
+        List of prompts to process
+    path
+        Path to file (with .json extension) to store batch state
+    wait
+        If True, wait for batch to complete
+
+    Return
+    ------
+    List of text responses (or None for failed requests)
+    """
+    chats = batch_chat(chat, prompts, path, wait=wait)
+
+    texts = []
+    for x in chats:
+        if x is None:
+            texts.append(None)
+            continue
+        last_turn = x.get_last_turn()
+        if last_turn is None:
+            texts.append(None)
+            continue
+        texts.append(last_turn.text)
+
+    return texts
+
+
+def batch_chat_structured(
+    chat: Chat,
+    prompts: list[ContentT] | list[list[ContentT]],
+    path: Union[str, Path],
+    data_model: type[BaseModelT],
+    wait: bool = True,
+) -> list[BaseModelT | None]:
+    """
+    Submit multiple structured data requests in a batch.
+
+    Parameters
+    ----------
+    chat
+        Chat instance to use for the batch
+    prompts
+        List of prompts to process
+    path
+        Path to file (with .json extension) to store batch state
+    data_model
+        Pydantic model class for structured responses
+    wait
+        If True, wait for batch to complete
+
+    Return
+    ------
+    List of structured data objects (or None for failed requests)
+    """
+    job = BatchJob(chat, prompts, path, data_model=data_model, wait=wait)
+    result = job.step_until_done()
+
+    if result is None:
+        return []
+
+    res: list[BaseModelT | None] = []
+    assistant_turns = job.result_turns()
+    for turn in assistant_turns:
+        if turn is None:
+            res.append(None)
+        else:
+            json = chat._extract_turn_json(turn)
+            model = data_model.model_validate(json)
+            res.append(model)
+
+    return res
+
+
+def batch_chat_completed(
+    chat: Chat,
+    prompts: list[ContentT] | list[list[ContentT]],
+    path: Union[str, Path],
+) -> bool:
+    """
+    Check if a batch job is completed without waiting.
+
+    Parameters
+    ----------
+    chat
+        Chat instance used for the batch
+    prompts
+        List of prompts used for the batch
+    path
+        Path to batch state file
+
+    Returns
+    -------
+    True if batch is complete, False otherwise
+    """
+    job = BatchJob(chat, prompts, path, wait=False)
+    stage = job.stage
+
+    if stage == "submitting":
+        return False
+    elif stage == "waiting":
+        status = job._poll()
+        return not status.working
+    elif stage == "retrieving" or stage == "done":
+        return True
+    else:
+        raise ValueError(f"Unknown batch stage: {stage}")