gllm-inference-binary 0.5.51__cp311-cp311-win_amd64.whl → 0.5.66__cp311-cp311-win_amd64.whl

gllm_inference/lm_invoker/portkey_lm_invoker.pyi

@@ -102,49 +102,138 @@ class PortkeyLMInvoker(OpenAIChatCompletionsLMInvoker):
         result = await lm_invoker.invoke([text, image])
         ```
 
+    Text output:
+        The `PortkeyLMInvoker` generates text outputs by default.
+        Text outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `texts` (all text outputs) or `text` (first text output) properties.
+
+        Output example:
+        ```python
+        LMOutput(outputs=[LMOutputItem(type="text", output="Hello, there!")])
+        ```
+
+    Structured output:
+        The `PortkeyLMInvoker` can be configured to generate structured outputs.
+        This feature can be enabled by providing a schema to the `response_schema` parameter.
+
+        Structured outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `structureds` (all structured outputs) or `structured` (first structured output) properties.
+
+        The schema must either be one of the following:
+        1. A Pydantic BaseModel class
+            The structured output will be a Pydantic model.
+        2. A JSON schema dictionary
+            JSON dictionary schema must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
+            Thus, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
+            The structured output will be a dictionary.
+
+        Usage example:
+        ```python
+        class Animal(BaseModel):
+            name: str
+            color: str
+
+        json_schema = Animal.model_json_schema()
+
+        lm_invoker = PortkeyLMInvoker(..., response_schema=Animal)  # Using Pydantic BaseModel class
+        lm_invoker = PortkeyLMInvoker(..., response_schema=json_schema)  # Using JSON schema dictionary
+        ```
+
+        Output example:
+        ```python
+        # Using Pydantic BaseModel class outputs a Pydantic model
+        LMOutput(outputs=[LMOutputItem(type="structured", output=Animal(name="dog", color="white"))])
+
+        # Using JSON schema dictionary outputs a dictionary
+        LMOutput(outputs=[LMOutputItem(type="structured", output={"name": "dog", "color": "white"})])
+        ```
+
+        When structured output is enabled, streaming is disabled.
+
     Tool calling:
-        Tools can be provided via the `tools` parameter to enable tool invocation.
+        The `PortkeyLMInvoker` can be configured to call tools to perform certain tasks.
+        This feature can be enabled by providing a list of `Tool` objects to the `tools` parameter.
+
+        Tool calls outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `tool_calls` property.
 
+        Usage example:
         ```python
         lm_invoker = PortkeyLMInvoker(..., tools=[tool_1, tool_2])
         ```
+
         Output example:
         ```python
         LMOutput(
-            response="Let me call the tools...",
-            tool_calls=[
-                ToolCall(id="123", name="tool_1", args={"key": "value"}),
+            outputs=[
+                LMOutputItem(type="text", output="I\'m using tools..."),
+                LMOutputItem(type="tool_call", output=ToolCall(id="123", name="tool_1", args={"key": "value"})),
+                LMOutputItem(type="tool_call", output=ToolCall(id="456", name="tool_2", args={"key": "value"})),
             ]
         )
         ```
 
-    Structured output:
-        The `response_schema` parameter enables structured responses (Pydantic BaseModel or JSON schema).
+    Thinking:
+        The `PortkeyLMInvoker` can be configured to perform step-by-step thinking process before answering.
+        This feature can be enabled by setting the `thinking` parameter to `True`.
 
+        Thinking outputs are stored in the `outputs` attribute of the `LMOutput` object
+        and can be accessed via the `thinkings` property.
+
+        Usage example:
         ```python
-        class Animal(BaseModel):
-            name: str
-            color: str
-        lm_invoker = PortkeyLMInvoker(..., response_schema=Animal)
+        lm_invoker = PortkeyLMInvoker(..., thinking=True, thinking_budget=1024)
         ```
+
         Output example:
         ```python
-        LMOutput(structured_output=Animal(name="Golden retriever", color="Golden"))
+        LMOutput(
+            outputs=[
+                LMOutputItem(type="thinking", output=Reasoning(type="thinking", reasoning="I\'m thinking...", ...)),
+                LMOutputItem(type="text", output="Golden retriever is a good dog breed."),
+            ]
+        )
         ```
 
+        Streaming output example:
+        ```python
+        {"type": "thinking_start", "value": "", ...}
+        {"type": "thinking", "value": "I\'m ", ...}
+        {"type": "thinking", "value": "thinking...", ...}
+        {"type": "thinking_end", "value": "", ...}
+        {"type": "response", "value": "Golden retriever ", ...}
+        {"type": "response", "value": "is a good dog breed.", ...}
+        ```
+        Note: By default, the thinking token will be streamed with the legacy `EventType.DATA` event type.
+        To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
+        LM invoker initialization. The legacy event format support will be removed in v0.6.
+
+        The amount of tokens allocated for the thinking process can be set via the `thinking_budget` parameter.
+        For more information, please refer to the following documentation:
+        https://portkey.ai/docs/product/ai-gateway/multimodal-capabilities/thinking-mode.
+
+        Thinking is only available for certain models depending on capabilities
+
     Analytics tracking:
-        When `output_analytics=True`, the invoker includes token usage, duration, and finish details.
+        The `PortkeyLMInvoker` can be configured to output additional information about the invocation.
+        This feature can be enabled by setting the `output_analytics` parameter to `True`.
+
+        When enabled, the following attributes will be stored in the output:
+        1. `token_usage`: The token usage.
+        2. `duration`: The duration in seconds.
+        3. `finish_details`: The details about how the generation finished.
 
+        Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
+            outputs=[...],
             token_usage=TokenUsage(input_tokens=100, output_tokens=50),
             duration=0.729,
-            finish_details={"finish_reason": "stop"},
+            finish_details={"stop_reason": "end_turn"},
         )
         ```
 
-        **Note:** When streaming is enabled, token usage analytics are not supported and will be `None`.
+        When streaming is enabled, token usage is not supported.
 
     Retry and timeout:
         The `PortkeyLMInvoker` supports retry and timeout configuration.
@@ -154,8 +243,6 @@ class PortkeyLMInvoker(OpenAIChatCompletionsLMInvoker):
         Retry config examples:
         ```python
         retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
-        retry_config = RetryConfig(max_retries=0, timeout=10.0)  # No retry, 10.0 seconds timeout
-        retry_config = RetryConfig(max_retries=5, timeout=None)  # 5 max retries, no timeout
         retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
         ```
 
@@ -163,57 +250,6 @@ class PortkeyLMInvoker(OpenAIChatCompletionsLMInvoker):
         ```python
         lm_invoker = PortkeyLMInvoker(..., retry_config=retry_config)
         ```
-
-    Thinking:
-        The `thinking` parameter enables enhanced reasoning capability for supported models.
-        Thinking mode allocates additional “reasoning tokens” up to `thinking_budget` (minimum 1024).
-        When enabled, the model’s reasoning trace is stored in the `reasoning` attribute.
-
-        ```python
-        lm_invoker = PortkeyLMInvoker(..., thinking=True, thinking_budget=1024)
-        ```
-        Output example:
-        ```python
-        LMOutput(
-            response="Golden retriever is a good dog breed.",
-            reasoning=[Reasoning(reasoning="Let me think about it...")],
-        )
-        ```
-
-        Streaming output example:
-        ```python
-        {"type": "thinking_start", "value": ""}
-        {"type": "thinking", "value": "Let me think "}
-        {"type": "thinking", "value": "about it..."}
-        {"type": "thinking_end", "value": ""}
-        {"type": "response", "value": "Golden retriever "}
-        {"type": "response", "value": "is a good dog breed."}
-        ```
-
-        Note: By default, the thinking token will be streamed with the legacy `EventType.DATA` event type.
-        To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
-        LM invoker initialization. The legacy event format support will be removed in v0.6.
-
-        When thinking is enabled, the amount of tokens allocated for the thinking process can be set via the
-        `thinking_budget` parameter. The `thinking_budget`:
-        1. Must be a positive integer.
-        2. Must be at least 1024.
-        3. Must be less than or equal to the model\'s maximum context length.
-        For more information, please refer to https://portkey.ai/docs/product/ai-gateway/multimodal-capabilities/thinking-mode
-
-        Setting reasoning-related parameters for non-reasoning models will raise an error.
-
-    Output types:
-        The output of the `PortkeyLMInvoker` can either be:
-        1. `str`: A simple text response.
-        2. `LMOutput`: A structured response model that may contain:
-            2.1. response (str)
-            2.2. tool_calls (list[ToolCall])
-            2.3. structured_output (dict[str, Any] | BaseModel | None)
-            2.4. token_usage (TokenUsage | None)
-            2.5. duration (float | None)
-            2.6. finish_details (dict[str, Any] | None)
-            2.7. reasoning (list[Reasoning])
     '''
     model_kwargs: Incomplete
     thinking: Incomplete

gllm_inference/lm_invoker/schema/google.pyi

@@ -7,8 +7,10 @@ class Key:
     FUNCTION: str
     FUNCTION_CALL: str
     HTTP_OPTIONS: str
+    ID: str
     NAME: str
     RETRY_OPTIONS: str
+    STATUS: str
     SYSTEM_INSTRUCTION: str
     THINKING_CONFIG: str
     TIMEOUT: str
@@ -16,9 +18,19 @@ class Key:
     RESPONSE_SCHEMA: str
     RESPONSE_MIME_TYPE: str
     VERTEXAI: str
+    CUSTOM_REQUEST_IDS: str
 
 class InputType:
     """Defines valid input types in Google."""
     APPLICATION_JSON: str
     MODEL: str
     USER: str
+
+class JobState:
+    """Defines valid output types in Google."""
+    JOB_STATE_CANCELLED: str
+    JOB_STATE_EXPIRED: str
+    JOB_STATE_FAILED: str
+    JOB_STATE_PENDING: str
+    JOB_STATE_RUNNING: str
+    JOB_STATE_SUCCEEDED: str

gllm_inference/lm_invoker/schema/openai.pyi

@@ -7,9 +7,11 @@ class Key:
     ARGS: str
     ARGUMENTS: str
     BASE_URL: str
+    BODY: str
     CALL_ID: str
     CONTAINER: str
     CONTENT: str
+    CUSTOM_ID: str
     DEFAULT: str
     DEFS: str
     DESCRIPTION: str
@@ -18,17 +20,24 @@ class Key:
     FILENAME: str
     FORMAT: str
     ID: str
+    IMAGE_GENERATION_CALL: str
     IMAGE_URL: str
     INCLUDE: str
     INCOMPLETE_DETAILS: str
+    INPUT: str
     INSTRUCTIONS: str
     JSON_SCHEMA: str
     MAX_RETRIES: str
+    METHOD: str
+    METHOD_POST: str
+    MODEL: str
     NAME: str
     OUTPUT: str
+    OUTPUTS: str
     PARAMETERS: str
     REASON: str
     REASONING: str
+    REFUSAL: str
     ROLE: str
     SCHEMA: str
     REQUIRE_APPROVAL: str
@@ -45,6 +54,8 @@ class Key:
     TOOL_NAME: str
     TOOLS: str
     TYPE: str
+    URL: str
+    USAGE: str
 
 class InputType:
     """Defines valid input types in OpenAI."""
@@ -54,6 +65,7 @@ class InputType:
     FUNCTION: str
     FUNCTION_CALL: str
     FUNCTION_CALL_OUTPUT: str
+    IMAGE_GENERATION: str
     INPUT_FILE: str
     INPUT_IMAGE: str
     INPUT_TEXT: str
@@ -69,21 +81,30 @@ class InputType:
 
 class OutputType:
     """Defines valid output types in OpenAI."""
+    CANCELLED: str
+    CANCELLING: str
     CODE_INTERPRETER_CALL: str
     CODE_INTERPRETER_CALL_DELTA: str
     CODE_INTERPRETER_CALL_DONE: str
     CODE_INTERPRETER_CALL_IN_PROGRESS: str
     COMPLETED: str
+    COMPLETED_BATCH: str
     CONTAINER_FILE_CITATION: str
+    EXPIRED: str
+    FAILED: str
+    FINALIZING: str
     FIND_IN_PAGE: str
     FUNCTION_CALL: str
     IMAGE: str
+    IMAGE_GENERATION_CALL: str
     INCOMPLETE: str
+    IN_PROGRESS: str
     ITEM_DONE: str
     MCP_CALL: str
     MCP_LIST_TOOLS: str
     MESSAGE: str
     OPEN_PAGE: str
+    PROCESSED: str
     REASONING: str
     REASONING_ADDED: str
     REASONING_DELTA: str
@@ -91,6 +112,7 @@ class OutputType:
     REFUSAL: str
     SEARCH: str
     TEXT_DELTA: str
+    VALIDATING: str
     WEB_SEARCH_CALL: str
 
 class ReasoningEffort(StrEnum):

gllm_inference/lm_invoker/sea_lion_lm_invoker.pyi

@@ -0,0 +1,48 @@
+from _typeshed import Incomplete
+from gllm_core.schema.tool import Tool as Tool
+from gllm_core.utils import RetryConfig as RetryConfig
+from gllm_inference.constants import INVOKER_PROPAGATED_MAX_RETRIES as INVOKER_PROPAGATED_MAX_RETRIES
+from gllm_inference.lm_invoker.openai_chat_completions_lm_invoker import OpenAIChatCompletionsLMInvoker as OpenAIChatCompletionsLMInvoker
+from gllm_inference.lm_invoker.schema.openai_chat_completions import Key as Key
+from gllm_inference.schema import ModelId as ModelId, ModelProvider as ModelProvider, ResponseSchema as ResponseSchema
+from langchain_core.tools import Tool as LangChainTool
+from typing import Any
+
+SEA_LION_URL: str
+SUPPORTED_ATTACHMENTS: Incomplete
+
+class SeaLionLMInvoker(OpenAIChatCompletionsLMInvoker):
+    """A language model invoker to interact with SEA-LION API.
+
+    Attributes:
+        model_id (str): The model ID of the language model.
+        model_provider (str): The provider of the language model.
+        model_name (str): The name of the language model.
+        client_kwargs (dict[str, Any]): The keyword arguments for the OpenAI client.
+        default_hyperparameters (dict[str, Any]): Default hyperparameters for invoking the model.
+        tools (list[Tool]): The list of tools provided to the model to enable tool calling.
+        response_schema (ResponseSchema | None): The schema of the response. If provided, the model will output a
+            structured response as defined by the schema. Supports both Pydantic BaseModel and JSON schema dictionary.
+        output_analytics (bool): Whether to output the invocation analytics.
+        retry_config (RetryConfig | None): The retry configuration for the language model.
+    """
+    client_kwargs: Incomplete
+    def __init__(self, model_name: str, api_key: str | None = None, model_kwargs: dict[str, Any] | None = None, default_hyperparameters: dict[str, Any] | None = None, tools: list[Tool | LangChainTool] | None = None, response_schema: ResponseSchema | None = None, output_analytics: bool = False, retry_config: RetryConfig | None = None) -> None:
+        """Initializes a new instance of the SeaLionLMInvoker class.
+
+        Args:
+            model_name (str): The name of the SEA-LION language model.
+            api_key (str | None, optional): The API key for authenticating with the SEA-LION API.
+                Defaults to None, in which case the `SEA_LION_API_KEY` environment variable will be used.
+            model_kwargs (dict[str, Any] | None, optional): Additional model parameters. Defaults to None.
+            default_hyperparameters (dict[str, Any] | None, optional): Default hyperparameters for invoking the model.
+                Defaults to None.
+            tools (list[Tool | LangChainTool] | None, optional): Tools provided to the model to enable tool calling.
+                Defaults to None.
+            response_schema (ResponseSchema | None, optional): The schema of the response. If provided, the model will
+                output a structured response as defined by the schema. Supports both Pydantic BaseModel and JSON schema
+                dictionary. Defaults to None.
+            output_analytics (bool, optional): Whether to output the invocation analytics. Defaults to False.
+            retry_config (RetryConfig | None, optional): The retry configuration for the language model.
+                Defaults to None, in which case a default config with no retry and 30.0 seconds timeout will be used.
+        """