gllm-inference-binary 0.5.52__cp312-cp312-win_amd64.whl → 0.5.54__cp312-cp312-win_amd64.whl

gllm_inference/lm_invoker/bedrock_lm_invoker.pyi

@@ -51,83 +51,82 @@ class BedrockLMInvoker(BaseLMInvoker):
         result = await lm_invoker.invoke([text, image])
         ```
 
-    Tool calling:
-        Tool calling is a feature that allows the language model to call tools to perform tasks.
-        Tools can be passed to the via the `tools` parameter as a list of `Tool` objects.
-        When tools are provided and the model decides to call a tool, the tool calls are stored in the
-        `tool_calls` attribute in the output.
-
-        Usage example:
-        ```python
-        lm_invoker = BedrockLMInvoker(..., tools=[tool_1, tool_2])
-        ```
+    Text output:
+        The `BedrockLMInvoker` generates text outputs by default.
+        Text outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `texts` (all text outputs) or `text` (first text output) properties.
 
         Output example:
         ```python
-        LMOutput(
-            response="Let me call the tools...",
-            tool_calls=[
-                ToolCall(id="123", name="tool_1", args={"key": "value"}),
-                ToolCall(id="456", name="tool_2", args={"key": "value"}),
-            ]
-        )
+        LMOutput(outputs=[LMOutputItem(type="text", output="Hello, there!")])
         ```
 
     Structured output:
-        Structured output is a feature that allows the language model to output a structured response.
+        The `BedrockLMInvoker` can be configured to generate structured outputs.
         This feature can be enabled by providing a schema to the `response_schema` parameter.
 
-        The schema must be either a JSON schema dictionary or a Pydantic BaseModel class.
-        If JSON schema is used, it must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
-        For this reason, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
-
-        Structured output is achieved by providing the schema name in the `tool_choice` parameter. This forces
-        the model to call the provided schema as a tool. Thus, structured output is not compatible with tool calling,
-        since the tool calling is reserved to force the model to call the provided schema as a tool.
-        The language model also doesn\'t need to stream anything when structured output is enabled. Thus, standard
-        invocation will be performed regardless of whether the `event_emitter` parameter is provided or not.
+        Structured outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `structureds` (all structured outputs) or `structured` (first structured output) properties.
 
-        When enabled, the structured output is stored in the `structured_output` attribute in the output.
-        1. If the schema is a JSON schema dictionary, the structured output is a dictionary.
-        2. If the schema is a Pydantic BaseModel class, the structured output is a Pydantic model.
+        The schema must either be one of the following:
+        1. A Pydantic BaseModel class
+            The structured output will be a Pydantic model.
+        2. A JSON schema dictionary
+            JSON dictionary schema must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
+            Thus, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
+            The structured output will be a dictionary.
 
-        # Example 1: Using a JSON schema dictionary
         Usage example:
         ```python
-        schema = {
-            "title": "Animal",
-            "description": "A description of an animal.",
-            "properties": {
-                "color": {"title": "Color", "type": "string"},
-                "name": {"title": "Name", "type": "string"},
-            },
-            "required": ["name", "color"],
-            "type": "object",
-        }
-        lm_invoker = BedrockLMInvoker(..., response_schema=schema)
+        class Animal(BaseModel):
+            name: str
+            color: str
+
+        json_schema = Animal.model_json_schema()
+
+        lm_invoker = BedrockLMInvoker(..., response_schema=Animal)  # Using Pydantic BaseModel class
+        lm_invoker = BedrockLMInvoker(..., response_schema=json_schema)  # Using JSON schema dictionary
         ```
+
         Output example:
         ```python
-        LMOutput(structured_output={"name": "Golden retriever", "color": "Golden"})
+        # Using Pydantic BaseModel class outputs a Pydantic model
+        LMOutput(outputs=[LMOutputItem(type="structured", output=Animal(name="dog", color="white"))])
+
+        # Using JSON schema dictionary outputs a dictionary
+        LMOutput(outputs=[LMOutputItem(type="structured", output={"name": "dog", "color": "white"})])
         ```
 
-        # Example 2: Using a Pydantic BaseModel class
+        Structured output is not compatible with tool calling.
+        When structured output is enabled, streaming is disabled.
+
+    Tool calling:
+        The `BedrockLMInvoker` can be configured to call tools to perform certain tasks.
+        This feature can be enabled by providing a list of `Tool` objects to the `tools` parameter.
+
+        Tool calls outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `tool_calls` property.
+
         Usage example:
         ```python
-        class Animal(BaseModel):
-            name: str
-            color: str
-
-        lm_invoker = BedrockLMInvoker(..., response_schema=Animal)
+        lm_invoker = BedrockLMInvoker(..., tools=[tool_1, tool_2])
         ```
+
         Output example:
         ```python
-        LMOutput(structured_output=Animal(name="Golden retriever", color="Golden"))
+        LMOutput(
+            outputs=[
+                LMOutputItem(type="text", output="I\'m using tools..."),
+                LMOutputItem(type="tool_call", output=ToolCall(id="123", name="tool_1", args={"key": "value"})),
+                LMOutputItem(type="tool_call", output=ToolCall(id="456", name="tool_2", args={"key": "value"})),
+            ]
+        )
         ```
 
     Analytics tracking:
-        Analytics tracking is a feature that allows the module to output additional information about the invocation.
+        The `BedrockLMInvoker` can be configured to output additional information about the invocation.
         This feature can be enabled by setting the `output_analytics` parameter to `True`.
+
         When enabled, the following attributes will be stored in the output:
         1. `token_usage`: The token usage.
         2. `duration`: The duration in seconds.
@@ -136,7 +135,7 @@ class BedrockLMInvoker(BaseLMInvoker):
         Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
+            outputs=[...],
             token_usage=TokenUsage(input_tokens=100, output_tokens=50),
             duration=0.729,
             finish_details={"stop_reason": "end_turn"},
@@ -151,8 +150,6 @@ class BedrockLMInvoker(BaseLMInvoker):
         Retry config examples:
         ```python
         retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
-        retry_config = RetryConfig(max_retries=0, timeout=10.0)  # No retry, 10.0 seconds timeout
-        retry_config = RetryConfig(max_retries=5, timeout=None)  # 5 max retries, no timeout
         retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
         ```
 
@@ -160,17 +157,6 @@ class BedrockLMInvoker(BaseLMInvoker):
         ```python
         lm_invoker = BedrockLMInvoker(..., retry_config=retry_config)
         ```
-
-    Output types:
-        The output of the `BedrockLMInvoker` can either be:
-        1. `str`: A text response.
-        2. `LMOutput`: A Pydantic model that may contain the following attributes:
-            2.1. response (str)
-            2.2. tool_calls (list[ToolCall])
-            2.3. structured_output (dict[str, Any] | BaseModel | None)
-            2.4. token_usage (TokenUsage | None)
-            2.5. duration (float | None)
-            2.6. finish_details (dict[str, Any] | None)
     '''
     session: Incomplete
     client_kwargs: Incomplete

gllm_inference/lm_invoker/datasaur_lm_invoker.pyi

@@ -44,9 +44,42 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         result = await lm_invoker.invoke([text, image])
         ```
 
+    Text output:
+        The `DatasaurLMInvoker` generates text outputs by default.
+        Text outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `texts` (all text outputs) or `text` (first text output) properties.
+
+        Output example:
+        ```python
+        LMOutput(outputs=[LMOutputItem(type="text", output="Hello, there!")])
+        ```
+
+    Citations:
+        The `DatasaurLMInvoker` can be configured to output the citations used to generate the response.
+        This feature can be enabled by setting the `citations` parameter to `True`.
+
+        Citations outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `citations` property.
+
+        Usage example:
+        ```python
+        lm_invoker = DatasaurLMInvoker(..., citations=True)
+        ```
+
+        Output example:
+        ```python
+        LMOutput(
+            outputs=[
+                LMOutputItem(type="citation", output=Chunk(id="123", content="...", metadata={...}, score=0.95)),
+                LMOutputItem(type="text", output="According to recent reports... ([Source](https://www.example.com))."),
+            ],
+        )
+        ```
+
     Analytics tracking:
-        Analytics tracking is a feature that allows the module to output additional information about the invocation.
+        The `DatasaurLMInvoker` can be configured to output additional information about the invocation.
         This feature can be enabled by setting the `output_analytics` parameter to `True`.
+
         When enabled, the following attributes will be stored in the output:
         1. `token_usage`: The token usage.
         2. `duration`: The duration in seconds.
@@ -55,16 +88,13 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
+            outputs=[...],
             token_usage=TokenUsage(input_tokens=100, output_tokens=50),
             duration=0.729,
-            finish_details={"finish_reason": "stop"},
+            finish_details={"stop_reason": "end_turn"},
         )
         ```
 
-        When streaming is enabled, token usage is not supported. Therefore, the `token_usage` attribute will be `None`
-        regardless of the value of the `output_analytics` parameter.
-
     Retry and timeout:
         The `DatasaurLMInvoker` supports retry and timeout configuration.
         By default, the max retries is set to 0 and the timeout is set to 30.0 seconds.
@@ -73,8 +103,6 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         Retry config examples:
         ```python
         retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
-        retry_config = RetryConfig(max_retries=0, timeout=10.0)  # No retry, 10.0 seconds timeout
-        retry_config = RetryConfig(max_retries=5, timeout=None)  # 5 max retries, no timeout
         retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
         ```
 
@@ -82,34 +110,6 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         ```python
         lm_invoker = DatasaurLMInvoker(..., retry_config=retry_config)
         ```
-
-    Citations:
-        The `DatasaurLMInvoker` can be configured to output the citations used to generate the response.
-        They can be enabled by setting the `citations` parameter to `True`.
-        When enabled, the citations will be stored as `Chunk` objects in the `citations` attribute in the output.
-
-        Usage example:
-        ```python
-        lm_invoker = DatasaurLMInvoker(..., citations=True)
-        ```
-
-        Output example:
-        ```python
-        LMOutput(
-            response="The winner of the match is team A ([Example title](https://www.example.com)).",
-            citations=[Chunk(id="123", content="...", metadata={...}, score=0.95)],
-        )
-        ```
-
-    Output types:
-        The output of the `DatasaurLMInvoker` can either be:
-        1. `str`: A text response.
-        2. `LMOutput`: A Pydantic model that may contain the following attributes:
-            2.1. response (str)
-            2.2. token_usage (TokenUsage | None)
-            2.3. duration (float | None)
-            2.4. finish_details (dict[str, Any] | None)
-            2.5. citations (list[Chunk])
     '''
     client_kwargs: Incomplete
     citations: Incomplete

gllm_inference/lm_invoker/google_lm_invoker.pyi

@@ -82,10 +82,61 @@ class GoogleLMInvoker(BaseLMInvoker):
         result = await lm_invoker.invoke([text, image])
         ```
 
+    Text output:
+        The `GoogleLMInvoker` generates text outputs by default.
+        Text outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `texts` (all text outputs) or `text` (first text output) properties.
+
+        Output example:
+        ```python
+        LMOutput(outputs=[LMOutputItem(type="text", output="Hello, there!")])
+        ```
+
+    Structured output:
+        The `GoogleLMInvoker` can be configured to generate structured outputs.
+        This feature can be enabled by providing a schema to the `response_schema` parameter.
+
+        Structured outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `structureds` (all structured outputs) or `structured` (first structured output) properties.
+
+        The schema must either be one of the following:
+        1. A Pydantic BaseModel class
+            The structured output will be a Pydantic model.
+        2. A JSON schema dictionary
+            JSON dictionary schema must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
+            Thus, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
+            The structured output will be a dictionary.
+
+        Usage example:
+        ```python
+        class Animal(BaseModel):
+            name: str
+            color: str
+
+        json_schema = Animal.model_json_schema()
+
+        lm_invoker = GoogleLMInvoker(..., response_schema=Animal)  # Using Pydantic BaseModel class
+        lm_invoker = GoogleLMInvoker(..., response_schema=json_schema)  # Using JSON schema dictionary
+        ```
+
+        Output example:
+        ```python
+        # Using Pydantic BaseModel class outputs a Pydantic model
+        LMOutput(outputs=[LMOutputItem(type="structured", output=Animal(name="dog", color="white"))])
+
+        # Using JSON schema dictionary outputs a dictionary
+        LMOutput(outputs=[LMOutputItem(type="structured", output={"name": "dog", "color": "white"})])
+        ```
+
+        Structured output is not compatible with tool calling.
+        When structured output is enabled, streaming is disabled.
+
     Image generation:
-        The `GoogleLMInvoker` supports image generation. This can be done by using an image generation model,
-        such as `gemini-2.5-flash-image`. Streaming is disabled for image generation models.
-        The generated image will be stored in the `attachments` attribute in the output.
+        The `GoogleLMInvoker` can be configured to generate images.
+        This feature can be enabled by using an image generation model, such as `gemini-2.5-flash-image`.
+
+        Image outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `attachments` property.
 
         Usage example:
         ```python
@@ -97,16 +148,24 @@ class GoogleLMInvoker(BaseLMInvoker):
         Output example:
         ```python
         LMOutput(
-            response="Let me call the tools...",
-            attachments=[Attachment(filename="image.png", mime_type="image/png", data=b"...")],
+            outputs=[
+                LMOutputItem(type="text", output="Creating a picture..."),
+                LMOutputItem(
+                    type="attachment",
+                    output=Attachment(filename="image.png", mime_type="image/png", data=b"..."),
+                ),
+            ],
         )
         ```
 
+        When image generation is enabled, streaming is disabled.
+
     Tool calling:
-        Tool calling is a feature that allows the language model to call tools to perform tasks.
-        Tools can be passed to the via the `tools` parameter as a list of `Tool` objects.
-        When tools are provided and the model decides to call a tool, the tool calls are stored in the
-        `tool_calls` attribute in the output.
+        The `GoogleLMInvoker` can be configured to call tools to perform certain tasks.
+        This feature can be enabled by providing a list of `Tool` objects to the `tools` parameter.
+
+        Tool calls outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `tool_calls` property.
 
         Usage example:
         ```python
@@ -116,67 +175,60 @@ class GoogleLMInvoker(BaseLMInvoker):
         Output example:
         ```python
         LMOutput(
-            response="Let me call the tools...",
-            tool_calls=[
-                ToolCall(id="123", name="tool_1", args={"key": "value"}),
-                ToolCall(id="456", name="tool_2", args={"key": "value"}),
+            outputs=[
+                LMOutputItem(type="text", output="I\'m using tools..."),
+                LMOutputItem(type="tool_call", output=ToolCall(id="123", name="tool_1", args={"key": "value"})),
+                LMOutputItem(type="tool_call", output=ToolCall(id="456", name="tool_2", args={"key": "value"})),
             ]
         )
         ```
 
-    Structured output:
-        Structured output is a feature that allows the language model to output a structured response.
-        This feature can be enabled by providing a schema to the `response_schema` parameter.
-
-        The schema must be either a JSON schema dictionary or a Pydantic BaseModel class.
-        If JSON schema is used, it must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
-        For this reason, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
-
-        Structured output is not compatible with tool calling. The language model also doesn\'t need to stream
-        anything when structured output is enabled. Thus, standard invocation will be performed regardless of
-        whether the `event_emitter` parameter is provided or not.
+    Thinking:
+        The `GoogleLMInvoker` can be configured to perform step-by-step thinking process before answering.
+        This feature can be enabled by setting the `thinking` parameter to `True`.
 
-        When enabled, the structured output is stored in the `structured_output` attribute in the output.
-        1. If the schema is a JSON schema dictionary, the structured output is a dictionary.
-        2. If the schema is a Pydantic BaseModel class, the structured output is a Pydantic model.
+        Thinking outputs are stored in the `outputs` attribute of the `LMOutput` object
+        and can be accessed via the `thinkings` property.
 
-        # Example 1: Using a JSON schema dictionary
         Usage example:
         ```python
-        schema = {
-            "title": "Animal",
-            "description": "A description of an animal.",
-            "properties": {
-                "color": {"title": "Color", "type": "string"},
-                "name": {"title": "Name", "type": "string"},
-            },
-            "required": ["name", "color"],
-            "type": "object",
-        }
-        lm_invoker = GoogleLMInvoker(..., response_schema=schema)
+        lm_invoker = GoogleLMInvoker(..., thinking=True, thinking_budget=1024)
         ```
+
         Output example:
         ```python
-        LMOutput(structured_output={"name": "Golden retriever", "color": "Golden"})
+        LMOutput(
+            outputs=[
+                LMOutputItem(type="thinking", output=Reasoning(type="thinking", reasoning="I\'m thinking...", ...)),
+                LMOutputItem(type="text", output="Golden retriever is a good dog breed."),
+            ]
+        )
         ```
 
-        # Example 2: Using a Pydantic BaseModel class
-        Usage example:
-        ```python
-        class Animal(BaseModel):
-            name: str
-            color: str
-
-        lm_invoker = GoogleLMInvoker(..., response_schema=Animal)
-        ```
-        Output example:
+        Streaming output example:
         ```python
-        LMOutput(structured_output=Animal(name="Golden retriever", color="Golden"))
+        {"type": "thinking_start", "value": "", ...}
+        {"type": "thinking", "value": "I\'m ", ...}
+        {"type": "thinking", "value": "thinking...", ...}
+        {"type": "thinking_end", "value": "", ...}
+        {"type": "response", "value": "Golden retriever ", ...}
+        {"type": "response", "value": "is a good dog breed.", ...}
         ```
+        Note: By default, the thinking token will be streamed with the legacy `EventType.DATA` event type.
+        To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
+        LM invoker initialization. The legacy event format support will be removed in v0.6.
+
+        The amount of tokens allocated for the thinking process can be set via the `thinking_budget` parameter.
+        For more information, please refer to the following documentation:
+        https://ai.google.dev/gemini-api/docs/thinking
+
+        Thinking is only available for certain models, starting from Gemini 2.5 series.
+        Thinking is required for Gemini 2.5 Pro models.
 
     Analytics tracking:
-        Analytics tracking is a feature that allows the module to output additional information about the invocation.
+        The `GoogleLMInvoker` can be configured to output additional information about the invocation.
         This feature can be enabled by setting the `output_analytics` parameter to `True`.
+
         When enabled, the following attributes will be stored in the output:
         1. `token_usage`: The token usage.
         2. `duration`: The duration in seconds.
@@ -185,15 +237,10 @@ class GoogleLMInvoker(BaseLMInvoker):
         Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
-            token_usage=TokenUsage(
-                input_tokens=1500,
-                output_tokens=200,
-                input_token_details=InputTokenDetails(cached_tokens=1200, uncached_tokens=300),
-                output_token_details=OutputTokenDetails(reasoning_tokens=180, response_tokens=20),
-            ),
+            outputs=[...],
+            token_usage=TokenUsage(input_tokens=100, output_tokens=50),
             duration=0.729,
-            finish_details={"finish_reason": "STOP", "finish_message": None},
+            finish_details={"stop_reason": "end_turn"},
         )
         ```
 
@@ -205,8 +252,6 @@ class GoogleLMInvoker(BaseLMInvoker):
         Retry config examples:
         ```python
         retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
-        retry_config = RetryConfig(max_retries=0, timeout=10.0)  # No retry, 10.0 seconds timeout
-        retry_config = RetryConfig(max_retries=5, timeout=None)  # 5 max retries, no timeout
         retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
         ```
 
@@ -214,61 +259,6 @@ class GoogleLMInvoker(BaseLMInvoker):
         ```python
         lm_invoker = GoogleLMInvoker(..., retry_config=retry_config)
         ```
-
-    Thinking:
-        Thinking is a feature that allows the language model to have enhanced reasoning capabilities for complex tasks,
-        while also providing transparency into its step-by-step thought process before it delivers its final answer.
-        It can be enabled by setting the `thinking` parameter to `True`.
-
-        Thinking is only available for certain models, starting from Gemini 2.5 series, and is required for
-        Gemini 2.5 Pro models. Therefore, `thinking` defaults to `True` for Gemini 2.5 Pro models and `False`
-        for other models. Setting `thinking` to `False` for Gemini 2.5 Pro models will raise a `ValueError`.
-        When enabled, the reasoning is stored in the `reasoning` attribute in the output.
-
-        Usage example:
-        ```python
-        lm_invoker = GoogleLMInvoker(..., thinking=True, thinking_budget=1024)
-        ```
-
-        Output example:
-        ```python
-        LMOutput(
-            response="Golden retriever is a good dog breed.",
-            reasoning=[Reasoning(reasoning="Let me think about it...")],
-        )
-        ```
-
-        Streaming output example:
-        ```python
-        {"type": "thinking_start", "value": "", ...}
-        {"type": "thinking", "value": "Let me think "\', ...}
-        {"type": "thinking", "value": "about it...", ...}
-        {"type": "thinking_end", "value": ""}\', ...}
-        {"type": "response", "value": "Golden retriever ", ...}
-        {"type": "response", "value": "is a good dog breed.", ...}
-        ```
-        Note: By default, the thinking token will be streamed with the legacy `EventType.DATA` event type.
-        To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
-        LM invoker initialization. The legacy event format support will be removed in v0.6.
-
-        When thinking is enabled, the amount of tokens allocated for the thinking process can be set via the
-        `thinking_budget` parameter. The `thinking_budget`:
-        1. Defaults to -1, in which case the model will control the budget automatically.
-        2. Must be greater than the model\'s minimum thinking budget.
-        For more details, please refer to https://ai.google.dev/gemini-api/docs/thinking
-
-    Output types:
-        The output of the `GoogleLMInvoker` can either be:
-        1. `str`: A text response.
-        2. `LMOutput`: A Pydantic model that may contain the following attributes:
-            2.1. response (str)
-            2.2. attachments (list[Attachment])
-            2.3. tool_calls (list[ToolCall])
-            2.4. structured_output (dict[str, Any] | BaseModel | None)
-            2.5. token_usage (TokenUsage | None)
-            2.6. duration (float | None)
-            2.7. finish_details (dict[str, Any])
-            2.8. reasoning (list[Reasoning])
     '''
     client_params: Incomplete
     generate_image: Incomplete