gllm-inference-binary 0.5.51__cp311-cp311-win_amd64.whl → 0.5.66__cp311-cp311-win_amd64.whl

gllm_inference/lm_invoker/bedrock_lm_invoker.pyi

@@ -51,83 +51,82 @@ class BedrockLMInvoker(BaseLMInvoker):
         result = await lm_invoker.invoke([text, image])
         ```
 
-    Tool calling:
-        Tool calling is a feature that allows the language model to call tools to perform tasks.
-        Tools can be passed to the via the `tools` parameter as a list of `Tool` objects.
-        When tools are provided and the model decides to call a tool, the tool calls are stored in the
-        `tool_calls` attribute in the output.
-
-        Usage example:
-        ```python
-        lm_invoker = BedrockLMInvoker(..., tools=[tool_1, tool_2])
-        ```
+    Text output:
+        The `BedrockLMInvoker` generates text outputs by default.
+        Text outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `texts` (all text outputs) or `text` (first text output) properties.
 
         Output example:
         ```python
-        LMOutput(
-            response="Let me call the tools...",
-            tool_calls=[
-                ToolCall(id="123", name="tool_1", args={"key": "value"}),
-                ToolCall(id="456", name="tool_2", args={"key": "value"}),
-            ]
-        )
+        LMOutput(outputs=[LMOutputItem(type="text", output="Hello, there!")])
         ```
 
     Structured output:
-        Structured output is a feature that allows the language model to output a structured response.
+        The `BedrockLMInvoker` can be configured to generate structured outputs.
         This feature can be enabled by providing a schema to the `response_schema` parameter.
 
-        The schema must be either a JSON schema dictionary or a Pydantic BaseModel class.
-        If JSON schema is used, it must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
-        For this reason, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
-
-        Structured output is achieved by providing the schema name in the `tool_choice` parameter. This forces
-        the model to call the provided schema as a tool. Thus, structured output is not compatible with tool calling,
-        since the tool calling is reserved to force the model to call the provided schema as a tool.
-        The language model also doesn\'t need to stream anything when structured output is enabled. Thus, standard
-        invocation will be performed regardless of whether the `event_emitter` parameter is provided or not.
+        Structured outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `structureds` (all structured outputs) or `structured` (first structured output) properties.
 
-        When enabled, the structured output is stored in the `structured_output` attribute in the output.
-        1. If the schema is a JSON schema dictionary, the structured output is a dictionary.
-        2. If the schema is a Pydantic BaseModel class, the structured output is a Pydantic model.
+        The schema must either be one of the following:
+        1. A Pydantic BaseModel class
+            The structured output will be a Pydantic model.
+        2. A JSON schema dictionary
+            JSON dictionary schema must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
+            Thus, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
+            The structured output will be a dictionary.
 
-        # Example 1: Using a JSON schema dictionary
         Usage example:
         ```python
-        schema = {
-            "title": "Animal",
-            "description": "A description of an animal.",
-            "properties": {
-                "color": {"title": "Color", "type": "string"},
-                "name": {"title": "Name", "type": "string"},
-            },
-            "required": ["name", "color"],
-            "type": "object",
-        }
-        lm_invoker = BedrockLMInvoker(..., response_schema=schema)
+        class Animal(BaseModel):
+            name: str
+            color: str
+
+        json_schema = Animal.model_json_schema()
+
+        lm_invoker = BedrockLMInvoker(..., response_schema=Animal)  # Using Pydantic BaseModel class
+        lm_invoker = BedrockLMInvoker(..., response_schema=json_schema)  # Using JSON schema dictionary
         ```
+
         Output example:
         ```python
-        LMOutput(structured_output={"name": "Golden retriever", "color": "Golden"})
+        # Using Pydantic BaseModel class outputs a Pydantic model
+        LMOutput(outputs=[LMOutputItem(type="structured", output=Animal(name="dog", color="white"))])
+
+        # Using JSON schema dictionary outputs a dictionary
+        LMOutput(outputs=[LMOutputItem(type="structured", output={"name": "dog", "color": "white"})])
         ```
 
-        # Example 2: Using a Pydantic BaseModel class
+        Structured output is not compatible with tool calling.
+        When structured output is enabled, streaming is disabled.
+
+    Tool calling:
+        The `BedrockLMInvoker` can be configured to call tools to perform certain tasks.
+        This feature can be enabled by providing a list of `Tool` objects to the `tools` parameter.
+
+        Tool calls outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `tool_calls` property.
+
         Usage example:
         ```python
-        class Animal(BaseModel):
-            name: str
-            color: str
-
-        lm_invoker = BedrockLMInvoker(..., response_schema=Animal)
+        lm_invoker = BedrockLMInvoker(..., tools=[tool_1, tool_2])
         ```
+
         Output example:
         ```python
-        LMOutput(structured_output=Animal(name="Golden retriever", color="Golden"))
+        LMOutput(
+            outputs=[
+                LMOutputItem(type="text", output="I\'m using tools..."),
+                LMOutputItem(type="tool_call", output=ToolCall(id="123", name="tool_1", args={"key": "value"})),
+                LMOutputItem(type="tool_call", output=ToolCall(id="456", name="tool_2", args={"key": "value"})),
+            ]
+        )
         ```
 
     Analytics tracking:
-        Analytics tracking is a feature that allows the module to output additional information about the invocation.
+        The `BedrockLMInvoker` can be configured to output additional information about the invocation.
         This feature can be enabled by setting the `output_analytics` parameter to `True`.
+
         When enabled, the following attributes will be stored in the output:
         1. `token_usage`: The token usage.
         2. `duration`: The duration in seconds.
@@ -136,7 +135,7 @@ class BedrockLMInvoker(BaseLMInvoker):
         Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
+            outputs=[...],
             token_usage=TokenUsage(input_tokens=100, output_tokens=50),
             duration=0.729,
             finish_details={"stop_reason": "end_turn"},
@@ -151,8 +150,6 @@ class BedrockLMInvoker(BaseLMInvoker):
         Retry config examples:
         ```python
         retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
-        retry_config = RetryConfig(max_retries=0, timeout=10.0)  # No retry, 10.0 seconds timeout
-        retry_config = RetryConfig(max_retries=5, timeout=None)  # 5 max retries, no timeout
         retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
         ```
 
@@ -160,17 +157,6 @@ class BedrockLMInvoker(BaseLMInvoker):
         ```python
         lm_invoker = BedrockLMInvoker(..., retry_config=retry_config)
         ```
-
-    Output types:
-        The output of the `BedrockLMInvoker` can either be:
-        1. `str`: A text response.
-        2. `LMOutput`: A Pydantic model that may contain the following attributes:
-            2.1. response (str)
-            2.2. tool_calls (list[ToolCall])
-            2.3. structured_output (dict[str, Any] | BaseModel | None)
-            2.4. token_usage (TokenUsage | None)
-            2.5. duration (float | None)
-            2.6. finish_details (dict[str, Any] | None)
     '''
     session: Incomplete
     client_kwargs: Incomplete

gllm_inference/lm_invoker/datasaur_lm_invoker.pyi

@@ -44,9 +44,42 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         result = await lm_invoker.invoke([text, image])
         ```
 
+    Text output:
+        The `DatasaurLMInvoker` generates text outputs by default.
+        Text outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `texts` (all text outputs) or `text` (first text output) properties.
+
+        Output example:
+        ```python
+        LMOutput(outputs=[LMOutputItem(type="text", output="Hello, there!")])
+        ```
+
+    Citations:
+        The `DatasaurLMInvoker` can be configured to output the citations used to generate the response.
+        This feature can be enabled by setting the `citations` parameter to `True`.
+
+        Citations outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `citations` property.
+
+        Usage example:
+        ```python
+        lm_invoker = DatasaurLMInvoker(..., citations=True)
+        ```
+
+        Output example:
+        ```python
+        LMOutput(
+            outputs=[
+                LMOutputItem(type="citation", output=Chunk(id="123", content="...", metadata={...}, score=0.95)),
+                LMOutputItem(type="text", output="According to recent reports... ([Source](https://www.example.com))."),
+            ],
+        )
+        ```
+
     Analytics tracking:
-        Analytics tracking is a feature that allows the module to output additional information about the invocation.
+        The `DatasaurLMInvoker` can be configured to output additional information about the invocation.
         This feature can be enabled by setting the `output_analytics` parameter to `True`.
+
         When enabled, the following attributes will be stored in the output:
         1. `token_usage`: The token usage.
         2. `duration`: The duration in seconds.
@@ -55,16 +88,13 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
+            outputs=[...],
             token_usage=TokenUsage(input_tokens=100, output_tokens=50),
             duration=0.729,
-            finish_details={"finish_reason": "stop"},
+            finish_details={"stop_reason": "end_turn"},
         )
         ```
 
-        When streaming is enabled, token usage is not supported. Therefore, the `token_usage` attribute will be `None`
-        regardless of the value of the `output_analytics` parameter.
-
     Retry and timeout:
         The `DatasaurLMInvoker` supports retry and timeout configuration.
         By default, the max retries is set to 0 and the timeout is set to 30.0 seconds.
@@ -73,8 +103,6 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         Retry config examples:
         ```python
         retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
-        retry_config = RetryConfig(max_retries=0, timeout=10.0)  # No retry, 10.0 seconds timeout
-        retry_config = RetryConfig(max_retries=5, timeout=None)  # 5 max retries, no timeout
         retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
         ```
 
@@ -82,34 +110,6 @@ class DatasaurLMInvoker(OpenAIChatCompletionsLMInvoker):
         ```python
         lm_invoker = DatasaurLMInvoker(..., retry_config=retry_config)
         ```
-
-    Citations:
-        The `DatasaurLMInvoker` can be configured to output the citations used to generate the response.
-        They can be enabled by setting the `citations` parameter to `True`.
-        When enabled, the citations will be stored as `Chunk` objects in the `citations` attribute in the output.
-
-        Usage example:
-        ```python
-        lm_invoker = DatasaurLMInvoker(..., citations=True)
-        ```
-
-        Output example:
-        ```python
-        LMOutput(
-            response="The winner of the match is team A ([Example title](https://www.example.com)).",
-            citations=[Chunk(id="123", content="...", metadata={...}, score=0.95)],
-        )
-        ```
-
-    Output types:
-        The output of the `DatasaurLMInvoker` can either be:
-        1. `str`: A text response.
-        2. `LMOutput`: A Pydantic model that may contain the following attributes:
-            2.1. response (str)
-            2.2. token_usage (TokenUsage | None)
-            2.3. duration (float | None)
-            2.4. finish_details (dict[str, Any] | None)
-            2.5. citations (list[Chunk])
     '''
     client_kwargs: Incomplete
     citations: Incomplete