gllm-inference-binary 0.5.53__cp312-cp312-win_amd64.whl → 0.5.55__cp312-cp312-win_amd64.whl

gllm_inference/lm_invoker/openai_lm_invoker.pyi

@@ -76,237 +76,199 @@ class OpenAILMInvoker(BaseLMInvoker):
         result = await lm_invoker.invoke([text, image])
         ```
 
-    Tool calling:
-        Tool calling is a feature that allows the language model to call tools to perform tasks.
-        Tools can be passed to the via the `tools` parameter as a list of `Tool` objects.
-        When tools are provided and the model decides to call a tool, the tool calls are stored in the
-        `tool_calls` attribute in the output.
-
-        Usage example:
-        ```python
-        lm_invoker = OpenAILMInvoker(..., tools=[tool_1, tool_2])
-        ```
+    Text output:
+        The `OpenAILMInvoker` generates text outputs by default.
+        Text outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `texts` (all text outputs) or `text` (first text output) properties.
 
         Output example:
         ```python
-        LMOutput(
-            response="Let me call the tools...",
-            tool_calls=[
-                ToolCall(id="123", name="tool_1", args={"key": "value"}),
-                ToolCall(id="456", name="tool_2", args={"key": "value"}),
-            ]
-        )
+        LMOutput(outputs=[LMOutputItem(type="text", output="Hello, there!")])
         ```
 
     Structured output:
-        Structured output is a feature that allows the language model to output a structured response.
+        The `OpenAILMInvoker` can be configured to generate structured outputs.
         This feature can be enabled by providing a schema to the `response_schema` parameter.
 
-        The schema must be either a JSON schema dictionary or a Pydantic BaseModel class.
-        If JSON schema is used, it must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
-        For this reason, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
-
-        The language model also doesn\'t need to stream anything when structured output is enabled. Thus, standard
-        invocation will be performed regardless of whether the `event_emitter` parameter is provided or not.
+        Structured outputs are stored in the `outputs` attribute of the `LMOutput` object and can be accessed
+        via the `structureds` (all structured outputs) or `structured` (first structured output) properties.
 
-        When enabled, the structured output is stored in the `structured_output` attribute in the output.
-        1. If the schema is a JSON schema dictionary, the structured output is a dictionary.
-        2. If the schema is a Pydantic BaseModel class, the structured output is a Pydantic model.
-
-        # Example 1: Using a JSON schema dictionary
-        Usage example:
-        ```python
-        schema = {
-            "title": "Animal",
-            "description": "A description of an animal.",
-            "properties": {
-                "color": {"title": "Color", "type": "string"},
-                "name": {"title": "Name", "type": "string"},
-            },
-            "required": ["name", "color"],
-            "type": "object",
-        }
-        lm_invoker = OpenAILMInvoker(..., response_schema=schema)
-        ```
-        Output example:
-        ```python
-        LMOutput(structured_output={"name": "Golden retriever", "color": "Golden"})
-        ```
+        The schema must either be one of the following:
+        1. A Pydantic BaseModel class
+            The structured output will be a Pydantic model.
+        2. A JSON schema dictionary
+            JSON dictionary schema must be compatible with Pydantic\'s JSON schema, especially for complex schemas.
+            Thus, it is recommended to create the JSON schema using Pydantic\'s `model_json_schema` method.
+            The structured output will be a dictionary.
 
-        # Example 2: Using a Pydantic BaseModel class
         Usage example:
         ```python
         class Animal(BaseModel):
             name: str
             color: str
 
-        lm_invoker = OpenAILMInvoker(..., response_schema=Animal)
+        json_schema = Animal.model_json_schema()
+
+        lm_invoker = OpenAILMInvoker(..., response_schema=Animal)  # Using Pydantic BaseModel class
+        lm_invoker = OpenAILMInvoker(..., response_schema=json_schema)  # Using JSON schema dictionary
         ```
+
         Output example:
         ```python
-        LMOutput(structured_output=Animal(name="Golden retriever", color="Golden"))
+        # Using Pydantic BaseModel class outputs a Pydantic model
+        LMOutput(outputs=[LMOutputItem(type="structured", output=Animal(name="dog", color="white"))])
+
+        # Using JSON schema dictionary outputs a dictionary
+        LMOutput(outputs=[LMOutputItem(type="structured", output={"name": "dog", "color": "white"})])
         ```
 
-    Analytics tracking:
-        Analytics tracking is a feature that allows the module to output additional information about the invocation.
-        This feature can be enabled by setting the `output_analytics` parameter to `True`.
-        When enabled, the following attributes will be stored in the output:
-        1. `token_usage`: The token usage.
-        2. `duration`: The duration in seconds.
-        3. `finish_details`: The details about how the generation finished.
+        When structured output is enabled, streaming is disabled.
+
+    Tool calling:
+        The `OpenAILMInvoker` can be configured to call tools to perform certain tasks.
+        This feature can be enabled by providing a list of `Tool` objects to the `tools` parameter.
+
+        Tool calls outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `tool_calls` property.
+
+        Usage example:
+        ```python
+        lm_invoker = OpenAILMInvoker(..., tools=[tool_1, tool_2])
+        ```
 
         Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
-            token_usage=TokenUsage(
-                input_tokens=1500,
-                output_tokens=200,
-                input_token_details=InputTokenDetails(cached_tokens=1200, uncached_tokens=300),
-                output_token_details=OutputTokenDetails(reasoning_tokens=180, response_tokens=20),
-            ),
-            duration=0.729,
-            finish_details={"status": "completed", "incomplete_details": {"reason": None}},
+            outputs=[
+                LMOutputItem(type="text", output="I\'m using tools..."),
+                LMOutputItem(type="tool_call", output=ToolCall(id="123", name="tool_1", args={"key": "value"})),
+                LMOutputItem(type="tool_call", output=ToolCall(id="456", name="tool_2", args={"key": "value"})),
+            ]
         )
         ```
 
-    Retry and timeout:
-        The `OpenAILMInvoker` supports retry and timeout configuration.
-        By default, the max retries is set to 0 and the timeout is set to 30.0 seconds.
-        They can be customized by providing a custom `RetryConfig` object to the `retry_config` parameter.
+    MCP tool calling:
+        The `OpenAILMInvoker` can be configured to call MCP tools to perform certain tasks.
+        This feature can be enabled by providing a list of MCP servers to the `mcp_servers` parameter.
 
-        Retry config examples:
-        ```python
-        retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
-        retry_config = RetryConfig(max_retries=0, timeout=10.0)  # No retry, 10.0 seconds timeout
-        retry_config = RetryConfig(max_retries=5, timeout=None)  # 5 max retries, no timeout
-        retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
-        ```
+        MCP calls outputs are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `mcp_calls` property.
 
         Usage example:
         ```python
-        lm_invoker = OpenAILMInvoker(..., retry_config=retry_config)
-        ```
+        from gllm_inference.schema import MCPServer
 
-    Reasoning:
-        OpenAI\'s GPT-5 models and o-series models are classified as reasoning models. Reasoning models think before
-        they answer, producing a long internal chain of thought before responding to the user. Reasoning models
-        excel in complex problem solving, coding, scientific reasoning, and multi-step planning for agentic workflows.
-
-        The reasoning effort of reasoning models can be set via the `reasoning_effort` parameter. This parameter
-        will guide the models on how many reasoning tokens it should generate before creating a response.
-        Available options include:
-        1. "minimal": Favors the least amount of reasoning, only supported for GPT-5 models onwards.
-        2. "low": Favors speed and economical token usage.
-        3. "medium": Favors a balance between speed and reasoning accuracy.
-        4. "high": Favors more complete reasoning at the cost of more tokens generated and slower responses.
-        When not set, the reasoning effort will be equivalent to `medium` by default.
-
-        OpenAI doesn\'t expose the raw reasoning tokens. However, the summary of the reasoning tokens can still be
-        generated. The summary level can be set via the `reasoning_summary` parameter. Available options include:
-        1. "auto": The model decides the summary level automatically.
-        2. "detailed": The model will generate a detailed summary of the reasoning tokens.
-        Reasoning summary is not compatible with tool calling.
-        When enabled, the reasoning summary will be stored in the `reasoning` attribute in the output.
+        mcp_server_1 = MCPServer(url="https://mcp_server_1.com", name="mcp_server_1")
+        lm_invoker = OpenAILMInvoker(..., mcp_servers=[mcp_server_1])
+        ```
 
         Output example:
         ```python
         LMOutput(
-            response="Golden retriever is a good dog breed.",
-            reasoning=[Reasoning(id="x", reasoning="Let me think about it...")],
+            outputs=[
+                LMOutputItem(type="text", output="I\'m using MCP tools..."),
+                LMOutputItem(
+                    type="mcp_call",
+                    output=MCPCall(
+                        id="123",
+                        server_name="mcp_server_1",
+                        tool_name="mcp_tool_1",
+                        args={"key": "value"},
+                        output="The result is 10."
+                    ),
+                ),
+            ],
         )
         ```
 
         Streaming output example:
         ```python
-        {"type": "thinking_start", "value": ""}\', ...}
-        {"type": "thinking", "value": "Let me think "}\', ...}
-        {"type": "thinking", "value": "about it..."}\', ...}
-        {"type": "thinking_end", "value": ""}\', ...}
-        {"type": "response", "value": "Golden retriever ", ...}
-        {"type": "response", "value": "is a good dog breed.", ...}
+        {"type": "activity", "value": {"type": "mcp_list_tools", ...}, ...}
+        {"type": "activity", "value": {"type": "mcp_call", ...}, ...}
+        {"type": "response", "value": "The result ", ...}
+        {"type": "response", "value": "is 10.", ...}
         ```
-        Note: By default, the thinking token will be streamed with the legacy `EventType.DATA` event type.
+        Note: By default, the activity token will be streamed with the legacy `EventType.DATA` event type.
         To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
         LM invoker initialization. The legacy event format support will be removed in v0.6.
 
-        Setting reasoning-related parameters for non-reasoning models will raise an error.
+    Reasoning:
+        The `OpenAILMInvoker` performs step-by-step reasoning before generating a response when reasoning
+        models are used, such as GPT-5 models and o-series models.
 
-    MCP tool calling:
-        The `OpenAILMInvoker` supports MCP tool calling. This feature can be enabled by providing a list of
-        MCP servers to the `mcp_servers` parameter. When MCP servers are provided and the model decides to call
-        an MCP tool, the MCP calls are stored in the `mcp_calls` attribute in the output.
+        The reasoning effort can be set via the `reasoning_effort` parameter, which guides the models on the amount
+        of reasoning tokens to generate. Available options include `minimal`, `low`, `medium`, and `high`.
+
+        While the raw reasoning tokens are not available, the summary of the reasoning tokens can still be generated.
+        This can be done by passing the desired summary level via the `reasoning_summary` parameter.
+        Available options include `auto` and `detailed`.
+
+        Reasoning summaries are stored in the `outputs` attribute of the `LMOutput` object
+        and can be accessed via the `thinkings` property.
 
         Usage example:
         ```python
-        from gllm_inference.schema import MCPServer
-        mcp_server_1 = MCPServer(
-            url="https://mcp_server_1.com",
-            name="mcp_server_1",
-        )
-        lm_invoker = OpenAILMInvoker(..., mcp_servers=[mcp_server_1])
+        lm_invoker = OpenAILMInvoker(..., reasoning_effort="high", reasoning_summary="detailed")
         ```
 
         Output example:
         ```python
         LMOutput(
-            response="The result is 10.",
-            mcp_calls=[
-                MCPCall(
-                    id="123",
-                    server_name="mcp_server_1",
-                    tool_name="mcp_tool_1",
-                    args={"key": "value"},
-                    output="The result is 10.",
-                ),
-            ],
+            outputs=[
+                LMOutputItem(type="thinking", output=Reasoning(type="thinking", reasoning="I\'m thinking...", ...)),
+                LMOutputItem(type="text", output="Golden retriever is a good dog breed."),
+            ]
         )
         ```
 
         Streaming output example:
         ```python
-        {"type": "activity", "value": {"type": "mcp_list_tools", ...}, ...}
-        {"type": "activity", "value": {"type": "mcp_call", ...}, ...}
-        {"type": "response", "value": "The result ", ...}
-        {"type": "response", "value": "is 10.", ...}
+        {"type": "thinking_start", "value": "", ...}
+        {"type": "thinking", "value": "I\'m ", ...}
+        {"type": "thinking", "value": "thinking...", ...}
+        {"type": "thinking_end", "value": "", ...}
+        {"type": "response", "value": "Golden retriever ", ...}
+        {"type": "response", "value": "is a good dog breed.", ...}
         ```
-        Note: By default, the activity token will be streamed with the legacy `EventType.DATA` event type.
+        Note: By default, the thinking token will be streamed with the legacy `EventType.DATA` event type.
         To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
         LM invoker initialization. The legacy event format support will be removed in v0.6.
 
+        Reasoning summary is not compatible with tool calling.
+
     Code interpreter:
-        The code interpreter is a feature that allows the language model to write and run Python code in a
-        sandboxed environment to solve complex problems in domains like data analysis, coding, and math.
+        The `OpenAILMInvoker` can be configured to write and run Python code in a sandboxed environment.
+        This is useful for solving complex problems in domains like data analysis, coding, and math.
         This feature can be enabled by setting the `code_interpreter` parameter to `True`.
 
-        Usage example:
-        ```python
-        lm_invoker = OpenAILMInvoker(..., code_interpreter=True)
-        ```
-
         When code interpreter is enabled, it is highly recommended to instruct the model to use the "python tool"
         in the system message, as "python tool" is the term recognized by the model to refer to the code interpreter.
 
-        Messages example:
+        Code execution results are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `code_exec_results` property.
+
+        Usage example:
         ```python
+        lm_invoker = OpenAILMInvoker(..., code_interpreter=True)
         messages = [
             Message.system("You are a data analyst. Use the python tool to generate a file."]),
             Message.user("Show an histogram of the following data: [1, 2, 1, 4, 1, 2, 4, 2, 3, 1]"),
         ]
+        result = await lm_invoker.invoke(messages)
         ```
 
-        When code interpreter is enabled, the code execution results are stored in the `code_exec_results`
-        attribute in the output.
-
         Output example:
         ```python
         LMOutput(
-            response="The histogram is attached.",
-            code_exec_results=[
-                CodeExecResult(
-                    id="123",
-                    code="import matplotlib.pyplot as plt...",
-                    output=[Attachment(data=b"...", mime_type="image/png")],
+            outputs=[
+                LMOutputItem(type="text", output="The histogram is attached."),
+                LMOutputItem(
+                    type="code_exec_result",
+                    output=CodeExecResult(
+                        id="123",
+                        code="import matplotlib.pyplot as plt...",
+                        output=[Attachment(data=b"...", mime_type="image/png")],
+                    ),
                 ),
             ],
         )
@@ -325,35 +287,24 @@ class OpenAILMInvoker(BaseLMInvoker):
         To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
         LM invoker initialization. The legacy event format support will be removed in v0.6.
 
-    Web search:
-        The web search is a feature that allows the language model to search the web for relevant information.
+    Web Search:
+        The `OpenAILMInvoker` can be configured to search the web for relevant information.
         This feature can be enabled by setting the `web_search` parameter to `True`.
 
+        Web search citations are stored in the `outputs` attribute of the `LMOutput` object and
+        can be accessed via the `citations` property.
+
         Usage example:
         ```python
         lm_invoker = OpenAILMInvoker(..., web_search=True)
         ```
 
-        When web search is enabled, the language model will search the web for relevant information and may cite the
-        relevant sources. The citations will be stored as `Chunk` objects in the `citations` attribute in the output.
-        The content of the `Chunk` object is the type of the citation, e.g. "url_citation".
-
         Output example:
         ```python
         LMOutput(
-            response="The winner of the match is team A ([Example title](https://www.example.com)).",
-            citations=[
-                Chunk(
-                    id="123",
-                    content="url_citation",
-                    metadata={
-                        "start_index": 164,
-                        "end_index": 275,
-                        "title": "Example title",
-                        "url": "https://www.example.com",
-                        "type": "url_citation",
-                    },
-                ),
+            outputs=[
+                LMOutputItem(type="citation", output=Chunk(id="123", content="...", metadata={...}, score=None)),
+                LMOutputItem(type="text", output="According to recent reports... ([Source](https://example.com))."),
             ],
         )
         ```
@@ -361,27 +312,47 @@ class OpenAILMInvoker(BaseLMInvoker):
         Streaming output example:
         ```python
         {"type": "activity", "value": {"query": "search query"}, ...}
-        {"type": "response", "value": "The winner of the match ", ...}
-        {"type": "response", "value": "is team A ([Example title](https://www.example.com)).", ...}
+        {"type": "response", "value": "According to recent ", ...}
+        {"type": "response", "value": "reports... ([Source](https://example.com)).", ...}
         ```
         Note: By default, the activity token will be streamed with the legacy `EventType.DATA` event type.
         To use the new simplified streamed event format, set the `simplify_events` parameter to `True` during
         LM invoker initialization. The legacy event format support will be removed in v0.6.
 
-    Output types:
-        The output of the `OpenAILMInvoker` can either be:
-        1. `str`: A text response.
-        2. `LMOutput`: A Pydantic model that may contain the following attributes:
-            2.1. response (str)
-            2.2. tool_calls (list[ToolCall])
-            2.3. structured_output (dict[str, Any] | BaseModel | None)
-            2.4. token_usage (TokenUsage | None)
-            2.5. duration (float | None)
-            2.6. finish_details (dict[str, Any])
-            2.7. reasoning (list[Reasoning])
-            2.8. citations (list[Chunk])
-            2.9. code_exec_results (list[CodeExecResult])
-            2.10. mcp_calls (list[MCPCall])
+    Analytics tracking:
+        The `OpenAILMInvoker` can be configured to output additional information about the invocation.
+        This feature can be enabled by setting the `output_analytics` parameter to `True`.
+
+        When enabled, the following attributes will be stored in the output:
+        1. `token_usage`: The token usage.
+        2. `duration`: The duration in seconds.
+        3. `finish_details`: The details about how the generation finished.
+
+        Output example:
+        ```python
+        LMOutput(
+            outputs=[...],
+            token_usage=TokenUsage(input_tokens=100, output_tokens=50),
+            duration=0.729,
+            finish_details={"stop_reason": "end_turn"},
+        )
+        ```
+
+    Retry and timeout:
+        The `OpenAILMInvoker` supports retry and timeout configuration.
+        By default, the max retries is set to 0 and the timeout is set to 30.0 seconds.
+        They can be customized by providing a custom `RetryConfig` object to the `retry_config` parameter.
+
+        Retry config examples:
+        ```python
+        retry_config = RetryConfig(max_retries=0, timeout=None)  # No retry, no timeout
+        retry_config = RetryConfig(max_retries=5, timeout=10.0)  # 5 max retries, 10.0 seconds timeout
+        ```
+
+        Usage example:
+        ```python
+        lm_invoker = OpenAILMInvoker(..., retry_config=retry_config)
+        ```
     '''
     client_kwargs: Incomplete
     def __init__(self, model_name: str, api_key: str | None = None, base_url: str = ..., model_kwargs: dict[str, Any] | None = None, default_hyperparameters: dict[str, Any] | None = None, tools: list[Tool | LangChainTool] | None = None, response_schema: ResponseSchema | None = None, output_analytics: bool = False, retry_config: RetryConfig | None = None, reasoning_effort: ReasoningEffort | None = None, reasoning_summary: ReasoningSummary | None = None, mcp_servers: list[MCPServer] | None = None, code_interpreter: bool = False, web_search: bool = False, simplify_events: bool = False) -> None: