@mate-academy/llm-gateway 7.3.0-canary.0 → 7.4.0

package/README.md

@@ -943,9 +943,40 @@ Interface for chat/assistance services with file handling capabilities.
   - `chatId`: The ID of the chat to send the message to
   - `storageId`: Optional storage ID to use for file context
   - `schema`: Optional schema for structured output (supports type-safe JSON responses)
+  - `tools`: Optional array of tool definitions (built with `LLMTool.create`) the model may call during the turn
+  - `maxToolIterations`: Optional cap on the number of tool-call rounds
   - `reporterContext`: Context data passed to reporter for metrics tracking (required if reporter is configured with a defined context type)
 - `countTokens(messages, model)`: Count tokens in messages for context management
 
+#### Tool-returned Images
+
+A tool's `execute` may return either a plain `string` (text-only, the legacy
+behavior) or an `LLMStructuredToolResult` to surface images the model can
+actually see:
+
+```typescript
+const screenshotTool = LLMTool.create({
+  name: 'get_node_screenshot',
+  description: 'Renders a screenshot of a design node.',
+  parameters: LLMSchema.object({ nodeId: LLMSchema.string() }),
+  execute: async ({ nodeId }) => ({
+    content: `Rendered a screenshot of node ${nodeId}.`,
+    images: [{ url: imageUrl, detail: 'high', label: `Node ${nodeId}` }],
+  }),
+});
+```
+
+`content` always becomes the tool/function-result message. Each provider
+then surfaces `images` the way its API allows:
+
+- **OpenAI** (Responses API) and **Gemini 3** (`multimodalToolResults`
+  capability) embed the images natively in the function-result message.
+- **LLMAPI** (Chat Completions) and **Gemini 2.5** cannot carry images in a
+  tool-result message, so the gateway appends the images as a separate user
+  message right after the tool results.
+
+`LLMToolResultImage.url` accepts an `https` URL or a base64 `data:` URL.
+
 ### LLMSpeechToTextService
 
 Interface for converting speech audio to text.