prompture 1.0.52.dev1__tar.gz → 1.0.53.dev1__tar.gz

{prompture-1.0.52.dev1 → prompture-1.0.53.dev1}/.github/workflows/dev.yml

@@ -23,6 +23,16 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           pip install -e ".[test,all]"
+          pip install ruff mypy
+
+      - name: Lint with ruff
+        run: ruff check .
+
+      - name: Type check with mypy
+        # Advisory: 500+ pre-existing errors. Fix incrementally before
+        # promoting to a required check.
+        run: mypy prompture
+        continue-on-error: true
 
       - name: Run tests
         run: pytest

prompture-1.0.53.dev1/.github/workflows/pricing-freshness.yml

@@ -0,0 +1,25 @@
+name: Pricing Freshness
+
+on:
+  schedule:
+    - cron: "0 6 * * *"
+  workflow_dispatch:
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[test,all]"
+
+      - name: Check pricing freshness
+        run: python scripts/check_pricing_freshness.py

prompture-1.0.53.dev1/AGENTS.md

@@ -0,0 +1,93 @@
+# AGENTS.md
+
+This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Prompture is a Python library for extracting structured JSON output from LLMs, with schema enforcement, Pydantic model integration, TOON (Token-Oriented Object Notation) input/output conversion, and multi-provider driver support. Published on PyPI as `prompture`.
+
+## Build & Development Commands
+
+```bash
+# Install in development mode
+pip install -e ".[test,dev]"
+
+# Run all tests (uses DEFAULT_MODEL from tests/conftest.py)
+python test.py
+
+# Run tests with pytest directly
+pytest
+
+# Run integration tests (require live LLM access)
+pytest --run-integration
+# or: RUN_INTEGRATION_TESTS=1 pytest
+
+# Run a single test file
+pytest tests/test_core.py
+
+# Run a single test
+pytest tests/test_core.py::TestCleanJsonText::test_basic_json
+
+# Skip integration tests when credentials are missing
+TEST_SKIP_NO_CREDENTIALS=true python test.py
+
+# Build distribution
+python -m build
+
+# CLI entry point
+prompture run <spec-file>
+```
+
+### Linting & Formatting (ruff)
+
+```bash
+# Format all files
+ruff format .
+
+# Check for lint violations
+ruff check .
+
+# Auto-fix lint violations
+ruff check --fix .
+
+# Auto-fix including unsafe fixes (e.g., typing modernization)
+ruff check --fix --unsafe-fixes .
+```
+
+Configuration is in `pyproject.toml` under `[tool.ruff]`.
+
+## Architecture
+
+### Module Layout
+
+- **`prompture/core.py`** — Primary business logic. All extraction functions live here: `ask_for_json()` (low-level schema enforcement), `extract_and_jsonify()` / `manual_extract_and_jsonify()` (text-to-JSON), `extract_with_model()` / `stepwise_extract_with_model()` (Pydantic-based), `extract_from_data()` / `extract_from_pandas()` (TOON input), `render_output()` (raw text formatting).
+- **`prompture/drivers/`** — One module per LLM provider (openai, Codex, google, groq, grok, azure, ollama, lmstudio, openrouter, local_http, huggingface, airllm). Each driver implements `generate(prompt, options)` returning a standardized response with token/cost metadata.
+- **`prompture/drivers/__init__.py`** — Central `DRIVER_REGISTRY` dict mapping provider name to factory lambda. `get_driver_for_model("provider/model")` parses the string and instantiates the right driver. `get_driver("provider")` is the legacy interface.
+- **`prompture/tools.py`** — Utilities: JSON/TOON text cleanup, type conversion (shorthand numbers, multilingual booleans, datetimes), field schema generation.
+- **`prompture/logging.py`** — Library logging configuration: `JSONFormatter` for structured JSON-lines output, `configure_logging()` for enabling library-level logging via Python stdlib `logging`.
+- **`prompture/callbacks.py`** — `DriverCallbacks` dataclass with `on_request`, `on_response`, `on_error` hooks for driver-level observability.
+- **`prompture/session.py`** — `UsageSession` dataclass for tracking token counts, costs, and errors across multiple driver calls.
+- **`prompture/field_definitions.py`** — Thread-safe global field registry with 50+ predefined fields, template variable substitution (`{{current_year}}`, `{{current_date}}`), and Pydantic Field generation via `field_from_registry()`.
+- **`prompture/settings.py`** — Pydantic-settings `Settings` class loading provider API keys/endpoints from `.env`.
+- **`prompture/discovery.py`** — `get_available_models()` auto-detects models from configured providers (static pricing tables + dynamic Ollama endpoint query).
+- **`prompture/runner.py`** — Spec-driven test suite runner for cross-model comparison.
+- **`prompture/validator.py`** — JSON schema validation via jsonschema with fallback.
+
+### Key Patterns
+
+- **Model strings** use `"provider/model"` format (e.g., `"ollama/llama3.1:8b"`, `"openai/gpt-4"`). The provider prefix routes to the correct driver.
+- **Driver responses** always include metadata: `prompt_tokens`, `completion_tokens`, `total_tokens`, `cost`, `raw_response`.
+- **Output formats**: JSON (default) and TOON (experimental, for compact output). Controlled via `output_format` parameter.
+- **TOON input conversion** uses `python-toon` and `tukuy` packages to reduce token usage by 45-60% when sending structured data to LLMs.
+
+### Testing
+
+Tests live in `tests/`. Integration tests that call live LLMs are marked with `@pytest.mark.integration` and skipped by default. The default test model is set in `tests/conftest.py` as `DEFAULT_MODEL`. Shared fixtures and assertion helpers (`assert_valid_usage_metadata`, `assert_jsonify_response_structure`) are in `conftest.py`.
+
+### Configuration
+
+Provider API keys and endpoints are configured via environment variables loaded from `.env` (see `.env.copy` for the template). The `Settings` class in `settings.py` manages all provider config.
+
+### Versioning
+
+Uses `setuptools_scm` for automatic version from git tags. The `VERSION` file contains the current dev version.

{prompture-1.0.52.dev1 → prompture-1.0.53.dev1}/BREAKING_CHANGES.md

@@ -2,7 +2,7 @@
 
 This log records every backward-incompatible change to Prompture's
 **stable** APIs. Experimental surfaces may change without an entry here
-(see [API Stability](docs/source/stability.rst) for tier definitions).
+(see [API Stability](prompture/pages/06-stability.md) for tier definitions).
 
 Changes are listed newest-first.
 

{prompture-1.0.52.dev1 → prompture-1.0.53.dev1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: prompture
-Version: 1.0.52.dev1
+Version: 1.0.53.dev1
 Summary: Ask LLMs to return structured JSON and run cross-model tests. API-first.
 Author-email: Juan Denis <juan@vene.co>
 License-Expression: MIT
@@ -95,7 +95,10 @@ print(person.name)  # Maria
 ## Key Features
 
 - **Structured output** — JSON schema enforcement and direct Pydantic model population
-- **12 providers** — OpenAI, Claude, Google, Groq, Grok, Azure, Ollama, LM Studio, OpenRouter, HuggingFace, AirLLM, and generic HTTP
+- **17+ providers** — OpenAI, Claude, Google, Groq, Grok, Azure, Ollama, LM Studio, OpenRouter, HuggingFace, Moonshot, ModelScope, Z.ai, Vertex AI, AirLLM, CachiBot, and generic HTTP
+- **Multi-modal** — Drivers for embeddings, image generation (DALL-E, Imagen, Grok, Stability), text-to-speech, and speech-to-text (Whisper, ElevenLabs)
+- **Multi-model fallback** — Try a list of models in sequence with per-attempt cost, token, and capability accounting
+- **Strategy cascade** — Auto-selects between provider-native JSON mode, tool-call extraction, and prompted repair so extraction works on any model
 - **TOON input conversion** — 45-60% token savings when sending structured data via [Token-Oriented Object Notation](https://github.com/jhd3197/python-toon)
 - **Stepwise extraction** — Per-field prompts with smart type coercion (shorthand numbers, multilingual booleans, dates)
 - **Field registry** — 50+ predefined extraction fields with template variables and Pydantic integration
@@ -172,14 +175,42 @@ Model strings use `"provider/model"` format. The provider prefix routes to the c
 | `openai` | `openai/gpt-4` | Automatic |
 | `claude` | `claude/claude-3` | Automatic |
 | `google` | `google/gemini-1.5-pro` | Automatic |
+| `google_vertexai` | `google_vertexai/gemini-1.5-pro` | Automatic |
 | `groq` | `groq/llama2-70b-4096` | Automatic |
 | `grok` | `grok/grok-4-fast-reasoning` | Automatic |
 | `azure` | `azure/deployed-name` | Automatic |
 | `openrouter` | `openrouter/anthropic/claude-2` | Automatic |
+| `moonshot` | `moonshot/kimi-k2` | Automatic |
+| `modelscope` | `modelscope/Qwen2.5-72B-Instruct` | Automatic |
+| `zai` | `zai/glm-4` | Automatic |
+| `cachibot` | `cachibot/openai/gpt-4o-mini` | Automatic |
 | `ollama` | `ollama/llama3.1:8b` | Free (local) |
 | `lmstudio` | `lmstudio/local-model` | Free (local) |
 | `huggingface` | `hf/model-name` | Free (local) |
-| `http` | `http/self-hosted` | Free |
+| `airllm` | `airllm/Qwen2-7B` | Free (local) |
+| `local_http` | `local_http/self-hosted` | Free |
+
+Aliases (`anthropic`, `gemini`, `chatgpt`, `xai`, `lm_studio`, `zhipu`, `hf`, `dalle`) route to their canonical providers.
+
+## Multi-Modal
+
+Beyond text LLMs, Prompture exposes drivers for adjacent modalities under the same `provider/model` routing:
+
+- **Embeddings** — OpenAI (`text-embedding-3-*`) and Ollama (`nomic-embed-text`)
+- **Image generation** — OpenAI DALL-E, Google Imagen, Grok, Stability AI
+- **Text-to-speech** — OpenAI (`tts-1`) and ElevenLabs
+- **Speech-to-text** — OpenAI Whisper and ElevenLabs
+
+```python
+from prompture.drivers.img_gen_registry import get_img_gen_driver_for_model
+
+driver = get_img_gen_driver_for_model("openai/dall-e-3")
+result = driver.generate_image(
+    "a cat on a surfboard at sunset",
+    {"size": "1024x1024", "quality": "hd"},
+)
+print(result["meta"]["cost"], result["meta"]["image_count"])
+```
 
 ## Usage
 
@@ -256,6 +287,48 @@ print(result["json_object"])  # {"name": "John", "age": 28}
 print(result["usage"])        # token counts and cost
 ```
 
+### Strategy Cascade
+
+Prompture picks how to obtain structured JSON based on each model's capabilities. The cascade is `provider_native` (built-in JSON mode / schema enforcement) → `tool_call` (encode the schema as a function definition and read it back from the tool call) → `prompted_repair` (prompt for JSON, repair malformed output via AI cleanup). Pass `strategy="auto"` (default) to let Prompture select per model, or pin a specific strategy via the `StructuredOutputStrategy` enum or its string value. The strategy used is recorded in the response so you can see which path each call took.
+
+### Multi-Model Fallback
+
+Try a list of models in priority order, with full per-attempt accounting — every model tried (success, failure, or skipped) is recorded with its cost, tokens, duration, capabilities, and strategy. The first success wins; if all fail, an optional `fallback` Pydantic instance is returned instead of raising.
+
+```python
+from prompture import extract_with_models
+
+result = extract_with_models(
+    Person,
+    "Maria is 32, a software developer in NYC.",
+    models=[
+        "openai/gpt-4o-mini",        # try first
+        "claude/claude-3-5-haiku",   # fallback
+        "ollama/llama3.1:8b",        # last resort, free
+    ],
+    fallback=Person(name="unknown", age=0, profession="unknown"),
+)
+
+print(result["selected_model"])     # winning model string
+print(result["model"])              # validated Pydantic instance
+print(result["total_cost"])         # cumulative cost across all attempts
+print(result["total_attempts"])     # number of models actually called
+
+for attempt in result["attempts"]:
+    print(
+        attempt["model"],
+        attempt["status"],          # "success" | "failed" | "skipped"
+        attempt["strategy"],        # "single" | "stepwise"
+        attempt["cost"],
+        attempt["prompt_tokens"],
+        attempt["completion_tokens"],
+        attempt["duration_ms"],
+        attempt["capabilities"],    # {"json_mode": bool, "json_schema": bool}
+    )
+```
+
+If every model fails and no `fallback` is provided, an `ExtractionError` is raised with the full `attempts` list, `total_cost`, and `total_tokens` attached as attributes.
+
 ### TOON Input — Token Savings
 
 Analyze structured data with automatic TOON conversion for 45-60% fewer tokens:

{prompture-1.0.52.dev1 → prompture-1.0.53.dev1}/README.md

@@ -31,7 +31,10 @@ print(person.name)  # Maria
 ## Key Features
 
 - **Structured output** — JSON schema enforcement and direct Pydantic model population
-- **12 providers** — OpenAI, Claude, Google, Groq, Grok, Azure, Ollama, LM Studio, OpenRouter, HuggingFace, AirLLM, and generic HTTP
+- **17+ providers** — OpenAI, Claude, Google, Groq, Grok, Azure, Ollama, LM Studio, OpenRouter, HuggingFace, Moonshot, ModelScope, Z.ai, Vertex AI, AirLLM, CachiBot, and generic HTTP
+- **Multi-modal** — Drivers for embeddings, image generation (DALL-E, Imagen, Grok, Stability), text-to-speech, and speech-to-text (Whisper, ElevenLabs)
+- **Multi-model fallback** — Try a list of models in sequence with per-attempt cost, token, and capability accounting
+- **Strategy cascade** — Auto-selects between provider-native JSON mode, tool-call extraction, and prompted repair so extraction works on any model
 - **TOON input conversion** — 45-60% token savings when sending structured data via [Token-Oriented Object Notation](https://github.com/jhd3197/python-toon)
 - **Stepwise extraction** — Per-field prompts with smart type coercion (shorthand numbers, multilingual booleans, dates)
 - **Field registry** — 50+ predefined extraction fields with template variables and Pydantic integration
@@ -108,14 +111,42 @@ Model strings use `"provider/model"` format. The provider prefix routes to the c
 | `openai` | `openai/gpt-4` | Automatic |
 | `claude` | `claude/claude-3` | Automatic |
 | `google` | `google/gemini-1.5-pro` | Automatic |
+| `google_vertexai` | `google_vertexai/gemini-1.5-pro` | Automatic |
 | `groq` | `groq/llama2-70b-4096` | Automatic |
 | `grok` | `grok/grok-4-fast-reasoning` | Automatic |
 | `azure` | `azure/deployed-name` | Automatic |
 | `openrouter` | `openrouter/anthropic/claude-2` | Automatic |
+| `moonshot` | `moonshot/kimi-k2` | Automatic |
+| `modelscope` | `modelscope/Qwen2.5-72B-Instruct` | Automatic |
+| `zai` | `zai/glm-4` | Automatic |
+| `cachibot` | `cachibot/openai/gpt-4o-mini` | Automatic |
 | `ollama` | `ollama/llama3.1:8b` | Free (local) |
 | `lmstudio` | `lmstudio/local-model` | Free (local) |
 | `huggingface` | `hf/model-name` | Free (local) |
-| `http` | `http/self-hosted` | Free |
+| `airllm` | `airllm/Qwen2-7B` | Free (local) |
+| `local_http` | `local_http/self-hosted` | Free |
+
+Aliases (`anthropic`, `gemini`, `chatgpt`, `xai`, `lm_studio`, `zhipu`, `hf`, `dalle`) route to their canonical providers.
+
+## Multi-Modal
+
+Beyond text LLMs, Prompture exposes drivers for adjacent modalities under the same `provider/model` routing:
+
+- **Embeddings** — OpenAI (`text-embedding-3-*`) and Ollama (`nomic-embed-text`)
+- **Image generation** — OpenAI DALL-E, Google Imagen, Grok, Stability AI
+- **Text-to-speech** — OpenAI (`tts-1`) and ElevenLabs
+- **Speech-to-text** — OpenAI Whisper and ElevenLabs
+
+```python
+from prompture.drivers.img_gen_registry import get_img_gen_driver_for_model
+
+driver = get_img_gen_driver_for_model("openai/dall-e-3")
+result = driver.generate_image(
+    "a cat on a surfboard at sunset",
+    {"size": "1024x1024", "quality": "hd"},
+)
+print(result["meta"]["cost"], result["meta"]["image_count"])
+```
 
 ## Usage
 
@@ -192,6 +223,48 @@ print(result["json_object"])  # {"name": "John", "age": 28}
 print(result["usage"])        # token counts and cost
 ```
 
+### Strategy Cascade
+
+Prompture picks how to obtain structured JSON based on each model's capabilities. The cascade is `provider_native` (built-in JSON mode / schema enforcement) → `tool_call` (encode the schema as a function definition and read it back from the tool call) → `prompted_repair` (prompt for JSON, repair malformed output via AI cleanup). Pass `strategy="auto"` (default) to let Prompture select per model, or pin a specific strategy via the `StructuredOutputStrategy` enum or its string value. The strategy used is recorded in the response so you can see which path each call took.
+
+### Multi-Model Fallback
+
+Try a list of models in priority order, with full per-attempt accounting — every model tried (success, failure, or skipped) is recorded with its cost, tokens, duration, capabilities, and strategy. The first success wins; if all fail, an optional `fallback` Pydantic instance is returned instead of raising.
+
+```python
+from prompture import extract_with_models
+
+result = extract_with_models(
+    Person,
+    "Maria is 32, a software developer in NYC.",
+    models=[
+        "openai/gpt-4o-mini",        # try first
+        "claude/claude-3-5-haiku",   # fallback
+        "ollama/llama3.1:8b",        # last resort, free
+    ],
+    fallback=Person(name="unknown", age=0, profession="unknown"),
+)
+
+print(result["selected_model"])     # winning model string
+print(result["model"])              # validated Pydantic instance
+print(result["total_cost"])         # cumulative cost across all attempts
+print(result["total_attempts"])     # number of models actually called
+
+for attempt in result["attempts"]:
+    print(
+        attempt["model"],
+        attempt["status"],          # "success" | "failed" | "skipped"
+        attempt["strategy"],        # "single" | "stepwise"
+        attempt["cost"],
+        attempt["prompt_tokens"],
+        attempt["completion_tokens"],
+        attempt["duration_ms"],
+        attempt["capabilities"],    # {"json_mode": bool, "json_schema": bool}
+    )
+```
+
+If every model fails and no `fallback` is provided, an `ExtractionError` is raised with the full `attempts` list, `total_cost`, and `total_tokens` attached as attributes.
+
 ### TOON Input — Token Savings
 
 Analyze structured data with automatic TOON conversion for 45-60% fewer tokens:

{prompture-1.0.52.dev1 → prompture-1.0.53.dev1}/prompture/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '1.0.52.dev1'
-__version_tuple__ = version_tuple = (1, 0, 52, 'dev1')
+__version__ = version = '1.0.53.dev1'
+__version_tuple__ = version_tuple = (1, 0, 53, 'dev1')
 
 __commit_id__ = commit_id = None

{prompture-1.0.52.dev1 → prompture-1.0.53.dev1}/prompture/drivers/async_claude_driver.py

@@ -14,7 +14,15 @@ except Exception:
 
 from ..infra.cost_mixin import CostMixin
 from .async_base import AsyncDriver
-from .claude_driver import ClaudeDriver
+from .claude_driver import (
+    ClaudeDriver,
+    _build_anthropic_json_mode_tool_def,
+    _build_anthropic_meta,
+    _build_anthropic_stream_done,
+    _convert_tools_to_anthropic,
+    _extract_anthropic_system_and_messages,
+    _extract_anthropic_text_and_tool_calls,
+)
 
 
 class AsyncClaudeDriver(CostMixin, AsyncDriver):
@@ -60,10 +68,8 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
 
         client = anthropic.AsyncAnthropic(api_key=self.api_key)
 
-        # Anthropic requires system messages as a top-level parameter
-        system_content, api_messages = self._extract_system_and_messages(messages)
+        system_content, api_messages = _extract_anthropic_system_and_messages(messages)
 
-        # Build common kwargs
         common_kwargs: dict[str, Any] = {
             "model": model,
             "messages": api_messages,
@@ -77,14 +83,9 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
         if options.get("json_mode"):
             json_schema = options.get("json_schema")
             if json_schema:
-                tool_def = {
-                    "name": "extract_json",
-                    "description": "Extract structured data matching the schema",
-                    "input_schema": json_schema,
-                }
                 resp = await client.messages.create(  # type: ignore[call-overload]
                     **common_kwargs,
-                    tools=[tool_def],
+                    tools=[_build_anthropic_json_mode_tool_def(json_schema)],
                     tool_choice={"type": "tool", "name": "extract_json"},
                 )
                 text = ""
@@ -99,27 +100,14 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
             resp = await client.messages.create(**common_kwargs)
             text = resp.content[0].text
 
-        # Extract reasoning/thinking content from content blocks
         reasoning_content = ClaudeDriver._extract_thinking(resp.content)
-
-        # Fallback: use reasoning as text if content is empty
         if not text and reasoning_content:
             text = reasoning_content
 
-        prompt_tokens = resp.usage.input_tokens
-        completion_tokens = resp.usage.output_tokens
-        total_tokens = prompt_tokens + completion_tokens
-
-        total_cost = self._calculate_cost("claude", model, prompt_tokens, completion_tokens)
-
-        meta = {
-            "prompt_tokens": prompt_tokens,
-            "completion_tokens": completion_tokens,
-            "total_tokens": total_tokens,
-            "cost": round(total_cost, 6),
-            "raw_response": dict(resp),
-            "model_name": model,
-        }
+        total_cost = self._calculate_cost(
+            "claude", model, resp.usage.input_tokens, resp.usage.output_tokens
+        )
+        meta = _build_anthropic_meta(resp, model, total_cost)
 
         result: dict[str, Any] = {"text": text, "meta": meta}
         if reasoning_content is not None:
@@ -131,78 +119,7 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
     # ------------------------------------------------------------------
 
     def _extract_system_and_messages(self, messages: list[dict[str, Any]]) -> tuple[str | None, list[dict[str, Any]]]:
-        """Separate system message and translate tool messages for Anthropic API.
-
-        The conversation layer stores tool use in OpenAI-compatible shape
-        (assistant messages with a ``tool_calls`` list and separate
-        ``role: "tool"`` result messages). Anthropic rejects role=tool and
-        expects:
-
-        - Assistant tool calls → ``content=[{"type": "text", ...}, {"type": "tool_use", "id": ..., "name": ..., "input": {...}}]``
-        - Tool results → a user message whose content is ``[{"type": "tool_result", "tool_use_id": ..., "content": ...}]``
-
-        Consecutive tool results are merged into a single user message so
-        the API sees one user turn per assistant turn, as required.
-        """
-        system_content: str | None = None
-        api_messages: list[dict[str, Any]] = []
-        for msg in messages:
-            role = msg.get("role")
-            if role == "system":
-                system_content = msg.get("content", "")
-                continue
-
-            if role == "assistant" and msg.get("tool_calls"):
-                content = msg.get("content") or ""
-                blocks: list[dict[str, Any]] = []
-                if content:
-                    blocks.append({"type": "text", "text": content})
-                for tc in msg["tool_calls"]:
-                    fn = tc.get("function", tc)
-                    raw_args = fn.get("arguments", tc.get("arguments", {}))
-                    if isinstance(raw_args, str):
-                        try:
-                            tool_input = json.loads(raw_args) if raw_args else {}
-                        except json.JSONDecodeError:
-                            tool_input = {"_raw": raw_args}
-                    else:
-                        tool_input = raw_args or {}
-                    blocks.append(
-                        {
-                            "type": "tool_use",
-                            "id": tc.get("id", ""),
-                            "name": fn.get("name", tc.get("name", "")),
-                            "input": tool_input,
-                        }
-                    )
-                api_messages.append({"role": "assistant", "content": blocks})
-                continue
-
-            if role == "tool":
-                result_block = {
-                    "type": "tool_result",
-                    "tool_use_id": msg.get("tool_call_id", ""),
-                    "content": msg.get("content", ""),
-                }
-                # Merge with a preceding user message whose content is a list
-                # of tool_result blocks — Anthropic prefers one user turn per
-                # assistant turn even with multiple tool calls.
-                if (
-                    api_messages
-                    and api_messages[-1].get("role") == "user"
-                    and isinstance(api_messages[-1].get("content"), list)
-                    and all(
-                        isinstance(b, dict) and b.get("type") == "tool_result"
-                        for b in api_messages[-1]["content"]
-                    )
-                ):
-                    api_messages[-1]["content"].append(result_block)
-                else:
-                    api_messages.append({"role": "user", "content": [result_block]})
-                continue
-
-            api_messages.append(msg)
-        return system_content, api_messages
+        return _extract_anthropic_system_and_messages(messages)
 
     # ------------------------------------------------------------------
     # Tool use
@@ -225,26 +142,8 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
 
         client = anthropic.AsyncAnthropic(api_key=self.api_key)
 
-        system_content, api_messages = self._extract_system_and_messages(messages)
-
-        # Convert tools from OpenAI format to Anthropic format if needed
-        anthropic_tools = []
-        for t in tools:
-            if "type" in t and t["type"] == "function":
-                # OpenAI format -> Anthropic format
-                fn = t["function"]
-                anthropic_tools.append(
-                    {
-                        "name": fn["name"],
-                        "description": fn.get("description", ""),
-                        "input_schema": fn.get("parameters", {"type": "object", "properties": {}}),
-                    }
-                )
-            elif "input_schema" in t:
-                # Already Anthropic format
-                anthropic_tools.append(t)
-            else:
-                anthropic_tools.append(t)
+        system_content, api_messages = _extract_anthropic_system_and_messages(messages)
+        anthropic_tools = _convert_tools_to_anthropic(tools)
 
         kwargs: dict[str, Any] = {
             "model": model,
@@ -258,34 +157,12 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
 
         resp = await client.messages.create(**kwargs)
 
-        prompt_tokens = resp.usage.input_tokens
-        completion_tokens = resp.usage.output_tokens
-        total_tokens = prompt_tokens + completion_tokens
-        total_cost = self._calculate_cost("claude", model, prompt_tokens, completion_tokens)
-
-        meta = {
-            "prompt_tokens": prompt_tokens,
-            "completion_tokens": completion_tokens,
-            "total_tokens": total_tokens,
-            "cost": round(total_cost, 6),
-            "raw_response": dict(resp),
-            "model_name": model,
-        }
-
-        text = ""
-        tool_calls_out: list[dict[str, Any]] = []
-        for block in resp.content:
-            if block.type == "text":
-                text += block.text
-            elif block.type == "tool_use":
-                tool_calls_out.append(
-                    {
-                        "id": block.id,
-                        "name": block.name,
-                        "arguments": block.input,
-                    }
-                )
+        total_cost = self._calculate_cost(
+            "claude", model, resp.usage.input_tokens, resp.usage.output_tokens
+        )
+        meta = _build_anthropic_meta(resp, model, total_cost)
 
+        text, tool_calls_out = _extract_anthropic_text_and_tool_calls(resp.content)
         reasoning_content = ClaudeDriver._extract_thinking(resp.content)
 
         result: dict[str, Any] = {
@@ -315,7 +192,7 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
         model = options.get("model", self.model)
         client = anthropic.AsyncAnthropic(api_key=self.api_key)
 
-        system_content, api_messages = self._extract_system_and_messages(messages)
+        system_content, api_messages = _extract_anthropic_system_and_messages(messages)
 
         kwargs: dict[str, Any] = {
             "model": model,
@@ -353,21 +230,7 @@ class AsyncClaudeDriver(CostMixin, AsyncDriver):
                         if usage:
                             prompt_tokens = getattr(usage, "input_tokens", 0)
 
-        total_tokens = prompt_tokens + completion_tokens
         total_cost = self._calculate_cost("claude", model, prompt_tokens, completion_tokens)
-
-        done_chunk: dict[str, Any] = {
-            "type": "done",
-            "text": full_text,
-            "meta": {
-                "prompt_tokens": prompt_tokens,
-                "completion_tokens": completion_tokens,
-                "total_tokens": total_tokens,
-                "cost": round(total_cost, 6),
-                "raw_response": {},
-                "model_name": model,
-            },
-        }
-        if full_reasoning:
-            done_chunk["reasoning_content"] = full_reasoning
-        yield done_chunk
+        yield _build_anthropic_stream_done(
+            model, full_text, full_reasoning, prompt_tokens, completion_tokens, total_cost
+        )