aidial-sdk 0.47.0.dev5__tar.gz → 0.47.0.dev6__tar.gz

{aidial_sdk-0.47.0.dev5 → aidial_sdk-0.47.0.dev6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: aidial-sdk
-Version: 0.47.0.dev5
+Version: 0.47.0.dev6
 Summary: Framework to create applications and model adapters for AI DIAL
 License-Expression: Apache-2.0
 License-File: LICENSE
@@ -14,8 +14,12 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Provides-Extra: httpx
 Provides-Extra: telemetry
+Provides-Extra: translators
+Requires-Dist: anthropic (>=0.105.0,<1.0) ; extra == "translators"
 Requires-Dist: fastapi (>=0.51,<1.0)
 Requires-Dist: httpx (>=0.25.0,<1.0) ; extra == "httpx"
+Requires-Dist: httpx (>=0.25.0,<1.0) ; extra == "translators"
+Requires-Dist: openai (>=2.46.0,<3.0) ; extra == "translators"
 Requires-Dist: opentelemetry-api (>=1.22.0,<2.0) ; extra == "telemetry"
 Requires-Dist: opentelemetry-exporter-otlp-proto-grpc (>=1.22.0,<2.0) ; extra == "telemetry"
 Requires-Dist: opentelemetry-exporter-prometheus (>=0.64b0) ; extra == "telemetry"
@@ -61,6 +65,7 @@ Description-Content-Type: text/markdown
     - [Run](#run)
     - [Check](#check)
   - [Endpoints](#endpoints)
+  - [Anthropic Messages to Chat Completions translator](#anthropic-messages-to-chat-completions-translator)
 - [Development](#development)
   - [Development Environment](#development-environment)
   - [Setup](#setup)
@@ -205,6 +210,72 @@ An `/openai/v1` request with none of the headers set fails with `500 Internal Se
 
 ---
 
+### Anthropic Messages to Chat Completions translator
+
+The translator lets an Anthropic-native client (e.g. Claude Code) talk to a model that only speaks the [Chat Completions API](https://platform.openai.com/docs/api-reference/chat). It converts an Anthropic Messages request into a Chat Completions request to DIAL Core at `DIAL_URL`, and converts the response, streaming or not, back into the Anthropic Messages shape.
+
+The translator requires the `translators` extra and Pydantic V2 models (`PYDANTIC_V2=True`):
+
+```sh
+pip install "aidial-sdk[translators]"
+```
+
+Mount it on your application:
+
+```python
+import httpx
+
+from aidial_sdk import DIALApp
+from aidial_sdk.translators.messages_to_chat_completions import mount_translator
+
+app = DIALApp()
+mount_translator(app, httpx.AsyncClient())
+```
+
+The second argument is the HTTP client the translator calls DIAL Core with: either an `httpx.AsyncClient` or an async function that takes the incoming `Request` and returns one. The `path` (`/openaiChatCompletions` by default) and `name` keyword arguments configure the mount.
+
+|Endpoint|Calls on DIAL Core|
+|---|---|
+|`POST /openaiChatCompletions/anthropic/v1/messages`|`POST {DIAL_URL}/openai/v1/chat/completions`|
+|`POST /openaiChatCompletions/legacy/anthropic/v1/messages`|`POST {DIAL_URL}/openai/deployments/{deployment}/chat/completions`|
+
+`DIAL_URL` must be set, otherwise the endpoints answer `500`.
+
+To expose a Chat Completions model under `/anthropic/v1/messages`, add an `anthropicMessages` interface to its DIAL Core config, pointing `base_url` at the mount path of the application that runs the translator (or at `/openaiChatCompletions/legacy` to call DIAL Core's legacy per-deployment endpoint):
+
+<details>
+<summary>DIAL Core config example</summary>
+
+```json
+{
+  "models": {
+    "gpt-5.5": {
+      "type": "chat",
+      "interfaces": {
+        "openaiChatCompletions": {
+          "base_url": "${OPENAI_ADAPTER_ORIGIN}"
+        },
+        "anthropicMessages": {
+          "base_url": "${TRANSLATOR_ORIGIN}/openaiChatCompletions"
+        }
+      },
+      "upstreams": [
+        {
+          "endpoint": "https://api.openai.com/v1/chat/completions",
+          "key": "..."
+        }
+      ]
+    }
+  }
+}
+```
+
+</details>
+
+See the [translation guide](docs/messages-to-chat-translation.md) for how request parameters, content blocks, tools, caching, usage, and errors are translated, and for the translator's limitations.
+
+---
+
 ## Development
 
 ### Development Environment

{aidial_sdk-0.47.0.dev5 → aidial_sdk-0.47.0.dev6}/README.md

@@ -23,6 +23,7 @@
     - [Run](#run)
     - [Check](#check)
   - [Endpoints](#endpoints)
+  - [Anthropic Messages to Chat Completions translator](#anthropic-messages-to-chat-completions-translator)
 - [Development](#development)
   - [Development Environment](#development-environment)
   - [Setup](#setup)
@@ -167,6 +168,72 @@ An `/openai/v1` request with none of the headers set fails with `500 Internal Se
 
 ---
 
+### Anthropic Messages to Chat Completions translator
+
+The translator lets an Anthropic-native client (e.g. Claude Code) talk to a model that only speaks the [Chat Completions API](https://platform.openai.com/docs/api-reference/chat). It converts an Anthropic Messages request into a Chat Completions request to DIAL Core at `DIAL_URL`, and converts the response, streaming or not, back into the Anthropic Messages shape.
+
+The translator requires the `translators` extra and Pydantic V2 models (`PYDANTIC_V2=True`):
+
+```sh
+pip install "aidial-sdk[translators]"
+```
+
+Mount it on your application:
+
+```python
+import httpx
+
+from aidial_sdk import DIALApp
+from aidial_sdk.translators.messages_to_chat_completions import mount_translator
+
+app = DIALApp()
+mount_translator(app, httpx.AsyncClient())
+```
+
+The second argument is the HTTP client the translator calls DIAL Core with: either an `httpx.AsyncClient` or an async function that takes the incoming `Request` and returns one. The `path` (`/openaiChatCompletions` by default) and `name` keyword arguments configure the mount.
+
+|Endpoint|Calls on DIAL Core|
+|---|---|
+|`POST /openaiChatCompletions/anthropic/v1/messages`|`POST {DIAL_URL}/openai/v1/chat/completions`|
+|`POST /openaiChatCompletions/legacy/anthropic/v1/messages`|`POST {DIAL_URL}/openai/deployments/{deployment}/chat/completions`|
+
+`DIAL_URL` must be set, otherwise the endpoints answer `500`.
+
+To expose a Chat Completions model under `/anthropic/v1/messages`, add an `anthropicMessages` interface to its DIAL Core config, pointing `base_url` at the mount path of the application that runs the translator (or at `/openaiChatCompletions/legacy` to call DIAL Core's legacy per-deployment endpoint):
+
+<details>
+<summary>DIAL Core config example</summary>
+
+```json
+{
+  "models": {
+    "gpt-5.5": {
+      "type": "chat",
+      "interfaces": {
+        "openaiChatCompletions": {
+          "base_url": "${OPENAI_ADAPTER_ORIGIN}"
+        },
+        "anthropicMessages": {
+          "base_url": "${TRANSLATOR_ORIGIN}/openaiChatCompletions"
+        }
+      },
+      "upstreams": [
+        {
+          "endpoint": "https://api.openai.com/v1/chat/completions",
+          "key": "..."
+        }
+      ]
+    }
+  }
+}
+```
+
+</details>
+
+See the [translation guide](docs/messages-to-chat-translation.md) for how request parameters, content blocks, tools, caching, usage, and errors are translated, and for the translator's limitations.
+
+---
+
 ## Development
 
 ### Development Environment

aidial_sdk-0.47.0.dev6/aidial_sdk/translators/messages_to_chat_completions/__init__.py

@@ -0,0 +1,28 @@
+import httpx
+from fastapi import FastAPI
+
+from aidial_sdk._pydantic import PYDANTIC_V2
+
+if not PYDANTIC_V2:
+    raise ImportError(
+        "The Anthropic Messages API translator requires Pydantic v2: "
+        "install pydantic>=2 and set PYDANTIC_V2=True"
+    )
+
+from aidial_sdk.translators.messages_to_chat_completions.chat_completions.app import (  # noqa: E402
+    ClientFactory,
+    create_anthropic_api_app,
+)
+
+__all__ = ["ClientFactory", "create_anthropic_api_app", "mount_translator"]
+
+
+def mount_translator(
+    app: FastAPI,
+    get_client: ClientFactory[httpx.AsyncClient],
+    *,
+    path: str = "/openaiChatCompletions",
+    name: str = "Anthropic Messages API to Chat Completions API translator",
+) -> None:
+    translator = create_anthropic_api_app(get_client)
+    app.mount(path=path, app=translator, name=name)

aidial_sdk-0.47.0.dev6/aidial_sdk/translators/messages_to_chat_completions/chat_completions/app.py

@@ -0,0 +1,134 @@
+import json
+import logging
+from collections.abc import Awaitable, Callable
+from typing import TypeVar
+
+import httpx
+from anthropic.types.beta.message_create_params import MessageCreateParams
+from fastapi import FastAPI, Request
+from fastapi.responses import JSONResponse, Response, StreamingResponse
+from openai import AsyncStream
+from openai.types.chat import ChatCompletion, ChatCompletionChunk
+
+from aidial_sdk.translators.messages_to_chat_completions.chat_completions.from_chat_completions import (
+    to_anthropic_response,
+)
+from aidial_sdk.translators.messages_to_chat_completions.chat_completions.streaming import (
+    translate_stream,
+)
+from aidial_sdk.translators.messages_to_chat_completions.chat_completions.to_chat_completions import (
+    to_chat_completions_request,
+)
+from aidial_sdk.translators.messages_to_chat_completions.common import (
+    not_found,
+    with_debug_logging,
+)
+from aidial_sdk.translators.messages_to_chat_completions.core_client import (
+    core_headers,
+    core_legacy_client,
+    core_v1_client,
+)
+from aidial_sdk.translators.messages_to_chat_completions.errors import (
+    AnthropicErrorType,
+    AnthropicHTTPError,
+    translator_error_handler,
+)
+from aidial_sdk.translators.messages_to_chat_completions.settings import (
+    get_dial_url,
+)
+from aidial_sdk.translators.messages_to_chat_completions.translation_state import (
+    TranslationState,
+)
+from aidial_sdk.utils.logging import logger as log
+
+ClientT = TypeVar("ClientT")
+ClientFactory = Callable[[Request], Awaitable[ClientT]] | ClientT
+
+
+async def _handle_messages(
+    request: Request,
+    get_client: ClientFactory[httpx.AsyncClient],
+    legacy: bool,
+) -> Response:
+    dial_url = get_dial_url()
+    translation_state = TranslationState()
+
+    try:
+        req: MessageCreateParams = await request.json()
+    except ValueError as e:
+        raise AnthropicHTTPError(
+            AnthropicErrorType.INVALID_REQUEST,
+            f"Request body is not valid JSON: {e}",
+        ) from e
+
+    deployment = request.headers.get("x-dial-deployment-id") or req["model"]
+    chat_request = to_chat_completions_request(
+        req, deployment, translation_state
+    )
+
+    body = chat_request.model_dump(mode="json", exclude_none=True)
+    if log.isEnabledFor(logging.DEBUG):
+        log.debug(f"Translated request to Core: {json.dumps(body)}")
+
+    http_client = (
+        get_client
+        if isinstance(get_client, httpx.AsyncClient)
+        else await get_client(request)
+    )
+    client = (
+        core_legacy_client(dial_url, deployment, http_client)
+        if legacy
+        else core_v1_client(dial_url, http_client)
+    )
+    headers = core_headers(request)
+
+    if chat_request.stream:
+        events = await client.post(
+            "/chat/completions",
+            cast_to=ChatCompletion,
+            body=body,
+            options={"headers": headers},
+            stream=True,
+            stream_cls=AsyncStream[ChatCompletionChunk],
+        )
+        return StreamingResponse(
+            translate_stream(events, deployment, translation_state),
+            media_type="text/event-stream",
+        )
+
+    response = await client.post(
+        "/chat/completions",
+        cast_to=ChatCompletion,
+        body=body,
+        options={"headers": headers},
+    )
+    message = to_anthropic_response(response, deployment, translation_state)
+    return JSONResponse(
+        content=message.model_dump(mode="json", exclude_unset=True)
+    )
+
+
+def create_anthropic_api_app(
+    get_client: ClientFactory[httpx.AsyncClient],
+) -> FastAPI:
+    app = FastAPI()
+
+    @app.post("/anthropic/v1/messages")
+    @with_debug_logging
+    @translator_error_handler
+    async def _handle_v1_messages(request: Request) -> Response:
+        return await _handle_messages(request, get_client, legacy=False)
+
+    @app.post("/legacy/anthropic/v1/messages")
+    @with_debug_logging
+    @translator_error_handler
+    async def _handle_legacy_messages(request: Request) -> Response:
+        return await _handle_messages(request, get_client, legacy=True)
+
+    app.add_api_route(
+        "/{full_path:path}",
+        not_found,
+        methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"],
+    )
+
+    return app

aidial_sdk-0.47.0.dev6/aidial_sdk/translators/messages_to_chat_completions/chat_completions/cache_breakpoints.py

@@ -0,0 +1,48 @@
+import re
+from datetime import UTC, datetime, timedelta
+
+from anthropic.types.beta import BetaCacheControlEphemeralParam as CacheControl
+
+from aidial_sdk.chat_completion.request import CacheBreakpoint
+from aidial_sdk.utils.logging import logger as log
+
+_TTL = re.compile(r"^([1-9]\d*)([mh])$")
+
+_UNITS = {
+    "m": timedelta(minutes=1),
+    "h": timedelta(hours=1),
+}
+
+
+def to_dial_cache_breakpoint(
+    controls: list[CacheControl],
+) -> CacheBreakpoint | None:
+    if not controls:
+        return None
+    expires_at: str | None = None
+    for control in controls:
+        try:
+            if (ttl := _ttl(control)) is not None:
+                candidate = _expire_at(ttl)
+                expires_at = (
+                    max(expires_at, candidate) if expires_at else candidate
+                )
+        except (OverflowError, ValueError):
+            log.warning(
+                f"Ignoring unreadable cache_control.ttl: {control.get('ttl')}"
+            )
+    return CacheBreakpoint(expire_at=expires_at)
+
+
+def _ttl(control: CacheControl) -> timedelta | None:
+    raw = control.get("ttl")
+    if raw is None:
+        return None
+    if isinstance(raw, str) and (match := _TTL.match(raw)):
+        return int(match.group(1)) * _UNITS[match.group(2)]
+    raise ValueError("Invalid ttl")
+
+
+def _expire_at(ttl: timedelta) -> str:
+    expires_at = datetime.now(UTC) + ttl
+    return expires_at.replace(microsecond=0).isoformat().replace("+00:00", "Z")

aidial_sdk-0.47.0.dev6/aidial_sdk/translators/messages_to_chat_completions/chat_completions/dial_extensions.py

@@ -0,0 +1,66 @@
+import re
+from collections.abc import Mapping
+
+from openai.types.chat.chat_completion_message import Annotation
+from pydantic import ValidationError
+
+from aidial_sdk.utils.logging import logger as log
+from aidial_sdk.utils.pydantic import ExtraAllowModel
+
+_REASONING_STAGE = re.compile(r"think|thought|reason", re.IGNORECASE)
+
+
+class ClaudeMessageBlock(ExtraAllowModel):
+    type: str | None = None
+    thinking: str | None = None
+    signature: str | None = None
+
+
+class MessageState(ExtraAllowModel):
+    claude_message_content: list[ClaudeMessageBlock] | None = None
+
+
+class Stage(ExtraAllowModel):
+    index: int | None = None
+    name: str | None = None
+    content: str | None = None
+
+
+class CustomContent(ExtraAllowModel):
+    stages: list[Stage] | None = None
+    state: MessageState | None = None
+
+
+class DialExtras(ExtraAllowModel):
+    custom_content: CustomContent | None = None
+    annotations: list[Annotation] | None = None
+
+
+def parse_extras(model_extra: Mapping[str, object] | None) -> DialExtras:
+    try:
+        return DialExtras.model_validate(model_extra or {})
+    except ValidationError as e:
+        log.error(f"Failed to parse DIAL extension fields: {e}")
+        return DialExtras()
+
+
+def is_reasoning_stage(name: str | None) -> bool:
+    return name is not None and bool(_REASONING_STAGE.search(name))
+
+
+def get_claude_thinking_block(
+    custom_content: CustomContent | None,
+) -> ClaudeMessageBlock | None:
+    state = custom_content.state if custom_content else None
+    for block in (state.claude_message_content if state else None) or []:
+        if block.type == "thinking" and block.thinking:
+            return block
+    return None
+
+
+def get_dial_reasoning_content(custom_content: CustomContent | None) -> str:
+    return "".join(
+        stage.content or ""
+        for stage in (custom_content.stages if custom_content else None) or []
+        if is_reasoning_stage(stage.name)
+    )

aidial_sdk-0.47.0.dev6/aidial_sdk/translators/messages_to_chat_completions/chat_completions/from_chat_completions.py

@@ -0,0 +1,237 @@
+import json
+import uuid
+from typing import assert_never
+
+from anthropic.types import (
+    Message,
+    ServerToolUseBlock,
+    StopReason,
+    TextBlock,
+    ThinkingBlock,
+    ToolUseBlock,
+    Usage,
+    WebSearchToolResultBlock,
+)
+from anthropic.types.content_block import ContentBlock as AnthropicContentBlock
+from anthropic.types.output_tokens_details import OutputTokensDetails
+from anthropic.types.web_search_result_block import WebSearchResultBlock
+from openai.types.chat import ChatCompletion
+from openai.types.chat.chat_completion_message import (
+    Annotation,
+    ChatCompletionMessage,
+)
+from openai.types.chat.chat_completion_message_tool_call import (
+    ChatCompletionMessageToolCallUnion,
+)
+from openai.types.completion_usage import CompletionUsage, PromptTokensDetails
+
+from aidial_sdk.translators.messages_to_chat_completions.chat_completions.dial_extensions import (
+    CustomContent,
+    get_claude_thinking_block,
+    get_dial_reasoning_content,
+    parse_extras,
+)
+from aidial_sdk.translators.messages_to_chat_completions.translation_state import (
+    TranslationState,
+)
+from aidial_sdk.utils.logging import logger as log
+
+
+def to_anthropic_response(
+    response: ChatCompletion,
+    requested_model: str,
+    translation_state: TranslationState,
+) -> Message:
+    if len(response.choices) > 1:
+        log.warning(
+            f"Ignoring {len(response.choices) - 1} choices after the first one"
+        )
+    choice = response.choices[0] if response.choices else None
+    message = choice.message if choice else None
+    tool_blocks = _to_anthropic_tool_use_blocks(
+        message.tool_calls if message else None, translation_state
+    )
+
+    return Message(
+        id=response.id,
+        type="message",
+        role="assistant",
+        model=requested_model,
+        content=_to_anthropic_content_blocks(message, tool_blocks),
+        stop_reason=to_anthropic_stop_reason(
+            choice.finish_reason if choice else None,
+            bool(tool_blocks),
+            bool(message and message.refusal),
+        ),
+        usage=to_anthropic_usage(response.usage),
+    )
+
+
+def _to_anthropic_content_blocks(
+    message: ChatCompletionMessage | None,
+    tool_blocks: list[ToolUseBlock],
+) -> list[AnthropicContentBlock]:
+    if message is None:
+        return [_text_block("")]
+
+    content: list[AnthropicContentBlock] = []
+
+    extras = parse_extras(message.model_extra)
+    if thinking := to_anthropic_thinking_block(extras.custom_content):
+        content.append(thinking)
+
+    content.extend(
+        to_anthropic_web_search_blocks(_citations(message.annotations))
+    )
+
+    if message.content:
+        content.append(_text_block(message.content))
+
+    if message.refusal:
+        content.append(_text_block(message.refusal))
+
+    content.extend(tool_blocks)
+
+    return content or [_text_block("")]
+
+
+def _text_block(text: str) -> TextBlock:
+    return TextBlock(type="text", text=text)
+
+
+def to_anthropic_thinking_block(
+    custom_content: CustomContent | None,
+) -> ThinkingBlock | None:
+    if block := get_claude_thinking_block(custom_content):
+        return ThinkingBlock(
+            type="thinking",
+            thinking=block.thinking or "",
+            signature=block.signature or "",
+        )
+    if text := get_dial_reasoning_content(custom_content):
+        return ThinkingBlock(type="thinking", thinking=text, signature="")
+    return None
+
+
+def _citations(annotations: list[Annotation] | None) -> list[tuple[str, str]]:
+    return [
+        (annotation.url_citation.url, annotation.url_citation.title)
+        for annotation in annotations or []
+    ]
+
+
+def to_anthropic_web_search_blocks(
+    citations: list[tuple[str, str]],
+) -> list[AnthropicContentBlock]:
+    if not citations:
+        return []
+    tool_use_id = f"srvtoolu_{uuid.uuid4().hex}"
+    return [
+        ServerToolUseBlock(
+            type="server_tool_use",
+            id=tool_use_id,
+            name="web_search",
+            input={"query": ""},
+        ),
+        WebSearchToolResultBlock(
+            type="web_search_tool_result",
+            tool_use_id=tool_use_id,
+            content=[
+                WebSearchResultBlock(
+                    type="web_search_result",
+                    url=url,
+                    title=title,
+                    encrypted_content="",
+                )
+                for url, title in citations
+            ],
+        ),
+    ]
+
+
+def _to_anthropic_tool_use_blocks(
+    tool_calls: list[ChatCompletionMessageToolCallUnion] | None,
+    translation_state: TranslationState,
+) -> list[ToolUseBlock]:
+    blocks: list[ToolUseBlock] = []
+    for call in tool_calls or []:
+        match call.type:
+            case "custom":
+                log.warning(f"Skipping unsupported tool call type: {call.type}")
+                continue
+            case "function":
+                pass
+            case unexpected:
+                assert_never(unexpected)
+        blocks.append(
+            ToolUseBlock(
+                type="tool_use",
+                id=call.id,
+                name=translation_state.to_client_tool_name(call.function.name),
+                input=_parse_arguments(call.function.arguments),
+            )
+        )
+    return blocks
+
+
+def _parse_arguments(arguments: str) -> dict[str, object]:
+    try:
+        parsed = json.loads(arguments) if arguments else {}
+    except json.JSONDecodeError:
+        parsed = None
+    if not isinstance(parsed, dict):
+        log.warning("Failed to parse function_call arguments as a JSON object")
+        return {}
+    return parsed
+
+
+def to_anthropic_stop_reason(
+    finish_reason: str | None,
+    has_tool_call: bool,
+    is_refusal: bool,
+) -> StopReason:
+    if finish_reason == "length":
+        return "max_tokens"
+    if finish_reason == "content_filter":
+        return "refusal"
+    if has_tool_call:
+        return "tool_use"
+    if is_refusal:
+        return "refusal"
+    return "end_turn"
+
+
+def to_anthropic_usage(usage: CompletionUsage | None) -> Usage:
+    if usage is None:
+        return Usage(input_tokens=0, output_tokens=0)
+
+    prompt_tokens = usage.prompt_tokens or 0
+    prompt_details = usage.prompt_tokens_details
+    # Anthropic's input_tokens excludes cache reads and writes, which
+    # prompt_tokens includes: cap the breakdown so the counters add up.
+    cache_read = min(
+        (prompt_details.cached_tokens if prompt_details else 0) or 0,
+        prompt_tokens,
+    )
+    cache_write = min(
+        _cache_write_tokens(prompt_details), prompt_tokens - cache_read
+    )
+
+    completion_details = usage.completion_tokens_details
+    thinking = (
+        completion_details.reasoning_tokens if completion_details else 0
+    ) or 0
+
+    return Usage(
+        input_tokens=prompt_tokens - cache_read - cache_write,
+        output_tokens=usage.completion_tokens or 0,
+        cache_read_input_tokens=cache_read,
+        cache_creation_input_tokens=cache_write,
+        output_tokens_details=OutputTokensDetails(thinking_tokens=thinking),
+    )
+
+
+def _cache_write_tokens(details: PromptTokensDetails | None) -> int:
+    if details is None:
+        return 0
+    return details.cache_write_tokens or 0