llm-actor 0.1.0__tar.gz

llm_actor-0.1.0/.gitignore

@@ -0,0 +1,14 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv
+
+# Локальные секреты для смоук-скрипта
+scripts/
+.env

llm_actor-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

llm_actor-0.1.0/PKG-INFO

@@ -0,0 +1,179 @@
+Metadata-Version: 2.4
+Name: llm-actor
+Version: 0.1.0
+Summary: LLM actor: pool, circuit breaker, retries, metrics
+Project-URL: Homepage, https://github.com/SergeyArc/llm-actor
+Project-URL: Repository, https://github.com/SergeyArc/llm-actor
+Project-URL: Issues, https://github.com/SergeyArc/llm-actor/issues
+Author-email: Sergey Arс <sergey-vd@outlook.com>
+License: MIT
+License-File: LICENSE
+Keywords: actor,anthropic,circuit-breaker,concurrent,gigachat,llm,openai,parallel,resilience
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: <3.14,>=3.13
+Requires-Dist: loguru>=0.7.0
+Requires-Dist: opentelemetry-api>=1.25.0
+Requires-Dist: pydantic-settings~=2.5.2
+Requires-Dist: pydantic~=2.9.2
+Provides-Extra: gigachat
+Requires-Dist: gigachat; extra == 'gigachat'
+Provides-Extra: metrics
+Requires-Dist: prometheus-client>=0.19.0; extra == 'metrics'
+Provides-Extra: openai
+Requires-Dist: openai; extra == 'openai'
+Description-Content-Type: text/markdown
+
+# LLM Actor: Resilient Throughput, Structured Output & Tool Calling
+
+<p align="center">
+  <a href="https://pypi.org/project/llm-actor/"><img src="https://img.shields.io/pypi/v/llm-actor.svg" alt="PyPI version"></a>
+  <a href="https://github.com/SergeyArc/llm-actor/actions/workflows/test.yml"><img src="https://github.com/SergeyArc/llm-actor/actions/workflows/test.yml/badge.svg" alt="Tests status"></a>
+  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
+  <a href="https://www.python.org/downloads/release/python-3130/"><img src="https://img.shields.io/badge/python-3.13-blue.svg" alt="Python 3.13"></a>
+</p>
+
+<p align="center">
+  <i>Documentation: <b>English</b> | <a href="docs/README.ru.md">Russian</a></i>
+</p>
+
+**LLM Actor** is a high-performance orchestration layer designed to handle Large Language Model (LLM) requests at scale. Inspired by the **Actor Model**, it solves the "last mile" of production LLM integration: handling concurrency, ensuring resilience, and providing **guaranteed structured output** and **agentic tool calling** without the boilerplate.
+
+---
+
+## Why LLM Actor?
+
+Most developers start with simple API calls. But when you move to production, you quickly hit:
+- **Rate Limit Exhaustion**: No global coordination for token usage.
+- **Provider Outages**: One slow response can hang your entire app.
+- **Unreliable Parsing**: Hard to get **guaranteed Structured Output** from raw strings.
+- **Complex Agentic Flows**: Orchestrating **Tool Calling** (especially in parallel) is error-prone.
+- **Lack of Priority**: Background tasks block high-priority user UI requests.
+
+**LLM Actor** fixes this. It’s not just a wrapper; it’s a **resilient worker pool** built to sit between your application logic and your LLM providers.
+
+---
+
+## Key Features
+
+- **High Throughput Actor Pool**: Efficiently manage hundreds of concurrent requests using a dedicated worker pool.
+- **Intelligent Resilience**: 
+    - **Circuit Breaker**: Detects provider failures and "fails fast" to protect your infrastructure.
+    - **Exponential Backoff**: Automatic retries for transient HTTP errors (429, 502, 503).
+    - **Semantic Validation**: Typed response validation with Pydantic; auto-retry on schema mismatch.
+- **Built-in Tool Calling Loop**: Native support for complex agentic flows. Run multiple tools **in parallel** to slash latency.
+- **Global Priority Queue**: Assign priorities to tasks. Ensure user-facing interactions always jump to the front of the line.
+- **Multi-Provider & Self-Hosted**: 
+    - Native support: OpenAI, Anthropic, Sber GigaChat.
+    - Proxy support: **vLLM**, **Ollama**, and any OpenAI-compatible endpoint.
+- **Deep Observability**: Full **OpenTelemetry** integration. Trace every request from the queue through the actor to the final provider response.
+
+---
+
+## Installation
+
+```bash
+# Install core package
+pip install llm-actor
+
+# Install with your preferred providers
+pip install "llm-actor[openai,anthropic]"
+
+# Full installation (all providers + metrics)
+pip install "llm-actor[all]"
+```
+
+---
+
+## Quick Start: 60 Seconds to Scale
+
+Create a service and start processing tasks with priority and auto-recovery:
+
+```python
+from llm_actor import LLMActorService, LLMActorSettings, Priority
+from pydantic import BaseModel
+
+# 1. Setup Service
+service = LLMActorService.from_openai(
+    api_key="sk-...", 
+    model="gpt-4o",
+    settings=LLMActorSettings(LLM_NUM_ACTORS=10) # 10 concurrent workers
+)
+
+# 2. Define Output Schema
+class UserProfile(BaseModel):
+    name: str
+    skills: list[str]
+
+# 3. Use via Context Manager (handles Start/Stop automatically)
+async with service:
+    # 4. Queue a High-Priority Task
+    request = service.request(
+        "Extract profile from: Alex is a Senior Python Dev with LLM expertise.",
+        response_model=UserProfile,
+        priority=Priority.HIGH
+    )
+
+    # 5. Get Your Results (Blocking or Async)
+    result = request.get()
+    print(f"Found: {result.name} with skills: {result.skills}")
+```
+
+---
+
+## Provider Support Matrix
+
+| Provider | Generations | Parallel Tools | Tested |
+|---|---|---|---|
+| **OpenAI / compatible** | Yes | Yes | Yes Full |
+| **Anthropic** | Yes | Yes | Yes Full |
+| **vLLM / Ollama** | Yes | Yes* | Yes Full |
+| **Sber GigaChat** | Yes | Warning | Experimental |
+
+*\*Tool calling in vLLM requires specific server-side flags.*
+
+---
+
+## Built for Reliability
+
+| Mechanism | Description |
+|---|---|
+| **Actor Supervision** | If a worker process crashes, it's automatically restarted by the supervisor. |
+| **Backpressure** | Prevents system overload by limiting the number of active tasks. |
+| **Otel Tracing** | Visualize latency including "Queue Wait Time" vs "In-LLM Time". |
+
+---
+
+## Contributing
+
+We love contributions! Whether it's adding a new provider adapter, fixing a bug, or improving documentation.
+
+1. Fork the repo.
+2. Install dev dependencies: `uv sync --all-extras --group dev`
+3. Run tests: `pytest tests/unit`
+4. Submit your PR!
+
+---
+
+## License
+
+Distributed under the **MIT License**. See `LICENSE` for more information.
+
+---
+
+## Examples & Advanced Usage
+
+Check out the [examples/](examples/) directory for complete, runnable scripts:
+
+1.  **[Basic Generation](examples/01_basic_generation.py)**: Quick start with any provider.
+2.  **[Structured Output](examples/02_structured_output.py)**: Extract data into Pydantic models.
+3.  **[Tool Calling](examples/03_tool_calling.py)**: Orchestrate complex agentic loops with parallel tool execution.
+
+---
+
+<p align="center">Built for the AI Developer Community.</p>

llm_actor-0.1.0/README.md

@@ -0,0 +1,148 @@
+# LLM Actor: Resilient Throughput, Structured Output & Tool Calling
+
+<p align="center">
+  <a href="https://pypi.org/project/llm-actor/"><img src="https://img.shields.io/pypi/v/llm-actor.svg" alt="PyPI version"></a>
+  <a href="https://github.com/SergeyArc/llm-actor/actions/workflows/test.yml"><img src="https://github.com/SergeyArc/llm-actor/actions/workflows/test.yml/badge.svg" alt="Tests status"></a>
+  <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
+  <a href="https://www.python.org/downloads/release/python-3130/"><img src="https://img.shields.io/badge/python-3.13-blue.svg" alt="Python 3.13"></a>
+</p>
+
+<p align="center">
+  <i>Documentation: <b>English</b> | <a href="docs/README.ru.md">Russian</a></i>
+</p>
+
+**LLM Actor** is a high-performance orchestration layer designed to handle Large Language Model (LLM) requests at scale. Inspired by the **Actor Model**, it solves the "last mile" of production LLM integration: handling concurrency, ensuring resilience, and providing **guaranteed structured output** and **agentic tool calling** without the boilerplate.
+
+---
+
+## Why LLM Actor?
+
+Most developers start with simple API calls. But when you move to production, you quickly hit:
+- **Rate Limit Exhaustion**: No global coordination for token usage.
+- **Provider Outages**: One slow response can hang your entire app.
+- **Unreliable Parsing**: Hard to get **guaranteed Structured Output** from raw strings.
+- **Complex Agentic Flows**: Orchestrating **Tool Calling** (especially in parallel) is error-prone.
+- **Lack of Priority**: Background tasks block high-priority user UI requests.
+
+**LLM Actor** fixes this. It’s not just a wrapper; it’s a **resilient worker pool** built to sit between your application logic and your LLM providers.
+
+---
+
+## Key Features
+
+- **High Throughput Actor Pool**: Efficiently manage hundreds of concurrent requests using a dedicated worker pool.
+- **Intelligent Resilience**: 
+    - **Circuit Breaker**: Detects provider failures and "fails fast" to protect your infrastructure.
+    - **Exponential Backoff**: Automatic retries for transient HTTP errors (429, 502, 503).
+    - **Semantic Validation**: Typed response validation with Pydantic; auto-retry on schema mismatch.
+- **Built-in Tool Calling Loop**: Native support for complex agentic flows. Run multiple tools **in parallel** to slash latency.
+- **Global Priority Queue**: Assign priorities to tasks. Ensure user-facing interactions always jump to the front of the line.
+- **Multi-Provider & Self-Hosted**: 
+    - Native support: OpenAI, Anthropic, Sber GigaChat.
+    - Proxy support: **vLLM**, **Ollama**, and any OpenAI-compatible endpoint.
+- **Deep Observability**: Full **OpenTelemetry** integration. Trace every request from the queue through the actor to the final provider response.
+
+---
+
+## Installation
+
+```bash
+# Install core package
+pip install llm-actor
+
+# Install with your preferred providers
+pip install "llm-actor[openai,anthropic]"
+
+# Full installation (all providers + metrics)
+pip install "llm-actor[all]"
+```
+
+---
+
+## Quick Start: 60 Seconds to Scale
+
+Create a service and start processing tasks with priority and auto-recovery:
+
+```python
+from llm_actor import LLMActorService, LLMActorSettings, Priority
+from pydantic import BaseModel
+
+# 1. Setup Service
+service = LLMActorService.from_openai(
+    api_key="sk-...", 
+    model="gpt-4o",
+    settings=LLMActorSettings(LLM_NUM_ACTORS=10) # 10 concurrent workers
+)
+
+# 2. Define Output Schema
+class UserProfile(BaseModel):
+    name: str
+    skills: list[str]
+
+# 3. Use via Context Manager (handles Start/Stop automatically)
+async with service:
+    # 4. Queue a High-Priority Task
+    request = service.request(
+        "Extract profile from: Alex is a Senior Python Dev with LLM expertise.",
+        response_model=UserProfile,
+        priority=Priority.HIGH
+    )
+
+    # 5. Get Your Results (Blocking or Async)
+    result = request.get()
+    print(f"Found: {result.name} with skills: {result.skills}")
+```
+
+---
+
+## Provider Support Matrix
+
+| Provider | Generations | Parallel Tools | Tested |
+|---|---|---|---|
+| **OpenAI / compatible** | Yes | Yes | Yes Full |
+| **Anthropic** | Yes | Yes | Yes Full |
+| **vLLM / Ollama** | Yes | Yes* | Yes Full |
+| **Sber GigaChat** | Yes | Warning | Experimental |
+
+*\*Tool calling in vLLM requires specific server-side flags.*
+
+---
+
+## Built for Reliability
+
+| Mechanism | Description |
+|---|---|
+| **Actor Supervision** | If a worker process crashes, it's automatically restarted by the supervisor. |
+| **Backpressure** | Prevents system overload by limiting the number of active tasks. |
+| **Otel Tracing** | Visualize latency including "Queue Wait Time" vs "In-LLM Time". |
+
+---
+
+## Contributing
+
+We love contributions! Whether it's adding a new provider adapter, fixing a bug, or improving documentation.
+
+1. Fork the repo.
+2. Install dev dependencies: `uv sync --all-extras --group dev`
+3. Run tests: `pytest tests/unit`
+4. Submit your PR!
+
+---
+
+## License
+
+Distributed under the **MIT License**. See `LICENSE` for more information.
+
+---
+
+## Examples & Advanced Usage
+
+Check out the [examples/](examples/) directory for complete, runnable scripts:
+
+1.  **[Basic Generation](examples/01_basic_generation.py)**: Quick start with any provider.
+2.  **[Structured Output](examples/02_structured_output.py)**: Extract data into Pydantic models.
+3.  **[Tool Calling](examples/03_tool_calling.py)**: Orchestrate complex agentic loops with parallel tool execution.
+
+---
+
+<p align="center">Built for the AI Developer Community.</p>

llm_actor-0.1.0/pyproject.toml

@@ -0,0 +1,117 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "llm-actor"
+version = "0.1.0"
+description = "LLM actor: pool, circuit breaker, retries, metrics"
+authors = [
+    { name = "Sergey Arс", email = "sergey-vd@outlook.com" }
+]
+license = { text = "MIT" }
+readme = "README.md"
+requires-python = ">=3.13,<3.14"
+dependencies = [
+    "pydantic~=2.9.2",
+    "pydantic-settings~=2.5.2",
+    "loguru>=0.7.0",
+    "opentelemetry-api>=1.25.0",
+]
+keywords = ["llm", "actor", "concurrent", "parallel", "resilience", "circuit-breaker", "openai", "anthropic", "gigachat"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+
+[project.urls]
+Homepage = "https://github.com/SergeyArc/llm-actor"
+Repository = "https://github.com/SergeyArc/llm-actor"
+Issues = "https://github.com/SergeyArc/llm-actor/issues"
+
+[project.optional-dependencies]
+gigachat = ["gigachat"]
+metrics = ["prometheus-client>=0.19.0"]
+openai = ["openai"]
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0.0",
+    "pytest-asyncio>=0.23.0",
+    "mypy>=1.11.0",
+    "prometheus-client>=0.19.0",
+    "ruff>=0.6.0",
+    "opentelemetry-sdk>=1.25.0",
+]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/llm_actor"]
+
+[tool.hatch.build.targets.sdist]
+include = ["src/llm_actor"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py313"
+
+[tool.ruff.lint]
+select = [
+    "E",
+    "F",
+    "I",
+    "B",
+    "C4",
+    "UP",
+    "C90",
+    "N",
+]
+ignore = ["E501"]
+
+[tool.ruff.format]
+quote-style = "double"
+indent-style = "space"
+line-ending = "lf"
+
+[tool.mypy]
+python_version = "3.13"
+mypy_path = "src"
+files = ["src"]
+check_untyped_defs = true
+disallow_any_generics = true
+disallow_incomplete_defs = true
+disallow_untyped_defs = true
+no_implicit_optional = true
+warn_redundant_casts = true
+warn_unused_ignores = true
+warn_return_any = true
+warn_unused_configs = true
+strict_equality = true
+show_error_codes = true
+disallow_any_unimported = true
+plugins = ["pydantic.mypy"]
+
+[[tool.mypy.overrides]]
+module = "anthropic"
+ignore_missing_imports = true
+
+[[tool.mypy.overrides]]
+module = "openai"
+ignore_missing_imports = true
+
+[[tool.mypy.overrides]]
+module = "gigachat"
+ignore_missing_imports = true
+
+[[tool.mypy.overrides]]
+module = "gigachat.models"
+ignore_missing_imports = true
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]
+asyncio_mode = "auto"

llm_actor-0.1.0/src/llm_actor/__init__.py

@@ -0,0 +1,76 @@
+from llm_actor.actors.pool import HealthStatus, SupervisedActorPool
+from llm_actor.actors.worker import ModelActor
+from llm_actor.client.interface import (
+    LLMClientInterface,
+    LLMClientWithCircuitBreakerInterface,
+    ToolCapableClientInterface,
+)
+from llm_actor.client.llm import LLMClientWithCircuitBreaker, build_json_prompt
+from llm_actor.client.retry import LLMClientWithRetry
+from llm_actor.core.messages import ActorMessage
+from llm_actor.core.request import LLMRequest
+from llm_actor.core.tools import LLMResponse, Tool, ToolCall, ToolResult
+from llm_actor.exceptions import (
+    ActorFailedError,
+    CircuitBreakerOpenError,
+    LLMActorError,
+    LLMServiceError,
+    LLMServiceGeneralError,
+    LLMServiceHTTPError,
+    LLMServiceOverloadedError,
+    LLMServiceTimeoutError,
+    LLMServiceUnavailableError,
+    OverloadError,
+    PoolShuttingDownError,
+    ToolExecutionError,
+    ToolExecutionTimeoutError,
+    ToolLoopMaxIterationsError,
+)
+from llm_actor.logger import ActorLogger
+from llm_actor.metrics import (
+    MetricsCollector,
+    default_metrics_collector,
+    is_prometheus_metrics_available,
+)
+from llm_actor.resilience.circuit_breaker import CircuitBreaker
+from llm_actor.service import LLMActorService
+from llm_actor.settings import LLMActorSettings
+
+__all__ = [
+    "ActorMessage",
+    "LLMRequest",
+    "LLMResponse",
+    "ActorFailedError",
+    "ActorLogger",
+    "CircuitBreaker",
+    "CircuitBreakerOpenError",
+    "HealthStatus",
+    "LLMActorError",
+    "LLMActorService",
+    "LLMActorSettings",
+    "LLMClientInterface",
+    "LLMClientWithCircuitBreaker",
+    "build_json_prompt",
+    "LLMClientWithCircuitBreakerInterface",
+    "ToolCapableClientInterface",
+    "LLMClientWithRetry",
+    "LLMServiceError",
+    "LLMServiceGeneralError",
+    "LLMServiceHTTPError",
+    "LLMServiceOverloadedError",
+    "LLMServiceTimeoutError",
+    "LLMServiceUnavailableError",
+    "MetricsCollector",
+    "default_metrics_collector",
+    "is_prometheus_metrics_available",
+    "ModelActor",
+    "OverloadError",
+    "PoolShuttingDownError",
+    "SupervisedActorPool",
+    "Tool",
+    "ToolCall",
+    "ToolExecutionError",
+    "ToolExecutionTimeoutError",
+    "ToolLoopMaxIterationsError",
+    "ToolResult",
+]