anydeploy 0.2.2__tar.gz → 0.2.4__tar.gz

{anydeploy-0.2.2 → anydeploy-0.2.4}/.github/workflows/ci.yml

@@ -12,7 +12,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
 
     steps:
       - uses: actions/checkout@v4

anydeploy-0.2.4/PKG-INFO

@@ -0,0 +1,241 @@
+Metadata-Version: 2.4
+Name: anydeploy
+Version: 0.2.4
+Summary: CLI tool and library to export ML models to production formats and containerize them with Docker
+Project-URL: Homepage, https://www.nrl.ai
+Project-URL: Repository, https://github.com/vietanhdev/anydeploy
+Project-URL: Documentation, https://github.com/vietanhdev/anydeploy#readme
+Project-URL: Issues, https://github.com/vietanhdev/anydeploy/issues
+Author-email: Viet-Anh Nguyen <vietanh.dev@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: deployment,docker,machine-learning,model-serving,onnx,tflite,torchscript
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Requires-Dist: click>=8.0
+Requires-Dist: numpy>=1.20
+Requires-Dist: pyyaml>=6.0
+Provides-Extra: all
+Requires-Dist: fastapi>=0.68; extra == 'all'
+Requires-Dist: onnx>=1.10; extra == 'all'
+Requires-Dist: onnxruntime>=1.10; extra == 'all'
+Requires-Dist: tensorflow>=2.5; extra == 'all'
+Requires-Dist: torch>=1.9; extra == 'all'
+Requires-Dist: uvicorn>=0.15; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff>=0.1.0; extra == 'dev'
+Provides-Extra: onnx
+Requires-Dist: onnx>=1.10; extra == 'onnx'
+Requires-Dist: onnxruntime>=1.10; extra == 'onnx'
+Provides-Extra: serve
+Requires-Dist: fastapi>=0.68; extra == 'serve'
+Requires-Dist: uvicorn>=0.15; extra == 'serve'
+Provides-Extra: tflite
+Requires-Dist: tensorflow>=2.5; extra == 'tflite'
+Provides-Extra: torch
+Requires-Dist: torch>=1.9; extra == 'torch'
+Description-Content-Type: text/markdown
+
+<h1 align="center">anydeploy</h1>
+<p align="center"><em>Export, serve, and containerize any ML model — plus auto-generate MCP servers for AI agents.</em></p>
+
+<p align="center">
+<img src="https://img.shields.io/pypi/v/anydeploy.svg" alt="PyPI">
+<img src="https://img.shields.io/pypi/pyversions/anydeploy.svg" alt="Python">
+<img src="https://img.shields.io/pypi/l/anydeploy.svg" alt="License">
+</p>
+
+**anydeploy** is the last-mile deployment toolkit for ML models. It exports PyTorch or sklearn models to ONNX, TorchScript, or TFLite with smart defaults; generates a FastAPI server with health checks and OpenAPI docs; auto-creates a Model Context Protocol (MCP) server so any AI agent (Claude Desktop, Continue, Cursor) can call your model as a tool; and produces Dockerfiles + requirements files for reproducible deployment. Three deployment profiles (`edge`, `balanced`, `quality`) pick quantization and precision for you.
+
+Built by [Viet-Anh Nguyen](https://github.com/vietanhdev) at [NRL.ai](https://www.nrl.ai).
+
+## Why anydeploy?
+
+- **One-liner API** — `anydeploy.export(model, "onnx")` handles shape inference, opset, and validation
+- **Plugin architecture** — Register custom exporters, servers, or container targets
+- **Local-first** — Everything runs on your machine; no cloud account needed
+- **Minimal core deps** — Base install has zero heavy deps; torch/tf are optional
+- **Production-ready** — MCP integration, FastAPI generation, Dockerfile scaffolding
+
+## Installation
+
+```bash
+pip install anydeploy
+```
+
+For optional features:
+
+```bash
+pip install anydeploy[onnx]      # ONNX export + onnxruntime verification
+pip install anydeploy[torch]     # TorchScript export
+pip install anydeploy[tflite]    # TFLite conversion
+pip install anydeploy[serve]     # FastAPI + uvicorn server
+pip install anydeploy[mcp]       # Model Context Protocol server generation
+pip install anydeploy[all]       # everything
+```
+
+**Python 3.8+ supported** (tested on 3.8, 3.9, 3.10, 3.11, 3.12, 3.13)
+
+## Quick Start
+
+```python
+import anydeploy
+import torch
+
+model = torch.load("resnet50.pt").eval()
+
+# 1. Export to ONNX with smart defaults (opset, dynamic axes, validation)
+anydeploy.export(
+    model,
+    format="onnx",
+    out="resnet50.onnx",
+    example_input=torch.randn(1, 3, 224, 224),
+    profile="balanced",          # edge | balanced | quality
+)
+
+# 2. Generate a FastAPI server with health check + OpenAPI docs
+anydeploy.serve("resnet50.onnx", host="0.0.0.0", port=8000)
+
+# 3. Generate an MCP server so Claude Desktop / Cursor can call the model
+anydeploy.mcp("resnet50.onnx", out="my_mcp_server/", name="image-classifier")
+
+# 4. Generate a Dockerfile + requirements.txt for reproducible deployment
+anydeploy.containerize("resnet50.onnx", out="docker/", base="python:3.11-slim")
+```
+
+## Models & Methods
+
+### Export formats
+
+| Format | How it works | Notes |
+|---|---|---|
+| **ONNX** | `torch.onnx.export` with auto-derived dynamic axes + opset 17 defaults | Validates via onnxruntime after export |
+| **TorchScript** | `torch.jit.trace` (default) or `torch.jit.script` | Python-free runtime |
+| **TFLite** | `torch -> onnx -> tf -> tflite` via onnx-tf + TensorFlow converter | Mobile / embedded |
+
+All exports include automatic **shape inference**, **input/output naming**, and a **round-trip validation step** that runs a dummy input through both the original and the exported model and compares outputs.
+
+### Deployment profiles
+
+| Profile | Precision | Quantization | Intended target |
+|---|---|---|---|
+| `edge` | int8 | Post-training static quantization | Raspberry Pi, phones, MCUs |
+| `balanced` (default) | fp16 | Optional fp16 conversion | Laptop / workstation CPU |
+| `quality` | fp32 | None | Server / GPU inference |
+
+### FastAPI server generation
+
+`anydeploy.serve(model_path)` generates and launches a FastAPI app with:
+
+- `POST /predict` — accepts JSON or multipart image upload
+- `GET /health` — liveness check
+- `GET /docs` — interactive OpenAPI UI (Swagger)
+- Automatic request/response Pydantic schemas inferred from the model's input/output shapes
+- Optional batching, CORS, and API-key authentication
+
+### MCP (Model Context Protocol) server generation
+
+`anydeploy.mcp(model_path, name=...)` generates a complete MCP server implementation that exposes your model as an AI-callable tool. Any MCP-compatible client — **Claude Desktop**, **Cursor**, **Continue**, **Zed** — can then invoke your model via natural language.
+
+The generated server:
+
+- Exposes a `run_model` tool with a JSON schema derived from model inputs
+- Handles image decoding, tensor conversion, and postprocessing
+- Ships with a `claude_desktop_config.json` snippet ready to copy
+
+### Containerization
+
+`anydeploy.containerize(model_path)` generates:
+
+- `Dockerfile` — minimal base image (python-slim by default) with only the runtime dependencies your model needs
+- `requirements.txt` — pinned versions discovered from the export step
+- `.dockerignore` — sensible defaults
+- `docker-compose.yml` (optional) — for multi-container setups
+
+## API Reference
+
+| Function | Purpose |
+|---|---|
+| `anydeploy.export(model, format, out, **opts)` | Export to ONNX/TorchScript/TFLite |
+| `anydeploy.serve(model_path, host, port)` | Launch a FastAPI server |
+| `anydeploy.generate_server(model_path, out)` | Generate FastAPI code to disk |
+| `anydeploy.mcp(model_path, out, name)` | Generate an MCP tool server |
+| `anydeploy.containerize(model_path, out)` | Generate Dockerfile + requirements |
+| `anydeploy.quantize(model_path, mode="int8")` | Post-training quantization |
+| `anydeploy.benchmark(model_path)` | Measure latency + throughput |
+
+## CLI Usage
+
+```bash
+# Export
+anydeploy export model.pt --format onnx --out model.onnx --profile edge
+
+# Serve
+anydeploy serve model.onnx --port 8000
+
+# Generate MCP server
+anydeploy mcp model.onnx --out mcp_server/ --name my-model
+
+# Containerize
+anydeploy containerize model.onnx --out docker/
+
+# Benchmark
+anydeploy benchmark model.onnx --runs 100
+```
+
+## Examples
+
+### Train with traincv, deploy with anydeploy
+
+```python
+import traincv, anydeploy
+
+# Train a YOLOv8 detector
+run = traincv.train("datasets/pets/", task="detect", model="yolov8n", epochs=50)
+
+# Export to ONNX, edge-quantized
+anydeploy.export(run.weights_path, format="onnx",
+                 out="pets.onnx", profile="edge")
+
+# Expose as an MCP tool for Claude Desktop
+anydeploy.mcp("pets.onnx", out="pets_mcp/", name="pet-detector")
+```
+
+### Auto-generate a Docker image and run it
+
+```python
+import anydeploy
+
+anydeploy.containerize("model.onnx", out="deploy/")
+
+# Then:
+#   cd deploy && docker build -t my-model .
+#   docker run -p 8000:8000 my-model
+```
+
+### Benchmark before and after quantization
+
+```python
+import anydeploy
+
+print(anydeploy.benchmark("model.onnx"))              # fp32 baseline
+anydeploy.quantize("model.onnx", mode="int8", out="model_int8.onnx")
+print(anydeploy.benchmark("model_int8.onnx"))         # int8 quantized
+```
+
+## License
+
+MIT (c) Viet-Anh Nguyen

anydeploy-0.2.4/README.md

@@ -0,0 +1,190 @@
+<h1 align="center">anydeploy</h1>
+<p align="center"><em>Export, serve, and containerize any ML model — plus auto-generate MCP servers for AI agents.</em></p>
+
+<p align="center">
+<img src="https://img.shields.io/pypi/v/anydeploy.svg" alt="PyPI">
+<img src="https://img.shields.io/pypi/pyversions/anydeploy.svg" alt="Python">
+<img src="https://img.shields.io/pypi/l/anydeploy.svg" alt="License">
+</p>
+
+**anydeploy** is the last-mile deployment toolkit for ML models. It exports PyTorch or sklearn models to ONNX, TorchScript, or TFLite with smart defaults; generates a FastAPI server with health checks and OpenAPI docs; auto-creates a Model Context Protocol (MCP) server so any AI agent (Claude Desktop, Continue, Cursor) can call your model as a tool; and produces Dockerfiles + requirements files for reproducible deployment. Three deployment profiles (`edge`, `balanced`, `quality`) pick quantization and precision for you.
+
+Built by [Viet-Anh Nguyen](https://github.com/vietanhdev) at [NRL.ai](https://www.nrl.ai).
+
+## Why anydeploy?
+
+- **One-liner API** — `anydeploy.export(model, "onnx")` handles shape inference, opset, and validation
+- **Plugin architecture** — Register custom exporters, servers, or container targets
+- **Local-first** — Everything runs on your machine; no cloud account needed
+- **Minimal core deps** — Base install has zero heavy deps; torch/tf are optional
+- **Production-ready** — MCP integration, FastAPI generation, Dockerfile scaffolding
+
+## Installation
+
+```bash
+pip install anydeploy
+```
+
+For optional features:
+
+```bash
+pip install anydeploy[onnx]      # ONNX export + onnxruntime verification
+pip install anydeploy[torch]     # TorchScript export
+pip install anydeploy[tflite]    # TFLite conversion
+pip install anydeploy[serve]     # FastAPI + uvicorn server
+pip install anydeploy[mcp]       # Model Context Protocol server generation
+pip install anydeploy[all]       # everything
+```
+
+**Python 3.8+ supported** (tested on 3.8, 3.9, 3.10, 3.11, 3.12, 3.13)
+
+## Quick Start
+
+```python
+import anydeploy
+import torch
+
+model = torch.load("resnet50.pt").eval()
+
+# 1. Export to ONNX with smart defaults (opset, dynamic axes, validation)
+anydeploy.export(
+    model,
+    format="onnx",
+    out="resnet50.onnx",
+    example_input=torch.randn(1, 3, 224, 224),
+    profile="balanced",          # edge | balanced | quality
+)
+
+# 2. Generate a FastAPI server with health check + OpenAPI docs
+anydeploy.serve("resnet50.onnx", host="0.0.0.0", port=8000)
+
+# 3. Generate an MCP server so Claude Desktop / Cursor can call the model
+anydeploy.mcp("resnet50.onnx", out="my_mcp_server/", name="image-classifier")
+
+# 4. Generate a Dockerfile + requirements.txt for reproducible deployment
+anydeploy.containerize("resnet50.onnx", out="docker/", base="python:3.11-slim")
+```
+
+## Models & Methods
+
+### Export formats
+
+| Format | How it works | Notes |
+|---|---|---|
+| **ONNX** | `torch.onnx.export` with auto-derived dynamic axes + opset 17 defaults | Validates via onnxruntime after export |
+| **TorchScript** | `torch.jit.trace` (default) or `torch.jit.script` | Python-free runtime |
+| **TFLite** | `torch -> onnx -> tf -> tflite` via onnx-tf + TensorFlow converter | Mobile / embedded |
+
+All exports include automatic **shape inference**, **input/output naming**, and a **round-trip validation step** that runs a dummy input through both the original and the exported model and compares outputs.
+
+### Deployment profiles
+
+| Profile | Precision | Quantization | Intended target |
+|---|---|---|---|
+| `edge` | int8 | Post-training static quantization | Raspberry Pi, phones, MCUs |
+| `balanced` (default) | fp16 | Optional fp16 conversion | Laptop / workstation CPU |
+| `quality` | fp32 | None | Server / GPU inference |
+
+### FastAPI server generation
+
+`anydeploy.serve(model_path)` generates and launches a FastAPI app with:
+
+- `POST /predict` — accepts JSON or multipart image upload
+- `GET /health` — liveness check
+- `GET /docs` — interactive OpenAPI UI (Swagger)
+- Automatic request/response Pydantic schemas inferred from the model's input/output shapes
+- Optional batching, CORS, and API-key authentication
+
+### MCP (Model Context Protocol) server generation
+
+`anydeploy.mcp(model_path, name=...)` generates a complete MCP server implementation that exposes your model as an AI-callable tool. Any MCP-compatible client — **Claude Desktop**, **Cursor**, **Continue**, **Zed** — can then invoke your model via natural language.
+
+The generated server:
+
+- Exposes a `run_model` tool with a JSON schema derived from model inputs
+- Handles image decoding, tensor conversion, and postprocessing
+- Ships with a `claude_desktop_config.json` snippet ready to copy
+
+### Containerization
+
+`anydeploy.containerize(model_path)` generates:
+
+- `Dockerfile` — minimal base image (python-slim by default) with only the runtime dependencies your model needs
+- `requirements.txt` — pinned versions discovered from the export step
+- `.dockerignore` — sensible defaults
+- `docker-compose.yml` (optional) — for multi-container setups
+
+## API Reference
+
+| Function | Purpose |
+|---|---|
+| `anydeploy.export(model, format, out, **opts)` | Export to ONNX/TorchScript/TFLite |
+| `anydeploy.serve(model_path, host, port)` | Launch a FastAPI server |
+| `anydeploy.generate_server(model_path, out)` | Generate FastAPI code to disk |
+| `anydeploy.mcp(model_path, out, name)` | Generate an MCP tool server |
+| `anydeploy.containerize(model_path, out)` | Generate Dockerfile + requirements |
+| `anydeploy.quantize(model_path, mode="int8")` | Post-training quantization |
+| `anydeploy.benchmark(model_path)` | Measure latency + throughput |
+
+## CLI Usage
+
+```bash
+# Export
+anydeploy export model.pt --format onnx --out model.onnx --profile edge
+
+# Serve
+anydeploy serve model.onnx --port 8000
+
+# Generate MCP server
+anydeploy mcp model.onnx --out mcp_server/ --name my-model
+
+# Containerize
+anydeploy containerize model.onnx --out docker/
+
+# Benchmark
+anydeploy benchmark model.onnx --runs 100
+```
+
+## Examples
+
+### Train with traincv, deploy with anydeploy
+
+```python
+import traincv, anydeploy
+
+# Train a YOLOv8 detector
+run = traincv.train("datasets/pets/", task="detect", model="yolov8n", epochs=50)
+
+# Export to ONNX, edge-quantized
+anydeploy.export(run.weights_path, format="onnx",
+                 out="pets.onnx", profile="edge")
+
+# Expose as an MCP tool for Claude Desktop
+anydeploy.mcp("pets.onnx", out="pets_mcp/", name="pet-detector")
+```
+
+### Auto-generate a Docker image and run it
+
+```python
+import anydeploy
+
+anydeploy.containerize("model.onnx", out="deploy/")
+
+# Then:
+#   cd deploy && docker build -t my-model .
+#   docker run -p 8000:8000 my-model
+```
+
+### Benchmark before and after quantization
+
+```python
+import anydeploy
+
+print(anydeploy.benchmark("model.onnx"))              # fp32 baseline
+anydeploy.quantize("model.onnx", mode="int8", out="model_int8.onnx")
+print(anydeploy.benchmark("model_int8.onnx"))         # int8 quantized
+```
+
+## License
+
+MIT (c) Viet-Anh Nguyen

{anydeploy-0.2.2 → anydeploy-0.2.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "anydeploy"
-version = "0.2.2"
+version = "0.2.4"
 description = "CLI tool and library to export ML models to production formats and containerize them with Docker"
 readme = "README.md"
 license = "MIT"
@@ -24,6 +24,7 @@ classifiers = [
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
     "Topic :: Scientific/Engineering :: Artificial Intelligence",
 ]
 dependencies = [

anydeploy-0.2.2/PKG-INFO

@@ -1,234 +0,0 @@
-Metadata-Version: 2.4
-Name: anydeploy
-Version: 0.2.2
-Summary: CLI tool and library to export ML models to production formats and containerize them with Docker
-Project-URL: Homepage, https://www.nrl.ai
-Project-URL: Repository, https://github.com/vietanhdev/anydeploy
-Project-URL: Documentation, https://github.com/vietanhdev/anydeploy#readme
-Project-URL: Issues, https://github.com/vietanhdev/anydeploy/issues
-Author-email: Viet-Anh Nguyen <vietanh.dev@gmail.com>
-License-Expression: MIT
-License-File: LICENSE
-Keywords: deployment,docker,machine-learning,model-serving,onnx,tflite,torchscript
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: Science/Research
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
-Requires-Python: >=3.8
-Requires-Dist: click>=8.0
-Requires-Dist: numpy>=1.20
-Requires-Dist: pyyaml>=6.0
-Provides-Extra: all
-Requires-Dist: fastapi>=0.68; extra == 'all'
-Requires-Dist: onnx>=1.10; extra == 'all'
-Requires-Dist: onnxruntime>=1.10; extra == 'all'
-Requires-Dist: tensorflow>=2.5; extra == 'all'
-Requires-Dist: torch>=1.9; extra == 'all'
-Requires-Dist: uvicorn>=0.15; extra == 'all'
-Provides-Extra: dev
-Requires-Dist: pytest-cov>=4.0; extra == 'dev'
-Requires-Dist: pytest>=7.0; extra == 'dev'
-Requires-Dist: ruff>=0.1.0; extra == 'dev'
-Provides-Extra: onnx
-Requires-Dist: onnx>=1.10; extra == 'onnx'
-Requires-Dist: onnxruntime>=1.10; extra == 'onnx'
-Provides-Extra: serve
-Requires-Dist: fastapi>=0.68; extra == 'serve'
-Requires-Dist: uvicorn>=0.15; extra == 'serve'
-Provides-Extra: tflite
-Requires-Dist: tensorflow>=2.5; extra == 'tflite'
-Provides-Extra: torch
-Requires-Dist: torch>=1.9; extra == 'torch'
-Description-Content-Type: text/markdown
-
-<h1 align="center">anydeploy</h1>
-<p align="center"><em>Deploy ML models anywhere</em></p>
-
-![PyPI](https://img.shields.io/pypi/v/anydeploy)
-![Python](https://img.shields.io/pypi/pyversions/anydeploy)
-![License](https://img.shields.io/pypi/l/anydeploy)
-
-**Export ML models to production formats (ONNX, TFLite, TorchScript) and deploy them locally or at the edge.**
-
-`anydeploy` makes model deployment easy. Convert your trained models to optimized inference formats, benchmark performance, validate correctness, generate serving code, and containerize everything -- all from a single CLI or Python API.
-
-**Edge-first deployment.** Supports ONNX Runtime (CPU/GPU/edge), TFLite (mobile/edge), and llama.cpp (local LLM serving). All deployment targets work completely offline.
-
-Built and maintained by [Viet-Anh Nguyen](https://github.com/vietanhdev) at [NRL.ai](https://www.nrl.ai).
-
-## Installation
-
-```bash
-# Core (CLI + config + benchmarking)
-pip install anydeploy
-
-# With specific framework support
-pip install anydeploy[torch]      # PyTorch + TorchScript
-pip install anydeploy[onnx]       # ONNX + ONNX Runtime
-pip install anydeploy[tflite]     # TensorFlow Lite
-pip install anydeploy[serve]      # FastAPI serving
-
-# Everything
-pip install anydeploy[all]
-```
-
-## Quick Start
-
-### CLI
-
-```bash
-# Export a PyTorch model to ONNX
-anydeploy export model.pt --format onnx --input-shape 1,3,224,224
-
-# Export to TFLite
-anydeploy export model.pt --format tflite --input-shape 1,3,224,224
-
-# Benchmark an exported model
-anydeploy benchmark model.onnx --runs 100
-
-# Serve a model with FastAPI
-anydeploy serve model.onnx --backend fastapi --port 8000
-
-# Generate a Docker container for deployment
-anydeploy dockerize model.onnx --base python:3.11-slim
-```
-
-### Python API
-
-```python
-import anydeploy
-
-# Export a model
-anydeploy.export(model, format="onnx", input_shape=(1, 3, 224, 224))
-
-# Benchmark performance
-result = anydeploy.benchmark("model.onnx", runs=100)
-print(f"Mean latency: {result.mean_latency_ms:.2f} ms")
-print(f"P95 latency:  {result.p95_latency_ms:.2f} ms")
-print(f"Throughput:   {result.throughput:.1f} inferences/sec")
-
-# Validate exported model against original
-report = anydeploy.validate(original_model, "model.onnx", test_input)
-print(f"Max difference: {report.max_diff}")
-print(f"Passed: {report.passed}")
-
-# Generate Dockerfile and serving code
-from anydeploy.config import DockerConfig
-docker_cfg = DockerConfig(base_image="python:3.11-slim")
-anydeploy.dockerize("model.onnx", docker_cfg)
-
-# Register a custom exporter
-from anydeploy.export.base import BaseExporter
-class MyExporter(BaseExporter):
-    def export(self, model, output_path, config=None):
-        ...
-anydeploy.register_exporter("myformat", MyExporter)
-```
-
-## Export Format Comparison
-
-| Format      | Framework    | Hardware        | Optimization     | File Size |
-|-------------|-------------|-----------------|------------------|-----------|
-| ONNX        | Any (via ONNX Runtime) | CPU, GPU, Edge  | Graph optimization | Medium    |
-| TFLite      | TensorFlow  | Mobile, Edge    | Quantization     | Small     |
-| TorchScript | PyTorch     | CPU, GPU        | JIT compilation  | Large     |
-
-## Serving
-
-`anydeploy` generates production-ready serving code for multiple backends:
-
-```bash
-# FastAPI server for ONNX/TFLite/TorchScript models
-anydeploy serve model.onnx --backend fastapi --port 8000
-
-# llama.cpp server for GGUF language models (edge LLM deployment)
-anydeploy serve model.gguf --backend llamacpp --port 8080
-```
-
-### FastAPI Backend
-
-Creates a FastAPI application with:
-- `/predict` endpoint accepting JSON or binary input
-- `/health` health check endpoint
-- Automatic input validation
-- Configurable batch size
-
-### llama.cpp Backend
-
-Creates deployment scripts for serving GGUF language models locally:
-- Shell script to launch llama.cpp server
-- Dockerfile for containerized LLM serving
-- OpenAI-compatible `/v1/chat/completions` endpoint
-- Works on CPU, GPU, and edge devices
-
-## Docker Deployment
-
-Generate a complete Docker setup for your model:
-
-```bash
-anydeploy dockerize model.onnx --base python:3.11-slim --port 8000
-```
-
-This creates:
-- `Dockerfile` with optimized layers
-- `serve.py` FastAPI application
-- `requirements.txt` with pinned dependencies
-
-## Extensibility
-
-`anydeploy` uses a plugin architecture. You can register custom exporters and serving backends:
-
-```python
-import anydeploy
-from anydeploy.export.base import BaseExporter
-
-class CoreMLExporter(BaseExporter):
-    format_name = "coreml"
-
-    def export(self, model, output_path, config=None):
-        # Your export logic
-        ...
-
-    def validate_model(self, model):
-        return True
-
-anydeploy.register_exporter("coreml", CoreMLExporter)
-```
-
-See [CONTRIBUTING.md](CONTRIBUTING.md) for details on adding new exporters and backends.
-
-## Local-First / Edge AI
-
-This package is designed for edge and local deployment. All export formats
-(ONNX, TFLite, TorchScript) produce models that run completely offline.
-The llama.cpp backend enables local LLM serving without any cloud dependencies.
-
-```bash
-# Export for edge deployment
-anydeploy export model.pt --format onnx       # ONNX Runtime (CPU/GPU/edge)
-anydeploy export model.pt --format tflite      # TFLite (mobile/edge)
-
-# Serve an LLM locally
-anydeploy serve model.gguf --backend llamacpp
-```
-
-## Contributing
-
-Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
-
-## License
-
-MIT License. See [LICENSE](LICENSE) for details.
-
-## Links
-
-- [NRL.ai](https://www.nrl.ai)
-- [GitHub](https://github.com/vietanhdev/anydeploy)
-- [PyPI](https://pypi.org/project/anydeploy/)

anydeploy-0.2.2/README.md

@@ -1,184 +0,0 @@
-<h1 align="center">anydeploy</h1>
-<p align="center"><em>Deploy ML models anywhere</em></p>
-
-![PyPI](https://img.shields.io/pypi/v/anydeploy)
-![Python](https://img.shields.io/pypi/pyversions/anydeploy)
-![License](https://img.shields.io/pypi/l/anydeploy)
-
-**Export ML models to production formats (ONNX, TFLite, TorchScript) and deploy them locally or at the edge.**
-
-`anydeploy` makes model deployment easy. Convert your trained models to optimized inference formats, benchmark performance, validate correctness, generate serving code, and containerize everything -- all from a single CLI or Python API.
-
-**Edge-first deployment.** Supports ONNX Runtime (CPU/GPU/edge), TFLite (mobile/edge), and llama.cpp (local LLM serving). All deployment targets work completely offline.
-
-Built and maintained by [Viet-Anh Nguyen](https://github.com/vietanhdev) at [NRL.ai](https://www.nrl.ai).
-
-## Installation
-
-```bash
-# Core (CLI + config + benchmarking)
-pip install anydeploy
-
-# With specific framework support
-pip install anydeploy[torch]      # PyTorch + TorchScript
-pip install anydeploy[onnx]       # ONNX + ONNX Runtime
-pip install anydeploy[tflite]     # TensorFlow Lite
-pip install anydeploy[serve]      # FastAPI serving
-
-# Everything
-pip install anydeploy[all]
-```
-
-## Quick Start
-
-### CLI
-
-```bash
-# Export a PyTorch model to ONNX
-anydeploy export model.pt --format onnx --input-shape 1,3,224,224
-
-# Export to TFLite
-anydeploy export model.pt --format tflite --input-shape 1,3,224,224
-
-# Benchmark an exported model
-anydeploy benchmark model.onnx --runs 100
-
-# Serve a model with FastAPI
-anydeploy serve model.onnx --backend fastapi --port 8000
-
-# Generate a Docker container for deployment
-anydeploy dockerize model.onnx --base python:3.11-slim
-```
-
-### Python API
-
-```python
-import anydeploy
-
-# Export a model
-anydeploy.export(model, format="onnx", input_shape=(1, 3, 224, 224))
-
-# Benchmark performance
-result = anydeploy.benchmark("model.onnx", runs=100)
-print(f"Mean latency: {result.mean_latency_ms:.2f} ms")
-print(f"P95 latency:  {result.p95_latency_ms:.2f} ms")
-print(f"Throughput:   {result.throughput:.1f} inferences/sec")
-
-# Validate exported model against original
-report = anydeploy.validate(original_model, "model.onnx", test_input)
-print(f"Max difference: {report.max_diff}")
-print(f"Passed: {report.passed}")
-
-# Generate Dockerfile and serving code
-from anydeploy.config import DockerConfig
-docker_cfg = DockerConfig(base_image="python:3.11-slim")
-anydeploy.dockerize("model.onnx", docker_cfg)
-
-# Register a custom exporter
-from anydeploy.export.base import BaseExporter
-class MyExporter(BaseExporter):
-    def export(self, model, output_path, config=None):
-        ...
-anydeploy.register_exporter("myformat", MyExporter)
-```
-
-## Export Format Comparison
-
-| Format      | Framework    | Hardware        | Optimization     | File Size |
-|-------------|-------------|-----------------|------------------|-----------|
-| ONNX        | Any (via ONNX Runtime) | CPU, GPU, Edge  | Graph optimization | Medium    |
-| TFLite      | TensorFlow  | Mobile, Edge    | Quantization     | Small     |
-| TorchScript | PyTorch     | CPU, GPU        | JIT compilation  | Large     |
-
-## Serving
-
-`anydeploy` generates production-ready serving code for multiple backends:
-
-```bash
-# FastAPI server for ONNX/TFLite/TorchScript models
-anydeploy serve model.onnx --backend fastapi --port 8000
-
-# llama.cpp server for GGUF language models (edge LLM deployment)
-anydeploy serve model.gguf --backend llamacpp --port 8080
-```
-
-### FastAPI Backend
-
-Creates a FastAPI application with:
-- `/predict` endpoint accepting JSON or binary input
-- `/health` health check endpoint
-- Automatic input validation
-- Configurable batch size
-
-### llama.cpp Backend
-
-Creates deployment scripts for serving GGUF language models locally:
-- Shell script to launch llama.cpp server
-- Dockerfile for containerized LLM serving
-- OpenAI-compatible `/v1/chat/completions` endpoint
-- Works on CPU, GPU, and edge devices
-
-## Docker Deployment
-
-Generate a complete Docker setup for your model:
-
-```bash
-anydeploy dockerize model.onnx --base python:3.11-slim --port 8000
-```
-
-This creates:
-- `Dockerfile` with optimized layers
-- `serve.py` FastAPI application
-- `requirements.txt` with pinned dependencies
-
-## Extensibility
-
-`anydeploy` uses a plugin architecture. You can register custom exporters and serving backends:
-
-```python
-import anydeploy
-from anydeploy.export.base import BaseExporter
-
-class CoreMLExporter(BaseExporter):
-    format_name = "coreml"
-
-    def export(self, model, output_path, config=None):
-        # Your export logic
-        ...
-
-    def validate_model(self, model):
-        return True
-
-anydeploy.register_exporter("coreml", CoreMLExporter)
-```
-
-See [CONTRIBUTING.md](CONTRIBUTING.md) for details on adding new exporters and backends.
-
-## Local-First / Edge AI
-
-This package is designed for edge and local deployment. All export formats
-(ONNX, TFLite, TorchScript) produce models that run completely offline.
-The llama.cpp backend enables local LLM serving without any cloud dependencies.
-
-```bash
-# Export for edge deployment
-anydeploy export model.pt --format onnx       # ONNX Runtime (CPU/GPU/edge)
-anydeploy export model.pt --format tflite      # TFLite (mobile/edge)
-
-# Serve an LLM locally
-anydeploy serve model.gguf --backend llamacpp
-```
-
-## Contributing
-
-Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
-
-## License
-
-MIT License. See [LICENSE](LICENSE) for details.
-
-## Links
-
-- [NRL.ai](https://www.nrl.ai)
-- [GitHub](https://github.com/vietanhdev/anydeploy)
-- [PyPI](https://pypi.org/project/anydeploy/)