adaptive-serve 0.1.0__tar.gz

adaptive_serve-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mandar Wagh
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

adaptive_serve-0.1.0/PKG-INFO

@@ -0,0 +1,176 @@
+Metadata-Version: 2.4
+Name: adaptive-serve
+Version: 0.1.0
+Summary: Adaptive cloud-to-robot serving layer that scales robotics model quality to fit device, power, latency, and bandwidth constraints.
+Author: Mandar Wagh
+License: MIT
+Project-URL: Homepage, https://github.com/mandarwagh9/adaptive-serve
+Project-URL: Repository, https://github.com/mandarwagh9/adaptive-serve
+Keywords: robotics,serving,inference,edge,vla,adaptive,cloud
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.11; extra == "dev"
+Provides-Extra: torch
+Requires-Dist: torch>=2.0; extra == "torch"
+Dynamic: license-file
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/mandarwagh9/adaptive-serve/main/assets/banner.png" alt="adaptive-serve" width="880">
+</p>
+
+<p align="center">
+  <a href="https://github.com/mandarwagh9/adaptive-serve/actions/workflows/ci.yml"><img src="https://github.com/mandarwagh9/adaptive-serve/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <img src="https://img.shields.io/badge/python-3.10+-EDE8DD?style=flat-square&labelColor=0B0E14" alt="python 3.10+">
+  <img src="https://img.shields.io/badge/license-MIT-EDE8DD?style=flat-square&labelColor=0B0E14" alt="MIT">
+  <img src="https://img.shields.io/badge/lint-ruff-FFB000?style=flat-square&labelColor=0B0E14" alt="ruff">
+  <img src="https://img.shields.io/badge/tests-152_passing-FFB000?style=flat-square&labelColor=0B0E14" alt="152 tests passing">
+</p>
+
+# adaptive-serve
+
+Adaptive cloud-to-robot serving layer for robotics models of all types.
+
+It takes any trained robotics model and serves it to a robot, scaling model
+quality up or down to fit the binding constraint: on-robot compute (TOPS/VRAM),
+power budget, latency tolerance, and live network bandwidth. Think adaptive
+bitrate streaming, but the renditions are model variants and the network is the
+robot's compute and link.
+
+The whole core runs with no GPU, no network, and no external dependencies. An
+optional torch backend and a stdlib HTTP cloud adapter add real execution when
+you want it.
+
+## Why
+
+A robot cannot round-trip to the cloud for every control step, and it must keep
+acting if the link drops. adaptive-serve picks the right variant and serving mode
+for the device in front of it, runs a slow cloud planner and a fast local reflex
+loop at the same time, and degrades safely when the cloud goes away.
+
+## Install
+
+```bash
+pip install -e ".[dev]"        # core plus test tooling
+pip install -e ".[dev,torch]"  # also pull torch for the real backend
+```
+
+## Try it
+
+```bash
+adaptive-serve demo                                  # full end to end demo
+adaptive-serve serve --model planner --vram 2 --bandwidth 200 --difficulty hard
+adaptive-serve serve --model planner --vram 80 --bandwidth 0  --difficulty hard
+```
+
+The same model adapts to the device, exactly like adaptive bitrate:
+
+```
+device               mode    variant                 ~latency ms  meets
+tiny-edge-offline    local   planner-vlm:2B:int8            36.0  False
+edge-weak-link       split   planner-vlm:7B:fp16           761.8   True
+edge-fast-link       cloud   planner-vlm:7B:fp16            21.0   True
+workstation          local   planner-vlm:7B:fp16             4.2   True
+```
+
+And it keeps acting through a cloud outage, degrading from NOMINAL to a latched
+emergency stop only when the last plan is long gone:
+
+```
+Multi-rate session with a cloud drop at 1000 ms:
+  reflex steps: 1200
+  plans published: 2
+  safety mode timeline: {'nominal': 200, 'degraded': 531, 'hold': 400, 'kill': 69}
+  final safety mode: kill
+```
+
+## Calibration: decide on real numbers, not guesses
+
+By default the router uses portable heuristics for footprint and latency. To
+ground its decisions in real hardware, measure actual models and route on those
+numbers instead:
+
+```bash
+adaptive-serve calibrate            # measure real models -> calibration.json
+adaptive-serve pipeline             # calibrate -> route on measured -> run in torch
+```
+
+`calibrate` records the exact parameter count, the weight memory at each
+precision, and the measured forward latency per variant. A `ModelRegistry`
+built with that table feeds the router measured footprints (heuristics remain the
+fallback for any variant you have not measured), so a model that is actually small
+gets served where the heuristic would have offloaded it.
+
+Honest scope: weight bytes and params are device-independent and drive the VRAM
+fit directly; latency is device-specific, so it stays informational. On a CPU box
+the harness measures real but small reference models and INT8 via torch dynamic
+quantization; the 7B/70B rungs and true INT4 need target hardware, which the same
+harness supports. Requires the `torch` extra: `pip install -e ".[dev,torch]"`.
+
+## Supported model classes
+
+The registry and router are generic across these, never hardcoded to one:
+
+- VLA / action policies (diffusion policies, ACT, flow-matching)
+- world / dynamics models (Dreamer-style, latent predictors, video predictors)
+- perception (detection, segmentation, depth, optical flow, 6DoF pose, keypoints)
+- VLMs / high-level reasoners (planning, language grounding)
+- state estimation / SLAM / odometry
+- classical or learned planners and controllers
+
+Each declares its role in the stack, native rate (Hz), latency budget, and which
+scaling techniques it supports. Adding a new class touches no core.
+
+## Architecture
+
+```
+DeviceProfile  ---+
+                  |
+ModelSpec --> Registry: variant ladder (size x quant) per model, any class
+                  |
+                  v
+             Router: pick variant + quant + serving mode (local / cloud / split)
+                  |
+   +--------------+--------------+
+   |                             |
+Scaling strategies          Orchestrator: cloud System-2 planner ~1 Hz
+size/quant/input/exit/      + local System-1 reflex 50-1000 Hz
+cascade/split                    |
+   |                             v
+   +--------------->  Safety governor: hold / kill on loss
+                             |
+                             v
+                      Runtime + backends: run local (stub or torch) or
+                      offload over HTTP, fall back to local on failure
+                             |
+                             v
+                      Telemetry: record every step
+```
+
+Two invariants hold throughout, both covered by tests:
+
+- **Generic across model classes.** The router and scaling cores dispatch on a
+  spec's declared role, tier, latency budget, and supported techniques, never on
+  class identity.
+- **The robot never blocks on the cloud.** The reflex tier runs every control
+  tick. On cloud loss the planner stops publishing, the reflex keeps acting on
+  its last plan, the safety governor degrades to HOLD and latches KILL when the
+  plan is too stale, and a late or failed cloud call is dropped for a safe local
+  action rather than awaited.
+
+## Develop
+
+```bash
+ruff check .
+pytest
+```
+
+See `STATUS.md` for the full component list and `PLAN.md` / `ADAPTERS_PLAN.md`
+for the build history. MIT licensed.

adaptive_serve-0.1.0/README.md

@@ -0,0 +1,152 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/mandarwagh9/adaptive-serve/main/assets/banner.png" alt="adaptive-serve" width="880">
+</p>
+
+<p align="center">
+  <a href="https://github.com/mandarwagh9/adaptive-serve/actions/workflows/ci.yml"><img src="https://github.com/mandarwagh9/adaptive-serve/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <img src="https://img.shields.io/badge/python-3.10+-EDE8DD?style=flat-square&labelColor=0B0E14" alt="python 3.10+">
+  <img src="https://img.shields.io/badge/license-MIT-EDE8DD?style=flat-square&labelColor=0B0E14" alt="MIT">
+  <img src="https://img.shields.io/badge/lint-ruff-FFB000?style=flat-square&labelColor=0B0E14" alt="ruff">
+  <img src="https://img.shields.io/badge/tests-152_passing-FFB000?style=flat-square&labelColor=0B0E14" alt="152 tests passing">
+</p>
+
+# adaptive-serve
+
+Adaptive cloud-to-robot serving layer for robotics models of all types.
+
+It takes any trained robotics model and serves it to a robot, scaling model
+quality up or down to fit the binding constraint: on-robot compute (TOPS/VRAM),
+power budget, latency tolerance, and live network bandwidth. Think adaptive
+bitrate streaming, but the renditions are model variants and the network is the
+robot's compute and link.
+
+The whole core runs with no GPU, no network, and no external dependencies. An
+optional torch backend and a stdlib HTTP cloud adapter add real execution when
+you want it.
+
+## Why
+
+A robot cannot round-trip to the cloud for every control step, and it must keep
+acting if the link drops. adaptive-serve picks the right variant and serving mode
+for the device in front of it, runs a slow cloud planner and a fast local reflex
+loop at the same time, and degrades safely when the cloud goes away.
+
+## Install
+
+```bash
+pip install -e ".[dev]"        # core plus test tooling
+pip install -e ".[dev,torch]"  # also pull torch for the real backend
+```
+
+## Try it
+
+```bash
+adaptive-serve demo                                  # full end to end demo
+adaptive-serve serve --model planner --vram 2 --bandwidth 200 --difficulty hard
+adaptive-serve serve --model planner --vram 80 --bandwidth 0  --difficulty hard
+```
+
+The same model adapts to the device, exactly like adaptive bitrate:
+
+```
+device               mode    variant                 ~latency ms  meets
+tiny-edge-offline    local   planner-vlm:2B:int8            36.0  False
+edge-weak-link       split   planner-vlm:7B:fp16           761.8   True
+edge-fast-link       cloud   planner-vlm:7B:fp16            21.0   True
+workstation          local   planner-vlm:7B:fp16             4.2   True
+```
+
+And it keeps acting through a cloud outage, degrading from NOMINAL to a latched
+emergency stop only when the last plan is long gone:
+
+```
+Multi-rate session with a cloud drop at 1000 ms:
+  reflex steps: 1200
+  plans published: 2
+  safety mode timeline: {'nominal': 200, 'degraded': 531, 'hold': 400, 'kill': 69}
+  final safety mode: kill
+```
+
+## Calibration: decide on real numbers, not guesses
+
+By default the router uses portable heuristics for footprint and latency. To
+ground its decisions in real hardware, measure actual models and route on those
+numbers instead:
+
+```bash
+adaptive-serve calibrate            # measure real models -> calibration.json
+adaptive-serve pipeline             # calibrate -> route on measured -> run in torch
+```
+
+`calibrate` records the exact parameter count, the weight memory at each
+precision, and the measured forward latency per variant. A `ModelRegistry`
+built with that table feeds the router measured footprints (heuristics remain the
+fallback for any variant you have not measured), so a model that is actually small
+gets served where the heuristic would have offloaded it.
+
+Honest scope: weight bytes and params are device-independent and drive the VRAM
+fit directly; latency is device-specific, so it stays informational. On a CPU box
+the harness measures real but small reference models and INT8 via torch dynamic
+quantization; the 7B/70B rungs and true INT4 need target hardware, which the same
+harness supports. Requires the `torch` extra: `pip install -e ".[dev,torch]"`.
+
+## Supported model classes
+
+The registry and router are generic across these, never hardcoded to one:
+
+- VLA / action policies (diffusion policies, ACT, flow-matching)
+- world / dynamics models (Dreamer-style, latent predictors, video predictors)
+- perception (detection, segmentation, depth, optical flow, 6DoF pose, keypoints)
+- VLMs / high-level reasoners (planning, language grounding)
+- state estimation / SLAM / odometry
+- classical or learned planners and controllers
+
+Each declares its role in the stack, native rate (Hz), latency budget, and which
+scaling techniques it supports. Adding a new class touches no core.
+
+## Architecture
+
+```
+DeviceProfile  ---+
+                  |
+ModelSpec --> Registry: variant ladder (size x quant) per model, any class
+                  |
+                  v
+             Router: pick variant + quant + serving mode (local / cloud / split)
+                  |
+   +--------------+--------------+
+   |                             |
+Scaling strategies          Orchestrator: cloud System-2 planner ~1 Hz
+size/quant/input/exit/      + local System-1 reflex 50-1000 Hz
+cascade/split                    |
+   |                             v
+   +--------------->  Safety governor: hold / kill on loss
+                             |
+                             v
+                      Runtime + backends: run local (stub or torch) or
+                      offload over HTTP, fall back to local on failure
+                             |
+                             v
+                      Telemetry: record every step
+```
+
+Two invariants hold throughout, both covered by tests:
+
+- **Generic across model classes.** The router and scaling cores dispatch on a
+  spec's declared role, tier, latency budget, and supported techniques, never on
+  class identity.
+- **The robot never blocks on the cloud.** The reflex tier runs every control
+  tick. On cloud loss the planner stops publishing, the reflex keeps acting on
+  its last plan, the safety governor degrades to HOLD and latches KILL when the
+  plan is too stale, and a late or failed cloud call is dropped for a safe local
+  action rather than awaited.
+
+## Develop
+
+```bash
+ruff check .
+pytest
+```
+
+See `STATUS.md` for the full component list and `PLAN.md` / `ADAPTERS_PLAN.md`
+for the build history. MIT licensed.

adaptive_serve-0.1.0/pyproject.toml

@@ -0,0 +1,48 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "adaptive-serve"
+version = "0.1.0"
+description = "Adaptive cloud-to-robot serving layer that scales robotics model quality to fit device, power, latency, and bandwidth constraints."
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "Mandar Wagh" }]
+keywords = ["robotics", "serving", "inference", "edge", "vla", "adaptive", "cloud"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = []
+
+[project.optional-dependencies]
+dev = ["pytest>=8", "ruff>=0.11"]
+torch = ["torch>=2.0"]
+
+[project.urls]
+Homepage = "https://github.com/mandarwagh9/adaptive-serve"
+Repository = "https://github.com/mandarwagh9/adaptive-serve"
+
+[project.scripts]
+adaptive-serve = "adaptive_serve.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.ruff]
+line-length = 100
+src = ["src", "tests"]
+extend-exclude = ["assets"]
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"
+pythonpath = ["src"]

adaptive_serve-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

adaptive_serve-0.1.0/src/adaptive_serve/__init__.py

@@ -0,0 +1,194 @@
+"""Adaptive cloud-to-robot serving layer.
+
+Serves any trained robotics model to a robot, scaling model quality up or down
+to fit the binding constraint: on-robot compute (TOPS/VRAM), power budget,
+latency tolerance, and live network bandwidth.
+
+The package is generic across model classes (action policies, world models,
+perception, VLMs, state estimation, planners). The registry and router never
+hardcode a single class; adding a new class must not touch the router core.
+"""
+
+from adaptive_serve.backends import Backend, InferenceResult, LocalStubBackend
+from adaptive_serve.calibration import CalibrationTable, Measurement, measure, run_calibration
+from adaptive_serve.cloud_http import HttpCloudError, HttpCloudLink, HttpCloudPlanner
+from adaptive_serve.orchestrator import (
+    AlwaysOnline,
+    CallableLink,
+    CloudLink,
+    Orchestrator,
+    OrchestratorTrace,
+    Plan,
+    PlannerRunner,
+    PlanSlot,
+    ReflexRunner,
+    ReflexStep,
+    ScheduledOutage,
+)
+from adaptive_serve.pipeline import PipelineResult, run_calibrated_pipeline
+from adaptive_serve.profiler import (
+    AutoProbe,
+    BandwidthMonitor,
+    DeviceProbe,
+    DeviceProfile,
+    DeviceProfiler,
+    ReplayBandwidthMonitor,
+    StaticBandwidthMonitor,
+    StaticProbe,
+    default_profiler,
+)
+from adaptive_serve.registry import (
+    DEFAULT_QUANTS,
+    DEFAULT_SIZES,
+    ModelRegistry,
+    ModelVariant,
+    Quant,
+    RegisteredModel,
+    SizeTier,
+)
+from adaptive_serve.router import (
+    RoutingDecision,
+    ServingMode,
+    TaskDifficulty,
+    TierRouter,
+)
+from adaptive_serve.runtime import CloudExecutor, ExecutionResult, ServingRuntime
+from adaptive_serve.safety import (
+    SafetyDecision,
+    SafetyGovernor,
+    SafetyMode,
+    SafetyObservation,
+)
+from adaptive_serve.scaling import (
+    CascadeStrategy,
+    EarlyExitStrategy,
+    InputScalingStrategy,
+    QuantLadderStrategy,
+    ScalingAction,
+    ScalingContext,
+    ScalingStrategy,
+    SizeLadderStrategy,
+    SplitInferenceStrategy,
+    StrategyRegistry,
+    default_strategy_registry,
+    estimate_latency_ms,
+)
+from adaptive_serve.serving import ServingLayer, ServingPlan
+from adaptive_serve.taxonomy import (
+    ModelClass,
+    ModelSpec,
+    ScalingTechnique,
+    StackRole,
+    Tier,
+    get_model_class,
+    register_model_class,
+    registered_model_classes,
+)
+from adaptive_serve.telemetry import (
+    CallbackSink,
+    InMemorySink,
+    NdjsonSink,
+    NullSink,
+    TelemetryRecord,
+    TelemetryRecorder,
+    TelemetrySummary,
+    percentile,
+)
+from adaptive_serve.torch_backend import TorchBackend
+
+__version__ = "0.1.0"
+
+__all__ = [
+    # taxonomy
+    "ModelClass",
+    "ModelSpec",
+    "ScalingTechnique",
+    "StackRole",
+    "Tier",
+    "get_model_class",
+    "register_model_class",
+    "registered_model_classes",
+    # profiler
+    "AutoProbe",
+    "BandwidthMonitor",
+    "DeviceProbe",
+    "DeviceProfile",
+    "DeviceProfiler",
+    "ReplayBandwidthMonitor",
+    "StaticBandwidthMonitor",
+    "StaticProbe",
+    "default_profiler",
+    # registry
+    "DEFAULT_QUANTS",
+    "DEFAULT_SIZES",
+    "ModelRegistry",
+    "ModelVariant",
+    "Quant",
+    "RegisteredModel",
+    "SizeTier",
+    # router
+    "RoutingDecision",
+    "ServingMode",
+    "TaskDifficulty",
+    "TierRouter",
+    # scaling
+    "CascadeStrategy",
+    "EarlyExitStrategy",
+    "InputScalingStrategy",
+    "QuantLadderStrategy",
+    "ScalingAction",
+    "ScalingContext",
+    "ScalingStrategy",
+    "SizeLadderStrategy",
+    "SplitInferenceStrategy",
+    "StrategyRegistry",
+    "default_strategy_registry",
+    "estimate_latency_ms",
+    # orchestrator
+    "AlwaysOnline",
+    "CallableLink",
+    "CloudLink",
+    "Orchestrator",
+    "OrchestratorTrace",
+    "Plan",
+    "PlanSlot",
+    "PlannerRunner",
+    "ReflexRunner",
+    "ReflexStep",
+    "ScheduledOutage",
+    # safety
+    "SafetyDecision",
+    "SafetyGovernor",
+    "SafetyMode",
+    "SafetyObservation",
+    # serving facade
+    "ServingLayer",
+    "ServingPlan",
+    # telemetry
+    "CallbackSink",
+    "InMemorySink",
+    "NdjsonSink",
+    "NullSink",
+    "TelemetryRecord",
+    "TelemetryRecorder",
+    "TelemetrySummary",
+    "percentile",
+    # adapters
+    "Backend",
+    "PipelineResult",
+    "run_calibrated_pipeline",
+    "CalibrationTable",
+    "Measurement",
+    "measure",
+    "run_calibration",
+    "CloudExecutor",
+    "ExecutionResult",
+    "HttpCloudError",
+    "HttpCloudLink",
+    "HttpCloudPlanner",
+    "InferenceResult",
+    "LocalStubBackend",
+    "ServingRuntime",
+    "TorchBackend",
+    "__version__",
+]

adaptive_serve-0.1.0/src/adaptive_serve/backends.py

@@ -0,0 +1,84 @@
+"""Inference backends.
+
+A Backend turns a chosen ModelVariant into a runnable handle and executes it on
+inputs, returning an InferenceResult with the output and the real measured (or
+estimated) latency. The serving layer dispatches a ServingPlan to a backend to
+actually run a model rather than only decide how to run it.
+
+LocalStubBackend is a deterministic CPU backend with no ML framework dependency,
+so the whole stack can run end to end on any machine. The torch-based backend
+lives in its own module behind a lazy import so this one never pulls heavy deps.
+"""
+
+from __future__ import annotations
+
+import zlib
+from dataclasses import dataclass
+from typing import Protocol
+
+from adaptive_serve.profiler import DeviceProfile
+from adaptive_serve.registry import ModelVariant
+from adaptive_serve.scaling import estimate_latency_ms
+
+# Latency used when no device profile is available to estimate against.
+_DEFAULT_LATENCY_MS = 1.0
+
+
+@dataclass(frozen=True)
+class InferenceResult:
+    """The output of running one variant on one set of inputs."""
+
+    output: object
+    latency_ms: float
+    variant_id: str
+    backend: str
+
+
+class Backend(Protocol):
+    """Loads and runs a model variant. The seam real engines implement."""
+
+    name: str
+
+    def load(self, variant: ModelVariant) -> object: ...
+
+    def infer(self, handle: object, inputs: object) -> InferenceResult: ...
+
+
+class LocalStubBackend:
+    """A deterministic CPU backend that needs no ML framework.
+
+    It produces a reproducible digest of the inputs and a latency derived from the
+    variant footprint and the device profile, so the serving layer can be driven
+    end to end with no GPU. Same inputs and variant always give the same result.
+    """
+
+    name = "local-stub"
+
+    def __init__(self, profile: DeviceProfile | None = None) -> None:
+        self._profile = profile
+
+    def load(self, variant: ModelVariant) -> ModelVariant:
+        # Nothing heavy to load; the variant itself is the handle.
+        return variant
+
+    def infer(self, handle: object, inputs: object) -> InferenceResult:
+        if not isinstance(handle, ModelVariant):
+            raise TypeError("handle must be a ModelVariant from load()")
+        variant = handle
+        digest = zlib.crc32(repr(inputs).encode("utf-8"))
+        latency = (
+            estimate_latency_ms(variant, self._profile)
+            if self._profile is not None
+            else _DEFAULT_LATENCY_MS
+        )
+        output = {
+            "variant_id": variant.variant_id,
+            "digest": digest,
+            "quality": variant.quality,
+        }
+        return InferenceResult(
+            output=output,
+            latency_ms=latency,
+            variant_id=variant.variant_id,
+            backend=self.name,
+        )