cortexgrid 0.2.85__tar.gz → 0.2.87__tar.gz

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: cortexgrid
-Version: 0.2.85
+Version: 0.2.87
 Summary: Connect your ML code to the RoboLab compute cluster — Ray, MLflow, and S3
 Project-URL: Homepage, https://github.com/robodatalab/cortexgrid
 Project-URL: Repository, https://github.com/robodatalab/cortexgrid
@@ -13,6 +13,7 @@ Requires-Dist: cloudpickle>=3.0
 Requires-Dist: fabric>=3.2.3
 Requires-Dist: haikunator>=2.1.0
 Requires-Dist: mlflow<4,>=3.11
+Requires-Dist: packaging>=24
 Requires-Dist: pip>=23.0
 Requires-Dist: pydantic-settings>=2.13.1
 Requires-Dist: pydantic>=2.13.3
@@ -115,9 +116,10 @@ print(f"Submitted: {job_id}")
 A separate service — the **jobs control plane** — polls MLflow for pending job requests, matches them against the set of Ray submissions the cluster already has, and submits anything missing. It is also responsible for retrying failed jobs and honouring user-requested stops.
 
 Each submission captures the code and dependencies the entry function needs automatically ([_bundle.py](https://github.com/robodatalab/cortexgrid/blob/main/cortexgrid/_bundle.py)):
-- `bundle(entry)` traces the import graph from the function's source file, resolving each import the way the interpreter does (via `sys.path`), and returns every file needed to run it -- your own modules and third-party packages alike, wherever they live. The standard library is excluded (it ships with the interpreter)
-- Everything ships **as source**: the bundle is staged at each file's import path and tarred into the Ray `working_dir`. Nothing is `pip`-installed on the worker
-- Dependencies the worker image already has are subtracted rather than shipped: `bundle(entry) - worker_provides()`, where `worker_provides()` is the bundle of the packages baked into the ray image (torch and its CUDA stack, ray, mlflow, ...). See [k8s/docker/ray/Dockerfile](https://github.com/robodatalab/cortexgrid/blob/main/k8s/docker/ray/Dockerfile)
+- `bundle(entry)` traces the import graph from the function's source file, resolving each import the way the interpreter does (via `sys.path`). The standard library is excluded (it ships with the interpreter)
+- Your own modules -- anything outside site-packages / dist-packages -- ship **as source**: they are staged at their import paths and tarred into the Ray `working_dir`
+- Third-party packages are recorded as the installed distribution that owns the imported file, pinned to its installed version (`tqdm==4.67.3`), and Ray **pip-installs** them on the worker into a per-node cached virtualenv layered on the image (`runtime_env["pip"]`). An import into site-packages that no installed distribution owns fails `cortexgrid.remote` with `UnownedDependencyError`
+- Distributions the worker image already has are not installed again: `worker_provides()` is the dependency closure of the packages baked into the ray image (torch and its CUDA stack, ray, mlflow, ...), and is subtracted from the pip list. See [k8s/docker/ray/Dockerfile](https://github.com/robodatalab/cortexgrid/blob/main/k8s/docker/ray/Dockerfile) and `_WORKER_BAKED` in `_bundle.py`, which must list the same packages
 - Injects MLflow/S3 credentials so task code running on the DGX can reach all services
 
 ##### Retries
@@ -153,9 +155,24 @@ s3_client = cortexgrid.get_s3_client()           # boto3 S3 client
 
 #### Model registry and serving
 
-Save a trained model's weights together with the serve-app that fronts it, then deploy it as a Ray Serve application:
+Save a trained model's weights together with the serve-app that fronts it, then deploy it as a Ray Serve application. A serve-app is a class fronted by a FastAPI app, marked with cortexgrid's `serve.ingress` (not Ray's):
 
 ```python
+from cortexgrid import serve
+from fastapi import FastAPI
+
+app = FastAPI()
+
+@serve.ingress(app)
+class MyServeApp:
+    num_gpus = 1
+
+    def __init__(self, family: str, suffix: str, run_name: str) -> None:
+        self._weights_dir = cortexgrid.load_model(family, suffix, run_name)
+
+    @app.post("/complete")
+    async def complete(self, body: dict): ...
+
 saved = cortexgrid.save_model(weights_dir, MyServeApp, family="qwen", suffix="instruct")
 deployed = cortexgrid.deploy_model("qwen", "instruct", saved.run_name, wait=True)
 print(deployed.url)

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/cortexgrid/_bundle.py

@@ -3,27 +3,38 @@
 `bundle(seed)` describes what is needed to run the module at `seed`: the local
 files it reaches -- following its import graph and each package's __init__
 chain, resolving imports the way the interpreter does -- and the third-party
-dependencies those files import. A file is local unless it lives in an installed
-package location (site-packages / dist-packages); imports that resolve into one
-are not followed. The standard library is excluded (it ships with the
-interpreter). Bundles of several seeds combine with `BundleDesc.merge`.
+distributions those files import. A file is local unless it lives in an installed
+package location (site-packages / dist-packages). An import that resolves into
+one is recorded as the distribution that installed the file, at its installed
+version, and is not followed: installing that distribution brings its own
+dependencies. The standard library is excluded (it ships with the interpreter).
+Bundles of several seeds combine with `BundleDesc.merge`.
 
 `stage(files, dest)` lays a bundle out under `dest` at each file's import path,
 so `dest` on sys.path (e.g. a Ray working_dir) makes every module importable.
+
+`BundleDesc.pip_requirements(worker_provides())` pins the third-party
+distributions the Ray worker image does not already have, for a Ray `pip`
+runtime_env to install on the worker.
 """
 
 from __future__ import annotations
 
 import ast
-from collections.abc import Iterator
+from collections.abc import Iterable, Iterator
 from dataclasses import dataclass
 import functools
 import importlib.machinery
+import importlib.metadata
 import importlib.util
+import os
 from pathlib import Path
 import shutil
 import sys
 
+from packaging.requirements import Requirement
+from packaging.utils import canonicalize_name
+
 
 ThirdPartyDependencyName = str
 ThirdPartyDependencyVersion = str
@@ -39,17 +50,43 @@ class BundleDesc:
             tp_deps={**self.tp_deps, **other.tp_deps},
         )
 
+    def pip_requirements(
+        self, provided: frozenset[ThirdPartyDependencyName]
+    ) -> list[str]:
+        """The third-party distributions, pinned (`name==version`) and sorted,
+        minus the `provided` ones."""
+        return sorted(
+            f"{name}=={version}"
+            for name, version in self.tp_deps.items()
+            if name not in provided
+        )
+
+
+class UnownedDependencyError(LookupError):
+    """An import resolved into an installed package location, but no installed
+    distribution owns the file, so it can be neither shipped nor installed."""
+
 
 def bundle(seed: Path) -> BundleDesc:
     """What is needed to run the module at `seed`: its local files, each at its
-    real path. Third-party dependency detection is not implemented yet, so
-    `tp_deps` is always empty."""
+    real path, and the third-party distributions they import, keyed by
+    canonical name, each at its installed version.
+
+    Raises UnownedDependencyError for an import that resolves into an installed
+    package location no distribution owns."""
     seed = seed.resolve()
     files: set[Path] = set()
+    tp_deps: dict[ThirdPartyDependencyName, ThirdPartyDependencyVersion] = {}
+    visited: set[Path] = set()
     queue: list[Path] = [seed]
     while queue:
         file = queue.pop()
-        if file in files or not _is_local(file):
+        if file in visited:
+            continue
+        visited.add(file)
+        if not _is_local(file):
+            dist = _owning_distribution(file)
+            tp_deps[canonicalize_name(dist.metadata["Name"])] = dist.version
             continue
         files.add(file)
         queue.extend(_init_chain(file))  # importing a module runs its __init__ chain
@@ -58,35 +95,135 @@ def bundle(seed: Path) -> BundleDesc:
                 dep = _module_file(name)
                 if dep is not None:
                     queue.append(dep)
-    return BundleDesc(local_files=files, tp_deps={})
+    return BundleDesc(local_files=files, tp_deps=tp_deps)
 
 
 def stage(files: set[Path], dest: Path) -> None:
     """Copy `files` under `dest`, each at its import path (relative to the
-    sys.path entry it lives under), so `dest` on sys.path imports them all."""
+    sys.path entry it lives under), so `dest` on sys.path imports them all.
+    `dest` is created even when `files` is empty (e.g. code that lives entirely
+    in installed distributions)."""
+    dest.mkdir(parents=True, exist_ok=True)
     for file in files:
         target = dest / file.relative_to(_sys_path_root(file))
         target.parent.mkdir(parents=True, exist_ok=True)
         shutil.copy2(file, target)
 
 
-# Distributions already present in the Ray worker image (k8s/docker/ray/Dockerfile).
-# They and their whole dependency trees -- torch's CUDA stack, sympy, numpy, ray,
-# mlflow, ... -- are on the worker already, so a caller subtracts them from a
-# bundle rather than shipping them again.
-_WORKER_BAKED = ("ray", "mlflow", "torch", "smart_open", "dotenv", "psutil")
+# What the Ray worker image pip-installs, as the Dockerfile spells it. They and
+# their dependency trees are on the worker already, so they are never installed
+# there again -- a second copy in the job's virtualenv would shadow the image's.
+#
+# KEEP IN SYNC with k8s/docker/ray/Dockerfile, by hand: every package it
+# pip-installs must be listed here. A package missing here gets installed a
+# second time on the worker; one listed here but no longer in the image is
+# never installed at all.
+_WORKER_BAKED = (
+    "ray[default,serve]",
+    "smart_open[s3]",
+    "mlflow",
+    "python-dotenv",
+    "psutil",
+    "nvidia-cublas-cu12",
+    "nvidia-cudnn-cu12",
+    "nvidia-cuda-nvrtc-cu12",
+    "nvidia-cuda-runtime-cu12",
+    "nvidia-cuda-cupti-cu12",
+    "nvidia-cufft-cu12",
+    "nvidia-curand-cu12",
+    "nvidia-cusolver-cu12",
+    "nvidia-cusparse-cu12",
+    "nvidia-cusparselt-cu12",
+    "nvidia-nccl-cu12",
+    "nvidia-nvshmem-cu12",
+    "nvidia-nvtx-cu12",
+    "nvidia-nvjitlink-cu12",
+    "nvidia-cufile-cu12",
+    "cuda-bindings",
+    "triton",
+    "torch",
+    "filelock",
+    "typing-extensions",
+    "sympy",
+    "networkx",
+    "jinja2",
+    "fsspec",
+)
+
+
+@functools.lru_cache(maxsize=1)
+def worker_provides() -> frozenset[ThirdPartyDependencyName]:
+    """Every distribution the Ray worker image already provides:
+    `distribution_closure(_WORKER_BAKED)`."""
+    return distribution_closure(_WORKER_BAKED)
+
+
+def distribution_closure(
+    requirements: Iterable[str],
+) -> frozenset[ThirdPartyDependencyName]:
+    """Canonical names of the distributions `requirements` name plus everything
+    they depend on, transitively, as this environment's installed metadata
+    declares it -- extras followed where requested, environment markers
+    evaluated here. A requirement not installed here contributes only its own
+    name: its dependencies are unknown."""
+    names: set[ThirdPartyDependencyName] = set()
+    seen: set[tuple[str, frozenset[str]]] = set()
+    queue = [Requirement(spec) for spec in requirements]
+    while queue:
+        requirement = queue.pop()
+        name = canonicalize_name(requirement.name)
+        key = (name, frozenset(requirement.extras))
+        if key in seen:
+            continue
+        seen.add(key)
+        names.add(name)
+        try:
+            dist = importlib.metadata.distribution(name)
+        except importlib.metadata.PackageNotFoundError:
+            continue
+        extras = requirement.extras or {""}
+        for spec in dist.requires or ():
+            dependency = Requirement(spec)
+            if dependency.marker is None or any(
+                dependency.marker.evaluate({"extra": extra}) for extra in extras
+            ):
+                queue.append(dependency)
+    return frozenset(names)
+
+
+def _owning_distribution(file: Path) -> importlib.metadata.Distribution:
+    """The installed distribution whose file list (RECORD) contains `file`.
+    Rebuilds the index once on a miss, in case something was installed since it
+    was built."""
+    path = tuple(sys.path)
+    dist = _distribution_index(path).get(file)
+    if dist is None:
+        _distribution_index.cache_clear()
+        dist = _distribution_index(path).get(file)
+    if dist is None:
+        raise UnownedDependencyError(
+            f"{file} is imported from an installed package location, but no "
+            "installed distribution lists it among its files, so it can be "
+            "neither shipped nor pip-installed on the worker. Install it with "
+            "pip or uv so it carries distribution metadata."
+        )
+    return dist
 
 
 @functools.lru_cache(maxsize=1)
-def worker_provides() -> frozenset[Path]:
-    """Every file the Ray worker image already provides. Subtract from a bundle
-    before shipping: `bundle(entry).local_files - worker_provides()`."""
-    provided: set[Path] = set()
-    for name in _WORKER_BAKED:
-        origin = _module_file(name)
-        if origin is not None:
-            provided |= bundle(origin).local_files
-    return frozenset(provided)
+def _distribution_index(
+    path: tuple[str, ...],
+) -> dict[Path, importlib.metadata.Distribution]:
+    """Every file installed by a distribution found on `path` (sys.path, which
+    the cache is keyed on), mapped to that distribution. Each distribution's
+    root is resolved once; its files are joined onto it lexically, which keeps
+    this fast for environments with tens of thousands of files."""
+    index: dict[Path, importlib.metadata.Distribution] = {}
+    for dist in importlib.metadata.distributions(path=list(path)):
+        root = Path(dist.locate_file("")).resolve()
+        for file in dist.files or ():
+            index[Path(os.path.normpath(root / file))] = dist
+    return index
 
 
 def _is_local(file: Path) -> bool:

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/cortexgrid/_serve_entry.py

@@ -2,7 +2,8 @@
 
 Ray Serve's REST `import_path` resolves to `cortexgrid._serve_entry:build`.
 On the cluster replica, `build` imports the serve-app class bundled at
-`save_model` time (its import path was stored as an MLflow tag), reads its
+`save_model` time (its import path was stored as an MLflow tag), applies Ray's
+ingress with the app it was marked with by `cortexgrid.serve.ingress`, reads its
 `num_gpus`/`num_replicas` class attributes for actor placement, wraps it as a
 Ray Serve deployment, and binds it with the (family, suffix, run_name)
 identifiers.
@@ -27,10 +28,17 @@ from typing import Any
 from ray import serve
 from ray.serve.deployment import Application
 
+from cortexgrid.serve import ingress_app
+
 
 def build(args: dict[str, Any]) -> Application:
     module_name, class_name = args["class_import_path"].split(":")
     serve_app = getattr(importlib.import_module(module_name), class_name)
+    # Models saved with a class wrapped by ray.serve.ingress itself carry no
+    # mark and are deployed as they are.
+    app = ingress_app(serve_app)
+    if app is not None:
+        serve_app = serve.ingress(app)(serve_app)
     num_gpus = getattr(serve_app, "num_gpus", 0)
     num_replicas = getattr(serve_app, "num_replicas", 1)
     return serve.deployment(serve_app).options(

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/cortexgrid/jobs.py

@@ -63,6 +63,9 @@ class JobLifecycle:
     retry: bool = False  # static flag set at job creation
     num_gpus: int = 0
     num_cpus: int = 1
+    # static: pinned third-party requirements the worker pip-installs (the
+    # bundle's distributions the Ray image does not already provide)
+    pip_requirements: list[str] = field(default_factory=list)
     history: list[LifecycleEvent] = field(default_factory=list)
 
     def to_json(self) -> str:
@@ -229,11 +232,17 @@ def schedule_remote_job(
     job_id = Haikunator().haikunate(token_length=2, token_chars="0123456789")
     entry_file = Path(inspect.getfile(fn)).resolve()
     driver_file = Path(__file__).with_name("_ray_job_driver.py")
-    files = bundle(entry_file).merge(bundle(driver_file)).local_files - worker_provides()
+    desc = bundle(entry_file).merge(bundle(driver_file))
+    pip_requirements = desc.pip_requirements(worker_provides())
     with tempfile.TemporaryDirectory() as tmp:
         code_root = Path(tmp, "project_code_root")
-        stage(files, code_root)
-        log.info("Submitting job %s (%d files)", job_id, len(files))
+        stage(desc.local_files, code_root)
+        log.info(
+            "Submitting job %s (%d files, pip: %s)",
+            job_id,
+            len(desc.local_files),
+            pip_requirements,
+        )
         Payload(
             experiment_name=experiment_name,
             run_id=run_id,
@@ -252,6 +261,7 @@ def schedule_remote_job(
             retry=retry,
             num_gpus=num_gpus,
             num_cpus=num_cpus,
+            pip_requirements=pip_requirements,
         ).save_to_mlflow()
     return job_id
 

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/cortexgrid/model_serving.py

@@ -19,11 +19,12 @@ the end-to-end design.
 from __future__ import annotations
 
 import inspect
+import json
 import logging
 import shutil
 import tempfile
 import time
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Any
 
@@ -90,6 +91,9 @@ class BundleMetadata:
 
     bundle_url: str
     class_import_path: str
+    # pinned third-party requirements the replica pip-installs (the bundle's
+    # distributions the Ray image does not already provide)
+    pip_requirements: list[str] = field(default_factory=list)
 
 
 def bundle_class(
@@ -100,15 +104,32 @@ def bundle_class(
 
     Returns the metadata `deploy_model` needs later; callers (typically
     `save_model`) persist it on the ModelVersion so the deploy step can run
-    without holding the class object."""
+    without holding the class object.
+
+    Raises ValueError for a class wrapped by `ray.serve.ingress`: that wrapper
+    is a subclass Ray defines in its own module, and on older Ray (e.g. 2.9) it
+    reports that module as its own, so the class's source and import path would
+    resolve to Ray instead of the serve-app. `cortexgrid.serve.ingress` leaves
+    the class unwrapped."""
+    if any(klass.__module__.startswith("ray.serve") for klass in cls.__mro__):
+        raise ValueError(
+            f"{cls.__name__} is wrapped by ray.serve.ingress; decorate it with "
+            "cortexgrid.serve.ingress instead (from cortexgrid import serve)"
+        )
     entry_file = Path(inspect.getfile(cls)).resolve()
     serve_entry = Path(__file__).with_name("_serve_entry.py")
-    files = bundle(entry_file).merge(bundle(serve_entry)).local_files - worker_provides()
+    desc = bundle(entry_file).merge(bundle(serve_entry))
+    pip_requirements = desc.pip_requirements(worker_provides())
     with tempfile.TemporaryDirectory() as tmp:
         code_root = Path(tmp) / "code"
-        stage(files, code_root)
+        stage(desc.local_files, code_root)
         log.info(
-            "Serve bundle for %s/%s/%s: %d files", family, suffix, run_name, len(files)
+            "Serve bundle for %s/%s/%s: %d files, pip: %s",
+            family,
+            suffix,
+            run_name,
+            len(desc.local_files),
+            pip_requirements,
         )
         zip_base = Path(tmp) / f"{family}__{suffix}"
         shutil.make_archive(str(zip_base), "zip", root_dir=str(code_root))
@@ -119,6 +140,7 @@ def bundle_class(
     return BundleMetadata(
         bundle_url=bundle_url,
         class_import_path=f"{cls.__module__}:{cls.__name__}",
+        pip_requirements=pip_requirements,
     )
 
 
@@ -126,6 +148,13 @@ def _build_application_spec(
     family: str, suffix: str, run_name: str, meta: BundleMetadata
 ) -> dict[str, Any]:
     """Assemble a Ray Serve application schema from pre-bundled metadata."""
+    # working_dir carries the serve-app's own source; Ray pip-installs the
+    # third-party distributions the image lacks into a per-node cached
+    # virtualenv layered on the image. No pip key when there are none, so Ray
+    # builds no virtualenv.
+    runtime_env: dict[str, Any] = {"working_dir": meta.bundle_url}
+    if meta.pip_requirements:
+        runtime_env["pip"] = meta.pip_requirements
     return {
         "name": _app_name(family, suffix, run_name),
         "route_prefix": _route_prefix(family, suffix, run_name),
@@ -140,9 +169,7 @@ def _build_application_spec(
             "suffix": suffix,
             "run_name": run_name,
         },
-        # Every dependency ships as source inside the bundle, so working_dir
-        # alone makes the serve app importable; nothing is pip-installed.
-        "runtime_env": {"working_dir": meta.bundle_url},
+        "runtime_env": runtime_env,
     }
 
 
@@ -150,6 +177,7 @@ def _build_application_spec(
 # `deploy_model` reads back.
 _CLASS_IMPORT_PATH_TAG = "class_import_path"
 _BUNDLE_URL_TAG = "serve_bundle_url"
+_PIP_REQUIREMENTS_TAG = "serve_pip_requirements"
 
 
 def metadata_to_tags(meta: BundleMetadata) -> dict[str, str]:
@@ -159,6 +187,7 @@ def metadata_to_tags(meta: BundleMetadata) -> dict[str, str]:
     return {
         _CLASS_IMPORT_PATH_TAG: meta.class_import_path,
         _BUNDLE_URL_TAG: meta.bundle_url,
+        _PIP_REQUIREMENTS_TAG: json.dumps(meta.pip_requirements),
     }
 
 
@@ -180,6 +209,8 @@ def _load_bundle_metadata(
         return BundleMetadata(
             bundle_url=tags[_BUNDLE_URL_TAG],
             class_import_path=tags[_CLASS_IMPORT_PATH_TAG],
+            # Absent on models saved before dependencies were pip-installed.
+            pip_requirements=json.loads(tags.get(_PIP_REQUIREMENTS_TAG, "[]")),
         )
     except KeyError as exc:
         raise ValueError(

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/cortexgrid/model_storage.py

@@ -143,7 +143,8 @@ def save_model(
     the caller's concern. That directory boundary is the open-closed extension
     point - new model kinds need no change here.
 
-    `serve_app` is the Ray Serve ingress class that will front these weights.
+    `serve_app` is the `cortexgrid.serve.ingress` class that will front these
+    weights.
     Its code is bundled and its import path, bundle URL, and pip list are
     stored as tags on the ModelVersion so `deploy_model` can bind it later
     without the caller holding the class object."""

cortexgrid-0.2.87/cortexgrid/serve.py

@@ -0,0 +1,43 @@
+"""Declare a serve-app's HTTP ingress without importing Ray.
+
+    from cortexgrid import serve
+
+    @serve.ingress(app)
+    class MyServeApp: ...
+
+Same shape as `ray.serve.ingress`, but the class is left exactly as written: the
+FastAPI app is only recorded on it, and `cortexgrid._serve_entry.build` applies
+Ray's ingress when it builds the Serve application on the cluster.
+
+Ray's decorator replaces the class with a wrapper subclass defined in
+ray/serve/api.py; older Ray (e.g. 2.9) leaves the wrapper's __module__ naming
+that module. Everything that locates a serve-app by its module - bundling its
+source, recording its import path - would then find Ray instead of the user's
+code. Deferring the wrap to the one place Serve needs it keeps the class
+locatable everywhere else (the laptop, Ray jobs, tests).
+"""
+
+from __future__ import annotations
+
+from typing import Any, Callable, TypeVar
+
+
+_T = TypeVar("_T", bound=type)
+
+_INGRESS_APP_ATTR = "__cortexgrid_ingress_app__"
+
+
+def ingress(app: Any) -> Callable[[_T], _T]:
+    """Mark a serve-app class as fronted by the ASGI `app` (e.g. a FastAPI
+    instance). Returns the class itself, unwrapped."""
+
+    def decorator(cls: _T) -> _T:
+        setattr(cls, _INGRESS_APP_ATTR, app)
+        return cls
+
+    return decorator
+
+
+def ingress_app(cls: type) -> Any | None:
+    """The app `cls` was marked with by `ingress`, or None if it was not."""
+    return getattr(cls, _INGRESS_APP_ATTR, None)

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/docs/cortexgrid/README.md

@@ -88,9 +88,10 @@ print(f"Submitted: {job_id}")
 A separate service — the **jobs control plane** — polls MLflow for pending job requests, matches them against the set of Ray submissions the cluster already has, and submits anything missing. It is also responsible for retrying failed jobs and honouring user-requested stops.
 
 Each submission captures the code and dependencies the entry function needs automatically ([_bundle.py](https://github.com/robodatalab/cortexgrid/blob/main/cortexgrid/_bundle.py)):
-- `bundle(entry)` traces the import graph from the function's source file, resolving each import the way the interpreter does (via `sys.path`), and returns every file needed to run it -- your own modules and third-party packages alike, wherever they live. The standard library is excluded (it ships with the interpreter)
-- Everything ships **as source**: the bundle is staged at each file's import path and tarred into the Ray `working_dir`. Nothing is `pip`-installed on the worker
-- Dependencies the worker image already has are subtracted rather than shipped: `bundle(entry) - worker_provides()`, where `worker_provides()` is the bundle of the packages baked into the ray image (torch and its CUDA stack, ray, mlflow, ...). See [k8s/docker/ray/Dockerfile](https://github.com/robodatalab/cortexgrid/blob/main/k8s/docker/ray/Dockerfile)
+- `bundle(entry)` traces the import graph from the function's source file, resolving each import the way the interpreter does (via `sys.path`). The standard library is excluded (it ships with the interpreter)
+- Your own modules -- anything outside site-packages / dist-packages -- ship **as source**: they are staged at their import paths and tarred into the Ray `working_dir`
+- Third-party packages are recorded as the installed distribution that owns the imported file, pinned to its installed version (`tqdm==4.67.3`), and Ray **pip-installs** them on the worker into a per-node cached virtualenv layered on the image (`runtime_env["pip"]`). An import into site-packages that no installed distribution owns fails `cortexgrid.remote` with `UnownedDependencyError`
+- Distributions the worker image already has are not installed again: `worker_provides()` is the dependency closure of the packages baked into the ray image (torch and its CUDA stack, ray, mlflow, ...), and is subtracted from the pip list. See [k8s/docker/ray/Dockerfile](https://github.com/robodatalab/cortexgrid/blob/main/k8s/docker/ray/Dockerfile) and `_WORKER_BAKED` in `_bundle.py`, which must list the same packages
 - Injects MLflow/S3 credentials so task code running on the DGX can reach all services
 
 ##### Retries
@@ -126,9 +127,24 @@ s3_client = cortexgrid.get_s3_client()           # boto3 S3 client
 
 #### Model registry and serving
 
-Save a trained model's weights together with the serve-app that fronts it, then deploy it as a Ray Serve application:
+Save a trained model's weights together with the serve-app that fronts it, then deploy it as a Ray Serve application. A serve-app is a class fronted by a FastAPI app, marked with cortexgrid's `serve.ingress` (not Ray's):
 
 ```python
+from cortexgrid import serve
+from fastapi import FastAPI
+
+app = FastAPI()
+
+@serve.ingress(app)
+class MyServeApp:
+    num_gpus = 1
+
+    def __init__(self, family: str, suffix: str, run_name: str) -> None:
+        self._weights_dir = cortexgrid.load_model(family, suffix, run_name)
+
+    @app.post("/complete")
+    async def complete(self, body: dict): ...
+
 saved = cortexgrid.save_model(weights_dir, MyServeApp, family="qwen", suffix="instruct")
 deployed = cortexgrid.deploy_model("qwen", "instruct", saved.run_name, wait=True)
 print(deployed.url)

{cortexgrid-0.2.85 → cortexgrid-0.2.87}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "cortexgrid"
-version = "0.2.85"
+version = "0.2.87"
 description = "Connect your ML code to the RoboLab compute cluster — Ray, MLflow, and S3"
 readme = "docs/cortexgrid/README.md"
 license = "Apache-2.0"
@@ -9,6 +9,7 @@ requires-python = ">=3.11"
 dependencies = [
     "ray[default]>=2.9,<3",
     "mlflow>=3.11,<4",
+    "packaging>=24",
     "boto3>=1.34",
     "setuptools>=82.0.1",
     "tqdm>=4.60",