lablink-cli 0.1.0a1__tar.gz

lablink_cli-0.1.0a1/PKG-INFO

@@ -0,0 +1,76 @@
+Metadata-Version: 2.4
+Name: lablink-cli
+Version: 0.1.0a1
+Summary: CLI tool for deploying and managing LabLink infrastructure
+Author-email: Elizabeth Berrigan <eberrigan@salk.edu>, Talmo Pereira <talmo@salk.edu>, Andrew Park <hep003@ucsd.edu>
+Project-URL: Homepage, https://github.com/talmolab/lablink
+Project-URL: Issues, https://github.com/talmolab/lablink/issues
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: lablink-allocator-service[config]>=0.2.0
+Requires-Dist: typer>=0.15
+Requires-Dist: textual>=3.0
+Requires-Dist: rich>=13.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: boto3>=1.35
+Requires-Dist: psutil>=5.9
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+
+# LabLink CLI
+
+Command-line tool for deploying and managing LabLink teaching lab infrastructure —
+on AWS (EC2 via Terraform) or on a machine you already have (`provider: manual`,
+deployed with docker-compose).
+
+## Installation
+
+This package is not yet published to PyPI. Install it from source — the repo is a
+`uv` workspace, so sync all three packages into the shared root venv:
+
+```bash
+git clone https://github.com/talmolab/lablink.git
+cd lablink
+uv sync --all-packages
+```
+
+## Usage
+
+```bash
+lablink --help
+lablink --version   # or -v
+```
+
+### Commands
+
+| Command          | Description                                                                              |
+| ---------------- | ---------------------------------------------------------------------------------------- |
+| `configure`      | Create or edit LabLink configuration (interactive TUI)                                   |
+| `setup`          | Provision provider-specific bootstrap resources (AWS: S3 + DynamoDB for Terraform state) |
+| `doctor`         | Check prerequisites and configuration                                                    |
+| `deploy`         | Deploy LabLink infrastructure (AWS Terraform or docker-compose)                          |
+| `destroy`        | Tear down LabLink infrastructure                                                         |
+| `status`         | Show deployment health and inventory                                                     |
+| `logs`           | View allocator and client logs                                                           |
+| `export-metrics` | Export deployment metrics to CSV or JSON                                                 |
+| `stats`          | Show a cohort session-metrics summary in the terminal                                    |
+| `cleanup`        | Remove deployment resources and local state                                              |
+| `show-config`    | View the current LabLink configuration                                                   |
+| `cache-clear`    | Clear LabLink caches (Terraform templates, deployment metrics)                           |
+
+### Client fleet commands
+
+| Command                | Description                                                                      |
+| ---------------------- | -------------------------------------------------------------------------------- |
+| `client launch`        | Launch client VMs via the allocator service (AWS provider)                       |
+| `client register`      | Register this bring-your-own box as a manual client and run the client container |
+| `client unregister`    | Tear down a registered BYO box                                                   |
+| `client reset-overlay` | Discard this box's persisted mesh-overlay node identity                          |
+
+Run `lablink <command> --help` for details on any command.
+
+## Documentation
+
+Full CLI documentation: https://talmolab.github.io/lablink/cli/

lablink_cli-0.1.0a1/README.md

@@ -0,0 +1,55 @@
+# LabLink CLI
+
+Command-line tool for deploying and managing LabLink teaching lab infrastructure —
+on AWS (EC2 via Terraform) or on a machine you already have (`provider: manual`,
+deployed with docker-compose).
+
+## Installation
+
+This package is not yet published to PyPI. Install it from source — the repo is a
+`uv` workspace, so sync all three packages into the shared root venv:
+
+```bash
+git clone https://github.com/talmolab/lablink.git
+cd lablink
+uv sync --all-packages
+```
+
+## Usage
+
+```bash
+lablink --help
+lablink --version   # or -v
+```
+
+### Commands
+
+| Command          | Description                                                                              |
+| ---------------- | ---------------------------------------------------------------------------------------- |
+| `configure`      | Create or edit LabLink configuration (interactive TUI)                                   |
+| `setup`          | Provision provider-specific bootstrap resources (AWS: S3 + DynamoDB for Terraform state) |
+| `doctor`         | Check prerequisites and configuration                                                    |
+| `deploy`         | Deploy LabLink infrastructure (AWS Terraform or docker-compose)                          |
+| `destroy`        | Tear down LabLink infrastructure                                                         |
+| `status`         | Show deployment health and inventory                                                     |
+| `logs`           | View allocator and client logs                                                           |
+| `export-metrics` | Export deployment metrics to CSV or JSON                                                 |
+| `stats`          | Show a cohort session-metrics summary in the terminal                                    |
+| `cleanup`        | Remove deployment resources and local state                                              |
+| `show-config`    | View the current LabLink configuration                                                   |
+| `cache-clear`    | Clear LabLink caches (Terraform templates, deployment metrics)                           |
+
+### Client fleet commands
+
+| Command                | Description                                                                      |
+| ---------------------- | -------------------------------------------------------------------------------- |
+| `client launch`        | Launch client VMs via the allocator service (AWS provider)                       |
+| `client register`      | Register this bring-your-own box as a manual client and run the client container |
+| `client unregister`    | Tear down a registered BYO box                                                   |
+| `client reset-overlay` | Discard this box's persisted mesh-overlay node identity                          |
+
+Run `lablink <command> --help` for details on any command.
+
+## Documentation
+
+Full CLI documentation: https://talmolab.github.io/lablink/cli/

lablink_cli-0.1.0a1/pyproject.toml

@@ -0,0 +1,59 @@
+[project]
+name = "lablink-cli"
+authors = [
+  {name = "Elizabeth Berrigan", email = "eberrigan@salk.edu"},
+  {name = "Talmo Pereira", email = "talmo@salk.edu"},
+  {name = "Andrew Park", email = "hep003@ucsd.edu"},
+]
+version = "0.1.0a1"
+description = "CLI tool for deploying and managing LabLink infrastructure"
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = [
+    # >=0.2.0: deploy_compose imports PUBLIC_HOSTNAME_HINT,
+    # is_valid_public_hostname and is_weak_admin_password, none of which
+    # exist in 0.1.2 — an unpinned resolve breaks `provider: manual`.
+    "lablink-allocator-service[config]>=0.2.0",
+    "typer>=0.15",
+    "textual>=3.0",
+    "rich>=13.0",
+    "pyyaml>=6.0",
+    "boto3>=1.35",
+    "psutil>=5.9",
+]
+
+[project.optional-dependencies]
+dev = ["pytest", "pytest-cov", "ruff"]
+
+[project.scripts]
+lablink = "lablink_cli.app:main"
+
+[build-system]
+requires = ["setuptools>=75"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+lablink_cli = ["templates/*.yml"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+markers = ["integration: real network tests (deselect with '-m not integration')"]
+addopts = "-m 'not integration'"
+
+[tool.coverage.run]
+omit = [
+    "src/lablink_cli/tui/*",
+]
+
+[tool.coverage.report]
+exclude_lines = [
+    "pragma: no cover",
+    "if __name__",
+]
+
+[project.urls]
+Homepage = "https://github.com/talmolab/lablink"
+Issues = "https://github.com/talmolab/lablink/issues"

lablink_cli-0.1.0a1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

lablink_cli-0.1.0a1/src/lablink_cli/__init__.py

@@ -0,0 +1,8 @@
+"""LabLink CLI - Deploy and manage LabLink infrastructure."""
+
+TEMPLATE_REPO = "talmolab/lablink-template"
+TEMPLATE_VERSION = "v0.3.0"
+# SHA-256 of the GitHub release tarball for TEMPLATE_VERSION.
+# Update this when bumping TEMPLATE_VERSION.
+# To compute: curl -sL <tarball_url> | sha256sum
+TEMPLATE_SHA256 = "3d3f8803990f80e6bcda0b3098178d909bc454315fdf2304e70dbee5d43e3bb6"

lablink_cli-0.1.0a1/src/lablink_cli/api.py

@@ -0,0 +1,428 @@
+"""HTTP client for the LabLink allocator service."""
+
+from __future__ import annotations
+
+import base64
+import json
+import ssl
+import time
+from typing import Callable, NoReturn
+from urllib.error import HTTPError, URLError
+from urllib.parse import urlencode
+from urllib.request import Request, urlopen
+
+try:
+    from importlib.metadata import version as _pkg_version
+
+    _CLI_VERSION = _pkg_version("lablink-cli")
+except Exception:  # pragma: no cover - package metadata unavailable
+    _CLI_VERSION = "0.0.0"
+
+# Product User-Agent for all CLI HTTP requests. urllib's default
+# "Python-urllib/x.y" is blocked with HTTP 403 by Cloudflare-proxied
+# allocators, so every Request must identify itself with this instead.
+USER_AGENT = f"lablink-cli/{_CLI_VERSION}"
+
+
+def ssl_context(ssl_provider: str) -> ssl.SSLContext:
+    """Verifying context, except under `self_signed` where there is no CA
+    to verify against and the operator has opted into that trade."""
+    ctx = ssl.create_default_context()
+    if ssl_provider == "self_signed":
+        ctx.check_hostname = False
+        ctx.verify_mode = ssl.CERT_NONE
+    return ctx
+
+
+def basic_auth_header(admin_user: str, admin_password: str) -> str:
+    credentials = base64.b64encode(
+        f"{admin_user}:{admin_password}".encode()
+    ).decode()
+    return f"Basic {credentials}"
+
+
+def _read_body(
+    url: str,
+    *,
+    method: str,
+    auth_header: str,
+    ssl_ctx: ssl.SSLContext,
+    timeout: int,
+    data: bytes | None = None,
+    content_type: str | None = None,
+) -> str:
+    """Send one authenticated request and return the decoded body.
+
+    Owns the plumbing every caller in this module shares — the header
+    set, the context manager, the decode — and nothing else. HTTPError
+    and URLError propagate untranslated: the three callers each map them
+    differently (raw for `authenticated_json_request`, typed exceptions
+    with 404/502 handling for `AllocatorAPI`, typed with 409/400 for
+    `RegistrationClient`), and that mapping is the part that must stay
+    per-caller.
+    """
+    req = Request(url, data=data, method=method)
+    req.add_header("User-Agent", USER_AGENT)
+    req.add_header("Authorization", auth_header)
+    req.add_header("Accept", "application/json")
+    if content_type:
+        req.add_header("Content-Type", content_type)
+
+    # S310: URL scheme is operator-supplied by design (allocator base URL
+    # from the CLI config); skipping the scheme allowlist check.
+    with urlopen(req, timeout=timeout, context=ssl_ctx) as resp:  # noqa: S310
+        return resp.read().decode()
+
+
+def authenticated_json_request(
+    url: str,
+    admin_user: str,
+    admin_password: str,
+    *,
+    ssl_provider: str = "none",
+    timeout: int = 60,
+) -> dict:
+    """GET `url` with HTTP Basic auth and return the parsed JSON body.
+
+    Raises the underlying HTTPError/URLError/json.JSONDecodeError
+    unwrapped — stats/logs/export_metrics each need different
+    presentation for the same failures (a red console message here, a
+    TUI error dict there), so this only owns the request plumbing, not
+    the error translation.
+    """
+    raw = _read_body(
+        url,
+        method="GET",
+        auth_header=basic_auth_header(admin_user, admin_password),
+        ssl_ctx=ssl_context(ssl_provider),
+        timeout=timeout,
+    )
+    return json.loads(raw)
+
+# /destroy and /api/launch are async (they return a job id immediately;
+# the allocator runs OpenTofu on a background thread). Individual
+# requests (submit or poll) are fast, so a short per-request timeout
+# replaces the old single-request timeouts of up to 1800s;
+# _POLL_TIMEOUT_SECONDS is the new overall ceiling for how long we'll
+# keep polling before giving up (matches the old ceiling).
+_REQUEST_TIMEOUT_SECONDS = 30
+_POLL_INTERVAL_SECONDS = 2.0
+_POLL_TIMEOUT_SECONDS = 1800
+
+# The exact message providers/aws.py's destroy_hosts raises (wrapped by
+# main.py's /destroy route into a failed operation with this same text)
+# when no client VMs were ever launched. Matched here so destroy_vms()
+# can keep raising AllocatorNotFoundError for this one case, preserving
+# the contract deploy.py's callers already handle (non-fatal — continue
+# tearing down the allocator).
+_NO_VMS_LAUNCHED_ERROR = (
+    "tfvars does not exist — no client VMs were launched"
+)
+
+
+class AllocatorError(Exception):
+    """Base exception for allocator API errors."""
+
+
+class AllocatorAuthError(AllocatorError):
+    """401 Unauthorized."""
+
+
+class AllocatorNotFoundError(AllocatorError):
+    """404 Not Found (no client VMs launched)."""
+
+
+class AllocatorUnavailableError(AllocatorError):
+    """502 Bad Gateway or connection failure."""
+
+
+class AllocatorOperationTimeout(AllocatorError):
+    """An operation did not reach a terminal status within the poll deadline."""
+
+
+class AllocatorAPI:
+    """HTTP client for the allocator service."""
+
+    def __init__(
+        self,
+        base_url: str,
+        admin_user: str,
+        admin_password: str,
+        ssl_provider: str = "none",
+    ) -> None:
+        self.base_url = base_url.rstrip("/")
+        self._auth_header = basic_auth_header(admin_user, admin_password)
+        self._ssl_ctx = ssl_context(ssl_provider)
+
+    def destroy_vms(
+        self,
+        on_progress: Callable[[int | None, int | None], None] | None = None,
+    ) -> dict | None:
+        """POST /destroy to tear down client VMs, then poll until the
+        job reaches a terminal status.
+
+        Returns {"status": "success", "output": <str>}. Raises
+        AllocatorNotFoundError if no client VMs were ever launched
+        (same as the old synchronous contract), or
+        AllocatorError/AllocatorOperationTimeout otherwise.
+        """
+        return self._submit_and_poll(
+            "POST", "/destroy", on_progress=on_progress,
+        )
+
+    def launch_vms(
+        self,
+        num_vms: int,
+        on_progress: Callable[[int | None, int | None], None] | None = None,
+    ) -> dict | None:
+        """POST /api/launch to provision num_vms new client VMs, then
+        poll until the job reaches a terminal status.
+
+        Returns {"status": "success", "output": <str>} on success.
+        """
+        data = urlencode({"num_vms": str(num_vms)}).encode()
+        return self._submit_and_poll(
+            "POST",
+            "/api/launch",
+            data,
+            content_type="application/x-www-form-urlencoded",
+            on_progress=on_progress,
+        )
+
+    def _submit_and_poll(
+        self,
+        method: str,
+        path: str,
+        data: bytes | None = b"",
+        *,
+        content_type: str | None = None,
+        on_progress: Callable[[int | None, int | None], None] | None = None,
+    ) -> dict | None:
+        """Submit an async apply/destroy job and poll GET
+        /api/operations/<id> until it reaches a terminal status.
+
+        If on_progress is given, it's called once per poll tick with
+        (resources_completed, resources_total) read off the operation
+        response. Both are None if the allocator predates progress
+        reporting (the keys are simply absent from its response) —
+        callers must handle that, not assume real numbers."""
+        submitted = self._request(
+            method, path, data, content_type=content_type
+        )
+        if not submitted or "job_id" not in submitted:
+            raise AllocatorError(
+                f"Unexpected response from {path}: {submitted}"
+            )
+        job_id = submitted["job_id"]
+
+        deadline = time.monotonic() + _POLL_TIMEOUT_SECONDS
+        while True:
+            try:
+                op = self._request("GET", f"/api/operations/{job_id}")
+            except AllocatorUnavailableError:
+                # Transient network blip while polling — the job keeps
+                # running server-side regardless of whether our poll
+                # request succeeds, so retry rather than abort an
+                # operation that may still finish successfully.
+                op = None
+
+            if op is not None:
+                if on_progress:
+                    on_progress(
+                        op.get("resources_completed"),
+                        op.get("resources_total"),
+                    )
+                status = op.get("status")
+                if status == "succeeded":
+                    return {
+                        "status": "success",
+                        "output": op.get("output") or "",
+                    }
+                if status in ("failed", "interrupted"):
+                    error_text = (
+                        op.get("error") or f"Operation #{job_id} {status}"
+                    )
+                    if _NO_VMS_LAUNCHED_ERROR in error_text:
+                        raise AllocatorNotFoundError(error_text)
+                    raise AllocatorError(error_text)
+
+            if time.monotonic() > deadline:
+                raise AllocatorOperationTimeout(
+                    f"Operation #{job_id} did not finish within "
+                    f"{_POLL_TIMEOUT_SECONDS}s"
+                )
+            time.sleep(_POLL_INTERVAL_SECONDS)
+
+    def _request(
+        self,
+        method: str,
+        path: str,
+        data: bytes | None = None,
+        *,
+        content_type: str | None = None,
+    ) -> dict | None:
+        """Send an HTTP request to the allocator."""
+        try:
+            raw = _read_body(
+                f"{self.base_url}{path}",
+                method=method,
+                auth_header=self._auth_header,
+                ssl_ctx=self._ssl_ctx,
+                timeout=_REQUEST_TIMEOUT_SECONDS,
+                data=data,
+                content_type=content_type,
+            )
+        except HTTPError as e:
+            self._handle_http_error(e)
+        except URLError as e:
+            raise AllocatorUnavailableError(str(e.reason)) from e
+
+        try:
+            body = json.loads(raw)
+        except (json.JSONDecodeError, ValueError):
+            return None
+
+        if isinstance(body, dict) and body.get("status") == "error":
+            raise AllocatorError(
+                body.get("error", "unknown error")
+            )
+
+        return body
+
+    def _handle_http_error(self, e: HTTPError) -> NoReturn:
+        """Translate HTTPError to typed exception."""
+        if e.code == 401:
+            raise AllocatorAuthError("Authentication failed") from e
+        if e.code == 404:
+            raise AllocatorNotFoundError(
+                "No client VMs found"
+            ) from e
+        if e.code == 502:
+            raise AllocatorUnavailableError(
+                "Allocator is unhealthy (502)"
+            ) from e
+
+        try:
+            body = json.loads(e.read().decode())
+            msg = body.get("error", str(e))
+        except (json.JSONDecodeError, UnicodeDecodeError):
+            msg = str(e)
+
+        raise AllocatorError(
+            f"HTTP {e.code}: {msg}"
+        ) from e
+
+
+class AllocatorConflictError(AllocatorError):
+    """409 Conflict (already-registered machine_identity)."""
+
+
+class RegistrationClient:
+    """HTTP client for the registration endpoint (Bearer auth, no admin creds).
+
+    Used by `lablink client register` on the BYO box. Unlike AllocatorAPI which
+    requires admin Basic auth, RegistrationClient authenticates with the
+    bootstrap register_token only — which is what a BYO box has at
+    onboarding time.
+    """
+
+    def __init__(
+        self,
+        base_url: str,
+        register_token: str,
+        *,
+        ssl_provider: str = "none",
+    ) -> None:
+        self.base_url = base_url.rstrip("/")
+        self._auth_header = f"Bearer {register_token}"
+        self._ssl_ctx = ssl_context(ssl_provider)
+
+    def register(
+        self,
+        *,
+        hostname: str,
+        machine_identity: str,
+        gpu_present: bool,
+        gpu_model: str | None,
+        lan_ip: str | None = None,
+        overlay_hostname: str | None = None,
+        reverse_tunnel: bool = False,
+    ) -> dict:
+        """POST /api/v1/clients/register; return parsed JSON.
+
+        Exactly one of ``lan_ip`` (real BYO box, LAN-direct connectivity),
+        ``overlay_hostname`` (mesh-overlay connectivity, e.g. a
+        Run:AI-hosted workload), or ``reverse_tunnel=True`` (reverse-tunnel
+        connectivity — the client dials out, so it has no address for the
+        allocator to reach) is expected — enforced by the caller
+        (``run_register``), not here.
+
+        Raises AllocatorAuthError on 401, AllocatorConflictError on 409,
+        AllocatorUnavailableError on connection failure, AllocatorError
+        on other HTTP error codes / malformed response.
+        """
+        if reverse_tunnel:
+            # The sentinel the allocator validates against its configured
+            # connectivity. No endpoint_url: a tunnel client has no address
+            # the allocator can dial -- the tunnel is the path.
+            provider_metadata = {"reverse_tunnel": True}
+            endpoint_url = None
+        elif overlay_hostname is not None:
+            provider_metadata = {"overlay_hostname": overlay_hostname}
+            endpoint_url = None
+        else:
+            provider_metadata = {"lan_ip": lan_ip}
+            endpoint_url = f"http://{lan_ip}:7070"
+        body = {
+            "hostname": hostname,
+            "machine_identity": machine_identity,
+            "provider": "manual",
+            "endpoint_url": endpoint_url,
+            "provider_metadata": provider_metadata,
+            "gpu_present": gpu_present,
+            "gpu_model": gpu_model,
+        }
+        return self._post("/api/v1/clients/register", body)
+
+    def _post(self, path: str, body: dict) -> dict:
+        try:
+            raw = _read_body(
+                f"{self.base_url}{path}",
+                method="POST",
+                auth_header=self._auth_header,
+                ssl_ctx=self._ssl_ctx,
+                timeout=60,
+                data=json.dumps(body).encode(),
+                content_type="application/json",
+            )
+        except HTTPError as e:
+            self._handle_http_error(e)
+        except URLError as e:
+            raise AllocatorUnavailableError(str(e.reason)) from e
+
+        try:
+            return json.loads(raw)
+        except (json.JSONDecodeError, ValueError) as e:
+            raise AllocatorError("Allocator returned non-JSON response") from e
+
+    def _handle_http_error(self, e: HTTPError) -> NoReturn:
+        try:
+            err_body = json.loads(e.read().decode())
+            msg = err_body.get("error", str(e))
+        except (json.JSONDecodeError, UnicodeDecodeError):
+            msg = str(e)
+
+        if e.code == 401:
+            raise AllocatorAuthError(
+                "Registration rejected — register_token may be stale "
+                "(allocator restarts mint a new one) or wrong."
+            ) from e
+        if e.code == 409:
+            raise AllocatorConflictError(
+                "Already registered with this machine_identity. "
+                "Re-register with --force or pass a different "
+                "--machine-identity."
+            ) from e
+        if e.code == 400:
+            raise AllocatorError(f"Bad request: {msg}") from e
+        raise AllocatorError(f"HTTP {e.code}: {msg}") from e