@openhands/extensions 0.25.0 → 0.27.0

package/plugins/runtime-api-configs/README.md

@@ -0,0 +1,59 @@
+# runtime-api-configs
+
+Python CLI for managing warm sandbox pools on OpenHands Enterprise (OHE) via
+the `runtime-api` admin endpoints. This plugin supersedes the ad-hoc
+`warm-runtime-configs.sh` snippets embedded in the docs.
+
+- **Skill definition:** [`SKILL.md`](./SKILL.md)
+- **CLI:** [`scripts/warm_runtime_configs.py`](./scripts/warm_runtime_configs.py)
+- **Docs:** <https://docs.openhands.dev/enterprise/custom-sandbox-images/multiple-images-warm-pools>
+
+## Quick start
+
+Export the runtime-api URL and the admin password you set in the OHE Admin
+Console (VM installs) or that lives in the `admin-password` Kubernetes
+secret (Helm installs):
+
+```bash
+export RUNTIME_API_URL=https://runtime-api.<your-base-domain>
+export ADMIN_PASSWORD=<the-runtime-api-admin-password>
+```
+
+Then drive the CLI over HTTPS. No `kubectl`, no SSH:
+
+```bash
+# 1. Inspect the effective set
+python3 scripts/warm_runtime_configs.py list
+
+# 2. Register a new custom sandbox image
+python3 scripts/warm_runtime_configs.py template v1_current \
+  --image ghcr.io/your-org/openhands-php:8.4-v1 --count 1 \
+  | python3 scripts/warm_runtime_configs.py save php-web --file -
+
+# 3. Delete when finished
+python3 scripts/warm_runtime_configs.py delete php-web
+```
+
+If you would rather pull all three env vars from Kubernetes secrets in one
+step, see the `bootstrap` subcommand in [`SKILL.md`](./SKILL.md) - on
+Replicated VM installs it needs `sudo -E`; on Helm it needs
+`kubectl port-forward` and `--skip-url`.
+
+## Why a plugin instead of a shell script
+
+- **Runs anywhere Python 3.9+ is available.**
+- **No third-party dependencies** - pure stdlib (`urllib`, `hashlib`,
+  `argparse`).
+- **First-class `template` subcommand** replaces `jq` gymnastics when
+  deriving custom configurations from the installer default.
+- **Testable**: `tests/test_warm_runtime_configs.py` exercises the full
+  request/response path against an in-process HTTP server.
+
+## Development
+
+```bash
+python3 -m unittest discover -s tests
+```
+
+The tests spin up a stdlib `http.server` on an ephemeral port, so they need
+no cluster access.

package/plugins/runtime-api-configs/SKILL.md

@@ -0,0 +1,240 @@
+---
+name: runtime-api-configs
+description: >-
+  Manage warm sandbox pools on OpenHands Enterprise by driving the runtime-api
+  admin endpoints (list, save, delete warm runtime configurations) from a
+  Python CLI. Use when an OpenHands Enterprise administrator asks to add,
+  update, delete, or inspect custom sandbox images, warm pools, or
+  `warm-runtime-configs`, or when the docs point to
+  `enterprise/custom-sandbox-images/multiple-images-warm-pools`.
+triggers:
+- warm-runtime-config
+- warm runtime config
+- warm pool
+- warm-runtime-configs
+- custom sandbox image
+- runtime-api admin
+---
+
+# Runtime API - Warm Runtime Configurations
+
+This plugin manages the warm sandbox pools used by OpenHands Enterprise. It
+replaces the ad-hoc `warm-runtime-configs.sh` shell script embedded in the
+docs and gives administrators a single, testable Python CLI for the same
+workflow.
+
+The CLI talks HTTP to the runtime-api service. It never modifies the cluster
+directly, and it uses only the Python standard library, so it runs anywhere
+`python3` is available.
+
+## When to use this skill
+
+Reach for this skill whenever the user wants to:
+
+- **List** the effective warm-runtime configurations on an OHE install.
+- **Add or update** a custom sandbox image (e.g. `openhands-php:8.4-v1`) so
+  users can select it from the **Settings → Application → Default Sandbox**
+  dropdown.
+- **Delete** a database-managed configuration, restoring an installer-managed
+  entry with the same name.
+- **Refresh configurations after upgrading OHE**, when API-managed frozen
+  snapshots need to be re-derived from the new installer template.
+- **Bootstrap credentials** (`RUNTIME_API_URL`, `API_KEY`, `ADMIN_PASSWORD`)
+  from Kubernetes secrets so subsequent runs need no cluster access.
+
+If the user is instead asking how to **build** a custom sandbox image
+(Dockerfile, versioning), point them at
+[Building a Custom Image](https://docs.openhands.dev/enterprise/custom-sandbox-images/building-custom-images)
+- image building is out of scope for this skill.
+
+## Prerequisites
+
+- OpenHands Enterprise **0.64.0 or later**.
+- Python **3.9+** on the machine running the CLI. No third-party packages.
+- The **Runtime API Admin Password**:
+  - **VM (Replicated) installs**: set via *Admin Console → Config → Sandbox
+    Configuration → Runtime API Admin Password*.
+  - **Helm installs**: the value stored in the `admin-password` Kubernetes
+    secret in the runtime-api namespace. Helm has no Admin Console.
+- **Helm installs only**: the runtime-api chart must be running in **overlay
+  mode** and have warm pools enabled, so `v1_current` shows up in `list` and
+  can be used as a `template` source. Set both in your umbrella values:
+
+  ```yaml
+  runtime-api:
+    warmRuntimes:
+      enabled: true
+    env:
+      WARM_RUNTIME_CONFIG_OVERLAY: "1"
+  ```
+
+  Overlay mode is on by default on Replicated VM installs. Without it on
+  Helm, `list` returns `[]` on a fresh install and `template v1_current`
+  fails.
+
+## Setup
+
+On any machine with HTTPS reachability to the runtime-api, export two env
+vars:
+
+```bash
+export RUNTIME_API_URL=https://runtime-api.<your-base-domain>
+export ADMIN_PASSWORD=<runtime-api-admin-password>   # see Prerequisites above
+```
+
+That is the full setup on any install where the runtime-api ingress is
+reachable from the machine running the CLI. No `kubectl`, no SSH, no
+cluster access. If the runtime-api is cluster-internal (Helm default), see
+the Helm entry in the Advanced section below to port-forward first.
+
+Under the hood, the CLI uses the admin password directly for `save` and
+`delete` (via the PBKDF2 handshake), and for `list` and `template` it logs
+in as admin and fetches the `default` read-only API key over HTTPS from
+`/api/admin/api-keys`.
+
+If you already know the read-only API key (for example, from an operator
+runbook), export it too and the CLI will skip the extra admin-login round
+trip on reads:
+
+```bash
+export API_KEY=<default api key>
+```
+
+If you have `kubectl` access to the cluster and prefer to pull all three
+env vars from Kubernetes secrets in one step, see
+**[Advanced: bootstrap from Kubernetes](#advanced-bootstrap-from-kubernetes)**
+at the bottom of this file.
+
+## CLI reference
+
+Run with `python3 scripts/warm_runtime_configs.py <subcommand>`. Each
+subcommand exits non-zero on error and prints the runtime-api's HTTP body
+on failure.
+
+| Subcommand | Auth | Purpose |
+|---|---|---|
+| `list` | `API_KEY` if set, else admin login | Print the effective set of configurations. |
+| `template <source>` | `API_KEY` if set, else admin login | Fetch a config, strip identity fields, override image/count, print JSON ready to `save`. |
+| `save <name> [--file F]` | Admin password | Upsert a configuration. Reads JSON from `--file` or stdin. |
+| `delete <name>` | Admin password | Delete a database-managed configuration. |
+| `bootstrap` | kubectl | Print `export` lines for `RUNTIME_API_URL`, `API_KEY`, `ADMIN_PASSWORD`. Optional; see [Advanced](#advanced-bootstrap-from-kubernetes) below. |
+
+Full help: `python3 scripts/warm_runtime_configs.py --help` and
+`python3 scripts/warm_runtime_configs.py <subcommand> --help`.
+
+## Typical workflow
+
+Register a new PHP sandbox image once credentials are in the environment:
+
+```bash
+# 1. Derive a config JSON from the current default, changing only the image
+python3 scripts/warm_runtime_configs.py template v1_current \
+  --image ghcr.io/your-org/openhands-php:8.4-v1 --count 1 \
+  --output php-web.json
+
+# 2. Save it
+python3 scripts/warm_runtime_configs.py save php-web --file php-web.json
+
+# 3. Confirm
+python3 scripts/warm_runtime_configs.py list
+```
+
+Within about a minute the pool is ready and `php-web` appears in
+**Settings → Application → Default Sandbox**.
+
+Deleting an API-managed configuration removes it from the effective set:
+
+```bash
+python3 scripts/warm_runtime_configs.py delete php-web
+```
+
+`delete` operates on database rows only, so it 404s for names that have
+never been saved through this CLI (installer-managed entries like
+`v1_current` on a fresh install fall in that bucket). In overlay mode, if
+you had previously saved a database row that shadowed a same-named
+ConfigMap entry, deleting the row reverts to the ConfigMap entry on the
+next reconciler cycle.
+
+## After an OHE upgrade
+
+Frozen API-managed configurations do not follow release bumps. After every
+OHE upgrade, re-derive each API-managed configuration from the refreshed
+default template, keeping each configuration's current image tag and pool
+size unless you deliberately change them:
+
+```bash
+# One "name image count" tuple per API-managed config; edit for your fleet.
+# Preserving the existing pool count matters: hardcoding it would silently
+# shrink pools on every refresh.
+while read -r name image count; do
+  [ -z "$name" ] && continue
+  python3 scripts/warm_runtime_configs.py template v1_current \
+    --image "$image" --count "$count" \
+    | python3 scripts/warm_runtime_configs.py save "$name" --file -
+done <<'EOF'
+php-web        ghcr.io/your-org/openhands-php:8.4-v2    3
+ruby-app       ghcr.io/your-org/openhands-ruby:3.3-v2   2
+node-monorepo  ghcr.io/your-org/openhands-node:20-v2    1
+EOF
+```
+
+Two caveats worth spelling out:
+
+- `template` refreshes the *command* and *environment* to match the new
+  `v1_current`, but it keeps whatever image tag you pass. To actually
+  pick up a new agent-server version, the image itself must be rebuilt
+  against the matching agent-server base. This loop alone does not do
+  that.
+- Skipping this step leaves configurations pointing at the previous
+  agent-server command/environment, and new conversations fail with a
+  version mismatch until the configurations are updated.
+
+## Environment variables
+
+| Variable | Used by | Notes |
+|---|---|---|
+| `RUNTIME_API_URL` | all HTTP subcommands | e.g. `https://runtime-api.example.com` or `http://localhost:5000`. |
+| `ADMIN_PASSWORD` | `save`, `delete`; also `list`/`template` when `API_KEY` is unset | Runs the PBKDF2 challenge-response handshake. |
+| `ADMIN_TOKEN` | Anywhere `ADMIN_PASSWORD` is used | Pre-obtained admin JWT. When set, it wins over `ADMIN_PASSWORD` and skips the handshake. |
+| `API_KEY` | `list`, `template` (optional) | Sent as `X-API-Key`. Optional: when unset, the CLI fetches the `default` key over HTTPS via admin login. |
+| `NAMESPACE` | `bootstrap` | Defaults to `openhands`. Overridable via `--namespace`. |
+
+## Advanced: bootstrap from Kubernetes
+
+`bootstrap` extracts all three env vars from Kubernetes secrets in one
+step. Handy for cluster operators and CI, but the default HTTPS-only
+workflow above is preferred for interactive admin use.
+
+### VM (Replicated) install
+
+The k0s kubeconfig is root-owned, so the invocation runs under sudo:
+
+```bash
+eval "$(sudo -E python3 scripts/warm_runtime_configs.py bootstrap --namespace openhands)"
+```
+
+### Helm install
+
+The runtime-api is cluster-internal on Helm (chart default is
+`ingress.enabled: false`). The Kubernetes service name is release-prefixed
+via `include "runtime-api.fullname"`, so it is not literally `runtime-api`
+- discover it and port-forward, then bootstrap without touching
+`RUNTIME_API_URL`:
+
+```bash
+SVC=$(kubectl -n openhands get svc \
+  -l app.kubernetes.io/name=runtime-api \
+  -o jsonpath='{.items[0].metadata.name}')
+kubectl -n openhands port-forward "svc/$SVC" 5000:5000 &
+export RUNTIME_API_URL=http://localhost:5000
+eval "$(python3 scripts/warm_runtime_configs.py bootstrap --namespace openhands --skip-url)"
+```
+
+`--skip-url` leaves `RUNTIME_API_URL` alone. If the operator set
+`nameOverride` on the runtime-api subchart, adjust the label selector to
+match.
+
+## See also
+
+- Docs: [Configuring Custom Sandbox Images](https://docs.openhands.dev/enterprise/custom-sandbox-images/multiple-images-warm-pools)
+- Docs: [Using Custom Images](https://docs.openhands.dev/enterprise/custom-sandbox-images/using-custom-images)

package/plugins/runtime-api-configs/scripts/warm_runtime_configs.py

@@ -0,0 +1,331 @@
+#!/usr/bin/env python3
+"""Manage warm sandbox configurations on the OpenHands Enterprise runtime-api.
+
+Standard-library-only CLI. Runs on any host with Python 3.9+, including
+inside the runtime-api pod on Helm installs.
+
+Environment variables:
+    RUNTIME_API_URL   Base URL, e.g. https://runtime-api.example.com
+    ADMIN_PASSWORD    Password used for the PBKDF2 challenge/response handshake
+    API_KEY           Sent as X-API-Key on read operations. Optional: when
+                      unset, list/template log in as admin and fetch the
+                      'default' API key over HTTPS via /api/admin/api-keys
+    ADMIN_TOKEN       Pre-obtained admin JWT; when set, skips the handshake
+    NAMESPACE         Kubernetes namespace, default 'openhands' (bootstrap only)
+
+Exit codes:
+    0  success
+    1  precondition failure (missing env vars, no matching config, etc.)
+    2  HTTP or connection error against the runtime-api, or argparse usage
+       error (argparse's own default)
+"""
+
+from __future__ import annotations
+
+import argparse
+import base64
+import binascii
+import hashlib
+import json
+import os
+import shlex
+import shutil
+import subprocess
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+from typing import Any
+
+
+# ---------- HTTP helpers ---------------------------------------------------
+
+
+def _runtime_api_url() -> str:
+    url = os.environ.get("RUNTIME_API_URL", "").rstrip("/")
+    if not url:
+        _fail(
+            "RUNTIME_API_URL is not set. Export it (e.g. "
+            "https://runtime-api.<your-base-domain>) or run `bootstrap`."
+        )
+    return url
+
+
+def _request(
+    path: str,
+    method: str = "GET",
+    body: Any = None,
+    headers: dict[str, str] | None = None,
+) -> Any:
+    api_url = _runtime_api_url()
+    req_headers = {"Content-Type": "application/json", **(headers or {})}
+    req = urllib.request.Request(
+        f"{api_url}{path}", method=method, headers=req_headers
+    )
+    if body is not None:
+        req.data = json.dumps(body).encode()
+    try:
+        with urllib.request.urlopen(req) as resp:
+            payload = resp.read().decode()
+            return json.loads(payload) if payload else {}
+    except urllib.error.HTTPError as exc:
+        _fail(f"HTTP {exc.code} {method} {path}: {exc.read().decode()}", code=2)
+    except urllib.error.URLError as exc:
+        _fail(f"Could not connect to {api_url}: {exc.reason}", code=2)
+
+
+# ---------- Auth -----------------------------------------------------------
+
+
+def _api_key() -> str:
+    key = os.environ.get("API_KEY")
+    if key:
+        return key
+    # No API_KEY exported. If admin creds are available, log in and pull the
+    # 'default' key value over HTTPS - the same value bootstrap would extract
+    # from the k8s secret. Lets an admin drive the CLI end-to-end with just
+    # RUNTIME_API_URL + ADMIN_PASSWORD, no cluster access.
+    if not (os.environ.get("ADMIN_TOKEN") or os.environ.get("ADMIN_PASSWORD")):
+        _fail(
+            "API_KEY is not set, and no ADMIN_PASSWORD (or ADMIN_TOKEN) to "
+            "fall back to. Either export API_KEY, or export ADMIN_PASSWORD "
+            "so the CLI can fetch the default API key over HTTPS."
+        )
+    token = _admin_token()
+    keys = _request(
+        "/api/admin/api-keys",
+        headers={"Authorization": f"Bearer {token}"},
+    )
+    for k in keys or []:
+        if k.get("name") == "default":
+            return k["key_value"]
+    _fail("No API key named 'default' returned by /api/admin/api-keys.")
+
+
+def _admin_token() -> str:
+    """Return a valid admin JWT, using ADMIN_TOKEN if set or logging in."""
+    token = os.environ.get("ADMIN_TOKEN")
+    if token:
+        return token
+    password = os.environ.get("ADMIN_PASSWORD")
+    if not password:
+        _fail("ADMIN_PASSWORD (or ADMIN_TOKEN) is not set.")
+
+    challenge = _request("/api/admin/challenge")
+    derived = hashlib.pbkdf2_hmac(
+        "sha256",
+        password.encode(),
+        (challenge["salt"] + challenge["challenge"]).encode(),
+        challenge["iterations"],
+        dklen=32,
+    )
+    resp = _request(
+        "/api/admin/login",
+        method="POST",
+        body={
+            "challenge": challenge["challenge"],
+            "hash": binascii.hexlify(derived).decode(),
+        },
+    )
+    return resp["token"]
+
+
+# ---------- Subcommand implementations -------------------------------------
+
+
+def cmd_list(args: argparse.Namespace) -> None:
+    resp = _request(
+        "/api/warm-runtime-configs", headers={"X-API-Key": _api_key()}
+    )
+    print(json.dumps(resp["configs"], indent=2))
+
+
+def cmd_template(args: argparse.Namespace) -> None:
+    resp = _request(
+        "/api/warm-runtime-configs", headers={"X-API-Key": _api_key()}
+    )
+    match = next(
+        (c for c in resp["configs"] if c.get("name") == args.source), None
+    )
+    if match is None:
+        _fail(f"No configuration named {args.source!r} in the effective set.")
+
+    # Strip identity fields that must not appear in a save body.
+    for field in ("name", "source"):
+        match.pop(field, None)
+
+    if args.image is not None:
+        match["image"] = args.image
+    if args.count is not None:
+        match["count"] = args.count
+
+    payload = json.dumps(match, indent=2)
+    if args.output and args.output != "-":
+        with open(args.output, "w") as fh:
+            fh.write(payload + "\n")
+    else:
+        print(payload)
+
+
+def cmd_save(args: argparse.Namespace) -> None:
+    if args.file in (None, "-"):
+        raw = sys.stdin.read()
+        if not raw.strip():
+            _fail("No JSON body on stdin. Pass --file, or pipe JSON in.")
+    else:
+        with open(args.file) as fh:
+            raw = fh.read()
+    try:
+        body = json.loads(raw)
+    except json.JSONDecodeError as exc:
+        _fail(f"Invalid JSON: {exc}")
+
+    name = urllib.parse.quote(args.name, safe="")
+    token = _admin_token()
+    saved = _request(
+        f"/api/admin/warm-runtime-configs/{name}",
+        method="PUT",
+        body=body,
+        headers={"Authorization": f"Bearer {token}"},
+    )
+    print(
+        f"Saved {saved['name']} image={saved['image']} count={saved.get('count')}"
+    )
+
+
+def cmd_delete(args: argparse.Namespace) -> None:
+    name = urllib.parse.quote(args.name, safe="")
+    token = _admin_token()
+    resp = _request(
+        f"/api/admin/warm-runtime-configs/{name}",
+        method="DELETE",
+        headers={"Authorization": f"Bearer {token}"},
+    )
+    print(resp.get("message", f"Deleted {args.name}"))
+
+
+def cmd_bootstrap(args: argparse.Namespace) -> None:
+    if shutil.which("kubectl") is None:
+        _fail("kubectl not found on PATH. bootstrap needs cluster access.")
+
+    ns = args.namespace
+    lines: list[str] = []
+
+    if not args.skip_url:
+        host = _kubectl(
+            [
+                "get", "ingress", "-n", ns,
+                "-l", "app.kubernetes.io/name=runtime-api",
+                "-o", "jsonpath={.items[0].spec.rules[0].host}",
+            ],
+            allow_empty=True,
+        )
+        if host:
+            lines.append(f"export RUNTIME_API_URL={shlex.quote(f'https://{host}')}")
+        else:
+            print(
+                "# No runtime-api ingress found; export RUNTIME_API_URL manually",
+                file=sys.stderr,
+            )
+
+    api_key = _decode_secret(ns, "default-api-key", "default-api-key")
+    admin_password = _decode_secret(ns, "admin-password", "admin-password")
+    lines.append(f"export API_KEY={shlex.quote(api_key)}")
+    lines.append(f"export ADMIN_PASSWORD={shlex.quote(admin_password)}")
+    print("\n".join(lines))
+
+
+# ---------- kubectl helpers (bootstrap only) -------------------------------
+
+
+def _kubectl(argv: list[str], *, allow_empty: bool = False) -> str:
+    try:
+        out = subprocess.check_output(["kubectl", *argv], stderr=subprocess.PIPE)
+    except subprocess.CalledProcessError as exc:
+        _fail(
+            f"kubectl {' '.join(argv)} failed:\n{exc.stderr.decode()}",
+            code=2,
+        )
+    text = out.decode().strip()
+    if not text and not allow_empty:
+        _fail(f"kubectl {' '.join(argv)} returned no output.", code=2)
+    return text
+
+
+def _decode_secret(namespace: str, secret: str, key: str) -> str:
+    encoded = _kubectl(
+        [
+            "get", "secret", secret, "-n", namespace,
+            "-o", f"jsonpath={{.data.{key}}}",
+        ]
+    )
+    try:
+        return base64.b64decode(encoded).decode()
+    except (binascii.Error, ValueError) as exc:
+        _fail(f"Could not decode secret {secret}/{key}: {exc}", code=2)
+
+
+# ---------- Utility --------------------------------------------------------
+
+
+def _fail(msg: str, *, code: int = 1) -> None:
+    print(msg, file=sys.stderr)
+    sys.exit(code)
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(
+        prog="warm_runtime_configs",
+        description="Manage warm sandbox configurations on OpenHands Enterprise.",
+    )
+    sub = p.add_subparsers(dest="command", required=True)
+
+    sub.add_parser("list", help="List effective warm runtime configurations.").set_defaults(func=cmd_list)
+
+    t = sub.add_parser(
+        "template",
+        help="Fetch a config, strip name/source, override image/count, print JSON.",
+    )
+    t.add_argument("source", help="Name of an existing config to derive from (e.g. v1_current).")
+    t.add_argument("--image", help="Override the image field.")
+    t.add_argument("--count", type=int, help="Override the count field.")
+    t.add_argument("-o", "--output", help="Write to file instead of stdout ('-' = stdout).")
+    t.set_defaults(func=cmd_template)
+
+    s = sub.add_parser("save", help="Create or update a configuration (upsert).")
+    s.add_argument("name", help="Configuration name (URL identity).")
+    s.add_argument(
+        "--file", "-f",
+        help="Path to JSON body. Use '-' or omit to read from stdin.",
+    )
+    s.set_defaults(func=cmd_save)
+
+    d = sub.add_parser("delete", help="Delete a database-managed configuration.")
+    d.add_argument("name", help="Configuration name to delete.")
+    d.set_defaults(func=cmd_delete)
+
+    b = sub.add_parser(
+        "bootstrap",
+        help="Print `export` lines for RUNTIME_API_URL, API_KEY, ADMIN_PASSWORD.",
+    )
+    b.add_argument(
+        "--namespace", "-n",
+        default=os.environ.get("NAMESPACE", "openhands"),
+        help="Kubernetes namespace (default: %(default)s).",
+    )
+    b.add_argument(
+        "--skip-url", action="store_true",
+        help="Do not emit RUNTIME_API_URL (useful with `kubectl port-forward`).",
+    )
+    b.set_defaults(func=cmd_bootstrap)
+
+    return p
+
+
+def main(argv: list[str] | None = None) -> None:
+    args = build_parser().parse_args(argv)
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()