cassis-cli 0.2.0__tar.gz → 0.3.0__tar.gz

{cassis_cli-0.2.0 → cassis_cli-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cassis-cli
-Version: 0.2.0
+Version: 0.3.0
 Summary: Cassis CLI — run Cassis actions (ontology validation, upload and publish) from your CI pipelines
 License: Proprietary
 Keywords: cassis,ontology,ci,text-to-sql
@@ -24,6 +24,8 @@ Run Cassis actions from your CI pipelines:
 
 - `cassis ontology check` validates the ontology files in your repository with the exact same checks as the Cassis GitHub PR check (YAML parsing, round-trip, import validation) — so you can gate merges in any CI system, not just GitHub.
 - `cassis ontology upload` uploads the ontology files to a Cassis project (full replace) and, by default, publishes them immediately as a new version — so a merge to your main branch can go live in one CI step.
+- `cassis ontology pull` downloads the project's unpublished ontology into your repository checkout (full sync — stale local YAML files are pruned), so you can start editing from the current state, or bootstrap a repo that isn't git-synced (e.g. Bitbucket).
+- `cassis eval run` runs the project's eval suite against your local ontology files (scored in-memory — nothing is pushed to Cassis) and prints per-question results, so you can test the changes on your git branch before merging.
 
 ## Install
 
@@ -35,7 +37,7 @@ pip install cassis-cli
 
 1. Create an API key in Cassis under **Organization settings → API keys** (keys start with `sk-k6-`).
 2. Store it as a CI secret and expose it as `CASSIS_API_KEY`.
-3. For `upload`: find the project ID (UUID) in the project's URL and expose it as `CASSIS_PROJECT_ID` (or pass `--project`).
+3. For `pull`, `upload` and `eval run`: find the project ID (UUID) in the project's URL and expose it as `CASSIS_PROJECT_ID` (or pass `--project`).
 
 ## Usage
 
@@ -46,6 +48,10 @@ cassis ontology check
 # Or point at the checkout explicitly:
 cassis ontology check /path/to/checkout
 
+# Download the project's unpublished ontology into the checkout (full sync;
+# review with git diff — pass --no-prune to keep local files it would delete):
+cassis ontology pull --project 019f0000-0000-7000-8000-000000000000
+
 # Upload the ontology to a project and publish it immediately:
 cassis ontology upload --project 019f0000-0000-7000-8000-000000000000
 
@@ -57,7 +63,19 @@ cassis ontology upload --project ... --label "release 1.2"
 
 # Machine-readable output:
 cassis ontology check --json
+cassis ontology pull --project ... --json
 cassis ontology upload --project ... --json
+cassis eval run --project ... --json
+
+# Run the eval suite against the local ontology files and wait for results
+# (the run is labelled with your git branch name in the Evals page):
+cassis eval run --project ...
+
+# Run against an existing Cassis ontology branch, or the unpublished ontology:
+cassis eval run --project ... --branch feature-x
+
+# Start the run and return immediately (poll in the webapp):
+cassis eval run --project ... --no-wait
 ```
 
 Configuration (flags take precedence over env vars):
@@ -67,20 +85,29 @@ Configuration (flags take precedence over env vars):
 | `--api-key` | `CASSIS_API_KEY` | — (required)                |
 | `--api-url` | `CASSIS_API_URL` | `https://app.getcassis.com` |
 | `--base-path` | `CASSIS_BASE_PATH` | `cassis` — must match the project's git-sync "Path" setting |
-| `--project` (upload only) | `CASSIS_PROJECT_ID` | — (required)      |
+| `--project` (pull, upload, eval run) | `CASSIS_PROJECT_ID` | — (required)      |
+
+`cassis eval run` also accepts `--label` (run label in the Evals page; defaults
+to the branch name from the CI environment or the local git checkout; rejected
+with `--branch`, whose runs are labelled with the branch name), `--wait/--no-wait`, `--poll-interval` (5 s),
+`--timeout` (30 min — the run keeps going server-side if the CLI stops waiting),
+and Ctrl-C cancels the run (exit 130). It prints a deep link to the run's page
+in the Evals UI; `--app-url` / `CASSIS_APP_URL` overrides the link's base URL
+when the webapp is not served from the API host (defaults to `--api-url`).
 
 ### Exit codes
 
 | Code | Meaning                                                                        |
 | ---- | ------------------------------------------------------------------------------ |
-| 0    | Ontology is valid (check) / uploaded (upload)                                  |
-| 1    | Validation failed (check: findings printed; upload: nothing imported)          |
+| 0    | Ontology is valid (check) / pulled (pull) / uploaded (upload) / eval run completed all-passed (eval run) |
+| 1    | Validation failed (check: findings printed; upload: nothing imported; eval run: invalid tree, failed cases, or failed/cancelled run) |
 | 2    | Usage error (missing API key or project, no ontology directory, unreadable file, tree over the size limits) |
-| 3    | Transport/API error (unreachable API, invalid key, inaccessible project, unexpected response) |
+| 3    | Transport/API error (unreachable API, invalid key, inaccessible project, unexpected response), another run already active, or `--timeout` reached |
 
-Both commands accept up to 2000 YAML files / 5 MB total — far above real
-ontologies (a few hundred small files). Beyond that the CLI fails fast with
-exit 2 before uploading anything; double-check `--base-path` if you hit it.
+Commands that send the local tree (`check`, `upload`, `eval run`) accept up to
+2000 YAML files / 5 MB total — far above real ontologies (a few hundred small
+files). Beyond that the CLI fails fast with exit 2 before uploading anything;
+double-check `--base-path` if you hit it.
 
 `upload` replaces the project's entire ontology with the uploaded tree. A
 never-published project always goes live immediately on first upload (even
@@ -105,6 +132,20 @@ jobs:
         env:
           CASSIS_API_KEY: ${{ secrets.CASSIS_API_KEY }}
 
+  ontology-eval:
+    runs-on: ubuntu-latest
+    if: github.event_name == 'pull_request'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install cassis-cli
+      - run: cassis eval run
+        env:
+          CASSIS_API_KEY: ${{ secrets.CASSIS_API_KEY }}
+          CASSIS_PROJECT_ID: ${{ vars.CASSIS_PROJECT_ID }}
+
   ontology-publish:
     runs-on: ubuntu-latest
     if: github.ref == 'refs/heads/main'
@@ -131,6 +172,17 @@ ontology-check:
   variables:
     CASSIS_API_KEY: $CASSIS_API_KEY
 
+ontology-eval:
+  image: python:3.12-slim
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+  script:
+    - pip install cassis-cli
+    - cassis eval run
+  variables:
+    CASSIS_API_KEY: $CASSIS_API_KEY
+    CASSIS_PROJECT_ID: $CASSIS_PROJECT_ID
+
 ontology-publish:
   image: python:3.12-slim
   rules:

{cassis_cli-0.2.0 → cassis_cli-0.3.0}/README.md

@@ -4,6 +4,8 @@ Run Cassis actions from your CI pipelines:
 
 - `cassis ontology check` validates the ontology files in your repository with the exact same checks as the Cassis GitHub PR check (YAML parsing, round-trip, import validation) — so you can gate merges in any CI system, not just GitHub.
 - `cassis ontology upload` uploads the ontology files to a Cassis project (full replace) and, by default, publishes them immediately as a new version — so a merge to your main branch can go live in one CI step.
+- `cassis ontology pull` downloads the project's unpublished ontology into your repository checkout (full sync — stale local YAML files are pruned), so you can start editing from the current state, or bootstrap a repo that isn't git-synced (e.g. Bitbucket).
+- `cassis eval run` runs the project's eval suite against your local ontology files (scored in-memory — nothing is pushed to Cassis) and prints per-question results, so you can test the changes on your git branch before merging.
 
 ## Install
 
@@ -15,7 +17,7 @@ pip install cassis-cli
 
 1. Create an API key in Cassis under **Organization settings → API keys** (keys start with `sk-k6-`).
 2. Store it as a CI secret and expose it as `CASSIS_API_KEY`.
-3. For `upload`: find the project ID (UUID) in the project's URL and expose it as `CASSIS_PROJECT_ID` (or pass `--project`).
+3. For `pull`, `upload` and `eval run`: find the project ID (UUID) in the project's URL and expose it as `CASSIS_PROJECT_ID` (or pass `--project`).
 
 ## Usage
 
@@ -26,6 +28,10 @@ cassis ontology check
 # Or point at the checkout explicitly:
 cassis ontology check /path/to/checkout
 
+# Download the project's unpublished ontology into the checkout (full sync;
+# review with git diff — pass --no-prune to keep local files it would delete):
+cassis ontology pull --project 019f0000-0000-7000-8000-000000000000
+
 # Upload the ontology to a project and publish it immediately:
 cassis ontology upload --project 019f0000-0000-7000-8000-000000000000
 
@@ -37,7 +43,19 @@ cassis ontology upload --project ... --label "release 1.2"
 
 # Machine-readable output:
 cassis ontology check --json
+cassis ontology pull --project ... --json
 cassis ontology upload --project ... --json
+cassis eval run --project ... --json
+
+# Run the eval suite against the local ontology files and wait for results
+# (the run is labelled with your git branch name in the Evals page):
+cassis eval run --project ...
+
+# Run against an existing Cassis ontology branch, or the unpublished ontology:
+cassis eval run --project ... --branch feature-x
+
+# Start the run and return immediately (poll in the webapp):
+cassis eval run --project ... --no-wait
 ```
 
 Configuration (flags take precedence over env vars):
@@ -47,20 +65,29 @@ Configuration (flags take precedence over env vars):
 | `--api-key` | `CASSIS_API_KEY` | — (required)                |
 | `--api-url` | `CASSIS_API_URL` | `https://app.getcassis.com` |
 | `--base-path` | `CASSIS_BASE_PATH` | `cassis` — must match the project's git-sync "Path" setting |
-| `--project` (upload only) | `CASSIS_PROJECT_ID` | — (required)      |
+| `--project` (pull, upload, eval run) | `CASSIS_PROJECT_ID` | — (required)      |
+
+`cassis eval run` also accepts `--label` (run label in the Evals page; defaults
+to the branch name from the CI environment or the local git checkout; rejected
+with `--branch`, whose runs are labelled with the branch name), `--wait/--no-wait`, `--poll-interval` (5 s),
+`--timeout` (30 min — the run keeps going server-side if the CLI stops waiting),
+and Ctrl-C cancels the run (exit 130). It prints a deep link to the run's page
+in the Evals UI; `--app-url` / `CASSIS_APP_URL` overrides the link's base URL
+when the webapp is not served from the API host (defaults to `--api-url`).
 
 ### Exit codes
 
 | Code | Meaning                                                                        |
 | ---- | ------------------------------------------------------------------------------ |
-| 0    | Ontology is valid (check) / uploaded (upload)                                  |
-| 1    | Validation failed (check: findings printed; upload: nothing imported)          |
+| 0    | Ontology is valid (check) / pulled (pull) / uploaded (upload) / eval run completed all-passed (eval run) |
+| 1    | Validation failed (check: findings printed; upload: nothing imported; eval run: invalid tree, failed cases, or failed/cancelled run) |
 | 2    | Usage error (missing API key or project, no ontology directory, unreadable file, tree over the size limits) |
-| 3    | Transport/API error (unreachable API, invalid key, inaccessible project, unexpected response) |
+| 3    | Transport/API error (unreachable API, invalid key, inaccessible project, unexpected response), another run already active, or `--timeout` reached |
 
-Both commands accept up to 2000 YAML files / 5 MB total — far above real
-ontologies (a few hundred small files). Beyond that the CLI fails fast with
-exit 2 before uploading anything; double-check `--base-path` if you hit it.
+Commands that send the local tree (`check`, `upload`, `eval run`) accept up to
+2000 YAML files / 5 MB total — far above real ontologies (a few hundred small
+files). Beyond that the CLI fails fast with exit 2 before uploading anything;
+double-check `--base-path` if you hit it.
 
 `upload` replaces the project's entire ontology with the uploaded tree. A
 never-published project always goes live immediately on first upload (even
@@ -85,6 +112,20 @@ jobs:
         env:
           CASSIS_API_KEY: ${{ secrets.CASSIS_API_KEY }}
 
+  ontology-eval:
+    runs-on: ubuntu-latest
+    if: github.event_name == 'pull_request'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install cassis-cli
+      - run: cassis eval run
+        env:
+          CASSIS_API_KEY: ${{ secrets.CASSIS_API_KEY }}
+          CASSIS_PROJECT_ID: ${{ vars.CASSIS_PROJECT_ID }}
+
   ontology-publish:
     runs-on: ubuntu-latest
     if: github.ref == 'refs/heads/main'
@@ -111,6 +152,17 @@ ontology-check:
   variables:
     CASSIS_API_KEY: $CASSIS_API_KEY
 
+ontology-eval:
+  image: python:3.12-slim
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+  script:
+    - pip install cassis-cli
+    - cassis eval run
+  variables:
+    CASSIS_API_KEY: $CASSIS_API_KEY
+    CASSIS_PROJECT_ID: $CASSIS_PROJECT_ID
+
 ontology-publish:
   image: python:3.12-slim
   rules:

{cassis_cli-0.2.0 → cassis_cli-0.3.0}/cassis_cli/__init__.py

@@ -1,3 +1,3 @@
 """Cassis CLI — run Cassis actions from your CI pipelines."""
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"

cassis_cli-0.3.0/cassis_cli/api.py

@@ -0,0 +1,286 @@
+"""Thin HTTP client for the Cassis API."""
+
+from __future__ import annotations
+
+from typing import Any, Optional
+
+import httpx
+
+DEFAULT_API_URL = "https://app.getcassis.com"
+TIMEOUT_SECONDS = 60.0
+
+
+class ApiError(Exception):
+    """Transport or HTTP-level failure talking to the Cassis API."""
+
+
+class AuthError(ApiError):
+    """The API rejected the API key (401)."""
+
+
+class UploadValidationError(ApiError):
+    """The API rejected the uploaded ontology tree as invalid (400)."""
+
+
+class EvalStartValidationError(ApiError):
+    """The API rejected the eval-run start request (400): invalid tree or no test cases.
+
+    ``detail`` keeps the structured payload (``{"message", "findings"}`` for an
+    invalid tree, or a plain string) for display.
+    """
+
+    def __init__(self, detail: object) -> None:
+        super().__init__(str(detail))
+        self.detail = detail
+
+
+class EvalRunActiveError(ApiError):
+    """Another eval run is already active for the project (409)."""
+
+
+def _project_scope_error(response: httpx.Response) -> ApiError:
+    # Surface the server's own message when it names the missing resource
+    # (e.g. "Branch 'x' not found") — the generic hint covers the rest.
+    try:
+        detail = response.json().get("detail")
+    except ValueError:
+        detail = None
+    prefix = f"{detail} " if isinstance(detail, str) and detail else ""
+    return ApiError(
+        f"{prefix}(HTTP {response.status_code}). Check --project and that "
+        "the API key belongs to the project's organization and can edit it."
+    )
+
+
+def _parse_json_response(response: httpx.Response, url: str) -> Any:
+    try:
+        return response.json()
+    except ValueError as exc:  # json.JSONDecodeError — e.g. a proxy or portal answering HTML with a 200
+        raise ApiError(
+            f"The Cassis API at {url} returned a non-JSON response — check the API URL and any proxy in between."
+        ) from exc
+
+
+def post_ontology_check(
+    *,
+    api_url: str,
+    api_key: str,
+    files: dict[str, str],
+    transport: Optional[httpx.BaseTransport] = None,
+) -> dict[str, Any]:
+    """POST the ontology tree to /api/ci/ontology-check and return the response body."""
+    url = api_url.rstrip("/") + "/api/ci/ontology-check"
+    try:
+        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
+            response = client.post(
+                url,
+                json={"files": files},
+                headers={"Authorization": f"Bearer {api_key}"},
+            )
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
+    result = _parse_json_response(response, url)
+    if (
+        not isinstance(result, dict)
+        or not all(key in result for key in ("passed", "title", "summary"))
+        or not isinstance(result.get("findings"), list)
+    ):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result
+
+
+def post_ontology_import(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    files: dict[str, str],
+    publish: bool,
+    label: Optional[str] = None,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> dict[str, Any]:
+    """POST the ontology tree to /api/ci/projects/{project_id}/ontology/import and return the response body."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/ontology/import"
+    body: dict[str, Any] = {"files": files, "publish": publish}
+    if label is not None:
+        body["label"] = label
+    try:
+        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
+            response = client.post(
+                url,
+                json=body,
+                headers={"Authorization": f"Bearer {api_key}"},
+            )
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    if response.status_code == 400:
+        try:
+            detail = response.json().get("detail") or response.text[:500]
+        except ValueError:
+            detail = response.text[:500]
+        raise UploadValidationError(str(detail))
+    if response.status_code in (403, 404):
+        raise _project_scope_error(response)
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
+    result = _parse_json_response(response, url)
+    if not isinstance(result, dict) or not all(
+        key in result for key in ("domain_count", "table_count", "join_count", "metric_count", "published_version")
+    ):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result
+
+
+def get_ontology_export(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> dict[str, str]:
+    """GET /api/ci/projects/{project_id}/ontology/export and return the files tree."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/ontology/export"
+    try:
+        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
+            response = client.get(url, headers={"Authorization": f"Bearer {api_key}"})
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    if response.status_code in (403, 404):
+        raise _project_scope_error(response)
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
+    result = _parse_json_response(response, url)
+    if not isinstance(result, dict) or not isinstance(result.get("files"), dict):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result["files"]
+
+
+def post_eval_run_start(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    files: Optional[dict[str, str]] = None,
+    branch: Optional[str] = None,
+    label: Optional[str] = None,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> dict[str, Any]:
+    """POST to /api/ci/projects/{project_id}/eval/runs and return the run record."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/eval/runs"
+    body: dict[str, Any] = {}
+    if files is not None:
+        body["files"] = files
+    if branch is not None:
+        body["branch"] = branch
+    if label is not None:
+        body["label"] = label
+    try:
+        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
+            response = client.post(url, json=body, headers={"Authorization": f"Bearer {api_key}"})
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    if response.status_code == 400:
+        try:
+            detail = response.json().get("detail") or response.text[:500]
+        except ValueError:
+            detail = response.text[:500]
+        raise EvalStartValidationError(detail)
+    if response.status_code == 409:
+        raise EvalRunActiveError(
+            "An eval run is already active for this project — wait for it to finish or cancel it "
+            "(in the webapp's Evals page, or with the run id printed when it was started)."
+        )
+    if response.status_code in (403, 404):
+        raise _project_scope_error(response)
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
+    result = _parse_json_response(response, url)
+    if not isinstance(result, dict) or not all(key in result for key in ("run_id", "status", "total_cases")):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result
+
+
+def _get_eval_json(url: str, api_key: str, transport: Optional[httpx.BaseTransport]) -> Any:
+    """GET an eval-run URL with the shared error mapping."""
+    try:
+        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
+            response = client.get(url, headers={"Authorization": f"Bearer {api_key}"})
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    if response.status_code in (403, 404):
+        raise _project_scope_error(response)
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
+    return _parse_json_response(response, url)
+
+
+def get_eval_run(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    run_id: str,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> dict[str, Any]:
+    """GET /api/ci/projects/{project_id}/eval/runs/{run_id} and return the run record."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/eval/runs/{run_id}"
+    result = _get_eval_json(url, api_key, transport)
+    if not isinstance(result, dict) or not all(key in result for key in ("run_id", "status", "total_cases")):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result
+
+
+def get_eval_run_results(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    run_id: str,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> list[dict[str, Any]]:
+    """GET /api/ci/projects/{project_id}/eval/runs/{run_id}/results and return the result list."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/eval/runs/{run_id}/results"
+    result = _get_eval_json(url, api_key, transport)
+    if not isinstance(result, list):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result
+
+
+def post_eval_run_cancel(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    run_id: str,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> None:
+    """POST /api/ci/projects/{project_id}/eval/runs/{run_id}/cancel."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/eval/runs/{run_id}/cancel"
+    try:
+        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
+            response = client.post(url, headers={"Authorization": f"Bearer {api_key}"})
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    if response.status_code in (403, 404):
+        raise _project_scope_error(response)
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")

cassis_cli-0.3.0/cassis_cli/common.py

@@ -0,0 +1,95 @@
+"""Helpers shared by the `cassis` subcommands (tree collection, auth, exit codes)."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Optional
+
+import typer
+
+# Repository directory the ontology tree is exported under. Must match the
+# project's git-sync "Path" setting in Cassis (default "cassis").
+DEFAULT_BASE_PATH = "cassis"
+
+# Exit codes (documented in the README; stable contract for CI scripts).
+EXIT_OK = 0
+EXIT_VALIDATION_FAILED = 1
+EXIT_USAGE = 2
+EXIT_TRANSPORT = 3
+
+# Request ceilings of the /api/ci file-tree endpoints, mirrored so oversized
+# trees fail fast with a clear message before any upload. Source of truth:
+# backend/app/schemas/ci.py (the server's 422 remains the backstop).
+MAX_FILES = 2000
+MAX_TOTAL_BYTES = 5 * 1024 * 1024
+
+
+def collect_files(ontology_dir: Path) -> dict[str, str]:
+    """Read every YAML file under the ontology dir, keyed by posix relpath.
+
+    Exits 2 (usage) on an unreadable or non-UTF-8 file — a local checkout
+    problem, reported before anything is sent to the API.
+    """
+    files: dict[str, str] = {}
+    for pattern in ("**/*.yml", "**/*.yaml"):
+        for file in sorted(ontology_dir.glob(pattern)):
+            if file.is_file():
+                try:
+                    files[file.relative_to(ontology_dir).as_posix()] = file.read_text(encoding="utf-8")
+                except (UnicodeDecodeError, OSError) as exc:
+                    typer.secho(f"Cannot read {file}: {exc}", fg=typer.colors.RED, err=True)
+                    raise typer.Exit(EXIT_USAGE) from exc
+    return files
+
+
+def require_api_key(api_key: Optional[str]) -> str:
+    """Exit 2 (usage) when no API key was provided."""
+    if not api_key:
+        typer.secho(
+            "No API key. Set CASSIS_API_KEY or pass --api-key "
+            "(create one in Cassis under Organization settings -> API keys).",
+            fg=typer.colors.RED,
+            err=True,
+        )
+        raise typer.Exit(EXIT_USAGE)
+    return api_key
+
+
+def collect_tree(path: Path, base_path: str) -> "tuple[dict[str, str], str]":
+    """Resolve the ontology dir under the checkout and read its YAML tree.
+
+    Returns ``(files, normalized_base_path)``. Exits 2 (usage) on an empty or
+    missing dir, or a tree beyond the API request ceilings — all local checkout
+    problems, reported before anything is sent to the API.
+    """
+    base_path = base_path.strip().strip("/")
+    if not base_path:
+        typer.secho("--base-path must not be empty.", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_USAGE)
+
+    ontology_dir = path / Path(base_path)
+    if not ontology_dir.is_dir():
+        typer.secho(
+            f"No {base_path}/ directory found under {path}. "
+            "If the project exports to a custom path, pass it with --base-path (or CASSIS_BASE_PATH).",
+            fg=typer.colors.RED,
+            err=True,
+        )
+        raise typer.Exit(EXIT_USAGE)
+
+    files = collect_files(ontology_dir)
+    if not files:
+        typer.secho(f"No YAML files found under {ontology_dir}.", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_USAGE)
+
+    total_bytes = sum(len(content.encode()) for content in files.values())
+    if len(files) > MAX_FILES or total_bytes > MAX_TOTAL_BYTES:
+        typer.secho(
+            f"Ontology tree too large: {len(files)} files / {total_bytes / (1024 * 1024):.1f} MB "
+            f"(limits: {MAX_FILES} files / {MAX_TOTAL_BYTES // (1024 * 1024)} MB). "
+            "Check that --base-path points at the ontology directory, not a larger tree.",
+            fg=typer.colors.RED,
+            err=True,
+        )
+        raise typer.Exit(EXIT_USAGE)
+    return files, base_path

cassis_cli-0.3.0/cassis_cli/eval.py

@@ -0,0 +1,356 @@
+"""`cassis eval` subcommands."""
+
+from __future__ import annotations
+
+import json
+import os
+import subprocess
+import time
+from pathlib import Path
+from typing import Any, Optional
+from uuid import UUID
+
+import typer
+from cassis_cli.api import (
+    DEFAULT_API_URL,
+    ApiError,
+    AuthError,
+    EvalRunActiveError,
+    EvalStartValidationError,
+    get_eval_run,
+    get_eval_run_results,
+    post_eval_run_cancel,
+    post_eval_run_start,
+)
+from cassis_cli.common import (
+    DEFAULT_BASE_PATH,
+    EXIT_OK,
+    EXIT_TRANSPORT,
+    EXIT_USAGE,
+    EXIT_VALIDATION_FAILED,
+    collect_tree,
+    require_api_key,
+)
+
+app = typer.Typer(no_args_is_help=True, help="Eval commands.")
+
+EXIT_INTERRUPTED = 130
+
+# Result statuses that count as "passed"; everything else is a failure or error.
+_PASSED = "passed"
+_TERMINAL_RUN_STATUSES = {"completed", "failed", "cancelled"}
+_STATUS_COLORS = {
+    "passed": typer.colors.GREEN,
+    "failed": typer.colors.RED,
+    "error": typer.colors.RED,
+    "gold_sql_error": typer.colors.YELLOW,
+    "missing_concept": typer.colors.YELLOW,
+    "plan_unexecutable": typer.colors.YELLOW,
+}
+
+
+def _run_page_url(app_url: str, project_id: str, run_id: str) -> str:
+    """Webapp URL of the run's detail view (Evals page, runs tab)."""
+    return f"{app_url.rstrip('/')}/eval?projectId={project_id}&tab=runs&runId={run_id}"
+
+
+# CI checkouts are usually detached HEAD (`git rev-parse` says "HEAD"), but
+# the CI systems expose the branch through env vars — checked first so PR/MR
+# eval runs get their branch label, the feature's primary use case.
+_CI_BRANCH_ENV_VARS = (
+    "GITHUB_HEAD_REF",  # GitHub Actions, pull_request events
+    "GITHUB_REF_NAME",  # GitHub Actions, branch pushes
+    "CI_MERGE_REQUEST_SOURCE_BRANCH_NAME",  # GitLab CI, merge_request pipelines
+    "CI_COMMIT_REF_NAME",  # GitLab CI, branch pipelines
+    "BITBUCKET_BRANCH",  # Bitbucket Pipelines
+)
+
+
+def _git_branch(path: Path) -> Optional[str]:
+    """Return the checkout's branch name: CI env vars first, then git.
+
+    None outside a repo or on a detached HEAD with no CI env var.
+    """
+    for var in _CI_BRANCH_ENV_VARS:
+        value = os.environ.get(var, "").strip()
+        if value:
+            return value
+    try:
+        proc = subprocess.run(
+            ["git", "rev-parse", "--abbrev-ref", "HEAD"],
+            cwd=path,
+            capture_output=True,
+            text=True,
+            timeout=5,
+        )
+    # Two separate handlers: the repo-wide black targets py314 and would strip
+    # the parens off a tuple form, which is a SyntaxError on the CLI's
+    # supported Python (>=3.10).
+    except OSError:
+        return None
+    except subprocess.TimeoutExpired:
+        return None
+    branch = proc.stdout.strip()
+    if proc.returncode != 0 or not branch or branch == "HEAD":
+        return None
+    return branch
+
+
+def _print_validation_failure(detail: object, base_path: str) -> None:
+    """Print a structured 400 from the start endpoint (invalid tree findings, or a plain message)."""
+    if isinstance(detail, dict) and isinstance(detail.get("findings"), list):
+        typer.secho("Ontology validation failed:", fg=typer.colors.RED, bold=True, err=True)
+        message = detail.get("message")
+        if message:
+            typer.echo(message, err=True)
+        for finding in detail["findings"]:
+            location = f"{base_path}/{finding.get('path')}: " if finding.get("path") else ""
+            typer.echo(f"  {location}{finding.get('message', '')} ({finding.get('stage', '?')})", err=True)
+    else:
+        typer.secho(str(detail), fg=typer.colors.RED, err=True)
+
+
+def _print_results_table(results: list[dict[str, Any]]) -> None:
+    for r in sorted(results, key=lambda r: (r.get("status") == _PASSED, r.get("question") or "")):
+        status = r.get("status", "?")
+        color = _STATUS_COLORS.get(status, typer.colors.WHITE)
+        icon = "✓" if status == _PASSED else "✗"
+        duration = f"{r['duration_seconds']:.0f}s" if r.get("duration_seconds") is not None else "-"
+        question = (r.get("question") or "").replace("\n", " ")
+        if len(question) > 70:
+            question = question[:67] + "..."
+        line = f"  {icon} {status:<17} {duration:>5}  {question}"
+        typer.secho(line, fg=color)
+        if r.get("error"):
+            typer.echo(f"      {str(r['error'])[:200]}")
+
+
+def _print_summary(run: dict[str, Any]) -> None:
+    summary = run.get("summary") or {}
+    total = summary.get("total", run.get("total_cases"))
+    passed = summary.get("passed", 0)
+    accuracy = summary.get("accuracy")
+    parts = [f"{passed}/{total} passed"]
+    if accuracy is not None:
+        parts.append(f"accuracy {accuracy:.0%}")
+    timing = summary.get("timing") or {}
+    if timing.get("p50") is not None:
+        parts.append(f"p50 {timing['p50']:.0f}s")
+    typer.echo(", ".join(parts))
+
+
+@app.command()
+def run(
+    path: Path = typer.Argument(
+        Path("."),
+        help="Repository checkout root (the directory containing the ontology export path).",
+    ),
+    project_id: str = typer.Option(
+        ...,
+        "--project",
+        envvar="CASSIS_PROJECT_ID",
+        help="Target Cassis project ID (UUID, shown in the project's URL).",
+    ),
+    api_key: Optional[str] = typer.Option(
+        None,
+        "--api-key",
+        envvar="CASSIS_API_KEY",
+        help="Cassis API key (sk-k6-...). Create one in Organization settings -> API keys.",
+    ),
+    api_url: str = typer.Option(
+        DEFAULT_API_URL,
+        "--api-url",
+        envvar="CASSIS_API_URL",
+        help="Cassis API base URL.",
+    ),
+    base_path: str = typer.Option(
+        DEFAULT_BASE_PATH,
+        "--base-path",
+        envvar="CASSIS_BASE_PATH",
+        help="Repository directory the ontology is exported under (the project's git-sync Path setting).",
+    ),
+    branch: Optional[str] = typer.Option(
+        None,
+        "--branch",
+        help="Run against an existing Cassis ontology branch by name instead of local files.",
+    ),
+    label: Optional[str] = typer.Option(
+        None,
+        "--label",
+        help="Run label shown in the webapp's Evals page (default: the local git branch name).",
+    ),
+    wait: bool = typer.Option(
+        True,
+        "--wait/--no-wait",
+        help="Wait for the run to finish and print results (default), or just print the run id.",
+    ),
+    poll_interval: float = typer.Option(5.0, "--poll-interval", help="Seconds between status polls."),
+    timeout: float = typer.Option(1800.0, "--timeout", help="Give up waiting after this many seconds."),
+    json_output: bool = typer.Option(False, "--json", help="Print the final run and results as raw JSON."),
+    app_url: Optional[str] = typer.Option(
+        None,
+        "--app-url",
+        envvar="CASSIS_APP_URL",
+        help="Cassis webapp base URL, used for the run-details link (default: the API URL).",
+    ),
+) -> None:
+    """Run the project's eval suite against your local ontology files.
+
+    Uploads the local YAML tree and scores it in-memory — nothing is pushed or
+    persisted in Cassis besides the eval run itself. With --branch, runs against
+    an existing Cassis branch instead (no files are sent). Exits 0 when the run
+    completes with every case passed, 1 on any failed case / failed run /
+    invalid tree, 2 on usage errors, 3 on transport errors or --timeout.
+    """
+    api_key = require_api_key(api_key)
+    try:
+        UUID(project_id)
+    except ValueError:
+        typer.secho(f"--project must be a project ID (UUID), got {project_id!r}.", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_USAGE)
+    if branch is not None and label is not None:
+        typer.secho(
+            "--label cannot be used with --branch: branch runs are labelled with the branch name.",
+            fg=typer.colors.RED,
+            err=True,
+        )
+        raise typer.Exit(EXIT_USAGE)
+
+    files: Optional[dict[str, str]] = None
+    if branch is None:
+        files, base_path = collect_tree(path, base_path)
+        if label is None:
+            label = _git_branch(path)
+
+    try:
+        run_record = post_eval_run_start(
+            api_url=api_url,
+            api_key=api_key,
+            project_id=project_id,
+            files=files,
+            branch=branch,
+            label=label,
+        )
+    except EvalStartValidationError as exc:
+        _print_validation_failure(exc.detail, base_path)
+        raise typer.Exit(EXIT_VALIDATION_FAILED) from exc
+    except ApiError as exc:  # covers AuthError and EvalRunActiveError too
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+
+    run_id = str(run_record["run_id"])
+    total = run_record.get("total_cases", 0)
+    ontology_label = run_record.get("ontology_label") or "unpublished ontology"
+    run_url = _run_page_url(app_url or api_url, project_id, run_id)
+
+    if not wait:
+        if json_output:
+            typer.echo(json.dumps({"run": run_record, "run_url": run_url}, indent=2))
+        else:
+            typer.echo(f"Eval run {run_id} started: {total} cases against {ontology_label!r}.")
+            typer.echo(f"Follow it at: {run_url}")
+        raise typer.Exit(EXIT_OK)
+
+    typer.echo(f"Eval run {run_id} started: {total} cases against {ontology_label!r}.")
+
+    try:
+        final_run, results = _wait_for_run(
+            api_url=api_url,
+            api_key=api_key,
+            project_id=project_id,
+            run_id=run_id,
+            total=total,
+            poll_interval=poll_interval,
+            timeout=timeout,
+        )
+    except KeyboardInterrupt:
+        typer.echo("")
+        typer.echo("Interrupted — cancelling the run...")
+        try:
+            post_eval_run_cancel(api_url=api_url, api_key=api_key, project_id=project_id, run_id=run_id)
+            typer.echo("Run cancelled.")
+        except ApiError as exc:
+            typer.secho(f"Could not cancel the run: {exc}", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_INTERRUPTED)
+
+    if json_output:
+        typer.echo(json.dumps({"run": final_run, "results": results, "run_url": run_url}, indent=2))
+    else:
+        typer.echo("")
+        _print_results_table(results)
+        typer.echo("")
+        _print_summary(final_run)
+        typer.echo(f"View details: {run_url}")
+
+    status = final_run.get("status")
+    if status == "completed" and all(r.get("status") == _PASSED for r in results):
+        raise typer.Exit(EXIT_OK)
+    raise typer.Exit(EXIT_VALIDATION_FAILED)
+
+
+# Consecutive poll failures tolerated before giving up: covers transient
+# blips (LB hiccup, brief network loss) without letting a permanently-broken
+# poll (deleted run/project) spin until --timeout.
+_MAX_CONSECUTIVE_POLL_FAILURES = 5
+
+
+def _wait_for_run(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    run_id: str,
+    total: int,
+    poll_interval: float,
+    timeout: float,
+) -> "tuple[dict[str, Any], list[dict[str, Any]]]":
+    """Poll the run until terminal. Returns (run, results).
+
+    Exits 3 directly on timeout, on an auth failure (fail fast — retrying a
+    revoked key can't succeed), or after several consecutive poll failures.
+    The server-side run keeps going in all three cases.
+    """
+    deadline = time.monotonic() + timeout
+    last_line = ""
+    failures = 0
+    while time.monotonic() < deadline:
+        try:
+            run_record = get_eval_run(api_url=api_url, api_key=api_key, project_id=project_id, run_id=run_id)
+            results = get_eval_run_results(api_url=api_url, api_key=api_key, project_id=project_id, run_id=run_id)
+        except AuthError as exc:
+            typer.secho(f"{exc} The run keeps going server-side.", fg=typer.colors.RED, err=True)
+            raise typer.Exit(EXIT_TRANSPORT) from exc
+        except ApiError as exc:
+            failures += 1
+            if failures >= _MAX_CONSECUTIVE_POLL_FAILURES:
+                typer.secho(
+                    f"Polling failed {failures} times in a row ({exc}). "
+                    "Giving up — the run keeps going server-side; see the webapp's Evals page.",
+                    fg=typer.colors.RED,
+                    err=True,
+                )
+                raise typer.Exit(EXIT_TRANSPORT) from exc
+            # A transient poll failure shouldn't kill a multi-minute run; keep waiting.
+            typer.secho(f"(poll failed, retrying: {exc})", fg=typer.colors.YELLOW, err=True)
+            time.sleep(poll_interval)
+            continue
+        failures = 0
+
+        done = len(results)
+        passed = sum(1 for r in results if r.get("status") == _PASSED)
+        line = f"{done}/{total} cases done — {passed} ✓ {done - passed} ✗"
+        if line != last_line:
+            typer.echo(line)
+            last_line = line
+
+        if run_record.get("status") in _TERMINAL_RUN_STATUSES:
+            return run_record, results
+        time.sleep(poll_interval)
+
+    typer.secho(
+        f"Timed out after {timeout:.0f}s waiting for run {run_id}. "
+        "The run keeps going server-side — see the webapp's Evals page.",
+        fg=typer.colors.YELLOW,
+        err=True,
+    )
+    raise typer.Exit(EXIT_TRANSPORT)

{cassis_cli-0.2.0 → cassis_cli-0.3.0}/cassis_cli/main.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import typer
 from cassis_cli import __version__
+from cassis_cli.eval import app as eval_app
 from cassis_cli.ontology import app as ontology_app
 
 app = typer.Typer(
@@ -11,6 +12,7 @@ app = typer.Typer(
     help="Cassis CLI — run Cassis actions from your CI pipelines.",
 )
 app.add_typer(ontology_app, name="ontology")
+app.add_typer(eval_app, name="eval")
 
 
 @app.command()

{cassis_cli-0.2.0 → cassis_cli-0.3.0}/cassis_cli/ontology.py

@@ -13,99 +13,23 @@ from cassis_cli.api import (
     ApiError,
     AuthError,
     UploadValidationError,
+    get_ontology_export,
     post_ontology_check,
     post_ontology_import,
 )
+from cassis_cli.common import (
+    DEFAULT_BASE_PATH,
+    EXIT_OK,
+    EXIT_TRANSPORT,
+    EXIT_USAGE,
+    EXIT_VALIDATION_FAILED,
+)
+from cassis_cli.common import collect_files as _collect_files
+from cassis_cli.common import collect_tree as _collect_tree
+from cassis_cli.common import require_api_key as _require_api_key
 
 app = typer.Typer(no_args_is_help=True, help="Ontology commands.")
 
-# Repository directory the ontology tree is exported under. Must match the
-# project's git-sync "Path" setting in Cassis (default "cassis").
-DEFAULT_BASE_PATH = "cassis"
-
-# Exit codes (documented in the README; stable contract for CI scripts).
-EXIT_OK = 0
-EXIT_VALIDATION_FAILED = 1
-EXIT_USAGE = 2
-EXIT_TRANSPORT = 3
-
-# Request ceilings of POST /api/ci/ontology-check, mirrored so oversized trees
-# fail fast with a clear message before any upload. Source of truth:
-# backend/app/schemas/ci.py (the server's 422 remains the backstop).
-MAX_FILES = 2000
-MAX_TOTAL_BYTES = 5 * 1024 * 1024
-
-
-def _collect_files(ontology_dir: Path) -> dict[str, str]:
-    """Read every YAML file under the ontology dir, keyed by posix relpath.
-
-    Exits 2 (usage) on an unreadable or non-UTF-8 file — a local checkout
-    problem, reported before anything is sent to the API.
-    """
-    files: dict[str, str] = {}
-    for pattern in ("**/*.yml", "**/*.yaml"):
-        for file in sorted(ontology_dir.glob(pattern)):
-            if file.is_file():
-                try:
-                    files[file.relative_to(ontology_dir).as_posix()] = file.read_text(encoding="utf-8")
-                except (UnicodeDecodeError, OSError) as exc:
-                    typer.secho(f"Cannot read {file}: {exc}", fg=typer.colors.RED, err=True)
-                    raise typer.Exit(EXIT_USAGE) from exc
-    return files
-
-
-def _require_api_key(api_key: Optional[str]) -> str:
-    """Exit 2 (usage) when no API key was provided."""
-    if not api_key:
-        typer.secho(
-            "No API key. Set CASSIS_API_KEY or pass --api-key "
-            "(create one in Cassis under Organization settings -> API keys).",
-            fg=typer.colors.RED,
-            err=True,
-        )
-        raise typer.Exit(EXIT_USAGE)
-    return api_key
-
-
-def _collect_tree(path: Path, base_path: str) -> "tuple[dict[str, str], str]":
-    """Resolve the ontology dir under the checkout and read its YAML tree.
-
-    Returns ``(files, normalized_base_path)``. Exits 2 (usage) on an empty or
-    missing dir, or a tree beyond the API request ceilings — all local checkout
-    problems, reported before anything is sent to the API.
-    """
-    base_path = base_path.strip().strip("/")
-    if not base_path:
-        typer.secho("--base-path must not be empty.", fg=typer.colors.RED, err=True)
-        raise typer.Exit(EXIT_USAGE)
-
-    ontology_dir = path / Path(base_path)
-    if not ontology_dir.is_dir():
-        typer.secho(
-            f"No {base_path}/ directory found under {path}. "
-            "If the project exports to a custom path, pass it with --base-path (or CASSIS_BASE_PATH).",
-            fg=typer.colors.RED,
-            err=True,
-        )
-        raise typer.Exit(EXIT_USAGE)
-
-    files = _collect_files(ontology_dir)
-    if not files:
-        typer.secho(f"No YAML files found under {ontology_dir}.", fg=typer.colors.RED, err=True)
-        raise typer.Exit(EXIT_USAGE)
-
-    total_bytes = sum(len(content.encode()) for content in files.values())
-    if len(files) > MAX_FILES or total_bytes > MAX_TOTAL_BYTES:
-        typer.secho(
-            f"Ontology tree too large: {len(files)} files / {total_bytes / (1024 * 1024):.1f} MB "
-            f"(limits: {MAX_FILES} files / {MAX_TOTAL_BYTES // (1024 * 1024)} MB). "
-            "Check that --base-path points at the ontology directory, not a larger tree.",
-            fg=typer.colors.RED,
-            err=True,
-        )
-        raise typer.Exit(EXIT_USAGE)
-    return files, base_path
-
 
 @app.command()
 def check(
@@ -165,6 +89,109 @@ def check(
     raise typer.Exit(EXIT_OK if result["passed"] else EXIT_VALIDATION_FAILED)
 
 
+@app.command()
+def pull(
+    path: Path = typer.Argument(
+        Path("."),
+        help="Repository checkout root (the directory containing the ontology export path).",
+    ),
+    project_id: str = typer.Option(
+        ...,
+        "--project",
+        envvar="CASSIS_PROJECT_ID",
+        help="Source Cassis project ID (UUID, shown in the project's URL).",
+    ),
+    api_key: Optional[str] = typer.Option(
+        None,
+        "--api-key",
+        envvar="CASSIS_API_KEY",
+        help="Cassis API key (sk-k6-...). Create one in Organization settings -> API keys.",
+    ),
+    api_url: str = typer.Option(
+        DEFAULT_API_URL,
+        "--api-url",
+        envvar="CASSIS_API_URL",
+        help="Cassis API base URL.",
+    ),
+    base_path: str = typer.Option(
+        DEFAULT_BASE_PATH,
+        "--base-path",
+        envvar="CASSIS_BASE_PATH",
+        help="Repository directory the ontology is exported under (the project's git-sync Path setting).",
+    ),
+    prune: bool = typer.Option(
+        True,
+        "--prune/--no-prune",
+        help="Delete local YAML files that no longer exist in the project's ontology (default: prune).",
+    ),
+    json_output: bool = typer.Option(False, "--json", help="Print a JSON summary of written/deleted files."),
+) -> None:
+    """Download the project's unpublished ontology into a repository checkout.
+
+    Writes the ontology YAML tree under the export path (full sync: files are
+    overwritten and, unless --no-prune, stale local YAML files are deleted, so
+    the checkout ends up matching the project exactly). Review the changes with
+    git diff before committing. Exits 0 on success, 2 on usage errors, 3 on
+    transport/API errors.
+    """
+    api_key = _require_api_key(api_key)
+    try:
+        UUID(project_id)
+    except ValueError:
+        typer.secho(f"--project must be a project ID (UUID), got {project_id!r}.", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_USAGE)
+    base_path = base_path.strip().strip("/")
+    if not base_path:
+        typer.secho("--base-path must not be empty.", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_USAGE)
+
+    try:
+        files = get_ontology_export(api_url=api_url, api_key=api_key, project_id=project_id)
+    except AuthError as exc:
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+    except ApiError as exc:
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+
+    ontology_dir = (path / Path(base_path)).resolve()
+    written: list[str] = []
+    for rel, content in sorted(files.items()):
+        dest = (ontology_dir / rel).resolve()
+        # The server controls these paths; refuse anything escaping the
+        # ontology dir rather than trusting it blindly.
+        if not dest.is_relative_to(ontology_dir):
+            typer.secho(f"Refusing to write outside {ontology_dir}: {rel!r}", fg=typer.colors.RED, err=True)
+            raise typer.Exit(EXIT_TRANSPORT)
+        try:
+            dest.parent.mkdir(parents=True, exist_ok=True)
+            dest.write_text(content, encoding="utf-8")
+        except OSError as exc:  # local checkout problem (permissions, dir/file collision) — usage, not validation
+            typer.secho(f"Cannot write {dest}: {exc}", fg=typer.colors.RED, err=True)
+            raise typer.Exit(EXIT_USAGE) from exc
+        written.append(rel)
+
+    deleted: list[str] = []
+    if prune and ontology_dir.is_dir():
+        local = _collect_files(ontology_dir)
+        for rel in sorted(set(local) - set(files)):
+            try:
+                (ontology_dir / rel).unlink()
+            except OSError as exc:
+                typer.secho(f"Cannot delete {ontology_dir / rel}: {exc}", fg=typer.colors.RED, err=True)
+                raise typer.Exit(EXIT_USAGE) from exc
+            deleted.append(rel)
+
+    if json_output:
+        typer.echo(json.dumps({"written": written, "deleted": deleted}, indent=2))
+    else:
+        summary = f"✓ Pulled {len(written)} files into {ontology_dir}"
+        if deleted:
+            summary += f" ({len(deleted)} stale files deleted)"
+        typer.secho(f"{summary}.", fg=typer.colors.GREEN)
+    raise typer.Exit(EXIT_OK)
+
+
 @app.command()
 def upload(
     path: Path = typer.Argument(

{cassis_cli-0.2.0 → cassis_cli-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "cassis-cli"
-version = "0.2.0"
+version = "0.3.0"
 description = "Cassis CLI — run Cassis actions (ontology validation, upload and publish) from your CI pipelines"
 readme = "README.md"
 license = { text = "Proprietary" }

cassis_cli-0.2.0/cassis_cli/api.py

@@ -1,113 +0,0 @@
-"""Thin HTTP client for the Cassis API."""
-
-from __future__ import annotations
-
-from typing import Any, Optional
-
-import httpx
-
-DEFAULT_API_URL = "https://app.getcassis.com"
-TIMEOUT_SECONDS = 60.0
-
-
-class ApiError(Exception):
-    """Transport or HTTP-level failure talking to the Cassis API."""
-
-
-class AuthError(ApiError):
-    """The API rejected the API key (401)."""
-
-
-class UploadValidationError(ApiError):
-    """The API rejected the uploaded ontology tree as invalid (400)."""
-
-
-def post_ontology_check(
-    *,
-    api_url: str,
-    api_key: str,
-    files: dict[str, str],
-    transport: Optional[httpx.BaseTransport] = None,
-) -> dict[str, Any]:
-    """POST the ontology tree to /api/ci/ontology-check and return the response body."""
-    url = api_url.rstrip("/") + "/api/ci/ontology-check"
-    try:
-        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
-            response = client.post(
-                url,
-                json={"files": files},
-                headers={"Authorization": f"Bearer {api_key}"},
-            )
-    except httpx.HTTPError as exc:
-        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
-
-    if response.status_code == 401:
-        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
-    if response.status_code >= 400:
-        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
-    try:
-        result = response.json()
-    except ValueError as exc:  # json.JSONDecodeError — e.g. a proxy or portal answering HTML with a 200
-        raise ApiError(
-            f"The Cassis API at {url} returned a non-JSON response — check the API URL and any proxy in between."
-        ) from exc
-    if (
-        not isinstance(result, dict)
-        or not all(key in result for key in ("passed", "title", "summary"))
-        or not isinstance(result.get("findings"), list)
-    ):
-        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
-    return result
-
-
-def post_ontology_import(
-    *,
-    api_url: str,
-    api_key: str,
-    project_id: str,
-    files: dict[str, str],
-    publish: bool,
-    label: Optional[str] = None,
-    transport: Optional[httpx.BaseTransport] = None,
-) -> dict[str, Any]:
-    """POST the ontology tree to /api/ci/projects/{project_id}/ontology/import and return the response body."""
-    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/ontology/import"
-    body: dict[str, Any] = {"files": files, "publish": publish}
-    if label is not None:
-        body["label"] = label
-    try:
-        with httpx.Client(timeout=TIMEOUT_SECONDS, transport=transport) as client:
-            response = client.post(
-                url,
-                json=body,
-                headers={"Authorization": f"Bearer {api_key}"},
-            )
-    except httpx.HTTPError as exc:
-        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
-
-    if response.status_code == 401:
-        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
-    if response.status_code == 400:
-        try:
-            detail = response.json().get("detail") or response.text[:500]
-        except ValueError:
-            detail = response.text[:500]
-        raise UploadValidationError(str(detail))
-    if response.status_code in (403, 404):
-        raise ApiError(
-            f"Project not found or not accessible (HTTP {response.status_code}). Check --project and that "
-            "the API key belongs to the project's organization and can edit it."
-        )
-    if response.status_code >= 400:
-        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
-    try:
-        result = response.json()
-    except ValueError as exc:  # json.JSONDecodeError — e.g. a proxy or portal answering HTML with a 200
-        raise ApiError(
-            f"The Cassis API at {url} returned a non-JSON response — check the API URL and any proxy in between."
-        ) from exc
-    if not isinstance(result, dict) or not all(
-        key in result for key in ("domain_count", "table_count", "join_count", "metric_count", "published_version")
-    ):
-        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
-    return result