cassis-cli 1.1.1__tar.gz → 1.2.0__tar.gz

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cassis-cli
-Version: 1.1.1
+Version: 1.2.0
 Summary: Cassis CLI — run Cassis actions (ontology validation, upload and publish) from your CI pipelines
 License: Apache-2.0
 License-File: LICENSE
@@ -24,7 +24,8 @@ Description-Content-Type: text/markdown
 
 Run Cassis actions from your CI pipelines:
 
-- `cassis ontology check` validates the ontology files in your repository with the exact same checks as the Cassis GitHub PR check (YAML parsing, round-trip, import validation) — so you can gate merges in any CI system, not just GitHub.
+- `cassis ontology check` validates the ontology files in your repository with the exact same checks as the Cassis GitHub PR check (YAML parsing, round-trip, import validation) — so you can gate merges in any CI system, not just GitHub. In a checkout bound to a project (`project.yml`, `--project`, or `CASSIS_PROJECT_ID`), it also cross-checks the tree against the project's source schema: references to tables or columns the warehouse doesn't have print as **warnings** — advisory only (the object may simply not be built or synced yet), never a failed check.
+- `cassis schema pull` downloads the data source's full source schema (as Cassis last introspected it) into `<base-path>/.schema.json` — a **gitignored** local snapshot (the command maintains the ignore entry) with a `pulled_at` stamp. The warehouse stays authoritative; the snapshot is a cache for offline/bulk work — e.g. a coding agent grepping table and column names during a modeling pass instead of paging through the MCP `get_source_schema` tool. Re-run to refresh.
 - `cassis ontology fmt` rewrites the ontology files in canonical form (think `black`/`gofmt` for the ontology), so hand or agent edits pass the round-trip check.
 - `cassis ontology upload` uploads the ontology files to a Cassis project (full replace) and, by default, publishes them immediately as a new version — so a merge to your main branch can go live in one CI step.
 - `cassis ontology pull` downloads the project's unpublished ontology into your repository checkout (full sync — stale local ontology files are pruned), so you can start editing from the current state, or bootstrap a repo that isn't git-synced (e.g. Bitbucket).
@@ -33,6 +34,7 @@ Run Cassis actions from your CI pipelines:
 - `cassis eval run` runs the project's eval suite against your local ontology files (scored in-memory — nothing is pushed to Cassis) and prints per-question results, so you can test the changes on your git branch before merging.
 - `cassis ontology test` runs individual questions through the text-to-SQL agent using your local ontology files, so you can check that a change actually works (e.g. a new column gets picked) — where `eval run` only checks for regressions on existing eval cases.
 - `cassis eval add-case` adds a gold question/SQL case to the project's eval suite — after fixing an ontology issue, add the question users were failing on so `eval run` guards it from regressing.
+- `cassis eval list-cases` and `cassis eval delete-case` maintain the suite: list the current cases with their ids, and prune one that is stale or wrong (e.g. its gold SQL encodes a definition the ontology has since changed).
 
 ## Install
 
@@ -54,7 +56,7 @@ The ontology tree under `<base-path>` (default `cassis/`) is:
 
 1. Create an API key in Cassis under **Organization settings → API keys** (keys start with `sk-k6-`).
 2. Store it as a CI secret and expose it as `CASSIS_API_KEY`.
-3. For `pull`, `upload`, `eval run`, `ontology test`, and `eval add-case`: the project ID (UUID) is taken from `<base-path>/project.yml` in the checkout (written by `pull` and by publishing) — so once a repo is pulled you don't need to pass it. To override, or before the first pull, set `CASSIS_PROJECT_ID` or pass `--project` (find the UUID in the project's URL).
+3. For `pull`, `upload`, `schema pull`, `eval run`, `ontology test`, and the `eval` case commands (`add-case`, `list-cases`, `delete-case`): the project ID (UUID) is taken from `<base-path>/project.yml` in the checkout (written by `pull` and by publishing) — so once a repo is pulled you don't need to pass it. To override, or before the first pull, set `CASSIS_PROJECT_ID` or pass `--project` (find the UUID in the project's URL). `ontology check` uses the same resolution but treats it as optional: unbound checkouts get the project-less validation (no schema reference warnings).
 
 ## Usage
 
@@ -101,6 +103,13 @@ cassis ontology test --project ... -q "How much was refunded last month?" -q "Ne
 # Add a gold case to the eval suite (rejected if the exact question already exists):
 cassis eval add-case --project ... -q "How much was refunded last month?" \
   --gold-sql "SELECT SUM(refunded_cents) / 100.0 FROM public.orders WHERE ..."
+
+# List the suite's cases (id + question; --json adds the gold SQL), then prune one:
+cassis eval list-cases --project ...
+cassis eval delete-case 019f0000-0000-7000-8000-0000000000ca --project ...
+
+# Pull the source schema into <base-path>/.schema.json (gitignored local snapshot):
+cassis schema pull
 ```
 
 Configuration (flags take precedence over env vars):
@@ -110,7 +119,7 @@ Configuration (flags take precedence over env vars):
 | `--api-key` | `CASSIS_API_KEY` | — (required)                |
 | `--api-url` | `CASSIS_API_URL` | `https://app.getcassis.com` |
 | `--base-path` | `CASSIS_BASE_PATH` | `cassis` — must match the project's git-sync "Path" setting |
-| `--project` (pull, upload, eval run, eval add-case, test) | `CASSIS_PROJECT_ID` | — (required)      |
+| `--project` (check, pull, upload, schema pull, eval run, eval add-case, eval list-cases, eval delete-case, test) | `CASSIS_PROJECT_ID` | the id in `<base-path>/project.yml` (required before the first pull; `check` alone falls back to the project-less validation when unbound) |
 
 `cassis eval run` also accepts `--label` (run label in the Evals page; defaults
 to the branch name from the CI environment or the local git checkout; rejected
@@ -139,7 +148,7 @@ cassis ontology fmt --check
 | Code | Meaning                                                                        |
 | ---- | ------------------------------------------------------------------------------ |
 | 0    | Ontology is valid (check) / pulled (pull) / uploaded (upload) / eval run completed all-passed (eval run) / every probe completed (test — whatever its outcome; probes are informational, don't gate CI on them) |
-| 1    | Validation failed (check: findings printed; upload: nothing imported; eval run: invalid tree, failed cases, or failed/cancelled run; test: invalid tree or a probe failed) |
+| 1    | Validation failed (check: findings printed; upload: nothing imported; eval run: invalid tree, failed cases, or failed/cancelled run; test: invalid tree or a probe failed; add-case: duplicate question or gold SQL that does not run; delete-case: no such case in the project) |
 | 2    | Usage error (missing API key or project, no ontology directory, unreadable file, tree over the size limits) |
 | 3    | Transport/API error (unreachable API, invalid key, inaccessible project, unexpected response), another run already active, out of credits, or `--timeout` reached |
 

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/README.md

@@ -2,7 +2,8 @@
 
 Run Cassis actions from your CI pipelines:
 
-- `cassis ontology check` validates the ontology files in your repository with the exact same checks as the Cassis GitHub PR check (YAML parsing, round-trip, import validation) — so you can gate merges in any CI system, not just GitHub.
+- `cassis ontology check` validates the ontology files in your repository with the exact same checks as the Cassis GitHub PR check (YAML parsing, round-trip, import validation) — so you can gate merges in any CI system, not just GitHub. In a checkout bound to a project (`project.yml`, `--project`, or `CASSIS_PROJECT_ID`), it also cross-checks the tree against the project's source schema: references to tables or columns the warehouse doesn't have print as **warnings** — advisory only (the object may simply not be built or synced yet), never a failed check.
+- `cassis schema pull` downloads the data source's full source schema (as Cassis last introspected it) into `<base-path>/.schema.json` — a **gitignored** local snapshot (the command maintains the ignore entry) with a `pulled_at` stamp. The warehouse stays authoritative; the snapshot is a cache for offline/bulk work — e.g. a coding agent grepping table and column names during a modeling pass instead of paging through the MCP `get_source_schema` tool. Re-run to refresh.
 - `cassis ontology fmt` rewrites the ontology files in canonical form (think `black`/`gofmt` for the ontology), so hand or agent edits pass the round-trip check.
 - `cassis ontology upload` uploads the ontology files to a Cassis project (full replace) and, by default, publishes them immediately as a new version — so a merge to your main branch can go live in one CI step.
 - `cassis ontology pull` downloads the project's unpublished ontology into your repository checkout (full sync — stale local ontology files are pruned), so you can start editing from the current state, or bootstrap a repo that isn't git-synced (e.g. Bitbucket).
@@ -11,6 +12,7 @@ Run Cassis actions from your CI pipelines:
 - `cassis eval run` runs the project's eval suite against your local ontology files (scored in-memory — nothing is pushed to Cassis) and prints per-question results, so you can test the changes on your git branch before merging.
 - `cassis ontology test` runs individual questions through the text-to-SQL agent using your local ontology files, so you can check that a change actually works (e.g. a new column gets picked) — where `eval run` only checks for regressions on existing eval cases.
 - `cassis eval add-case` adds a gold question/SQL case to the project's eval suite — after fixing an ontology issue, add the question users were failing on so `eval run` guards it from regressing.
+- `cassis eval list-cases` and `cassis eval delete-case` maintain the suite: list the current cases with their ids, and prune one that is stale or wrong (e.g. its gold SQL encodes a definition the ontology has since changed).
 
 ## Install
 
@@ -32,7 +34,7 @@ The ontology tree under `<base-path>` (default `cassis/`) is:
 
 1. Create an API key in Cassis under **Organization settings → API keys** (keys start with `sk-k6-`).
 2. Store it as a CI secret and expose it as `CASSIS_API_KEY`.
-3. For `pull`, `upload`, `eval run`, `ontology test`, and `eval add-case`: the project ID (UUID) is taken from `<base-path>/project.yml` in the checkout (written by `pull` and by publishing) — so once a repo is pulled you don't need to pass it. To override, or before the first pull, set `CASSIS_PROJECT_ID` or pass `--project` (find the UUID in the project's URL).
+3. For `pull`, `upload`, `schema pull`, `eval run`, `ontology test`, and the `eval` case commands (`add-case`, `list-cases`, `delete-case`): the project ID (UUID) is taken from `<base-path>/project.yml` in the checkout (written by `pull` and by publishing) — so once a repo is pulled you don't need to pass it. To override, or before the first pull, set `CASSIS_PROJECT_ID` or pass `--project` (find the UUID in the project's URL). `ontology check` uses the same resolution but treats it as optional: unbound checkouts get the project-less validation (no schema reference warnings).
 
 ## Usage
 
@@ -79,6 +81,13 @@ cassis ontology test --project ... -q "How much was refunded last month?" -q "Ne
 # Add a gold case to the eval suite (rejected if the exact question already exists):
 cassis eval add-case --project ... -q "How much was refunded last month?" \
   --gold-sql "SELECT SUM(refunded_cents) / 100.0 FROM public.orders WHERE ..."
+
+# List the suite's cases (id + question; --json adds the gold SQL), then prune one:
+cassis eval list-cases --project ...
+cassis eval delete-case 019f0000-0000-7000-8000-0000000000ca --project ...
+
+# Pull the source schema into <base-path>/.schema.json (gitignored local snapshot):
+cassis schema pull
 ```
 
 Configuration (flags take precedence over env vars):
@@ -88,7 +97,7 @@ Configuration (flags take precedence over env vars):
 | `--api-key` | `CASSIS_API_KEY` | — (required)                |
 | `--api-url` | `CASSIS_API_URL` | `https://app.getcassis.com` |
 | `--base-path` | `CASSIS_BASE_PATH` | `cassis` — must match the project's git-sync "Path" setting |
-| `--project` (pull, upload, eval run, eval add-case, test) | `CASSIS_PROJECT_ID` | — (required)      |
+| `--project` (check, pull, upload, schema pull, eval run, eval add-case, eval list-cases, eval delete-case, test) | `CASSIS_PROJECT_ID` | the id in `<base-path>/project.yml` (required before the first pull; `check` alone falls back to the project-less validation when unbound) |
 
 `cassis eval run` also accepts `--label` (run label in the Evals page; defaults
 to the branch name from the CI environment or the local git checkout; rejected
@@ -117,7 +126,7 @@ cassis ontology fmt --check
 | Code | Meaning                                                                        |
 | ---- | ------------------------------------------------------------------------------ |
 | 0    | Ontology is valid (check) / pulled (pull) / uploaded (upload) / eval run completed all-passed (eval run) / every probe completed (test — whatever its outcome; probes are informational, don't gate CI on them) |
-| 1    | Validation failed (check: findings printed; upload: nothing imported; eval run: invalid tree, failed cases, or failed/cancelled run; test: invalid tree or a probe failed) |
+| 1    | Validation failed (check: findings printed; upload: nothing imported; eval run: invalid tree, failed cases, or failed/cancelled run; test: invalid tree or a probe failed; add-case: duplicate question or gold SQL that does not run; delete-case: no such case in the project) |
 | 2    | Usage error (missing API key or project, no ontology directory, unreadable file, tree over the size limits) |
 | 3    | Transport/API error (unreachable API, invalid key, inaccessible project, unexpected response), another run already active, out of credits, or `--timeout` reached |
 

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/__init__.py

@@ -1,3 +1,3 @@
 """Cassis CLI — run Cassis actions from your CI pipelines."""
 
-__version__ = "1.1.1"
+__version__ = "1.2.0"

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/api.py

@@ -134,10 +134,19 @@ def post_ontology_check(
     api_url: str,
     api_key: str,
     files: dict[str, str],
+    project_id: Optional[str] = None,
     transport: Optional[httpx.BaseTransport] = None,
 ) -> dict[str, Any]:
-    """POST the ontology tree to /api/ci/ontology-check and return the response body."""
-    url = api_url.rstrip("/") + "/api/ci/ontology-check"
+    """POST the ontology tree to the check endpoint and return the response body.
+
+    With ``project_id``, calls the project-scoped route, which additionally
+    cross-checks the tree against the project's source schema and returns
+    advisory ``warnings``; without it, the pure tree check.
+    """
+    if project_id:
+        url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/ontology/check"
+    else:
+        url = api_url.rstrip("/") + "/api/ci/ontology-check"
     try:
         with _client(transport=transport) as client:
             response = client.post(
@@ -150,6 +159,8 @@ def post_ontology_check(
 
     if response.status_code == 401:
         raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    if project_id and response.status_code in (403, 404):
+        raise _project_scope_error(response)
     if response.status_code >= 400:
         raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
     result = _parse_json_response(response, url)
@@ -232,6 +243,47 @@ def get_ontology_export(
     return result["files"]
 
 
+class NoSourceSchemaError(ApiError):
+    """The project's data source has no introspected schema to pull."""
+
+
+def get_schema_export(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> dict[str, Any]:
+    """GET /api/ci/projects/{project_id}/schema and return the response body."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/schema"
+    try:
+        with _client(transport=transport) as client:
+            response = client.get(url, headers={"Authorization": f"Bearer {api_key}"})
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    # "No source schema" is the marker the server's export endpoint puts in its
+    # 404 detail (backend endpoints/ci.py::export_source_schema — reworded only
+    # with a paired CLI release). Without this routing, a schema-less project
+    # would surface as the misleading "check --project / key access" hint below.
+    if response.status_code == 404 and "No source schema" in response.text:
+        raise NoSourceSchemaError(str(_detail_or_text(response)))
+    if response.status_code in (400, 403, 404):
+        raise _project_scope_error(response)
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
+    result = _parse_json_response(response, url)
+    if (
+        not isinstance(result, dict)
+        or not isinstance(result.get("tables"), list)
+        or not isinstance(result.get("schema_version"), dict)
+    ):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result
+
+
 def post_eval_run_start(
     *,
     api_url: str,
@@ -321,6 +373,54 @@ def post_eval_case_create(
     return result
 
 
+class EvalCaseNotFoundError(ApiError):
+    """The project has no current eval case with this id."""
+
+
+def get_eval_cases(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> list[dict[str, Any]]:
+    """GET /api/ci/projects/{project_id}/eval/cases and return the case list."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/eval/cases"
+    result = _get_eval_json(url, api_key, transport)
+    if not isinstance(result, list):
+        raise ApiError(f"Unexpected response shape from the Cassis API at {url}.")
+    return result
+
+
+def delete_eval_case(
+    *,
+    api_url: str,
+    api_key: str,
+    project_id: str,
+    case_id: str,
+    transport: Optional[httpx.BaseTransport] = None,
+) -> None:
+    """DELETE /api/ci/projects/{project_id}/eval/cases/{case_id}."""
+    url = api_url.rstrip("/") + f"/api/ci/projects/{project_id}/eval/cases/{case_id}"
+    try:
+        with _client(transport=transport) as client:
+            response = client.delete(url, headers={"Authorization": f"Bearer {api_key}"})
+    except httpx.HTTPError as exc:
+        raise ApiError(f"Could not reach the Cassis API at {url}: {exc}") from exc
+
+    if response.status_code == 401:
+        raise AuthError("The Cassis API rejected the API key (invalid or expired).")
+    # Exact-match wire contract with the DELETE endpoint's 404 detail (see
+    # `delete_eval_case` in backend/app/endpoints/ci.py): it distinguishes a
+    # missing case (exit 1) from a project-scope 404 (exit 3).
+    if response.status_code == 404 and _detail_or_text(response) == "Eval case not found":
+        raise EvalCaseNotFoundError(f"No current eval case {case_id} in this project (already deleted, or wrong id?).")
+    if response.status_code in (403, 404):
+        raise _project_scope_error(response)
+    if response.status_code >= 400:
+        raise ApiError(f"Cassis API returned HTTP {response.status_code}: {response.text[:500]}")
+
+
 def _get_eval_json(url: str, api_key: str, transport: Optional[httpx.BaseTransport]) -> Any:
     """GET an eval-run URL with the shared error mapping."""
     try:

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/common.py

@@ -93,20 +93,27 @@ def read_project_id_from_dir(ontology_dir: Path) -> Optional[str]:
     return None
 
 
-def resolve_project_id(project_id: Optional[str], ontology_dir: Path) -> str:
+def resolve_project_id(
+    project_id: Optional[str], ontology_dir: Path, *, optional: bool = False, quiet: bool = False
+) -> Optional[str]:
     """Resolve the target project id, defaulting to the checkout's ``project.yml``.
 
     Precedence: an explicit ``--project`` / ``CASSIS_PROJECT_ID`` wins; otherwise
     the ``project_id`` recorded in ``<base-path>/project.yml`` (written by
     ``pull`` / publish) is used, and where it came from is noted on stderr so a
-    stale value in a copied repo is visible. Exits 2 (usage) when neither is
-    available or the value isn't a UUID.
+    stale value in a copied repo is visible (``quiet`` suppresses the note for
+    machine-readable output). Exits 2 (usage) when the value isn't a UUID, or —
+    unless ``optional`` — when no value is available at all; with ``optional``,
+    an unbound checkout returns None (``check`` falls back to the project-less
+    validation).
     """
     from_file = False
     if not project_id:
         project_id = read_project_id_from_dir(ontology_dir)
         from_file = project_id is not None
     if not project_id:
+        if optional:
+            return None
         typer.secho(
             f"No project. Pass --project (or set CASSIS_PROJECT_ID), or run in a checkout whose "
             f"{ontology_dir.name}/project.yml records it (written by `cassis ontology pull` or a publish).",
@@ -119,7 +126,7 @@ def resolve_project_id(project_id: Optional[str], ontology_dir: Path) -> str:
     except ValueError:
         typer.secho(f"--project must be a project ID (UUID), got {project_id!r}.", fg=typer.colors.RED, err=True)
         raise typer.Exit(EXIT_USAGE)
-    if from_file:
+    if from_file and not quiet:
         typer.secho(f"Using project {project_id} from {ontology_dir.name}/project.yml.", fg=typer.colors.CYAN, err=True)
     return project_id
 

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/eval.py

@@ -8,6 +8,7 @@ import subprocess
 import time
 from pathlib import Path
 from typing import Any, Optional
+from uuid import UUID
 
 import typer
 from cassis_cli.api import (
@@ -16,8 +17,11 @@ from cassis_cli.api import (
     AuthError,
     EvalCaseExistsError,
     EvalCaseGoldSqlError,
+    EvalCaseNotFoundError,
     EvalRunActiveError,
     EvalStartValidationError,
+    delete_eval_case,
+    get_eval_cases,
     get_eval_run,
     get_eval_run_results,
     post_eval_case_create,
@@ -221,6 +225,134 @@ def add_case(
     raise typer.Exit(EXIT_OK)
 
 
+@app.command(name="list-cases")
+def list_cases(
+    path: Path = typer.Argument(
+        Path("."),
+        help="Repository checkout root (holds <base-path>/project.yml for the --project default).",
+    ),
+    project_id: Optional[str] = typer.Option(
+        None,
+        "--project",
+        envvar="CASSIS_PROJECT_ID",
+        help="Target Cassis project ID (UUID). Defaults to the id in <base-path>/project.yml.",
+    ),
+    api_key: Optional[str] = typer.Option(
+        None,
+        "--api-key",
+        envvar="CASSIS_API_KEY",
+        help="Cassis API key (sk-k6-...). Create one in Organization settings -> API keys.",
+    ),
+    api_url: str = typer.Option(
+        DEFAULT_API_URL,
+        "--api-url",
+        envvar="CASSIS_API_URL",
+        help="Cassis API base URL.",
+    ),
+    base_path: str = typer.Option(
+        DEFAULT_BASE_PATH,
+        "--base-path",
+        envvar="CASSIS_BASE_PATH",
+        help="Repository directory the ontology is exported under (holds project.yml for the --project default).",
+    ),
+    json_output: bool = typer.Option(False, "--json", help="Print the cases as raw JSON (includes gold SQL)."),
+) -> None:
+    """List the project's eval cases: the suite `cassis eval run` scores.
+
+    Prints each case's id and question (--json adds the gold SQL); the id is
+    what `cassis eval delete-case` takes. Exits 0 on success, 2 on usage
+    errors, 3 on transport/API errors.
+    """
+    api_key = require_api_key(api_key)
+    project_id = resolve_project_id(project_id, path / Path(base_path))
+
+    try:
+        cases = get_eval_cases(api_url=api_url, api_key=api_key, project_id=project_id)
+    except AuthError as exc:
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+    except ApiError as exc:
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+
+    if json_output:
+        typer.echo(json.dumps(cases, indent=2))
+    else:
+        if not cases:
+            typer.echo("No eval cases yet — add one with `cassis eval add-case`.")
+        for case in cases:
+            question = str(case.get("question", "")).replace("\n", " ")
+            typer.echo(f"{case.get('id')}  {question}")
+    raise typer.Exit(EXIT_OK)
+
+
+@app.command(name="delete-case")
+def delete_case(
+    case_id: str = typer.Argument(
+        ...,
+        help="Id of the eval case to delete (shown by `cassis eval list-cases`).",
+    ),
+    path: Path = typer.Option(
+        Path("."),
+        "--path",
+        help="Repository checkout root (holds <base-path>/project.yml for the --project default).",
+    ),
+    project_id: Optional[str] = typer.Option(
+        None,
+        "--project",
+        envvar="CASSIS_PROJECT_ID",
+        help="Target Cassis project ID (UUID). Defaults to the id in <base-path>/project.yml.",
+    ),
+    api_key: Optional[str] = typer.Option(
+        None,
+        "--api-key",
+        envvar="CASSIS_API_KEY",
+        help="Cassis API key (sk-k6-...). Create one in Organization settings -> API keys.",
+    ),
+    api_url: str = typer.Option(
+        DEFAULT_API_URL,
+        "--api-url",
+        envvar="CASSIS_API_URL",
+        help="Cassis API base URL.",
+    ),
+    base_path: str = typer.Option(
+        DEFAULT_BASE_PATH,
+        "--base-path",
+        envvar="CASSIS_BASE_PATH",
+        help="Repository directory the ontology is exported under (holds project.yml for the --project default).",
+    ),
+) -> None:
+    """Delete an eval case from the project's suite.
+
+    For pruning a case that is stale or wrong — e.g. its gold SQL encodes a
+    definition the ontology has since changed. Exits 0 on deletion, 1 when
+    the case does not exist in the project, 2 on usage errors, 3 on
+    transport/API errors.
+    """
+    api_key = require_api_key(api_key)
+    project_id = resolve_project_id(project_id, path / Path(base_path))
+    try:
+        UUID(case_id)
+    except ValueError:
+        typer.secho(f"CASE_ID must be a UUID, got {case_id!r}.", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_USAGE)
+
+    try:
+        delete_eval_case(api_url=api_url, api_key=api_key, project_id=project_id, case_id=case_id)
+    except EvalCaseNotFoundError as exc:
+        typer.secho(str(exc), fg=typer.colors.YELLOW, err=True)
+        raise typer.Exit(EXIT_VALIDATION_FAILED) from exc
+    except AuthError as exc:
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+    except ApiError as exc:
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+
+    typer.secho(f"✓ Deleted eval case {case_id}.", fg=typer.colors.GREEN)
+    raise typer.Exit(EXIT_OK)
+
+
 @app.command()
 def run(
     path: Path = typer.Argument(

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/guide.py

@@ -31,7 +31,7 @@ GUIDE_FILENAME = "AGENTS.md"
 # Monotonic version of the doctrine text below. Bump it whenever
 # ontology_design_guide.md changes (a backend test enforces the pairing) — it
 # is what lets an older writer recognize a newer guide and leave it alone.
-DOCTRINE_VERSION = 4
+DOCTRINE_VERSION = 5
 
 # Must stay byte-identical to backend/app/services/ontology_guide.py::_BANNER —
 # the server-side git export writes the same file, and differing banners would

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/main.py

@@ -6,6 +6,7 @@ import typer
 from cassis_cli import __version__
 from cassis_cli.eval import app as eval_app
 from cassis_cli.ontology import app as ontology_app
+from cassis_cli.schema import app as schema_app
 
 app = typer.Typer(
     no_args_is_help=True,
@@ -13,6 +14,7 @@ app = typer.Typer(
 )
 app.add_typer(ontology_app, name="ontology")
 app.add_typer(eval_app, name="eval")
+app.add_typer(schema_app, name="schema")
 
 
 @app.command()

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/ontology.py

@@ -56,6 +56,12 @@ def check(
         Path("."),
         help="Repository checkout root (the directory containing the ontology export path).",
     ),
+    project_id: Optional[str] = typer.Option(
+        None,
+        "--project",
+        envvar="CASSIS_PROJECT_ID",
+        help="Target Cassis project ID (UUID). Defaults to the id in <base-path>/project.yml.",
+    ),
     api_key: Optional[str] = typer.Option(
         None,
         "--api-key",
@@ -79,14 +85,27 @@ def check(
     """Validate the ontology files in a repository checkout.
 
     Runs the same checks as the Cassis GitHub PR check (YAML parsing,
-    round-trip, import validation). Exits 0 when valid, 1 when validation
-    fails, 2 on usage errors, 3 on transport/API errors.
+    round-trip, import validation). When the checkout is bound to a project
+    (``project.yml``, ``--project``, or ``CASSIS_PROJECT_ID``), the tree is
+    additionally cross-checked against the project's source schema; unmatched
+    references print as warnings and never fail the check. Exits 0 when valid,
+    1 when validation fails, 2 on usage errors, 3 on transport/API errors.
     """
     api_key = _require_api_key(api_key)
     files, base_path = _collect_tree(path, base_path)
 
+    # Soft resolution: an unbound checkout is not an error — the check falls
+    # back to the project-less route (no schema reference stage).
+    project_id = _resolve_project_id(project_id, path / base_path, optional=True, quiet=json_output)
+    if not project_id and not json_output:
+        typer.secho(
+            "No project binding — schema reference checks skipped (bind with `cassis ontology pull` or --project).",
+            fg=typer.colors.CYAN,
+            err=True,
+        )
+
     try:
-        result = post_ontology_check(api_url=api_url, api_key=api_key, files=files)
+        result = post_ontology_check(api_url=api_url, api_key=api_key, files=files, project_id=project_id)
     except AuthError as exc:
         typer.secho(str(exc), fg=typer.colors.RED, err=True)
         raise typer.Exit(EXIT_TRANSPORT) from exc
@@ -94,10 +113,21 @@ def check(
         typer.secho(str(exc), fg=typer.colors.RED, err=True)
         raise typer.Exit(EXIT_TRANSPORT) from exc
 
+    warnings = result.get("warnings") or []
     if json_output:
         typer.echo(json.dumps(result, indent=2))
     elif result["passed"]:
         typer.secho(f"✓ {result['summary']}", fg=typer.colors.GREEN)
+        # Disambiguate silence: "no warnings" must never read as "references
+        # verified" when the stage didn't run.
+        if project_id and result.get("references_checked") and not warnings:
+            typer.secho("✓ Schema references resolve against the source schema.", fg=typer.colors.GREEN)
+        elif project_id and not result.get("references_checked"):
+            typer.secho(
+                "Schema reference check skipped — the project has no source schema yet.",
+                fg=typer.colors.YELLOW,
+                err=True,
+            )
     else:
         typer.secho(result["title"], fg=typer.colors.RED, bold=True)
         typer.echo(result["summary"])
@@ -105,6 +135,16 @@ def check(
             location = f"{base_path}/{finding.get('path')}: " if finding.get("path") else ""
             typer.echo(f"  {location}{finding.get('message', '')} ({finding.get('stage', '?')})")
 
+    if warnings and not json_output:
+        typer.secho(
+            f"{len(warnings)} schema reference warning(s) — advisory, expected if the objects "
+            "haven't been built or synced yet:",
+            fg=typer.colors.YELLOW,
+            bold=True,
+        )
+        for warning in warnings:
+            typer.secho(f"  {warning.get('message', '')}", fg=typer.colors.YELLOW)
+
     raise typer.Exit(EXIT_OK if result["passed"] else EXIT_VALIDATION_FAILED)
 
 

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/cassis_cli/ontology_design_guide.md

@@ -357,6 +357,16 @@ description, so a wrong example is worse than none.
   real mechanic diverges.
 - **Don't describe things that don't exist.** Every table, column, and value you
   reference must be real.
+- **A hedge is not a safeguard.** "Likely `'M:SS.mmm'`", "probably", "appears to
+  be" reach the agent as the claim itself; the qualifier does not survive into
+  its reasoning. If a storage format, a unit, or a reason for nullability is not
+  in the source, leave it out and record the open question instead.
+- **Never invent a contrast with a column you did not read.** When a new column
+  shares a name with one on another table (`position_order`, `status_id`,
+  `lap`), the tempting sentence is "here it means X, unlike there". A schema
+  dump cannot support that. Open the sibling's description and reuse it if it
+  applies; if you can't confirm the relationship, say it's unverified. A false
+  distinction drawn confidently survives review as a documented decision.
 
 ---
 

cassis_cli-1.2.0/cassis_cli/schema.py

@@ -0,0 +1,129 @@
+"""`cassis schema` — local snapshot of the data source's source schema.
+
+The source schema is OBSERVED state (the warehouse is authoritative), so the
+snapshot is a gitignored cache, never a committed file: `pull` writes
+`<base-path>/.schema.json` and keeps it out of git via the ontology dir's
+`.gitignore`. Agents working in a checkout grep it instead of paging through
+the MCP `get_source_schema` tool; `pulled_at` records how stale it is.
+"""
+
+from __future__ import annotations
+
+import json
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Optional
+
+import typer
+from cassis_cli.api import DEFAULT_API_URL, ApiError, AuthError, get_schema_export
+from cassis_cli.common import (
+    DEFAULT_BASE_PATH,
+    EXIT_OK,
+    EXIT_TRANSPORT,
+    EXIT_USAGE,
+    require_api_key,
+    resolve_project_id,
+)
+
+app = typer.Typer(help="Pull a local, gitignored snapshot of the data source's schema.")
+
+SNAPSHOT_FILENAME = ".schema.json"
+_GITIGNORE_HEADER = "# Cassis local caches (observed state — never commit)"
+
+
+@app.command()
+def pull(
+    path: Path = typer.Argument(
+        Path("."),
+        help="Repository checkout root (the directory containing the ontology export path).",
+    ),
+    project_id: Optional[str] = typer.Option(
+        None,
+        "--project",
+        envvar="CASSIS_PROJECT_ID",
+        help="Target Cassis project ID (UUID). Defaults to the id in <base-path>/project.yml.",
+    ),
+    api_key: Optional[str] = typer.Option(
+        None,
+        "--api-key",
+        envvar="CASSIS_API_KEY",
+        help="Cassis API key (sk-k6-...). Create one in Organization settings -> API keys.",
+    ),
+    api_url: str = typer.Option(
+        DEFAULT_API_URL,
+        "--api-url",
+        envvar="CASSIS_API_URL",
+        help="Cassis API base URL.",
+    ),
+    base_path: str = typer.Option(
+        DEFAULT_BASE_PATH,
+        "--base-path",
+        envvar="CASSIS_BASE_PATH",
+        help="Repository directory the ontology is exported under (the project's git-sync Path setting).",
+    ),
+) -> None:
+    """Download the source schema into `<base-path>/.schema.json` (gitignored).
+
+    The snapshot is the schema as Cassis last introspected it from the
+    warehouse (or parsed from an uploaded DDL) — every table with its columns
+    and types, plus a `pulled_at` stamp so staleness is visible. Re-run after
+    a warehouse sync to refresh. Exits 0 on success, 2 on usage errors, 3 on
+    transport/API errors.
+    """
+    api_key = require_api_key(api_key)
+    ontology_dir = path / base_path
+    project_id = resolve_project_id(project_id, ontology_dir)
+
+    try:
+        result = get_schema_export(api_url=api_url, api_key=api_key, project_id=project_id)
+    except (AuthError, ApiError) as exc:
+        # NoSourceSchemaError lands here too: the server's message already says
+        # what to do (sync or upload a DDL); the class exists so api.py doesn't
+        # bury it under the misleading project-scope hint.
+        typer.secho(str(exc), fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_TRANSPORT) from exc
+
+    snapshot = {
+        "pulled_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
+        "project_id": project_id,
+        "schema_version": result["schema_version"],
+        "tables": result["tables"],
+    }
+
+    try:
+        ontology_dir.mkdir(parents=True, exist_ok=True)
+        snapshot_path = ontology_dir / SNAPSHOT_FILENAME
+        snapshot_path.write_text(json.dumps(snapshot, indent=2) + "\n", encoding="utf-8")
+        ensure_gitignored(ontology_dir)
+    except OSError as exc:
+        # Usage-class exit (2), like the other local-file failures in the exit
+        # table — the API call succeeded, the checkout is what's broken.
+        typer.secho(f"Could not write the snapshot: {exc}", fg=typer.colors.RED, err=True)
+        raise typer.Exit(EXIT_USAGE) from exc
+
+    table_count = len(result["tables"])
+    column_count = sum(len(t.get("columns") or []) for t in result["tables"])
+    version = result["schema_version"].get("version")
+    typer.secho(
+        f"✓ Pulled source schema v{version}: {table_count} tables, {column_count} columns "
+        f"-> {snapshot_path} (gitignored)",
+        fg=typer.colors.GREEN,
+    )
+
+
+def ensure_gitignored(ontology_dir: Path) -> None:
+    """Make sure the snapshot never lands in git: keep `.gitignore` covering it.
+
+    Appends to (or creates) the ontology dir's own `.gitignore` — local to the
+    export directory, so it survives repo-level `.gitignore` rewrites and needs
+    no knowledge of the checkout layout.
+    """
+    gitignore = ontology_dir / ".gitignore"
+    try:
+        existing = gitignore.read_text(encoding="utf-8")
+    except (OSError, UnicodeDecodeError) as _exc:  # `as` keeps black from stripping the parens (3.14-only syntax)
+        existing = ""
+    if SNAPSHOT_FILENAME in existing.splitlines():
+        return
+    prefix = "" if not existing else existing.rstrip("\n") + "\n"
+    gitignore.write_text(f"{prefix}{_GITIGNORE_HEADER}\n{SNAPSHOT_FILENAME}\n", encoding="utf-8")

{cassis_cli-1.1.1 → cassis_cli-1.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "cassis-cli"
-version = "1.1.1"
+version = "1.2.0"
 description = "Cassis CLI — run Cassis actions (ontology validation, upload and publish) from your CI pipelines"
 readme = "README.md"
 license = { text = "Apache-2.0" }