orchestra-cli 0.0.5__tar.gz → 0.1.2__tar.gz

orchestra_cli-0.1.2/PKG-INFO

@@ -0,0 +1,196 @@
+Metadata-Version: 2.4
+Name: orchestra-cli
+Version: 0.1.2
+Summary: Orchestra CLI. Perform operations with Orchestra locally.
+Author-email: Orchestra Technologies <support@getorchestra.io>
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: pyyaml>=6.0.2
+Requires-Dist: typer>=0.16.0
+
+# orchestra-cli
+
+Orchestra CLI for working with Orchestra pipelines from your terminal.
+
+Two entrypoints are available: `orchestra` and `orchestra-cli` (they are equivalent).
+
+## Installation
+
+```bash
+pip install orchestra-cli
+```
+
+Or with pipx:
+
+```bash
+pipx install orchestra-cli
+```
+
+## Environment variables
+
+- `ORCHESTRA_API_KEY`: Required for actions that call the API (`import`, `run`).
+- `BASE_URL`: Optional. Override the default API base
+  (`https://app.getorchestra.io/api/engine/public/pipelines/{}`) for non‑production/testing.
+
+## Commands overview
+
+- `validate`: Validate a pipeline YAML locally against the Orchestra API schema.
+- `import`: Register a pipeline YAML (from a git repo) with Orchestra under an alias.
+- `run`: Start a pipeline run by alias, optionally pinning branch/commit and waiting for completion.
+
+Use `orchestra --help` or `orchestra <command> --help` for built-in help.
+
+---
+
+## validate
+
+Validate a YAML file against the Orchestra API schema.
+
+```bash
+orchestra validate path/to/pipeline.yaml
+# or
+orchestra-cli validate path/to/pipeline.yaml
+```
+
+Options
+
+- `file` (positional): Path to the YAML file to validate.
+
+Behavior
+
+- Prints a success message on valid input.
+- On validation errors, prints the failing location(s), readable messages, and a YAML snippet when possible.
+- Exit codes: `0` on success, `1` on invalid file/validation failure/HTTP error.
+
+Example output (failure)
+
+```text
+❌ Validation failed with status 422
+Error at: pipeline.tasks[0].type
+  Invalid task type "foo"
+
+YAML snippet:
+pipeline:
+  tasks:
+    - type: foo
+```
+
+---
+
+## import
+
+Create (import) a pipeline in Orchestra by referencing a YAML file inside a git repository. The command infers your repository host/provider, default branch, and YAML path relative to the repo root.
+
+```bash
+export ORCHESTRA_API_KEY=...  # required
+
+orchestra import \
+  --alias my-pipeline \
+  --path ./pipelines/pipeline.yaml \
+  --working-branch my-feature-branch   # optional; defaults to current local branch
+# or
+orchestra-cli import -a my-pipeline -p ./pipelines/pipeline.yaml
+```
+
+Options
+
+- `-a, --alias` (required): The alias you want to register the pipeline under.
+- `-p, --path` (required): Path to the YAML file. Must be inside a git repository.
+- `-w, --working-branch` (optional): Branch to associate as the working branch for this pipeline import. If omitted, the current local git branch is used.
+
+Notes
+
+- The YAML is validated with the API before import; failures are printed clearly.
+- Git details are detected automatically:
+  - Supported providers: GitHub, GitLab, Azure DevOps.
+  - The default branch is detected from `origin`.
+  - The working branch defaults to your current local branch when not specified.
+  - The YAML path is computed relative to the repository root.
+- On success, the command prints the created pipeline ID (or a success message).
+- Exit codes: `0` on success, `1` on failure.
+
+Common errors
+
+- Missing `ORCHESTRA_API_KEY`.
+- Not running inside a git repository, or no `origin` remote configured.
+- Could not detect storage provider or default branch.
+
+---
+
+## run
+
+Start a pipeline run by alias. Optionally specify a branch and/or commit. By default, the command waits and polls the run status until completion.
+
+```bash
+export ORCHESTRA_API_KEY=...
+
+# Start and wait for completion
+orchestra run --alias my-pipeline
+
+# Start without waiting (prints run id and exits)
+orchestra run -a my-pipeline --no-wait
+
+# Start for a specific branch/commit
+orchestra run -a my-pipeline -b feature/my-change -c 0123abc
+```
+
+Options
+
+- `-a, --alias` (required): Pipeline alias to run.
+- `-b, --branch` (optional): Git branch name to associate with this run.
+- `-c, --commit` (optional): Commit SHA to associate with this run.
+- `--wait/--no-wait` (default: `--wait`): Poll until the run ends.
+- `--force/--no-force` (default: `--no-force`): Skip confirmation if local git warnings are detected.
+
+Behavior
+
+- Prints the run ID when known and a link to the run lineage page.
+- When waiting, polls status every ~5s until a terminal state:
+  - `SUCCEEDED` (exit `0`), `WARNING` (exit `0`), `SKIPPED` (exit `0`)
+  - `FAILED` or `CANCELLED` (exit `1`)
+- When not waiting, exits after start and prints the run ID.
+
+Non-interactive usage
+
+- If your repo has warnings (e.g., uncommitted changes), the CLI prompts for confirmation unless `--force` is provided. For CI or scripts, pass `--force` or ensure a clean repo.
+
+---
+
+## Examples
+
+```bash
+# Validate a pipeline file
+orchestra validate ./examples/etl.yaml
+
+# Import a pipeline and capture the created ID
+PIPELINE_ID=$(orchestra import -a finance-etl -p ./pipelines/etl.yaml)
+
+# Start a run and wait for completion
+orchestra run -a finance-etl
+
+# Start a run and exit immediately
+orchestra run -a finance-etl --no-wait
+```
+
+---
+
+## Development
+
+- Make sure [uv](https://github.com/astral-sh/uv) is installed
+- Use `uv pip install -e ".[dev]"` to install the CLI in editable mode for development
+- For local development, run `uv run orchestra` to start the CLI
+  - you can use `uv run --env-file .env ...` to run the CLI with env vars
+- For testing, run `uv run pytest`
+- For linting, run `uv run ruff check .`
+- For formatting, run `uv run black --check .`
+- For type checking, run `uv run pyright`
+
+## Building and Releasing
+
+- Bump the version in `pyproject.toml` or by running `uv version --bump <major/minor/patch>`
+- Run `uv sync` to install the dependencies
+- Run `uv build` to build the CLI
+- Run `uv publish` to publish the CLI (you will need to pass the `--token` flag)
+
+**Note: Failure to bump the version will result in a failed release.**

orchestra_cli-0.1.2/README.md

@@ -0,0 +1,185 @@
+# orchestra-cli
+
+Orchestra CLI for working with Orchestra pipelines from your terminal.
+
+Two entrypoints are available: `orchestra` and `orchestra-cli` (they are equivalent).
+
+## Installation
+
+```bash
+pip install orchestra-cli
+```
+
+Or with pipx:
+
+```bash
+pipx install orchestra-cli
+```
+
+## Environment variables
+
+- `ORCHESTRA_API_KEY`: Required for actions that call the API (`import`, `run`).
+- `BASE_URL`: Optional. Override the default API base
+  (`https://app.getorchestra.io/api/engine/public/pipelines/{}`) for non‑production/testing.
+
+## Commands overview
+
+- `validate`: Validate a pipeline YAML locally against the Orchestra API schema.
+- `import`: Register a pipeline YAML (from a git repo) with Orchestra under an alias.
+- `run`: Start a pipeline run by alias, optionally pinning branch/commit and waiting for completion.
+
+Use `orchestra --help` or `orchestra <command> --help` for built-in help.
+
+---
+
+## validate
+
+Validate a YAML file against the Orchestra API schema.
+
+```bash
+orchestra validate path/to/pipeline.yaml
+# or
+orchestra-cli validate path/to/pipeline.yaml
+```
+
+Options
+
+- `file` (positional): Path to the YAML file to validate.
+
+Behavior
+
+- Prints a success message on valid input.
+- On validation errors, prints the failing location(s), readable messages, and a YAML snippet when possible.
+- Exit codes: `0` on success, `1` on invalid file/validation failure/HTTP error.
+
+Example output (failure)
+
+```text
+❌ Validation failed with status 422
+Error at: pipeline.tasks[0].type
+  Invalid task type "foo"
+
+YAML snippet:
+pipeline:
+  tasks:
+    - type: foo
+```
+
+---
+
+## import
+
+Create (import) a pipeline in Orchestra by referencing a YAML file inside a git repository. The command infers your repository host/provider, default branch, and YAML path relative to the repo root.
+
+```bash
+export ORCHESTRA_API_KEY=...  # required
+
+orchestra import \
+  --alias my-pipeline \
+  --path ./pipelines/pipeline.yaml \
+  --working-branch my-feature-branch   # optional; defaults to current local branch
+# or
+orchestra-cli import -a my-pipeline -p ./pipelines/pipeline.yaml
+```
+
+Options
+
+- `-a, --alias` (required): The alias you want to register the pipeline under.
+- `-p, --path` (required): Path to the YAML file. Must be inside a git repository.
+- `-w, --working-branch` (optional): Branch to associate as the working branch for this pipeline import. If omitted, the current local git branch is used.
+
+Notes
+
+- The YAML is validated with the API before import; failures are printed clearly.
+- Git details are detected automatically:
+  - Supported providers: GitHub, GitLab, Azure DevOps.
+  - The default branch is detected from `origin`.
+  - The working branch defaults to your current local branch when not specified.
+  - The YAML path is computed relative to the repository root.
+- On success, the command prints the created pipeline ID (or a success message).
+- Exit codes: `0` on success, `1` on failure.
+
+Common errors
+
+- Missing `ORCHESTRA_API_KEY`.
+- Not running inside a git repository, or no `origin` remote configured.
+- Could not detect storage provider or default branch.
+
+---
+
+## run
+
+Start a pipeline run by alias. Optionally specify a branch and/or commit. By default, the command waits and polls the run status until completion.
+
+```bash
+export ORCHESTRA_API_KEY=...
+
+# Start and wait for completion
+orchestra run --alias my-pipeline
+
+# Start without waiting (prints run id and exits)
+orchestra run -a my-pipeline --no-wait
+
+# Start for a specific branch/commit
+orchestra run -a my-pipeline -b feature/my-change -c 0123abc
+```
+
+Options
+
+- `-a, --alias` (required): Pipeline alias to run.
+- `-b, --branch` (optional): Git branch name to associate with this run.
+- `-c, --commit` (optional): Commit SHA to associate with this run.
+- `--wait/--no-wait` (default: `--wait`): Poll until the run ends.
+- `--force/--no-force` (default: `--no-force`): Skip confirmation if local git warnings are detected.
+
+Behavior
+
+- Prints the run ID when known and a link to the run lineage page.
+- When waiting, polls status every ~5s until a terminal state:
+  - `SUCCEEDED` (exit `0`), `WARNING` (exit `0`), `SKIPPED` (exit `0`)
+  - `FAILED` or `CANCELLED` (exit `1`)
+- When not waiting, exits after start and prints the run ID.
+
+Non-interactive usage
+
+- If your repo has warnings (e.g., uncommitted changes), the CLI prompts for confirmation unless `--force` is provided. For CI or scripts, pass `--force` or ensure a clean repo.
+
+---
+
+## Examples
+
+```bash
+# Validate a pipeline file
+orchestra validate ./examples/etl.yaml
+
+# Import a pipeline and capture the created ID
+PIPELINE_ID=$(orchestra import -a finance-etl -p ./pipelines/etl.yaml)
+
+# Start a run and wait for completion
+orchestra run -a finance-etl
+
+# Start a run and exit immediately
+orchestra run -a finance-etl --no-wait
+```
+
+---
+
+## Development
+
+- Make sure [uv](https://github.com/astral-sh/uv) is installed
+- Use `uv pip install -e ".[dev]"` to install the CLI in editable mode for development
+- For local development, run `uv run orchestra` to start the CLI
+  - you can use `uv run --env-file .env ...` to run the CLI with env vars
+- For testing, run `uv run pytest`
+- For linting, run `uv run ruff check .`
+- For formatting, run `uv run black --check .`
+- For type checking, run `uv run pyright`
+
+## Building and Releasing
+
+- Bump the version in `pyproject.toml` or by running `uv version --bump <major/minor/patch>`
+- Run `uv sync` to install the dependencies
+- Run `uv build` to build the CLI
+- Run `uv publish` to publish the CLI (you will need to pass the `--token` flag)
+
+**Note: Failure to bump the version will result in a failed release.**

orchestra_cli-0.1.2/orchestra_cli/src/cli.py

@@ -0,0 +1,12 @@
+import typer
+
+from .import_pipeline import import_pipeline
+from .run_pipeline import run_pipeline
+from .validate_pipeline import validate
+
+app = typer.Typer(help="Orchestra CLI – perform operations with Orchestra locally.")
+
+
+app.command(name="validate")(validate)
+app.command(name="import")(import_pipeline)
+app.command(name="run")(run_pipeline)

orchestra_cli-0.1.2/orchestra_cli/src/import_pipeline.py

@@ -0,0 +1,223 @@
+import json
+import os
+import re
+from pathlib import Path
+
+import httpx
+import typer
+import yaml
+
+from ..utils.constants import get_api_url
+from ..utils.git import detect_repo_root, git_warnings, run_git_command
+from ..utils.styling import bold, green, indent_message, red, yellow
+
+
+def _detect_repository_url(repo_root: Path) -> str | None:
+    ok, remote = run_git_command(["remote", "get-url", "origin"], repo_root)
+    if not ok or not remote:
+        return None
+
+    # First, handle special cases like Azure URLs by removing segments like /_git/
+    cleaned_remote = re.sub(r"/(?:_git|scm|v3)(?=/)", "", remote.strip())
+
+    # A single regex can then capture the owner/repo from the cleaned URL
+    pattern = r".*[:/]([^/]+)/([^/]+?)(?:\.git)?/?$"
+
+    if match := re.search(pattern, cleaned_remote):
+        return f"{match.group(1)}/{match.group(2)}"
+
+    return None
+
+
+def _get_remote_url(repo_root: Path) -> str | None:
+    """Return the raw git remote URL for origin without transformation."""
+    ok, remote = run_git_command(["remote", "get-url", "origin"], repo_root)
+    if not ok:
+        return None
+    return remote
+
+
+def _detect_default_branch(repo_root: Path) -> str | None:
+    # Try symbolic-ref of remote HEAD first
+    ok, out = run_git_command(["symbolic-ref", "refs/remotes/origin/HEAD"], repo_root)
+    if ok and out:
+        # refs/remotes/origin/main -> main
+        return out.split("/")[-1]
+    # Fallback to parsing `git remote show origin`
+    ok, out = run_git_command(["remote", "show", "origin"], repo_root)
+    if ok and out:
+        m = re.search(r"HEAD branch:\s*(\S+)", out)
+        if m:
+            return m.group(1)
+    return None
+
+
+def _detect_current_branch(repo_root: Path) -> str | None:
+    ok, out = run_git_command(["rev-parse", "--abbrev-ref", "HEAD"], repo_root)
+    if ok and out:
+        return out
+    return None
+
+
+def _detect_storage_provider(repository_url: str | None) -> str:
+    if not repository_url:
+        typer.echo(red("Could not detect storage provider - no repository URL"))
+        raise typer.Exit(code=1)
+    url = repository_url.lower()
+    if "github.com" in url:
+        return "GITHUB"
+    if "gitlab.com" in url:
+        return "GITLAB"
+    if any(host in url for host in ["dev.azure.com", "azure.com", "visualstudio.com"]):
+        return "AZURE_DEVOPS"
+    typer.echo(red("Could not detect storage provider - no matching host"))
+    raise typer.Exit(code=1)
+
+
+def _load_yaml(file: Path) -> tuple[dict | None, str | None]:
+    try:
+        with file.open("r") as f:
+            data = yaml.safe_load(f)
+        return data, None
+    except Exception as e:
+        return None, str(e)
+
+
+def _validate_yaml_with_api(data: dict) -> tuple[bool, str | None]:
+    try:
+        response = httpx.post(get_api_url("schema"), json=data, timeout=15)
+    except Exception as e:
+        return False, f"HTTP request failed: {e}"
+
+    if response.status_code == 200:
+        return True, None
+    try:
+        errors = response.json()
+        return False, json.dumps(errors, indent=2)
+    except Exception:
+        return False, response.text
+
+
+def import_pipeline(
+    alias: str = typer.Option(..., "--alias", "-a", help="Pipeline alias"),
+    path: Path = typer.Option(
+        ...,
+        "--path",
+        "-p",
+        exists=True,
+        file_okay=True,
+        dir_okay=False,
+        readable=True,
+        resolve_path=True,
+        help="Path to pipeline YAML inside a git repository",
+    ),
+    working_branch: str | None = typer.Option(
+        None,
+        "--working-branch",
+        "-w",
+        help="Git branch to use for the imported pipeline (defaults to current local branch)",
+    ),
+):
+    """
+    Create a pipeline in Orchestra by referencing a YAML file in your git repository.
+    """
+
+    if not path.exists():
+        typer.echo(red(f"File not found: {path}"))
+        raise typer.Exit(code=1)
+
+    api_key = os.getenv("ORCHESTRA_API_KEY")
+    if not api_key:
+        typer.echo(red("ORCHESTRA_API_KEY is not set"))
+        raise typer.Exit(code=1)
+
+    # Load and validate YAML with API first
+    data, err = _load_yaml(path)
+    if err is not None:
+        typer.echo(red(f"Invalid YAML: {err}"))
+        raise typer.Exit(code=1)
+
+    ok, err_msg = _validate_yaml_with_api(data or {})
+    if not ok:
+        typer.echo(red("❌ Validation failed"))
+        if err_msg:
+            typer.echo(yellow(indent_message(err_msg)))
+        raise typer.Exit(code=1)
+
+    # Detect git repository info
+    repo_root = detect_repo_root(path.parent)
+    if repo_root is None:
+        typer.echo(red("Not a git repository (could not detect repository root)"))
+        raise typer.Exit(code=1)
+
+    repository_slug = _detect_repository_url(repo_root)
+    if not repository_slug:
+        typer.echo(red("Could not detect repository URL from git"))
+        raise typer.Exit(code=1)
+    default_branch = _detect_default_branch(repo_root)
+    if not default_branch:
+        typer.echo(red("Could not detect default branch from git"))
+        raise typer.Exit(code=1)
+
+    # Determine working branch (explicit option or current branch)
+    if working_branch is None:
+        working_branch = _detect_current_branch(repo_root)
+        if not working_branch:
+            typer.echo(red("Could not detect current branch from git"))
+            raise typer.Exit(code=1)
+
+    # Compute YAML path relative to repo root
+    try:
+        yaml_path = str(path.resolve().relative_to(repo_root.resolve()))
+    except Exception:
+        typer.echo(red("YAML file must be inside the git repository"))
+        raise typer.Exit(code=1)
+
+    # Show warnings
+    for w in git_warnings(repo_root):
+        typer.echo(yellow(f"⚠ {w}"))
+
+    payload = {
+        "storage_provider": _detect_storage_provider(_get_remote_url(repo_root)),
+        "repository": repository_slug,
+        "default_branch": default_branch,
+        "working_branch": working_branch,
+        "yaml_path": yaml_path,
+        "alias": alias,
+    }
+
+    try:
+        response = httpx.post(
+            get_api_url("import"),
+            json=payload,
+            timeout=30,
+            headers={"Authorization": f"Bearer {api_key}"},
+        )
+    except Exception as e:
+        typer.echo(red(f"HTTP request failed: {e}"))
+        raise typer.Exit(code=1)
+
+    if response.status_code == 201:
+        try:
+            body = response.json()
+        except Exception:
+            body = {}
+        pipeline_id = body.get("id")
+        if pipeline_id:
+            # Print only the pipeline id as requested
+            typer.echo(f"Pipeline with alias '{alias}' imported successfully: {pipeline_id}")
+            raise typer.Exit(code=0)
+        else:
+            # Fallback if server does not return a field we expect
+            typer.echo(green("✅ Pipeline imported successfully"))
+            if body:
+                typer.echo(bold(json.dumps(body)))
+            raise typer.Exit(code=0)
+
+    # Error handling for non-201 responses
+    typer.echo(red(f"❌ Import failed with status {response.status_code}"))
+    try:
+        typer.echo(yellow(indent_message(json.dumps(response.json(), indent=2))))
+    except Exception:
+        typer.echo(yellow(indent_message(response.text)))
+    raise typer.Exit(code=1)