vellum-ai 1.7.13__tar.gz → 1.9.7__tar.gz

vellum_ai-1.9.7/PKG-INFO

@@ -0,0 +1,125 @@
+Metadata-Version: 2.1
+Name: vellum-ai
+Version: 1.9.7
+Summary: 
+License: MIT
+Requires-Python: >=3.9,<4.0
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Operating System :: OS Independent
+Classifier: Operating System :: POSIX
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Dist: Jinja2 (>=3.1.0,<4.0.0)
+Requires-Dist: click (>=8.1.7,<9.0.0)
+Requires-Dist: docker (>=7.1.0,<8.0.0)
+Requires-Dist: httpx (>=0.21.2)
+Requires-Dist: openai (>=1.0.0,<2.0.0)
+Requires-Dist: orderly-set (>=5.2.2,<6.0.0)
+Requires-Dist: publication (==0.0.3)
+Requires-Dist: pydantic (>=1.9.2)
+Requires-Dist: pydantic-core (>=2.18.2)
+Requires-Dist: pydash (>=7.0.0,<8.0.0)
+Requires-Dist: python-dateutil (>=2.8.0,<3.0.0)
+Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
+Requires-Dist: pytz (>=2022.0,<2026.0)
+Requires-Dist: pyyaml (>=6.0.0,<7.0.0)
+Requires-Dist: requests (>=2.31.0,<3.0.0)
+Requires-Dist: tomli (>=2.0.0,<3.0.0)
+Requires-Dist: typing_extensions (>=4.0.0)
+Project-URL: Repository, https://github.com/vellum-ai/vellum-python-sdks
+Description-Content-Type: text/markdown
+
+<p align="center">
+   <a href="https://vellum.ai">
+    <picture>
+      <img alt="Vellum README banner" src="https://storage.googleapis.com/vellum-public/assets/github-readme-banner.png" height="128">
+    </picture>
+  </a>
+  <p align="center">
+    <a href="https://vellum.ai">Learn more</a>
+    ·
+    <a href="https://www.vellum.ai/landing-pages/request-demo">Talk to us</a>
+  </p>
+</p>
+
+<p align="center">
+  <a href="https://pepy.tech/project/vellum-ai">
+    <img src="https://img.shields.io/pypi/dm/vellum-ai" alt="Vellum AI PyPI - Downloads" >
+  </a>
+  <a href="https://pypi.org/project/vellum-ai" target="_blank">
+    <img src="https://img.shields.io/pypi/v/vellum-ai?color=%2334D058&label=pypi%20package" alt="Package version">
+  </a>
+  <a href="https://www.ycombinator.com/companies/vellum">
+    <img src="https://img.shields.io/badge/Y%20Combinator-W23-orange?style=flat-square" alt="Y Combinator S24">
+  </a>
+</p>
+
+# Introduction
+
+[Vellum](https://www.vellum.ai/) is the end-to-end development platform for building production-grade AI applications.
+
+### Core Features
+
+- **Orchestration:** A powerful SDK and IDE for defining and debugging the control flow of your AI applications
+- **Prompting:** A best-in-class prompt playground for iterating on and refining prompts between models from any provider
+- **Evaluations**: An evaluations framework that makes it easy to measure the quality of your AI systems at scale
+- **Retrieval:** A ready-to-go service for turning unstructured content into intelligent, context-aware solutions
+  optimized for AI systems
+- **Deployment:** Decouple updates to your AI systems from your application code with an easy integration +
+  one-click deploy
+- **Observability:** Monitor and debug your AI systems in real-time with detailed logs, metrics, and end-user feedback
+
+## Table of Contents
+
+- [Get Started](#get-started)
+- [Client SDK](#client-sdk)
+- [Workflows SDK](#workflows-sdk)
+- [Contributing](#contributing)
+- [Open-source vs paid](#open-source-vs-paid)
+
+## Get Started
+
+Most functionality within the Vellum SDK requires a Vellum account and API key. You can sign up for free [here](https://app.vellum.ai/signup?f=wsdk&utm_source=github&utm_medium=repo_readme&utm_campaign=sdk) or visit our [pricing page](https://www.vellum.ai/pricing) for paid options.
+
+Even without a Vellum account, you can use the Workflows SDK to define the control flow of your AI systems. [Learn
+more below](#workflows-sdk).
+
+## Client SDK
+
+The Vellum Client SDK, found within `src/vellum/client` is a low-level client used to interact directly with the Vellum API.
+Learn more and get started by visiting the [Vellum Client SDK README](/src/vellum/client/README.md).
+
+## Workflows SDK
+
+The Vellum Workflows SDK is a high-level framework for defining and debugging the control flow of AI systems. At
+it's core, it's a powerful workflow engine with syntactic sugar for declaratively defining graphs, the nodes within,
+and the relationships between them.
+
+The Workflows SDK can be used with or without a Vellum account, but a Vellum account is required to use certain
+out-of-box nodes and features, including the ability to push and pull your Workflow definition to Vellum for editing
+and debugging via a UI.
+
+To learn more and get started, visit the [Vellum Workflows SDK README](/src/vellum/workflows/README.md).
+
+## Contributing
+
+See the [CONTRIBUTING.md](/CONTRIBUTING.md) for information on how to contribute to the Vellum SDKs.
+
+## Open-Source vs. Paid
+
+This repo is available under the [MIT expat license](https://github.com/vellum-ai/vellum-python-sdks/blob/main/LICENSE), except
+for the `ee` directory (which has its [license here](https://github.com/vellum-ai/vellum-python-sdks/blob/main/ee/LICENSE)) if applicable.
+
+To learn more, [book a demo](https://www.vellum.ai/landing-pages/request-demo) or see our [pricing page](https://www.vellum.ai/pricing).
+

vellum_ai-1.9.7/ee/vellum_cli/ping.py

@@ -0,0 +1,45 @@
+from dotenv import load_dotenv
+
+from vellum.client.core.api_error import ApiError
+from vellum.workflows.vellum_client import create_vellum_client
+from vellum_cli.logger import load_cli_logger
+
+
+def ping_command():
+    load_dotenv()
+    logger = load_cli_logger()
+
+    client = create_vellum_client()
+
+    try:
+        workspace = client.workspaces.workspace_identity()
+        organization = client.organizations.organization_identity()
+        environment = client.environments.environment_identity()
+    except ApiError as e:
+        # If user did not provide an API key, we will get a 403 error
+        if e.status_code == 401 or e.status_code == 403:
+            raise Exception(
+                "Please make sure your `VELLUM_API_KEY` environment variable is set correctly."  # noqa: E501
+            ) from e
+        raise Exception(
+            "The API we tried to ping returned an invalid response. Please make sure your `VELLUM_API_URL` environment variable is set correctly."  # noqa: E501
+        )
+
+    logger.info(
+        f"""\
+Successfully authenticated with Vellum!
+
+Organization:
+    ID: {organization.id}
+    Name: {organization.name}
+
+Workspace:
+    ID: {workspace.id}
+    Name: {workspace.name}
+
+Environment:
+    ID: {environment.id}
+    Name: {environment.name}
+    Label: {environment.label}
+"""
+    )

vellum_ai-1.9.7/ee/vellum_cli/push.py

@@ -0,0 +1,342 @@
+from importlib import metadata
+import io
+import json
+import os
+import sys
+import tarfile
+from uuid import UUID
+from typing import List, Optional
+
+from dotenv import load_dotenv
+from pydantic import ValidationError
+
+from vellum.client.core.api_error import ApiError
+from vellum.client.types import WorkflowPushDeploymentConfigRequest
+from vellum.resources.workflows.client import OMIT
+from vellum.workflows.vellum_client import create_vellum_client
+from vellum_cli.config import DEFAULT_WORKSPACE_CONFIG, WorkflowConfig, WorkflowDeploymentConfig, load_vellum_cli_config
+from vellum_cli.logger import handle_cli_error, load_cli_logger
+from vellum_ee.workflows.display.nodes.utils import to_kebab_case
+from vellum_ee.workflows.display.workflows.base_workflow_display import BaseWorkflowDisplay
+
+
+def push_command(
+    module: Optional[str] = None,
+    workflow_sandbox_id: Optional[str] = None,
+    deploy: Optional[bool] = None,
+    deployment_label: Optional[str] = None,
+    deployment_name: Optional[str] = None,
+    deployment_description: Optional[str] = None,
+    release_tags: Optional[List[str]] = None,
+    release_description: Optional[str] = None,
+    dry_run: Optional[bool] = None,
+    strict: Optional[bool] = None,
+    workspace: Optional[str] = None,
+) -> None:
+    load_dotenv(dotenv_path=os.path.join(os.getcwd(), ".env"))
+    logger = load_cli_logger()
+    config = load_vellum_cli_config()
+
+    workflow_configs = (
+        [
+            w
+            for w in config.workflows
+            if (module and w.module == module) or (workflow_sandbox_id and w.workflow_sandbox_id == workflow_sandbox_id)
+        ]
+        if module or workflow_sandbox_id
+        else config.workflows
+    )
+
+    if len(workflow_configs) > 1:
+        if workspace:
+            workflow_configs = [w for w in workflow_configs if w.workspace == workspace]
+        else:
+            workflow_configs = [w for w in workflow_configs if w.workspace == DEFAULT_WORKSPACE_CONFIG.name]
+
+    if len(workflow_configs) == 0:
+        if module and module_exists(module):
+            new_config = WorkflowConfig(
+                module=module,
+                workflow_sandbox_id=workflow_sandbox_id,
+                workspace=workspace or DEFAULT_WORKSPACE_CONFIG.name,
+            )
+            config.workflows.append(new_config)
+            workflow_configs = [new_config]
+        elif not module:
+            handle_cli_error(
+                logger,
+                title="No workflows found in project to push",
+                message="No workflow configurations were found in vellum.lock.json.",
+                suggestion="If you pulled a workflow with a specific sandbox ID"
+                "(e.g., vellum workflows pull --workflow-sandbox-id=<id>), "
+                "you should push without specifying a different sandbox ID:\n\n"
+                "  vellum workflows push\n\n"
+                "Or, if you want to push to a specific module:\n\n"
+                "  vellum workflows push --module=<module-name>",
+            )
+            return
+        else:
+            raise ValueError(f"No workflow config for '{module}' found in project to push.")
+
+    if len(workflow_configs) > 1:
+        raise ValueError(
+            "Multiple workflows found. Please specify a single workflow to push. "
+            f"Found: {', '.join([w.module for w in workflow_configs])}"
+        )
+
+    workflow_config = workflow_configs[0]
+
+    logger.info(f"Loading workflow from {workflow_config.module}")
+    resolved_workspace = workspace or workflow_config.workspace or DEFAULT_WORKSPACE_CONFIG.name
+    workspace_config = next((w for w in config.workspaces if w.name == resolved_workspace), None)
+
+    if workspace_config is None:
+        if resolved_workspace == DEFAULT_WORKSPACE_CONFIG.name:
+            workspace_config = DEFAULT_WORKSPACE_CONFIG
+        else:
+            available_workspaces = sorted(set([w.name for w in config.workspaces] + [DEFAULT_WORKSPACE_CONFIG.name]))
+            handle_cli_error(
+                logger,
+                title=f"Workspace '{resolved_workspace}' not found in config",
+                message=f"Available workspaces: {', '.join(available_workspaces)}",
+            )
+            return
+
+    api_key = os.getenv(workspace_config.api_key)
+    if not api_key:
+        raise ValueError(f"No API key value found in environment for workspace '{workspace_config.name}'.")
+
+    if workspace_config.name != workflow_config.workspace:
+        # We are pushing to a new workspace, so we need a new workflow config
+        workflow_config = WorkflowConfig(
+            module=workflow_config.module,
+            workspace=workspace_config.name,
+        )
+        config.workflows.append(workflow_config)
+
+    if (
+        workflow_sandbox_id
+        and workflow_config.workflow_sandbox_id
+        and workflow_config.workflow_sandbox_id != workflow_sandbox_id
+    ):
+        logger.warning(
+            f"Workflow sandbox id '{workflow_sandbox_id}' is already associated with '{workflow_config.module}'. "
+            f"Continuing with the provided workflow sandbox id '{workflow_sandbox_id}'."
+        )
+
+    client = create_vellum_client(
+        api_key=api_key,
+        api_url=workspace_config.api_url,
+    )
+    sys.path.insert(0, os.getcwd())
+
+    # Remove this once we could serialize using the artifact in Vembda
+    # https://app.shortcut.com/vellum/story/5585
+    serialization_result = BaseWorkflowDisplay.serialize_module(
+        workflow_config.module,
+        client=client,
+        dry_run=dry_run or False,
+    )
+    exec_config = serialization_result.exec_config
+
+    container_tag = workflow_config.container_image_tag
+    if workflow_config.container_image_name and not workflow_config.container_image_tag:
+        container_tag = "latest"
+
+    exec_config["runner_config"] = {
+        "sdk_version": metadata.version("vellum-ai"),  # Deprecated in favor of codegen_version
+        "codegen_version": metadata.version("vellum-ai"),
+        "container_image_tag": container_tag,
+        "container_image_name": workflow_config.container_image_name,
+    }
+
+    deployment_config: WorkflowPushDeploymentConfigRequest = OMIT
+    deployment_config_serialized: str = OMIT
+    if deploy:
+        cli_deployment_config = (
+            workflow_config.deployments[0] if workflow_config.deployments else WorkflowDeploymentConfig()
+        )
+
+        try:
+            module_name = workflow_config.module.split(".")[-1]
+            deployment_config = WorkflowPushDeploymentConfigRequest(
+                label=deployment_label or cli_deployment_config.label,
+                name=deployment_name or cli_deployment_config.name or to_kebab_case(module_name),
+                description=deployment_description or cli_deployment_config.description,
+                release_tags=release_tags or cli_deployment_config.release_tags,
+                release_description=release_description or cli_deployment_config.release_description,
+            )
+        except ValidationError as e:
+            for error in e.errors():
+                if "release_tags" in str(error.get("loc", [])):
+                    handle_cli_error(
+                        logger,
+                        title="Invalid release tag format",
+                        message="Release tags must be provided as separate arguments. "
+                        "Use: --release-tag tag1 --release-tag tag2",
+                    )
+                    return
+
+            # Re-raise if it's not a release_tags validation error
+            raise e
+
+        # We should check with fern if we could auto-serialize typed fields for us
+        # https://app.shortcut.com/vellum/story/5568
+        deployment_config_serialized = json.dumps({k: v for k, v in deployment_config.dict().items() if v is not None})
+
+    artifact = io.BytesIO()
+    with tarfile.open(fileobj=artifact, mode="w:gz") as tar:
+        module_dir = workflow_config.module.replace(".", os.path.sep)
+        for root, _, files in os.walk(module_dir):
+            for filename in files:
+                if not filename.endswith(".py"):
+                    continue
+
+                file_path = os.path.join(root, filename)
+                # Get path relative to module_dir for tar archive
+                relative_path = os.path.relpath(file_path, module_dir)
+                content_bytes = open(file_path, "rb").read()
+                file_buffer = io.BytesIO(content_bytes)
+
+                tarinfo = tarfile.TarInfo(name=relative_path)
+                tarinfo.size = len(content_bytes)
+
+                tar.addfile(tarinfo, file_buffer)
+
+    artifact.seek(0)
+    artifact.name = f"{workflow_config.module.replace('.', '__')}.tar.gz"
+
+    provided_id = workflow_config.workflow_sandbox_id or workflow_sandbox_id
+
+    try:
+        response = client.workflows.push(
+            # Remove this once we could serialize using the artifact in Vembda
+            # https://app.shortcut.com/vellum/story/5585
+            exec_config=json.dumps(exec_config),
+            workflow_sandbox_id=provided_id,
+            artifact=artifact,
+            # We should check with fern if we could auto-serialize typed object fields for us
+            # https://app.shortcut.com/vellum/story/5568
+            deployment_config=deployment_config_serialized,  # type: ignore[arg-type]
+            dry_run=dry_run,
+            strict=strict,
+        )
+    except ApiError as e:
+        if e.status_code == 404:
+            if provided_id:
+                handle_cli_error(
+                    logger,
+                    title="Workflow Sandbox not found",
+                    message=f"Could not find Workflow Sandbox with ID '{provided_id}' "
+                    f"in workspace '{resolved_workspace}'.",
+                )
+            else:
+                error_detail = e.body.get("detail") if isinstance(e.body, dict) else None
+                default_message = "The `/workflows/push` endpoint failed with a 404 response."
+                handle_cli_error(
+                    logger,
+                    title="Workflow Sandbox not found",
+                    message=error_detail or default_message,
+                )
+            return
+
+        if e.status_code == 400 and isinstance(e.body, dict) and "diffs" in e.body:
+            diffs: dict = e.body["diffs"]
+            generated_only = diffs.get("generated_only", [])
+            generated_only_str = (
+                "\n".join(
+                    ["Files that were generated but not found in the original project:"]
+                    + [f"- {file}" for file in generated_only]
+                )
+                if generated_only
+                else ""
+            )
+
+            original_only = diffs.get("original_only", [])
+            original_only_str = (
+                "\n".join(
+                    ["Files that were found in the original project but not generated:"]
+                    + [f"- {file}" for file in original_only]
+                )
+                if original_only
+                else ""
+            )
+
+            modified = diffs.get("modified", {})
+            modified_str = (
+                "\n\n".join(
+                    ["Files that were different between the original project and the generated artifact:"]
+                    + ["\n".join(line.strip() for line in lines) for lines in modified.values()]
+                )
+                if modified
+                else ""
+            )
+
+            reported_diffs = f"""\
+{e.body.get("detail")}
+
+{generated_only_str}
+
+{original_only_str}
+
+{modified_str}
+"""
+            logger.error(reported_diffs)
+            return
+
+        if e.status_code == 400 and isinstance(e.body, dict) and "detail" in e.body:
+            handle_cli_error(logger, title="API request to /workflows/push failed.", message=e.body["detail"])
+            return
+
+        raise e
+
+    if dry_run:
+        error_messages = [error.message for error in serialization_result.errors]
+        error_message = "\n".join(error_messages) if error_messages else "No errors found."
+        logger.info(
+            f"""\
+# Workflow Push Report
+
+## Errors
+{error_message}
+
+## Proposed Diffs
+{json.dumps(response.proposed_diffs, indent=2)}
+"""
+        )  # type: ignore[attr-defined]
+
+        error_list = serialization_result.errors
+        has_errors = len(error_list) > 0
+        has_diffs = response.proposed_diffs is not None and response.proposed_diffs
+
+        if has_errors or has_diffs:
+            exit(1)
+    else:
+        default_api_url = client._client_wrapper._environment.default
+        base_url = default_api_url.split("/v1")[0].replace("//api.", "//app.")
+        logger.info(
+            f"""Successfully pushed {workflow_config.module} to Vellum!
+Visit at: {base_url}/workflow-sandboxes/{response.workflow_sandbox_id}"""
+        )
+
+    if not workflow_config.workflow_sandbox_id:
+        workflow_config.workflow_sandbox_id = response.workflow_sandbox_id
+
+    if not workflow_config.deployments and response.workflow_deployment_id:
+        stored_deployment_config = WorkflowDeploymentConfig(
+            id=UUID(response.workflow_deployment_id),
+            label=deployment_config.label if deploy else None,
+            name=deployment_config.name if deploy else None,
+            description=deployment_config.description if deploy else None,
+            release_tags=deployment_config.release_tags if deploy else None,
+            release_description=getattr(deployment_config, "release_description", None) if deploy else None,
+        )
+        workflow_config.deployments.append(stored_deployment_config)
+
+    config.save()
+    logger.info("Updated vellum.lock.json file.")
+
+
+def module_exists(module_name: str) -> bool:
+    module_path = os.path.join(os.getcwd(), *module_name.split("."))
+    return os.path.exists(module_path) and os.path.isdir(module_path)

vellum_ai-1.9.7/ee/vellum_cli/tests/test_ping.py

@@ -0,0 +1,97 @@
+import pytest
+from datetime import datetime
+
+from click.testing import CliRunner
+
+from vellum.client.core.api_error import ApiError
+from vellum.client.types.environment_read import EnvironmentRead
+from vellum.client.types.organization_read import OrganizationRead
+from vellum.client.types.workspace_read import WorkspaceRead
+from vellum_cli import main as cli_main
+
+
+@pytest.mark.usefixtures("info_log_level")
+def test_ping__happy_path(vellum_client):
+    # GIVEN a cli
+    runner = CliRunner()
+
+    # AND a valid response from both the workflow and organization API calls
+    vellum_client.workspaces.workspace_identity.return_value = WorkspaceRead(
+        id="1234567890",
+        name="Test Workspace",
+        label="Test Workspace",
+        created=datetime.now(),
+    )
+    vellum_client.organizations.organization_identity.return_value = OrganizationRead(
+        id="1234567890",
+        name="Test Organization",
+        new_member_join_behavior="AUTO_ACCEPT_FROM_SHARED_DOMAIN",
+        allow_staff_access=True,
+        limit_config={},
+    )
+    vellum_client.environments.environment_identity.return_value = EnvironmentRead(
+        id="1234567890",
+        name="development",
+        label="Development",
+        display_config={},
+    )
+
+    # WHEN calling `vellum ping`
+    result = runner.invoke(cli_main, ["ping"])
+
+    # THEN it should return Status information about the user's workspace
+    assert result.exit_code == 0
+    assert (
+        result.output
+        == """\x1b[38;20m\
+Successfully authenticated with Vellum!
+
+Organization:
+    ID: 1234567890
+    Name: Test Organization
+
+Workspace:
+    ID: 1234567890
+    Name: Test Workspace
+
+Environment:
+    ID: 1234567890
+    Name: development
+    Label: Development
+\x1b[0m
+"""
+    )
+
+
+def test_ping__error_path(vellum_client):
+    # GIVEN a cli
+    runner = CliRunner()
+
+    vellum_client.workspaces.workspace_identity.side_effect = ApiError(status_code=400, body="Invalid JSON")
+
+    # WHEN the user runs the ping command
+    result = runner.invoke(cli_main, ["ping"])
+
+    # THEN the command returns an error
+    assert result.exit_code == 1
+    assert (
+        str(result.exception)
+        == "The API we tried to ping returned an invalid response. Please make sure your `VELLUM_API_URL` environment variable is set correctly."  # noqa: E501
+    )
+
+
+def test_ping__unauthorized_error_path(vellum_client):
+    # GIVEN a cli
+    runner = CliRunner()
+
+    # GIVEN an unauthorized error with the error message from the API
+    vellum_client.workspaces.workspace_identity.side_effect = ApiError(
+        status_code=403, body={"detail": "Authentication credentials were not provided."}
+    )
+
+    # WHEN the user runs the ping command
+    result = runner.invoke(cli_main, ["ping"])
+
+    # THEN the command returns an error
+    assert result.exit_code == 1
+    assert str(result.exception) == "Please make sure your `VELLUM_API_KEY` environment variable is set correctly."