agent-starter-pack 0.0.1b0__py3-none-any.whl

src/base_template/README.md

@@ -0,0 +1,91 @@
+# {{cookiecutter.project_name}}
+
+{{cookiecutter.agent_description}}
+
+Template generated with `googleCloudPlatform/agent-starter-pack`
+
+## Project Structure
+
+This project is organized as follows:
+
+```
+{{cookiecutter.project_name}}/
+├── app/                 # Core application code
+│   ├── agent.py         # Main agent logic
+│   ├── agent_engine_app.py # Agent Engine application logic
+│   └── utils/           # Utility functions and helpers
+├── deployment/          # Infrastructure and deployment scripts
+├── notebooks/           # Jupyter notebooks for prototyping and evaluation
+├── tests/               # Unit, integration, and load tests
+├── Makefile             # Makefile for common commands
+└── pyproject.toml       # Project dependencies and configuration
+```
+
+
+### Installation
+
+Install required packages using uv:
+
+```bash
+uv sync --extra jupyter --extra streamlit --frozen
+```
+
+### Setup
+
+If not done during the initialization, set your default Google Cloud project and Location:
+
+```bash
+export PROJECT_ID="YOUR_PROJECT_ID"
+export LOCATION="us-central1"
+gcloud config set project $PROJECT_ID
+gcloud auth application-default login
+gcloud auth application-default set-quota-project $PROJECT_ID
+```
+
+## Commands
+
+| Command              | Description                                                                                 |
+| -------------------- | ------------------------------------------------------------------------------------------- |
+{%- if cookiecutter.deployment_target == 'cloud_run' %}
+| `make playground`    | Launch local development environment with backend and frontend |
+| `make backend`       | Start backend server only |
+| `make ui`           | Launch Streamlit frontend without local backend |
+{%- elif cookiecutter.deployment_target == 'agent_engine' %}
+| `make playground`    | Launch Streamlit interface for testing agent locally and remotely |
+| `make backend`       | Deploy agent to Agent Engine service |
+{%- endif %}
+| `make test`          | Run unit and integration tests                                                              |
+| `uv run jupyter lab` | Launch Jupyter notebook                                                                     |
+
+For full command options and usage, refer to the [Makefile](Makefile).
+
+## Usage
+1. **Prototype:** Build your Generative AI Agent using the intro notebooks in `notebooks/` for guidance. Use Vertex AI Evaluation to assess performance.
+2. **Integrate:** Import your chain into the app by editing `app/agent.py`.
+3. **Test:** Explore your chain's functionality using the Streamlit playground with `make playground`. The playground offers features like chat history, user feedback, and various input types, and automatically reloads your agent on code changes.
+4. **Deploy:** Configure and trigger the CI/CD pipelines, editing tests if needed. See the [deployment section](#deployment) for details.
+5. **Monitor:** Track performance and gather insights using Cloud Logging, Tracing, and the Looker Studio dashboard to iterate on your application.
+
+## Deployment
+
+### Dev Environment
+
+You can test deployment towards a Dev Environment using the following command:
+
+```bash
+gcloud config set project <your-dev-project-id>
+make backend
+```
+
+The repository includes a Terraform configuration for the setup of the Dev Google Cloud project.
+See [deployment/README.md](deployment/README.md) for instructions.
+
+### Production Deployment
+
+The repository includes a Terraform configuration for the setup of a production Google Cloud project. Refer to [deployment/README.md](deployment/README.md) for detailed instructions on how to deploy the infrastructure and application.
+
+## Monitoring and Observability
+
+>> You can use [this Looker Studio dashboard](https://lookerstudio.google.com/c/reporting/fa742264-4b4b-4c56-81e6-a667dd0f853f/page/tEnnC) template for visualizing events being logged in BigQuery. See the "Setup Instructions" tab to getting started.
+
+The application uses OpenTelemetry for comprehensive observability with all events being sent to Google Cloud Trace and Logging for monitoring and to BigQuery for long term storage. 

src/base_template/app/utils/tracing.py

@@ -0,0 +1,143 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import logging
+from collections.abc import Sequence
+from typing import Any
+
+from google.cloud import logging as google_cloud_logging
+from google.cloud import storage
+from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
+from opentelemetry.sdk.trace import ReadableSpan
+from opentelemetry.sdk.trace.export import SpanExportResult
+
+
+class CloudTraceLoggingSpanExporter(CloudTraceSpanExporter):
+    """
+    An extended version of CloudTraceSpanExporter that logs span data to Google Cloud Logging
+    and handles large attribute values by storing them in Google Cloud Storage.
+
+    This class helps bypass the 256 character limit of Cloud Trace for attribute values
+    by leveraging Cloud Logging (which has a 256KB limit) and Cloud Storage for larger payloads.
+    """
+
+    def __init__(
+        self,
+        logging_client: google_cloud_logging.Client | None = None,
+        storage_client: storage.Client | None = None,
+        bucket_name: str | None = None,
+        debug: bool = False,
+        **kwargs: Any,
+    ) -> None:
+        """
+        Initialize the exporter with Google Cloud clients and configuration.
+
+        :param logging_client: Google Cloud Logging client
+        :param storage_client: Google Cloud Storage client
+        :param bucket_name: Name of the GCS bucket to store large payloads
+        :param debug: Enable debug mode for additional logging
+        :param kwargs: Additional arguments to pass to the parent class
+        """
+        super().__init__(**kwargs)
+        self.debug = debug
+        self.logging_client = logging_client or google_cloud_logging.Client(
+            project=self.project_id
+        )
+        self.logger = self.logging_client.logger(__name__)
+        self.storage_client = storage_client or storage.Client(project=self.project_id)
+        self.bucket_name = bucket_name or f"{self.project_id}-logs-data"
+        self.bucket = self.storage_client.bucket(self.bucket_name)
+
+    def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult:
+        """
+        Export the spans to Google Cloud Logging and Cloud Trace.
+
+        :param spans: A sequence of spans to export
+        :return: The result of the export operation
+        """
+        for span in spans:
+            span_context = span.get_span_context()
+            trace_id = format(span_context.trace_id, "x")
+            span_id = format(span_context.span_id, "x")
+            span_dict = json.loads(span.to_json())
+
+            span_dict["trace"] = f"projects/{self.project_id}/traces/{trace_id}"
+            span_dict["span_id"] = span_id
+
+            span_dict = self._process_large_attributes(
+                span_dict=span_dict, span_id=span_id
+            )
+
+            if self.debug:
+                print(span_dict)
+
+            # Log the span data to Google Cloud Logging
+            self.logger.log_struct(span_dict, severity="INFO")
+
+        # Export spans to Google Cloud Trace using the parent class method
+        return super().export(spans)
+
+    def store_in_gcs(self, content: str, span_id: str) -> str:
+        """
+        Initiate storing large content in Google Cloud Storage/
+
+        :param content: The content to store
+        :param span_id: The ID of the span
+        :return: The  GCS URI of the stored content
+        """
+        if not self.storage_client.bucket(self.bucket_name).exists():
+            logging.warning(
+                f"Bucket {self.bucket_name} not found. "
+                "Unable to store span attributes in GCS."
+            )
+            return "GCS bucket not found"
+
+        blob_name = f"spans/{span_id}.json"
+        blob = self.bucket.blob(blob_name)
+
+        blob.upload_from_string(content, "application/json")
+        return f"gs://{self.bucket_name}/{blob_name}"
+
+    def _process_large_attributes(self, span_dict: dict, span_id: str) -> dict:
+        """
+        Process large attribute values by storing them in GCS if they exceed the size
+        limit of Google Cloud Logging.
+
+        :param span_dict: The span data dictionary
+        :param trace_id: The trace ID
+        :param span_id: The span ID
+        :return: The updated span dictionary
+        """
+        attributes = span_dict["attributes"]
+        if len(json.dumps(attributes).encode()) > 255 * 1024:  # 250 KB
+            # Separate large payload from other attributes
+            attributes_payload = dict(attributes.items())
+            attributes_retain = dict(attributes.items())
+
+            # Store large payload in GCS
+            gcs_uri = self.store_in_gcs(json.dumps(attributes_payload), span_id)
+            attributes_retain["uri_payload"] = gcs_uri
+            attributes_retain["url_payload"] = (
+                f"https://storage.mtls.cloud.google.com/"
+                f"{self.bucket_name}/spans/{span_id}.json"
+            )
+
+            span_dict["attributes"] = attributes_retain
+            logging.info(
+                "Length of payload span above 250 KB, storing attributes in GCS "
+                "to avoid large log entry errors"
+            )
+
+        return span_dict

src/base_template/app/utils/typing.py

@@ -0,0 +1,115 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import uuid
+from typing import (
+    Annotated,
+    Any,
+    Literal,
+)
+
+from langchain_core.load.serializable import Serializable
+from langchain_core.messages import (
+    AIMessage,
+    HumanMessage,
+    ToolMessage,
+)
+from langchain_core.runnables import RunnableConfig
+from pydantic import (
+    BaseModel,
+    Field,
+)
+
+
+class InputChat(BaseModel):
+    """Represents the input for a chat session."""
+
+    messages: list[
+        Annotated[HumanMessage | AIMessage | ToolMessage, Field(discriminator="type")]
+    ] = Field(
+        ..., description="The chat messages representing the current conversation."
+    )
+
+{% if cookiecutter.deployment_target == 'cloud_run' %}
+class Request(BaseModel):
+    """Represents the input for a chat request with optional configuration.
+
+    Attributes:
+        input: The chat input containing messages and other chat-related data
+        config: Optional configuration for the runnable, including tags, callbacks, etc.
+    """
+
+    input: InputChat
+    config: RunnableConfig | None = None
+
+{% endif %}
+class Feedback(BaseModel):
+    """Represents feedback for a conversation."""
+
+    score: int | float
+    text: str | None = ""
+    run_id: str
+    log_type: Literal["feedback"] = "feedback"
+
+
+def ensure_valid_config(config: RunnableConfig | None) -> RunnableConfig:
+    """Ensures a valid RunnableConfig by setting defaults for missing fields."""
+    if config is None:
+        config = RunnableConfig()
+    if config.get("run_id") is None:
+        config["run_id"] = uuid.uuid4()
+    if config.get("metadata") is None:
+        config["metadata"] = {}
+    return config
+
+
+def default_serialization(obj: Any) -> Any:
+    """
+    Default serialization for LangChain objects.
+    Converts BaseModel instances to JSON strings.
+    """
+    if isinstance(obj, Serializable):
+        return obj.to_json()
+
+
+def dumps(obj: Any) -> str:
+    """
+    Serialize an object to a JSON string.
+
+    For LangChain objects (BaseModel instances), it converts them to
+    dictionaries before serialization.
+
+    Args:
+        obj: The object to serialize
+
+    Returns:
+        JSON string representation of the object
+    """
+    return json.dumps(obj, default=default_serialization)
+{% if cookiecutter.deployment_target == 'agent_engine' %}
+
+def dumpd(obj: Any) -> Any:
+    """
+    Convert an object to a JSON-serializable dict.
+    Uses default_serialization for handling BaseModel instances.
+
+    Args:
+        obj: The object to convert
+
+    Returns:
+        Dict/list representation of the object that can be JSON serialized
+    """
+    return json.loads(dumps(obj))
+{% endif %}

src/base_template/deployment/README.md

@@ -0,0 +1,123 @@
+# Deployment README.md
+
+This folder contains the infrastructure-as-code and CI/CD pipeline configurations for deploying a conversational Generative AI application on Google Cloud.
+
+The application leverages [**Terraform**](http://terraform.io) to define and provision the underlying infrastructure, while [**Cloud Build**](https://cloud.google.com/build/) orchestrates the continuous integration and continuous deployment (CI/CD) pipeline.
+
+## Deployment Workflow
+
+![Deployment Workflow](https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/deployment_workflow.png)
+
+**Description:**
+
+1. CI Pipeline (`deployment/ci/pr_checks.yaml`):
+
+   - Triggered on pull request creation/update
+   - Runs unit and integration tests
+
+2. CD Pipeline (`deployment/cd/staging.yaml`):
+
+   - Triggered on merge to `main` branch
+   - Builds and pushes application to Artifact Registry
+   - Deploys to staging environment
+   - Performs load testing
+
+3. Production Deployment (`deployment/cd/deploy-to-prod.yaml`):
+   - Triggered after successful staging deployment
+   - Requires manual approval
+   - Deploys to production environment
+
+## Setup
+
+**Prerequisites:**
+
+1. A set of Google Cloud projects:
+   - Staging project
+   - Production project
+   - CI/CD project (can be the same as staging or production)
+2. Terraform installed on your local machine
+3. Enable required APIs in the CI/CD project. This will be required for the Terraform deployment:
+
+   ```bash
+   gcloud config set project $YOUR_CI_CD_PROJECT_ID
+   gcloud services enable serviceusage.googleapis.com cloudresourcemanager.googleapis.com cloudbuild.googleapis.com secretmanager.googleapis.com
+   ```
+
+## Step-by-Step Guide
+
+1. **Create a Git Repository using your favorite Git provider (GitHub, GitLab, Bitbucket, etc.)**
+
+2. **Connect Your Repository to Cloud Build**
+   For detailed instructions, visit: [Cloud Build Repository Setup](https://cloud.google.com/build/docs/repositories#whats_next).<br>
+
+   ![Alt text](https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/connection_cb.gif)
+
+3. **Configure Terraform Variables**
+
+   - Edit [`deployment/terraform/vars/env.tfvars`](../terraform/vars/env.tfvars) with your Google Cloud settings.
+
+   | Variable               | Description                                                     | Required |
+   | ---------------------- | --------------------------------------------------------------- | :------: |
+   | prod_project_id        | **Production** Google Cloud Project ID for resource deployment. |   Yes    |
+   | staging_project_id     | **Staging** Google Cloud Project ID for resource deployment.    |   Yes    |
+   | cicd_runner_project_id | Google Cloud Project ID where CI/CD pipelines will execute.     |   Yes    |
+   | region                 | Google Cloud region for resource deployment.                    |   Yes    |
+   | host_connection_name   | Name of the host connection you created in Cloud Build          |   Yes    |
+   | repository_name        | Name of the repository you added to Cloud Build                 |   Yes    |
+
+   Other optional variables include: telemetry and feedback BigQuery dataset IDs, log filters, sink names, service account names, bucket name suffixes, artifact registry repository name, and various role assignments.
+
+4. **Deploy Infrastructure with Terraform**
+
+   - Open a terminal and navigate to the Terraform directory:
+
+   ```bash
+   cd deployment/terraform
+   ```
+
+   - Initialize Terraform:
+
+   ```bash
+   terraform init
+   ```
+
+   - Apply the Terraform configuration:
+
+   ```bash
+   terraform apply --var-file vars/env.tfvars
+   ```
+
+   - Type 'yes' when prompted to confirm
+
+After completing these steps, your infrastructure will be set up and ready for deployment!
+
+## Dev Deployment
+
+For End-to-end testing of the application, including tracing and feedback sinking to BigQuery, without the need to trigger a CI/CD pipeline.
+
+First, enable required Google Cloud APIs:
+
+```bash
+gcloud config set project <your-dev-project-id>
+gcloud services enable serviceusage.googleapis.com cloudresourcemanager.googleapis.com
+```
+
+After you edited the relative [`env.tfvars` file](../terraform/dev/vars/env.tfvars), follow the following instructions:
+
+```bash
+cd deployment/terraform/dev
+terraform init
+terraform apply --var-file vars/env.tfvars
+```
+
+Then deploy the application using the following command (from the root of the repository):
+
+```bash
+make backend
+```
+
+### End-to-end Demo video
+
+<a href="https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/template_deployment_demo.mp4">
+  <img src="https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/preview_video.png" alt="Watch the video" width="300"/>
+</a>

src/base_template/deployment/cd/deploy-to-prod.yaml

@@ -0,0 +1,98 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+steps:
+{%- if cookiecutter.data_ingestion %}
+  - name: "python:3.11"
+    id: deploy-data-ingestion-pipeline-prod
+    entrypoint: bash
+    args:
+      - -c
+      - |
+        cd data_ingestion && pip install uv --user && uv sync --frozen && \
+        uv run python data_ingestion_pipeline/submit_pipeline.py
+    env:
+      - "PIPELINE_ROOT=${_PIPELINE_GCS_ROOT}"
+      - "REGION=${_REGION}"
+      - "DATA_STORE_REGION=${_DATA_STORE_REGION}"
+      - "DATA_STORE_ID=${_DATA_STORE_ID}"
+      - "PROJECT_ID=${_PROD_PROJECT_ID}"
+      - "SERVICE_ACCOUNT=${_PIPELINE_SA_EMAIL}"
+      - "PIPELINE_NAME=${_PIPELINE_NAME}"
+      - "CRON_SCHEDULE=${_PIPELINE_CRON_SCHEDULE}"
+      - "DISABLE_CACHING=TRUE"
+      - 'PATH=/usr/local/bin:/usr/bin:~/.local/bin'
+{%- endif %}
+{%- if cookiecutter.deployment_target == 'cloud_run' %}
+   
+  - name: "gcr.io/cloud-builders/gcloud"
+    id: trigger-deployment
+    entrypoint: gcloud
+    args:
+      - "run"
+      - "deploy"
+      - "{{cookiecutter.project_name}}"
+      - "--image"
+      - "$_REGION-docker.pkg.dev/$PROJECT_ID/$_ARTIFACT_REGISTRY_REPO_NAME/$_CONTAINER_NAME"
+      - "--region"
+      - "$_REGION"
+      - "--project"
+      - $_PROD_PROJECT_ID
+      - "--min-instances"
+      - "1"
+      - "--no-cpu-throttling"
+      - "--cpu"
+      - "4"
+      - "--memory"
+      - "4Gi"
+      - "--concurrency"
+      - "40"
+      - "--service-account"
+      - "${_CLOUD_RUN_APP_SA_NAME}@${_PROD_PROJECT_ID}.iam.gserviceaccount.com"
+      - "--set-env-vars"
+      - "COMMIT_SHA=${COMMIT_SHA}{%- if cookiecutter.data_ingestion %},DATA_STORE_ID=${_DATA_STORE_ID},DATA_STORE_REGION=${_DATA_STORE_REGION}{%- endif %}"
+{%- elif cookiecutter.deployment_target == 'agent_engine' %}
+  - name: "python:3.11" 
+    id: install-dependencies
+    entrypoint: /bin/bash
+    args:
+      - "-c"
+      - |
+        pip install uv --user && uv sync --frozen
+    env:
+      - 'PATH=/usr/local/bin:/usr/bin:~/.local/bin'
+
+  - name: "python:3.11"
+    id: trigger-deployment
+    entrypoint: /bin/bash
+    args:
+      - "-c"
+      - |
+        uv export --no-hashes --no-sources --no-header --no-emit-project --frozen > .requirements.txt
+        uv run app/agent_engine_app.py \
+          --project ${_PROD_PROJECT_ID} \
+          --location ${_REGION} \
+          --set-env-vars COMMIT_SHA=${COMMIT_SHA}{%- if cookiecutter.data_ingestion %},DATA_STORE_ID=${_DATA_STORE_ID},DATA_STORE_REGION=${_DATA_STORE_REGION}{%- endif %}
+    env:
+      - 'PATH=/usr/local/bin:/usr/bin:~/.local/bin'
+{%- endif %}
+
+substitutions:
+  _PROD_PROJECT_ID: YOUR_PROD_PROJECT_ID
+  _REGION: us-central1
+
+logsBucket: gs://${PROJECT_ID}-logs-data/build-logs
+options:
+  substitutionOption: ALLOW_LOOSE
+  defaultLogsBucketBehavior: REGIONAL_USER_OWNED_BUCKET