agent-starter-pack 0.3.3__py3-none-any.whl → 0.21.0__py3-none-any.whl

src/base_template/deployment/README.md

@@ -1,126 +0,0 @@
-# Deployment README.md
-
-This folder contains the infrastructure-as-code and CI/CD pipeline configurations for deploying a conversational Generative AI application on Google Cloud.
-
-The application leverages [**Terraform**](http://terraform.io) to define and provision the underlying infrastructure, while [**Cloud Build**](https://cloud.google.com/build/) orchestrates the continuous integration and continuous deployment (CI/CD) pipeline.
-
-## Deployment Workflow
-
-![Deployment Workflow](https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/deployment_workflow.png)
-
-**Description:**
-
-1. CI Pipeline (`deployment/ci/pr_checks.yaml`):
-
-   - Triggered on pull request creation/update
-   - Runs unit and integration tests
-
-2. CD Pipeline (`deployment/cd/staging.yaml`):
-
-   - Triggered on merge to `main` branch
-   - Builds and pushes application to Artifact Registry
-   - Deploys to staging environment
-   - Performs load testing
-
-3. Production Deployment (`deployment/cd/deploy-to-prod.yaml`):
-   - Triggered after successful staging deployment
-   - Requires manual approval
-   - Deploys to production environment
-
-## Setup
-
-> **Note:** For a streamlined one-command deployment of the entire CI/CD pipeline and infrastructure using Terraform, you can use the [`agent-starter-pack setup-cicd` CLI command](https://github.com/GoogleCloudPlatform/agent-starter-pack/blob/main/docs/cli/setup_cicd.md). Currently only supporting Github.
-
-**Prerequisites:**
-
-1. A set of Google Cloud projects:
-   - Staging project
-   - Production project
-   - CI/CD project (can be the same as staging or production)
-2. Terraform installed on your local machine
-3. Enable required APIs in the CI/CD project. This will be required for the Terraform deployment:
-
-   ```bash
-   gcloud config set project $YOUR_CI_CD_PROJECT_ID
-   gcloud services enable serviceusage.googleapis.com cloudresourcemanager.googleapis.com cloudbuild.googleapis.com secretmanager.googleapis.com
-   ```
-
-## Step-by-Step Guide
-
-1. **Create a Git Repository using your favorite Git provider (GitHub, GitLab, Bitbucket, etc.)**
-
-2. **Connect Your Repository to Cloud Build**
-   For detailed instructions, visit: [Cloud Build Repository Setup](https://cloud.google.com/build/docs/repositories#whats_next).<br>
-
-   ![Alt text](https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/connection_cb.gif)
-
-3. **Configure Terraform Variables**
-
-   - Edit [`deployment/terraform/vars/env.tfvars`](../terraform/vars/env.tfvars) with your Google Cloud settings.
-
-   | Variable               | Description                                                     | Required |
-   | ---------------------- | --------------------------------------------------------------- | :------: |
-   | project_name           | Project name used as a base for resource naming                 |   Yes    |
-   | prod_project_id        | **Production** Google Cloud Project ID for resource deployment. |   Yes    |
-   | staging_project_id     | **Staging** Google Cloud Project ID for resource deployment.    |   Yes    |
-   | cicd_runner_project_id | Google Cloud Project ID where CI/CD pipelines will execute.     |   Yes    |
-   | region                 | Google Cloud region for resource deployment.                    |   Yes    |
-   | host_connection_name   | Name of the host connection you created in Cloud Build          |   Yes    |
-   | repository_name        | Name of the repository you added to Cloud Build                 |   Yes    |
-
-   Other optional variables may include: telemetry and feedback log filters, service account roles, and for projects requiring data ingestion: pipeline cron schedule, pipeline roles, and datastore-specific configurations.
-
-4. **Deploy Infrastructure with Terraform**
-
-   - Open a terminal and navigate to the Terraform directory:
-
-   ```bash
-   cd deployment/terraform
-   ```
-
-   - Initialize Terraform:
-
-   ```bash
-   terraform init
-   ```
-
-   - Apply the Terraform configuration:
-
-   ```bash
-   terraform apply --var-file vars/env.tfvars
-   ```
-
-   - Type 'yes' when prompted to confirm
-
-After completing these steps, your infrastructure will be set up and ready for deployment!
-
-## Dev Deployment
-
-For End-to-end testing of the application, including tracing and feedback sinking to BigQuery, without the need to trigger a CI/CD pipeline.
-
-First, enable required Google Cloud APIs:
-
-```bash
-gcloud config set project <your-dev-project-id>
-gcloud services enable serviceusage.googleapis.com cloudresourcemanager.googleapis.com
-```
-
-After you edited the relative [`env.tfvars` file](../terraform/dev/vars/env.tfvars), follow the following instructions:
-
-```bash
-cd deployment/terraform/dev
-terraform init
-terraform apply --var-file vars/env.tfvars
-```
-
-Then deploy the application using the following command (from the root of the repository):
-
-```bash
-make backend
-```
-
-### End-to-end Demo video
-
-<a href="https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/template_deployment_demo.mp4">
-  <img src="https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/preview_video.png" alt="Watch the video" width="300"/>
-</a>

src/base_template/deployment/cd/staging.yaml

@@ -1,216 +0,0 @@
-# Copyright 2025 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-steps:
-{%- if cookiecutter.data_ingestion %}
-  - name: "python:3.11-slim"
-    id: deploy-data-ingestion-pipeline-staging
-    entrypoint: bash
-    args:
-      - -c
-      - |
-        cd data_ingestion && pip install uv==0.6.12 --user && uv sync --frozen && \
-        uv run python data_ingestion_pipeline/submit_pipeline.py
-    env:
-      - "PIPELINE_ROOT=${_PIPELINE_GCS_ROOT}"
-      - "REGION=${_REGION}"
-{%- if cookiecutter.datastore_type == "vertex_ai_search" %}
-      - "DATA_STORE_REGION=${_DATA_STORE_REGION}"
-      - "DATA_STORE_ID=${_DATA_STORE_ID}"
-{%- elif cookiecutter.datastore_type == "vertex_ai_vector_search" %}
-      - "VECTOR_SEARCH_INDEX=${_VECTOR_SEARCH_INDEX}"
-      - "VECTOR_SEARCH_INDEX_ENDPOINT=${_VECTOR_SEARCH_INDEX_ENDPOINT}"
-      - "VECTOR_SEARCH_BUCKET=${_VECTOR_SEARCH_BUCKET}"
-{%- endif %}
-      - "PROJECT_ID=${_STAGING_PROJECT_ID}"
-      - "SERVICE_ACCOUNT=${_PIPELINE_SA_EMAIL}"
-      - "PIPELINE_NAME=${_PIPELINE_NAME}"
-      - 'PATH=/usr/local/bin:/usr/bin:~/.local/bin'
-{%- endif %}
-{%- if cookiecutter.deployment_target == 'cloud_run' %}
-  # Build and Push
-  - name: "gcr.io/cloud-builders/docker"
-    args:
-      [
-        "build",
-        "-t",
-        "$_REGION-docker.pkg.dev/$PROJECT_ID/$_ARTIFACT_REGISTRY_REPO_NAME/$_CONTAINER_NAME",
-        ".",
-      ]
-  - name: "gcr.io/cloud-builders/docker"
-    args:
-      [
-        "push",
-        "$_REGION-docker.pkg.dev/$PROJECT_ID/$_ARTIFACT_REGISTRY_REPO_NAME/$_CONTAINER_NAME",
-      ]
-
-  # Deploy to Staging
-  - name: "gcr.io/cloud-builders/gcloud"
-    id: deploy-staging
-    entrypoint: gcloud
-    args:
-      - "run"
-      - "deploy"
-      - "{{cookiecutter.project_name}}"
-      - "--image"
-      - "$_REGION-docker.pkg.dev/$PROJECT_ID/$_ARTIFACT_REGISTRY_REPO_NAME/$_CONTAINER_NAME"
-      - "--region"
-      - "${_REGION}"
-      - "--project"
-      - "${_STAGING_PROJECT_ID}"
-      - "--min-instances"
-      - "1"
-      - "--no-cpu-throttling"
-      - "--cpu"
-      - "4"
-      - "--memory"
-      - "4Gi"
-      - "--concurrency"
-      - "40"
-      - "--service-account"
-      - "${_CLOUD_RUN_APP_SA_EMAIL}"
-      - "--set-env-vars"
-      - "COMMIT_SHA=${COMMIT_SHA}{%- if cookiecutter.data_ingestion %}{%- if cookiecutter.datastore_type == "vertex_ai_search" %},DATA_STORE_ID=${_DATA_STORE_ID},DATA_STORE_REGION=${_DATA_STORE_REGION}{%- elif cookiecutter.datastore_type == "vertex_ai_vector_search" %},VECTOR_SEARCH_INDEX=${_VECTOR_SEARCH_INDEX},VECTOR_SEARCH_INDEX_ENDPOINT=${_VECTOR_SEARCH_INDEX_ENDPOINT},VECTOR_SEARCH_BUCKET=${_VECTOR_SEARCH_BUCKET}{%- endif %}{%- endif %}"
-
-  # Fetch Staging Service URL
-  - name: "gcr.io/cloud-builders/gcloud"
-    id: fetch-staging-url
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-        echo $(gcloud run services describe {{cookiecutter.project_name}} \
-        --region ${_REGION} --project ${_STAGING_PROJECT_ID} --format="value(status.url)") > staging_url.txt
-
-  # Fetch ID Token
-  - name: gcr.io/cloud-builders/gcloud
-    id: fetch-id-token
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-        echo $(gcloud auth print-identity-token -q) > id_token.txt
-{%- elif cookiecutter.deployment_target == 'agent_engine' %}
-  - name: "python:3.11-slim" 
-    id: install-dependencies
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-        pip install uv==0.6.12 --user && uv sync --frozen
-    env:
-      - 'PATH=/usr/local/bin:/usr/bin:~/.local/bin'
-
-  - name: "python:3.11-slim"
-    id: deploy-staging
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-        uv export --no-hashes --no-sources --no-header --no-dev --no-emit-project --no-annotate --frozen > .requirements.txt
-        uv run app/agent_engine_app.py \
-          --project ${_STAGING_PROJECT_ID} \
-          --location ${_REGION} \
-          --set-env-vars="COMMIT_SHA=${COMMIT_SHA}{%- if cookiecutter.data_ingestion %}{%- if cookiecutter.datastore_type == "vertex_ai_search" %},DATA_STORE_ID=${_DATA_STORE_ID},DATA_STORE_REGION=${_DATA_STORE_REGION}{%- elif cookiecutter.datastore_type == "vertex_ai_vector_search" %},VECTOR_SEARCH_INDEX=${_VECTOR_SEARCH_INDEX},VECTOR_SEARCH_INDEX_ENDPOINT=${_VECTOR_SEARCH_INDEX_ENDPOINT},VECTOR_SEARCH_BUCKET=${_VECTOR_SEARCH_BUCKET}{%- endif %}{%- endif %}"
-    env:
-      - 'PATH=/usr/local/bin:/usr/bin:~/.local/bin'
-
-
-  - name: gcr.io/cloud-builders/gcloud
-    id: fetch-auth-token
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-        echo $(gcloud auth print-access-token -q) > auth_token.txt
-{%- endif %}
-
-  # Load Testing
-  - name: "python:3.11-slim"
-    id: load_test
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-{%- if cookiecutter.deployment_target == 'cloud_run' %}
-        export _ID_TOKEN=$(cat id_token.txt)
-        export _STAGING_URL=$(cat staging_url.txt)
-        pip install uv==0.6.12 --user && uv sync --frozen
-{%- elif cookiecutter.deployment_target == 'agent_engine' %}
-        export _AUTH_TOKEN=$(cat auth_token.txt)
-{%- endif %}
-        uv add locust==2.32.6
-        uv run locust -f tests/load_test/load_test.py \
-        --headless \
-{%- if cookiecutter.deployment_target == 'cloud_run' %}
-        -H $$_STAGING_URL \
-        -t 30s -u 10 -r 0.5 \
-{%- elif cookiecutter.deployment_target == 'agent_engine' %}
-        -t 30s -u 2 -r 0.5 \
-{%- endif %}
-        --csv=tests/load_test/.results/results \
-        --html=tests/load_test/.results/report.html
-    env:
-      - 'PATH=/usr/local/bin:/usr/bin:~/.local/bin'
-
-  # Export Load Test Results to GCS
-  - name: gcr.io/cloud-builders/gcloud
-    id: export-results-to-gcs
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-        export _TIMESTAMP=$(date +%Y%m%d-%H%M%S)
-        gsutil -m cp -r tests/load_test/.results gs://${_BUCKET_NAME_LOAD_TEST_RESULTS}/results-$${_TIMESTAMP}
-        echo "_________________________________________________________________________"
-        echo "Load test results copied to gs://${_BUCKET_NAME_LOAD_TEST_RESULTS}/results-$${_TIMESTAMP}"
-        echo "HTTP link: https://console.cloud.google.com/storage/browser/${_BUCKET_NAME_LOAD_TEST_RESULTS}/results-$${_TIMESTAMP}"
-        echo "_________________________________________________________________________"
-
-  # Trigger Prod Deployment
-  - name: gcr.io/cloud-builders/gcloud
-    id: trigger-prod-deployment
-    entrypoint: gcloud
-    args:
-      - "beta"
-      - "builds"
-      - "triggers"
-      - "run"
-      - "deploy-{{cookiecutter.project_name}}"
-      - "--region"
-      - "$LOCATION"
-      - "--project"
-      - "$PROJECT_ID"
-      - "--sha"
-      - $COMMIT_SHA
-
-  - name: gcr.io/cloud-builders/gcloud
-    id: echo-view-build-trigger-link
-    entrypoint: /bin/bash
-    args:
-      - "-c"
-      - |
-        echo "_________________________________________________________________________"
-        echo "Production deployment triggered. View progress and / or approve on the Cloud Build Console:"
-        echo "https://console.cloud.google.com/cloud-build/builds;region=$LOCATION"
-        echo "_________________________________________________________________________"
-
-substitutions:
-  _STAGING_PROJECT_ID: YOUR_STAGING_PROJECT_ID
-  _REGION: us-central1
-
-logsBucket: gs://${PROJECT_ID}-{{ cookiecutter.project_name | replace('_', '-') }}-logs-data/build-logs
-options:
-  substitutionOption: ALLOW_LOOSE
-  defaultLogsBucketBehavior: REGIONAL_USER_OWNED_BUCKET

src/base_template/deployment/terraform/dev/log_sinks.tf

@@ -1,63 +0,0 @@
-# Copyright 2025 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-resource "google_project_iam_member" "bigquery_data_editor" {
-
-  project = var.dev_project_id
-  role    = "roles/bigquery.dataEditor"
-  member  = module.log_export_to_bigquery.writer_identity
-}
-
-resource "google_bigquery_dataset" "feedback_dataset" {
-  project       = var.dev_project_id
-  dataset_id    = replace("${var.project_name}_feedback", "-", "_")
-  friendly_name = "${var.project_name}_feedback"
-  location      = var.region
-  depends_on    = [resource.google_project_service.services]
-}
-
-resource "google_bigquery_dataset" "telemetry_logs_dataset" {
-  project       = var.dev_project_id
-  dataset_id    = replace("${var.project_name}_telemetry", "-", "_")
-  friendly_name = "${var.project_name}_telemetry"
-  location      = var.region
-  depends_on    = [resource.google_project_service.services]
-}
-
-module "feedback_export_to_bigquery" {
-  source                 = "terraform-google-modules/log-export/google"
-  version                = "10.0.0"
-  log_sink_name          = "${var.project_name}_feedback"
-  parent_resource_type   = "project"
-  parent_resource_id     = var.dev_project_id
-  destination_uri        = "bigquery.googleapis.com/projects/${var.dev_project_id}/datasets/${google_bigquery_dataset.feedback_dataset.dataset_id}"
-  filter                 = var.feedback_logs_filter
-  bigquery_options       = { use_partitioned_tables = true }
-  unique_writer_identity = true
-  depends_on             = [google_bigquery_dataset.feedback_dataset]
-}
-
-module "log_export_to_bigquery" {
-  source  = "terraform-google-modules/log-export/google"
-  version = "10.0.0"
-
-  log_sink_name          = "${var.project_name}_telemetry"
-  parent_resource_type   = "project"
-  parent_resource_id     = var.dev_project_id
-  destination_uri        = "bigquery.googleapis.com/projects/${var.dev_project_id}/datasets/${google_bigquery_dataset.telemetry_logs_dataset.dataset_id}"
-  filter                 = var.telemetry_logs_filter
-  bigquery_options       = { use_partitioned_tables = true }
-  unique_writer_identity = true
-  depends_on             = [google_bigquery_dataset.telemetry_logs_dataset]
-}

src/base_template/deployment/terraform/log_sinks.tf

@@ -1,70 +0,0 @@
-# Copyright 2025 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-resource "google_project_iam_member" "bigquery_data_editor" {
-  for_each = local.deploy_project_ids
-
-  project = each.value
-  role    = "roles/bigquery.dataEditor"
-  member  = module.log_export_to_bigquery[each.key].writer_identity
-}
-
-resource "google_bigquery_dataset" "feedback_dataset" {
-  for_each      = local.deploy_project_ids
-  project       = each.value
-  dataset_id    = replace("${var.project_name}_feedback", "-", "_")
-  friendly_name = "${var.project_name}_feedback"
-  location      = var.region
-  depends_on    = [resource.google_project_service.cicd_services, resource.google_project_service.shared_services]
-}
-
-resource "google_bigquery_dataset" "telemetry_logs_dataset" {
-  for_each      = local.deploy_project_ids
-  project       = each.value
-  dataset_id    = replace("${var.project_name}_telemetry", "-", "_")
-  friendly_name = "${var.project_name}_telemetry"
-  location      = var.region
-  depends_on    = [resource.google_project_service.cicd_services, resource.google_project_service.shared_services]
-}
-
-module "log_export_to_bigquery" {
-  for_each = local.deploy_project_ids
-
-  source  = "terraform-google-modules/log-export/google"
-  version = "10.0.0"
-
-  log_sink_name          = "${var.project_name}_telemetry"
-  parent_resource_type   = "project"
-  parent_resource_id     = each.value
-  destination_uri        = "bigquery.googleapis.com/projects/${each.value}/datasets/${google_bigquery_dataset.telemetry_logs_dataset[each.key].dataset_id}"
-  filter                 = var.telemetry_logs_filter
-  bigquery_options       = { use_partitioned_tables = true }
-  unique_writer_identity = true
-  depends_on             = [google_bigquery_dataset.telemetry_logs_dataset]
-}
-
-module "feedback_export_to_bigquery" {
-  for_each = local.deploy_project_ids
-
-  source                 = "terraform-google-modules/log-export/google"
-  version                = "10.0.0"
-  log_sink_name          = "${var.project_name}_feedback"
-  parent_resource_type   = "project"
-  parent_resource_id     = each.value
-  destination_uri        = "bigquery.googleapis.com/projects/${each.value}/datasets/${google_bigquery_dataset.feedback_dataset[each.key].dataset_id}"
-  filter                 = var.feedback_logs_filter
-  bigquery_options       = { use_partitioned_tables = true }
-  unique_writer_identity = true
-  depends_on             = [google_bigquery_dataset.feedback_dataset]
-}

src/base_template/deployment/terraform/providers.tf

@@ -1,37 +0,0 @@
-# Copyright 2025 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-terraform {
-  required_version = ">= 1.0.0"
-  required_providers {
-    google = {
-      source  = "hashicorp/google"
-      version = "< 7.0.0"
-    }
-  }
-}
-
-provider "google" {
-  alias                 = "staging_billing_override"
-  billing_project       = var.staging_project_id
-  region = var.region
-  user_project_override = true
-}
-
-provider "google" {
-  alias                 = "prod_billing_override"
-  billing_project       = var.prod_project_id
-  region = var.region
-  user_project_override = true
-}