google-agents-cli 0.6.0__tar.gz → 1.0.0__tar.gz

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: google-agents-cli
-Version: 0.6.0
+Version: 1.0.0
 Summary: Agents CLI — Agent Development Lifecycle toolchain for ADK
 Author-email: Google LLC <agents-cli-dev-pypi@google.com>
 Requires-Python: >=3.11
@@ -22,6 +22,8 @@ Requires-Dist: requests>=2.31.0
 Requires-Dist: a2a-sdk~=0.3.22
 Requires-Dist: tomlkit>=0.13.0,<1
 Requires-Dist: psutil>=5.9.0
+Requires-Dist: python-dotenv>=1.0.0
+Requires-Dist: pathspec>=0.12.1
 Requires-Dist: ruff ; extra == "lint"
 Requires-Dist: ty ; extra == "lint"
 Requires-Dist: codespell ; extra == "lint"
@@ -55,7 +57,7 @@ Turn your favorite coding assistant into an expert at building and deploying age
 **Agents CLI in Agent Platform** (`agents-cli`) gives your coding agent the skills and commands to build, scale, govern, and optimize enterprise-grade agents — so you don't have to learn every CLI and service yourself.
 
 **Works seamlessly with:**
-[Gemini CLI](https://github.com/google-gemini/gemini-cli) &nbsp;•&nbsp; [Claude Code](https://docs.anthropic.com/en/docs/claude-code) &nbsp;•&nbsp; [Codex](https://github.com/openai/codex) &nbsp;•&nbsp; [Antigravity](https://antigravity.google/) &nbsp;•&nbsp; *and any other coding agent.*
+[Antigravity CLI](https://antigravity.google/) &nbsp;•&nbsp; [Claude Code](https://docs.anthropic.com/en/docs/claude-code) &nbsp;•&nbsp; [Codex](https://github.com/openai/codex) &nbsp;•&nbsp; *and any other coding agent.*
 
 ## Get Started
 
@@ -78,7 +80,7 @@ npx skills add google/agents-cli
 
 ### 2. Open your coding agent
 
-Launch [Gemini CLI](https://github.com/google-gemini/gemini-cli), [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://github.com/openai/codex), or any coding agent you prefer.
+Launch [Antigravity CLI](https://antigravity.google/), [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://github.com/openai/codex), or any coding agent you prefer.
 
 ### 3. Build your first agent
 
@@ -170,7 +172,7 @@ The Google Cloud agent stack that `agents-cli` builds on:
 
 ## FAQ
 
-**Is this an alternative to Gemini CLI, Claude Code, or Codex?**<br>
+**Is this an alternative to Antigravity CLI, Claude Code, or Codex?**<br>
 No. **`agents-cli` is a tool *for* coding agents, not a coding agent itself.** It provides the CLI commands and skills that make your coding agent better at building, evaluating, and deploying ADK agents on Google Cloud.
 
 **How is this different from just using `adk` directly?**<br>

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/README.md

@@ -22,7 +22,7 @@ Turn your favorite coding assistant into an expert at building and deploying age
 **Agents CLI in Agent Platform** (`agents-cli`) gives your coding agent the skills and commands to build, scale, govern, and optimize enterprise-grade agents — so you don't have to learn every CLI and service yourself.
 
 **Works seamlessly with:**
-[Gemini CLI](https://github.com/google-gemini/gemini-cli)  •  [Claude Code](https://docs.anthropic.com/en/docs/claude-code)  •  [Codex](https://github.com/openai/codex)  •  [Antigravity](https://antigravity.google/)  •  *and any other coding agent.*
+[Antigravity CLI](https://antigravity.google/)  •  [Claude Code](https://docs.anthropic.com/en/docs/claude-code)  •  [Codex](https://github.com/openai/codex)  •  *and any other coding agent.*
 
 ## Get Started
 
@@ -45,7 +45,7 @@ npx skills add google/agents-cli
 
 ### 2. Open your coding agent
 
-Launch [Gemini CLI](https://github.com/google-gemini/gemini-cli), [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://github.com/openai/codex), or any coding agent you prefer.
+Launch [Antigravity CLI](https://antigravity.google/), [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://github.com/openai/codex), or any coding agent you prefer.
 
 ### 3. Build your first agent
 
@@ -137,7 +137,7 @@ The Google Cloud agent stack that `agents-cli` builds on:
 
 ## FAQ
 
-**Is this an alternative to Gemini CLI, Claude Code, or Codex?**<br>
+**Is this an alternative to Antigravity CLI, Claude Code, or Codex?**<br>
 No. **`agents-cli` is a tool *for* coding agents, not a coding agent itself.** It provides the CLI commands and skills that make your coding agent better at building, evaluating, and deploying ADK agents on Google Cloud.
 
 **How is this different from just using `adk` directly?**<br>

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "google-agents-cli"
-version = "0.6.0"
+version = "1.0.0"
 description = "Agents CLI — Agent Development Lifecycle toolchain for ADK"
 readme = "README.md"
 license = "Apache-2.0"
@@ -25,6 +25,8 @@ dependencies = [
   "a2a-sdk~=0.3.22",
   "tomlkit>=0.13.0,<1",
   "psutil>=5.9.0",
+  "python-dotenv>=1.0.0",
+  "pathspec>=0.12.1",
 ]
 
 [project.urls]

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-adk-code/SKILL.md

@@ -12,7 +12,7 @@ description: >
 metadata:
   author: Google
   license: Apache-2.0
-  version: 0.6.0
+  version: 1.0.0
   requires:
     bins:
       - agents-cli

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-adk-code/references/adk-python.md

@@ -298,8 +298,8 @@ class EscalationChecker(BaseAgent):
 ### Google Gemini (Default)
 
 ```python
-# AI Studio (dev)
-# Set: GOOGLE_API_KEY, GOOGLE_GENAI_USE_VERTEXAI=False
+# AI Studio (dev): in the project .env, comment the GOOGLE_* lines and
+# uncomment GEMINI_API_KEY (GOOGLE_API_KEY is also accepted).
 
 # Vertex AI (prod)
 # Set: GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION, GOOGLE_GENAI_USE_VERTEXAI=True
@@ -842,7 +842,7 @@ Sessions are ephemeral by default (`InMemorySessionService`); use `DatabaseSessi
 
 Since ambient agents have no interactive user, route outputs via structured logging (JSON stdout → Cloud Logging → Cloud Monitoring alerts), Pub/Sub, or tool-based integrations (email, Jira, Slack).
 
-**Before implementing an ambient agent, clone and study the production sample** — it covers trigger wiring, middleware, structured logging, and Terraform. See the Notable Samples table in `/google-agents-cli-workflow` Phase 1. [Full docs](https://adk.dev/runtime/ambient-agents/).
+**Before implementing an ambient agent, clone and study the production sample** — it covers trigger wiring, middleware, structured logging, and Terraform. See the Notable Samples catalog in `/google-agents-cli-workflow` Phase 1. [Full docs](https://adk.dev/runtime/ambient-agents/).
 
 ---
 

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-deploy/SKILL.md

@@ -12,7 +12,7 @@ description: >
 metadata:
   author: Google
   license: Apache-2.0
-  version: 0.6.0
+  version: 1.0.0
   requires:
     bins:
       - agents-cli
@@ -62,7 +62,7 @@ Choose the right deployment target based on your requirements:
 
 > **Ambient / scheduled / event-driven agents:** ADK's `trigger_sources` registers `/apps/{app}/trigger/*` endpoints on the same FastAPI app for **all** targets. On **Cloud Run** / **GKE** these are public HTTP routes you point a Pub/Sub push subscription or Eventarc trigger at; on **Agent Runtime** the same routes are reachable through the Agent Engine `/api` passthrough (e.g. `.../reasoningEngines/v1/{resource}/api/apps/{app}/trigger/pubsub`). Cloud Run remains the simplest target for unauthenticated trigger sources. See `/google-agents-cli-adk-code` (`references/adk-python.md`, section "12. Event-Driven / Ambient Agents") for the `trigger_sources` pattern.
 
-> **OAuth / user consent agents:** Use **Agent Runtime** with Gemini Enterprise for agents that need OAuth 2.0 user consent (e.g., accessing Google Drive, Calendar, or other user-scoped APIs). Cloud Run does not currently support managed OAuth flows. See the `adk-ae-oauth` sample in `/google-agents-cli-workflow` Phase 1.
+> **OAuth / user consent agents:** Use **Agent Runtime** with Gemini Enterprise for agents that need OAuth 2.0 user consent (e.g., accessing Google Drive, Calendar, or other user-scoped APIs). Cloud Run does not currently support managed OAuth flows. See the `adk-ae-oauth` sample in the `/google-agents-cli-workflow` sample catalog (Phase 1).
 
 ---
 
@@ -114,7 +114,6 @@ agents-cli infra single-project
 | `--min-instances` | Minimum number of instances (default: `1`) | Agent Runtime, Cloud Run |
 | `--max-instances` | Maximum number of instances (default: `10`) | Agent Runtime, Cloud Run |
 | `--concurrency` | Concurrent requests per container (default: `8`; see [Sizing a deployment](#sizing-a-deployment)) | Agent Runtime, Cloud Run |
-| `--num-workers` | Worker processes per container (default: `1`) | Agent Runtime |
 | `--port` | Container port | Cloud Run, Agent Runtime |
 | `--build-args` | Comma-separated `KEY=VALUE` Docker build args | Agent Runtime |
 | `--iap` | Enable Identity-Aware Proxy | Cloud Run |
@@ -135,22 +134,22 @@ Run `agents-cli deploy --help` for the full flag reference.
 
 ## Sizing a deployment
 
-Defaults (same on Agent Runtime, Cloud Run, and the generated `service.tf`): `--cpu 1`, `--memory 4Gi`, `--num-workers 1`, `--concurrency 8`, `--min-instances 1`, `--max-instances 10`.
+Defaults (same on Agent Runtime, Cloud Run, and the generated `service.tf`): `--cpu 1`, `--memory 4Gi`, `--concurrency 8`, `--min-instances 1`, `--max-instances 10`.
 
 The params are coupled — scale them together:
 
-- **Workers = vCPUs.** Each worker is one GIL-bound process that saturates one core, so raise `--num-workers` with `--cpu` (e.g. `--cpu 4` → `--num-workers 4`) or you pay for idle cores.
+- **One async process — scale out, not up.** The container runs a single `uvicorn` process that serves many requests concurrently on the event loop, so throughput comes from `--concurrency` and horizontal scale (`--max-instances`), not extra worker processes. Raise `--cpu` only if profiling shows the event loop or synchronous tool calls are CPU-bound.
 - **Memory bounds concurrency.** Each concurrent request keeps its full working set (context window, history, RAG chunks, response buffer) in memory while it waits on the model, so peak ≈ base + `concurrency × per-request memory`. Memory — not CPU — is the first limit, so raising `--concurrency` without `--memory` is the main OOM cause.
 - **Concurrency default is conservative.** An async worker can serve many concurrent requests while it waits on the model, but per-request memory is agent-specific, so `8` protects a memory-heavy (RAG/multimodal) agent. Light agents can raise it to 16–32+ after load-testing. See [Underutilized asynchronous workers](https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/runtime/optimize-and-scale#underutilized-workers).
 
 ```bash
 # 4x throughput: scale every param, not just one
-agents-cli deploy --cpu 4 --num-workers 4 --concurrency 16 --memory 16Gi
+agents-cli deploy --cpu 4 --concurrency 16 --memory 16Gi --max-instances 20
 ```
 
 **Tune with the scaffolded load test** (`tests/load_test/`, run locally or in the CI/CD staging pipeline): drive load, watch *max* latency and memory/OOM restarts, then adjust — high max latency → raise concurrency (+ workers/cpu); OOM → raise memory or lower concurrency.
 
-> `--num-workers` is Agent-Runtime-only (Cloud Run runs one uvicorn process). On **GKE** these flags are rejected — size via the Terraform manifests + HorizontalPodAutoscaler under `deployment/terraform/`.
+> On **GKE** these sizing flags are rejected — size via the Terraform manifests + HorizontalPodAutoscaler under `deployment/terraform/`.
 
 ---
 

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-deploy/references/agent-runtime.md

@@ -4,7 +4,9 @@
 
 ## Deployment Architecture
 
-Agent Runtime uses **container-based deployment**. `agents-cli deploy` packages your project (all git-tracked files) and Agent Engine builds a container image from your project's `Dockerfile` — the same `fast_api_app:app` image that serves Cloud Run and GKE. A `Dockerfile` at the project root is required (scaffolded projects ship one).
+Agent Runtime uses **container-based deployment**: `agents-cli deploy` packages your project and Agent Engine builds a container image from your project's `Dockerfile` (required at the project root; scaffolded projects ship one).
+
+File selection honors the project-root `.gcloudignore`, else the project-root `.gitignore` (nested `.gitignore` files are not consulted).
 
 **App object:** `fast_api_app.py` builds a single FastAPI `app` via
 `get_fast_api_app(web=True, lifespan=...)`. The lifespan builds one `Runner`
@@ -41,16 +43,17 @@ https://{location}-aiplatform.googleapis.com/reasoningEngines/v1/{resource}/api/
 
 `{agent_directory}` is the app name (the project's `agent_directory`, recorded in
 `deployment_metadata.json`). This is the exact URL `deploy` advertises on
-success, `publish` registers with Gemini Enterprise, and `run` constructs for
-`--mode a2a` against an Agent Runtime URL. All three build it from the shared
-`_agent_runtime_a2a` helper.
+success and `run` constructs for `--mode a2a` against an Agent Runtime URL — both
+authenticate with your Google credentials. On Agent Runtime, `publish` registers
+with **ADK** (`:streamQuery` against the reasoning-engine resource name), not
+this card URL.
 
 ## Deploying
 
 Deploy with `agents-cli deploy` (run `agents-cli deploy --help` for the full flag reference). CI/CD pipelines invoke the same command.
 
 **Deployment flow:**
-1. `agents-cli deploy` packages the project's git-tracked files (including the `Dockerfile`)
+1. `agents-cli deploy` packages the project files (honoring `.gcloudignore`/`.gitignore`)
 2. Agent Engine builds the container image and creates/updates the Agent Runtime instance
 3. Writes `deployment_metadata.json` with the engine resource ID
 
@@ -74,7 +77,7 @@ Written by `agents-cli deploy` after a successful deployment:
 }
 ```
 
-Used by: subsequent deploys (update vs create), `agents-cli run --url`, and `agents-cli publish` (to construct the A2A card URL). Cloud Run does not use this file.
+Used by: subsequent deploys (update vs create), `agents-cli run --url`, and `agents-cli publish` (reads the runtime ID for the default ADK registration on Agent Runtime, and constructs the A2A card URL only when A2A registration is explicitly chosen). Cloud Run does not use this file.
 
 If deployment times out but the engine was created, manually populate this file with the engine resource ID.
 

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-deploy/references/cicd-pipeline.md

@@ -73,7 +73,7 @@ The pipeline has three stages:
 
 1. **CI (PR checks)** — Triggered on pull request. Runs unit and integration tests.
 2. **Staging CD** — Triggered on merge to `main`. Builds container, deploys to staging, runs load tests.
-   > **Path filter:** Staging CD only triggers when relevant paths change — the agent directory (`app/**` by default), `data_ingestion/**`, `tests/**`, `deployment/**`, or `uv.lock`. The first push after `infra cicd` won't trigger staging CD unless one of these changes. If nothing happens after pushing, this is why.
+   > **Path filter:** Staging CD only triggers when relevant paths change — the agent directory (`app/**` by default), `tests/**`, `deployment/**`, or `uv.lock`. The first push after `infra cicd` won't trigger staging CD unless one of these changes. If nothing happens after pushing, this is why.
 3. **Production CD** — Triggered after successful staging deploy via `workflow_run`. Might require **manual approval** before deploying to production.
    > **Approving:** Go to GitHub Actions → the production workflow run → click "Review deployments" → approve the pending `production` environment. This is GitHub's environment protection rules, not a custom mechanism.
 

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-eval/SKILL.md

@@ -11,7 +11,7 @@ description: >
 metadata:
   author: Google
   license: Apache-2.0
-  version: 0.6.0
+  version: 1.0.0
   requires:
     bins:
       - agents-cli
@@ -44,7 +44,7 @@ Improving agent quality is iterative. The 5 stages below describe the loop. Each
 
 **Default:** Use or edit the scaffolded `tests/eval/datasets/basic-dataset.json` to define single-turn eval inputs. Start with 1–2 cases.
 
-**Opt-in:** `agents-cli eval dataset synthesize` — runs e2e user simulation against your live agent to synthesize multi-turn eval datasets. Prefer when testing multi-turn conversations but lacking data. Output includes traces, so you can skip Stage 2 and go directly to `eval grade`.
+**Opt-in:** `agents-cli eval dataset synthesize` — user-simulate multi-turn datasets when you lack data; its output includes traces, so skip Stage 2 and grade directly. See *Eval Commands* and `references/user-simulation.md`.
 
 ### 2. Run Inference
 
@@ -60,7 +60,7 @@ Improving agent quality is iterative. The 5 stages below describe the loop. Each
 
 **Default:** Open the latest `artifacts/grade_results/results_<ts>.html` (or `.json`) and identify failed metrics — see *What to fix when scores fail* below for the fix table.
 
-**Opt-in:** `agents-cli eval analyze` — runs LLM-based failure clustering and root-cause analysis over the grade results. Prefer when you have 10+ failing cases and want categorized failure modes instead of case-by-case reading.
+**Opt-in:** `agents-cli eval analyze` — LLM-based failure clustering; prefer when you have 10+ failing cases and want categorized failure modes. See *Eval Commands*.
 
 ### 5. Optimize & Code Fix
 
@@ -275,75 +275,11 @@ Two shapes are supported.
 }
 ```
 
-**(b) Multi-turn continuation via `agent_data`** — partial conversation, last turn ends with a user message. Use to continue an existing conversation; the agent's next response is what gets evaluated.
-
-```json
-{
-  "eval_cases": [
-    {
-      "eval_case_id": "booking_followup",
-      "agent_data": {
-        "agents": {
-          "flight_booking_agent": {
-            "agent_id": "flight_booking_agent",
-            "instruction": "You are a helpful flight booking assistant."
-          }
-        },
-        "turns": [
-          {
-            "turn_index": 0,
-            "events": [
-              {"author": "user", "content": {"parts": [{"text": "I want to book a flight to Paris."}]}},
-              {"author": "flight_booking_agent", "content": {"parts": [{"text": "I found a flight for $800. Do you want to book it?"}]}}
-            ]
-          },
-          {
-            "turn_index": 1,
-            "events": [
-              {"author": "user", "content": {"parts": [{"text": "Yes, please book it."}]}}
-            ]
-          }
-        ]
-      }
-    }
-  ]
-}
-```
+**(b) Multi-turn continuation via `agent_data`** — a partial conversation whose last turn ends with a user message; the agent's next response is evaluated. See `references/dataset_schema.md` (*Multi-Turn / Multi-Agent Dataset*) for the JSON shape.
 
 ### Grading input format (traces)
 
-Complete trace — agent responses, tool calls, and tool responses all present. Normally produced by `eval generate` or `eval dataset synthesize`; shown here so you can recognize the shape when debugging.
-
-```json
-{
-  "eval_cases": [
-    {
-      "eval_case_id": "weather_query",
-      "agent_data": {
-        "agents": {
-          "weather_agent": {
-            "agent_id": "weather_agent",
-            "instruction": "You are a helpful weather assistant."
-          }
-        },
-        "turns": [
-          {
-            "turn_index": 0,
-            "events": [
-              {"author": "user", "content": {"parts": [{"text": "What's the weather in San Francisco?"}]}},
-              {"author": "weather_agent", "content": {"parts": [{"function_call": {"name": "get_weather", "args": {"city": "San Francisco"}}}]}},
-              {"author": "weather_agent", "content": {"parts": [{"function_response": {"name": "get_weather", "response": {"temp_f": 62, "conditions": "foggy"}}}]}},
-              {"author": "weather_agent", "content": {"parts": [{"text": "It's currently 62°F and foggy in San Francisco."}]}}
-            ]
-          }
-        ]
-      }
-    }
-  ]
-}
-```
-
-**Key conventions:** authors are `"user"`, agent IDs from the `agents` map, or `"tool"`; tool calls use `function_call` parts and tool results use `function_response` parts. See `references/dataset_schema.md` for multi-agent examples and the full type reference.
+A complete trace — agent responses plus `function_call` / `function_response` parts — normally produced by `eval generate` / `eval dataset synthesize` (you don't write these by hand). Authors are `"user"`, an agent ID from the `agents` map, or `"tool"`. See `references/dataset_schema.md` for the trace shape, multi-agent examples, and the full type reference.
 
 ---
 
@@ -421,14 +357,14 @@ Each eval case runs in its own fresh in-memory session (`eval generate` creates
 
 ### Vertex eval region
 
-`eval grade`, `eval submit`, and `eval dataset synthesize` **default to the `global` endpoint** — they don't inherit the manifest `region` (the eval services support only a subset of regions). `eval analyze` is `global`-only; `eval generate` runs locally and follows the project region. So you normally don't configure anything for eval.
-
-Override per run with `--region <REGION>` (e.g. data residency); the service rejects an unsupported one:
+`eval grade` and `eval submit` **default to the `global` endpoint** — they don't inherit the manifest `region` (the eval services support only a subset of regions), and `eval analyze` is `global`-only. Override these per run with `--region <REGION>` (e.g. data residency); the service rejects an unsupported one:
 
 ```
 400 FAILED_PRECONDITION: Unsupported region for Vertex Evaluation Service: <region>
 ```
 
+`eval generate` and `eval dataset synthesize` run your agent locally, so they honor the agent's own `.env` — notably `GOOGLE_CLOUD_LOCATION`, which selects the model endpoint **when the agent uses Vertex AI** (`GOOGLE_GENAI_USE_VERTEXAI=true`); it's unused with a `GEMINI_API_KEY` (AI Studio). They take **no** `--region` and never override your `.env` with the manifest `region`; change the model region by editing `.env`. One caveat for `synthesize`: its scenario-generation step is a **server-side** eval call at `GOOGLE_CLOUD_LOCATION`, so keep that an eval-supported region (`global` by default) even though the agent itself could run elsewhere.
+
 **No eval region fits your data-residency rules?** Fall back to **local custom metrics** — a `custom_metrics` entry with a `custom_function` (`execution: local`, the default) grades in-process with no GCP region required. You lose the managed built-in metrics, but your `custom_function` can still call an LLM judge in a compliant region itself — so LLM-as-judge grading stays available anywhere.
 
 ### The `before_agent_callback` Pattern (State Initialization)

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-eval/references/metrics-guide.md

@@ -44,6 +44,8 @@ Custom metrics are declared in `eval_config.yaml` (or `.json`) under `custom_met
 
 Code-based metrics default to **local in-process execution** (no GCP project or region required); opt into the Vertex AI sandbox with `execution: "remote"`.
 
+> **Scaffolded default metric.** The scaffolded `eval_config.yaml` ships `custom_response_quality` as a local LLM-judge in `tests/eval/metrics.py` (referenced via `custom_function_file`, run in-process via `google-genai`). It grades on either backend — `genai.Client()` uses `GEMINI_API_KEY` (AI Studio) or ADC (Vertex) — and reads each case's `reference` (ground truth) when present. To grade with the managed Vertex eval service instead, replace it with a built-in metric or an `LLMMetric` (`prompt_template`).
+
 ### Example
 
 ```yaml
@@ -81,11 +83,11 @@ custom_metrics:
           return {'score': n}
 ```
 
-For datasets produced by `agents-cli eval generate` / `eval dataset synthesize`, each eval case exposes three standard fields to a metric: `{prompt}` (user message), `{response}` (final agent text, populated from the last text-bearing event), and `{agent_data}` (the full `turns`/`events` trace, used when the judge or custom function needs to reason about tool calls or intermediate steps). `{reference}` and `{context}` resolve only when the eval case has `reference` / `context` fields populated (e.g., golden-answer datasets); they are not populated by `eval generate` / `eval dataset synthesize`.
+Metrics receive the eval case's `{prompt}`, `{response}`, and `{agent_data}` (and `{reference}` / `{context}` when the case populates them) — see SKILL.md's *Evaluation Configuration Schema → Agent trace field model* for details.
 
 ### Schema reference
 
-Each entry in `custom_metrics` must conform to one of two Agent Platform evaluation metric schemas. The presence of `custom_function` selects `CodeExecutionMetric`; otherwise it's `LLMMetric`.
+Each entry in `custom_metrics` must conform to one of two Agent Platform evaluation metric schemas. The presence of `custom_function` or `custom_function_file` selects `CodeExecutionMetric`; otherwise it's `LLMMetric`.
 
 #### Code Execution Metric (`CodeExecutionMetric`)
 
@@ -94,9 +96,30 @@ Evaluates responses using custom Python code.
 | Field | Required | Description |
 |-------|----------|-------------|
 | `name` | yes | Unique identifier for the metric. |
-| `custom_function` | yes | Python source containing `def evaluate(instance):`. Receives an evaluation instance, returns a numeric score or a `{'score', 'explanation'}` dict. |
+| `custom_function` | one of | Python source containing `def evaluate(instance):`. Receives an evaluation instance, returns a numeric score or a `{'score', 'explanation'}` dict. |
+| `custom_function_file` | one of | Path to a `.py` file containing `def evaluate(instance):`, **resolved relative to the eval config file's directory** (absolute paths honored). Keeps the metric a real, lintable/testable module instead of an inline blob. Mutually exclusive with `custom_function`. Works with both `execution` modes (for `remote`, the file's source is uploaded). |
 | `execution` | no | Where the function runs. `"local"` (default) — executed in the CLI process; no GCP project or region required; **runs with the CLI's privileges**, so only use trusted code. `"remote"` — uploaded and executed inside Vertex AI's `CodeExecutionMetric` sandbox; requires a configured GCP project + region. |
 
+**Minimal `custom_function_file` example** — point the metric at a sibling `.py` file instead of an inline blob:
+
+```yaml
+# tests/eval/eval_config.yaml
+metrics_to_run:
+  - turn_count
+custom_metrics:
+  - name: turn_count
+    custom_function_file: metrics.py   # resolved next to this config file
+```
+
+```python
+# tests/eval/metrics.py  (same directory as the config)
+def evaluate(instance):
+    turns = (instance.get("agent_data") or {}).get("turns", [])
+    return {"score": len(turns)}
+```
+
+Grade with `agents-cli eval grade --config tests/eval/eval_config.yaml`.
+
 #### LLM-as-a-Judge Metric (`LLMMetric`)
 
 Evaluates responses using an LLM judge driven by a prompt template.

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-eval/references/user-simulation.md

@@ -39,7 +39,8 @@ CLI flags exposed by `agents-cli eval dataset synthesize`:
 | `--model` | Model used for **scenario generation** (server-side; not the simulated user model) |
 | `--max-turns` | Cap on user↔agent turns per scenario (default 5) |
 | `-o / --output` | Output path; defaults to `artifacts/traces/traces_<ts>.json` |
-| `--project` / `--region` | GCP project / region overrides. `synthesize` defaults to the `global` eval endpoint (ignores the manifest `region`); pass `--region` only for data residency — the service rejects an unsupported one. |
+
+`synthesize` runs your agent locally and reads its config from the agent's `.env` (the whole file — `GOOGLE_GENAI_USE_VERTEXAI`, `GEMINI_API_KEY`, `GOOGLE_CLOUD_*`, app vars); there are no `--project` / `--region` flags. On Vertex AI, `GOOGLE_CLOUD_LOCATION` also picks the endpoint for the **server-side scenario-generation** call, which only supports a subset of eval regions — keep it `global` (the scaffold default) unless you know your region is supported.
 
 **Simulator internals are NOT user-configurable from agents-cli.** The LLM-backed user simulator that plays the user side runs inside `_synthesize_runner.py` with hardcoded ADK defaults (`gemini-2.5-flash` for the user voice, default thinking config, no `custom_instructions`). Only `--max-turns` reaches it (as `LlmBackedUserSimulatorConfig.max_allowed_invocations`). There is no `eval_config.yaml` key, no `--simulator-model` flag, and no way to override `custom_instructions` or `model_configuration` short of editing `_synthesize_runner.py` directly.
 

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-observability/SKILL.md

@@ -13,7 +13,7 @@ description: >
 metadata:
   author: Google
   license: Apache-2.0
-  version: 0.6.0
+  version: 1.0.0
   requires:
     bins:
       - agents-cli

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-publish/SKILL.md

@@ -15,7 +15,7 @@ description: >
 metadata:
   author: Google
   license: Apache-2.0
-  version: 0.6.0
+  version: 1.0.0
   requires:
     bins:
       - agents-cli
@@ -40,29 +40,22 @@ metadata:
 
 ## Registration Modes
 
-### A2A Registration (default)
+### A2A Registration (Cloud Run / GKE)
 
-Every scaffolded agent serves the Agent-to-Agent protocol, so A2A is the default on **every** deployment target — Agent Runtime included. Pass the agent card URL and the command fetches the card and registers it; display name and description default to the card's `name`/`description`.
+Every scaffolded agent serves the Agent-to-Agent protocol. A2A is the default — and only — registration type on **Cloud Run** and **GKE**, which have no reasoning engine, so Gemini Enterprise registers them over A2A. Pass the agent card URL and the command fetches the card and registers it; display name and description default to the card's `name`/`description`.
 
 ```bash
-# A2A on Agent Runtime (card served through the /api passthrough)
-agents-cli publish gemini-enterprise \
-  --agent-card-url "https://us-east1-aiplatform.googleapis.com/reasoningEngines/v1/projects/123456/locations/us-east1/reasoningEngines/789/api/a2a/app/.well-known/agent-card.json" \
-  --gemini-enterprise-app-id projects/123456/locations/global/collections/default_collection/engines/my-app
-
 # A2A on Cloud Run / GKE
 agents-cli publish gemini-enterprise \
   --agent-card-url https://my-service-abc123.us-east1.run.app/a2a/app/.well-known/agent-card.json \
   --gemini-enterprise-app-id projects/123456/locations/global/collections/default_collection/engines/my-app
 ```
 
-Pass `--display-name` / `--description` to override the card defaults. On Agent Runtime, `--agent-card-url` can be omitted when `deployment_metadata.json` is present — see [Auto-Detection from Metadata](#auto-detection-from-metadata).
-
-> **OAuth authorization is not supported yet for A2A on Agent Runtime.** If your agent needs an OAuth authorization (`--authorization-id`), register it as ADK instead (`--registration-type adk`) — Gemini Enterprise invokes it natively via `:streamQuery`, which does support authorizations.
+Pass `--display-name` / `--description` to override the card defaults. For Agent Runtime, use ADK registration (below).
 
-### ADK Registration
+### ADK Registration (default on Agent Runtime)
 
-Opt in with `--registration-type adk` for Agent Runtime agents you want Gemini Enterprise to invoke natively via `:streamQuery` instead of over A2A — also the path to use when the agent needs an OAuth authorization (`--authorization-id`), which A2A on Agent Runtime does not yet support. The agent is registered directly via its reasoning engine resource name.
+This is the **default and recommended registration for Agent Runtime** deployments: Gemini Enterprise invokes the agent natively via `:streamQuery` on its reasoning engine resource, authenticating end-to-end. Under the hood, `:streamQuery` dispatches to the `AdkApp`'s `streaming_agent_run_with_events` method — when debugging an ADK invocation, search the runtime's `reasoning_engine_stderr` logs for that method name to trace the failure. It's also the path to use when the agent needs an OAuth authorization (`--authorization-id`). The agent is registered directly via its reasoning engine resource name; no agent card URL is needed.
 
 ```bash
 agents-cli publish gemini-enterprise \
@@ -124,9 +117,9 @@ agents-cli publish gemini-enterprise --interactive
 | `--display-name` | `GEMINI_DISPLAY_NAME` | Display name in Gemini Enterprise |
 | `--description` | `GEMINI_DESCRIPTION` | Agent description |
 | `--tool-description` | `GEMINI_TOOL_DESCRIPTION` | Tool description (ADK mode only, defaults to description) |
-| `--registration-type` | `REGISTRATION_TYPE` | `adk` or `a2a` (defaults to `a2a` for scaffolded agents if not set) |
+| `--registration-type` | `REGISTRATION_TYPE` | `adk` or `a2a` (defaults to `adk` on Agent Runtime, `a2a` on Cloud Run / GKE) |
 | `--agent-card-url` | `AGENT_CARD_URL` | Agent card URL for A2A registration |
-| `--deployment-target` | `DEPLOYMENT_TARGET` | `agent_runtime`, `cloud_run`, or `gke` (affects A2A auth method) |
+| `--deployment-target` | `DEPLOYMENT_TARGET` | `agent_runtime`, `cloud_run`, or `gke` (sets the default registration type — ADK on Agent Runtime, A2A on Cloud Run / GKE — and the A2A auth method) |
 | `--project-id` | `GOOGLE_CLOUD_PROJECT` | GCP project ID for billing |
 | `--project-number` | `PROJECT_NUMBER` | GCP project number (used for Gemini Enterprise lookup) |
 | `--authorization-id` | `GEMINI_AUTHORIZATION_ID` | OAuth authorization resource name |
@@ -141,11 +134,10 @@ agents-cli publish gemini-enterprise --interactive
 When `deployment_metadata.json` exists, the command automatically:
 
 - Reads the **agent runtime ID** (`remote_agent_runtime_id`)
-- Determines the **registration type**: defaults to **A2A** on every deployment target, since every scaffolded agent serves A2A. Opt into native ADK invocation (`:streamQuery`) with `--registration-type adk` on Agent Runtime.
-- Constructs the **agent card URL** for A2A registrations on Agent Runtime. The card is served through the Agent Engine `/api` passthrough, so the URL is `https://{location}-aiplatform.googleapis.com/reasoningEngines/v1/{resource}/api/a2a/{agent_directory}/.well-known/agent-card.json` — built from `remote_agent_runtime_id` and `agent_directory` in `deployment_metadata.json` (re-deploy if the metadata predates `agent_directory`).
+- Determines the **registration type**: defaults to **ADK** (native `:streamQuery`) on **Agent Runtime**, and **A2A** on **Cloud Run / GKE** (which have no reasoning engine). Override with `--registration-type`.
 - Determines the **deployment target** for authentication
 
-This means that for the simplest case (an A2A agent on Agent Runtime), you only need to provide the Gemini Enterprise app ID:
+This means that for the simplest case (an agent on Agent Runtime, registered as ADK), you only need to provide the Gemini Enterprise app ID:
 
 ```bash
 agents-cli publish gemini-enterprise \
@@ -198,6 +190,7 @@ Docs: https://docs.cloud.google.com/agent-registry/manage-agents
 | "Gemini Enterprise App ID is required" | Provide `--gemini-enterprise-app-id` or set the `ID` / `GEMINI_ENTERPRISE_APP_ID` env var |
 | Re-publishing the same agent | Registration is idempotent — re-running updates the existing registration in place instead of creating a duplicate |
 | HTTP 403 on registration | Check that your account has Discovery Engine Editor permissions on the Gemini Enterprise project |
+| Debugging ADK invocation failures on Agent Runtime | Gemini Enterprise calls the agent via the `AdkApp`'s `streaming_agent_run_with_events` method (the native `:streamQuery` contract). Grep the runtime's `reasoning_engine_stderr` logs for `streaming_agent_run_with_events` to find the underlying error |
 | "Could not fetch agent card" | Verify the agent is running and the URL is correct; for Cloud Run, ensure `gcloud auth login` is done |
 
 ---

{google_agents_cli-0.6.0 → google_agents_cli-1.0.0}/skills/google-agents-cli-scaffold/SKILL.md

@@ -12,7 +12,7 @@ description: >
 metadata:
   author: Google
   license: Apache-2.0
-  version: 0.6.0
+  version: 1.0.0
   requires:
     bins:
       - agents-cli
@@ -39,8 +39,7 @@ Use the `agents-cli` CLI to create new ADK agent projects or enhance existing on
 
 | Choice | CLI flag |
 |--------|----------|
-| RAG with vector search | `--agent agentic_rag --datastore agent_platform_vector_search` |
-| RAG with document search | `--agent agentic_rag --datastore agent_platform_search` |
+| RAG (vector or document search) | Not a scaffold flag — clone-and-study `rag-vector-search` / `rag-agent-search` (see `/google-agents-cli-workflow` Phase 1) |
 | A2A protocol | built into every ADK agent — scaffold normally (`--agent adk`) |
 | Prototype (no deployment) | `--prototype` |
 | Deployment target | `--deployment-target <agent_runtime\|cloud_run\|gke>` |
@@ -52,9 +51,8 @@ Use the `agents-cli` CLI to create new ADK agent projects or enhance existing on
 Older names → CLI values (`vertexai` SDK package name unchanged):
 
 - Agent Engine / Vertex AI Agent Engine → `--deployment-target agent_runtime`
-- Vertex AI Search / Agent Search → `--datastore agent_platform_search`
-- Vertex AI Vector Search / Vector Search → `--datastore agent_platform_vector_search`
 - Agent Engine sessions / Agent Platform Sessions → `--session-type agent_platform_sessions`
+- Vertex AI Search / Vertex AI Vector Search / RAG → clone-and-study recipe, not a flag (see `/google-agents-cli-workflow` Phase 1)
 
 ---
 
@@ -73,7 +71,7 @@ agents-cli scaffold create <project-name> \
 **Constraints:**
 - Project name must be **26 characters or less**, lowercase letters, numbers, and hyphens only.
 - Do NOT `mkdir` the project directory before running `create` — the CLI creates it automatically. If you mkdir first, `create` will fail or behave unexpectedly.
-- Auto-detect the guidance filename based on the IDE you are running in and pass `--agent-guidance-filename` accordingly (`GEMINI.md` for Gemini CLI, `CLAUDE.md` for Claude Code, `AGENTS.md` for OpenAI Codex/other).
+- Auto-detect the guidance filename based on the IDE you are running in and pass `--agent-guidance-filename` accordingly (`GEMINI.md` for Antigravity CLI, `CLAUDE.md` for Claude Code, `AGENTS.md` for OpenAI Codex/other).
 - When enhancing an existing project, check where the agent code lives. If it's not in `app/`, pass `--agent-directory <dir>` (e.g. `--agent-directory agent`). Getting this wrong causes enhance to miss or misplace files.
 
 ### Reference Files
@@ -125,7 +123,10 @@ agents-cli scaffold enhance . --cicd-runner github_actions
 | Template | Deployment | Description |
 |----------|------------|-------------|
 | `adk` | Agent Runtime, Cloud Run, GKE | Standard ADK agent (default); A2A protocol built in |
-| `agentic_rag` | Agent Runtime, Cloud Run, GKE | RAG with data ingestion pipeline; A2A protocol built in |
+
+> **RAG is a clone-and-study recipe, not a template.** Build it by studying `rag-vector-search` or
+> `rag-agent-search` and adapting the sample into your project — see `/google-agents-cli-workflow`
+> Phase 1.
 
 ---
 
@@ -165,15 +166,11 @@ After scaffolding, immediately load `/google-agents-cli-workflow` — it contain
 **Key files to customize:** `app/agent.py` (instruction, tools, model), `app/tools.py` (custom tool functions), `.env` (project ID, location, API keys).
 **Files to preserve:** `agents-cli-manifest.yaml` (CLI reads this), deployment configs under `deployment/`, `Makefile`, `app/__init__.py` (the `App(name=...)` must match the directory name — default `app`), and the generated runtime/A2A infra (`app/fast_api_app.py`, `app/app_utils/a2a.py`, `app/app_utils/services.py`, `Dockerfile`) — these wire up serving, sessions, and the built-in A2A surface; don't hand-edit them.
 
-**RAG projects (`agentic_rag`) — provision datastore first:**
-Before running `agents-cli playground` or testing your RAG agent, you must provision the datastore and ingest data:
-```bash
-agents-cli infra datastore   # Provision datastore infrastructure
-agents-cli data-ingestion    # Ingest data into the datastore
-```
-Use `infra datastore` — **not** `infra single-project`. Both provision the datastore, but `infra datastore` is faster because it skips unrelated Terraform. Without this step, the agent won't have data to search over.
-
-> **Vector Search region:** `vector_search_location` defaults to `us-central1`, separate from `region` (`us-east1`). It sets both the Vector Search collection region and the BQ ingestion dataset region, kept colocated to avoid cross-region data movement. Override per-invocation with `agents-cli data-ingestion --vector-search-location <region>`.
+**RAG projects — clone-and-study, not a template:**
+RAG isn't a scaffold option. Build it by studying `rag-vector-search` or `rag-agent-search` (see
+`/google-agents-cli-workflow` Phase 1) and adapting the sample's `app/`, `infra/terraform/`, and
+ingestion into your project. Provisioning and ingestion run from the sample's own `Makefile`
+(`make setup-infra`, `make data-ingestion`).
 
 **Verifying your agent works:** Use `agents-cli run "test prompt"` for quick smoke tests, then `agents-cli eval generate` and `agents-cli eval grade` for systematic validation. Do NOT write pytest tests that assert on LLM response content — that belongs in eval.
 
@@ -208,7 +205,7 @@ This is useful for:
 - **Agent Runtime clears session_type** — if deploying to `agent_runtime`, remove any `session_type` setting from your code
 - **Start with `--prototype`** for quick iteration — add deployment later with `enhance`
 - **Project names** must be ≤26 characters, lowercase, letters/numbers/hyphens only
-- **NEVER write A2A code from scratch** — A2A is built into every Python ADK agent (`adk`, `agentic_rag`); the A2A Python API surface (import paths, `AgentCard` schema, `to_a2a()` signature) is non-trivial and changes across versions. Scaffold normally; never hand-write the A2A surface.
+- **NEVER write A2A code from scratch** — A2A is built into every Python ADK agent (`adk`); the A2A Python API surface (import paths, `AgentCard` schema, `to_a2a()` signature) is non-trivial and changes across versions. Scaffold normally; never hand-write the A2A surface.
 
 ---