@intentsolutionsio/jeremy-vertex-engine 2.1.0

package/skills/vertex-engine-inspector/references/PRD.md

@@ -0,0 +1,69 @@
+# PRD: Vertex Engine Inspector
+
+**Version:** 2.1.0
+**Author:** Jeremy Longshore <jeremy@intentsolutions.io>
+**Status:** Active
+**Marketplace:** [tonsofskills.com](https://tonsofskills.com) by [Intent Solutions](https://intentsolutions.io)
+**Portfolio:** [jeremylongshore.com](https://jeremylongshore.com)
+
+---
+
+## Problem Statement
+
+Vertex AI Agent Engine deployments involve seven interconnected configuration surfaces (runtime, Code Execution Sandbox, Memory Bank, A2A protocol, security, performance, monitoring) that are validated manually and inconsistently. Teams deploy agents without knowing their production readiness score, leading to security gaps (unhardened IAM, missing VPC-SC), reliability issues (no alerting, stale memory), and protocol non-compliance (broken A2A endpoints). Without a systematic inspection, problems surface only after production incidents.
+
+## Target Users
+
+| User | Context | Primary Need |
+|------|---------|-------------|
+| Platform Engineer | Preparing a new Agent Engine deployment for production launch | Comprehensive readiness score with prioritized fix list before go-live |
+| Security Auditor | Reviewing IAM, VPC-SC, and encryption posture after configuration changes | Targeted security inspection confirming least-privilege and perimeter integrity |
+| SRE / On-Call Engineer | Investigating elevated error rates or latency on a deployed agent | Performance metrics retrieval with root-cause correlation (scaling, tokens, errors) |
+| DevOps Lead | Establishing baseline quality gates for agent deployments | Repeatable inspection producing consistent scores across all team deployments |
+
+## Success Criteria
+
+1. Inspect all seven categories (runtime, sandbox, memory, A2A, security, performance, monitoring) in a single invocation
+2. Generate a weighted production-readiness score (0-100%) with per-category breakdowns
+3. Produce actionable recommendations with estimated score improvement per remediation item
+4. Complete a full inspection within 5 minutes for a standard deployment
+5. Security category catches 100% of missing VPC-SC, overprivileged IAM, and unencrypted configurations
+6. A2A compliance matrix clearly shows pass/fail for each protocol endpoint
+
+## Functional Requirements
+
+1. Retrieve agent metadata via the Python SDK (`vertexai.Client().agent_engines.get()`) and parse runtime configuration
+2. Validate Code Execution Sandbox settings: TTL range (7-14 days), sandbox type (`SECURE_ISOLATED`), scoped IAM
+3. Check Memory Bank: enabled status, retention policy (min 100 memories), Firestore encryption, indexing, auto-cleanup
+4. Test A2A protocol endpoints: `/.well-known/agent-card`, `POST /v1/tasks:send`, `GET /v1/tasks/<id>`
+5. Audit security posture: IAM least-privilege, VPC-SC perimeter, Model Armor, encryption, no hardcoded credentials
+6. Query Cloud Monitoring for 24-hour metrics: error rate, latency percentiles (p50/p95/p99), token usage, cost
+7. Assess observability: dashboards, alerting policies, structured logging, OpenTelemetry, Cloud Error Reporting
+8. Calculate weighted scores and generate a prioritized recommendation list
+
+## Non-Functional Requirements
+
+- Read-only inspection: skill must not modify the inspected deployment or its configuration
+- All Agent Engine operations use the Python SDK, never gcloud CLI (no gcloud surface exists for Agent Engine)
+- Inspection must work from outside VPC-SC perimeters when access levels are properly configured
+- Output format is YAML for machine-parseability and human readability
+- Scoring weights must reflect production impact: security and reliability weighted higher than monitoring
+- Inspection must handle partial failures gracefully (skip unavailable categories, still produce report)
+- Results must be deterministic: same deployment state always produces the same score
+
+## Dependencies
+
+- `google-cloud-aiplatform[agent_engines]>=1.120.0` Python SDK
+- `gcloud` CLI authenticated for IAM and monitoring queries
+- IAM roles: `roles/aiplatform.user` and `roles/monitoring.viewer` on target project
+- Cloud Monitoring API enabled
+- `curl` for A2A endpoint testing
+
+## Out of Scope
+
+- Modifying or remediating Agent Engine configurations (inspection only, never writes)
+- Deploying new agents or updating existing deployments (handled by adk-deployment-specialist)
+- Infrastructure provisioning with Terraform (handled by adk-infra-expert)
+- Cost optimization recommendations beyond basic model selection guidance
+- Load testing or performance benchmarking (inspection uses existing metrics only)
+- Cross-project inspection (each invocation targets a single project/agent pair)

package/skills/vertex-engine-inspector/references/errors.md

@@ -0,0 +1,96 @@
+# Error Handling Reference
+
+## SDK / Authentication Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `ModuleNotFoundError: No module named 'vertexai'` | Vertex AI SDK not installed | `pip install google-cloud-aiplatform[agent_engines]>=1.120.0` |
+| `google.auth.exceptions.DefaultCredentialsError` | No application default credentials configured | Run `gcloud auth application-default login` or set `GOOGLE_APPLICATION_CREDENTIALS` |
+| `google.api_core.exceptions.PermissionDenied (403)` | Service account or user lacks required IAM roles | Grant `roles/aiplatform.user` and `roles/monitoring.viewer` on the project |
+| `google.api_core.exceptions.NotFound (404)` | Agent engine ID or resource name is incorrect | Verify the full resource name: `projects/PROJECT/locations/LOCATION/reasoningEngines/ID`. List engines with `client.agent_engines.list()` |
+| `google.api_core.exceptions.InvalidArgument (400)` | Malformed request — wrong location, invalid config | Confirm the location matches where the engine was deployed (e.g., `us-central1`) |
+
+## Agent Engine Retrieval Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `client.agent_engines.get()` returns `None` | Engine was deleted or ID is stale | Re-list engines with `client.agent_engines.list()` to find the current resource name |
+| `UNAVAILABLE` / `DEADLINE_EXCEEDED` | Transient network or service issue | Retry with exponential backoff; check [Vertex AI status](https://status.cloud.google.com/) |
+| `RESOURCE_EXHAUSTED` | Quota limit hit on Agent Engine API | Check quotas in Cloud Console under IAM & Admin > Quotas; request an increase if needed |
+
+## Common gcloud CLI Misconceptions
+
+**There is no `gcloud` CLI for Agent Engine.** The following commands do NOT exist and will fail:
+- `gcloud ai agents describe` / `gcloud ai agents list`
+- `gcloud ai reasoning-engines list`
+- `gcloud alpha ai agent-engines list`
+- `gcloud ai agents update`
+
+All Agent Engine operations must use the Python SDK:
+
+```python
+import vertexai
+
+client = vertexai.Client(project="PROJECT_ID", location="LOCATION")
+
+# List all agent engines
+for engine in client.agent_engines.list():
+    print(engine.name, engine.display_name)
+
+# Get a specific agent engine
+engine = client.agent_engines.get(
+    name="projects/PROJECT_ID/locations/LOCATION/reasoningEngines/ENGINE_ID"
+)
+
+# Create / deploy a new agent engine
+from google.adk.agents import Agent
+agent = Agent(name="my-agent", model="gemini-2.5-flash")
+engine = client.agent_engines.create(agent=agent, config={"display_name": "my-agent"})
+```
+
+## A2A Protocol Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| AgentCard endpoint returns 404 | Agent not configured for A2A protocol or wrong endpoint URL | Verify A2A is enabled in the agent config; check that `/.well-known/agent-card` path is correct |
+| Task API returns 401/403 | Missing or invalid auth token in request header | Include `Authorization: Bearer $(gcloud auth print-access-token)` header |
+| Task API returns 500 | Agent crashed while processing the task | Check Cloud Logging for agent error logs; inspect the agent's error handler |
+| Status API timeout | Long-running task with no status update mechanism | Implement streaming or polling with reasonable timeout (30-60s) |
+
+## Monitoring and Observability Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| Cloud Monitoring returns no data | Monitoring API not enabled or no recent agent traffic | Run `gcloud services enable monitoring.googleapis.com`; generate test traffic first |
+| Metrics query returns `INVALID_ARGUMENT` | Incorrect metric type string or filter syntax | Verify metric type with [Metrics Explorer](https://console.cloud.google.com/monitoring/metrics-explorer) |
+| Log queries return empty results | Wrong resource type filter or time range | Use `resource.type="aiplatform.googleapis.com/Agent"` and verify timestamps are UTC |
+| Cloud Trace shows no spans | OpenTelemetry not configured in the agent | Add Cloud Trace exporter to the agent's OpenTelemetry setup |
+
+## Security Posture Check Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| IAM policy query fails | User lacks `resourcemanager.projects.getIamPolicy` permission | Grant `roles/iam.securityReviewer` for read-only IAM inspection |
+| VPC-SC perimeter query fails | No organization-level access | VPC-SC queries require org-level permissions; ask an org admin or skip this check |
+| Model Armor status unknown | Feature not available in the agent's region | Model Armor availability varies by region; check [region support](https://cloud.google.com/vertex-ai/docs/general/locations) |
+| Secret scan false positive | Environment variable names match secret patterns | Add false positive patterns to the exclusion list in the security checker config |
+
+## Code Execution Sandbox Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| State TTL rejected (> 14 days) | Agent Engine enforces max 14-day TTL | Set `state_ttl_days` between 1 and 14 |
+| Sandbox timeout | Code execution exceeded the configured timeout | Increase timeout or optimize the executed code; check for infinite loops |
+| Sandbox OOM (out of memory) | Code execution consumed too much memory | Reduce data size processed in sandbox; increase memory limits if available |
+| Sandbox network error | Code tried to make external network calls from isolated sandbox | Sandbox is network-isolated by design; move network calls outside the sandbox |
+
+## Memory Bank Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| Memory Bank query slow (> 500ms) | Indexing not enabled or index stale | Enable indexing in Memory Bank config; rebuild index if needed |
+| Memory quota exceeded | Too many memories stored without cleanup | Enable auto-cleanup or increase max_memories limit |
+| Firestore permission denied | Agent service account lacks Firestore access | Grant `roles/datastore.user` to the agent's service account |
+
+---
+*[Tons of Skills](https://tonsofskills.com) by [Intent Solutions](https://intentsolutions.io) | [jeremylongshore.com](https://jeremylongshore.com)*

package/skills/vertex-engine-inspector/references/example-inspection-report.md

@@ -0,0 +1,50 @@
+# Example Inspection Report
+
+## Example Inspection Report
+
+```yaml
+Agent ID: gcp-deployer-agent
+Deployment Status: RUNNING
+Inspection Date: 2025-12-09
+
+Runtime Configuration:
+  Model: gemini-2.5-flash
+  Code Execution: ✅ Enabled (TTL: 14 days)
+  Memory Bank: ✅ Enabled (retention: 90 days)
+  VPC: ✅ Configured (private-vpc-prod)
+
+A2A Protocol Compliance:
+  AgentCard: ✅ Valid
+  Task API: ✅ Functional
+  Status API: ✅ Functional
+  Protocol Version: 1.0
+
+Security Posture:
+  IAM: ✅ Least privilege (score: 95%)
+  VPC-SC: ✅ Enabled
+  Model Armor: ✅ Enabled
+  Encryption: ✅ At-rest & in-transit
+  Overall: 🟢 SECURE (92%)
+
+Performance Metrics (24h):
+  Request Count: 12,450
+  Error Rate: 2.3% 🟢
+  Latency (p95): 1,850ms 🟢
+  Token Usage: 450K tokens
+  Cost Estimate: $12.50/day
+
+Production Readiness:
+  Security: 92% (28/30 points)
+  Performance: 88% (22/25 points)
+  Monitoring: 95% (19/20 points)
+  Compliance: 80% (12/15 points)
+  Reliability: 70% (7/10 points)
+
+  Overall Score: 87% 🟢 PRODUCTION READY
+
+Recommendations:
+  1. Enable multi-region deployment (reliability +10%)
+  2. Configure automated backups (compliance +5%)
+  3. Add circuit breaker pattern (reliability +5%)
+  4. Optimize memory bank indexing (performance +3%)
+```