@1aboveio/skills 0.10.0

package/skills/cicd-pipeline/cloud-debug/references/pipeline-failures.md

@@ -0,0 +1,378 @@
+# Cloud Deploy Pipeline Failure Patterns
+
+## Common Failures
+
+| Symptom | Likely Cause | Resolution |
+|---------|--------------|------------|
+| Release render fails | Invalid Kustomize/Skaffold config | Run `kubectl kustomize` locally to validate |
+| `{{.param}}` appears literally | rawYaml doesn't support Go templates | Use Kustomize overlays instead |
+| Rollout stuck in "Pending" | Waiting for approval | Approve: `gcloud deploy rollouts approve` |
+| Custom action fails | Job execution error | Check job logs with `gcloud run jobs executions logs` |
+| Verify phase fails | Test job failed | Check verify job execution and logs |
+| `Job not found` in custom action | Job not created before release | Ensure job creation step runs before release |
+| Image not found | Image tag mismatch | Verify `--images` flag matches built image |
+| `RENDER_FAILED` | Skaffold/manifest error | Check targetRenders in release describe |
+
+---
+
+## Release Creation Failures
+
+### Skaffold File Not Found
+
+**Symptom:**
+```
+Error: skaffold.yaml not found
+```
+
+**Fix:** Specify correct path in release create:
+```bash
+gcloud deploy releases create $RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --skaffold-file=path/to/skaffold.yaml
+```
+
+### Image Reference Error
+
+**Symptom:**
+```
+Error: image "my-image" not found in --images flag
+```
+
+**Fix:** Ensure image name matches skaffold.yaml artifact:
+```yaml
+# skaffold.yaml
+build:
+  artifacts:
+    - image: my-service  # This name must match --images flag
+```
+
+```bash
+gcloud deploy releases create $RELEASE \
+  --images=my-service=$REGION-docker.pkg.dev/$PROJECT/repo/my-service:$TAG
+```
+
+### Variable Interpolation in Release Name
+
+**Symptom:** Release name contains literal `${SHORT_SHA}` instead of actual value.
+
+**Fix:** Use shell variable expansion, not Cloud Build substitution:
+```bash
+# In cloudbuild.yaml bash script
+RELEASE_NAME="my-release-${SHORT_SHA}"
+gcloud deploy releases create "$RELEASE_NAME" ...
+```
+
+---
+
+## Render Failures
+
+### Kustomize Overlay Error
+
+**Symptom:**
+```
+RENDER_FAILED: kustomize build failed
+```
+
+**Diagnose:**
+```bash
+# Check render status
+gcloud deploy releases describe $RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --format="yaml(targetRenders)"
+```
+
+**Fix - Validate locally:**
+```bash
+kubectl kustomize deploy/overlays/dev
+```
+
+### Missing Kustomization File
+
+**Symptom:**
+```
+Error: unable to find kustomization.yaml
+```
+
+**Fix:** Ensure directory structure:
+```
+deploy/overlays/dev/
+├── kustomization.yaml
+└── service.yaml
+```
+
+### Go Template Not Substituting (rawYaml)
+
+**Symptom:** `{{.serviceName}}` appears literally in rendered manifest.
+
+**Cause:** Skaffold's rawYaml deployer does NOT support Go template substitution.
+
+**Fix:** Use Kustomize overlays with hardcoded values per environment instead:
+```yaml
+# skaffold.yaml - Use Kustomize, not rawYaml
+profiles:
+  - name: dev
+    manifests:
+      kustomize:
+        paths:
+          - deploy/overlays/dev
+```
+
+### Invalid YAML in Manifest
+
+**Symptom:**
+```
+Error: error parsing manifest: yaml: line X: ...
+```
+
+**Fix:** Validate YAML syntax:
+```bash
+# Check YAML validity
+python -c "import yaml; yaml.safe_load(open('service.yaml'))"
+
+# Or use yq
+yq eval '.' service.yaml
+```
+
+---
+
+## Rollout Failures
+
+### Rollout Stuck in Pending
+
+**Cause:** Target requires approval.
+
+**Check:**
+```bash
+gcloud deploy rollouts describe $ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --format="yaml(approvalState,state)"
+```
+
+**Fix:**
+```bash
+gcloud deploy rollouts approve $ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION
+```
+
+### Rollout Failed in Deploy Phase
+
+**Diagnose:**
+```bash
+# Get phase details
+gcloud deploy rollouts describe $ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --format="yaml(phases)"
+```
+
+**Common causes:**
+1. Cloud Run service deployment failed (see deploy-failures.md)
+2. Resource quota exceeded
+3. IAM permissions missing
+
+### Revision Limit Reached
+
+**Symptom:**
+```
+Error: Revision limit of 100 reached
+```
+
+**Fix:** Delete old revisions:
+```bash
+# List revisions
+gcloud run revisions list --service=$SERVICE --region=$REGION
+
+# Delete old revision
+gcloud run revisions delete $OLD_REVISION --region=$REGION
+```
+
+---
+
+## Verification Failures
+
+### Verify Job Failed
+
+**Symptom:** Rollout fails in verify phase.
+
+**Diagnose:**
+```bash
+# List job runs for the rollout
+gcloud deploy job-runs list \
+  --rollout=$ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION
+
+# Get job run details
+gcloud deploy job-runs describe $JOB_RUN \
+  --rollout=$ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION
+```
+
+**Check underlying job execution:**
+```bash
+# The job run invokes a Cloud Run job - check its execution
+gcloud run jobs executions list --job=$VERIFY_JOB --region=$REGION
+gcloud run jobs executions logs $EXECUTION --job=$VERIFY_JOB --region=$REGION
+```
+
+### Test Timeout
+
+**Symptom:** Verify phase times out.
+
+**Fix:** Increase job timeout in skaffold.yaml verify config or Cloud Run job settings.
+
+---
+
+## Custom Action Failures
+
+### Predeploy/Postdeploy Action Failed
+
+**Symptom:**
+```
+Phase "predeploy" failed: job "run-migrations" failed
+```
+
+**Diagnose:**
+```bash
+# List job runs
+gcloud deploy job-runs list \
+  --rollout=$ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --filter="jobRunId:run-migrations"
+```
+
+### Job Not Found
+
+**Symptom:**
+```
+Error: Cloud Run job "my-job-dev" not found
+```
+
+**Cause:** Job must exist before custom action tries to execute it.
+
+**Fix:** Ensure Cloud Build creates the job before creating the release:
+```yaml
+# cloudbuild.yaml
+steps:
+  - id: 'create-jobs'
+    # Create Cloud Run jobs FIRST
+
+  - id: 'create-release'
+    waitFor: ['create-jobs']
+    # Now create release that uses those jobs
+```
+
+### Custom Action Environment Variables
+
+**Note:** Custom actions receive these variables automatically:
+- `$ENV` - From target's deployParameters
+- `$CLOUD_DEPLOY_LOCATION` - Region of the deployment
+- `$CLOUD_DEPLOY_PROJECT` - Project ID
+
+```yaml
+# skaffold.yaml custom action example
+customActions:
+  - name: run-migrations
+    containers:
+      - name: migrate
+        image: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
+        command: ["/bin/sh", "-c"]
+        args: ["gcloud run jobs execute my-job-$ENV --args=migrate --region=$CLOUD_DEPLOY_LOCATION --wait"]
+```
+
+---
+
+## Recovery Commands
+
+### Retry Failed Job
+
+```bash
+gcloud deploy rollouts retry-job $ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --job-id=$JOB_ID \
+  --phase-id=$PHASE_ID
+```
+
+To find job-id and phase-id:
+```bash
+gcloud deploy rollouts describe $ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --format="yaml(phases)"
+```
+
+### Cancel Rollout
+
+```bash
+gcloud deploy rollouts cancel $ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION
+```
+
+### Rollback to Previous Release
+
+```bash
+# List releases to find previous
+gcloud deploy releases list \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --limit=5
+
+# Create rollout for previous release
+gcloud deploy rollouts create $NEW_ROLLOUT \
+  --release=$PREVIOUS_RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION
+```
+
+---
+
+## Diagnostic Commands
+
+```bash
+# Pipeline overview
+gcloud deploy delivery-pipelines describe $PIPELINE --region=$REGION
+
+# Release render details
+gcloud deploy releases describe $RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --format="yaml(renderState,targetRenders,targetArtifacts)"
+
+# All rollouts for a release
+gcloud deploy rollouts list \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION
+
+# Rollout with full phase breakdown
+gcloud deploy rollouts describe $ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --format="yaml(state,phases[].state,phases[].childRolloutJobs)"
+
+# All job runs for a rollout
+gcloud deploy job-runs list \
+  --rollout=$ROLLOUT \
+  --release=$RELEASE \
+  --delivery-pipeline=$PIPELINE \
+  --region=$REGION \
+  --format="table(name,jobRunId,state,startTime,endTime)"
+```

package/skills/cicd-pipeline/cloud-deploy/LICENSE

@@ -0,0 +1,3 @@
+MIT License
+
+Copyright (c) 2026 1AboveIO

package/skills/cicd-pipeline/cloud-deploy/SKILL.md

@@ -0,0 +1,229 @@
+---
+name: cloud-deploy
+description: >
+  Cloud Deploy + Skaffold + Kustomize conventions. Read when designing multi-environment
+  CI/CD pipelines with staged promotion, verification gates, or GitFlow branching.
+  Also use when debugging Cloud Deploy pipeline failures. NOT for simple single-container
+  deploys — use cloud-build instead.
+---
+
+# Cloud Deploy + Skaffold + Kustomize for Cloud Run
+
+**Stack:** Cloud Build | Cloud Deploy | Skaffold | Kustomize | Cloud Run | Cloud Run Jobs
+
+## Critical Rules
+
+1. **Skaffold's rawYaml deployer does NOT support Go template substitution.** The `{{.param}}` syntax and `--set` flags are ignored for Cloud Run rawYaml manifests. Use Kustomize overlays instead.
+
+2. **Use GitFlow with two pipelines:** dev branch → dev pipeline, master branch → release pipeline (staging → prod).
+
+---
+
+## GitFlow Architecture
+
+```
+dev branch push ─────→ cloudbuild.yaml ─→ {service}-pipeline-dev ─────→ dev env
+                              │                   │
+                       ({service}-job-dev)   (verify: tests)
+
+master branch push ──→ cloudbuild.yaml ─→ {service}-pipeline-release ─→ staging ───→ prod
+                              │                   │                    │           │
+                  ({service}-job-staging,prod) (verify: tests)     (auto)      (auto)
+```
+
+---
+
+## Directory Structure
+
+```
+backend/{service}/
+├── skaffold.yaml           # Profiles reference Kustomize overlays
+├── cloudbuild.yaml         # Single file with $BRANCH_NAME conditional logic
+├── clouddeploy.yaml        # Two pipelines (dev, release) and three targets
+├── Dockerfile
+├── entrypoint.sh           # Handles: serve, migrate, seed, test
+└── deploy/
+    └── overlays/
+        ├── dev/
+        │   ├── kustomization.yaml
+        │   └── service.yaml
+        ├── staging/
+        │   ├── kustomization.yaml
+        │   └── service.yaml
+        └── prod/
+            ├── kustomization.yaml
+            └── service.yaml
+```
+
+---
+
+## Next.js + Prisma Dockerfile Pattern (pnpm Monorepo)
+
+For frontend services using Next.js standalone output with Prisma:
+
+```dockerfile
+FROM node:22-alpine AS base
+RUN corepack enable && corepack prepare pnpm@9.15.0 --activate
+
+# === BUILDER ===
+FROM base AS builder
+ENV NEXT_TELEMETRY_DISABLED=1
+ENV DATABASE_URL="postgresql://user:pass@localhost:5432/db"  # dummy for prisma generate
+WORKDIR /app
+COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
+COPY packages/ ./packages/
+COPY frontend/{app}/ ./frontend/{app}/
+RUN pnpm install --frozen-lockfile
+RUN pnpm --filter @repo/database db:generate
+RUN pnpm --filter @repo/{app} build
+WORKDIR /app/frontend/{app}
+RUN mkdir -p public
+WORKDIR /app
+RUN pnpm --filter @repo/database deploy /runtime-deps
+
+# === RUNNER ===
+FROM node:22-alpine AS runner
+WORKDIR /app
+RUN addgroup --system --gid 1001 nodejs && \
+    adduser --system --uid 1001 nextjs
+COPY --from=builder --chown=nextjs:nodejs /app/frontend/{app}/.next/standalone ./
+COPY --from=builder --chown=nextjs:nodejs /app/frontend/{app}/.next/static ./frontend/{app}/.next/static
+COPY --from=builder --chown=nextjs:nodejs /app/frontend/{app}/public ./frontend/{app}/public
+COPY --from=builder --chown=nextjs:nodejs /app/packages/database/prisma ./packages/database/prisma
+COPY --from=builder --chown=nextjs:nodejs /app/packages/database/prisma.config.ts ./packages/database/prisma.config.ts
+COPY --from=builder --chown=nextjs:nodejs /app/packages/database/src/generated ./packages/database/src/generated
+COPY --from=builder /runtime-deps /runtime-deps
+RUN mkdir -p /app/node_modules/@prisma && \
+    ln -s /runtime-deps/node_modules/@prisma/client /app/node_modules/@prisma/client
+ENV PATH="/runtime-deps/node_modules/.bin:/app/node_modules/.bin:$PATH"
+ENV NODE_PATH="/runtime-deps/node_modules"
+COPY frontend/{app}/entrypoint.sh /entrypoint.sh
+RUN chmod +x /entrypoint.sh
+EXPOSE 3000
+ENV NODE_ENV=production PORT=3000 HOSTNAME="0.0.0.0"
+ENTRYPOINT ["/entrypoint.sh"]
+CMD ["serve"]
+```
+
+### Key Points
+
+| Aspect | Details |
+|--------|---------|
+| Build context | Repository root (not app directory) for monorepo package access |
+| Prisma generate | Run in builder with dummy DATABASE_URL |
+| Runtime deps | Use `pnpm deploy` to copy database package with all dependencies |
+| Symlink | Link @prisma/client so generated client can resolve runtime |
+| Cloud SQL | Support Unix sockets via `?host=/cloudsql/instance` |
+
+### Entrypoint Script
+
+```bash
+#!/bin/sh
+set -e
+
+# Construct DATABASE_URL from components if not set
+if [ -z "$DATABASE_URL" ] && [ -n "$DB_USER" ] && [ -n "$DB_PASSWORD" ] && [ -n "$DB_NAME" ]; then
+    ENCODED_PASSWORD=$(DB_PASSWORD="$DB_PASSWORD" node -p "encodeURIComponent(process.env.DB_PASSWORD)")
+    if [ -n "$DB_SOCKET" ]; then
+        export DATABASE_URL="postgresql://${DB_USER}:${ENCODED_PASSWORD}@localhost/${DB_NAME}?host=${DB_SOCKET}"
+    else
+        export DATABASE_URL="postgresql://${DB_USER}:${ENCODED_PASSWORD}@${DB_HOST:-localhost}:${DB_PORT:-5432}/${DB_NAME}"
+    fi
+fi
+
+case "$1" in
+    migrate)  cd packages/database && prisma migrate deploy ;;
+    seed)     cd packages/database && prisma db seed ;;
+    test)     # Run vitest or other test runner ;;
+    serve)    exec node frontend/{app}/server.js ;;
+    *)        exec "$@" ;;
+esac
+```
+
+---
+
+## Configuration Templates
+
+See [references/config-templates.md](references/config-templates.md) for complete
+`clouddeploy.yaml`, `cloudbuild.yaml`, `skaffold.yaml`, and trigger setup commands.
+Load when setting up a new service pipeline or adapting an existing one.
+
+---
+
+## Developer Workflow
+
+```
+1. Work on feature branch
+2. PR → dev branch (code review)
+3. Merge to dev → auto-deploy to dev env
+4. Test in dev, verify tests pass
+5. PR → master branch (release review)
+6. Merge to master → auto-deploy to staging
+7. Verify in staging, tests pass
+8. Auto-promote to prod (or manual approval if requireApproval: true)
+```
+
+---
+
+## Key Patterns
+
+| Pattern | Description |
+|---------|-------------|
+| Two pipelines | Separate pipelines for dev and release flows |
+| Single cloudbuild.yaml | `$BRANCH_NAME` conditional logic for branch-specific behavior |
+| Kustomize overlays | Environment config baked into overlays, not templates |
+| Cloud Run Jobs | Migrations, seed, tests run as jobs via custom actions |
+| Verify phase | Tests run after deploy, gate promotion to next stage |
+| Timestamp suffix | `$$TIMESTAMP` in release names to allow rebuilding same commit |
+| Shell variable escaping | `$$VAR` to escape shell variables from Cloud Build substitution |
+| pnpm deploy | Use instead of `npm install` for isolated deps (includes WASM files) |
+
+---
+
+## Common Issues
+
+### Release ALREADY_EXISTS Error
+
+Rebuilding the same commit creates a release with the same name. Fix: add timestamp suffix:
+```bash
+TIMESTAMP=$$(date +%H%M%S)
+RELEASE_NAME="{service}-release-${SHORT_SHA}-$$TIMESTAMP"
+```
+
+### Shell Variable Interpreted as Substitution
+
+Cloud Build interprets `${VAR}` as a substitution variable. Fix: use `$$VAR` for shell variables.
+
+### Template Not Substituting
+
+`{{.serviceName}}` appears literally — Skaffold rawYaml doesn't support Go templates. Fix: use Kustomize overlays.
+
+### Duplicate Container Name
+
+Multiple verify containers with the same name. Fix: use unique container names per step.
+
+### Prisma v7 Runtime Files Missing in Docker
+
+`Cannot find module '@prisma/client/runtime/query_compiler_bg.postgresql.mjs'`. Fix: use `pnpm deploy` instead of `npm install`:
+```dockerfile
+# ✅ CORRECT — copies all dependencies including WASM runtime files
+RUN pnpm --filter @repo/database deploy /runtime-deps
+```
+
+---
+
+## Debugging Commands
+
+```bash
+# List recent releases
+gcloud deploy releases list --delivery-pipeline={service}-pipeline-dev --region=$LOCATION --limit=3
+
+# Check release details
+gcloud deploy releases describe {release-name} --delivery-pipeline={service}-pipeline-{dev|release} --region=$LOCATION
+
+# Check build logs
+gcloud builds log {build-id} --region=$LOCATION
+
+# Test Kustomize locally
+kubectl kustomize deploy/overlays/dev
+```