cpflow 5.1.0 → 5.2.0

data/README.md

@@ -1,3 +1,7 @@
+<p align="center">
+  <img src="./docs/assets/logo/icon-tile.svg" alt="Control Plane Flow (cpflow) logo" width="160" height="160" />
+</p>
+
 # The power of Kubernetes with the ease of Heroku!
 
 <meta name="author" content="Justin Gordon and Sergey Tarasov" />
@@ -30,7 +34,7 @@ To bootstrap a new project, run three commands from the repo root:
 
 The generated scaffold is a starting point. After generation, adapt `.controlplane/` for app-specific workloads (Sidekiq, Node renderer), wire any private-dependency Docker build settings (SSH key, optional known-host overrides), and verify that the production Docker build succeeds.
 
-See [CI automation](./docs/ci-automation.md) for the full setup and required GitHub secrets and variables. For an AI agent rollout, see the [AI rollout prompt](./docs/ai-github-flow-prompt.md) or run `cpflow ai-github-flow-prompt` inside the target repo to print a copy-paste prompt with that repo's default app prefix filled in.
+See [CI automation](./docs/ci-automation.md) for the full setup and required GitHub secrets and variables. For an AI agent rollout, copy the [AI rollout prompt](./docs/ai-github-flow-prompt.md). If `cpflow` is already available in the target repo, `cpflow ai-github-flow-prompt` prints the same prompt with that repo's default app prefix filled in.
 
 For a live reference, see the [demo app](https://github.com/shakacode/react-webpack-rails-tutorial/tree/master/.controlplane) and its [GitHub Actions flow](https://github.com/shakacode/react-webpack-rails-tutorial/tree/master/.github).
 Here is a brief [video overview](https://www.youtube.com/watch?v=llaQoAV_6Iw).
@@ -61,11 +65,13 @@ Additionally, the documentation includes numerous examples and practical tips fo
 10. [Scheduled Jobs](#scheduled-jobs)
 11. [CLI Commands Reference](#cli-commands-reference)
 12. [Mapping of Heroku Commands to `cpflow` and `cpln`](#mapping-of-heroku-commands-to-cpflow-and-cpln)
-13. [Examples](#examples)
-14. [Migrating Postgres Database from Heroku Infrastructure](https://www.shakacode.com/control-plane-flow/docs/postgres/)
-15. [Migrating Redis Database from Heroku Infrastructure](https://www.shakacode.com/control-plane-flow/docs/redis/)
-16. [Tips](https://www.shakacode.com/control-plane-flow/docs/tips/)
-17. [Thruster HTTP/2 Proxy on Control Plane](https://www.shakacode.com/control-plane-flow/docs/thruster/)
+13. [Telemetry](https://www.shakacode.com/control-plane-flow/docs/telemetry/)
+14. [Examples](#examples)
+15. [Migrating Postgres Database from Heroku Infrastructure](https://www.shakacode.com/control-plane-flow/docs/postgres/)
+16. [Migrating Redis Database from Heroku Infrastructure](https://www.shakacode.com/control-plane-flow/docs/redis/)
+17. [Tips](https://www.shakacode.com/control-plane-flow/docs/tips/)
+18. [Thruster HTTP/2 Proxy on Control Plane](https://www.shakacode.com/control-plane-flow/docs/thruster/)
+19. [Grafana and OpenTelemetry on Control Plane](https://www.shakacode.com/control-plane-flow/docs/grafana-opentelemetry/)
 
 ## Key Features
 
@@ -262,9 +268,19 @@ aliases:
     # On the other hand, if you have a workload for Redis, that would NOT use the application Docker image
     # and not be listed here.
     app_workloads:
+      - node-renderer
       - rails
       - sidekiq
 
+    # Optional ordered deploy groups for `cpflow deploy-image` and
+    # `cpflow promote-app-from-upstream`. Each group is deployed and waited on
+    # before the next group starts. Any app workloads omitted here deploy last
+    # as an implicit final group. Explicit `deploy-image -w/--workload` options
+    # override this ordering for one-off deploys.
+    deploy_order:
+      - [node-renderer]
+      - [rails, sidekiq]
+
     # Additional "service type" workloads, using non-application Docker images.
     # These are only used by the `info` and `ps:` commands in order to get all of the defined workloads.
     additional_workloads:
@@ -322,6 +338,8 @@ apps:
       post_creation: bundle exec rake db:prepare
 
       # Used by the command `cpflow delete` to run a hook before deleting the app.
+      # For a shared database, prefer admin-side cleanup instead: `cpflow delete` runs this hook before removing the
+      # workloads, so live connections can block the drop. See docs/tips.md ("Share One Control Plane Postgres").
       pre_deletion: bundle exec rake db:drop
 
   my-app-production:
@@ -375,7 +393,7 @@ cpflow generate-github-actions
 bin/test-cpflow-github-flow
 ```
 
-`cpflow github-flow-readiness` exits non-zero when it finds blockers such as unpublished exact-pinned packages or a missing production Dockerfile, so use it as the gate before generation. Then review the generated `.controlplane/controlplane.yml` entries, adjust any app-specific workloads, and configure the GitHub repository variables and secrets described in [CI automation](./docs/ci-automation.md), including the optional Docker build settings for private GitHub dependencies and custom SSH known hosts. `cpflow generate-github-actions` also writes `bin/test-cpflow-github-flow` for local validation and `bin/pin-cpflow-github-ref` for temporarily pinning downstream wrappers to an upstream commit SHA during pre-release testing. `cpflow generate` already switches to persistent `db` and `storage` volumes when `config/database.yml` shows SQLite in production and preserves detected frontend precompile hooks, but you should still confirm that the generated Dockerfile picked a Ruby base image compatible with the app's declared Ruby requirement and that the emitted workload set matches the real app. If you want an AI agent to do this end to end, start with the [AI rollout prompt](./docs/ai-github-flow-prompt.md) or run `cpflow ai-github-flow-prompt` in the target repo rather than giving a vague "set up CI" request.
+`cpflow github-flow-readiness` exits non-zero when it finds blockers such as unpublished exact-pinned packages or a missing production Dockerfile, so use it as the gate before generation. Then review the generated `.controlplane/controlplane.yml` entries, adjust any app-specific workloads, and configure the GitHub repository variables and secrets described in [CI automation](./docs/ci-automation.md), including the optional Docker build settings for private GitHub dependencies and custom SSH known hosts. `cpflow generate-github-actions` also writes `bin/test-cpflow-github-flow` for local validation and `bin/pin-cpflow-github-ref` for temporarily pinning downstream wrappers to an upstream commit SHA during pre-release testing. `cpflow generate` already switches to persistent `db` and `storage` volumes when `config/database.yml` shows SQLite in production and preserves detected frontend precompile hooks, but you should still confirm that the generated Dockerfile picked a Ruby base image compatible with the app's declared Ruby requirement and that the emitted workload set matches the real app. If you want an AI agent to do this end to end, give it the [AI rollout prompt](./docs/ai-github-flow-prompt.md) rather than a vague "set up CI" request.
 
 For a live example, see the [react-webpack-rails-tutorial](https://github.com/shakacode/react-webpack-rails-tutorial/blob/master/.controlplane/readme.md) repository.
 

data/docs/ai-github-flow-prompt.md

@@ -1,32 +1,34 @@
 # AI Rollout Prompt for Control Plane GitHub Flow
 
-Use this file when you want an AI agent to add the reusable `cpflow` review-app,
-staging, and production-promotion flow to a repository.
-
-If `cpflow` is already installed in the target repo, you can print the current
-copy-paste version of this prompt with:
-
-```sh
-cpflow ai-github-flow-prompt
-```
-
-That local-only command works even before `cpln` is installed and fills in the
-repo-name default app prefix for the current checkout. You can also run
-`cpflow github-flow-readiness` first to check the same blocker categories the
-prompt tells the agent to stop on.
+Copy the recommended prompt below when you want an AI agent to add the reusable
+`cpflow` review-app, staging, and production-promotion flow to a repository.
+It works whether `cpflow` is already installed or the agent needs to install it
+first.
 
 ## Recommended Prompt
 
 ```text
-Set up Control Plane GitHub Flow for this repo. Start with `cpflow github-flow-readiness` and stop on any reported blockers. The repo must be deployable from a clean clone: published package versions, complete runtime scaffold, and a production Dockerfile that can build the app. If any package version is unpublished, inaccessible from CI, or requires credentials that are not already modeled in the repo or GitHub settings, stop and report the blocker instead of generating workflow files. If the repo is a legacy sample pinned to an obsolete Ruby or Bundler toolchain, if it does not even have a production Dockerfile yet, or if it is a monorepo without an already-decided single app boundary for this flow, stop and report that as a prerequisite instead of forcing the rollout.
+Set up Control Plane GitHub Flow for this repo. First make sure the `cpflow` CLI is available: use the repo's existing `bundle exec cpflow` if present, otherwise install the published `cpflow` Ruby gem with `gem install cpflow`; if neither is possible, stop and report that blocker. Use the same `cpflow` invocation for the rest of the rollout. Start with `cpflow github-flow-readiness` and stop on any reported blockers. The repo must be deployable from a clean clone: published package versions, complete runtime scaffold, and a production Dockerfile that can build the app. If any package version is unpublished, inaccessible from CI, or requires credentials that are not already modeled in the repo or GitHub settings, stop and report the blocker instead of generating workflow files. If the repo is a legacy sample pinned to an obsolete Ruby or Bundler toolchain, if it does not even have a production Dockerfile yet, or if it is a monorepo without an already-decided single app boundary for this flow, stop and report that as a prerequisite instead of forcing the rollout.
 
-If `.controlplane/` is missing, run `cpflow generate`. Treat the generated app names as the repo-name default and rename them only if the project needs a different prefix. Then run `cpflow generate-github-actions` (or `cpflow generate-github-actions --staging-branch BRANCH` when staging should deploy from a branch other than `main`/`master`), keep review apps opt-in via `+review-app-deploy`, make sure any `STAGING_APP_BRANCH` repository variable is also present in the generated staging workflow's `on.push.branches` filter, and list the GitHub secrets and variables that must be configured. Do not hand-edit duplicated upstream refs into the generated wrappers: the only downstream Control Plane Flow pin should be the reusable workflow `uses: ...@vX.Y.Z` value generated from the installed `cpflow` gem version, and upstream workflows load their matching shared actions automatically. When bumping the `cpflow` gem in a downstream repo, run `cpflow update-github-actions` (or `bundle exec cpflow update-github-actions`) and validate with `bin/test-cpflow-github-flow` in the same PR so the checked-in wrappers move to the matching release tag. Keep the standard path simple: review apps require only `CPLN_TOKEN_STAGING` when the generated review app config can be inferred. For shared review-app resources such as one staging database, use `shared_secret_grants` and `{{SHARED_SECRET_DATABASE}}` placeholders instead of hardcoding the base app secret name; this keeps review-app policy binding and cleanup automatic while avoiding per-PR database cost. Document the one-time Control Plane bootstrap command for persistent staging and production apps with `cpflow setup-app --skip-post-creation-hook`; for existing apps or later template updates, document `cpflow apply-template` and the need for the app identity to have `reveal` on the app secret policy. Do not imply the staging deploy or promotion workflows create those persistent GVCs. For production promotion, document a protected `production` GitHub Environment with required reviewers, prevent self-review, and `CPLN_TOKEN_PRODUCTION` stored as an environment secret, not as a repository or organization secret.
+If `.controlplane/` is missing, run `cpflow generate`. Treat the generated app names as the repo-name default and rename them only if the project needs a different prefix. Then run `cpflow generate-github-actions` (or `cpflow generate-github-actions --staging-branch BRANCH` when staging should deploy from a branch other than `main`/`master`), keep review apps opt-in via `+review-app-deploy`, make sure any `STAGING_APP_BRANCH` repository variable is also present in the generated staging workflow's `on.push.branches` filter, and list the GitHub secrets and variables that must be configured. Do not hand-edit duplicated upstream refs into the generated wrappers: the only downstream Control Plane Flow pin should be the reusable workflow `uses: ...@vX.Y.Z` value generated from the installed `cpflow` gem version, and upstream workflows load their matching shared actions automatically. When bumping the `cpflow` gem in a downstream repo, run `cpflow update-github-actions` (or `bundle exec cpflow update-github-actions`) and validate with `bin/test-cpflow-github-flow` in the same PR so the checked-in wrappers move to the matching release tag. Keep the normal generated review-app setup simple: review apps require only `CPLN_TOKEN_STAGING` when the generated review app config can be inferred. For public demos, starter staging apps, and long-lived review apps, keep the app workload `type: standard` with one warm replica, set its autoscaling metric to `disabled`, and enable `capacityAI: true` so Control Plane can right-size CPU and memory allocation at that fixed replica count. Shared Postgres and other stateful workloads are the usual exceptions and should stay manually sized; Capacity AI is for supported stateless app/service workloads. If true idle scale-to-zero is explicitly required, create a separate `serverless` workload before first deploy or plan a delete/recreate migration because Control Plane will not change an existing `standard` workload to `serverless` in place. For shared review-app resources such as one staging database, use `shared_secret_grants` and `{{SHARED_SECRET_DATABASE}}` placeholders instead of hardcoding the base app secret name; this keeps review-app policy binding and cleanup automatic while avoiding per-PR database cost. Document the one-time Control Plane bootstrap command for persistent staging and production apps with `cpflow setup-app --skip-post-creation-hook`; for existing apps or later template updates, document `cpflow apply-template` and the need for the app identity to have `reveal` on the app secret policy. Do not imply the staging deploy or promotion workflows create those persistent GVCs. For production promotion, document a protected `production` GitHub Environment with required reviewers, prevent self-review, and `CPLN_TOKEN_PRODUCTION` stored as an environment secret, not as a repository or organization secret.
 
 Keep Node available in the final image if asset compilation or SSR depends on ExecJS, Yarn, `pnpm`, or npm after the main install layer. Make sure the generated Dockerfile uses a Ruby base image compatible with the app's declared Ruby requirement. Preserve repo-defined frontend build hooks: if `config/shakapacker.yml` defines a `precompile_hook`, or React on Rails enables `config.auto_load_bundle = true`, confirm the generated Dockerfile runs that codegen step before `rails assets:precompile`. If `config/database.yml` shows SQLite in production, confirm that the generated scaffold uses persistent `db` and `storage` volumes plus a release script that runs `rails db:prepare`; otherwise keep the default Postgres workload. If the public workload is not named `rails`, set `PRIMARY_WORKLOAD` or adjust the generated workflows. Inspect the Dockerfile and package sources for private GitHub dependencies or `RUN --mount=type=ssh`; if present, wire `DOCKER_BUILD_SSH_KEY`, optionally set `DOCKER_BUILD_SSH_KNOWN_HOSTS` for non-GitHub SSH hosts, and keep `DOCKER_BUILD_EXTRA_ARGS` to newline-delimited single tokens such as `--build-arg=FOO=bar`.
 
 Run the real local validations you can: Docker build if feasible, repo tests or smoke checks, YAML validation, and any CI-equivalent build steps. Push the branch and check the GitHub Actions results. Only stop early for a real external blocker or a product decision that changes scope.
 ```
 
+## Local Shortcut
+
+If `cpflow` is already available in the target repo, this local-only command
+prints the same recommended prompt with the repo-name default app prefix filled
+in:
+
+```sh
+cpflow ai-github-flow-prompt
+```
+
+That command works even before `cpln` is installed.
+
 ## Hard Stop Conditions
 
 Stop and report the blocker instead of generating `cpflow-*` workflow files when:

data/docs/assets/logo/icon-tile.svg

@@ -0,0 +1,17 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1024 1024" role="img" aria-label="Control Plane Flow icon">
+<title>Control Plane Flow icon</title>
+<rect x="64" y="64" width="896" height="896" rx="192" fill="#0B1118" stroke="#26313D" stroke-width="12"/>
+<g fill="none" stroke-linecap="round" stroke-linejoin="round">
+  <path d="M290 650 C330 500 435 430 512 512 C600 606 704 560 746 390" stroke="#2BD7FF" stroke-width="68"/>
+  <path d="M746 390 L688 397" stroke="#2BD7FF" stroke-width="68"/>
+  <path d="M746 390 L731 446" stroke="#2BD7FF" stroke-width="68"/>
+</g>
+<circle cx="290" cy="650" r="92" fill="#8973FF"/>
+<circle cx="512" cy="512" r="92" fill="#70D187"/>
+<circle cx="746" cy="390" r="92" fill="#FFB000"/>
+<circle cx="290" cy="650" r="32" fill="#0B1118" opacity="0.88"/>
+<circle cx="512" cy="512" r="32" fill="#0B1118" opacity="0.88"/>
+<circle cx="746" cy="390" r="32" fill="#0B1118" opacity="0.88"/>
+<path d="M390 730 H612" stroke="#2BD7FF" stroke-width="40" stroke-linecap="round" opacity="0.86"/>
+<path d="M612 730 L555 692 M612 730 L555 768" stroke="#2BD7FF" stroke-width="40" stroke-linecap="round" stroke-linejoin="round" opacity="0.86"/>
+</svg>

data/docs/assets/logo/mark-transparent.svg

@@ -0,0 +1,16 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1024 1024" role="img" aria-label="Control Plane Flow icon transparent mark">
+<title>Control Plane Flow icon transparent mark</title>
+<g fill="none" stroke-linecap="round" stroke-linejoin="round">
+  <path d="M290 650 C330 500 435 430 512 512 C600 606 704 560 746 390" stroke="#2BD7FF" stroke-width="68"/>
+  <path d="M746 390 L688 397" stroke="#2BD7FF" stroke-width="68"/>
+  <path d="M746 390 L731 446" stroke="#2BD7FF" stroke-width="68"/>
+</g>
+<circle cx="290" cy="650" r="92" fill="#8973FF"/>
+<circle cx="512" cy="512" r="92" fill="#70D187"/>
+<circle cx="746" cy="390" r="92" fill="#FFB000"/>
+<circle cx="290" cy="650" r="32" fill="#0B1118" opacity="0.88"/>
+<circle cx="512" cy="512" r="32" fill="#0B1118" opacity="0.88"/>
+<circle cx="746" cy="390" r="32" fill="#0B1118" opacity="0.88"/>
+<path d="M390 730 H612" stroke="#2BD7FF" stroke-width="40" stroke-linecap="round" opacity="0.86"/>
+<path d="M612 730 L555 692 M612 730 L555 768" stroke="#2BD7FF" stroke-width="40" stroke-linecap="round" stroke-linejoin="round" opacity="0.86"/>
+</svg>

data/docs/ci-automation.md

@@ -10,6 +10,13 @@ The goal is to bring the Heroku Flow model into any `cpflow` project:
 4. Promote the already-built staging artifact to production from the Actions tab.
 5. Let a nightly workflow clean up stale review apps.
 
+For public repositories, review apps are intentionally limited to branches in the base repository. Fork pull requests can
+receive help comments, but generated deploy workflows skip fork heads because review-app deployment builds Docker images
+with repository secrets. The skip is enforced by complementary guards on different trigger axes; see
+[Review app security for repositories with external contributors](#review-app-security-for-repositories-with-external-contributors)
+before customizing these workflows. If a forked change needs a review app, a maintainer should first review the code and
+move the change into a trusted branch in the base repository.
+
 ## Quick Start
 
 End-to-end rollout in one view:
@@ -20,6 +27,8 @@ End-to-end rollout in one view:
 4. Configure the GitHub [repository secrets and variables](#required-github-repository-settings) the workflows expect.
 5. Push the branch, then comment `+review-app-deploy` on a PR to spin up a review environment.
 
+AI rollout: copy the [AI rollout prompt](./ai-github-flow-prompt.md) when you want an agent to run this setup. The prompt works whether `cpflow` is already installed or the agent needs to install it first. If `cpflow` is already available in the target repo, `cpflow ai-github-flow-prompt` prints the same prompt with the default app prefix filled in.
+
 See [Bootstrap a Project](#bootstrap-a-project) for command details, [Repo Readiness Checklist](#repo-readiness-checklist) for what "ready" means, and [AI Playbook](#ai-playbook) to run the rollout through an agent.
 
 ## Bootstrap a Project
@@ -137,7 +146,8 @@ apps:
     match_if_app_name_starts_with: true
     hooks:
       post_creation: bundle exec rails db:prepare
-      pre_deletion: bundle exec rails db:drop
+      # pre_deletion intentionally omitted for shared databases: `cpflow delete` runs it before removing the workloads,
+      # so live connections can block the drop. Prefer admin-side cleanup. See docs/tips.md ("Share One Control Plane Postgres").
 
   my-app-production:
     <<: *common
@@ -155,12 +165,18 @@ Important points:
 - Review-app workflows infer the staging Control Plane org from that review app entry's `cpln_org`.
 - `upstream: my-app-staging` is what lets the production promotion workflow copy the exact staging artifact.
 - If your main web workload is not named `rails`, set the optional `PRIMARY_WORKLOAD` repository variable described below.
+- For public demos, starter staging apps, and long-lived review apps, prefer `capacityAI: true` with one warm replica
+  and the workload's autoscaling metric disabled, so Control Plane can right-size CPU and memory allocation at that
+  fixed replica count. See
+  [Enable Capacity AI for Demo and Starter Staging Apps](tips.md#enable-capacity-ai-for-demo-and-starter-staging-apps).
 
 ## Required GitHub Repository Settings
 
 For a normal generated review-app setup, configure one repository secret:
 
-- `CPLN_TOKEN_STAGING`: token for the staging Control Plane org
+- `CPLN_TOKEN_STAGING`: token for the staging/review Control Plane org. Generate it from a Control Plane service account
+  whose policies only allow review/staging CI operations; it must not read production secrets or manage production
+  workloads.
 
 No GitHub repository variables are required for review apps when `.controlplane/controlplane.yml`
 has exactly one review app entry with `match_if_app_name_starts_with: true` and
@@ -173,7 +189,11 @@ disambiguate generated review-app config:
 
 - `CPLN_ORG_STAGING`: override the staging/review org inferred from `cpln_org`, for example `company-staging`
 - `REVIEW_APP_PREFIX`: override the inferred review-app prefix; required only when multiple review app prefixes exist in `controlplane.yml`
-- `PRIMARY_WORKLOAD`: override the public workload used to discover the public endpoint and do production health checks; defaults to `rails`
+- `PRIMARY_WORKLOAD`: override the public workload used to discover the public endpoint and do review/production health checks; defaults to `rails`
+- `REVIEW_APP_HEALTH_CHECK_RETRIES`: override review-app health polling attempts; defaults to `24`
+- `REVIEW_APP_HEALTH_CHECK_INTERVAL`: override seconds between review-app health attempts; defaults to `15`
+- `REVIEW_APP_HEALTH_CHECK_ACCEPTED_STATUSES`: override space-separated healthy HTTP statuses for review apps; defaults to `200 301 302`
+- `REVIEW_APP_HEALTH_CHECK_CURL_MAX_TIME`: override per-request review-app curl timeout in seconds; defaults to `10`
 
 If `controlplane.yml` defines more than one app with
 `match_if_app_name_starts_with: true`, inference intentionally fails. Set
@@ -196,6 +216,30 @@ For production promotion, also configure:
 - `CPLN_ORG_PRODUCTION` as a production environment variable, for example `company-production`
 - `PRODUCTION_APP_NAME` as a production environment variable, for example `my-app-production`
 
+Enter GitHub variables such as `CPLN_ORG_STAGING`,
+`CPLN_ORG_PRODUCTION`, `STAGING_APP_NAME`, and `PRODUCTION_APP_NAME`
+as plain single-line values. The generated production promotion workflow trims
+accidental leading/trailing whitespace and line endings from Control Plane org
+names before building registry URLs, but embedded line breaks are rejected
+because they could change the target org name after normalization.
+
+Production promotion copies the exact image currently deployed on the selected
+staging workload. If that staging image is digest-pinned, the digest is used for
+the source copy while the production tag is derived from the tag portion. Tags
+with a `_<commit>` suffix keep that suffix in production; plain numeric tags are
+also valid and promote to the next plain production tag. The copy step uses
+`docker buildx imagetools create --prefer-index=false --tag` with isolated
+Docker credentials, which preserves multi-architecture manifests, preserves
+single-platform manifest format when supported, and avoids pulling image layers
+onto the GitHub Actions runner.
+
+Before copying the image, production promotion compares the environment variable
+names exposed by staging and production at both the GVC level and each configured
+app workload's container level. Variables present in staging are treated as
+required for production, while production-only variables emit warnings. A missing
+production workload variable such as a renderer password or runtime secret fails
+the promotion before the image copy starts.
+
 Do not put `CPLN_TOKEN_PRODUCTION` in repository or organization secrets for
 sensitive production systems. Production promotion intentionally runs as a
 normal caller-repo workflow job with `environment: production`, then checks out
@@ -271,6 +315,12 @@ cpflow setup-app -a my-app-production --org my-org-production --skip-post-creati
 Use production-only runtime secrets and values for the production app. The
 protected GitHub Environment controls who can run the promotion workflow, but
 the production app resources still need to exist before the first promotion.
+After bootstrap, populate the production app secret dictionary with the values
+referenced by `.controlplane/templates`, then run `cpflow apply-template` against
+production when templates change so the workload env references remain persisted.
+Production promotion checks for missing GVC and workload container env names
+before copying the staging image, so a staging-only runtime variable will stop
+the run early instead of deploying an image that cannot boot.
 
 Review apps are different: the generated `+review-app-deploy` workflow creates
 temporary PR apps as needed, including the identity and secret policy binding.
@@ -316,6 +366,17 @@ The standard path is:
 7. Store `CPLN_ORG_PRODUCTION` and `PRODUCTION_APP_NAME` as `production`
    environment variables, or as repository variables only when those names are
    intentionally non-sensitive.
+8. Keep GitHub variable values single-line; a pasted trailing newline is trimmed
+   for Control Plane org names, but embedded line breaks are rejected before
+   deployment, copy, health-check, or rollback steps run.
+9. Bootstrap or re-apply the persistent production app templates before first
+   promotion so app workload container env references and Control Plane secret
+   dictionaries exist in production.
+10. Expect promotion to preserve the selected staging image reference. Digest
+   references are copied by digest, commit-suffixed tags keep the commit suffix,
+   and plain numeric tags remain valid.
+11. Expect production health and rollback readiness polling to require Control
+   Plane `status.ready` and `status.readyLatest` before checking the endpoint.
 
 GitHub only exposes environment secrets to jobs that reference the environment
 after configured protection rules pass. GitHub does not allow a caller job that
@@ -336,6 +397,9 @@ Recommended org layout:
 
 - keep review apps and staging in a staging org that developers can access
 - keep production in a separate org with tighter access controls
+- for public repositories, use a staging/review token generated from a service account whose policies only allow
+  review/staging CI operations; use a dedicated review-app org only if you also customize the generated
+  workflow/configuration to target that org separately
 
 Optional repository secret for private dependency builds:
 
@@ -352,6 +416,105 @@ Advanced optional repository variables:
 - `CPLN_CLI_VERSION`: pin only when Control Plane CLI compatibility requires it.
 - `CPFLOW_VERSION`: pin a published RubyGems version only when intentionally overriding the default build-from-ref behavior.
 
+## Review App Security for Repositories with External Contributors
+
+Review-app deployment and teardown can execute pull request code. Even when the workflow itself is trusted, the
+Dockerfile, package scripts, Rails initializers, server-rendering code, application runtime, any `release_script` or
+`hooks.post_creation` defined in the PR's `.controlplane/controlplane.yml`, and `.controlplane/templates/*.yaml` identity
+and policy templates applied by `cpflow setup-app` at first deploy with `CPLN_TOKEN_STAGING` can all be changed by the
+pull request being deployed. Teardown can also run a `hooks.pre_deletion` command through the latest PR-built image, even
+when the hook command comes from the base-branch config. A PR author can embed malicious code in that image; at runtime
+the code executes inside the review app workload and can read any secrets mounted into the workload environment.
+The generated setup action also exports `CPLN_TOKEN_STAGING` as `CPLN_TOKEN` via `GITHUB_ENV`, making it available in the
+process environment of every subsequent runner step for the rest of the job, so keep any custom runner steps after setup
+trusted. PR-controlled `release_script` and hook commands normally run through `cpflow run` in remote Control Plane
+containers from the latest image, not as runner shell steps, so they do not read that runner token directly unless a
+custom workflow passes a local token through. Inside the Control Plane workload, a separate runtime `CPLN_TOKEN` is
+available only when the workload spec has an `identityLink`; `skip_secrets_setup` skips automatic identity creation and
+binding, but templates can still attach an identity. Because `cpflow setup-app` reads `controlplane.yml` from the PR
+checkout, a PR can also set `skip_secrets_setup: false` or omit it to re-enable automatic identity binding unless the
+trusted workflow passes `--skip-secrets-setup` as a CLI flag. Do not treat `skip_secrets_setup` in `controlplane.yml` as
+a token-removal control or reliable security boundary. This is why workload-mounted secrets and the staging
+service-account token must remain disposable and scoped to minimum permissions.
+
+The generated flow uses these defaults:
+
+- same-repository pull requests can update existing review apps automatically on each push; creating the first review app
+  requires either a `+review-app-deploy` comment from a trusted commenter (`OWNER`, `MEMBER`, or `COLLABORATOR`
+  association, regardless of permission level) or a manual workflow dispatch by a repository collaborator with write
+  access. The trusted-commenter gate applies to every `+review-app-deploy` comment, whether or not a review app already
+  exists. Later pushes to a base-repository branch PR redeploy automatically without another approval because the
+  auto-push path (`pull_request` event) has no trusted-commenter check; only the `issue_comment` path does;
+- fork pull requests cannot deploy via the generated `pull_request` path because the caller workflow's job-level `if:`
+  condition explicitly skips fork-originated runs. For `issue_comment` events, the caller `if:` restricts commands to
+  trusted author associations (`OWNER`, `MEMBER`, `COLLABORATOR`) but does not check fork status; the fork check for
+  comment events is enforced inside the reusable workflow's source-validation step. These complementary guards cover
+  different axes, so preserve both. A trusted commenter can comment on a fork PR and pass the caller's
+  `author_association` check; the reusable workflow's source-validation step is what blocks that fork head from being
+  deployed. Removing the source-validation guard opens a path to deploy untrusted code with repository-secret access,
+  because `issue_comment` events execute with base-repository secret access. Removing the `pull_request` guard still
+  lets untrusted fork code into the staging environment even though GitHub withholds repository secrets from fork
+  `pull_request` runs. Keep both workflow guards in place because the workflow builds Docker images with repository
+  secrets;
+- review apps are also deleted automatically when the pull request closes; that PR-close path uses `pull_request_target`
+  so it runs in the base-repository context and has repository-secret access for teardown. That is also why you must
+  never check out PR or fork code in this job; see the customization guidance below. The PR-close path does not require a
+  trusted commenter. The generated `cpflow-delete-review-app.yml` pins `GITHUB_TOKEN` permissions to the minimum it needs
+  (`contents: read`, `issues: write`, `pull-requests: write`); if you customize this workflow, preserve that
+  `permissions:` block because omitting it can fall back to broader repository defaults;
+- manual workflow dispatch by a repository collaborator can also delete a review app without a `+review-app-delete`
+  comment, and does not require a trusted commenter;
+- trusted comments on fork PRs still do not deploy the fork head; the workflow posts no PR comment in this case. The
+  commenter receives a rocket reaction and the skip appears only in the Actions step summary. Review the fork code
+  carefully, then move the change to a branch in the base repository if it needs a generated review app. That build will
+  run with repository-secret access;
+- production promotion is manual and uses production environment secrets separately from review and staging.
+
+The generated review-app workflow targets the staging/review Control Plane org inferred from `cpln_org` in
+`controlplane.yml`, or overridden by `CPLN_ORG_STAGING`, and uses the token from `CPLN_TOKEN_STAGING`. A fully separate
+review-app org or token requires workflow/configuration customization; otherwise, keep review apps and staging in the
+generated staging/review org and make that token disposable and unable to access production resources.
+
+The PR-close teardown workflow runs trusted base-branch workflow code with repository secret access so it can delete fork
+PR review apps. The generated `cpflow-delete-control-plane-app` composite action script refuses to call `cpflow delete`
+on any app whose name does not match the review-app prefix. This shell-level guard is effective because the generated
+delete workflow checks out base/default-branch code, not PR or fork code: its repository checkout has no `ref:` override,
+so `pull_request_target` runs use the base branch. If you customize this workflow, never check out PR or fork code in the
+same job as the delete step; doing so could let a PR replace the guard script itself and would also make
+`hooks.pre_deletion` come from the PR's `controlplane.yml`. This is still not a token policy, so use a scoped staging
+service account limited to review/staging operations. A configured `hooks.pre_deletion` command still runs through the
+latest PR-built image on all delete paths: PR-close teardown, `+review-app-delete` comments, manual dispatch, and
+scheduled cleanup. Review-app credentials must remain disposable even during deletion.
+
+If you customize the generated `pull_request_target` workflow, never pass `github.event.pull_request.head.sha` or another
+fork-controlled ref to `actions/checkout`, `git fetch`, `git merge`, `git cherry-pick`, or any other step that fetches,
+materializes, or executes the ref. Those operations run untrusted fork code with repository secret access. Referencing the
+SHA as an opaque identifier for deployment status updates, comments, or API calls is safe because SHA values are hex-only
+and cannot contain shell metacharacters. Do not apply that reasoning to other attacker-controlled PR event fields such as
+`head.ref`, `head.label`, `title`, `body`, `head.repo.full_name`, or repository descriptions; pass user-controlled strings
+through environment variables instead of interpolating them directly into `run:` steps.
+
+These defaults protect repository secrets from direct fork PR execution, but they do not make deployed PR code harmless.
+For repositories with external contributors, keep review-app credentials and runtime values disposable:
+
+- do not mount production secrets, staging customer data, package-publishing tokens, payment keys, or monitoring tokens
+  into review apps;
+- do not use `DOCKER_BUILD_SSH_KEY` with a long-lived personal key or broad deploy key; see
+  [Docker Builds with Private Dependencies](#docker-builds-with-private-dependencies) for the minimum-access deploy-key
+  requirements;
+- do not use broad Control Plane `superusers` tokens for review-app CI; use a review/staging service account scoped only to
+  review/staging CI operations;
+- keep review databases, Redis instances, object stores, and renderer credentials separate from staging and production;
+- rotate any credential that may have been exposed to a malicious review-app build;
+- use scheduled cleanup so stale review apps stop consuming compute and secrets access. Cleanup deletes apps through
+  `cpflow delete`, so any configured `hooks.pre_deletion` still runs through the latest PR-built image; review-app
+  credentials must remain disposable for this path too.
+
+Secret indirection such as `cpln://secret/...` protects values in Control Plane configuration. It does not protect a
+value after that secret is mounted into a workload that runs pull request code. See
+[Secrets and ENV Values - Review app secrets](./secrets-and-env-values.md#review-app-secrets) for review-app secret
+handling guidance.
+
 ## Docker Builds with Private Dependencies
 
 Some apps need extra Docker build configuration before the generated workflows are turnkey. Common examples are:
@@ -376,6 +539,10 @@ DOCKER_BUILD_EXTRA_ARGS=--build-arg=BUNDLE_WITHOUT=development:test
 
 The action will start an SSH agent, add the key, write `known_hosts`, and pass `--ssh=default` to `cpflow build-image`. When `DOCKER_BUILD_SSH_KNOWN_HOSTS` is unset, the generated action uses pinned GitHub.com host keys by default. If your Dockerfile relies on `RUN --mount=type=ssh`, validate the build locally with `cpflow build-image -a <app> --ssh=default` before relying on CI.
 
+Do not configure `DOCKER_BUILD_SSH_KEY` unless the Dockerfile truly needs it. When configured, it is available to all
+review-app and staging Docker builds; follow the review-app security guidance above by using a read-only, revocable
+deploy key scoped to the minimum private dependency access, and never use a personal SSH key.
+
 ## Generated Workflow Behavior
 
 `cpflow-review-app-help.yml`
@@ -391,18 +558,31 @@ The action will start an SSH agent, add the key, write `known_hosts`, and pass `
 
 `cpflow-deploy-review-app.yml`
 
-- Creates a review app when someone comments `+review-app-deploy`.
+- Creates a review app when someone comments `+review-app-deploy` or via manual workflow dispatch by a repository
+  collaborator.
+- For manual dispatch, provide the PR number; the workflow rejects fork PRs at runtime because it builds Docker images
+  with repository secrets.
 - Redeploys an existing review app automatically on later PR pushes.
 - Creates a GitHub deployment and comments with the review URL and logs.
 - Leaves PR pushes alone until the first review app is explicitly requested, which keeps demo-app costs down.
+- Supports cost-conscious review apps when paired with one warm replica, Capacity AI, and a disabled autoscaling
+  metric for public demos, starter staging apps, and long-lived review apps; see
+  [Enable Capacity AI for Demo and Starter Staging Apps](tips.md#enable-capacity-ai-for-demo-and-starter-staging-apps).
 - Accepts `+review-app-deploy` only from trusted commenters (`OWNER`, `MEMBER`, or `COLLABORATOR`).
-- Skips fork-based PR deploys because the workflow builds Docker images with repository secrets.
+- Skips fork-based PR deploys because the workflow builds Docker images with repository secrets. A trusted comment on a
+  fork PR still does not deploy the fork head, and manual dispatch must use a base-repository PR number; dispatching with
+  a fork PR number causes a hard workflow failure. To give a fork PR a review app, review the code carefully first, then
+  move the change to a branch in the base repository. The build will then run with repository-secret access.
 
 `cpflow-delete-review-app.yml`
 
 - Deletes the review app on `+review-app-delete`.
-- Also deletes it automatically when the pull request closes.
-- Accepts `+review-app-delete` only from trusted commenters (`OWNER`, `MEMBER`, or `COLLABORATOR`).
+- Also supports manual workflow dispatch by a repository collaborator without requiring a trusted-commenter command.
+- Also deletes it automatically when the pull request closes through a `pull_request_target` event, so repository secrets
+  are available for teardown; `hooks.pre_deletion` still executes through the latest PR-built image on this path, so
+  review-app credentials must remain disposable.
+- Accepts `+review-app-delete` only from trusted commenters (`OWNER`, `MEMBER`, or `COLLABORATOR`); manual dispatch and
+  automatic PR-close teardown do not require a trusted commenter.
 
 `cpflow-deploy-staging.yml`
 
@@ -415,11 +595,17 @@ The action will start an SSH agent, add the key, write `known_hosts`, and pass `
 
 - Manually promotes the staging artifact to production with a confirmation input.
 - Runs the production job in the `production` GitHub Environment, so configured reviewers approve the job before production environment secrets are available.
-- Verifies that production has the env var names staging expects.
-- Runs a health check against `PRIMARY_WORKLOAD`.
+- Verifies that production has the GVC and app workload container env var names staging expects.
+- Runs a health check against `PRIMARY_WORKLOAD` only after Control Plane reports the latest workload version ready.
 - Attempts a rollback of every configured application workload if the new production image does not come up healthy.
 - Creates a GitHub release after a successful promotion.
 
+`cpflow-deploy-review-app.yml`
+
+- Builds and deploys the pull request image after the review app exists or is created.
+- Waits for `PRIMARY_WORKLOAD` to report `readyLatest=true` and return an accepted HTTP status before marking the GitHub deployment successful.
+- Uses `REVIEW_APP_HEALTH_CHECK_*` repository variables to tune review-app health checks independently from production promotion.
+
 `cpflow-cleanup-stale-review-apps.yml`
 
 - Runs nightly and on demand.
@@ -676,15 +862,17 @@ In practice, porting the flow into a demo app usually follows five phases.
 
 13. Validate the real production Docker build before relying on the workflows, especially if asset compilation or SSR requires Node, extra system packages, multiple processes, extra Docker build flags, or persistent writable paths.
 14. Expect review app deploys to run only for branches in the base repository; fork PRs still get help comments, but deploys are skipped because the workflow uses repository secrets.
+15. For public repositories, confirm that review-app secrets are disposable and that `CPLN_TOKEN_STAGING` cannot access production resources.
 
 ## AI Playbook
 
-If you want an AI agent to apply this flow to another project, start with
-`cpflow github-flow-readiness`, then use the standalone
-[AI rollout prompt](./ai-github-flow-prompt.md). It captures the exact wording,
-hard stop conditions, and definition of done for this workflow. You can also
-run `cpflow ai-github-flow-prompt` from inside the target repo to print the
-current prompt with that repo's default app prefix already filled in.
+If you want an AI agent to apply this flow to another project, use the
+standalone [AI rollout prompt](./ai-github-flow-prompt.md). It captures the
+exact wording, hard stop conditions, and definition of done for this workflow,
+and it works whether `cpflow` is already installed or the agent needs to install
+it first. When `cpflow` is already available in the target repo,
+`cpflow ai-github-flow-prompt` prints the same prompt with that repo's default
+app prefix already filled in.
 
 Short version:
 

data/docs/commands.md

@@ -149,6 +149,9 @@ cpflow delete -a $APP_NAME -w $WORKLOAD_NAME
 ### `deploy-image`
 
 - Deploys the latest image to app workloads
+- Use `--workload`/`-w` one or more times to deploy only selected app workloads
+- If `deploy_order` is configured and no `--workload` is provided, deploys ordered workload groups one at a time and waits for each group to be ready before continuing
+- Workloads listed in `app_workloads` but omitted from `deploy_order` deploy last as an implicit final group
 - Runs a release script before deploying if `release_script` is specified in the `.controlplane/controlplane.yml` file and `--run-release-phase` is provided
 - The release script is run in the context of `cpflow run` with the latest image
 - If the release script exits with a non-zero code, the command will stop executing and also exit with a non-zero code
@@ -156,7 +159,14 @@ cpflow delete -a $APP_NAME -w $WORKLOAD_NAME
 - Repairs missing `shared_secret_grants` policy bindings before running a release phase or updating workloads
 
 ```sh
+# Deploys the latest image to all app workloads.
 cpflow deploy-image -a $APP_NAME
+
+# Deploys only one app workload.
+cpflow deploy-image -a $APP_NAME -w node-renderer
+
+# Deploys only selected app workloads.
+cpflow deploy-image -a $APP_NAME -w node-renderer -w sidekiq
 ```
 
 ### `doctor`
@@ -315,6 +325,7 @@ cpflow maintenance -a $APP_NAME
 ### `maintenance:off`
 
 - Disables maintenance mode for an app
+- Safe to re-run: if a previous run timed out after switching the domain but before stopping the maintenance workload, re-running while maintenance mode is already disabled stops the maintenance workload to finish it (so it is not a pure no-op)
 - Specify the one-off workload through `one_off_workload` in the `.controlplane/controlplane.yml` file
 - Optionally specify the maintenance workload through `maintenance_workload` in the `.controlplane/controlplane.yml` file (defaults to 'maintenance')
 - Maintenance mode is only supported for domains that use path based routing mode and have a route configured for the prefix '/' on either port 80 or 443
@@ -326,6 +337,7 @@ cpflow maintenance:off -a $APP_NAME
 ### `maintenance:on`
 
 - Enables maintenance mode for an app
+- Safe to re-run: if a previous run timed out after switching the domain but before stopping the app workloads, re-running while maintenance mode is already enabled stops the app workloads to finish it (so it is not a pure no-op)
 - Specify the one-off workload through `one_off_workload` in the `.controlplane/controlplane.yml` file
 - Optionally specify the maintenance workload through `maintenance_workload` in the `.controlplane/controlplane.yml` file (defaults to 'maintenance')
 - Maintenance mode is only supported for domains that use path based routing mode and have a route configured for the prefix '/' on either port 80 or 443
@@ -372,6 +384,7 @@ cpflow open-console -a $APP_NAME
 - It performs the following steps:
   - Runs `cpflow copy-image-from-upstream` to copy the latest image from upstream
   - Runs `cpflow deploy-image` to deploy the image
+  - Honors `deploy_order` from `.controlplane/controlplane.yml` through `cpflow deploy-image`
   - If `.controlplane/controlplane.yml` includes the `release_script`, `cpflow deploy-image` will use the `--run-release-phase` option
   - If the release script exits with a non-zero code, the command will stop executing and also exit with a non-zero code
   - If `use_digest_image_ref` is `true` in the `.controlplane/controlplane.yml` file or `--use-digest-image-ref` option is provided, deployed image's reference will include its digest
@@ -466,6 +479,8 @@ timeout 300 cpflow ps:wait -a $APP_NAME
   and also overridden per job through `--cpu` and `--memory`)
 - By default, the job is stopped if it takes longer than 6 hours to finish
   (can be configured though `runner_job_timeout` in `controlplane.yml`)
+- Non-interactive jobs return the Control Plane cron job status even when the job finishes before
+  Control Plane exposes a runner replica to attach logs to
 
 ```sh
 # Opens shell (bash by default).
@@ -550,7 +565,7 @@ cpflow terraform import
 Regenerates the generated cpflow GitHub Actions wrappers and helper files
 from the currently installed cpflow gem. Use this after updating the
 cpflow gem so checked-in workflow wrappers move to the matching upstream
-release tag, for example `v5.0.4`.
+release tag, for example `v5.1.1`.
 
 If the existing generated staging workflow uses a custom single staging
 branch, the command preserves it. Pass `--staging-branch BRANCH` to set or