cpflow 5.3.0 → 6.0.0

data/docs/commands.md

@@ -30,6 +30,7 @@ cpflow ai-github-flow-prompt
 - Picks templates from the `.controlplane/templates` directory
 - Templates are ordinary Control Plane templates but with variable preprocessing
 - Use `--preserve-existing-runtime` to retain each workload container's configured app image, even when the workload is unready, and skip existing secret resources entirely while applying other template changes
+- Use `--exclude-secret-template NAME` and `--exclude-policy-template NAME` to exclude exact named templates without changing workload image handling
 - Missing or invalid workload images use only an unambiguous app image from ready workloads; refresh fails before applying templates when no safe fallback exists
 
 **Preprocessed template variables:**
@@ -85,7 +86,7 @@ cpflow cleanup-images -a $APP_NAME
 
 - Acts on stale apps based on the creation date of the latest image, or the GVC if no images exist
 - With `--mode=delete` (default): deletes the whole app (GVC with all workloads, all volumesets and all images), and unbinds the app from the secrets policy and any configured `shared_secret_grants` policies as long as both the identity and each policy exist (and are bound)
-- With `--mode=stop`: suspends all workloads via `cpflow ps:stop` — no GVC, volumeset, or image is removed; resume with `cpflow ps:start`
+- With `--mode=stop`: suspends configured workloads that exist in the live GVC through Control Plane — no GVC, volumeset, or image is removed; resume with `cpflow ps:start`
 - `--mode=stop` only suspends workloads listed in `app_workloads` + `additional_workloads`; workloads present in the live GVC but missing from the config are skipped silently
 - `--mode=stop` returns once each workload is marked suspended; it does not wait for the workload to reach a not-ready state
 - Specify the amount of days after an app should be considered stale through `stale_app_image_deployed_days` in the `.controlplane/controlplane.yml` file
@@ -135,6 +136,7 @@ cpflow copy-image-from-upstream -a $APP_NAME --upstream-token $UPSTREAM_TOKEN --
 - Deletes the whole app (GVC with all workloads, all volumesets and all images) or a specific workload
 - Also unbinds the app from the secrets policy and any configured `shared_secret_grants` policies, as long as both the identity and each policy exist (and are bound)
 - For the app-specific secrets policy, removes every permission held by the app identity; for `shared_secret_grants`, removes only `reveal`
+- Removes a marked per-app dictionary and exact-target unbound policy for generated review credentials, including after the opt-in is removed from a dynamically matched review-app entry
 - Will ask for explicit user confirmation
 - Runs a pre-deletion hook before the app is deleted if `hooks.pre_deletion` is specified in the `.controlplane/controlplane.yml` file
 - If the hook exits with a non-zero code, the command will stop executing and also exit with a non-zero code
@@ -217,7 +219,7 @@ Creates base Control Plane config and template files for a Rails project:
 - infers the app prefix from the current directory and wires staging, review, and production entries
 - infers the Docker base Ruby version from `.ruby-version`, `.tool-versions`, or the app's `Gemfile`
 - preserves repo-defined asset precompile hooks, including React on Rails auto bundle generation
-- detects SQLite in `config/database.yml` and generates persistent `db` and `storage` volume templates instead of the default Postgres workload
+- detects SQLite in `config/database.yml` and generates persistent `/app/data` and `/app/storage` volume templates without hiding image migrations under `/app/db`
 
 ```sh
 # Creates .controlplane directory with Control Plane config and starter templates
@@ -232,6 +234,9 @@ Creates GitHub Actions templates for a Heroku Flow style Control Plane pipeline:
 - manual promotion from staging to production
 - nightly cleanup and PR help workflows
 
+It also copies cpflow's composite actions into `.github/actions/cpflow-*`
+so every local `uses:` target is checked in and can be audited directly.
+
 Pass `--staging-branch BRANCH` when staging should auto-deploy from a branch
 other than `main` or `master`; the generator will bake that branch into the
 GitHub Actions push trigger and use it as the default STAGING_APP_BRANCH.
@@ -240,13 +245,13 @@ Pass `--force` to overwrite existing generated files. Prefer
 repo.
 
 ```sh
-# Creates thin .github/workflows wrappers for the Control Plane flow
+# Creates workflow wrappers, local composite actions, and validation helpers
 cpflow generate-github-actions
 
 # Creates the flow with staging deploys triggered from develop
 cpflow generate-github-actions --staging-branch develop
 
-# Overwrites existing generated wrappers from the installed cpflow gem
+# Overwrites existing generated GitHub Actions files from the installed cpflow gem
 cpflow generate-github-actions --force
 ```
 
@@ -483,6 +488,12 @@ timeout 300 cpflow ps:wait -a $APP_NAME
   (can be configured though `runner_job_timeout` in `controlplane.yml`)
 - Waiting for a runner replica is limited to the smaller of `runner_job_timeout` and 1000 seconds.
   A terminal cron status fails immediately, and reaching the observation deadline reports the last safe status
+- With non-interactive log methods 2 and 3, after a command prints its completion marker, Control Plane
+  has up to 20 minutes to reconcile the cron job to a terminal status. This can be configured through
+  `runner_job_status_reconciliation_timeout` in `controlplane.yml`; timing out exits nonzero and reports
+  the job, replica, and last observed status
+- Log method 1 does not emit a completion marker, so its job-status polling is not covered by the
+  post-command reconciliation timeout
 - Non-interactive jobs return the Control Plane cron job status even when the job finishes before
   Control Plane exposes a runner replica to attach logs to
 - Injects `CPFLOW_GVC_ID` and `CPFLOW_GVC_CREATED` into the job, exposing the app's immutable GVC
@@ -549,13 +560,14 @@ cpflow run -a $APP_NAME --entrypoint /app/alternative-entrypoint.sh -- rails db:
 - Configures app to have org-level secrets with default name `"{APP_PREFIX}-secrets"`
   using org-level policy with default name `"{APP_PREFIX}-secrets-policy"` (names can be customized, see docs)
 - Creates identity for secrets if it does not exist
+- For dynamically named review apps with `generated_review_secret_keys`, checks an existing policy before writing credentials, creates a per-app dictionary, skips its secret template during initial setup, and fills missing disposable keys without printing or rotating values
 - Binds the app identity to any configured `shared_secret_grants` policies as part of the secrets setup flow; skipped when `--skip-secrets-setup` or `--skip-secret-access-binding` is provided, or `skip_secrets_setup` is set
 - Use `--skip-secrets-setup` to prevent the automatic setup of secrets,
   or set it through `skip_secrets_setup` in the `.controlplane/controlplane.yml` file
 - Runs a post-creation hook after the app is created if `hooks.post_creation` is specified in the `.controlplane/controlplane.yml` file
 - If the hook exits with a non-zero code, the command will stop executing and also exit with a non-zero code
 - Use `--skip-post-creation-hook` to skip the hook if specified in `controlplane.yml`
-- Use `--refresh-templates` to apply configured templates noninteractively to an existing app while preserving each workload's configured app image even when workloads are unready or use mixed image versions, skipping existing secret resources entirely, repairing secrets access bindings, and skipping the post-creation hook
+- Use `--refresh-templates` to apply configured templates noninteractively to an existing app while preserving each workload's configured app image even when workloads are unready or use mixed image versions, skipping existing secret templates (but filling missing opt-in generated review keys), repairing secrets access bindings, and skipping the post-creation hook
 
 ```sh
 cpflow setup-app -a $APP_NAME
@@ -579,24 +591,35 @@ cpflow terraform import
 
 ### `update-github-actions`
 
-Regenerates the generated cpflow GitHub Actions wrappers and helper files
-from the currently installed cpflow gem. Use this after updating the
-cpflow gem so checked-in workflow wrappers move to the matching upstream
-release tag, for example `v5.2.0`.
+Refreshes local composite actions and helper files from the installed gem.
+All top-level workflows are preserved by default, including their refs,
+triggers, permissions, and deployment ownership. Use --workflows FILE...
+to explicitly add or replace named generated workflows. Replacement resets
+each selected workflow to its template and matching v6.0.0 ref;
+review and reapply any downstream customizations in the same PR.
+
+Selecting cpflow-deploy-staging.yml preserves a single existing push branch
+or the default main/master pair. Missing or ambiguous branches require
+--staging-branch BRANCH. This option requires selecting the staging workflow.
 
-If the existing generated staging workflow uses a custom single staging
-branch, the command preserves it. Pass `--staging-branch BRANCH` to set or
-replace the generated staging branch explicitly.
+A differing existing bin/test-cpflow-github-flow blocks all writes. Move
+downstream checks to executable bin/test-cpflow-github-flow-custom and
+remove or rename the old validator before updating. The generated validator
+invokes that extension after its baseline checks; the extension is never
+generated or overwritten. See docs/ci-automation.md for migration guidance.
 
 ```sh
-# After updating the cpflow gem, refresh generated GitHub Actions wrappers
+# Refresh actions/helpers while preserving downstream workflows
 cpflow update-github-actions
 
 # When running cpflow through Bundler
 bundle exec cpflow update-github-actions
 
-# Preserve or set a custom staging branch
-cpflow update-github-actions --staging-branch develop
+# Explicitly replace the review-app wrapper
+cpflow update-github-actions --workflows cpflow-deploy-review-app.yml
+
+# Explicitly add or replace staging deployment from develop
+cpflow update-github-actions --workflows cpflow-deploy-staging.yml --staging-branch develop
 ```
 
 ### `version`

data/docs/postgres-review-apps.md

@@ -0,0 +1,90 @@
+# PostgreSQL for disposable review apps
+
+Use the generated `.controlplane/templates/postgres.yml` for a PostgreSQL workload
+that belongs to one disposable review app. For production database migration, see
+[Migrating PostgreSQL from Heroku to RDS](./postgres.md).
+
+## Enable cleanup for new review apps
+
+The generated template marks three helper resources:
+
+| Resource | Purpose |
+| --- | --- |
+| `<APP>-pg` | Credential dictionary |
+| `<APP>-pg-script` | Initialization-script secret |
+| `<APP>-pg-access` | Policy granting access to those secrets |
+
+Each helper template carries this tag:
+
+```yaml
+tags:
+  cpflow-disposable-postgres-app: "{{APP_NAME}}"
+```
+
+`cpflow apply-template` retains the tag for dynamically named review apps and removes
+it for persistent apps. In an existing project, add the tag to all three helper
+**templates before creating new review apps**. Reapplying a marked template to an
+existing unmarked resource is refused; it does not adopt that resource.
+
+Before opting in, check that your deployment token can:
+
+- Read and delete the three helper resources.
+- Read each secret's effective access report (`viewAccessReport` on the org).
+
+A missing or incomplete access report blocks cleanup before app data is deleted.
+
+## Delete an app
+
+Run the normal app deletion command:
+
+```sh
+cpflow delete -a APP_NAME
+```
+
+Cleanup validates the helpers before deleting app data. It removes the marked
+policy and secrets before deleting the GVC. Deleting a single workload with
+`cpflow delete -a APP_NAME -w WORKLOAD_NAME` preserves the helpers.
+
+The ownership checks require:
+
+- The same app marker on every surviving helper.
+- A policy targeting exactly the two helper secrets.
+- Only `reveal` grants in that policy, for the app and PostgreSQL identities in the
+  app's GVC.
+- No helper listed in `shared_secret_grants`.
+- No effective secret access through another application policy or identity.
+
+Reapplying marked helpers with `cpflow apply-template` uses the same checks.
+
+## Understand a cleanup refusal
+
+Cleanup preserves resources when a marker, policy, or access report cannot establish
+exclusive ownership. A refusal aborts whole-app deletion before live app data is
+deleted. Fix the reported condition before retrying; do not remove a
+sharing grant until you know which applications depend on it.
+
+| Condition | Result |
+| --- | --- |
+| All helpers are unmarked legacy resources | App deletion proceeds; helpers are preserved |
+| Marked and unmarked helpers coexist | Cleanup refuses deletion |
+| A marker names another app | Cleanup refuses deletion |
+| Policy targets or principals differ from the expected app scope | Cleanup refuses deletion |
+| A helper is configured as shared | Cleanup refuses deletion |
+| Secret access reports are unavailable or incomplete | Cleanup refuses deletion |
+
+The access checks include permissions implied by `edit` or `manage`. Global `manage`
+grants to org groups, users, or service accounts count as org administration.
+Targeted or query grants, and grants to other GVC identities, block cleanup.
+
+For shared staging credentials, use the
+[shared-secret configuration guide](./secrets-and-env-values.md#shared-secrets-for-review-apps)
+instead of marking them as disposable helpers.
+
+## Recover after interruption
+
+Repeat `cpflow delete` with the same app name. Cleanup tolerates an already absent
+GVC, policy, or secret and rechecks the surviving helpers before deleting them.
+
+Unmarked legacy helpers need manual cleanup after you establish that no other app
+uses them. Rolling back to an older cpflow version leaves the tags in place but
+stops automatic helper cleanup.