cpflow 3.0.0

data/docs/redis.md

@@ -0,0 +1,128 @@
+# Migrating Redis databases
+
+There are two templates examples in this repo:
+- `redis` - basic non-persistent template. It is good for review-apps or staging or where no persistence is required
+- `redis2` - basic persistent template. Good for production where persistence is needed, but cluster is overkill.
+
+## Option 1: use SLAVEOF (easier way)
+
+1. create a redis workload that will accept data
+2. execute `SLAVEOF source_host source_port`, if needed use `masterauth` to provide auth details
+3. wait for replication to pick up all changes (usually quickly), use `INFO` or `DBSIZE` to check progress
+4. stop app completely and ensure nothing is writing to any of redises
+5. execute `SLAVEOF no one` to disconnect replication
+6. switch `REDIS_URL` in the app to point to new server
+7. start the app
+
+## Option 2: use Redis-RIOT (harder way, where option 1 is not possible)
+
+### General considerations:
+
+1. Heroku uses self-signed TLS certificates, which are not verifiable. It needs special handling by setting
+The tool that satisfies those criteria is [Redis-RIOT](https://developer.redis.com/riot/riot-redis/index.html)
+
+2. We are moving to private Redis that don't have a public URL, so have to do it from a Control Plane GVC container.
+
+The tool that satisfies those criteria is [Redis-RIOT](https://developer.redis.com/riot/riot-redis/index.html)
+
+### Heroku Redis:
+
+As Redis-RIOT says, master redis should have keyspace-notifications set to `KA` to be able to do live replication.
+To do that:
+
+```sh
+heroku redis:keyspace-notifications -c KA -a my-app
+```
+
+Connect to heroku Redis CLI:
+```sh
+heroku redis:cli -a my-app
+```
+
+### Control Plane Redis:
+
+Connect to Control Plane Redis CLI:
+
+```sh
+# open cpflow interactive shell
+cpflow run bash -a my-app
+
+# install redis CLI if you don't have it in Docker
+apt-get update
+apt-get install redis -y
+
+# connect to local cloud Redis
+redis-cli -u MY_CONTROL_PLANE_REDIS_URL -p 6379
+```
+
+### Useful Redis CLI commands:
+
+Quick-check keys qty:
+```
+info keyspace
+
+# Keyspace
+db0:keys=9496,expires=2941,avg_ttl=77670114535
+```
+
+### Create a Control Plane sync workload
+
+```
+name: riot-redis
+
+suspend: true
+min/max scale: 1/1
+
+firewall: all firewalls off
+
+image: fieldengineering/riot-redis
+
+CPU: 1 Core
+RAM: 1 GB
+
+command args:
+  --info
+  -u
+  rediss://...your_heroku_redis_url...
+  --tls-verify=NONE
+  replicate
+  -h
+  ...your_control_plane_redis_host...
+  --mode
+  live
+```
+
+### Sync process
+
+1. open 1st terminal window with heroku redis CLI, check keys qty
+2. open 2nd terminal window with controlplane redis CLI, check keys qty
+3. start sync container
+4. open logs with `cpflow logs -a my-app -w riot-redis`
+4. re-check keys sync qty again
+5. stop sync container
+
+Result:
+```
+Setting commit interval to default value (1)
+Setting commit interval to default value (1)
+Job: [SimpleJob: [name=snapshot-replication]] launched with the following parameters: [{}]
+Executing step: [snapshot-replication]
+Scanning   0% ╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━    0/8891 (0:00:00 / ?)Job: [SimpleJob: [name=scan-reader]] launched with the following parameters: [{}]
+Executing step: [scan-reader]
+Scanning  61% ━━━━━━━━━━━━━━━━╸━━━━━━━━━━ 5460/8891 (0:00:07 / 0:00:04) 780.0/sStep: [scan-reader] executed in 7s918ms
+Closing with items still in queue
+Job: [SimpleJob: [name=scan-reader]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 7s925ms
+Scanning 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9482/9482 (0:00:11 / 0:00:00) 862.0/s
+Step: [snapshot-replication] executed in 13s333ms
+Executing step: [verification]
+Verifying   0% ╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━    0/8942 (0:00:00 / ?)Job: [SimpleJob: [name=RedisItemReader]] launched with the following parameters: [{}]
+Executing step: [RedisItemReader]
+Verifying   2% ╺━━━━━━━━━━━━━━━━━  220/8942 (0:00:00 / 0:00:19) ?/s >0 T0 ≠Step: [RedisItemReader] executed in 7s521ms
+Closing with items still in queue
+Job: [SimpleJob: [name=RedisItemReader]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 7s522ms
+Verification completed - all OK
+Step: [verification] executed in 7s776ms
+Job: [SimpleJob: [name=snapshot-replication]] completed with the following parameters: [{}] and the following status: [COMPLETED] in 21s320ms
+```
+
+Total sync time ~1min

data/docs/secrets-and-env-values.md

@@ -0,0 +1,42 @@
+# Secrets and ENV Values
+
+You can store ENV values used by a container (within a workload) within Control Plane at the following levels:
+
+1. Workload Container
+2. GVC
+
+For your "review apps," it is convenient to have simple ENVs stored in plain text in your source code. You will want to
+keep some ENVs, like the Rails' `SECRET_KEY_BASE`, out of your source code. For staging and production apps, you will
+set these values directly at the GVC or workload levels, so none of these ENV values are committed to the source code.
+
+For storing ENVs in the source code, we can use a level of indirection so that you can store an ENV value in your source
+code like `cpln://secret/my-app-review-env-secrets.SECRET_KEY_BASE` and then have the secret value stored at the org
+level, which applies to your GVCs mapped to that org.
+
+For setting up secrets, you'll need:
+
+- **Org-level Secret:** This is where the values will be stored.
+- **GVC Identity:** An identity that must be associated with each workload that requires access to the secret.
+- **Org-level Policy:** A policy that binds the identity to the secret, granting the necessary permissions for the workload to access the secret.
+
+You can do this during the initial app setup, like this:
+
+1. Add the template for `app` to `.controlplane/templates`
+2. Ensure that the `app` template is listed in `setup_app_templates` for the app in `.controlplane/controlplane.yml`
+3. Run `cpflow setup-app -a $APP_NAME`
+4. The secrets, secrets policy and identity will be automatically created, along with the proper binding
+5. In the Control Plane console, upper left "Manage Org" menu, click on "Secrets"
+6. Find the created secret (it will be in the `$APP_PREFIX-secrets` format) and add the secret env vars there
+7. Use `cpln://secret/...` in the app to access the secret env vars (e.g., `cpln://secret/$APP_PREFIX-secrets.SOME_VAR`)
+
+Here are the manual steps for reference. We recommend that you follow the steps above:
+
+1. In the upper left of the Control Plane console, "Manage Org" menu, click on "Secrets"
+2. Create a secret with `Secret Type: Dictionary` (e.g., `my-secrets`) and add the secret env vars there
+3. In the upper left "Manage GVC" menu, click on "Identities"
+4. Create an identity (e.g., `my-identity`)
+5. Navigate to the workload that you want to associate with the identity created
+6. Click "Identity" on the left menu and select the identity created
+7. In the lower left "Access Control" menu, click on "Policies"
+8. Create a policy with `Target Kind: Secret` and add a binding with the `reveal` permission for the identity created
+9. Use `cpln://secret/...` in the app to access the secret env vars (e.g., `cpln://secret/my-secrets.SOME_VAR`)

data/docs/tips.md

@@ -0,0 +1,150 @@
+# Tips
+
+1. [GVCs vs. Orgs](#gvcs-vs-orgs)
+2. [RAM](#ram)
+3. [Remote IP](#remote-ip)
+4. [Secrets and ENV Values](/docs/secrets-and-env-values.md)
+5. [CI](#ci)
+6. [Memcached](#memcached)
+7. [Sidekiq](#sidekiq)
+   - [Quieting Non-Critical Workers During Deployments](#quieting-non-critical-workers-during-deployments)
+   - [Setting Up a Pre Stop Hook](#setting-up-a-pre-stop-hook)
+   - [Setting Up a Liveness Probe](#setting-up-a-liveness-probe)
+8. [Useful Links](#useful-links)
+
+## GVCs vs. Orgs
+
+- A "GVC" roughly corresponds to a Heroku "app."
+- Images are available at the org level.
+- Multiple GVCs within an org can use the same image.
+- You can have different images within a GVC and even within a workload. This flexibility is one of the key differences
+  compared to Heroku apps.
+
+## RAM
+
+Any workload replica that reaches the max memory is terminated and restarted. You can configure alerts for workload
+restarts and the percentage of memory used in the Control Plane UX.
+
+Here are the steps for configuring an alert for the percentage of memory used:
+
+1. Navigate to the workload that you want to configure the alert for
+2. Click "Metrics" on the left menu to go to Grafana
+3. On Grafana, go to the alerting page by clicking on the alert icon in the sidebar
+4. Click on "New alert rule"
+5. In the "Set a query and alert condition" section, select "Grafana managed alert"
+6. There should be a default query named `A`
+7. Change the data source of the query to `metrics`
+8. Click "Code" on the top right of the query and enter `mem_used{workload="workload_name"} / mem_reserved{workload="workload_name"} * 100`
+   (replace `workload_name` with the name of the workload)
+9. There should be a default expression named `B`, with the type `Reduce`, the function `Last`, and the input `A` (this
+   ensures that we're getting the last data point of the query)
+10. There should be a default expression named `C`, with the type `Threshold`, and the input `B` (this is where you
+    configure the condition for firing the alert, e.g., `IS ABOVE 95`)
+11. You can then preview the alert and check if it's firing or not based on the example time range of the query
+12. In the "Alert evaluation behavior" section, you can configure how often the alert should be evaluated and for how
+    long the condition should be true before firing (for example, you might want the alert only to be fired if the
+    percentage has been above `95` for more than 20 seconds)
+13. In the "Add details for your alert" section, fill out the name, folder, group, and summary for your alert
+14. In the "Notifications" section, you can configure a label for the alert if you're using a custom notification policy,
+    but there should be a default root route for all alerts
+15. Once you're done, save and exit in the top right of the page
+16. Click "Contact points" on the top menu
+17. Edit the `grafana-default-email` contact point and add the email where you want to receive notifications
+18. You should now receive notifications for the alert in your email
+
+![](assets/grafana-alert.png)
+
+The steps for configuring an alert for workload restarts are almost identical, but the code for the query would be
+`container_restarts`.
+
+For more information on Grafana alerts, see: https://grafana.com/docs/grafana/latest/alerting/
+
+## Remote IP
+
+The actual remote IP of the workload container is in the 127.0.0.x network, so that will be the value of the
+`REMOTE_ADDR` env var.
+
+However, Control Plane additionally sets the `x-forwarded-for` and `x-envoy-external-address` headers (and others - see:
+https://shakadocs.controlplane.com/concepts/security#headers). On Rails, the `ActionDispatch::RemoteIp` middleware should
+pick those up and automatically populate `request.remote_ip`.
+
+So `REMOTE_ADDR` should not be used directly, only `request.remote_ip`.
+
+## CI
+
+**Note:** Docker builds much slower on Apple Silicon, so try configuring CI to build the images when using Apple
+hardware.
+
+Make sure to create a profile on CI before running any `cpln` or `cpflow` commands.
+
+```sh
+CPLN_TOKEN=...
+cpln profile create default --token ${CPLN_TOKEN}
+```
+
+Also, log in to the Control Plane Docker repository if building and pushing an image.
+
+```sh
+cpln image docker-login
+```
+
+## Memcached
+
+On the workload container for Memcached (using the `memcached:alpine` image), configure the command with the args
+`-l 0.0.0.0`.
+
+To do this:
+
+1. Navigate to the workload container for Memcached
+2. Click "Command" on the top menu
+3. Add the args and save
+
+![](assets/memcached.png)
+
+## Sidekiq
+
+### Quieting Non-Critical Workers During Deployments
+
+To avoid locks in migrations, we can quiet non-critical workers during deployments. Doing this early enough in the CI
+allows all workers to finish jobs gracefully before deploying the new image.
+
+There's no need to unquiet the workers, as that will happen automatically after deploying the new image.
+
+```sh
+cpflow run 'rails runner "Sidekiq::ProcessSet.new.each { |w| w.quiet! unless w[%q(hostname)].start_with?(%q(criticalworker.)) }"' -a my-app
+```
+
+### Setting Up a Pre Stop Hook
+
+By setting up a pre stop hook in the lifecycle of the workload container for Sidekiq, which sends "QUIET" to the workers,
+we can ensure that all workers will finish jobs gracefully before Control Plane stops the replica. That also works
+nicely for multiple replicas.
+
+A couple of notes:
+
+- We can't use the process name as regex because it's Ruby, not Sidekiq.
+- We need to add a space after `sidekiq`; otherwise, it sends `TSTP` to the `sidekiqswarm` process as well, and for some
+  reason, that doesn't work.
+
+So with `^` and `\s`, we guarantee it's sent only to worker processes.
+
+```sh
+pkill -TSTP -f ^sidekiq\s
+```
+
+To do this:
+
+1. Navigate to the workload container for Sidekiq
+2. Click "Lifecycle" on the top menu
+3. Add the command and args below "Pre Stop Hook" and save
+
+![](assets/sidekiq-pre-stop-hook.png)
+
+### Setting Up a Liveness Probe
+
+To set up a liveness probe on port 7433, see: https://github.com/arturictus/sidekiq_alive
+
+## Useful Links
+
+- For best practices for the app's Dockerfile, see: https://lipanski.com/posts/dockerfile-ruby-best-practices
+- For migrating from Heroku Postgres to RDS, see: https://pelle.io/posts/hetzner-rds-postgres

data/docs/troubleshooting.md

@@ -0,0 +1,6 @@
+# Troubleshooting
+
+
+## App Web Page Shows `upstream request timeout`
+
+If you get a blank screen showing the message `upstream request timeout` on your app after running `cpflow open -a my-app-name`, check out the application logs. Your image has been promoted and your app crashing when starting.

data/examples/circleci.yml

@@ -0,0 +1,104 @@
+# Example config for staging app:
+# - triggers on push to master
+# - static app name
+# - no resource provisioning
+# - no database setup, only migration
+build-staging:
+  docker:
+    - image: cimg/ruby:3.1-node
+  resource_class: large
+  environment:
+    CPLN_ORG: my-org
+    APP_NAME: my-app-staging
+  steps:
+    - checkout
+    - setup_remote_docker:
+        docker_layer_caching: true
+    - run:
+        name: Install Control Plane tools
+        command: |
+          sudo npm install -g @controlplane/cli && cpln --version
+          cpln profile create default --token ${CPLN_TOKEN} --org ${CPLN_ORG} --gvc ${APP_NAME}
+          cpln image docker-login
+
+          gem install cpflow
+    - run:
+        name: Containerize and push image
+        command: cpflow build-image -a ${APP_NAME}
+    - run:
+        name: Database tasks
+        command: cpflow run -a ${APP_NAME} --image latest -- rails db:migrate
+    - run:
+        name: Deploy image
+        command: cpflow deploy-image -a ${APP_NAME}
+
+# Example config for review app:
+# - triggers manually if needed
+# - dynamic app name based on PR number
+# - resources provisioning for new apps
+# - initial database setup or migration
+build-review-app:
+  docker:
+    - image: cimg/ruby:3.1-node
+  resource_class: large
+  environment:
+    CPLN_ORG: my-org
+  steps:
+    - checkout
+    - setup_remote_docker:
+        docker_layer_caching: true
+    - run:
+        name: Setup environment
+        command: |
+          PR_NUM=$(echo $CIRCLE_PULL_REQUEST | grep -Eo '[0-9]+$')
+          echo "export APP_NAME=hichee-review-$PR_NUM" >> $BASH_ENV
+    - run:
+        name: Install Control Plane tools
+        command: |
+          sudo npm install -g @controlplane/cli && cpln --version
+          cpln profile create default --token ${CPLN_TOKEN} --org ${CPLN_ORG} --gvc ${APP_NAME}
+          cpln image docker-login
+
+          gem install cpflow
+    - run:
+        name: Provision review app if needed
+        command: |
+          if ! cpflow exists -a ${APP_NAME}; then
+            cpflow setup-app -a ${APP_NAME}
+            echo "export NEW_APP=true" >> $BASH_ENV
+          fi
+    - run:
+        name: Containerize and push image
+        command: |
+          cpflow build-image -a ${APP_NAME} --commit ${CIRCLE_SHA1::7}
+    - run:
+        name: Database tasks
+        command: |
+          if [ -n "${NEW_APP}" ]; then
+            cpflow run -a ${APP_NAME} --image latest -- rails db:reset
+          else
+            cpflow run -a ${APP_NAME} --image latest -- rails db:migrate
+          fi
+    - run:
+        name: Deploy image
+        command: cpflow deploy-image -a ${APP_NAME}
+
+review-app:
+  jobs:
+    - start:
+        filters:
+          branches:
+            ignore: master
+        type: approval
+    - build-review-app:
+        filters:
+          branches:
+            ignore: master
+        requires:
+          - start
+staging:
+  jobs:
+    - build-staging:
+        filters:
+          branches:
+            only: master

data/examples/controlplane.yml

@@ -0,0 +1,159 @@
+# Keys beginning with "cpln_" correspond to your settings in Control Plane.
+
+# Global settings that apply to `cpflow` usage.
+# You can opt out of allowing the use of CPLN_ORG and CPLN_APP env vars
+# to avoid any accidents with the wrong org / app.
+allow_org_override_by_env: true
+allow_app_override_by_env: true
+
+aliases:
+  common: &common
+    # Organization for staging and QA apps is typically set as an alias.
+    # Production apps will use a different organization, specified in `apps`, for security.
+    # Change this value to your organization name
+    # or set the CPLN_ORG env var and it will override this for all `cpflow` commands
+    # (provided that `allow_org_override_by_env` is set to `true`).
+    cpln_org: my-org-staging
+
+    # Control Plane offers the ability to use multiple locations.
+    # default_location is used for commands that require a location
+    # including `ps`, `run`, `apply-template`.
+    # This can be overridden with option --location=<location> and
+    # CPLN_LOCATION environment variable.
+    # TODO: Allow specification of multiple locations.
+    default_location: aws-us-east-2
+
+    # Allows running the command `cpflow setup-app`
+    # instead of `cpflow apply-template app redis postgres memcached rails sidekiq`.
+    #
+    # Note:
+    # 1. These names correspond to files in the `./controlplane/templates` directory.
+    # 2. Each file can contain many objects, such as in the case of templates that create a resource, like `postgres`.
+    # 3. While the naming often corresponds to a workload or other object name, the naming is arbitrary. 
+    #    Naming does not need to match anything other than the file name without the `.yml` extension.
+    setup_app_templates:
+      - app
+      - redis
+      - postgres
+      - memcached
+      - rails
+      - sidekiq
+
+    # Uncomment next line to skips secrets setup when running `cpflow setup-app`.
+    # skip_secrets_setup: true
+
+    # Only needed if using a custom secrets name.
+    # The default is '{APP_PREFIX}-secrets'. For example:
+    # - for an app 'my-app-staging' with `match_if_app_name_starts_with` set to `false`,
+    #   it would be 'my-app-staging-secrets'
+    # - for an app 'my-app-review-1234' with `match_if_app_name_starts_with` set to `true`,
+    #   it would be 'my-app-review-secrets'
+    secrets_name: my-secrets
+
+    # Only needed if using a custom secrets policy name.
+    # The default is '{APP_SECRETS}-policy'. For example:
+    # - for an app 'my-app-staging' with `match_if_app_name_starts_with` set to `false`,
+    #   it would be 'my-app-staging-secrets-policy'
+    # - for an app 'my-app-review-1234' with `match_if_app_name_starts_with` set to `true`,
+    #   it would be 'my-app-review-secrets-policy'
+    secrets_policy_name: my-secrets-policy
+
+    # Configure the workload name used as a template for one-off scripts, like a Heroku one-off dyno.
+    one_off_workload: rails
+
+    # Workloads that are for the application itself and are using application Docker images.
+    # These are updated with the new image when running the `deploy-image` command,
+    # and are also used by the `info` and `ps:` commands in order to get all of the defined workloads.
+    # On the other hand, if you have a workload for Redis, that would NOT use the application Docker image
+    # and not be listed here.
+    app_workloads:
+      - rails
+      - sidekiq
+
+    # Additional "service type" workloads, using non-application Docker images.
+    # These are only used by the `info` and `ps:` commands in order to get all of the defined workloads.
+    additional_workloads:
+      - redis
+      - postgres
+      - memcached
+
+    # Configure the workload name used when maintenance mode is on (defaults to "maintenance").
+    maintenance_workload: maintenance
+
+    # Fixes the remote terminal size to match the local terminal size
+    # when running `cpflow run`.
+    fix_terminal_size: true
+
+    # Sets a default CPU size for `cpflow run` jobs (can be overridden per job through `--cpu`).
+    # If not specified, defaults to "1" (1 core).
+    runner_job_default_cpu: "2"
+
+    # Sets a default memory size for `cpflow run` jobs (can be overridden per job through `--memory`).
+    # If not specified, defaults to "2Gi" (2 gibibytes).
+    runner_job_default_memory: "4Gi"
+
+    # Sets the maximum number of seconds that `cpflow run` jobs can execute before being stopped.
+    # If not specified, defaults to 21600 (6 hours).
+    runner_job_timeout: 1000
+
+    # Apps with a deployed image created before this amount of days will be listed for deletion
+    # when running the command `cpflow cleanup-stale-apps`.
+    stale_app_image_deployed_days: 5
+
+    # Images that exceed this quantity will be listed for deletion
+    # when running the command `cpflow cleanup-images`.
+    image_retention_max_qty: 20
+
+    # Images created before this amount of days will be listed for deletion
+    # when running the command `cpflow cleanup-images` (`image_retention_max_qty` takes precedence).
+    image_retention_days: 5
+
+apps:
+  my-app-staging:
+    # Use the values from the common section above.
+    <<: *common
+
+  my-app-review:
+    <<: *common
+
+    # If `match_if_app_name_starts_with` is `true`, then use this config for app names starting with this name,
+    # e.g., "my-app-review-pr123", "my-app-review-anything-goes", etc.
+    match_if_app_name_starts_with: true
+
+    # Hooks can be either a script path that exists in the app image or a command.
+    # They're run in the context of `cpflow run` with the latest image.
+    hooks:
+      # Used by the command `cpflow setup-app` to run a hook after creating the app.
+      post_creation: bundle exec rake db:prepare
+
+      # Used by the command `cpflow delete` to run a hook before deleting the app.
+      pre_deletion: bundle exec rake db:drop
+
+  my-app-production:
+    <<: *common
+
+    # You can also opt out of allowing the use of CPLN_ORG and CPLN_APP env vars per app.
+    # It's recommended to leave this off for production, to avoid any accidents.
+    allow_org_override_by_env: false
+    allow_app_override_by_env: false
+
+    # Use a different organization for production.
+    cpln_org: my-org-production
+
+    # Allows running the command `cpflow promote-app-from-upstream -a my-app-production`
+    # to promote the staging app to production.
+    upstream: my-app-staging
+
+    # Used by the command `cpflow promote-app-from-upstream` to run a release script before deploying.
+    # This is relative to the `.controlplane/` directory.
+    release_script: release_script
+
+    # default_domain is used for commands that require a domain
+    # including `maintenance`, `maintenance:on`, `maintenance:off`.
+    default_domain: domain.com
+
+  my-app-other:
+    <<: *common
+
+    # You can specify a different `Dockerfile` relative to the `.controlplane/` directory (defaults to "Dockerfile").
+    dockerfile: ../some_other/Dockerfile