cpl 2.0.0.rc.1 → 2.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '03793c0046fc403a4fb42583e57c3449394a52da4c03191d3af845e2e5e0b53f'
-  data.tar.gz: c882856e066cc3a63f0d8f52738e8e432a72cec33dc565224159a554949a2f5b
+  metadata.gz: 7601f22a541072ea22c5745496d77ee63409d8975903f14582e2681963e9e0ff
+  data.tar.gz: e3a189dbb1506571133924f9640bd144c6c1a7d1a7233096b9fbd13bef8b0a82
 SHA512:
-  metadata.gz: 788fa61c7d5710e72ea00db402770d3fcfd037938ce0ce1ef8134e0614a024cddb7e4629223598a509f33abf2dec499e64bf23a52b0b86b823eafe59b3522cd5
-  data.tar.gz: 6ba9d8cdab48da090c7efd86eb9e14f05845ed1f219c6d251ce18ad636e34d678562d092ad52806dae3a4d106cd4760402463c823d9e66158f14e6cd806fc520
+  metadata.gz: f0e2603b1e9d29e2f23deb2620edb94a51e13763a48985064facfd9ce2f9a61731267176766ad037e826430283c253300334e35514c9de60a5cda59a97c8b81a
+  data.tar.gz: 8c97ac2b7d5f349d90a41bf1acb8dea6e72d107cb275361792547ca9faa4b1bca66bc2566558a05fb336e1bee39d577c9c5cc0e3602971037cd2d44d5f975ed8

data/CHANGELOG.md

@@ -14,14 +14,23 @@ Changes since the last non-beta release.
 
 _Please add entries here for your pull requests that are not yet released._
 
-## [2.0.0.rc.0] - 2024-05-10
+### Fixed
+
+- Fixed issue where `cleanup-stale-apps` command fails to delete apps with volumesets. [PR 175](https://github.com/shakacode/heroku-to-control-plane/pull/175) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
+
+## [2.0.0] - 2024-05-14
 
 ### BREAKING CHANGES
 
 - Commands that finished with a failure now exit with code `64` instead of `1`. [PR 132](https://github.com/shakacode/heroku-to-control-plane/pull/132) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
+- Bumped minimum `cpln` version to `2.0.1` (`cpln workload cron get` is required). [PR 171](https://github.com/shakacode/heroku-to-control-plane/pull/171) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
 - `run:cleanup` command has been removed. [PR 151](https://github.com/shakacode/heroku-to-control-plane/pull/151) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
 - `deploy-image` command now runs the release script in the context of the `run` command. [PR 151](https://github.com/shakacode/heroku-to-control-plane/pull/151) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
 
+### Fixed
+
+- Fixed race conditions when using latest image in `run` command. [PR 163](https://github.com/shakacode/heroku-to-control-plane/pull/163) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
+
 ### Added
 
 - Added options to `run` command to override the workload container's `--cpu`, `--memory`, and `--entrypoint`. [PR 151](https://github.com/shakacode/heroku-to-control-plane/pull/151) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
@@ -32,9 +41,12 @@ _Please add entries here for your pull requests that are not yet released._
 
 ### Changed
 
+- An error is now raised if the org does not exist. [PR 167](https://github.com/shakacode/heroku-to-control-plane/pull/167) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
+- Common options are now shown in help. [PR 169](https://github.com/shakacode/heroku-to-control-plane/pull/169) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
 - `run` command now uses a single reusable cron workload and works for both interactive and non-interactive jobs. [PR 151](https://github.com/shakacode/heroku-to-control-plane/pull/151) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
 - `run:detached` command has been deprecated in favor of `run`. [PR 151](https://github.com/shakacode/heroku-to-control-plane/pull/151) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
 - `deploy-image` command now raises an error if image does not exist. [PR 153](https://github.com/shakacode/heroku-to-control-plane/pull/153) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
+- `delete` command now unbinds identity from policy (if bound) when deleting app. [PR 170](https://github.com/shakacode/heroku-to-control-plane/pull/170) by [Rafael Gomes](https://github.com/rafaelgomesxyz).
 
 ## [1.4.0] - 2024-03-20
 
@@ -178,8 +190,8 @@ _Please add entries here for your pull requests that are not yet released._
 
 - Initial release
 
-[Unreleased]: https://github.com/shakacode/heroku-to-control-plane/compare/v2.0.0.rc.0...HEAD
-[2.0.0.rc.0]: https://github.com/shakacode/heroku-to-control-plane/compare/v1.4.0...v2.0.0.rc.0
+[Unreleased]: https://github.com/shakacode/heroku-to-control-plane/compare/v2.0.0...HEAD
+[2.0.0]: https://github.com/shakacode/heroku-to-control-plane/compare/v1.4.0...v2.0.0
 [1.4.0]: https://github.com/shakacode/heroku-to-control-plane/compare/v1.3.0...v1.4.0
 [1.3.0]: https://github.com/shakacode/heroku-to-control-plane/compare/v1.2.0...v1.3.0
 [1.2.0]: https://github.com/shakacode/heroku-to-control-plane/compare/v1.1.2...v1.2.0

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    cpl (2.0.0.rc.1)
+    cpl (2.0.1)
       debug (~> 1.7.1)
       dotenv (~> 2.8.1)
       jwt (~> 2.8.1)
@@ -52,7 +52,7 @@ GEM
     rdoc (6.6.3.1)
       psych (>= 4.0.0)
     regexp_parser (2.9.0)
-    reline (0.5.6)
+    reline (0.5.7)
       io-console (~> 0.5)
     rexml (3.2.6)
     rspec (3.12.0)

data/docs/commands.md

@@ -66,7 +66,8 @@ cpl cleanup-images -a $APP_NAME
 
 ### `cleanup-stale-apps`
 
-- Deletes the whole app (GVC with all workloads and all images) for all stale apps
+- Deletes the whole app (GVC with all workloads, all volumesets and all images) for all stale apps
+- Also unbinds the app from the secrets policy, as long as both the identity and the policy exist (and are bound)
 - Stale apps are identified based on the creation date of the latest image
 - Specify the amount of days after an app should be considered stale through `stale_app_image_deployed_days` in the `.controlplane/controlplane.yml` file
 - If `match_if_app_name_starts_with` is `true` in the `.controlplane/controlplane.yml` file, it will delete all stale apps that start with the name
@@ -106,6 +107,7 @@ cpl copy-image-from-upstream -a $APP_NAME --upstream-token $UPSTREAM_TOKEN --ima
 ### `delete`
 
 - Deletes the whole app (GVC with all workloads, all volumesets and all images) or a specific workload
+- Also unbinds the app from the secrets policy, as long as both the identity and the policy exist (and are bound)
 - Will ask for explicit user confirmation
 
 ```sh

data/docs/redis.md

@@ -1,15 +1,31 @@
-# Migrating Redis database from Heroku infrastructure
+# Migrating Redis databases
 
-**General considerations:**
+There are two templates examples in this repo:
+- `redis` - basic non-persistent template. It is good for review-apps or staging or where no persistence is required
+- `redis2` - basic persistent template. Good for production where persistence is needed, but cluster is overkill.
+
+## Option 1: use SLAVEOF (easier way)
+
+1. create a redis workload that will accept data
+2. execute `SLAVEOF source_host source_port`, if needed use `masterauth` to provide auth details
+3. wait for replication to pick up all changes (usually quickly), use `INFO` or `DBSIZE` to check progress
+4. stop app completely and ensure nothing is writing to any of redises
+5. execute `SLAVEOF no one` to disconnect replication
+6. switch `REDIS_URL` in the app to point to new server
+7. start the app
+
+## Option 2: use Redis-RIOT (harder way, where option 1 is not possible)
+
+### General considerations:
 
 1. Heroku uses self-signed TLS certificates, which are not verifiable. It needs special handling by setting
-TLS verification to `none`, otherwise most apps are not able to connect.
+The tool that satisfies those criteria is [Redis-RIOT](https://developer.redis.com/riot/riot-redis/index.html)
 
 2. We are moving to private Redis that don't have a public URL, so have to do it from a Control Plane GVC container.
 
 The tool that satisfies those criteria is [Redis-RIOT](https://developer.redis.com/riot/riot-redis/index.html)
 
-**Heroku Redis:**
+### Heroku Redis:
 
 As Redis-RIOT says, master redis should have keyspace-notifications set to `KA` to be able to do live replication.
 To do that:
@@ -23,7 +39,7 @@ Connect to heroku Redis CLI:
 heroku redis:cli -a my-app
 ```
 
-**Control Plane Redis:**
+### Control Plane Redis:
 
 Connect to Control Plane Redis CLI:
 
@@ -36,10 +52,10 @@ apt-get update
 apt-get install redis -y
 
 # connect to local cloud Redis
-redis-cli -u MY_CONTROL_PLANE_REDIS_URL
+redis-cli -u MY_CONTROL_PLANE_REDIS_URL -p 6379
 ```
 
-**Useful Redis CLI commands:**
+### Useful Redis CLI commands:
 
 Quick-check keys qty:
 ```
@@ -49,7 +65,7 @@ info keyspace
 db0:keys=9496,expires=2941,avg_ttl=77670114535
 ```
 
-**Create a Control Plane sync workload**
+### Create a Control Plane sync workload
 
 ```
 name: riot-redis
@@ -76,7 +92,7 @@ command args:
   live
 ```
 
-**Sync process**
+### Sync process
 
 1. open 1st terminal window with heroku redis CLI, check keys qty
 2. open 2nd terminal window with controlplane redis CLI, check keys qty

data/lib/command/cleanup_stale_apps.rb

@@ -7,9 +7,10 @@ module Command
       app_option(required: true),
       skip_confirm_option
     ].freeze
-    DESCRIPTION = "Deletes the whole app (GVC with all workloads and all images) for all stale apps"
+    DESCRIPTION = "Deletes the whole app (GVC with all workloads, all volumesets and all images) for all stale apps"
     LONG_DESCRIPTION = <<~DESC
-      - Deletes the whole app (GVC with all workloads and all images) for all stale apps
+      - Deletes the whole app (GVC with all workloads, all volumesets and all images) for all stale apps
+      - Also unbinds the app from the secrets policy, as long as both the identity and the policy exist (and are bound)
       - Stale apps are identified based on the creation date of the latest image
       - Specify the amount of days after an app should be considered stale through `stale_app_image_deployed_days` in the `.controlplane/controlplane.yml` file
       - If `match_if_app_name_starts_with` is `true` in the `.controlplane/controlplane.yml` file, it will delete all stale apps that start with the name
@@ -28,8 +29,8 @@ module Command
 
       progress.puts
       stale_apps.each do |app|
-        delete_gvc(app)
-        delete_images(app)
+        delete_app(app[:name])
+        progress.puts
       end
     end
 
@@ -57,8 +58,7 @@ module Command
 
             apps.push({
                         name: app_name,
-                        date: created_date,
-                        images: images.map { |current_image| current_image["name"] }
+                        date: created_date
                       })
           end
 
@@ -72,18 +72,8 @@ module Command
       Shell.confirm("\nAre you sure you want to delete these #{stale_apps.length} apps?")
     end
 
-    def delete_gvc(app)
-      step("Deleting app '#{app[:name]}'") do
-        cp.gvc_delete(app[:name])
-      end
-    end
-
-    def delete_images(app)
-      app[:images].each do |image|
-        step("Deleting image '#{image}'") do
-          cp.image_delete(image)
-        end
-      end
+    def delete_app(app)
+      Cpl::Cli.start(["delete", "-a", app, "--yes"])
     end
   end
 end

data/lib/command/delete.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Command
-  class Delete < Base
+  class Delete < Base # rubocop:disable Metrics/ClassLength
     NAME = "delete"
     OPTIONS = [
       app_option(required: true),
@@ -11,6 +11,7 @@ module Command
     DESCRIPTION = "Deletes the whole app (GVC with all workloads, all volumesets and all images) or a specific workload"
     LONG_DESCRIPTION = <<~DESC
       - Deletes the whole app (GVC with all workloads, all volumesets and all images) or a specific workload
+      - Also unbinds the app from the secrets policy, as long as both the identity and the policy exist (and are bound)
       - Will ask for explicit user confirmation
     DESC
     EXAMPLES = <<~EX
@@ -35,7 +36,9 @@ module Command
     private
 
     def delete_single_workload(workload)
-      return progress.puts("Workload '#{workload}' does not exist.") if cp.fetch_workload(workload).nil?
+      if cp.fetch_workload(workload).nil?
+        return progress.puts("Workload '#{workload}' does not exist in app '#{config.app}'.")
+      end
       return unless confirm_delete(workload)
 
       delete_workload(workload)
@@ -48,6 +51,7 @@ module Command
       check_images
       return unless confirm_delete(config.app)
 
+      unbind_identity_from_policy
       delete_volumesets
       delete_gvc
       delete_images
@@ -55,9 +59,9 @@ module Command
 
     def check_volumesets
       @volumesets = cp.fetch_volumesets["items"]
-      return progress.puts("No volumesets to delete.") unless @volumesets.any?
+      return progress.puts("No volumesets to delete from app '#{config.app}'.") unless @volumesets.any?
 
-      message = "The following volumesets will be deleted along with the app:"
+      message = "The following volumesets will be deleted along with the app '#{config.app}':"
       volumesets_list = @volumesets.map { |volumeset| "- #{volumeset['name']}" }.join("\n")
       progress.puts("#{Shell.color(message, :red)}\n#{volumesets_list}\n\n")
     end
@@ -65,9 +69,9 @@ module Command
     def check_images
       @images = cp.query_images["items"]
                   .select { |image| image["name"].start_with?("#{config.app}:") }
-      return progress.puts("No images to delete.") unless @images.any?
+      return progress.puts("No images to delete from app '#{config.app}'.") unless @images.any?
 
-      message = "The following images will be deleted along with the app:"
+      message = "The following images will be deleted along with the app '#{config.app}':"
       images_list = @images.map { |image| "- #{image['name']}" }.join("\n")
       progress.puts("#{Shell.color(message, :red)}\n#{images_list}\n\n")
     end
@@ -89,14 +93,14 @@ module Command
     end
 
     def delete_workload(workload)
-      step("Deleting workload '#{workload}'") do
+      step("Deleting workload '#{workload}' from app '#{config.app}'") do
         cp.delete_workload(workload)
       end
     end
 
     def delete_volumesets
       @volumesets.each do |volumeset|
-        step("Deleting volumeset '#{volumeset['name']}'") do
+        step("Deleting volumeset '#{volumeset['name']}' from app '#{config.app}'") do
           # If the volumeset is attached to a workload, we need to delete the workload first
           workload = volumeset.dig("status", "usedByWorkload")&.split("/")&.last
           cp.delete_workload(workload) if workload
@@ -108,10 +112,26 @@ module Command
 
     def delete_images
       @images.each do |image|
-        step("Deleting image '#{image['name']}'") do
+        step("Deleting image '#{image['name']}' from app '#{config.app}'") do
           cp.image_delete(image["name"])
         end
       end
     end
+
+    def unbind_identity_from_policy
+      return if cp.fetch_identity(app_identity).nil?
+
+      policy = cp.fetch_policy(app_secrets_policy)
+      return if policy.nil?
+
+      is_bound = policy["bindings"].any? do |binding|
+        binding["principalLinks"].any? { |link| link == app_identity_link }
+      end
+      return unless is_bound
+
+      step("Unbinding identity from policy for app '#{config.app}'") do
+        cp.unbind_identity_from_policy(app_identity_link, app_secrets_policy)
+      end
+    end
   end
 end

data/lib/core/controlplane.rb

@@ -8,6 +8,8 @@ class Controlplane # rubocop:disable Metrics/ClassLength
     @api = ControlplaneApi.new
     @gvc = config.app
     @org = config.org
+
+    ensure_org_exists! if org
   end
 
   # profile
@@ -339,6 +341,11 @@ class Controlplane # rubocop:disable Metrics/ClassLength
     perform!(cmd)
   end
 
+  def unbind_identity_from_policy(identity_link, policy)
+    cmd = "cpln policy remove-binding #{policy} --org #{org} --identity #{identity_link} --permission reveal"
+    perform!(cmd)
+  end
+
   # apply
   def apply_template(data) # rubocop:disable Metrics/MethodLength
     Tempfile.create do |f|
@@ -404,6 +411,18 @@ class Controlplane # rubocop:disable Metrics/ClassLength
 
   private
 
+  def org_exists?
+    items = api.list_orgs["items"]
+    items.any? { |item| item["name"] == org }
+  end
+
+  def ensure_org_exists!
+    return if org_exists?
+
+    raise "Can't find org '#{org}', please create it in the Control Plane dashboard " \
+          "or ensure that the name is correct."
+  end
+
   # `output_mode` can be :all, :errors_only or :none.
   # If not provided, it will be determined based on the `HIDE_COMMAND_OUTPUT` env var
   # or the return value of `Shell.should_hide_output?`.

data/lib/core/controlplane_api.rb

@@ -1,6 +1,10 @@
 # frozen_string_literal: true
 
 class ControlplaneApi # rubocop:disable Metrics/ClassLength
+  def list_orgs
+    api_json("/org", method: :get)
+  end
+
   def gvc_list(org:)
     api_json("/org/#{org}/gvc", method: :get)
   end

data/lib/cpl/version.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
 module Cpl
-  VERSION = "2.0.0.rc.1"
-  MIN_CPLN_VERSION = "0.0.71"
+  VERSION = "2.0.1"
+  MIN_CPLN_VERSION = "2.0.1"
 end

data/lib/cpl.rb

@@ -146,9 +146,22 @@ module Cpl
       ::Command::Base.all_commands.merge(deprecated_commands)
     end
 
+    def self.process_option_params(params)
+      # Ensures that if no value is provided for a non-boolean option (e.g., `cpl command --option`),
+      # it defaults to an empty string instead of the option name (which is the default Thor behavior)
+      params[:lazy_default] ||= "" if params[:type] != :boolean
+
+      params
+    end
+
     @commands_with_required_options = []
     @commands_with_extra_options = []
 
+    ::Command::Base.common_options.each do |option|
+      params = process_option_params(option[:params])
+      class_option(option[:name], **params)
+    end
+
     all_base_commands.each do |command_key, command_class| # rubocop:disable Metrics/BlockLength
       deprecated = deprecated_commands[command_key]
 
@@ -157,7 +170,7 @@ module Cpl
       usage = command_class::USAGE.empty? ? name : command_class::USAGE
       requires_args = command_class::REQUIRES_ARGS
       default_args = command_class::DEFAULT_ARGS
-      command_options = command_class::OPTIONS + ::Command::Base.common_options
+      command_options = command_class::OPTIONS
       accepts_extra_options = command_class::ACCEPTS_EXTRA_OPTIONS
       description = command_class::DESCRIPTION
       long_description = command_class::LONG_DESCRIPTION
@@ -175,12 +188,7 @@ module Cpl
       long_desc(long_description)
 
       command_options.each do |option|
-        params = option[:params]
-
-        # Ensures that if no value is provided for a non-boolean option (e.g., `cpl command --option`),
-        # it defaults to an empty string instead of the option name (which is the default Thor behavior)
-        params[:lazy_default] ||= "" if params[:type] != :boolean
-
+        params = process_option_params(option[:params])
         method_option(option[:name], **params)
       end
 
@@ -208,7 +216,7 @@ module Cpl
         end
 
         begin
-          Cpl::Cli.validate_options!(options, command_options)
+          Cpl::Cli.validate_options!(options)
 
           config = Config.new(args, options, required_options)
 
@@ -227,11 +235,11 @@ module Cpl
     check_unknown_options!(except: @commands_with_extra_options)
     stop_on_unknown_option!
 
-    def self.validate_options!(options, command_options)
+    def self.validate_options!(options)
       options.each do |name, value|
         raise "No value provided for option '#{name}'." if value.to_s.strip.empty?
 
-        params = command_options.find { |option| option[:name].to_s == name }[:params]
+        params = ::Command::Base.all_options.find { |option| option[:name].to_s == name }[:params]
         next unless params[:valid_regex]
 
         raise "Invalid value provided for option '#{name}'." unless value.match?(params[:valid_regex])

data/templates/memcached.yml

@@ -4,8 +4,8 @@ spec:
   type: standard
   containers:
     - name: memcached
-      cpu: 3m
-      memory: 10Mi
+      cpu: 25m
+      memory: 32Mi
       args:
         - "-l"
         - 0.0.0.0
@@ -15,7 +15,7 @@ spec:
           protocol: tcp
   defaultOptions:
     autoscaling:
-      metric: latency
+      metric: disabled
       minScale: 1
       maxScale: 1
     capacityAI: false

data/templates/postgres.yml

@@ -23,7 +23,7 @@ spec:
           uri: "scratch://postgres-vol"
   defaultOptions:
     autoscaling:
-      metric: latency
+      metric: disabled
       minScale: 1
       maxScale: 1
     capacityAI: false

data/templates/redis.yml

@@ -4,15 +4,15 @@ spec:
   type: standard
   containers:
     - name: redis
-      cpu: 3m
-      memory: 20Mi
+      cpu: 25m
+      memory: 32Mi
       image: "redis:latest"
       ports:
         - number: 6379
           protocol: tcp
   defaultOptions:
     autoscaling:
-      metric: latency
+      metric: disabled
       minScale: 1
       maxScale: 1
     capacityAI: false

data/templates/redis2.yml

@@ -0,0 +1,37 @@
+kind: volumeset
+name: redis-data
+spec:
+  fileSystemType: ext4
+  initialCapacity: 10
+  performanceClass: general-purpose-ssd
+---
+kind: workload
+name: redis2
+spec:
+  type: stateful
+  containers:
+    - name: redis
+      args:
+        - '--appendonly'
+        - 'yes'
+        - '--maxmemory'
+        - 25mb
+      cpu: 25m
+      memory: 32Mi
+      image: "redis:latest"
+      ports:
+        - number: 6379
+          protocol: tcp
+      volumes:
+        - path: /data
+          recoveryPolicy: retain
+          uri: cpln://volumeset/redis-data
+  defaultOptions:
+    autoscaling:
+      metric: disabled
+      minScale: 1
+      maxScale: 1
+    capacityAI: false
+  firewallConfig:
+    internal:
+      inboundAllowType: same-gvc

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: cpl
 version: !ruby/object:Gem::Version
-  version: 2.0.0.rc.1
+  version: 2.0.1
 platform: ruby
 authors:
 - Justin Gordon
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-05-11 00:00:00.000000000 Z
+date: 2024-05-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: debug
@@ -310,6 +310,7 @@ files:
 - templates/postgres.yml
 - templates/rails.yml
 - templates/redis.yml
+- templates/redis2.yml
 - templates/secrets.yml
 - templates/sidekiq.yml
 homepage: https://github.com/shakacode/heroku-to-control-plane
@@ -328,9 +329,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: 2.7.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">"
+  - - ">="
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubygems_version: 3.4.21
 signing_key: