convox_installer 2.0.0 → 3.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b9b7140386f111c59db16b8db7429b2fb3a5c1a48372a2559e2e2c5499d59dd2
-  data.tar.gz: 6e94e2c459c01426f8b6d2adc7d232f630e9711722947d2899b20ea509089c47
+  metadata.gz: ea8a81c025b1a5046d3145beeee5cba92f8448cc8751182d911f3548b289c31a
+  data.tar.gz: 8eb61f167e06eb67bcd86eada0196d138995b3e61b237ce04f11891f6eb1426c
 SHA512:
-  metadata.gz: 64741ea4bd8a4e03d30172d366976e7d18c99cd2dff1f27ceaf5106d86de56c46fc0302fed774669f5a513d3b5cd8212b3b94e54687a539b235e545b3a9c90cb
-  data.tar.gz: 67966065258259f0b64c7572895ee884f4e34c55e4f9c1ddbff90cf431e8da78984706955b406655d8f0128f4edd21842559129aa5b7dde73a745a121ad44ce1
+  metadata.gz: db7f9e671230b4054a888397461056af9f8ee2d631e0d05fb4eda32a138ae947e7f7fdee8706ea3b9b695254d9fa8b5c2971222a68d8ba4ad8a1326917357f65
+  data.tar.gz: 8dd7d82786491eca18253d6dc405675fbe6c188c0276878ed352e3f9c17770aca9a4906b8819d061dd6940a203832e0808e9f1aa8427bddc9505d89974f39c69

data/.rubocop.yml

@@ -8,7 +8,7 @@ Style/MixinUsage:
   Exclude:
     - examples/**/*
 
-Metrics/LineLength:
+Layout/LineLength:
   Max: 100
 
 Style/StringLiterals:

data/.vscode/settings.json

@@ -1,4 +1,5 @@
 {
-  "editor.formatOnSave": false,
-  "editor.formatOnSaveTimeout": 5000
+  "editor.formatOnSave": true,
+  "editor.formatOnSaveTimeout": 5000,
+  "ruby.rubocop.executePath": "/usr/local/bin/rubocop-daemon-wrapper/"
 }

data/Gemfile

@@ -9,6 +9,7 @@ group :test do
   gem 'pry-byebug', require: false
   gem 'rspec'
   gem 'rubocop'
+  gem 'rubocop-daemon'
   gem 'rubocop-rspec'
   gem 'vcr'
   gem 'webmock'

data/README.md

@@ -1,10 +1,23 @@
 # Convox Installer
 
-A Ruby gem that makes it easier to build a Convox installation script. This is like Chef/Ansible/Terraform for your initial Convox setup.
+A Ruby gem that makes it easier to build a Convox installation script. The main purpose of this gem is to make it easier to set up on-premise installations of your app for enterprise users.
 
-# Requirements: Convox v2
+This gem provides a DSL so that you can write a script that walks your users through setting up Convox and getting your app and running, setting up S3 buckets, etc.
+
+## Requirements
+
+- MacOS
+- Convox v3 CLI
+- Runtime integration installed in your AWS account. See: https://docs.convox.com/getting-started/introduction/
+
+_Please let us know if you need to run this script on Linux. Linux support should not be too difficult to implement, but unfortunately we probably won't be able to support Windows._
+
+### Requires Convox >= 3
+
+This version of `convox_installer` is only designed to work with Convox 3 and later. You can run `convox version` to check your version. Please install the Convox v3 CLI by following the instructions here: https://docs.convox.com/getting-started/introduction/
+
+_If you want to set up a Convox v2 rack (deprecated), the last version of `convox_installer` that supports the v2 CLI is `1.0.9`. (Take a look at [the `convox2` branch](https://github.com/DocSpring/convox_installer/tree/convox2).)_
 
-This version of `convox_installer` is only designed to work with Convox v2. Please install the v2 CLI by following the instructions at their old documentation site: https://docsv2.convox.com/introduction/installation
 ## USE AT YOUR OWN RISK! THIS CODE IS PROVIDED WITHOUT ANY WARRANTIES OR GUARANTEES
 
 We have successfully set up a number of test and production deployments using this gem. Everything seems to work very well. The library also facilitates idempotency and crash-resistance, so you can easily re-run your installation script if something goes wrong. However, if anything goes wrong, then you can end up with a large AWS bill if you're not careful. If anything crashes then make sure you double-check everything in your AWS account and shut down any leftover resources. **USE THIS SOFTWARE AT YOUR OWN RISK.**
@@ -25,7 +38,7 @@ We have successfully set up a number of test and production deployments using th
 
 `convox_installer` is a Ruby gem that makes it much easier to build an installation script for `convox/rack` (the open source PaaS). The Convox CLI is awesome, but it's missing a nice way to script a full deployment. I originally wrote a bash script that made API calls and used [`jq`](https://stedolan.github.io/jq/) and `sed`, but this was very error-prone and it did not have good cross-platform support.
 
-I've rewritten this installation script in Ruby, which provides very good cross-platform support, and also allows me to write tests.
+I've written this installation script in Ruby, which provides very good cross-platform support, and also allows me to write tests.
 
 ## Usage
 
@@ -37,7 +50,7 @@ require 'bundler/inline'
 
 gemfile do
   source 'https://rubygems.org'
-  gem 'convox_installer'
+  gem 'convox_installer', '3.0.0'
 end
 
 require "convox_installer"
@@ -141,10 +154,6 @@ Makes sure that the `convox` and `aws` CLI tools are installed on this system. I
 Loads config from ENV vars, or from saved config at `./.installer_config.json`.
 If any config settings are missing, it prompts the user for input. Finally, it shows a summary of the config, and asks the user if they want to proceed with the installation. If the user enters `y` (or `yes`), the `prompt_for_config` method completes. If they enter `n` (or `no`), we loop over every setting and let them press "enter" to keep the current value, or provide a new value to correct any mistakes.
 
-#### `backup_convox_host_and_rack`
-
-If there are any existing files at `~/.convox/host` or `~/.convox/rack`, this method moves these to `~/.convox/host.bak` and `~/.convox/rack.bak`.
-
 #### `install_convox`
 
 - **Required Config:** `aws_region`, `aws_access_key_id`, `aws_secret_access_key`,
@@ -152,13 +161,13 @@ If there are any existing files at `~/.convox/host` or `~/.convox/rack`, this me
 
 Runs `convox rack install ...`. Has some validations to ensure that all required settings are present.
 
-#### `validate_convox_auth_and_write_host!`
+#### `validate_convox_rack_and_write_current!`
 
-After running `install_convox`, call this method to ensure that the the `~/.convox/auth` file has been updated with the correct details (checks the rack name and AWS region.) Then it sets the rack host in `~/.convox/host` (if not already set.)
+Ensures that the local machine contains a directory for the rack's terraform config, and sets the current rack for Convox CLI commands.
 
-#### `validate_convox_rack!`
+#### `validate_convox_rack_api!`
 
-Calls `convox api get /system` to get the Rack details, then makes sure that everything is correct.
+Makes an API request (`convox api get /system`) to get the rack details, and makes sure that everything is correct.
 
 #### `convox_rack_data`
 
@@ -174,10 +183,14 @@ Checks if the app already exists. If not, calls `convox apps create ... --wait`
 
 Writes the app name into `./.convox/app` (in the current directory.) The `convox` CLI reads this file, so you don't need to specify the `--app` flag for future commands.
 
-#### `create_s3_bucket!`
+#### `add_s3_bucket`
+
+Adds an S3 bucket to your Terraform config.
 
 - **Required Config:** `s3_bucket_name`
 
+NOTE: This method just writes a new Terraform configuration file. You must run `apply_terraform_update!` to apply the changes and create the S3 bucket.
+
 Creates an S3 bucket from the `:s3_bucket_name` config setting. This is not a default setting, so you can add something like this to your custom `@prompts`:
 
 ```ruby
@@ -190,41 +203,97 @@ Creates an S3 bucket from the `:s3_bucket_name` config setting. This is not a de
 
 The `:value` `Proc` will generate a bucket name with a random suffix. (Avoids conflicts when you are setting up multiple deployments for your app.)
 
-`create_s3_bucket!` will also call `set_s3_bucket_cors_policy` automatically, so you don't need to call this manually.
-
-#### `set_s3_bucket_cors_policy`
+You can also set a CORS policy for your S3 bucket. (`:s3_bucket_name`)
+We set the `cors_rule` option for the `aws_s3_bucket` resource in the Terraform configuration. Example:
 
-- **Required Config:** `s3_bucket_name`
+```
+   cors_rule {
+    allowed_headers = ["*"]
+    allowed_methods = ["PUT", "POST"]
+    allowed_origins = ["https://s3-website-test.hashicorp.com"]
+    expose_headers  = ["ETag"]
+    max_age_seconds = 3000
+  }
+```
 
-Set up a CORS policy for your S3 bucket. (`:s3_bucket_name`)
+See: https://registry.terraform.io/providers/hashicorp/aws/3.33.0/docs/resources/s3_bucket#using-cors
 
-_Note: If the `:s3_bucket_cors_policy` setting is not provided, then this method does nothing._
+_Note: If the `:s3_bucket_cors_rule` setting is not provided, then it is skipped._
 
-You should set `:s3_bucket_cors_policy` to a JSON string. Here's how I set this up in my own `install.rb` script:
+Here's how we set up a CORS policy in our own `install.rb` script:
 
 ```ruby
-S3_BUCKET_CORS_POLICY = <<-JSON
-{
-  "CORSRules": [
-    {
-      "AllowedOrigins": ["*"],
-      "AllowedHeaders": ["Authorization", "cache-control", "x-requested-with"],
-      "AllowedMethods": ["PUT", "POST", "GET"],
-      "MaxAgeSeconds": 3000
-    }
-  ]
-}
-JSON
+xxxxc = <<-TERRAFORM
+  cors_rule {
+    allowed_headers = ["Authorization", "cache-control", "x-requested-with"]
+    allowed_methods = ["PUT", "POST", "GET"]
+    allowed_origins = ["*"]
+    expose_headers  = []
+    max_age_seconds = 3000
+  }
+TERRAFORM
 
 @prompts = [
   {
-    key: :s3_bucket_cors_policy,
-    value: S3_BUCKET_CORS_POLICY,
+    key: :s3_bucket_cors_rule,
+    value: S3_BUCKET_CORS_RULE,
     hidden: true,
   }
 ]
 ```
 
+#### `add_rds_database`
+
+Adds an RDS database to your Terraform config.
+
+- **Required Config:**
+  - `database_username`
+  - `database_password`
+- **Optional Config:**
+  - `database_allocated_storage` _(default: 30)_
+  - `database_engine` _(default: 'postgres')_
+  - `database_engine_version` _(default: '14.2')_
+  - `database_instance_class` _(default: 'db.t3.medium')_
+  - `database_multi_az` _(default: true)_
+
+#### `add_add_elasticache_cluster`
+
+Adds an Elasticache cluster to your Terraform config.
+
+- **Optional Config:**
+  - `engine` _(default: 'redis')_
+  - `engine_version` _(default: '6.2.5')_
+  - `node_type` _(default: 'cache.m3.medium')_
+  - `database_instance_class` _(default: 'db.t3.medium')_
+  - `num_cache_nodes` _(default: 1)_
+  - `port` _(default: 6379)_
+
+_IMPORTANT: Make sure you specify a full version string (e.g. `6.2.5`), and not a partial version (e.g. `6.2`.) A partial version will cause Terraform to delete and recreate the cluster on every run._
+
+#### `apply_terraform_update!`
+
+Runs `terraform apply -auto-approve` to apply any changes to your Terraform configuration (add new resources, etc.)
+
+#### `rds_details`
+
+Returns information about the created RDS database resource.
+
+```ruby
+{
+  postgres_url: "Full URL for the RDS database (including auth)",
+}
+```
+
+#### `elasticache_details`
+
+Returns information about the created RDS database resource.
+
+```ruby
+{
+  redis_url: "Full URL for the Redis cluster",
+}
+```
+
 #### `s3_bucket_details`
 
 - **Required Config:** `s3_bucket_name`
@@ -249,13 +318,14 @@ Checks the list of registries to see if `docker_registry_url` has already been a
 
 #### `default_service_domain_name`
 
-- **Required Config:** `convox_app_name`, `default_service`
+- **Required Config:** `convox_app_name`
+- **Optional Config:** `default_service`
 
-Parses the rack router ELB name and region, and returns the default `convox.site` domain for your default service. (You can visit this URL in the browser to access your app.)
+Finds the default `*.convox.cloud` URL for the web service. (You can visit this URL in the browser to access your app.)
 
-Example: `myapp-web.rackname-route-abcdfe123456-123456789.us-west-2.convox.site`
+Example: `web.docspring.dc6bae48c2e36366.convox.cloud`
 
-Set a default service in your config prompts (e.g. `web`):
+You can override the default service name in your config (e.g. `web`):
 
 ```ruby
 @prompts = [

data/examples/full_installation.rb

@@ -20,18 +20,15 @@ include ConvoxInstaller
 MINIMAL_HEALTH_CHECK_PATH = '/health/site'
 COMPLETE_HEALTH_CHECK_PATH = '/health'
 
-S3_BUCKET_CORS_POLICY = <<~JSON
-  {
-    "CORSRules": [
-      {
-        "AllowedOrigins": ["*"],
-        "AllowedHeaders": ["Authorization", "cache-control", "x-requested-with"],
-        "AllowedMethods": ["PUT", "POST", "GET"],
-        "MaxAgeSeconds": 3000
-      }
-    ]
+S3_BUCKET_CORS_RULE = <<-TERRAFORM
+  cors_rule {
+    allowed_headers = ["Authorization", "cache-control", "x-requested-with"]
+    allowed_methods = ["PUT", "POST", "GET"]
+    allowed_origins = ["*"]
+    expose_headers  = []
+    max_age_seconds = 3000
   }
-JSON
+TERRAFORM
 
 @prompts = ConvoxInstaller::Config::DEFAULT_PROMPTS + [
   {
@@ -81,8 +78,18 @@ JSON
     value: -> { "app-uploads-#{SecureRandom.hex(4)}" }
   },
   {
-    key: :s3_bucket_cors_policy,
-    value: S3_BUCKET_CORS_POLICY,
+    key: :s3_bucket_cors_rule,
+    value: S3_BUCKET_CORS_RULE,
+    hidden: true
+  },
+  {
+    key: :database_username,
+    value: 'example_app',
+    hidden: true
+  },
+  {
+    key: :database_password,
+    value: -> { SecureRandom.hex(16) },
     hidden: true
   }
 ]
@@ -94,7 +101,7 @@ backup_convox_host_and_rack
 install_convox
 
 validate_convox_auth_and_write_host!
-validate_convox_rack!
+validate_convox_rack_api!
 
 create_convox_app!
 set_default_app_for_directory!
@@ -129,7 +136,7 @@ env_command_params = env.map { |k, v| "#{k}=\"#{v}\"" }.join(' ')
 run_convox_command! "env set #{env_command_params}"
 
 puts '=> Initial deploy...'
-puts '-----> Documentation: https://docs.convox.com/deployment/builds'
+puts '-----> Documentation: https://docs.convox.com/deployment/deploying-changes/'
 run_convox_command! 'deploy --wait'
 
 puts '=> Setting up the database...'

data/lib/convox/client.rb

@@ -4,12 +4,20 @@ require 'logger'
 require 'json'
 require 'fileutils'
 require 'rubygems'
+require 'os'
+require 'erb'
 
 module Convox
   class Client
-    CONVOX_DIR = File.expand_path('~/.convox').freeze
-    AUTH_FILE = File.join(CONVOX_DIR, 'auth')
-    HOST_FILE = File.join(CONVOX_DIR, 'host')
+    CONVOX_CONFIG_DIR = if OS.mac?
+                          # Convox v3 moved this to ~/Library/Preferences/convox/ on Mac
+                          File.expand_path('~/Library/Preferences/convox').freeze
+                        else
+                          File.expand_path('~/.convox').freeze
+                        end
+
+    AUTH_FILE = File.join(CONVOX_CONFIG_DIR, 'auth')
+    CURRENT_FILE = File.join(CONVOX_CONFIG_DIR, 'current')
 
     attr_accessor :logger, :config
 
@@ -57,17 +65,21 @@ module Convox
       @config = options[:config] || {}
     end
 
+    # Convox v3 creates a folder for each rack for the Terraform config
+    def rack_dir
+      stack_name = config.fetch(:stack_name)
+      File.join(CONVOX_CONFIG_DIR, 'racks', stack_name)
+    end
+
     def backup_convox_host_and_rack
-      FileUtils.mkdir_p CONVOX_DIR
+      FileUtils.mkdir_p CONVOX_CONFIG_DIR
 
-      %w[host rack].each do |f|
-        path = File.join(CONVOX_DIR, f)
-        next unless File.exist?(path)
+      path = File.join(CONVOX_CONFIG_DIR, 'current')
+      return unless File.exist?(path)
 
-        bak_file = "#{path}.bak"
-        logger.info "Moving existing #{path} to #{bak_file}..."
-        FileUtils.mv(path, bak_file)
-      end
+      bak_file = "#{path}.bak"
+      logger.info "Moving existing #{path} to #{bak_file}..."
+      FileUtils.mv(path, bak_file)
     end
 
     def install_convox
@@ -76,8 +88,12 @@ module Convox
       stack_name = config.fetch(:stack_name)
 
       if rack_already_installed?
-        logger.info "There is already a Convox stack named #{stack_name} " \
-                    "in the #{region} AWS region. Using this rack. "
+        logger.info "There is already a Convox rack named #{stack_name}. Using this rack."
+        logger.debug 'If you need to start over, you can run: ' \
+                     "convox rack uninstall #{stack_name}    " \
+                     '(Make sure you export AWS_ACCESS_KEY_ID and ' \
+                     "AWS_SECRET_ACCESS_KEY first.)\n" \
+                     "If this fails, you can try deleting the rack directory: rm -rf #{rack_dir}"
         return true
       end
 
@@ -97,11 +113,19 @@ module Convox
         'AWS_SECRET_ACCESS_KEY' => config.fetch(:aws_secret_access_key)
       }
       command = %(rack install aws \
---name "#{config.fetch(:stack_name)}" \
-"InstanceType=#{config.fetch(:instance_type)}" \
-"BuildInstance=")
+"#{config.fetch(:stack_name)}" \
+"node_type=#{config.fetch(:instance_type)}" \
+"region=#{config.fetch(:aws_region)}")
+      # us-east constantly has problems with the us-east-1c AZ:
+      # "Cannot create cluster 'ds-enterprise-cx3' because us-east-1c, the targeted
+      # availability zone, does not currently have sufficient capacity to support the cluster.
+      # Retry and choose from these availability zones:
+      # us-east-1a, us-east-1b, us-east-1d, us-east-1e, us-east-1f
+      if config.fetch(:aws_region) == 'us-east-1'
+        command += ' "availability_zones=us-east-1a,us-east-1b,us-east-1d,us-east-1e,us-east-1f"'
+      end
 
-      run_convox_command!(command, env)
+      run_convox_command!(command, env, rack_arg: false)
     end
 
     def rack_already_installed?
@@ -109,66 +133,51 @@ module Convox
 
       return unless File.exist?(AUTH_FILE)
 
-      region = config.fetch(:aws_region)
+      # region = config.fetch(:aws_region)
       stack_name = config.fetch(:stack_name)
+      return true if File.exist?(rack_dir)
 
-      auth.each do |host, _password|
-        if host.match?(/^#{stack_name}-\d+\.#{region}\.elb\.amazonaws\.com$/)
-          return true
-        end
+      auth.each do |rack_name, _password|
+        return true if rack_name == stack_name
       end
       false
     end
 
-    def validate_convox_auth_and_write_host!
+    # Auth for a detached rack is not saved in the auth file anymore.
+    # It can be found in the terraform state:
+    # ~/Library/Preferences/convox/racks/ds-enterprise-cx3/terraform.tfstate
+    # Under outputs/api/value. The API URL contains the convox username and API token as basic auth.
+    def validate_convox_rack_and_write_current!
       require_config(%i[aws_region stack_name])
 
-      unless File.exist?(AUTH_FILE)
-        raise "Could not find auth file at #{AUTH_FILE}!"
+      unless rack_already_installed?
+        raise "Could not find rack terraform directory at: #{rack_dir}"
       end
 
-      region = config.fetch(:aws_region)
-      stack = config.fetch(:stack_name)
-
-      match_count = 0
-      matching_host = nil
-      auth.each do |host, _password|
-        if host.match?(/^#{stack}-\d+\.#{region}\.elb\.amazonaws\.com$/)
-          matching_host = host
-          match_count += 1
-        end
-      end
-
-      if match_count == 1
-        write_host(matching_host)
-        return matching_host
-      end
-
-      error_message = if match_count > 1
-                        'Found multiple matching hosts for '
-                      else
-                        'Could not find matching authentication for '
-                      end
-      error_message += "region: #{region}, stack: #{stack}"
-      raise error_message
+      # Tells the Convox CLI to use our terraform stack
+      stack_name = config.fetch(:stack_name)
+      write_current(stack_name)
+      stack_name
     end
 
-    def write_host(host)
-      logger.debug "Setting convox host to #{host} (in #{HOST_FILE})..."
-      File.open(HOST_FILE, 'w') { |f| f.puts host }
+    def write_current(rack_name)
+      logger.debug "Setting convox rack to #{rack_name} (in #{CURRENT_FILE})..."
+      current_hash = { name: rack_name, type: 'terraform' }
+      File.open(CURRENT_FILE, 'w') { |f| f.puts current_hash.to_json }
     end
 
-    def validate_convox_rack!
+    def validate_convox_rack_api!
       require_config(%i[
                        aws_region
                        stack_name
                        instance_type
                      ])
       logger.debug 'Validating that convox rack has the correct attributes...'
+      # Convox 3 racks no longer return info about region or type. (These are blank strings.)
       {
         provider: 'aws',
-        region: config.fetch(:aws_region),
-        type: config.fetch(:instance_type),
+        # region: config.fetch(:aws_region),
+        # type: config.fetch(:instance_type),
         name: config.fetch(:stack_name)
       }.each do |k, v|
         convox_value = convox_rack_data[k.to_s]
@@ -184,8 +193,22 @@ module Convox
     def convox_rack_data
       @convox_rack_data ||= begin
         logger.debug 'Fetching convox rack attributes...'
-        convox_output = `convox api get /system`
-        raise 'convox command failed!' unless $CHILD_STATUS.success?
+        command = "convox api get /system --rack #{config.fetch(:stack_name)}"
+        logger.debug "+ #{command}"
+        # It can take a while for the API to be ready.
+        start_time = Time.now
+        convox_output = nil
+        loop do
+          convox_output = `#{command}`
+          break if $CHILD_STATUS.success?
+
+          if Time.now - start_time > 360
+            raise 'Could not connect to Convox rack API!'
+          end
+
+          logger.debug 'Waiting for Convox rack API to be ready... (can take a few minutes)'
+          sleep 5
+        end
 
         JSON.parse(convox_output)
       end
@@ -199,9 +222,10 @@ module Convox
 
       logger.info "Creating app: #{app_name}..."
       logger.info '=> Documentation: ' \
-                  'https://docs.convox.com/deployment/creating-an-application'
+                  'https://docs.convox.com/reference/cli/apps/'
 
-      run_convox_command! "apps create #{app_name} --wait"
+      # NOTE: --wait flags were removed in Convox 3. It now waits by default.
+      run_convox_command! "apps create #{app_name}"
 
       retries = 0
       loop do
@@ -232,7 +256,7 @@ module Convox
       app_name = config.fetch(:convox_app_name)
 
       logger.debug "Looking for existing #{app_name} app..."
-      convox_output = `convox api get /apps`
+      convox_output = `convox api get /apps --rack #{config.fetch(:stack_name)}`
       raise 'convox command failed!' unless $CHILD_STATUS.success?
 
       apps = JSON.parse(convox_output)
@@ -247,128 +271,117 @@ module Convox
     end
 
     # Create the s3 bucket, and also apply a CORS configuration
-    def create_s3_bucket!
+    # Convox v3 update - They removed support for S3 resources, so we have to do
+    # in terraform now (which is actually pretty nice!)
+    def add_s3_bucket
       require_config(%i[s3_bucket_name])
-      bucket_name = config.fetch(:s3_bucket_name)
-      if s3_bucket_exists?
-        logger.info "#{bucket_name} S3 bucket already exists!"
-      else
-        logger.info "Creating S3 bucket resource (#{bucket_name})..."
-        run_convox_command! 'rack resources create s3 ' \
-                            "--name \"#{bucket_name}\" " \
-                            '--wait'
-
-        retries = 0
-        loop do
-          break if s3_bucket_exists?
 
-          if retries > 10
-            raise "Something went wrong while creating the #{bucket_name} S3 bucket! " \
-                  '(Please wait a few moments and then restart the installation script.)'
-          end
-          logger.debug 'Waiting for S3 bucket to be ready...'
-          sleep 3
-          retries += 1
-        end
-
-        logger.debug '=> S3 bucket created!'
+      unless config.key? :s3_bucket_cors_rule
+        logger.debug 'No CORS rule provided in config: s3_bucket_cors_rule   (optional)'
+        return
       end
 
-      set_s3_bucket_cors_policy
+      write_terraform_template('s3_bucket')
     end
 
-    def s3_bucket_exists?
-      require_config(%i[s3_bucket_name])
-      bucket_name = config.fetch(:s3_bucket_name)
-      logger.debug "Looking up S3 bucket resource: #{bucket_name}"
-      `convox api get /resources/#{bucket_name} 2>/dev/null`
-      $CHILD_STATUS.success?
+    def add_rds_database
+      require_config(%i[database_username database_password])
+      write_terraform_template('rds')
     end
 
-    def s3_bucket_details
-      require_config(%i[s3_bucket_name])
-      @s3_bucket_details ||= begin
-        bucket_name = config.fetch(:s3_bucket_name)
-        logger.debug "Fetching S3 bucket resource details for #{bucket_name}..."
-
-        response = `convox api get /resources/#{bucket_name}`
-        raise 'convox command failed!' unless $CHILD_STATUS.success?
-
-        bucket_data = JSON.parse(response)
-        s3_url = bucket_data['url']
-        matches = s3_url.match(
-          %r{^s3://(?<access_key_id>[^:]*):(?<secret_access_key>[^@]*)@(?<bucket_name>.*)$}
-        )
-
-        match_keys = %i[access_key_id secret_access_key bucket_name]
-        unless matches && match_keys.all? { |k| matches[k].present? }
-          raise "#{s3_url} is an invalid S3 URL!"
-        end
+    def add_elasticache_cluster
+      write_terraform_template('elasticache')
+    end
 
-        {
-          access_key_id: matches[:access_key_id],
-          secret_access_key: matches[:secret_access_key],
-          name: matches[:bucket_name]
-        }
+    def write_terraform_template(name)
+      template_path = File.join(__dir__, "../../terraform/#{name}.tf.erb")
+      unless File.exist?(template_path)
+        raise "Could not find terraform template at: #{template_path}"
       end
+
+      template = ERB.new(File.read(template_path))
+      template_output = template.result(binding)
+
+      tf_file_path = File.join(rack_dir, "#{name}.tf")
+      logger.debug "Writing terraform config to #{tf_file_path}..."
+      File.open(tf_file_path, 'w') { |f| f.puts template_output }
     end
 
-    def set_s3_bucket_cors_policy
-      require_config(%i[aws_access_key_id aws_secret_access_key])
-      access_key_id = config.fetch(:aws_access_key_id)
-      secret_access_key = config.fetch(:aws_secret_access_key)
+    def apply_terraform_update!
+      logger.info 'Applying terraform update...'
+      command = if ENV['DEBUG_TERRAFORM']
+                  'terraform plan'
+                else
+                  'terraform apply -auto-approve'
+                end
+      logger.debug "+ #{command}"
 
-      unless config.key? :s3_bucket_cors_policy
-        logger.debug 'No CORS policy provided in config: s3_bucket_cors_policy'
-        return
+      env = {
+        'AWS_ACCESS_KEY_ID' => config.fetch(:aws_access_key_id),
+        'AWS_SECRET_ACCESS_KEY' => config.fetch(:aws_secret_access_key)
+      }
+      Dir.chdir(rack_dir) do
+        system env, command
+        raise 'terraform command failed!' unless $CHILD_STATUS.success?
       end
-      cors_policy_string = config.fetch(:s3_bucket_cors_policy)
-
-      bucket_name = s3_bucket_details[:name]
-
-      logger.debug "Looking up existing CORS policy for #{bucket_name}"
-      existing_cors_policy_string =
-        `AWS_ACCESS_KEY_ID=#{access_key_id} \
-        AWS_SECRET_ACCESS_KEY=#{secret_access_key} \
-        aws s3api get-bucket-cors --bucket #{bucket_name} 2>/dev/null`
-      if $CHILD_STATUS.success? && existing_cors_policy_string.present?
-        # Sort all the nested arrays so that the equality operator works
-        existing_cors_policy = JSON.parse(existing_cors_policy_string)
-        cors_policy_json = JSON.parse(cors_policy_string)
-        [existing_cors_policy, cors_policy_json].each do |policy_json|
-          next unless policy_json.is_a?(Hash) && policy_json['CORSRules']
-
-          policy_json['CORSRules'].each do |rule|
-            rule['AllowedHeaders']&.sort!
-            rule['AllowedMethods']&.sort!
-            rule['AllowedOrigins']&.sort!
-          end
-        end
+    end
 
-        if existing_cors_policy == cors_policy_json
-          logger.debug "=> CORS policy is already up to date for #{bucket_name}."
-          return
-        end
+    def terraform_state
+      tf_state_file = File.join(rack_dir, 'terraform.tfstate')
+      JSON.parse(File.read(tf_state_file))
+    end
+
+    def terraform_resource(resource_type, resource_name)
+      resource = terraform_state['resources'].find do |resource|
+        resource['type'] == resource_type && resource['name'] == resource_name
       end
+      return resource if resource
 
-      begin
-        logger.info "Setting CORS policy for #{bucket_name}..."
+      raise "Could not find #{resource_type} resource named #{resource_name} in terraform state!"
+    end
 
-        File.open('cors-policy.json', 'w') { |f| f.puts cors_policy_string }
+    def s3_bucket_details
+      require_config(%i[s3_bucket_name])
 
-        `AWS_ACCESS_KEY_ID=#{access_key_id} \
-        AWS_SECRET_ACCESS_KEY=#{secret_access_key} \
-          aws s3api put-bucket-cors \
-          --bucket #{bucket_name} \
-          --cors-configuration "file://cors-policy.json"`
-        unless $CHILD_STATUS.success?
-          raise 'Something went wrong while setting the S3 bucket CORS policy!'
-        end
+      s3_bucket = terraform_resource('aws_s3_bucket', 'docs_s3_bucket')
+      bucket_attributes = s3_bucket['instances'][0]['attributes']
+      access_key = terraform_resource('aws_iam_access_key', 'docspring_user_access_key')
+      key_attributes = access_key['instances'][0]['attributes']
 
-        logger.info "=> Successfully set CORS policy for #{bucket_name}."
-      ensure
-        FileUtils.rm_f 'cors-policy.json'
-      end
+      {
+        access_key_id: key_attributes['id'],
+        secret_access_key: key_attributes['secret'],
+        name: bucket_attributes['bucket']
+      }
+    end
+
+    def rds_details
+      require_config(%i[database_username database_password])
+
+      database = terraform_resource('aws_db_instance', 'rds_database')
+      database_attributes = database['instances'][0]['attributes']
+
+      username = database_attributes['username']
+      password = database_attributes['password']
+      endpoint = database_attributes['endpoint']
+      postgres_url = "postgres://#{username}:#{password}@#{endpoint}/app"
+      {
+        postgres_url: postgres_url
+      }
+    end
+
+    def elasticache_details
+      require_config(%i[s3_bucket_name])
+
+      # Just ensure that the bucket exists in the state
+      cluster = terraform_resource('aws_elasticache_cluster', 'elasticache_cluster')
+      cluster_attributes = cluster['instances'][0]['attributes']
+      cache_node = cluster_attributes['cache_nodes'][0]
+      redis_url = "redis://#{cache_node['address']}:#{cache_node['port']}/0"
+
+      {
+        redis_url: redis_url
+      }
     end
 
     def add_docker_registry!
@@ -377,7 +390,7 @@ module Convox
       registry_url = config.fetch(:docker_registry_url)
 
       logger.debug 'Looking up existing Docker registries...'
-      registries_response = `convox api get /registries`
+      registries_response = `convox api get /registries --rack #{config.fetch(:stack_name)}`
       unless $CHILD_STATUS.success?
         raise 'Something went wrong while fetching the list of registries!'
       end
@@ -391,36 +404,37 @@ module Convox
 
       logger.info "Adding Docker Registry: #{registry_url}..."
       logger.info '=> Documentation: ' \
-                  'https://docs.convox.com/deployment/private-registries'
+                  'https://docs.convox.com/configuration/private-registries/'
 
       `convox registries add "#{registry_url}" \
         "#{config.fetch(:docker_registry_username)}" \
-        "#{config.fetch(:docker_registry_password)}"`
+        "#{config.fetch(:docker_registry_password)}" \
+        --rack #{config.fetch(:stack_name)}`
       return if $CHILD_STATUS.success?
 
       raise "Something went wrong while adding the #{registry_url} registry!"
     end
 
     def default_service_domain_name
-      require_config(%i[convox_app_name default_service])
-
-      @default_service_domain_name ||= begin
-        convox_domain = convox_rack_data['domain']
-        elb_name_and_region = convox_domain[/([^.]*\.[^.]*)\..*/, 1]
-        unless elb_name_and_region.present?
-          raise 'Something went wrong while parsing the ELB name and region! ' \
-                "(#{elb_name_and_region})"
-        end
-        app = config.fetch(:convox_app_name)
-        service = config.fetch(:default_service)
+      require_config(%i[convox_app_name])
 
-        # Need to return downcase host so that `config.hosts` works with Rails applications
-        "#{app}-#{service}.#{elb_name_and_region}.convox.site".downcase
-      end
+      app_name = config.fetch(:convox_app_name)
+      default_service = config[:default_service] || 'web'
+
+      convox_api_url = terraform_state['outputs']['api']['value']
+      convox_router_host = convox_api_url.split('@').last.sub(/^api\./, '')
+
+      [default_service, app_name, convox_router_host].join('.').downcase
     end
 
-    def run_convox_command!(cmd, env = {})
+    def run_convox_command!(cmd, env = {}, rack_arg: true)
+      # Always include the rack as an argument, to
+      # make sure that 'convox switch' doesn't affect any commands
       command = "convox #{cmd}"
+      if rack_arg
+        command = "#{command} --rack #{config.fetch(:stack_name)}"
+      end
+      logger.debug "+ #{command}"
       system env, command
       raise "Error running: #{command}" unless $CHILD_STATUS.success?
     end

data/lib/convox_installer/config.rb

@@ -38,6 +38,12 @@ module ConvoxInstaller
       {
         key: :aws_secret_access_key,
         title: 'AWS Secret Access Key'
+      },
+      # Short random ID used to ensure that resources are always unique
+      {
+        key: :random_id,
+        value: -> { SecureRandom.hex(4) },
+        hidden: true
       }
     ].freeze
 
@@ -76,6 +82,8 @@ module ConvoxInstaller
 
         highline.say 'Please double check all of these configuration details.'
 
+        break if ENV['AUTOSTART_CONVOX_INSTALLATION']
+
         agree = highline.agree(
           'Would you like to start the Convox installation?' \
           " (press 'n' to correct any settings)"

data/lib/convox_installer/requirements.rb

@@ -34,6 +34,14 @@ module ConvoxInstaller
         }
       end
 
+      unless command_present? 'terraform'
+        @missing_packages << {
+          name: 'terraform',
+          brew: 'terraform',
+          docs: 'https://learn.hashicorp.com/tutorials/terraform/install-cli'
+        }
+      end
+
       if @missing_packages.any?
         logger.error 'This script requires the convox and AWS CLI tools.'
         if OS.mac?
@@ -49,12 +57,15 @@ module ConvoxInstaller
       end
 
       client = Convox::Client.new
-      return if client.convox_2_cli?
+      if client.convox_3_cli?
+        logger.debug "=> Convox CLI is version 3.x.x (#{client.cli_version_string})"
+        return
+      end
 
-      logger.error 'This script requires Convox CLI version 2.x.x. ' \
+      logger.error 'This script requires Convox CLI version 3.x.x. ' \
                    "Your Convox CLI version is: #{client.cli_version_string}"
-      logger.error 'Please follow the instructions here to downgrade your ' \
-                   'Convox CLI version: https://docsv2.convox.com/introduction/installation'
+      logger.error "Please run 'brew update convox' or follow the instructions " \
+                   'at https://docs.convox.com/getting-started/introduction'
       quit!
     end
 

data/lib/convox_installer/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ConvoxInstaller
-  VERSION = '2.0.0'
+  VERSION = '3.0.0'
 end

data/lib/convox_installer.rb

@@ -32,18 +32,24 @@ module ConvoxInstaller
   %w[
     backup_convox_host_and_rack
     install_convox
-    validate_convox_auth_and_write_host!
-    validate_convox_rack!
+    validate_convox_rack_and_write_current!
+    validate_convox_rack_api!
     convox_rack_data
     create_convox_app!
     set_default_app_for_directory!
-    create_s3_bucket!
-    set_s3_bucket_cors_policy
+    add_s3_bucket
+    add_rds_database
+    add_elasticache_cluster
+    apply_terraform_update!
+    terraform_state
     s3_bucket_details
+    elasticache_details
+    rds_details
     add_docker_registry!
     default_service_domain_name
     run_convox_command!
     logger
+    rack_already_installed?
   ].each do |method|
     define_method(method) do |*args|
       client.send(method, *args)

data/spec/lib/convox/client_spec.rb

@@ -138,7 +138,7 @@ RSpec.describe Convox::Client do
           stack_name: 'convox-test'
         }
       )
-      expect(client).to receive(:write_host).with(
+      expect(client).to receive(:write_current).with(
         'convox-test-697645520.us-west-2.elb.amazonaws.com'
       )
       expect(client.validate_convox_auth_and_write_host!).to(

data/spec/lib/convox_installer/requirements_spec.rb

@@ -3,7 +3,7 @@
 require 'convox_installer'
 
 RSpec.describe ConvoxInstaller::Requirements do
-  let(:convox_cli_version) { '20210208170413' }
+  let(:convox_cli_version) { '3.3.4' }
 
   before do
     allow_any_instance_of(
@@ -45,8 +45,8 @@ RSpec.describe ConvoxInstaller::Requirements do
       end
     end
 
-    context 'with Convox CLI version 3.3.4' do
-      let(:convox_cli_version) { '3.3.4' }
+    context 'with Convox CLI version 20210208170413' do
+      let(:convox_cli_version) { '20210208170413' }
 
       it 'shows the correct error message and quit' do
         req = described_class.new
@@ -55,11 +55,10 @@ RSpec.describe ConvoxInstaller::Requirements do
         expect(req).to receive(:quit!)
 
         expect(req.logger).to receive(:error).with(
-          'This script requires Convox CLI version 2.x.x. Your Convox CLI version is: 3.3.4'
+          'This script requires Convox CLI version 3.x.x. Your Convox CLI version is: 20210208170413'
         )
         expect(req.logger).to receive(:error).with(
-          'Please follow the instructions here to downgrade your ' \
-          'Convox CLI version: https://docsv2.convox.com/introduction/installation'
+          "Please run 'brew update convox' or follow the instructions at https://docs.convox.com/getting-started/introduction"
         )
 
         req.ensure_requirements!

data/terraform/elasticache.tf.erb

@@ -0,0 +1,46 @@
+resource "aws_elasticache_cluster" "elasticache_cluster" {
+  cluster_id           = "<%= config.fetch(:stack_name) %>-elasticache-<%= config.fetch(:random_id) %>"
+  engine               = "<%= config[:elasticache_engine] || 'redis' %>"
+  engine_version       = "<%= config[:elasticache_engine_version] || '6.x' %>"
+  node_type            = "<%= config[:elasticache_node_type] || 'cache.t3.medium' %>"
+  num_cache_nodes      = <%= config[:elasticache_num_cache_nodes] || 1 %>
+  port                 = <%= config[:elasticache_port] || 6379 %>
+  
+  subnet_group_name = aws_elasticache_subnet_group.elasticache_subnet_group.name
+  security_group_ids = [aws_security_group.elasticache_security_group.id]
+
+  # Workaround for weird engine_version issue where 6.x works for creation, and fails for update
+  # See: https://github.com/hashicorp/terraform-provider-aws/issues/15625#issuecomment-727759811
+  # Fixed in version 3.38.0 of the Terraform AWS provider.
+  lifecycle {
+    ignore_changes = [engine_version]
+  }
+}
+
+resource "aws_elasticache_subnet_group" "elasticache_subnet_group" {
+  name       = "<%= config.fetch(:stack_name) %>-elasticache-cluster-subnetgroup-<%= config.fetch(:random_id) %>"
+  subnet_ids = module.system.cluster.subnets
+}
+
+resource "aws_security_group" "elasticache_security_group" {
+  name = "<%= config.fetch(:stack_name) %>-elasticache-securitygroup-<%= config.fetch(:random_id) %>"
+
+  description = "Elasticache Security Group (Managed by Terraform)"
+  vpc_id      = module.system.cluster.vpc
+
+  # Only Redis in
+  ingress {
+    from_port   = 6379
+    to_port     = 6379
+    protocol    = "tcp"
+    cidr_blocks = ["10.1.0.0/16"]
+  }
+
+  # Allow all outbound traffic
+  egress {
+    from_port   = 0
+    to_port     = 0
+    protocol    = "-1"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+}

data/terraform/rds.tf.erb

@@ -0,0 +1,45 @@
+resource "aws_db_instance" "rds_database" {
+  allocated_storage    = <%= config[:database_allocated_storage] || 30 %>
+  engine               = "<%= config[:database_engine] || 'postgres' %>"
+  engine_version       = "<%= config[:database_engine_version] || '14.2' %>"
+  instance_class       = "<%= config[:database_instance_class] || 'db.t3.medium' %>"
+  name                 = "<%= config.fetch(:stack_name).gsub('-', '_') %>_database"
+  identifier           = "<%= config.fetch(:stack_name) %>-rds-<%= config.fetch(:random_id) %>"
+  multi_az             = <%= config[:database_multi_az] || true %>
+  username             = "<%= config.fetch(:database_username) %>"
+  password             = "<%= config.fetch(:database_password) %>"
+
+  final_snapshot_identifier = "<%= config.fetch(:stack_name) %>-rds-<%= config.fetch(:random_id) %>-final-snapshot"
+  skip_final_snapshot = false
+
+  db_subnet_group_name = aws_db_subnet_group.rds_subnet_group.name
+  vpc_security_group_ids = [aws_security_group.rds_security_group.id]
+}
+
+resource "aws_db_subnet_group" "rds_subnet_group" {
+  name       = "<%= config.fetch(:stack_name) %>-rds-subnetgroup-<%= config.fetch(:random_id) %>"
+  subnet_ids = module.system.cluster.subnets
+}
+
+resource "aws_security_group" "rds_security_group" {
+  name = "<%= config.fetch(:stack_name) %>-rds-database-securitygroup-<%= config.fetch(:random_id) %>"
+
+  description = "RDS Security Group (Managed by Terraform)"
+  vpc_id      = module.system.cluster.vpc
+
+  # Only Postgres in
+  ingress {
+    from_port   = 5432
+    to_port     = 5432
+    protocol    = "tcp"
+    cidr_blocks = ["10.1.0.0/16"]
+  }
+
+  # Allow all outbound traffic
+  egress {
+    from_port   = 0
+    to_port     = 0
+    protocol    = "-1"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+}

data/terraform/s3_bucket.tf.erb

@@ -0,0 +1,73 @@
+# AWS provider version is 3.33.0
+# https://registry.terraform.io/providers/hashicorp/aws/3.33.0
+
+provider "aws" {
+  region = "<%= config[:aws_region] %>"
+}
+
+resource "aws_kms_key" "docs_kms_key" {
+  description             = "This key is used to encrypt objects in the DocSpring S3 bucket"
+  deletion_window_in_days = 14
+}
+
+# Later versions of aws provider (e.g. 4.8.0) use separate resources for 
+# aws_s3_bucket_acl and aws_s3_bucket_cors_configuration.
+# This will need to be updated in the future.
+resource "aws_s3_bucket" "docs_s3_bucket" {
+  bucket = "<%= config.fetch(:stack_name) %>-<%= config.fetch(:s3_bucket_name) %>"
+  acl   = "private"
+
+  server_side_encryption_configuration {
+    rule {
+      apply_server_side_encryption_by_default {
+        kms_master_key_id = aws_kms_key.docs_kms_key.arn
+        sse_algorithm     = "aws:kms"
+      }
+    }
+  }
+
+  <%= config[:s3_bucket_cors_rule] %>
+}
+
+resource "aws_iam_user" "docspring_s3_user" {
+  name = "<%= config.fetch(:stack_name) %>-<%= config.fetch(:s3_bucket_name) %>"
+}
+
+resource "aws_iam_access_key" "docspring_user_access_key" {
+  user = aws_iam_user.docspring_s3_user.name
+}
+
+resource "aws_iam_user_policy" "docspring_user_s3_policy" {
+  name = "docspring_user_s3_policy"
+  user = aws_iam_user.docspring_s3_user.name
+
+  policy = jsonencode({
+    "Version": "2012-10-17",
+    "Statement": [
+      {
+        "Effect": "Allow",
+        "Action": [
+            "s3:PutObject",
+            "s3:PutObjectAcl",
+            "s3:GetObject",
+            "s3:GetObjectAcl",
+            "s3:DeleteObject"
+        ],
+        "Resource": [
+          "arn:aws:s3:::<%= config.fetch(:stack_name) %>-<%= config.fetch(:s3_bucket_name) %>/*",
+        ]
+      },
+      {
+        "Effect": "Allow",
+        "Action": [
+          "kms:Encrypt",
+          "kms:Decrypt",
+          "kms:ReEncrypt*",
+          "kms:GenerateDataKey*",
+          "kms:DescribeKey"
+        ],
+        "Resource": "*"
+      }
+    ]
+  })
+}

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: convox_installer
 version: !ruby/object:Gem::Version
-  version: 2.0.0
+  version: 3.0.0
 platform: ruby
 authors:
 - Form Applications Inc.
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-04-02 00:00:00.000000000 Z
+date: 2022-04-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -109,6 +109,9 @@ files:
 - spec/lib/convox_installer/config_spec.rb
 - spec/lib/convox_installer/requirements_spec.rb
 - spec/spec_helper.rb
+- terraform/elasticache.tf.erb
+- terraform/rds.tf.erb
+- terraform/s3_bucket.tf.erb
 homepage: https://github.com/FormAPI/convox_installer
 licenses:
 - MIT