jets 3.1.0 → 5.0.0

data/lib/jets/commands/credentials/USAGE

@@ -0,0 +1,103 @@
+## Storing Encrypted Credentials in Source Control
+
+The Jets `credentials` commands provide access to encrypted credentials,
+so you can safely store access tokens, database passwords, and the like
+safely inside the app without relying on a mess of ENVs.
+
+This also allows for atomic deploys: no need to coordinate key changes
+to get everything working as the keys are shipped with the code.
+
+## Setup
+
+Applications after Jets 5 automatically have a basic credentials file generated
+that just contains the secret_key_base used by MessageVerifiers/MessageEncryptors, like the ones
+signing and encrypting cookies.
+
+For applications created prior to Jets 5, we'll automatically generate a new
+credentials file in `config/credentials.yml.enc` the first time you run `bin/jets credentials:edit`.
+If you didn't have a master key saved in `config/master.key`, that'll be created too.
+
+Don't lose this master key! Put it in a password manager your team can access.
+Should you lose it no one, including you, will be able to access any encrypted
+credentials.
+
+Don't commit the key! Add `config/master.key` to your source control's
+ignore file. If you use Git, Jets handles this for you.
+
+Jets also looks for the master key in `ENV["JETS_MASTER_KEY"]`, if that's easier to manage.
+
+You could prepend that to your server's start command like this:
+
+    JETS_MASTER_KEY="very-secret-and-secure" server.start
+
+## Set up Git to Diff Credentials
+
+Jets provides `bin/jets credentials:diff --enroll` to instruct Git to call
+`bin/jets credentials:diff` when `git diff` is run on a credentials file.
+
+Running the command enrolls the project such that all credentials files use the
+"jets_credentials" diff driver in .gitattributes.
+
+Additionally since Git requires the driver itself to be set up in a config file
+that isn't tracked Jets automatically ensures it's configured when running
+`credentials:edit`.
+
+Otherwise each co-worker would have to run enable manually, including on each new
+repo clone.
+
+To disenroll from this feature, run `bin/jets credentials:diff --disenroll`.
+
+## Editing Credentials
+
+This will open a temporary file in `$EDITOR` with the decrypted contents to edit
+the encrypted credentials.
+
+When the temporary file is next saved the contents are encrypted and written to
+`config/credentials.yml.enc` while the file itself is destroyed to prevent credentials
+from leaking.
+
+## Environment Specific Credentials
+
+The `credentials` command supports passing an `--environment` option to create an
+environment specific override. That override will take precedence over the
+global `config/credentials.yml.enc` file when running in that environment. So:
+
+    bin/jets credentials:edit --environment development
+
+will create `config/credentials/development.yml.enc` with the corresponding
+encryption key in `config/credentials/development.key` if the credentials file
+doesn't exist.
+
+The encryption key can also be put in `ENV["JETS_MASTER_KEY"]`, which takes
+precedence over the file encryption key.
+
+In addition to that, the default credentials lookup paths can be overridden through
+`config.credentials.content_path` and `config.credentials.key_path`.
+
+## Editor wait
+
+For editors that fork and exit immediately, it's important to pass a wait flag,
+otherwise the credentials will be saved immediately with no chance to edit.
+
+    EDITOR="code --wait" jets credentials:edit
+
+## Accessing in App
+
+Let's say you have credentials like so:
+
+    ❯ jets credentials:show
+    secret_key_base: somesecretvalue
+    foo: bar
+
+You can access them in the app like so:
+
+    ❯ jets console
+    > Jets.application.credentials.secret_key_base
+    => "somesecretvalue"
+    > Jets.application.credentials.foo
+    => "bar"
+    > Jets.application.credentials.does_not_exist
+    => nil
+
+Remember, when you deploy this to Lambda you should set the `JETS_MASTER_KEY`. You set it with [Env Files](https://rubyonjets.com/docs/env-files/). Otherwise the `Jets.application.credentials.xx` values will return `nil`.
+

data/lib/jets/commands/credentials/credentials_command/diffing.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+module Jets::Command::CredentialsCommand::Diffing # :nodoc:
+  GITATTRIBUTES_ENTRY = <<~END
+    config/credentials/*.yml.enc diff=jets_credentials
+    config/credentials.yml.enc diff=jets_credentials
+  END
+
+  def enroll_project_in_credentials_diffing
+    if enrolled_in_credentials_diffing?
+      say "Project is already enrolled in credentials file diffing."
+    else
+      gitattributes.write(GITATTRIBUTES_ENTRY, mode: "a")
+
+      say "Enrolled project in credentials file diffing!"
+      say "Jets ensures the jets_credentials diff driver is set when running `credentials:edit`. See `credentials:help` for more."
+    end
+  end
+
+  def disenroll_project_from_credentials_diffing
+    if enrolled_in_credentials_diffing?
+      gitattributes.write(gitattributes.read.gsub(GITATTRIBUTES_ENTRY, ""))
+      gitattributes.delete if gitattributes.empty?
+
+      say "Disenrolled project from credentials file diffing!"
+    else
+      say "Project is not enrolled in credentials file diffing."
+    end
+  end
+
+  def ensure_diffing_driver_is_configured
+    configure_diffing_driver if enrolled_in_credentials_diffing? && !diffing_driver_configured?
+  end
+
+  private
+    def enrolled_in_credentials_diffing?
+      gitattributes.file? && gitattributes.read.include?(GITATTRIBUTES_ENTRY)
+    end
+
+    def diffing_driver_configured?
+      system "git config --get diff.jets_credentials.textconv", out: File::NULL
+    end
+
+    def configure_diffing_driver
+      system "git config diff.jets_credentials.textconv 'bin/jets credentials:diff'"
+    end
+
+    def gitattributes
+      Jets.root.join(".gitattributes")
+    end
+end

data/lib/jets/commands/credentials/credentials_command.rb

@@ -0,0 +1,139 @@
+# frozen_string_literal: true
+
+require "pathname"
+require "shellwords"
+require "active_support"
+
+module Jets
+  module Command
+    class CredentialsCommand < Jets::Command::Base # :nodoc:
+      include Helpers::Editor
+      include EnvironmentArgument
+
+      require_relative "credentials_command/diffing"
+      include Diffing
+
+      self.environment_desc = "Uses credentials from config/credentials/:environment.yml.enc encrypted by config/credentials/:environment.key key"
+
+      no_commands do
+        def help
+          say "Usage:\n  #{self.class.banner}"
+          say ""
+          say self.class.desc
+        end
+      end
+
+      def edit
+        extract_environment_option_from_argument(default_environment: nil)
+        require_application!
+
+        ensure_editor_available(command: "bin/jets credentials:edit") || (return)
+
+        ensure_encryption_key_has_been_added if credentials.key.nil?
+        ensure_credentials_have_been_added
+        ensure_diffing_driver_is_configured
+
+        catch_editing_exceptions do
+          change_credentials_in_system_editor
+        end
+
+        say "File encrypted and saved."
+      rescue ActiveSupport::MessageEncryptor::InvalidMessage
+        say "Couldn't decrypt #{content_path}. Perhaps you passed the wrong key?"
+      end
+
+      def show
+        extract_environment_option_from_argument(default_environment: nil)
+        require_application!
+
+        say credentials.read.presence || missing_credentials_message
+      end
+
+      option :enroll, type: :boolean, default: false,
+        desc: "Enrolls project in credentials file diffing with `git diff`"
+
+      option :disenroll, type: :boolean, default: false,
+        desc: "Disenrolls project from credentials file diffing"
+
+      def diff(content_path = nil)
+        if @content_path = content_path
+          extract_environment_option_from_argument(default_environment: extract_environment_from_path(content_path))
+          require_application!
+
+          say credentials.read.presence || credentials.content_path.read
+        else
+          require_application!
+          disenroll_project_from_credentials_diffing if options[:disenroll]
+          enroll_project_in_credentials_diffing if options[:enroll]
+        end
+      rescue ActiveSupport::MessageEncryptor::InvalidMessage
+        say credentials.content_path.read
+      end
+
+      private
+        def credentials
+          Jets.application.encrypted(content_path, key_path: key_path)
+        end
+
+        def ensure_encryption_key_has_been_added
+          encryption_key_file_generator.add_key_file(key_path)
+          encryption_key_file_generator.ignore_key_file(key_path)
+        end
+
+        def ensure_credentials_have_been_added
+          if options[:environment]
+            encrypted_file_generator.add_encrypted_file_silently(content_path, key_path)
+          else
+            credentials_generator.add_credentials_file_silently
+          end
+        end
+
+        def change_credentials_in_system_editor
+          credentials.change do |tmp_path|
+            system(*Shellwords.split(ENV["EDITOR"]), tmp_path.to_s)
+          end
+        end
+
+        def missing_credentials_message
+          if credentials.key.nil?
+            "Missing '#{key_path}' to decrypt credentials. See `bin/jets credentials:help`"
+          else
+            "File '#{content_path}' does not exist. Use `bin/jets credentials:edit` to change that."
+          end
+        end
+
+        def content_path
+          @content_path ||= options[:environment] ? "config/credentials/#{options[:environment]}.yml.enc" : "config/credentials.yml.enc"
+        end
+
+        def key_path
+          options[:environment] ? "config/credentials/#{options[:environment]}.key" : "config/master.key"
+        end
+
+        def extract_environment_from_path(path)
+          available_environments.find { |env| path.include? env } if path.end_with?(".yml.enc")
+        end
+
+        def encryption_key_file_generator
+          require "rails/generators"
+          require "rails/generators/rails/encryption_key_file/encryption_key_file_generator"
+
+          Rails::Generators::EncryptionKeyFileGenerator.new
+        end
+
+        def encrypted_file_generator
+          require "rails/generators"
+          require "rails/generators/rails/encrypted_file/encrypted_file_generator"
+
+          Rails::Generators::EncryptedFileGenerator.new
+        end
+
+        def credentials_generator
+          require "rails/generators"
+          require "rails/generators/rails/credentials/credentials_generator"
+
+          Rails::Generators::CredentialsGenerator.new
+        end
+    end
+  end
+end

data/lib/jets/commands/db/system/change/change_command.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module Jets
+  module Command
+    module Db
+      module System
+        class ChangeCommand < Base # :nodoc:
+          class_option :to, desc: "The database system to switch to."
+
+          def initialize(positional_args, option_args, *)
+            @argv = positional_args + option_args
+            super
+          end
+
+          def perform
+            require "rails/generators"
+            require "rails/generators/rails/db/system/change/change_generator"
+            Rails::Generators::Db::System::ChangeGenerator.start(@argv)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/jets/commands/dbconsole/dbconsole_command.rb

@@ -0,0 +1,184 @@
+# frozen_string_literal: true
+
+require "active_support/core_ext/string/filters"
+require "active_support/deprecation"
+
+module Jets
+  class DBConsole
+    def self.start(*args)
+      new(*args).start
+    end
+
+    def initialize(options = {})
+      @options = options
+    end
+
+    def start
+      ENV["JETS_ENV"] ||= @options[:environment] || environment
+      config = db_config.configuration_hash
+
+      case db_config.adapter
+      when /^(jdbc)?mysql/
+        args = {
+          host: "--host",
+          port: "--port",
+          socket: "--socket",
+          username: "--user",
+          encoding: "--default-character-set",
+          sslca: "--ssl-ca",
+          sslcert: "--ssl-cert",
+          sslcapath: "--ssl-capath",
+          sslcipher: "--ssl-cipher",
+          sslkey: "--ssl-key"
+        }.filter_map { |opt, arg| "#{arg}=#{config[opt]}" if config[opt] }
+
+        if config[:password] && @options[:include_password]
+          args << "--password=#{config[:password]}"
+        elsif config[:password] && !config[:password].to_s.empty?
+          args << "-p"
+        end
+
+        args << db_config.database
+
+        find_cmd_and_exec(["mysql", "mysql5"], *args)
+
+      when /^postgres|^postgis/
+        ENV["PGUSER"]         = config[:username] if config[:username]
+        ENV["PGHOST"]         = config[:host] if config[:host]
+        ENV["PGPORT"]         = config[:port].to_s if config[:port]
+        ENV["PGPASSWORD"]     = config[:password].to_s if config[:password] && @options[:include_password]
+        ENV["PGSSLMODE"]      = config[:sslmode].to_s if config[:sslmode]
+        ENV["PGSSLCERT"]      = config[:sslcert].to_s if config[:sslcert]
+        ENV["PGSSLKEY"]       = config[:sslkey].to_s if config[:sslkey]
+        ENV["PGSSLROOTCERT"]  = config[:sslrootcert].to_s if config[:sslrootcert]
+        find_cmd_and_exec("psql", db_config.database)
+
+      when "sqlite3"
+        args = []
+
+        args << "-#{@options[:mode]}" if @options[:mode]
+        args << "-header" if @options[:header]
+        args << File.expand_path(db_config.database, Jets.respond_to?(:root) ? Jets.root : nil)
+
+        find_cmd_and_exec("sqlite3", *args)
+
+      when "oracle", "oracle_enhanced"
+        logon = ""
+
+        if config[:username]
+          logon = config[:username].dup
+          logon << "/#{config[:password]}" if config[:password] && @options[:include_password]
+          logon << "@#{db_config.database}" if db_config.database
+        end
+
+        find_cmd_and_exec("sqlplus", logon)
+
+      when "sqlserver"
+        args = []
+
+        args += ["-d", "#{db_config.database}"] if db_config.database
+        args += ["-U", "#{config[:username]}"] if config[:username]
+        args += ["-P", "#{config[:password]}"] if config[:password]
+
+        if config[:host]
+          host_arg = +"tcp:#{config[:host]}"
+          host_arg << ",#{config[:port]}" if config[:port]
+          args += ["-S", host_arg]
+        end
+
+        find_cmd_and_exec("sqlcmd", *args)
+
+      else
+        abort "Unknown command-line client for #{db_config.database}."
+      end
+    end
+
+    def db_config
+      return @db_config if defined?(@db_config)
+
+      # If the user provided a database, use that. Otherwise find
+      # the first config in the database.yml
+      if database
+        @db_config = configurations.configs_for(env_name: environment, name: database, include_hidden: true)
+      else
+        @db_config = configurations.find_db_config(environment)
+      end
+
+      unless @db_config
+        missing_db = database ? "'#{database}' database is not" : "No databases are"
+        raise ActiveRecord::AdapterNotSpecified,
+          "#{missing_db} configured for '#{environment}'. Available configuration: #{configurations.inspect}"
+      end
+
+      @db_config
+    end
+
+    def environment
+      Jets.respond_to?(:env) ? Jets.env : Jets::Command.environment
+    end
+
+    def database
+      @options[:database]
+    end
+
+    private
+      def configurations # :doc:
+        # require APP_PATH
+        ActiveRecord::Base.configurations = Jets.application.config.database_configuration
+        ActiveRecord::Base.configurations
+      end
+
+      def find_cmd_and_exec(commands, *args) # :doc:
+        commands = Array(commands)
+
+        dirs_on_path = ENV["PATH"].to_s.split(File::PATH_SEPARATOR)
+        unless (ext = RbConfig::CONFIG["EXEEXT"]).empty?
+          commands = commands.map { |cmd| "#{cmd}#{ext}" }
+        end
+
+        full_path_command = nil
+        found = commands.detect do |cmd|
+          dirs_on_path.detect do |path|
+            full_path_command = File.join(path, cmd)
+            begin
+              stat = File.stat(full_path_command)
+            rescue SystemCallError
+            else
+              stat.file? && stat.executable?
+            end
+          end
+        end
+
+        if found
+          exec full_path_command, *args
+        else
+          abort("Couldn't find database client: #{commands.join(', ')}. Check your $PATH and try again.")
+        end
+      end
+  end
+
+  module Command
+    class DbconsoleCommand < Base # :nodoc:
+      include EnvironmentArgument
+
+      class_option :include_password, aliases: "-p", type: :boolean,
+        desc: "Automatically provide the password from database.yml"
+
+      class_option :mode, enum: %w( html list line column ), type: :string,
+        desc: "Automatically put the sqlite3 database in the specified mode (html, list, line, column)."
+
+      class_option :header, type: :boolean
+
+      class_option :database, aliases: "--db", type: :string,
+        desc: "Specifies the database to use."
+
+      desc "dbconsole", "Starts DB REPL console"
+      long_desc Help.text(:dbconsole)
+      def perform
+        extract_environment_option_from_argument
+        require_application_and_environment!
+        Jets::DBConsole.start(options)
+      end
+    end
+  end
+end

data/lib/jets/commands/degenerate/degenerate_command.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+require "jets/generators"
+require "jets/commands/generate/generate_command"
+
+module Jets
+  module Command
+    class DegenerateCommand < GenerateCommand # :nodoc:
+      desc "Degenerate", "Opposite of generate. Removes the generated code."
+      long_desc Help.text(:degenerate)
+      def perform(*)
+        super
+      end
+
+    private
+      def behavior
+        :revoke
+      end
+    end
+  end
+end

data/lib/jets/commands/delete/delete_command.rb

@@ -0,0 +1,155 @@
+module Jets::Command
+  class DeleteCommand < Base # :nodoc:
+    desc "delete", "Delete the Jets project and all its resources"
+    long_desc Help.text(:delete)
+    option :yes, aliases: %w[y], type: :boolean, desc: "Skip are you sure prompt."
+    option :wait, type: :boolean, default: true, desc: "Wait for stack deletion to complete."
+    def perform
+      Delete.new(options).run
+    end
+  end
+
+  class Delete
+    include Jets::AwsServices
+    include AwsHelpers
+
+    def initialize(options={})
+      @options = options
+    end
+
+    def run
+      puts("Deleting project...")
+      return if @options[:noop]
+
+      are_you_sure?
+      confirm_project_exists
+
+      # Must first remove all objects from s3 bucket in order to delete stack
+      puts "First, deleting objects in s3 bucket #{s3_bucket_name}" if s3_bucket_name
+      empty_s3_bucket
+
+      stack_in_progress?(parent_stack_name)
+
+      cfn.delete_stack(stack_name: parent_stack_name)
+      puts "Deleting #{Jets.project_namespace.color(:green)}..."
+
+      if @options[:wait]
+        stack = find_stack(parent_stack_name)
+        wait_for_stack
+        Jets::Cfn::Deployment.new(stack_name: stack.stack_id, message: "Deleted").create
+      end
+
+      delete_logs
+
+      puts "Project #{Jets.project_namespace.color(:green)} deleted!"
+    end
+
+    def wait_for_stack
+      status = Jets::Cfn::Status.new(@options)
+      start_time = Time.now
+      status.wait
+      took = Time.now - start_time
+      puts "Time took for deletion: #{status.pretty_time(took).color(:green)}."
+    end
+
+    def delete_logs
+      puts "Deleting CloudWatch logs"
+      log = Jets::Commands::Clean::Log.new(mute: true, yes: true)
+      log.clean
+    end
+
+    def confirm_project_exists
+      stack = find_stack(parent_stack_name)
+      return if stack
+      puts "ERROR: Project #{parent_stack_name} does not exist".color(:red)
+      exit 1
+    end
+
+    def empty_s3_bucket
+      return unless s3_bucket_name # Happens when minimal stack fails to build
+      return unless bucket_exists?(s3_bucket_name)
+
+      resp = s3.list_objects(bucket: s3_bucket_name)
+      if resp.contents.size > 0
+        # IE: objects = [{key: "objectkey1"}, {key: "objectkey2"}]
+        objects = resp.contents.map { |item| {key: item.key} }
+        s3.delete_objects(
+          bucket: s3_bucket_name,
+          delete: {
+            objects: objects,
+            quiet: false,
+          }
+        )
+        empty_s3_bucket # keep deleting objects until bucket is empty
+      end
+    end
+
+    def s3_bucket_name
+      return @s3_bucket_name if defined?(@s3_bucket_name)
+
+      resp = cfn.describe_stacks(stack_name: parent_stack_name)
+      outputs = resp.stacks[0].outputs
+      if outputs.empty?
+        @s3_bucket_name = false
+      else
+        @s3_bucket_name = outputs.find {|o| o.output_key == 'S3Bucket'}.output_value
+      end
+    end
+
+    # Thanks: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/s3-example-does-bucket-exist.html
+    def bucket_exists?(bucket_name)
+      bucket_exists = false
+      begin
+        resp = s3.head_bucket(bucket: bucket_name, use_accelerate_endpoint: false)
+        bucket_exists = true
+      rescue
+      end
+      bucket_exists
+    end
+
+    def parent_stack_name
+      Jets::Names.parent_stack_name
+    end
+
+    def are_you_sure?
+      if @options[:yes]
+        sure = 'y'
+      else
+        puts "Are you sure you want to want to delete the #{Jets.project_namespace.color(:green)} project? (y/N)"
+        sure = $stdin.gets
+      end
+
+      unless sure =~ /^y/
+        puts "Phew! Jets #{Jets.project_namespace.color(:green)} project was not deleted."
+        exit 0
+      end
+    end
+
+    # All CloudFormation states listed here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-describing-stacks.html
+    #
+    # Returns resp so we can use it to grab data about the stack without calling api again.
+    def check_deleteable_status
+      return true if !stack_exists?(@parent_stack_name)
+
+      # Assumes stack exists
+      resp = cfn.describe_stacks(stack_name: @parent_stack_name)
+      status = resp.stacks[0].stack_status
+
+      return true if status == 'ROLLBACK_COMPLETE'
+
+      if status =~ /_IN_PROGRESS$/
+        puts "The '#{@parent_stack_name}' stack status is #{status}. " \
+            "It is not in an updateable status. Please wait until the stack is ready and try again.".color(:red)
+        exit 0
+      elsif resp.stacks[0].outputs.empty?
+        # This Happens when the miminal stack fails at the very beginning.
+        # There is no s3 bucket at all.  User should delete the stack.
+        puts "The minimal stack failed to create. Please delete the stack first and try again. " \
+        "You can delete the CloudFormation stack or use the `jets delete` command"
+        exit 0
+      else
+        true
+      end
+    end
+  end
+end