kuby-core 0.7.2 → 0.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a163afc89c23d091afce04277e9308252ae8335f8137f2136b68355dccb74bc2
-  data.tar.gz: 617007a29710ec9359dc4f3d5b31d850e22a0eacdc70175c444cf6d94a53462f
+  metadata.gz: f61bf7246e7fed1b51421252d1fee25da0bf1ad35da792e9f705d306ad5aeeba
+  data.tar.gz: fcaed876fd5da816cd1458c41c19fc735a7d3e5de2d23879427c526f2f0f94ea
 SHA512:
-  metadata.gz: 8fd6302f94ffa2a740928bd324ca5e2bbe251f49356ce582e581c4d91740b66425b6c1a3b392e40fe7c52e649781fe939fa377d6894e435e00386ddaed0f4ee3
-  data.tar.gz: 4ae55a8fc6c4aeceac417b54e41f679a988c34788e5740e268ddeb728223f949915c17e8bc738ffcd744a257a1d5ba74cf431d1df90c8b5b496eaa51861229d6
+  metadata.gz: 78f6cad56fd00b1265a43ac36169da5d46b15c50ceb5afce6f519ef175ff3949f84204ccf40ad9531b9dbb6b357d38ac0396e6969d848bfb17b42ba1c273ba6d
+  data.tar.gz: 056b9a151fd6f11079eb87fd5c0b7c4f92087cd8455a361ac7e668e9b94e4fc99a866b91c4f30d0a3eda582b0c2cafaac2e3ee882daebec0a52f1fd748786871

data/CHANGELOG.md

@@ -1,3 +1,45 @@
+## 0.10.0
+* Set default database user and password in dev environment.
+* Add ability to run rake tasks in dev environment.
+* Disallow running rails and rake tasks in non-dev environments.
+* Don't run database config through ERB.
+  - Rails env often isn't loaded, so ERB rendering can blow up with `NoMethodError`s, etc.
+  - All we really need to know is what database engine to stand up.
+* Require database user/password to be added manually to Kuby config.
+
+## 0.9.1
+* Run dev setup when asked to.
+  - Bug caused dev setup to be skipped even when requested.
+* Deployer should be tolerant of missing namespace.
+
+## 0.9.0
+* Add support for developing your app using a local Kubernetes cluster.
+  - Includes a default `:development` Kuby environment.
+* Remove rake tasks in favor of a `kuby` executable powered by [GLI](https://github.com/davetron5000/gli).
+* Rename the `minikube` provider to `docker_desktop`, which is more accurate.
+* Add more tests.
+* Avoid running commands inside pods that aren't running or that are marked for deletion (#15).
+* Pass `RAILS_MASTER_KEY` to Docker build (#14).
+* Add `kuby remote restart` command for manually restarting Rails pods.
+* Automatically restart Rails pods if deploy doesn't change the Docker image URL (#11).
+
+## 0.8.1
+* Fix database config rewriter task.
+  - Broke with refactoring of database config code.
+* More correctly parse Docker image URLs.
+  - It can be challenging to identify the hostname in image URLs because 1) the host can be omitted, and 2) the scheme is often omitted.
+  - The new strategy is to look for a "." in the first segment of the URL if there is no scheme.  It's not bulletproof but is better than what we had before, which was to assume the first segment was the host. Eg. for an image URL like camertron/foo, we would identify the host as "camertron."
+* Add a number of tests and a Rails dummy app in spec/.
+
+## 0.8.0
+* Upgrade to Krane >= 1.1.4, < 2.0.
+* Remove Krane monkeypatch in ext/.
+* Implement a Rails static asset server.
+* Move plugins from `Kuby::Kubernetes` namespace to `Kuby` namespace.
+  - This is to eventually enable plugins to modify the Dockerfile and introduce additional Dockerfiles (i.e. to enable a development mode, etc).
+* Pass `Environment` instead of `Definition` instances around.
+  - Providers, plugins, etc all take `Definition` instances. `Definition#kubernetes`, for example, returns the Kubernetes spec for the `Environment` specified by `KUBY_ENV` (or the first env defined if `KUBY_ENV` is not set). This is a problem for Kuby configs that specify multiple environments, and causes plugins to make changes to the default environment instead of the one they've been specifically added to. For example, if the `:production` env is defined first, the `:development` env still gets a cluster issuer from cert-manager even though `enable_tls` is set to `false`.
+
 ## 0.7.2
 * Fix issue causing `Kuby.environment(...)` to raise an `UndefinedEnvironmentError` for existing environments.
 

data/Gemfile

@@ -9,4 +9,5 @@ end
 
 group :test do
   gem 'rspec', '~> 3.0'
+  gem 'timecop', '~> 0.9'
 end

data/README.md

@@ -8,6 +8,16 @@ At its core, Kuby is a set of tools and smart defaults that encapsulate and codi
 
 Under the hood, Kuby leverages the power of Docker and Kubernetes. It tries to make these technologies accessible to the average Rails dev without requiring a devops black belt.
 
+## Why Kuby?
+
+Kuby embraces the same convention-over-configuration approach that Rails does. It aims to reduce the cognitive overhead associated with learning a bunch of ops tools to get your app onto the internet. In other words, Kuby does a whole lot for you. Specifically, it:
+
+* leverages Docker and Kubernetes, industry-leading infrastructure tools.
+* automatically configures your app with a TLS certificate from [LetsEncrypt](https://letsencrypt.org/).
+* automatically spins up a database instance based on what's in your database.yml.
+* runs a separate server for your static assets.
+* features a powerful plugin system that allows, for example, easy [Sidekiq integration](https://github.com/getkuby/kuby-sidekiq).
+
 ## Getting Started
 
 See the [Quick Start Guide](https://github.com/getkuby/kuby-core/wiki/Quick-Start-Guide)
@@ -18,7 +28,7 @@ See the [wiki](https://github.com/getkuby/kuby-core/wiki).
 
 ## Running Tests
 
-`bundle exec rspec` should do the trick... or at least it would if there were any tests. Don't worry, it's on my radar.
+`bundle exec rspec` should do the trick. Test coverage is very minimal at the moment, however.
 
 ## License
 

data/bin/kuby

@@ -0,0 +1,4 @@
+#! /usr/bin/env ruby
+
+require 'kuby'
+exit Kuby::Commands.run(ARGV)

data/kuby-core.gemspec

@@ -14,17 +14,22 @@ Gem::Specification.new do |s|
 
   s.add_dependency 'colorize', '~> 0.8'
   s.add_dependency 'docker-remote', '~> 0.1'
-  s.add_dependency 'krane', '~> 1.0'
-  s.add_dependency 'kuby-cert-manager', '~> 0.2'
+  s.add_dependency 'gli', '~> 2.0'
+  s.add_dependency 'helm-cli', '~> 0.3'
+  # See: https://github.com/Shopify/krane/pull/720
+  # See: https://github.com/Shopify/krane/blob/master/CHANGELOG.md#114
+  s.add_dependency 'krane', '>= 1.1.4', '< 2.0'
+  s.add_dependency 'kuby-cert-manager', '>= 0.3'
   s.add_dependency 'kube-dsl', '~> 0.3'
-  s.add_dependency 'kuby-kube-db', '~> 0.4'
-  s.add_dependency 'kubernetes-cli', '~> 0.2'
+  s.add_dependency 'kuby-kube-db', '>= 0.6'
+  s.add_dependency 'kubernetes-cli', '~> 0.3'
   s.add_dependency 'railties', '>= 5.1'
   s.add_dependency 'rouge', '~> 3.0'
 
   s.add_development_dependency 'rspec'
 
   s.require_path = 'lib'
+  s.executables << 'kuby'
 
   s.files = Dir['{lib,spec}/**/*', 'Gemfile', 'LICENSE', 'CHANGELOG.md', 'README.md', 'Rakefile', 'kuby-core.gemspec']
 end

data/lib/kuby.rb

@@ -1,29 +1,46 @@
 require 'kuby/railtie'
 
 begin
-  require 'kuby/kubernetes/plugins/rails_app/generators/kuby'
+  require 'kuby/plugins/rails_app/generators/kuby'
 rescue NameError
 end
 
 module Kuby
-  autoload :BasicLogger,  'kuby/basic_logger'
-  autoload :CLIBase,      'kuby/cli_base'
-  autoload :Definition,   'kuby/definition'
-  autoload :Docker,       'kuby/docker'
-  autoload :Environment,  'kuby/environment'
-  autoload :Kubernetes,   'kuby/kubernetes'
-  autoload :Middleware,   'kuby/middleware'
-  autoload :Tasks,        'kuby/tasks'
-  autoload :TrailingHash, 'kuby/trailing_hash'
+  autoload :BasicLogger,    'kuby/basic_logger'
+  autoload :CLIBase,        'kuby/cli_base'
+  autoload :Commands,       'kuby/commands'
+  autoload :Definition,     'kuby/definition'
+  autoload :DevSetup,       'kuby/dev_setup'
+  autoload :Docker,         'kuby/docker'
+  autoload :Environment,    'kuby/environment'
+  autoload :Kubernetes,     'kuby/kubernetes'
+  autoload :Middleware,     'kuby/middleware'
+  autoload :Plugin,         'kuby/plugin'
+  autoload :PluginRegistry, 'kuby/plugin_registry'
+  autoload :Plugins,        'kuby/plugins'
+  autoload :RailsCommands,  'kuby/rails_commands'
+  autoload :Tasks,          'kuby/tasks'
+  autoload :TrailingHash,   'kuby/trailing_hash'
+
+  DEFAULT_ENV = 'development'.freeze
+  DEFAULT_DB_USER = 'root'.freeze
+  DEFAULT_DB_PASSWORD = 'password'.freeze
 
   class UndefinedEnvironmentError < StandardError; end
+  class MissingConfigError < StandardError; end
 
   class << self
     attr_reader :definition
     attr_writer :logger
 
-    def load!
-      require ENV['KUBY_CONFIG'] || File.join('.', 'kuby.rb')
+    def load!(config_file = nil)
+      config_file ||= ENV['KUBY_CONFIG'] || File.join('.', 'kuby.rb')
+
+      unless File.exist?(config_file)
+        raise MissingConfigError, "couldn't find Kuby config file at #{config_file}"
+      end
+
+      require config_file
     end
 
     def define(name, &block)
@@ -32,9 +49,27 @@ module Kuby
       @definition = Definition.new(name.to_s)
       @definition.instance_eval(&block)
 
+      # default development environment
+      @definition.environment(:development) do
+        kubernetes do
+          add_plugin(:rails_app) do
+            tls_enabled false
+
+            database do
+              user(DEFAULT_DB_USER) if respond_to?(:user)
+              password(DEFAULT_DB_PASSWORD) if respond_to?(:password)
+            end
+          end
+
+          provider :docker_desktop
+        end
+      end
+
       @definition.environments.each do |_, env|
         env.kubernetes.after_configuration
       end
+
+      @definition
     end
 
     def environment(name = env)
@@ -52,8 +87,8 @@ module Kuby
       @providers ||= {}
     end
 
-    def register_plugin(plugin_name, plugin_klass)
-      plugins[plugin_name] = plugin_klass
+    def register_plugin(*args, **kwargs)
+      plugins.register(*args, **kwargs)
     end
 
     def register_distro(distro_name, distro_klass)
@@ -65,7 +100,7 @@ module Kuby
     end
 
     def plugins
-      @plugins ||= {}
+      @plugins ||= PluginRegistry.new
     end
 
     def logger
@@ -95,20 +130,24 @@ module Kuby
       @packages ||= {}
     end
 
+    def env=(env_name)
+      @env = env_name.to_s
+    end
+
     def env
       ENV.fetch('KUBY_ENV') do
-        (definition.environments.keys.first || Rails.env).to_s
+        (@env || Rails.env rescue nil || DEFAULT_ENV).to_s
       end
     end
   end
 end
 
 # providers
-Kuby.register_provider(:minikube, Kuby::Kubernetes::MinikubeProvider)
+Kuby.register_provider(:docker_desktop, Kuby::Kubernetes::DockerDesktopProvider)
 
 # plugins
-Kuby.register_plugin(:rails_app, Kuby::Kubernetes::Plugins::RailsApp::Plugin)
-Kuby.register_plugin(:nginx_ingress, Kuby::Kubernetes::Plugins::NginxIngress)
+Kuby.register_plugin(:rails_app, Kuby::Plugins::RailsApp::Plugin)
+Kuby.register_plugin(:nginx_ingress, Kuby::Plugins::NginxIngress)
 
 # distros
 Kuby.register_distro(:debian, Kuby::Docker::Debian)

data/lib/kuby/basic_logger.rb

@@ -18,5 +18,18 @@ module Kuby
     def fatal(msg, *args)
       super(ColorizedString[msg].red, *args)
     end
+
+    # adhere to the "CLI" interface
+    def with_pipes(out = STDOUT, err = STDERR)
+      previous_logdev = @logdev || STDERR
+      reopen(err)
+      yield
+    ensure
+      reopen(previous_logdev)
+    end
+
+    def last_status
+      nil
+    end
   end
 end

data/lib/kuby/cli_base.rb

@@ -17,17 +17,59 @@ module Kuby
       @after_execute << block
     end
 
+    def with_pipes(out = STDOUT, err = STDERR)
+      previous_stdout = self.stdout
+      previous_stderr = self.stderr
+      self.stdout = out
+      self.stderr = err
+      yield
+    ensure
+      self.stdout = previous_stdout
+      self.stderr = previous_stderr
+    end
+
+    def stdout
+      Thread.current[stdout_key] || STDOUT
+    end
+
+    def stdout=(new_stdout)
+      Thread.current[stdout_key] = new_stdout
+    end
+
+    def stderr
+      Thread.current[stderr_key] || STDERR
+    end
+
+    def stderr=(new_stderr)
+      Thread.current[stderr_key] = new_stderr
+    end
+
     private
 
     def open3_w(env, cmd, opts = {}, &block)
       run_before_callbacks(cmd)
       cmd_s = cmd.join(' ')
 
-      Open3.pipeline_w([env, cmd_s], opts) do |stdin, wait_threads|
-        yield(stdin, wait_threads).tap do
-          stdin.close
-          self.last_status = wait_threads.last.value
+      Open3.popen3(env, cmd_s, opts) do |p_stdin, p_stdout, p_stderr, wait_thread|
+        Thread.new(stdout) do |t_stdout|
+          begin
+            p_stdout.each { |line| t_stdout.puts(line) }
+          rescue IOError
+          end
+        end
+
+        Thread.new(stderr) do |t_stderr|
+          begin
+            p_stderr.each { |line| t_stderr.puts(line) }
+          rescue IOError
+          end
+        end
+
+        yield(p_stdin).tap do
+          p_stdin.close
+          self.last_status = wait_thread.value
           run_after_callbacks(cmd)
+          wait_thread.join
         end
       end
     end
@@ -41,19 +83,42 @@ module Kuby
     def systemm(cmd)
       run_before_callbacks(cmd)
       cmd_s = cmd.join(' ')
-      system(cmd_s).tap do
-        self.last_status = $?
+
+      Open3.popen3(cmd_s) do |p_stdin, p_stdout, p_stderr, wait_thread|
+        Thread.new(stdout) do |t_stdout|
+          p_stdout.each { |line| t_stdout.puts(line) }
+        end
+
+        Thread.new(stderr) do |t_stderr|
+          p_stderr.each { |line| t_stderr.puts(line) }
+        end
+
+        self.last_status = wait_thread.value
         run_after_callbacks(cmd)
+        wait_thread.join
       end
     end
 
     def backticks(cmd)
       run_before_callbacks(cmd)
       cmd_s = cmd.join(' ')
-      `#{cmd_s}`.tap do
-        self.last_status = $?
+      result = StringIO.new
+
+      Open3.popen3(cmd_s) do |p_stdin, p_stdout, p_stderr, wait_thread|
+        Thread.new do
+          p_stdout.each { |line| result.puts(line) }
+        end
+
+        Thread.new(stderr) do |t_stderr|
+          p_stderr.each { |line| t_stderr.puts(line) }
+        end
+
+        self.last_status = wait_thread.value
         run_after_callbacks(cmd)
+        wait_thread.join
       end
+
+      result.string
     end
 
     def run_before_callbacks(cmd)
@@ -71,5 +136,13 @@ module Kuby
     def status_key
       raise NotImplementedError, "#{__method__} must be defined in derived classes"
     end
+
+    def stdout_key
+      raise NotImplementedError, "#{__method__} must be defined in derived classes"
+    end
+
+    def stderr_key
+      raise NotImplementedError, "#{__method__} must be defined in derived classes"
+    end
   end
 end

data/lib/kuby/commands.rb

@@ -0,0 +1,244 @@
+require 'kuby/version'
+require 'gli'
+
+module Kuby
+  class Commands
+    extend GLI::App
+
+    # GLI doesn't have a wildcard option, so it's impossible to tell it to
+    # slurp up all args after a certain point. In our case, we want to be
+    # able to invoke `rails` commands and pass through all the original
+    # flags, switches, etc. To get around GLI's limitations, we identify
+    # `rails` commands in this hijacked `run` method and only use GLI to
+    # parse global options (like -e). The rest of the Rails options are
+    # captured in an instance variable and thereby made available to the
+    # Rails command handlers defined below. We use Module#prepend here to
+    # avoid the usual series of cryptic alias_method calls (note that there
+    # is no singleton class version of #prepend in the Ruby language).
+    singleton_class.send(:prepend, Module.new do
+      def run(args)
+        if idx = args.index('rails') || idx = args.index('rake')
+          @rails_options = args[idx..-1]
+          super(args[0..idx])
+        else
+          super
+        end
+      end
+    end)
+
+    def self.tasks
+      Kuby::Tasks.new(Kuby.environment)
+    end
+
+    def self.must_be_dev_env!
+      unless Kuby.environment.development?
+        fail "Command not supported in the '#{Kuby.environment.name}' environment"
+      end
+    end
+
+    program_desc 'Kuby command-line interface. Kuby is a convention '\
+      'over configuration approach for running Rails apps in Kubernetes.'
+
+    version Kuby::VERSION
+
+    subcommand_option_handling :normal
+    arguments :loose
+
+    desc 'The Kuby environment to use. Overrides KUBY_ENV.'
+    flag [:e, :environment], type: String, required: false
+
+    desc 'Path to your Kuby config file. Overrides KUBY_CONFIG.'
+    default_value './kuby.rb'
+    flag [:c, :config]
+
+    pre do |global_options, options, args|
+      Kuby.env = global_options[:environment] if global_options[:environment]
+      Kuby.load!(global_options[:config])
+
+      # GLI will abort unless this block returns a truthy value
+      true
+    end
+
+    # These are only stubs included to fill out the help screens. Rails
+    # commands are handled by the RailsCommands class.
+    desc 'Runs a Rails command.'
+    command :rails do |rc|
+      rc.action do |global_options, options, args|
+        must_be_dev_env!
+        exit 1 unless tasks.dev_deployment_ok
+        Kuby::RailsCommands.run(@rails_options)
+      end
+
+      rc.desc 'Runs the rails server (run `rails server --help` for options)'
+      rc.command [:server, :s] do |c|
+        c.action do |global_options, options, args|
+          must_be_dev_env!
+          exit 1 unless tasks.dev_deployment_ok
+          Kuby::RailsCommands.run(@rails_options)
+        end
+      end
+
+      rc.desc 'Runs a script in the Rails environment (run `rails runner --help` for options)'
+      rc.command [:runner, :r] do |c|
+        c.action do |global_options, options, args|
+          must_be_dev_env!
+          exit 1 unless tasks.dev_deployment_ok
+          Kuby::RailsCommands.run(@rails_options)
+        end
+      end
+
+      rc.desc 'Starts an interactive Ruby console with the Rails environment loaded '\
+        '(run `rails console --help` for options)'
+      rc.command [:console, :c] do |c|
+        c.action do |global_options, options, args|
+          must_be_dev_env!
+          exit 1 unless tasks.dev_deployment_ok
+          Kuby::RailsCommands.run(@rails_options)
+        end
+      end
+    end
+
+    desc 'Runs a rake task.'
+    command :rake do |rc|
+      rc.action do |global_options, options, args|
+        must_be_dev_env!
+        exit 1 unless tasks.dev_deployment_ok
+        Kuby::RailsCommands.run(@rails_options)
+      end
+    end
+
+    desc 'Builds the Docker image.'
+    command :build do |c|
+      c.action do |global_options, options, args|
+        tasks.build
+      end
+    end
+
+    desc 'Pushes the Docker image to the configured registry.'
+    command :push do |c|
+      c.action do |global_options, options, args|
+        tasks.push
+      end
+    end
+
+    desc 'Gets your Kubernetes cluster ready to run your Rails app.'
+    command :setup do |c|
+      c.action do |global_options, options, args|
+        tasks.setup
+      end
+    end
+
+    desc 'Prints the effective Dockerfile used to build the Docker image.'
+    command :dockerfile do |c|
+      c.action do |global_options, options, args|
+        tasks.print_dockerfile
+      end
+    end
+
+    desc 'Deploys the application.'
+    command :deploy do |c|
+      c.desc 'The Docker tag to deploy. Defaults to the most recent tag.'
+      c.flag [:t, :tag], required: false
+      c.action do |global_options, options, args|
+        tasks.deploy(options[:tag])
+      end
+    end
+
+    desc 'Rolls back to the previous Docker tag.'
+    command :rollback do |c|
+      c.action do |global_options, options, args|
+        tasks.rollback
+      end
+    end
+
+    desc 'Prints the effective Kubernetes resources that will be applied on deploy.'
+    command :resources do |c|
+      c.action do |global_options, options, args|
+        tasks.print_resources
+      end
+    end
+
+    desc 'Prints out the contents of the kubeconfig Kuby is using to communicate with your cluster.'
+    command :kubeconfig do |c|
+      c.action do |global_options, options, args|
+        tasks.print_kubeconfig
+      end
+    end
+
+    desc 'Runs an arbitrary kubectl command.'
+    command :kubectl do |c|
+      c.desc 'Prefixes the kubectl command with the namespace associated with '\
+        'the current environment. For example, if the Kuby env is "production", '\
+        'this option will prefix the kubectl command with "-n myapp-production".'
+      c.switch [:N, :namespaced], default: false
+      c.action do |global_options, options, args|
+        if options[:namespaced]
+          # sorry Demeter
+          namespace = Kuby.definition.environment.kubernetes.namespace.metadata.name
+          args = ['-n', namespace, *args]
+        end
+
+        tasks.kubectl(*args)
+      end
+    end
+
+    desc 'Runs commands, etc against the Kubernetes cluster.'
+    command :remote do |rc|
+      rc.desc 'Tails (i.e. continuously streams) the Rails log from your running application.'
+      rc.command :logs do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_logs
+        end
+      end
+
+      rc.desc 'Lists running Kubernetes pods.'
+      rc.command :status do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_status
+        end
+      end
+
+      rc.desc 'Runs an arbitrary command inside a running Rails pod.'
+      rc.command :exec do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_exec(args)
+        end
+      end
+
+      rc.desc 'Establishes a shell inside a running Rails pod.'
+      rc.command :shell do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_shell
+        end
+      end
+
+      rc.desc 'Establishes a Rails console inside a running Rails pod.'
+      rc.command :console do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_console
+        end
+      end
+
+      rc.desc 'Establishes a database console inside a running Rails pod.'
+      rc.command :dbconsole do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_dbconsole
+        end
+      end
+
+      rc.desc "Restarts the Rails app's web pods."
+      rc.command :restart do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_restart
+        end
+      end
+    end
+  end
+end