kuby-core 0.7.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7d3f784993d379920477923911af9c73a7ac0f661a8831a3fdc21feea83e9d89
-  data.tar.gz: 97390f124219ba7e697253c551bbef2527b15d7234793a76431ebdde73e996ed
+  metadata.gz: f9367e316dfab7a0210bea18ab630b65a5996ce0622960f3be2a76f8ec4482c1
+  data.tar.gz: d48777e79f11eab678545a2d398d3f60463e7d1209e6d15987c3f3665d571b82
 SHA512:
-  metadata.gz: 95fceaebbb3959fb3b872b570b9202deb4dfd4a44dc363c25b20398f9e2a199091eb5fdecbeb07caceb2be104b94c07f3294e8b8158737416e2b83efe7e65ebe
-  data.tar.gz: 248b60b0c0ab90d10c780ce69e3f63ff6aa4b222cef82c5121abf2acb4b462c6534b245556a2683649ccf57440e758d5846e93322828f127c66784701ea11127
+  metadata.gz: cbfd215f8b4a7cb2b1a53bc8de8298e4dd85f487f0245da82bd19a390d55500c20e97e155a07fafa09284683ee9c220bb4cdd89194c3e5cb4c6e3c055fbcce74
+  data.tar.gz: 03c15b9ffb8672ef2e45d2d818c3d048e694dae2656db0be6325a1edf6853b1c9c8bc7060b021add66b43d87c9ca3ac46bb71346f2e2a4e68bd89ba6d45635f9

data/CHANGELOG.md

@@ -1,3 +1,38 @@
+## 0.9.0
+* Add support for developing your app using a local Kubernetes cluster.
+  - Includes a default `:development` Kuby environment.
+* Remove rake tasks in favor of a `kuby` executable powered by [GLI](https://github.com/davetron5000/gli).
+* Rename the `minikube` provider to `docker_desktop`, which is more accurate.
+* Add more tests.
+* Avoid running commands inside pods that aren't running or that are marked for deletion (#15).
+* Pass `RAILS_MASTER_KEY` to Docker build (#14).
+* Add `kuby remote restart` command for manually restarting Rails pods.
+* Automatically restart Rails pods if deploy doesn't change the Docker image URL (#11).
+
+## 0.8.1
+* Fix database config rewriter task.
+  - Broke with refactoring of database config code.
+* More correctly parse Docker image URLs.
+  - It can be challenging to identify the hostname in image URLs because 1) the host can be omitted, and 2) the scheme is often omitted.
+  - The new strategy is to look for a "." in the first segment of the URL if there is no scheme.  It's not bulletproof but is better than what we had before, which was to assume the first segment was the host. Eg. for an image URL like camertron/foo, we would identify the host as "camertron."
+* Add a number of tests and a Rails dummy app in spec/.
+
+## 0.8.0
+* Upgrade to Krane >= 1.1.4, < 2.0.
+* Remove Krane monkeypatch in ext/.
+* Implement a Rails static asset server.
+* Move plugins from `Kuby::Kubernetes` namespace to `Kuby` namespace.
+  - This is to eventually enable plugins to modify the Dockerfile and introduce additional Dockerfiles (i.e. to enable a development mode, etc).
+* Pass `Environment` instead of `Definition` instances around.
+  - Providers, plugins, etc all take `Definition` instances. `Definition#kubernetes`, for example, returns the Kubernetes spec for the `Environment` specified by `KUBY_ENV` (or the first env defined if `KUBY_ENV` is not set). This is a problem for Kuby configs that specify multiple environments, and causes plugins to make changes to the default environment instead of the one they've been specifically added to. For example, if the `:production` env is defined first, the `:development` env still gets a cluster issuer from cert-manager even though `enable_tls` is set to `false`.
+
+## 0.7.2
+* Fix issue causing `Kuby.environment(...)` to raise an `UndefinedEnvironmentError` for existing environments.
+
+## 0.7.1
+* Fix timestamp tag parsing regression caused by adding anchor tags to the regex.
+  - Instead, let's rely on `strptime` and return `nil` if it throws an `ArgumentError`.
+
 ## 0.7.0
 * Automatically perform `docker login` if not already logged into the Docker registry.
 * Fix timestamp tag parsing issue causing deploy to fail with no available tags.

data/Gemfile

@@ -9,4 +9,5 @@ end
 
 group :test do
   gem 'rspec', '~> 3.0'
+  gem 'timecop', '~> 0.9'
 end

data/README.md

@@ -8,6 +8,16 @@ At its core, Kuby is a set of tools and smart defaults that encapsulate and codi
 
 Under the hood, Kuby leverages the power of Docker and Kubernetes. It tries to make these technologies accessible to the average Rails dev without requiring a devops black belt.
 
+## Why Kuby?
+
+Kuby embraces the same convention-over-configuration approach that Rails does. It aims to reduce the cognitive overhead associated with learning a bunch of ops tools to get your app onto the internet. In other words, Kuby does a whole lot for you. Specifically, it:
+
+* leverages Docker and Kubernetes, industry-leading infrastructure tools.
+* automatically configures your app with a TLS certificate from [LetsEncrypt](https://letsencrypt.org/).
+* automatically spins up a database instance based on what's in your database.yml.
+* runs a separate server for your static assets.
+* features a powerful plugin system that allows, for example, easy [Sidekiq integration](https://github.com/getkuby/kuby-sidekiq).
+
 ## Getting Started
 
 See the [Quick Start Guide](https://github.com/getkuby/kuby-core/wiki/Quick-Start-Guide)
@@ -18,7 +28,7 @@ See the [wiki](https://github.com/getkuby/kuby-core/wiki).
 
 ## Running Tests
 
-`bundle exec rspec` should do the trick... or at least it would if there were any tests. Don't worry, it's on my radar.
+`bundle exec rspec` should do the trick. Test coverage is very minimal at the moment, however.
 
 ## License
 

data/bin/kuby

@@ -0,0 +1,4 @@
+#! /usr/bin/env ruby
+
+require 'kuby'
+exit Kuby::Commands.run(ARGV)

data/kuby-core.gemspec

@@ -14,15 +14,22 @@ Gem::Specification.new do |s|
 
   s.add_dependency 'colorize', '~> 0.8'
   s.add_dependency 'docker-remote', '~> 0.1'
-  s.add_dependency 'krane', '~> 1.0'
-  s.add_dependency 'kuby-cert-manager', '~> 0.2'
+  s.add_dependency 'gli', '~> 2.0'
+  s.add_dependency 'helm-cli', '~> 0.3'
+  # See: https://github.com/Shopify/krane/pull/720
+  # See: https://github.com/Shopify/krane/blob/master/CHANGELOG.md#114
+  s.add_dependency 'krane', '>= 1.1.4', '< 2.0'
+  s.add_dependency 'kuby-cert-manager', '>= 0.3'
   s.add_dependency 'kube-dsl', '~> 0.3'
-  s.add_dependency 'kuby-kube-db', '~> 0.4'
-  s.add_dependency 'kubernetes-cli', '~> 0.2'
+  s.add_dependency 'kuby-kube-db', '>= 0.6'
+  s.add_dependency 'kubernetes-cli', '~> 0.3'
   s.add_dependency 'railties', '>= 5.1'
   s.add_dependency 'rouge', '~> 3.0'
 
+  s.add_development_dependency 'rspec'
+
   s.require_path = 'lib'
+  s.executables << 'kuby'
 
   s.files = Dir['{lib,spec}/**/*', 'Gemfile', 'LICENSE', 'CHANGELOG.md', 'README.md', 'Rakefile', 'kuby-core.gemspec']
 end

data/lib/kuby.rb

@@ -1,29 +1,44 @@
 require 'kuby/railtie'
 
 begin
-  require 'kuby/kubernetes/plugins/rails_app/generators/kuby'
+  require 'kuby/plugins/rails_app/generators/kuby'
 rescue NameError
 end
 
 module Kuby
-  autoload :BasicLogger,  'kuby/basic_logger'
-  autoload :CLIBase,      'kuby/cli_base'
-  autoload :Definition,   'kuby/definition'
-  autoload :Docker,       'kuby/docker'
-  autoload :Environment,  'kuby/environment'
-  autoload :Kubernetes,   'kuby/kubernetes'
-  autoload :Middleware,   'kuby/middleware'
-  autoload :Tasks,        'kuby/tasks'
-  autoload :TrailingHash, 'kuby/trailing_hash'
+  autoload :BasicLogger,    'kuby/basic_logger'
+  autoload :CLIBase,        'kuby/cli_base'
+  autoload :Commands,       'kuby/commands'
+  autoload :Definition,     'kuby/definition'
+  autoload :DevSetup,       'kuby/dev_setup'
+  autoload :Docker,         'kuby/docker'
+  autoload :Environment,    'kuby/environment'
+  autoload :Kubernetes,     'kuby/kubernetes'
+  autoload :Middleware,     'kuby/middleware'
+  autoload :Plugin,         'kuby/plugin'
+  autoload :PluginRegistry, 'kuby/plugin_registry'
+  autoload :Plugins,        'kuby/plugins'
+  autoload :RailsCommands,  'kuby/rails_commands'
+  autoload :Tasks,          'kuby/tasks'
+  autoload :TrailingHash,   'kuby/trailing_hash'
+
+  DEFAULT_ENV = 'development'.freeze
 
   class UndefinedEnvironmentError < StandardError; end
+  class MissingConfigError < StandardError; end
 
   class << self
     attr_reader :definition
     attr_writer :logger
 
-    def load!
-      require ENV['KUBY_CONFIG'] || File.join('.', 'kuby.rb')
+    def load!(config_file = nil)
+      config_file ||= ENV['KUBY_CONFIG'] || File.join('.', 'kuby.rb')
+
+      unless File.exist?(config_file)
+        raise MissingConfigError, "couldn't find Kuby config file at #{config_file}"
+      end
+
+      require config_file
     end
 
     def define(name, &block)
@@ -32,16 +47,29 @@ module Kuby
       @definition = Definition.new(name.to_s)
       @definition.instance_eval(&block)
 
+      # default development environment
+      @definition.environment(:development) do
+        kubernetes do
+          add_plugin(:rails_app) do
+            tls_enabled false
+          end
+
+          provider :docker_desktop
+        end
+      end
+
       @definition.environments.each do |_, env|
         env.kubernetes.after_configuration
       end
+
+      @definition
     end
 
     def environment(name = env)
-      definition.environment(name.to_s) do
-        raise UndefinedEnvironmentError, "couldn't find a Kuby environment named "\
-          "'#{environment}'"
-      end
+      definition.environment(name.to_s) || raise(
+        UndefinedEnvironmentError, "couldn't find a Kuby environment named "\
+        "'#{name}'"
+      )
     end
 
     def register_provider(provider_name, provider_klass)
@@ -52,8 +80,8 @@ module Kuby
       @providers ||= {}
     end
 
-    def register_plugin(plugin_name, plugin_klass)
-      plugins[plugin_name] = plugin_klass
+    def register_plugin(*args, **kwargs)
+      plugins.register(*args, **kwargs)
     end
 
     def register_distro(distro_name, distro_klass)
@@ -65,7 +93,7 @@ module Kuby
     end
 
     def plugins
-      @plugins ||= {}
+      @plugins ||= PluginRegistry.new
     end
 
     def logger
@@ -95,20 +123,24 @@ module Kuby
       @packages ||= {}
     end
 
+    def env=(env_name)
+      @env = env_name.to_s
+    end
+
     def env
       ENV.fetch('KUBY_ENV') do
-        (definition.environments.keys.first || Rails.env).to_s
+        (@env || Rails.env rescue nil || DEFAULT_ENV).to_s
       end
     end
   end
 end
 
 # providers
-Kuby.register_provider(:minikube, Kuby::Kubernetes::MinikubeProvider)
+Kuby.register_provider(:docker_desktop, Kuby::Kubernetes::DockerDesktopProvider)
 
 # plugins
-Kuby.register_plugin(:rails_app, Kuby::Kubernetes::Plugins::RailsApp::Plugin)
-Kuby.register_plugin(:nginx_ingress, Kuby::Kubernetes::Plugins::NginxIngress)
+Kuby.register_plugin(:rails_app, Kuby::Plugins::RailsApp::Plugin)
+Kuby.register_plugin(:nginx_ingress, Kuby::Plugins::NginxIngress)
 
 # distros
 Kuby.register_distro(:debian, Kuby::Docker::Debian)

data/lib/kuby/basic_logger.rb

@@ -18,5 +18,18 @@ module Kuby
     def fatal(msg, *args)
       super(ColorizedString[msg].red, *args)
     end
+
+    # adhere to the "CLI" interface
+    def with_pipes(out = STDOUT, err = STDERR)
+      previous_logdev = @logdev || STDERR
+      reopen(err)
+      yield
+    ensure
+      reopen(previous_logdev)
+    end
+
+    def last_status
+      nil
+    end
   end
 end

data/lib/kuby/cli_base.rb

@@ -17,17 +17,59 @@ module Kuby
       @after_execute << block
     end
 
+    def with_pipes(out = STDOUT, err = STDERR)
+      previous_stdout = self.stdout
+      previous_stderr = self.stderr
+      self.stdout = out
+      self.stderr = err
+      yield
+    ensure
+      self.stdout = previous_stdout
+      self.stderr = previous_stderr
+    end
+
+    def stdout
+      Thread.current[stdout_key] || STDOUT
+    end
+
+    def stdout=(new_stdout)
+      Thread.current[stdout_key] = new_stdout
+    end
+
+    def stderr
+      Thread.current[stderr_key] || STDERR
+    end
+
+    def stderr=(new_stderr)
+      Thread.current[stderr_key] = new_stderr
+    end
+
     private
 
     def open3_w(env, cmd, opts = {}, &block)
       run_before_callbacks(cmd)
       cmd_s = cmd.join(' ')
 
-      Open3.pipeline_w([env, cmd_s], opts) do |stdin, wait_threads|
-        yield(stdin, wait_threads).tap do
-          stdin.close
-          self.last_status = wait_threads.last.value
+      Open3.popen3(env, cmd_s, opts) do |p_stdin, p_stdout, p_stderr, wait_thread|
+        Thread.new(stdout) do |t_stdout|
+          begin
+            p_stdout.each { |line| t_stdout.puts(line) }
+          rescue IOError
+          end
+        end
+
+        Thread.new(stderr) do |t_stderr|
+          begin
+            p_stderr.each { |line| t_stderr.puts(line) }
+          rescue IOError
+          end
+        end
+
+        yield(p_stdin).tap do
+          p_stdin.close
+          self.last_status = wait_thread.value
           run_after_callbacks(cmd)
+          wait_thread.join
         end
       end
     end
@@ -41,19 +83,42 @@ module Kuby
     def systemm(cmd)
       run_before_callbacks(cmd)
       cmd_s = cmd.join(' ')
-      system(cmd_s).tap do
-        self.last_status = $?
+
+      Open3.popen3(cmd_s) do |p_stdin, p_stdout, p_stderr, wait_thread|
+        Thread.new(stdout) do |t_stdout|
+          p_stdout.each { |line| t_stdout.puts(line) }
+        end
+
+        Thread.new(stderr) do |t_stderr|
+          p_stderr.each { |line| t_stderr.puts(line) }
+        end
+
+        self.last_status = wait_thread.value
         run_after_callbacks(cmd)
+        wait_thread.join
       end
     end
 
     def backticks(cmd)
       run_before_callbacks(cmd)
       cmd_s = cmd.join(' ')
-      `#{cmd_s}`.tap do
-        self.last_status = $?
+      result = StringIO.new
+
+      Open3.popen3(cmd_s) do |p_stdin, p_stdout, p_stderr, wait_thread|
+        Thread.new do
+          p_stdout.each { |line| result.puts(line) }
+        end
+
+        Thread.new(stderr) do |t_stderr|
+          p_stderr.each { |line| t_stderr.puts(line) }
+        end
+
+        self.last_status = wait_thread.value
         run_after_callbacks(cmd)
+        wait_thread.join
       end
+
+      result.string
     end
 
     def run_before_callbacks(cmd)
@@ -71,5 +136,13 @@ module Kuby
     def status_key
       raise NotImplementedError, "#{__method__} must be defined in derived classes"
     end
+
+    def stdout_key
+      raise NotImplementedError, "#{__method__} must be defined in derived classes"
+    end
+
+    def stderr_key
+      raise NotImplementedError, "#{__method__} must be defined in derived classes"
+    end
   end
 end

data/lib/kuby/commands.rb

@@ -0,0 +1,220 @@
+require 'kuby/version'
+require 'gli'
+
+module Kuby
+  class Commands
+    extend GLI::App
+
+    # GLI doesn't have a wildcard option, so it's impossible to tell it to
+    # slurp up all args after a certain point. In our case, we want to be
+    # able to invoke `rails` commands and pass through all the original
+    # flags, switches, etc. To get around GLI's limitations, we identify
+    # `rails` commands in this hijacked `run` method and only use GLI to
+    # parse global options (like -e). The rest of the Rails options are
+    # captured in an instance variable and thereby made available to the
+    # Rails command handlers defined below. We use Module#prepend here to
+    # avoid the usual series of cryptic alias_method calls (note that there
+    # is no singleton class version of #prepend in the Ruby language).
+    singleton_class.send(:prepend, Module.new do
+      def run(args)
+        if idx = args.index('rails')
+          @rails_options = args[(idx + 1)..-1]
+          super(args[0..(idx + 1)])
+        else
+          super
+        end
+      end
+    end)
+
+    def self.tasks
+      Kuby::Tasks.new(Kuby.definition.environment)
+    end
+
+    program_desc 'Kuby command-line interface. Kuby is a convention '\
+      'over configuration approach for running Rails apps in Kubernetes.'
+
+    version Kuby::VERSION
+
+    subcommand_option_handling :normal
+    arguments :loose
+
+    desc 'The Kuby environment to use. Overrides KUBY_ENV.'
+    flag [:e, :environment], type: String, required: false
+
+    desc 'Path to your Kuby config file. Overrides KUBY_CONFIG.'
+    default_value './kuby.rb'
+    flag [:c, :config]
+
+    pre do |global_options, options, args|
+      Kuby.env = global_options[:environment] if global_options[:environment]
+      Kuby.load!(global_options[:config])
+
+      # GLI will abort unless this block returns a truthy value
+      true
+    end
+
+    # These are only stubs included to fill out the help screens. Rails
+    # commands are handled by the RailsCommands class.
+    desc 'Runs a Rails command.'
+    command :rails do |rc|
+      rc.desc 'Runs the rails server (run `rails server --help` for options)'
+      rc.command [:server, :s] do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          Kuby::RailsCommands.run(@rails_options)
+        end
+      end
+
+      rc.desc 'Runs a script in the Rails environment (run `rails runner --help` for options)'
+      rc.command [:runner, :r] do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          Kuby::RailsCommands.run(@rails_options)
+        end
+      end
+
+      rc.desc 'Starts an interactive Ruby console with the Rails environment loaded '\
+        '(run `rails console --help` for options)'
+      rc.command [:console, :c] do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          Kuby::RailsCommands.run(@rails_options)
+        end
+      end
+    end
+
+    desc 'Builds the Docker image.'
+    command :build do |c|
+      c.action do |global_options, options, args|
+        tasks.build
+      end
+    end
+
+    desc 'Pushes the Docker image to the configured registry.'
+    command :push do |c|
+      c.action do |global_options, options, args|
+        tasks.push
+      end
+    end
+
+    desc 'Gets your Kubernetes cluster ready to run your Rails app.'
+    command :setup do |c|
+      c.action do |global_options, options, args|
+        tasks.setup
+      end
+    end
+
+    desc 'Prints the effective Dockerfile used to build the Docker image.'
+    command :dockerfile do |c|
+      c.action do |global_options, options, args|
+        tasks.print_dockerfile
+      end
+    end
+
+    desc 'Deploys the application.'
+    command :deploy do |c|
+      c.desc 'The Docker tag to deploy. Defaults to the most recent tag.'
+      c.flag [:t, :tag], required: false
+      c.action do |global_options, options, args|
+        tasks.deploy(options[:tag])
+      end
+    end
+
+    desc 'Rolls back to the previous Docker tag.'
+    command :rollback do |c|
+      c.action do |global_options, options, args|
+        tasks.rollback
+      end
+    end
+
+    desc 'Prints the effective Kubernetes resources that will be applied on deploy.'
+    command :resources do |c|
+      c.action do |global_options, options, args|
+        tasks.print_resources
+      end
+    end
+
+    desc 'Prints out the contents of the kubeconfig Kuby is using to communicate with your cluster.'
+    command :kubeconfig do |c|
+      c.action do |global_options, options, args|
+        tasks.print_kubeconfig
+      end
+    end
+
+    desc 'Runs an arbitrary kubectl command.'
+    command :kubectl do |c|
+      c.desc 'Prefixes the kubectl command with the namespace associated with '\
+        'the current environment. For example, if the Kuby env is "production", '\
+        'this option will prefix the kubectl command with "-n myapp-production".'
+      c.switch [:N, :namespaced], default: false
+      c.action do |global_options, options, args|
+        if options[:namespaced]
+          # sorry Demeter
+          namespace = Kuby.definition.environment.kubernetes.namespace.metadata.name
+          args = ['-n', namespace, *args]
+        end
+
+        tasks.kubectl(*args)
+      end
+    end
+
+    desc 'Runs commands, etc against the Kubernetes cluster.'
+    command :remote do |rc|
+      rc.desc 'Tails (i.e. continuously streams) the Rails log from your running application.'
+      rc.command :logs do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_logs
+        end
+      end
+
+      rc.desc 'Lists running Kubernetes pods.'
+      rc.command :status do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_status
+        end
+      end
+
+      rc.desc 'Runs an arbitrary command inside a running Rails pod.'
+      rc.command :exec do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_exec(args)
+        end
+      end
+
+      rc.desc 'Establishes a shell inside a running Rails pod.'
+      rc.command :shell do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_shell
+        end
+      end
+
+      rc.desc 'Establishes a Rails console inside a running Rails pod.'
+      rc.command :console do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_console
+        end
+      end
+
+      rc.desc 'Establishes a database console inside a running Rails pod.'
+      rc.command :dbconsole do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_dbconsole
+        end
+      end
+
+      rc.desc "Restarts the Rails app's web pods."
+      rc.command :restart do |c|
+        c.action do |global_options, options, args|
+          exit 1 unless tasks.dev_deployment_ok
+          tasks.remote_restart
+        end
+      end
+    end
+  end
+end