resque-kubernetes 0.10.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 15ceb41cf5857885f0b6777411e1f0980a4007bc
-  data.tar.gz: b1a5f893856a48ac7fdbd0059ebef225e938d474
+  metadata.gz: dd653a29a38d097f457584f8bb8a4b305dc14f1f
+  data.tar.gz: 4ac673bc617fdd91c99fe3b51f0c943f214655f6
 SHA512:
-  metadata.gz: 41c6a804868eb7e64ea25ee33fd08a63ea48756893417837a34440d6c5c01ead8e0eb8467d94888b94ac968047913eda67a0af06133e8aebb1585c80ab69e5e2
-  data.tar.gz: 1ef4222a4ec9b3508a5237125da66dfc64982a2891530feadbc4d289a5dfe7c747f756fde24bb4f4bdf7e4e4a0a03ed5756c30f3a2b13f83c2ea1207a2766dda
+  metadata.gz: 142a38eea17c227b6339eef837b58e4835d3fdf05d7f76f85f30cde00c25f6d20b3ee1fa59795b030549283bfcae28eb49da8159449bd3004066c258c5a7b4d0
+  data.tar.gz: 707c7a76de380e87d8da7c4f4651d924f9b7992d5bdedcf48150a0ce8c225cda21d9f252f4a4b907dae373c5c012be2f9ec449bb69d90501a711607f2161b3e2

data/.gitignore

@@ -1,6 +1,7 @@
-/.bundle/
+**/.bundle/
 /.yardoc
 /Gemfile.lock
+**/*.gemfile.lock
 /_yardoc/
 /coverage/
 /doc/

data/Appraisals

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+appraise "kubeclient-3" do
+  gem "kubeclient", "3.1.2"
+end
+
+appraise "kubeclient-4" do
+  gem "kubeclient", "4.0.0"
+end

data/CHANGELOG.md

@@ -1,3 +1,18 @@
+# v1.1.0
+- Fix design to set `INTERVAL=0` for the worker in the Kubernetes
+  job manifest, which tells `resque` to work until there are no more jobs,
+  rather than monkey-patching `Resque::Worker` to look for `TERM_ON_EMPTY`
+  environment variable.
+
+# v1.0.0
+**Breaking Change:**
+- Requires `kubeclient` 3.1.2 or 4.x
+
+**Changes:**
+- Add `kubeclient` configuration option for connecting to any Kubernetes server
+- Use kubernetes namespace provided by cluster or `kubectl` configuration when available
+- Add Appraisal for testing with kubeclient 3.1.2 and 4.x
+
 # v0.10.0
 - `kubeclient` may not be later than 3.0.0 due to change in signature of `Kubeclient::Config::Context#initialize`
   in `kubeclient` 3.1.0

data/README.md

@@ -7,8 +7,8 @@ the container finishes and then it terminates the pod (as opposed to trying to
 restart the container).
 
 This gem takes advantage of that feature by starting up a Kubernetes Job to
-run a worker when a Resque job or ActiveJob is enqueued. It then allows the
-Resque worker to be modified to terminate when there are no more jobs in the queue.
+run a worker when a Resque job or ActiveJob is enqueued. It then tells the
+Resque worker to run until there are no more jobs in the queue.
 
 Why would you do this?
 
@@ -122,15 +122,18 @@ class ResourceIntensiveJob < ApplicationJob
   end
 end
 ```
-
 ### Workers (for both)
 
-Make sure that the container image above, which is used to run the resque 
-worker, is built to include the `resque-kubernetes` gem as well. The gem will 
-add `TERM_ON_EMPTY` to the environment variables. This tells the worker that 
-whenever the queue is empty it should terminate the worker. Kubernetes will 
-then terminate the Job when the container is done running and will release the 
-resources.
+The resque worker can can be any container image that runs the `resque:work` `rake` task, for example:
+
+```bash
+bin/rails environment resque:work
+```
+
+The gem sets the environment variable `INTERVAL=0` for the Kubernetes Job which the `rake` task uses
+to when calling `Resque::Worker#work(interval)`. The value 0 tells Resque to terminate when the queue
+is empty. If your Docker image does not run the rake task, then you'll need to make sure you pass 0
+for the interval when calling `Resque::Worker#work`.
 
 ### Job manifest
 
@@ -200,13 +203,28 @@ class ResourceIntensiveJob
 end
 ```
 
-## To Do
+### kubeclient
+
+The gem will automatically connect to the Kubernetes server in the following cases:
+- You are running this in [a standard Kubernetes cluster](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/#accessing-the-api-from-a-pod)
+- You are running on a system with `kubeclient` installed and
+  - the default cluster context has credentials
+  - the default cluster is GKE and your system has 
+    [Google application default credentials](https://developers.google.com/identity/protocols/application-default-credentials)
+    installed
+
+There are many other ways to connect and you can do so by providing your own
+[configured `kubeclient`](https://github.com/abonas/kubeclient#usage):
+
+```ruby
+# config/initializers/resque-kubernetes.rb
+
+Resque::Kubernetes.configuration do |config|
+ config.kubeclient = Kubeclient::Client.new("http://localhost:8080/apis/batch")
+end
+```
 
-- Support for other authentication and server URL options for `kubeclient`.
-  See [the many examples](https://github.com/abonas/kubeclient#usage) in their
-  README.
-- We probably need better namespace support, particularly for reaping
-  finished jobs and pods.
+Because this uses the `Job` resource, make sure to connect to the `/apis/batch` API endpoint in your client.
 
 ## Contributing
 
@@ -231,7 +249,7 @@ experiment.
 
 Write test for any code that you add. Test all changes by running `rake`.
 This does the following, which you can also run separately while working.
-1. Tun unit tests: `rake spec`
+1. Run unit tests: `appraisal rake spec`
 2. Make sure that your code matches the styles: `rubocop`
 3. Verify if any dependent gems have open CVEs (you must update these):
    `rake bundle:audit` 
@@ -239,10 +257,10 @@ This does the following, which you can also run separately while working.
 ### End to End Tests
 
 We don't run End to End (e2e) tests in the regular suite because
-they require a connection to a cluster. You can run these on your changes
-if you want to verify that the jobs are created correctly.
+they require a connection to a cluster. You should run these on your changes
+to verify that the jobs are created correctly.
 
-This will use the default authentication on your system, which may is either
+This will use the default authentication on your system, which is either
 the cluster the tests are running in (if you are doing that), your `kubclient`
 configuration, or your Google Default Application Credentials.
 

data/Rakefile

@@ -1,16 +1,22 @@
 # frozen_string_literal: true
 
+require "appraisal/task"
+require "bundler/audit/task"
 require "bundler/gem_tasks"
 require "rspec/core/rake_task"
 require "rubocop/rake_task"
-require "bundler/audit/task"
 
 RSpec::Core::RakeTask.new(:spec)
 RuboCop::RakeTask.new
 Bundler::Audit::Task.new
+Appraisal::Task.new
 
 # Remove default and replace with a series of test tasks
 task default: []
 Rake::Task[:default].clear
 
-task default: %w[spec rubocop bundle:audit]
+if ENV["APPRAISAL_INITIALIZED"]
+  task default: %i[spec]
+else
+  task default: %i[rubocop bundle:audit appraisal]
+end

data/gemfiles/kubeclient_3.gemfile

@@ -0,0 +1,7 @@
+# This file was generated by Appraisal
+
+source "https://rubygems.org"
+
+gem "kubeclient", "3.1.2"
+
+gemspec path: "../"

data/gemfiles/kubeclient_4.gemfile

@@ -0,0 +1,7 @@
+# This file was generated by Appraisal
+
+source "https://rubygems.org"
+
+gem "kubeclient", "4.0.0"
+
+gemspec path: "../"

data/lib/resque/kubernetes/context/kubectl.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module Resque
+  module Kubernetes
+    module Context
+      # Kubeclient Context from `kubectl` config file.
+      class Kubectl
+        def applicable?
+          File.exist?(kubeconfig)
+        end
+
+        def context
+          config = Kubeclient::Config.read(kubeconfig)
+
+          Resque::Kubernetes::ContextFactory::Context.new(
+              config.context.api_endpoint,
+              config.context.api_version,
+              config.context.namespace,
+              auth_options: auth_options(config),
+              ssl_options:  config.context.ssl_options
+          )
+        end
+
+        private
+
+        def kubeconfig
+          File.join(ENV["HOME"], ".kube", "config")
+        end
+
+        def auth_options(config)
+          options = config.context.auth_options
+          return options unless options.empty?
+          google_application_default_credentials
+        end
+
+        def google_application_default_credentials
+          return unless defined?(Google) && defined?(Google::Auth)
+          {bearer_token: Kubeclient::GoogleApplicationDefaultCredentials.token}
+        end
+      end
+    end
+  end
+end

data/lib/resque/kubernetes/context/well_known.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+module Resque
+  module Kubernetes
+    module Context
+      # Kubeclient Context from well-known locations within a Kubernetes cluster.
+      class WellKnown
+        TOKEN_FILE     = "/var/run/secrets/kubernetes.io/serviceaccount/token"
+        CA_FILE        = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
+        NAMESPACE_FILE = "/var/run/secrets/kubernetes.io/serviceaccount/namespace"
+
+        def applicable?
+          File.exist?(TOKEN_FILE)
+        end
+
+        def context
+          Resque::Kubernetes::ContextFactory::Context.new(
+              "https://kubernetes.default.svc",
+              "v1",
+              namespace,
+              auth_options: {bearer_token_file: TOKEN_FILE},
+              ssl_options:  ssl_options
+          )
+        end
+
+        private
+
+        def namespace
+          return nil unless File.exist?(NAMESPACE_FILE)
+          File.read(NAMESPACE_FILE)
+        end
+
+        def ssl_options
+          return {} unless File.exist?(CA_FILE)
+          {ca_file: CA_FILE}
+        end
+      end
+    end
+  end
+end

data/lib/resque/kubernetes/context_factory.rb

@@ -6,64 +6,19 @@ module Resque
   module Kubernetes
     # Create a context for `Kubeclient` depending on the environment.
     class ContextFactory
+      Context = Struct.new(:endpoint, :version, :namespace, :options)
+
       class << self
         def context
           # TODO: Add ability to load this from config
-
-          if File.exist?("/var/run/secrets/kubernetes.io/serviceaccount/token")
-            # When running in GKE/k8s cluster, use the service account secret token and ca bundle
-            well_known_context
-          elsif File.exist?(kubeconfig)
-            # When running in development, use the config file for `kubectl` and default application credentials
-            kubectl_context
+          [
+              Resque::Kubernetes::Context::WellKnown,
+              Resque::Kubernetes::Context::Kubectl
+          ].each do |context_type|
+            context = context_type.new
+            return context.context if context.applicable?
           end
         end
-
-        private
-
-        def well_known_context
-          Kubeclient::Config::Context.new(
-              "https://kubernetes",
-              "v1",
-              {ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"},
-              bearer_token_file: "/var/run/secrets/kubernetes.io/serviceaccount/token"
-          )
-        end
-
-        def kubectl_context
-          config = Kubeclient::Config.read(kubeconfig)
-          auth_options = config.context.auth_options
-
-          auth_options = google_default_application_credentials(config) if auth_options.empty?
-
-          Kubeclient::Config::Context.new(
-              config.context.api_endpoint,
-              config.context.api_version,
-              config.context.ssl_options,
-              auth_options
-          )
-        end
-
-        def kubeconfig
-          File.join(ENV["HOME"], ".kube", "config")
-        end
-
-        # TODO: Move this logic to kubeclient. See abonas/kubeclient#213
-        def google_default_application_credentials(config)
-          return unless defined?(Google) && defined?(Google::Auth)
-
-          _cluster, user = config.send(:fetch_context, config.instance_variable_get(:@kcfg)["current-context"])
-          return {} unless user["auth-provider"] && user["auth-provider"]["name"] == "gcp"
-
-          {bearer_token: new_google_token}
-        end
-
-        def new_google_token
-          scopes = ["https://www.googleapis.com/auth/cloud-platform"]
-          authorization = Google::Auth.get_application_default(scopes)
-          authorization.apply({})
-          authorization.access_token
-        end
       end
     end
   end

data/lib/resque/kubernetes/jobs_manager.rb

@@ -10,7 +10,8 @@ module Resque
       private :owner
 
       def initialize(owner)
-        @owner = owner
+        @owner             = owner
+        @default_namespace = "default"
       end
 
       def reap_finished_jobs
@@ -39,20 +40,17 @@ module Resque
       private
 
       def jobs_client
-        return @jobs_client if @jobs_client
-        @jobs_client = client("/apis/batch")
+        @jobs_client ||= client("/apis/batch")
       end
 
       def client(scope)
+        return Resque::Kubernetes.kubeclient if Resque::Kubernetes.kubeclient
+
         context = ContextFactory.context
         return unless context
+        @default_namespace = context.namespace if context.namespace
 
-        Kubeclient::Client.new(
-            context.api_endpoint + scope,
-            context.api_version,
-            ssl_options:  context.ssl_options,
-            auth_options: context.auth_options
-        )
+        Kubeclient::Client.new(context.endpoint + scope, context.version, context.options)
       end
 
       def finished_jobs
@@ -92,12 +90,12 @@ module Resque
 
       def container_term_on_empty(container)
         container["env"] ||= []
-        term_on_empty = container["env"].find { |env| env["name"] == "TERM_ON_EMPTY" }
+        term_on_empty = container["env"].find { |env| env["name"] == "INTERVAL" }
         unless term_on_empty
-          term_on_empty = {"name" => "TERM_ON_EMPTY"}
+          term_on_empty = {"name" => "INTERVAL"}
           container["env"] << term_on_empty
         end
-        term_on_empty["value"] = "1"
+        term_on_empty["value"] = "0"
       end
 
       def ensure_reset_policy(manifest)
@@ -105,7 +103,7 @@ module Resque
       end
 
       def ensure_namespace(manifest)
-        manifest["metadata"]["namespace"] ||= "default"
+        manifest["metadata"]["namespace"] ||= @default_namespace
       end
 
       def update_job_name(manifest)

data/lib/resque/kubernetes/version.rb

@@ -2,6 +2,6 @@
 
 module Resque
   module Kubernetes
-    VERSION = "0.10.0"
+    VERSION = "1.1.0"
   end
 end

data/lib/resque/kubernetes.rb

@@ -1,13 +1,14 @@
 # frozen_string_literal: true
 
 require "resque/kubernetes/configurable"
+require "resque/kubernetes/context/kubectl"
+require "resque/kubernetes/context/well_known"
 require "resque/kubernetes/context_factory"
 require "resque/kubernetes/deep_hash"
 require "resque/kubernetes/dns_safe_random"
 require "resque/kubernetes/job"
 require "resque/kubernetes/jobs_manager"
 require "resque/kubernetes/version"
-require "resque/kubernetes/worker"
 
 module Resque
   # Run Resque Jobs as Kubernetes Jobs with autoscaling.
@@ -19,7 +20,8 @@ module Resque
 
     # Limit the number of workers that should be spun up, default 10
     define_setting :max_workers, 10
+
+    # A `kubeclient` for connection context, default attempts to read from cluster or `~/.kube/config`
+    define_setting :kubeclient, nil
   end
 end
-
-Resque::Worker.include Resque::Kubernetes::Worker

data/resque-kubernetes.gemspec

@@ -23,6 +23,7 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
+  spec.add_development_dependency "appraisal"
   spec.add_development_dependency "bundler", "~> 1.16"
   spec.add_development_dependency "bundler-audit", "~> 0"
   spec.add_development_dependency "googleauth", "~> 0.6"
@@ -30,6 +31,6 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "rspec", "~> 3.7"
   spec.add_development_dependency "rubocop", "~> 0.52", ">= 0.52.1"
 
-  spec.add_dependency "kubeclient", ">= 2.2", "<= 3.0.0"
+  spec.add_dependency "kubeclient", ">= 3.1.2", "< 5.0"
   spec.add_dependency "resque", "~> 1.26"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: resque-kubernetes
 version: !ruby/object:Gem::Version
-  version: 0.10.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Jeremy Wadsack
@@ -10,6 +10,20 @@ bindir: exe
 cert_chain: []
 date: 2018-09-27 00:00:00.000000000 Z
 dependencies:
+- !ruby/object:Gem::Dependency
+  name: appraisal
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -106,20 +120,20 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '2.2'
-    - - "<="
+        version: 3.1.2
+    - - "<"
       - !ruby/object:Gem::Version
-        version: 3.0.0
+        version: '5.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '2.2'
-    - - "<="
+        version: 3.1.2
+    - - "<"
       - !ruby/object:Gem::Version
-        version: 3.0.0
+        version: '5.0'
 - !ruby/object:Gem::Dependency
   name: resque
   requirement: !ruby/object:Gem::Requirement
@@ -147,6 +161,7 @@ files:
 - ".rubocop.yml"
 - ".ruby-gemset"
 - ".ruby-version"
+- Appraisals
 - CHANGELOG.md
 - Gemfile
 - LICENSE.txt
@@ -154,15 +169,18 @@ files:
 - Rakefile
 - bin/console
 - bin/setup
+- gemfiles/kubeclient_3.gemfile
+- gemfiles/kubeclient_4.gemfile
 - lib/resque/kubernetes.rb
 - lib/resque/kubernetes/configurable.rb
+- lib/resque/kubernetes/context/kubectl.rb
+- lib/resque/kubernetes/context/well_known.rb
 - lib/resque/kubernetes/context_factory.rb
 - lib/resque/kubernetes/deep_hash.rb
 - lib/resque/kubernetes/dns_safe_random.rb
 - lib/resque/kubernetes/job.rb
 - lib/resque/kubernetes/jobs_manager.rb
 - lib/resque/kubernetes/version.rb
-- lib/resque/kubernetes/worker.rb
 - resque-kubernetes.gemspec
 homepage: https://github.com/keylimetoolbox/resque-kubernetes
 licenses:

data/lib/resque/kubernetes/worker.rb

@@ -1,52 +0,0 @@
-# frozen_string_literal: true
-
-require "resque"
-
-module Resque
-  module Kubernetes
-    # Patches the resque worker to terminate when the queue is empty.
-    #
-    # This patch enables setting an environment variable, `TERM_ON_EMPTY`
-    # that causes the worker to shutdown when the queue no longer has any
-    # resque jobs. This allows running workers as Kuberenetes Jobs that will
-    # terminate when there is no longer any work to do.
-    #
-    # To use, make sure that the container images that hold your workers are
-    # built to include the `resque-kubernetes` gem and set `TERM_ON_EMPTY` in
-    # their environment to a truthy value (e.g. "1").
-    module Worker
-      def self.included(base)
-        base.class_eval do
-          prepend InstanceMethods
-        end
-      end
-
-      attr_accessor :term_on_empty
-
-      # Replace methods on the worker instance
-      module InstanceMethods
-        def prepare
-          self.term_on_empty = ENV["TERM_ON_EMPTY"] if ENV["TERM_ON_EMPTY"]
-          super
-        end
-
-        def shutdown?
-          if term_on_empty
-            if queues_empty?
-              log_with_severity :info, "shutdown: queues are empty"
-              shutdown
-            end
-          end
-
-          super
-        end
-      end
-
-      private
-
-      def queues_empty?
-        queues.all? { |queue| Resque.size(queue).zero? }
-      end
-    end
-  end
-end