k8s-client-renewed 0.10.5.pre.1

data/README.md

@@ -0,0 +1,233 @@
+# K8s::Client
+
+[![Build Status](https://travis-ci.com/kontena/k8s-client.svg?branch=master)](https://travis-ci.com/kontena/k8s-client)
+[![Gem Version](https://badge.fury.io/rb/k8s-client.svg)](https://badge.fury.io/rb/k8s-client)
+[![Yard Docs](http://img.shields.io/badge/yard-docs-blue.svg)](http://www.rubydoc.info/github/kontena/k8s-client/master)
+
+
+Ruby client library for the Kubernetes (1.9+) API
+
+## Highlights
+
+* Clean API for dynamic Kubernetes API Groups / Resources
+* Fast API requests using HTTP connection keepalive
+* Fast API discovery and resource listings using pipelined HTTP requests
+* Typed errors with useful debugging information
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'k8s-client'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install k8s-client
+
+And then load the code using:
+
+```ruby
+require 'k8s-client'
+```
+
+To use the [yajl-ruby](https://github.com/brianmario/yajl-ruby) JSON parser backend, add `yalj-ruby` to your Gemfile:
+
+```ruby
+gem 'yajl-ruby'
+gem 'k8s-client'
+```
+
+And then load the code using:
+
+```ruby
+require 'k8s-client'
+require 'k8s/json_parser/yajl'
+```
+
+## Usage
+
+### Overview
+The top-level `K8s::Client` provides access to separate `APIClient` instances for each Kubernetes API Group (`v1`, `apps/v1`, etc.), which in turns provides access to separate `ResourceClient` instances for each API resource type (`nodes`, `pods`, `deployments`, etc.).
+
+Individual resources are returned as `K8s::Resource` instances, which provide attribute access (`resource.metadata.name`). The resource instances are returned by methods such as `client.api('v1').resource('nodes').get('foo')`, and passed as arguments for `client.api('v1').resource('nodes').create_resource(res)`. Resources can also be loaded from disk using `K8s::Resource.from_files(path)`, and passed to the top-level methods such as `client.create_resource(res)`, which lookup the correct API/Resource client from the resource `apiVersion` and `kind`.
+
+The different `K8s::Error::API` subclasses represent different HTTP response codes, such as `K8s::Error::NotFound` or `K8s::Error::Conflict`.
+
+### Creating a client
+
+#### Unauthenticated client
+
+```ruby
+client = K8s.client('https://localhost:6443', ssl_verify_peer: false)
+```
+
+The keyword options are [Excon](https://github.com/excon/excon/) options.
+
+#### Client from kubeconfig
+
+```ruby
+client = K8s::Client.config(
+  K8s::Config.load_file(
+    File.expand_path '~/.kube/config'
+  )
+)
+```
+
+#### Supported kubeconfig options
+
+Not all kubeconfig options are supported, only the following kubeconfig options work:
+
+* `current-context`
+* `context.cluster`
+* `context.user`
+* `cluster.server`
+* `cluster.insecure_skip_tls_verify`
+* `cluster.certificate_authority`
+* `cluster.certificate_authority_data`
+* `user.client_certificate` + `user.client_key`
+* `user.client_certificate_data` + `user.client_key_data`
+* `user.token`
+
+##### With overrides
+
+```ruby
+client = K8s::Client.config(K8s::Config.load_file('~/.kube/config'),
+  server: 'http://localhost:8001',
+)
+```
+
+#### In-cluster client from pod envs/secrets
+
+```ruby
+client = K8s::Client.in_cluster_config
+```
+
+### Logging
+
+#### Quiet
+
+To supress any warning messages:
+
+```ruby
+K8s::Logging.quiet!
+K8s::Transport.quiet!
+```
+
+The `K8s::Transport` is quiet by default, but other components may log warnings in the future.
+
+#### Debugging
+
+Log all API requests
+
+```ruby
+K8s::Logging.debug!
+K8s::Transport.verbose!
+```
+
+```
+I, [2018-08-09T14:19:50.404739 #1]  INFO -- K8s::Transport: Using config with server=https://167.99.39.233:6443
+I, [2018-08-09T14:19:50.629521 #1]  INFO -- K8s::Transport<https://167.99.39.233:6443>: GET /version => HTTP 200: <K8s::Resource> in 0.224s
+I, [2018-08-09T14:19:50.681367 #1]  INFO -- K8s::Transport<https://167.99.39.233:6443>: GET /api/v1 => HTTP 200: <K8s::Resource> in 0.046s
+I, [2018-08-09T14:19:51.018740 #1]  INFO -- K8s::Transport<https://167.99.39.233:6443>: GET /api/v1/pods => HTTP 200: <K8s::Resource> in 0.316s
+```
+
+Using `K8s::Transport.debug!` will also log request/response bodies. The `EXCON_DEBUG=true` env will log all request/response attributes, including headers.
+
+### Prefetching API resources
+
+Operations like mapping a resource `kind` to an API resource URL require knowledge of the API resource lists for the API group. Mapping resources for multiple API groups would require fetching the API resource lists for each API group in turn, leading to additional request latency. This can be optimized using resource prefetching:
+
+```ruby
+client.apis(prefetch_resources: true)
+```
+
+This will fetch the API resource lists for all API groups in a single pipelined request.
+
+### Listing resources
+
+```ruby
+client.api('v1').resource('pods', namespace: 'default').list(labelSelector: {'role' => 'test'}).each do |pod|
+  puts "namespace=#{pod.metadata.namespace} pod: #{pod.metadata.name} node=#{pod.spec.nodeName}"
+end
+```
+
+### Updating resources
+
+```ruby
+node = client.api('v1').resource('nodes').get('test-node')
+
+node[:spec][:unschedulable] = true
+
+client.api('v1').resource('nodes').update_resource(node)
+```
+
+### Deleting resources
+
+```ruby
+pod = client.api('v1').resource('pods', namespace: 'default').delete('test-pod')
+```
+
+```ruby
+pods = client.api('v1').resource('pods', namespace: 'default').delete_collection(labelSelector: {'role' => 'test'})
+```
+
+### Creating resources
+
+#### Programmatically defined resources
+```ruby
+service = K8s::Resource.new(
+  apiVersion: 'v1',
+  kind: 'Service',
+  metadata: {
+    namespace: 'default',
+    name: 'test',
+  },
+  spec: {
+    type: 'ClusterIP',
+    ports: [
+      { port: 80 },
+    ],
+    selector: {'app' => 'test'},
+  },
+)
+
+logger.info "Create service=#{service.metadata.name} in namespace=#{service.metadata.namespace}"
+
+service = client.api('v1').resource('services').create_resource(service)
+```
+
+#### From file(s)
+
+```ruby
+resources = K8s::Resource.from_files('./test.yaml')
+
+for resource in resources
+  resource = client.create_resource(resource)
+end
+```
+
+### Patching resources
+
+```ruby
+client.api('apps/v1').resource('deployments', namespace: 'default').merge_patch('test', {
+    spec: { replicas: 3 },
+})
+```
+
+### Watching resources
+
+```ruby
+client.api('v1').resource('pods', namespace: 'default').watch(labelSelector: {'role' => 'test'}) do |watch_event|
+  puts "type=#{watch_event.type} pod=#{watch_event.resource.metadata.name}"
+end
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/kontena/k8s-client.

data/Rakefile

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+require 'rubocop/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+RuboCop::RakeTask.new(:rubocop).tap do |task|
+  task.options = %w(--fail-level A -S --format c)
+  if RUBY_ENGINE == 'ruby' && RbConfig::CONFIG['host_os'] !~ /mswin|msys|mingw|cygwin|bccwin|wince|emc/
+    task.options << '--parallel'
+  end
+end
+
+task default: [:spec, :rubocop]

data/docker-compose.yaml

@@ -0,0 +1,10 @@
+rspec:
+  build: .
+  volumes:
+  - .:/app
+  entrypoint: bundle exec rspec
+k8s-client:
+  build: .
+  volumes:
+  - .:/app
+  entrypoint: bundle exec bin/k8s-client

data/k8s-client.gemspec

@@ -0,0 +1,39 @@
+
+lib = File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "k8s/client/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "k8s-client-renewed"
+  spec.version       = K8s::Client::VERSION
+  spec.authors       = ["Kontena, Inc."]
+  spec.email         = ["info@kontena.io"]
+  spec.license       = "Apache-2.0"
+
+  spec.summary       = "Kubernetes client library"
+  spec.homepage      = "https://github.com/kontena/k8s-client"
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  end
+  spec.bindir        = "bin"
+  spec.executables   = []
+  spec.require_paths = ["lib"]
+#  spec.required_ruby_version = '> 2.4'
+
+  spec.add_runtime_dependency "excon"#, "~> 0.66"
+  spec.add_runtime_dependency "recursive-open-struct"#, "~> 1.1.0"
+  spec.add_runtime_dependency 'hashdiff'#, '~> 1.0.0'
+  spec.add_runtime_dependency 'jsonpath'#, '~> 0.9.5'
+  spec.add_runtime_dependency "yaml-safe_load_stream-renewed"#, "~> 0.1"
+
+#  spec.add_development_dependency "bundler"#, ">= 1.17", "< 3.0"
+  spec.add_development_dependency "byebug"#, "~> 10.0"
+  spec.add_development_dependency "rake"#, "~> 10.0"
+  spec.add_development_dependency "rspec"#, "~> 3.7"
+  spec.add_development_dependency "webmock"#, "~> 3.6.2"
+  spec.add_development_dependency "rubocop"#, "~> 0.59"
+  spec.add_development_dependency 'yajl-ruby'#, '~> 1.4.0'
+end

data/lib/k8s-client.rb

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require_relative 'k8s/client'

data/lib/k8s/api_client.rb

@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+
+module K8s
+  # Per-APIGroup/version client.
+  #
+  # Offers access to {ResourceClient} instances for the APIResource types defined in this apigroup/version
+  class APIClient
+    # @param api_version [String] either core version (v1) or apigroup/apiversion (apps/v1)
+    # @return [String]
+    def self.path(api_version)
+      if api_version.include? '/'
+        File.join('/apis', api_version)
+      else
+        File.join('/api', api_version)
+      end
+    end
+
+    # @param transport [K8s::Transport]
+    # @param api_version [String] "group/version" or "version" (core)
+    def initialize(transport, api_version)
+      @transport = transport
+      @api_version = api_version
+    end
+
+    # @return [String]
+    attr_reader :api_version
+
+    # @param path [Array<String>] join path from parts
+    # @return [String]
+    def path(*path)
+      @transport.path(self.class.path(@api_version), *path)
+    end
+
+    # @return [Bool] loaded yet?
+    def api_resources?
+      !!@api_resources
+    end
+
+    # @param api_resources [Array<K8s::Resource>]
+    attr_writer :api_resources
+
+    # Force-update APIResources
+    #
+    # @return [Array<K8s::Resource>]
+    def api_resources!
+      @api_resources = @transport.get(path).resources
+    end
+
+    # Cached APIResources
+    #
+    # @return [Array<K8s::Resource>]
+    def api_resources
+      @api_resources || api_resources!
+    end
+
+    # @param resource_name [String]
+    # @raise [K8s::Error::UndefinedResource]
+    # @return [K8s::Resource]
+    def find_api_resource(resource_name)
+      found_resource = api_resources.find{ |api_resource| api_resource.name == resource_name }
+      found_resource ||= api_resources!.find{ |api_resource| api_resource.name == resource_name }
+      raise K8s::Error::UndefinedResource, "Unknown resource #{resource_name} for #{@api_version}" unless found_resource
+
+      found_resource
+    end
+
+    # @param resource_name [String]
+    # @param namespace [String, nil]
+    # @raise [K8s::Error::UndefinedResource]
+    # @return [K8s::ResourceClient]
+    def resource(resource_name, namespace: nil)
+      ResourceClient.new(@transport, self, find_api_resource(resource_name), namespace: namespace)
+    end
+
+    # @param resource [K8s::Resource]
+    # @param namespace [String, nil] default if resource is missing namespace
+    # @raise [K8s::Error::UndefinedResource]
+    # @return [K8s::ResourceClient]
+    def client_for_resource(resource, namespace: nil)
+      unless @api_version == resource.apiVersion
+        raise K8s::Error::UndefinedResource, "Invalid apiVersion=#{resource.apiVersion} for #{@api_version} client"
+      end
+
+      found_resource = api_resources.find{ |api_resource| api_resource.kind == resource.kind }
+      found_resource ||= api_resources!.find{ |api_resource| api_resource.kind == resource.kind }
+      raise K8s::Error::UndefinedResource, "Unknown resource kind=#{resource.kind} for #{@api_version}" unless found_resource
+
+      ResourceClient.new(@transport, self, found_resource, namespace: resource.metadata.namespace || namespace)
+    end
+
+    # TODO: skip non-namespaced resources if namespace is given, or ignore namespace?
+    #
+    # @param namespace [String, nil]
+    # @return [Array<K8s::ResourceClient>]
+    def resources(namespace: nil)
+      api_resources.map{ |api_resource|
+        ResourceClient.new(@transport, self, api_resource,
+                           namespace: namespace)
+      }
+    end
+
+    # Pipeline list requests for multiple resource types.
+    #
+    # Returns flattened array with mixed resource kinds.
+    #
+    # @param resources [Array<K8s::ResourceClient>] default is all listable resources for api
+    # @param options @see [K8s::ResourceClient#list]
+    # @return [Array<K8s::Resource>]
+    def list_resources(resources = nil, **options)
+      resources ||= self.resources.select(&:list?)
+
+      ResourceClient.list(resources, @transport, **options)
+    end
+  end
+end

data/lib/k8s/client.rb

@@ -0,0 +1,282 @@
+# frozen_string_literal: true
+
+require 'openssl'
+require 'base64'
+
+require 'monitor'
+require 'uri'
+
+require 'k8s/json_parser'
+require 'k8s/util'
+
+require 'k8s/config'
+require 'k8s/logging'
+require 'k8s/api_client'
+require "k8s/error"
+require 'k8s/resource'
+require 'k8s/resource_client'
+require 'k8s/stack'
+require 'k8s/transport'
+
+module K8s
+  # @param server [String] http/s URL
+  # @param options [Hash] @see Transport.new
+  # @return [K8s::Client]
+  def self.client(server, **options)
+    Client.new(Transport.new(server, **options))
+  end
+
+  # Top-level client wrapper.
+  # Uses a {Transport} instance to talk to the kube API.
+  # Offers access to {APIClient} and {ResourceClient} instances.
+  class Client
+    # @param config [Phraos::Kube::Config]
+    # @param namespace [String] @see #initialize
+    # @param options [Hash] @see Transport.config
+    # @return [K8s::Client]
+    def self.config(config, namespace: nil, **options)
+      new(
+        Transport.config(config, **options),
+        namespace: namespace
+      )
+    end
+
+    # An K8s::Client instance from in-cluster config within a kube pod, using the kubernetes service envs and serviceaccount secrets
+    # @see K8s::Transport#in_cluster_config
+    #
+    # @param namespace [String] default namespace for all operations
+    # @param options [Hash] options passed to transport, @see Transport#in_cluster_config
+    # @return [K8s::Client]
+    # @raise [K8s::Error::Config,Errno::ENOENT,Errno::EACCES]
+    def self.in_cluster_config(namespace: nil, **options)
+      new(Transport.in_cluster_config(**options), namespace: namespace)
+    end
+
+    # Attempts to create a K8s::Client instance automatically using environment variables, existing configuration
+    # files or in cluster configuration.
+    #
+    # Look-up order:
+    #   - KUBE_TOKEN, KUBE_CA, KUBE_SERVER environment variables
+    #   - KUBECONFIG environment variable
+    #   - $HOME/.kube/config file
+    #   - In cluster configuration
+    #
+    # Will raise when no means of configuration is available
+    #
+    # @param options [Hash] default namespace for all operations
+    # @raise [K8s::Error::Config,Errno::ENOENT,Errno::EACCES]
+    # @return [K8s::Client]
+    def self.autoconfig(namespace: nil, **options)
+      if ENV.values_at('KUBE_TOKEN', 'KUBE_CA', 'KUBE_SERVER').none? { |v| v.nil? || v.empty? }
+        unless Base64.decode64(ENV['KUBE_CA']).match?(/CERTIFICATE/)
+          raise ArgumentError, 'KUBE_CA does not seem to be base64 encoded'
+        end
+
+        begin
+          token = options[:auth_token] || Base64.strict_decode64(ENV['KUBE_TOKEN'])
+        rescue ArgumentError
+          raise ArgumentError, 'KUBE_TOKEN does not seem to be base64 encoded'
+        end
+
+        configuration = K8s::Config.build(server: ENV['KUBE_SERVER'], ca: ENV['KUBE_CA'], auth_token: token)
+      elsif !ENV['KUBECONFIG'].to_s.empty?
+        configuration = K8s::Config.from_kubeconfig_env(ENV['KUBECONFIG'])
+      else
+        found_config = [
+          File.join(Dir.home, '.kube', 'config'),
+          '/etc/kubernetes/admin.conf',
+          '/etc/kubernetes/kubelet.conf'
+        ].find { |f| File.exist?(f) && File.readable?(f) }
+
+        configuration = K8s::Config.load_file(found_config) if found_config
+      end
+
+      if configuration
+        config(configuration, namespace: namespace, **options)
+      else
+        in_cluster_config(namespace: namespace, **options)
+      end
+    end
+
+    include MonitorMixin
+
+    attr_reader :transport
+
+    # @param transport [K8s::Transport]
+    # @param namespace [String, nil] default namespace for all operations
+    def initialize(transport, namespace: nil)
+      @transport = transport
+      @namespace = namespace
+
+      @api_clients = {}
+      super()
+    end
+
+    # @raise [K8s::Error]
+    # @return [K8s::Resource]
+    def version
+      @version ||= @transport.version
+    end
+
+    # @param api_version [String] "group/version" or "version" (core)
+    # @return [APIClient]
+    def api(api_version = 'v1')
+      @api_clients[api_version] ||= APIClient.new(@transport, api_version)
+    end
+
+    # Force-update /apis cache.
+    # Required if creating new CRDs/apiservices.
+    #
+    # @return [Array<String>]
+    def api_groups!
+      synchronize do
+        @api_groups = @transport.get(
+          '/apis',
+          response_class: K8s::Resource
+        ).groups.flat_map{ |api_group| api_group.versions.map(&:groupVersion) }
+
+        @api_clients.clear
+      end
+
+      @api_groups
+    end
+
+    # Cached /apis preferred group apiVersions
+    # @return [Array<String>]
+    def api_groups
+      @api_groups || api_groups!
+    end
+
+    # @param api_versions [Array<String>] defaults to all APIs
+    # @param prefetch_resources [Boolean] prefetch any missing api_resources for each api_version
+    # @param skip_missing [Boolean] return APIClient without api_resources? if 404
+    # @return [Array<APIClient>]
+    def apis(api_versions = nil, prefetch_resources: false, skip_missing: false)
+      api_versions ||= ['v1'] + api_groups
+
+      if prefetch_resources
+        # api groups that are missing their api_resources
+        api_paths = api_versions
+                    .uniq
+                    .reject{ |api_version| api(api_version).api_resources? }
+                    .map{ |api_version| APIClient.path(api_version) }
+
+        # load into APIClient.api_resources=
+        begin
+          @transport.gets(*api_paths, skip_missing: skip_missing).each do |api_resource_list|
+            api(api_resource_list.groupVersion).api_resources = api_resource_list.resources if api_resource_list
+          end
+        rescue K8s::Error::NotFound, K8s::Error::ServiceUnavailable # rubocop:disable Lint/HandleExceptions
+          # kubernetes api is in unstable state
+          # because this is only performance optimization, better to skip prefetch and move on
+        end
+      end
+
+      api_versions.map{ |api_version| api(api_version) }
+    end
+
+    # @param namespace [String, nil]
+    # @return [Array<K8s::ResourceClient>]
+    def resources(namespace: nil)
+      apis(prefetch_resources: true).map { |api|
+        begin
+          api.resources(namespace: namespace)
+        rescue K8s::Error::ServiceUnavailable, K8s::Error::NotFound
+          []
+        end
+      }.flatten
+    end
+
+    # Pipeline list requests for multiple resource types.
+    #
+    # Returns flattened array with mixed resource kinds.
+    #
+    # @param resources [Array<K8s::ResourceClient>] default is all listable resources for api
+    # @param options @see K8s::ResourceClient#list
+    # @return [Array<K8s::Resource>]
+    def list_resources(resources = nil, **options)
+      cached_clients = @api_clients.size.positive?
+      resources ||= self.resources.select(&:list?)
+
+      begin
+        ResourceClient.list(resources, @transport, **options)
+      rescue K8s::Error::NotFound
+        raise unless cached_clients
+
+        cached_clients = false
+        api_groups!
+        retry
+      end
+    end
+
+    # @param resource [K8s::Resource]
+    # @param namespace [String, nil] default if resource is missing namespace
+    # @raise [K8s::Error::NotFound] API Group does not exist
+    # @raise [K8s::Error::UndefinedResource]
+    # @return [K8s::ResourceClient]
+    def client_for_resource(resource, namespace: nil)
+      api(resource.apiVersion).client_for_resource(resource, namespace: namespace)
+    end
+
+    # @param resource [K8s::Resource]
+    # @return [K8s::Resource]
+    def create_resource(resource)
+      client_for_resource(resource).create_resource(resource)
+    end
+
+    # @param resource [K8s::Resource]
+    # @return [K8s::Resource]
+    def get_resource(resource)
+      client_for_resource(resource).get_resource(resource)
+    end
+
+    # Returns nils for any resources that do not exist.
+    # This includes custom resources that were not yet defined.
+    #
+    # @param resources [Array<K8s::Resource>]
+    # @return [Array<K8s::Resource, nil>] matching resources array 1:1
+    def get_resources(resources)
+      # prefetch api resources, skip missing APIs
+      resource_apis = apis(resources.map(&:apiVersion), prefetch_resources: true, skip_missing: true)
+
+      # map each resource to excon request options, or nil if resource is not (yet) defined
+      requests = resources.zip(resource_apis).map{ |resource, api_client|
+        next nil unless api_client.api_resources?
+
+        resource_client = api_client.client_for_resource(resource)
+
+        {
+          method: 'GET',
+          path: resource_client.path(resource.metadata.name, namespace: resource.metadata.namespace),
+          response_class: resource_client.resource_class
+        }
+      }
+
+      # map non-nil requests to response objects, or nil for nil request options
+      Util.compact_map(requests) { |reqs|
+        @transport.requests(*reqs, skip_missing: true)
+      }
+    end
+
+    # @param resource [K8s::Resource]
+    # @return [K8s::Resource]
+    def update_resource(resource)
+      client_for_resource(resource).update_resource(resource)
+    end
+
+    # @param resource [K8s::Resource]
+    # @param options [Hash]
+    # @see ResourceClient#delete for options
+    # @return [K8s::Resource]
+    def delete_resource(resource, **options)
+      client_for_resource(resource).delete_resource(resource, **options)
+    end
+
+    # @param resource [K8s::Resource]
+    # @param attrs [Hash]
+    # @return [K8s::Client]
+    def patch_resource(resource, attrs)
+      client_for_resource(resource).json_patch(resource.metadata.name, attrs)
+    end
+  end
+end