vanagon 0.16.1 → 0.17.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 29954dadaa5567df164293efed2f042795e1cf1a54096cb066e2a898782800c9
-  data.tar.gz: cd8ee9e9c6e88c650449a062577ed10d7520642e94a73082e4fd0708b8ab2dfd
+  metadata.gz: e09a1563ed504da68687d85b7c7a70e62028b7d1ba24568476d5cae338ae1ebf
+  data.tar.gz: 9da1804bfbf40c86ac845ccf60a76c133b217357117dcd22d43d97c35299aa1d
 SHA512:
-  metadata.gz: f3dd4b2c8be84d868ef66e56aa31ab841b050a72bd6ee52cea7e55851b117ac6de2f638697aaa2b2fab0b34c1cb3b31d743d4759021e814878410855817c9144
-  data.tar.gz: cafda3c0792d436bc133fe7fb064b0c3d685a7112040ea4d46d893690e8e9d539a84a9eabf97059621e3d23a1e557df239f3d91f83a98f8be4fda7c278b18e6d
+  metadata.gz: 19591339a372853aa653311214de761ead6dfd173218d68c0086691ede75da8d981079408afaf04e5fd362e2c9250c72ac9b1cb8cc3fdbeb16006a6510707176
+  data.tar.gz: 74e995f55e05778820809bbed4c2dcd501896a8d4894e1a38921bb8057cee049d62c42e136e8929c84c9609790738d77be356ed55b499974b3399c6a032d724c

data/README.md

@@ -45,9 +45,10 @@ Also, Vanagon ships with a number of engines which may include additional option
 
 ### Local Host:
 
-- [Ruby](https://www.ruby-lang.org/en/) (Ruby 2.1.x is the miniumum supported version)
+- [Ruby](https://www.ruby-lang.org/en/) (Ruby 2.3.x is the miniumum supported version)
 - [fustigit](https://github.com/mckern/fustigit)
 - [ruby-git](https://github.com/schacon/ruby-git)
+- [docopt](https://github.com/docopt/docopt.rb)
 - The command line tool `ssh` ([homepage](https://www.openssh.com/)) available on the local `${PATH}` (any modern version should suffice)
 - The command line tool `rsync` ([homepage](https://rsync.samba.org/)) available on the local `${PATH}` (At least rsync 2.6.x)
 - The command line tool `git` ([homepage](https://git-scm.com/)) available on the local `${PATH}` (Vanagon is tested against Git version 1.8.x but should work with any newer version)
@@ -84,10 +85,28 @@ wheezy and build my project against it.
 For more detailed examples of the DSLs available, please see the
 [examples](examples) directory and the YARD documentation for Vanagon.
 
-### `build` usage
+### CLI changes and deprecations (from version 0.16.0)
+
+Prior to 0.16.0, the vanagon command line contained these commands
+
+* `build`
+* `build_host_info`
+* `build_requirements`
+* `inspect`
+* `render`
+* `repo`
+* `ship`
+* `sign`
+
+With the exception of `repo`, which calls `packaging` methods, the remaining commands
+have been moved to a git-like pattern of `vanagon <subcommand>`. `vangon build` replaced `build`, `vanagon ship` replaced `ship` and so forth.
+
+The older calling usage is deprecated.
+
+### `vanagon build` usage
 The build command has positional arguments and position independent flags.
 
-#### Arguments (position dependent)
+#### Positional arguments
 
 ##### project name
 The name of the project to build; a file named `<project_name>.rb` must be
@@ -114,7 +133,8 @@ Build machines should be cleaned between builds.
 
 #### Flagged arguments
 
-**Note:** command flags can be used anywhere in the command.
+**Note:** command flags currently can be used anywhere in the command. Recommended usages
+is to use flagged arguments before the positional arguments.
 
 ##### -w DIR, --workdir DIR
 Specifies a directory on the local host where the sources should be placed and builds performed.
@@ -134,9 +154,10 @@ Currently supported engines are:
 * `base` - Pure ssh backend; no teardown currently defined
 * `local` - Build on the local machine; platform name must match the local machine
 * `docker` - Builds in a docker container
-* `pooler` - Selects a vm from Puppet Labs' vm pooler to build on
+* `pooler` - Selects a vm from Puppet's internal vmpooler to build on
 * `hardware` - Build on a specific taget and lock it in redis
 * `ec2` - Build on a specific AWS instance.
+* `ABS` - Selects a vm from Puppet's internal Always-be-scheduling service to build on
 
 #### Flags
 
@@ -147,9 +168,6 @@ Indicates that the host used for building the project should be left intact
 after the build instead of destroyed. The host is usually destroyed after a
 successful build, or left after a failed build.
 
-##### -v, --verbose
-(Reserved for future implementation) Will increase the verbosity of output, when implemented.
-
 ##### -h, --help
 Display command-line help.
 
@@ -195,16 +213,16 @@ integer value these components to fail after the `VANAGON_TIMEOUT` count is reac
 Note that this value is expected to be in seconds.
 
 #### Example usage
-`build --preserve puppet-agent el-6-i386` will build the puppet-agent project
+`vanagon build --preserve puppet-agent el-6-i386` will build the puppet-agent project
 on the el-6-i386 platform and leave the host intact afterward.
 
-`build --engine=docker puppet-agent el-6-i386` will build the puppet-agent
+`vanagon build --engine=docker puppet-agent el-6-i386` will build the puppet-agent
 project on the el-6-i386 platform using the docker engine (the platform must
 have a docker\_image defined in its config).
 
 ---
 
-### `inspect` usage
+### `vanagon inspect` usage
 
 The `inspect` command has positional arguments and position independent flags. It
 mirrors the `build` command, but exits with success after loading and interpolating
@@ -212,7 +230,7 @@ all of the components in the given project. No attempt is made to actually build
 the given project; instead, a JSON formatted array of hashes is returned and printed
 to `stdout`. This JSON array can be further processed by external tooling, such as `jq`.
 
-#### Arguments (position dependent)
+#### Positional arguments
 
 ##### project name
 The name of the project to build, and a file named \<project\_name\>.rb must be
@@ -227,7 +245,8 @@ Platform can also be a comma separated list of platforms such as platform1,platf
 
 #### Flagged arguments
 
-**Note:** command flags can be used anywhere in the command.
+**Note:** command flags currently can be used anywhere in the command. Recommended usages
+is to use flagged arguments before the positional arguments.
 
 ##### -w DIR, --workdir DIR
 Specifies a directory where the sources should be placed and builds performed.
@@ -239,29 +258,24 @@ Specifies where project configuration is found. Defaults to $pwd/configs.
 ##### -e ENGINE, --engine ENGINE
 Choose a different virtualization engine to use to select the build target.
 Engines are respected, but only insofar as components and projects are
-rendered -- the `inspect` command performs no compilation.
+rendered -- the `vanagon inspect` command performs no compilation.
 
-Supported engines are the same as the `build` command.
+Supported engines are the same as the `vanagon build` command.
 
 #### Flags
 
-**Note:** command flags can be used anywhere in the command.
-
-##### -v, --verbose (not yet implemented)
-Increase verbosity of output.
-
 ##### -h, --help
 Display command-line help.
 
 #### Environment variables
 
 Environment variables are respected, but only insofar as components and projects are
-rendered -- the `inspect` command has no behavior to alter.
+rendered -- the `vanagon inspect` command has no behavior to alter.
 
-Supported environment variables are the same as the `build` command.
+Supported environment variables are the same as the `vanagon build` command.
 
 #### Example usage
-`inspect puppet-agent el-6-i386` will load the puppet-agent project
+`vanagon inspect puppet-agent el-6-i386` will load the puppet-agent project
 on the el-6-i386 platform and print the resulting list of dependencies,
 build-time configuration, environment variables, and expected artifacts.
 
@@ -297,6 +311,16 @@ platform "el-7-x86_64" do |plat|
 end
 ```
 
+### ABS (internal)
+When using the ABS engine, there is a variety of ways you can specify your token: 
+- the environment variable ABS_TOKEN
+- or vanagon token file ~/.vanagon-token (note this is the same file read by the pooler engine)
+- or [vmfloaty](https://github.com/puppetlabs/vmfloaty)'s config file ~/.vmfloaty.yml
+
+In order to modify or track the VM via floaty or bit-bar you can optionally add the vmpooler token (different from the ABS token) via
+- VMPOOLER_TOKEN
+- or as a second line to the file ~/.vanagon-token
+- or by using the existing mechanism in floaty using a vmpooler_fallback
 
 Contributing
 ---
@@ -310,3 +334,4 @@ See [LICENSE](LICENSE) file.
 
 ## Maintainers
 See [MAINTAINERS](MAINTAINERS) file.
+

data/lib/vanagon/engine/always_be_scheduling.rb

@@ -1,5 +1,6 @@
-require 'vanagon/engine/base'
 require 'json'
+require 'vanagon/engine/base'
+require 'yaml'
 
 class Vanagon
   class Engine
@@ -82,9 +83,15 @@ class Vanagon
     # {"name":"aix-53-ppc","engine":"always_be_scheduling"}
     # ```
     class AlwaysBeScheduling < Base
+      attr_reader :token
+      attr_reader :token_vmpooler
+
       def initialize(platform, target, **opts)
         super
 
+        @available_abs_endpoint = "https://abs-prod.k8s.infracore.puppet.net/api/v2"
+        @token_vmpooler = ENV['VMPOOLER_TOKEN']
+        @token = load_token
         Vanagon::Driver.logger.debug "AlwaysBeScheduling engine invoked."
       end
 
@@ -94,6 +101,7 @@ class Vanagon
       end
 
       # return the platform name as the "host" name
+      # order of preference: abs_resource_name, vmpooler_template or name
       def build_host_name
         if @platform.abs_resource_name
           @platform.abs_resource_name
@@ -103,6 +111,268 @@ class Vanagon
           @platform.name
         end
       end
+
+      # Retrieve the ABS token from an environment variable
+      # ("ABS_TOKEN") or from a number of potential configuration
+      # files (~/.vanagon-token or ~/.vmfloaty.yml).
+      # @return [String, nil] token for use with the vmpooler
+      def load_token
+        ENV['ABS_TOKEN'] || token_from_file
+      end
+
+      # a wrapper method around retrieving a token,
+      # with an explicitly ordered preference for a Vanagon-specific
+      # token file or a preexisting vmfoaty yaml file.
+      #
+      # @return [String, nil] token for use with the vmpooler
+      def token_from_file
+        read_vanagon_token || read_vmfloaty_token
+      end
+      private :token_from_file
+
+      # Read an ABS/vmpooler token from the plaintext vanagon-token file,
+      # as outlined in the project README.
+      # The first line should be the ABS token
+      # and the second (optional) line should be the vmpooler token
+      # Saves the second line into the @token_vmpooler variable
+      #
+      # @return [String, nil] the vanagon vmpooler token value
+      def read_vanagon_token(path = "~/.vanagon-token")
+        absolute_path = File.expand_path(path)
+        return nil unless File.exist?(absolute_path)
+
+        warn "Reading ABS token from: #{path}"
+        contents = File.read(absolute_path).chomp
+        lines = contents.each_line.map(&:chomp)
+
+        abs = lines.shift
+        @token_vmpooler = lines.shift
+
+        warn "Please add a second line with the vmpooler token to be able to modify or see the VM in floaty/bit-bar" if @token_vmpooler.nil?
+        return abs
+      end
+      private :read_vanagon_token
+
+      # Read a vmpooler token from the yaml formatted vmfloaty config,
+      # as outlined by the vmfloaty project:
+      # https://github.com/puppetlabs/vmfloaty
+      #
+      # It returns the top-level token value or the first 'abs' service
+      # token value it finds in the services list
+      # it sets @token_vmpooler with the optional vmpooler token
+      # if the abs services has a vmpooler_fallback, and that service
+      # has a token eg
+      #
+      # TOP-LEVEL DEFINED  => would get only the abs token
+      # user: 'jdoe'
+      # url: 'https://abs.example.net'
+      # token: '456def789'
+      #
+      # MULTIPLE SERVICES DEFINED => would get the abs token in 'abs-prod' and the vmpooler token in 'vmpooler-prod'
+      # user: 'jdoe'
+      # services:
+      #   abs-prod:
+      #     type: 'abs'
+      #     url: 'https://abs.example.net/api/v2'
+      #     token: '123abc456'
+      #     vmpooler_fallback: 'vmpooler-prod'
+      #   vmpooler-dev:
+      #     type: 'vmpooler'
+      #     url: 'https://vmpooler-dev.example.net'
+      #     token: '987dsa654'
+      #   vmpooler-prod:
+      #     type: 'vmpooler'
+      #     url: 'https://vmpooler.example.net'
+      #     token: '456def789'
+      #
+      # @return [String, nil] the vmfloaty vmpooler token value
+      def read_vmfloaty_token(path = "~/.vmfloaty.yml") # rubocop:disable Metrics/AbcSize, Metrics/PerceivedComplexity
+        absolute_path = File.expand_path(path)
+        return nil unless File.exist?(absolute_path)
+
+        yaml_config = YAML.load_file(absolute_path)
+        abs_token = nil
+        abs_service_name = nil
+        if yaml_config['token'] # top level
+          abs_token = yaml_config['token']
+        elsif yaml_config['services']
+          yaml_config['services'].each do |name, value|
+            if value['type'] == "abs" && value['token']
+              abs_token = value['token']
+              abs_service_name = name
+              vmpooler_fallback = value['vmpooler_fallback']
+              unless vmpooler_fallback.nil? || yaml_config['services'][vmpooler_fallback].nil? || yaml_config['services'][vmpooler_fallback]['token'].nil?
+                @token_vmpooler = yaml_config['services'][vmpooler_fallback]['token']
+              end
+              break
+            end
+          end
+        end
+        message = "Reading ABS token from: #{path}"
+        if abs_service_name
+          message.concat(" for service named #{abs_service_name}")
+          if @token_vmpooler.nil?
+            message.concat(" but there was no vmpooler_fallback value, please add one if you want to be able to modify the VM via bit-bar/floaty")
+          else
+            message.concat(" with a vmpooler_fallback token")
+          end
+        end
+        warn message
+        return abs_token
+      end
+      private :read_vmfloaty_token
+
+      # This method is used to obtain a vm to build upon using Puppet's internal
+      # ABS (https://github.com/puppetlabs/always-be-scheduling) which is a level of abstraction for other
+      # engines using similar APIs
+      # @raise [Vanagon::Error] if a target cannot be obtained
+      def select_target
+        @pooler = select_target_from(@available_abs_endpoint)
+        raise Vanagon::Error, "Something went wrong getting a target vm to build on" if @pooler.empty?
+      end
+
+      # Attempt to provision a host from a specific pooler.
+      def select_target_from(pooler) # rubocop:disable Metrics/AbcSize
+        request_object = build_request_object
+
+        warn "Requesting VMs with job_id: #{@saved_job_id}.  Will poll for up to an hour."
+        #the initial request is always replied with "come back again"
+        response = Vanagon::Utilities.http_request_generic(
+          "#{pooler}/request",
+          'POST',
+          request_object.to_json,
+          { 'X-AUTH-TOKEN' => @token }
+        )
+
+        unless response.code == "202"
+          warn "failed to request ABS with code #{response.code}"
+          if valid_json?(response.body)
+            response_json = JSON.parse(response.body)
+            warn "reason: #{response_json['reason']}"
+          end
+          return ''
+        end
+        response_body = check_queue(pooler, request_object)
+
+        return '' unless response_body["ok"]
+        @target = response_body[build_host_name]['hostname']
+        Vanagon::Driver.logger.info "Reserving #{@target} (#{build_host_name}) [#{@token ? 'token used' : 'no token used'}]"
+        return pooler
+      end
+
+      # main loop where the status of the request is checked, to see if the request
+      # has been allocated
+      def check_queue(pooler, request_object)
+        retries = 360 # ~ one hour
+        response_body = nil
+        begin
+          (1..retries).each do |i|
+            response = Vanagon::Utilities.http_request_generic(
+              "#{pooler}/request",
+              'POST',
+              request_object.to_json,
+              { 'X-AUTH-TOKEN' => @token }
+            )
+            response_body = validate_queue_status_response(response.code, response.body)
+            break if response_body
+
+            sleep_seconds = 10 if i >= 10
+            sleep_seconds = i if i < 10
+            warn "Waiting #{sleep_seconds} seconds to check if ABS request has been filled. (x#{i})"
+
+            sleep(sleep_seconds)
+          end
+        rescue SystemExit, Interrupt
+          warn "\nVanagon interrupted during mains ABS polling. Make sure you delete the requested job_id #{@saved_job_id}"
+          raise
+        end
+        response_body = translated(response_body, @saved_job_id)
+        response_body
+      end
+
+      def validate_queue_status_response(status_code, body)
+        case status_code
+        when "200"
+          return JSON.parse(body) unless body.empty? || !valid_json?(body)
+        when "202"
+          return nil
+        when "401"
+          raise Vanagon::Error, "HTTP #{status_code}: The token provided could not authenticate.\n#{body}"
+        when "503"
+          return nil
+        else
+          raise Vanagon::Error, "HTTP #{status_code}: request to ABS failed!\n#{body}"
+        end
+      end
+
+      # This method is used to tell the ABS to delete the job_id requested
+      # otherwise the resources will eventually get allocated asynchronously
+      # and will keep running until the end of their lifetime.
+      def teardown # rubocop:disable Metrics/AbcSize
+        request_object = {
+            'job_id' => @saved_job_id,
+        }
+
+        response = Vanagon::Utilities.http_request_generic(
+          "#{@available_abs_endpoint}/return",
+          "POST",
+          request_object.to_json,
+          { 'X-AUTH-TOKEN' => @token }
+        )
+        if response && response.body == 'OK'
+          Vanagon::Driver.logger.info "#{@saved_job_id} has been scheduled for removal"
+          warn "#{@saved_job_id} has been scheduled for removal"
+        else
+          Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal: #{response.body}"
+          warn "#{@saved_job_id} could not be scheduled for removal"
+        end
+      rescue Vanagon::Error => e
+        Vanagon::Driver.logger.info "#{@saved_job_id} could not be scheduled for removal (#{e.message})"
+        warn "#{@saved_job_id} could not be scheduled for removal (#{e.message})"
+      end
+
+      private
+
+      def translated(response_body, job_id)
+        vmpooler_formatted_body = { 'job_id' => job_id }
+
+        response_body.each do |host| # in this context there should be only one host
+          vmpooler_formatted_body[host['type']] = { 'hostname' => host['hostname'] }
+        end
+        vmpooler_formatted_body['ok'] = true
+
+        vmpooler_formatted_body
+      end
+
+      def build_request_object
+        user = ENV['USER'] || ENV['USERNAME'] || 'unknown'
+
+        @saved_job_id = user + "-" + DateTime.now.strftime('%Q')
+        request_object = {
+            :resources => { build_host_name => 1 },
+            :job       => {
+                :id   => @saved_job_id,
+                :tags => {
+                    :jenkins_build_url => ENV['BUILD_URL'],
+                    :project           => ENV['JOB_NAME'] || 'vanagon',
+                    :created_by        => user,
+                    :user              => user
+                },
+            },
+            :priority => 1, # DO NOT use priority 1 in automated CI runs
+        }
+        unless @token_vmpooler.nil?
+          request_object[:vm_token] = @token_vmpooler
+        end
+        request_object
+      end
+
+      def valid_json?(json)
+        JSON.parse(json)
+        return true
+      rescue TypeError, JSON::ParserError
+        return false
+      end
     end
   end
 end

data/lib/vanagon/engine/pooler.rb

@@ -1,6 +1,10 @@
 require 'vanagon/engine/base'
 require 'yaml'
 
+### Note this class is deprecated in favor of using the ABS Engine. The pooler has changed it's API with regards to
+# getting VMs for ondemand provisioning and would need to be updated here.
+# See DIO-1066
+
 class Vanagon
   class Engine
     class Pooler < Base
@@ -10,7 +14,7 @@ class Vanagon
       def initialize(platform, target = nil, **opts)
         super
 
-        @available_poolers = ["http://vmpooler.delivery.puppetlabs.net", "https://nspooler-service-prod-1.delivery.puppetlabs.net"]
+        @available_poolers = ["https://vmpooler.delivery.puppetlabs.net", "https://nspooler-service-prod-1.delivery.puppetlabs.net"]
         @token = load_token
         @required_attributes << "vmpooler_template"
       end
@@ -63,7 +67,7 @@ class Vanagon
 
       # Read a vmpooler token from the yaml formatted vmfloaty config,
       # as outlined by the vmfloaty project:
-      # https://github.com/briancain/vmfloaty
+      # https://github.com/puppetlabs/vmfloaty
       #
       # @return [String, nil] the vmfloaty vmpooler token value
       def read_vmfloaty_token(path = "~/.vmfloaty.yml")
@@ -75,7 +79,7 @@ class Vanagon
       end
       private :read_vmfloaty_token
 
-      # This method is used to obtain a vm to build upon using the Puppet Labs'
+      # This method is used to obtain a vm to build upon using Puppet's internal
       # vmpooler (https://github.com/puppetlabs/vmpooler) or other pooler technologies
       # leveraging the same API
       # @raise [Vanagon::Error] if a target cannot be obtained

data/lib/vanagon/utilities.rb

@@ -39,17 +39,16 @@ class Vanagon
     end
 
     # Simple wrapper around Net::HTTP. Will make a request of the given type to
-    # the given url and return the body as parsed by JSON.
+    # the given url and return the response object
     #
     # @param url [String] The url to make the request against (needs to be parsable by URI
     # @param type [String] One of the supported request types (currently 'get', 'post', 'delete')
     # @param payload [String] The request body data payload used for POST and PUT
     # @param header [Hash] Send additional information in the HTTP request header
-    # @return [Hash] The response body is parsed by JSON and returned
+    # @return [Net::HTTPAccepted] The response object
     # @raise [RuntimeError, Vanagon::Error] an exception is raised if the
-    # action is not supported, or if there is a problem with the http request,
-    # or if the response is not JSON
-    def http_request(url, type, payload = {}.to_json, header = nil) # rubocop:disable Metrics/AbcSize
+    # action is not supported, or if there is a problem with the http request
+    def http_request_generic(url, type, payload = {}.to_json, header = nil) # rubocop:disable Metrics/AbcSize
       uri = URI.parse(url)
       http = Net::HTTP.new(uri.host, uri.port)
       http.use_ssl = true if uri.scheme == 'https'
@@ -77,14 +76,37 @@ class Vanagon
       end
 
       response = http.request(request)
-
-      JSON.parse(response.body)
+      response
     rescue Errno::ETIMEDOUT, Timeout::Error, Errno::EINVAL, Errno::ECONNRESET,
       EOFError, Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError,
       Net::ProtocolError => e
       raise Vanagon::Error.wrap(e, "Problem reaching #{url}. Is #{uri.host} down?")
+    end
+
+    # uses http_request_generic and returns the body as parsed by JSON.
+    # @param url [String] The url to make the request against (needs to be parsable by URI
+    # @param type [String] One of the supported request types (currently 'get', 'post', 'delete')
+    # @param payload [String] The request body data payload used for POST and PUT
+    # @param header [Hash] Send additional information in the HTTP request header
+    # @return [Hash] The response in JSON format
+    # @raise [RuntimeError, Vanagon::Error] an exception is raised if the response
+    # body cannot be parsed as JSON
+    def http_request(url, type, payload = {}.to_json, header = nil)
+      response = http_request_generic(url, type, payload, header)
+      JSON.parse(response.body)
     rescue JSON::ParserError => e
-      raise Vanagon::Error.wrap(e, "#{uri.host} handed us a response that doesn't look like JSON.")
+      raise Vanagon::Error.wrap(e, "#{url} handed us a response that doesn't look like JSON.")
+    end
+
+    # uses http_request_generic and returns the response code.
+    # @param url [String] The url to make the request against (needs to be parsable by URI
+    # @param type [String] One of the supported request types (currently 'get', 'post', 'delete')
+    # @param payload [String] The request body data payload used for POST and PUT
+    # @param header [Hash] Send additional information in the HTTP request header
+    # @return [String] The response code eg 202, 200 etc
+    def http_request_code(url, type, payload = {}.to_json, header = nil)
+      response = http_request_generic(url, type, payload, header)
+      response.code
     end
 
     # Similar to rake's sh, the passed command will be executed and an

data/spec/lib/vanagon/engine/always_be_scheduling_spec.rb

@@ -2,6 +2,7 @@ require 'vanagon/engine/always_be_scheduling'
 require 'vanagon/driver'
 require 'vanagon/platform'
 require 'logger'
+require 'spec_helper'
 
 class Vanagon
   class Driver
@@ -45,6 +46,8 @@ describe 'Vanagon::Engine::AlwaysBeScheduling' do
                     end")
     plat._platform
   }
+  let(:pooler_token_file) { File.expand_path('~/.vanagon-token') }
+  let(:floaty_config) { File.expand_path('~/.vmfloaty.yml') }
 
   describe '#validate_platform' do
     it 'returns true if the platform has the required attributes' do
@@ -81,4 +84,113 @@ describe 'Vanagon::Engine::AlwaysBeScheduling' do
         .to eq('always_be_scheduling')
     end
   end
-end
+
+  describe '#read_vanagon_token' do
+    it 'takes the first line for abs token, second line is optional' do
+      token_value = 'decade'
+      allow(File).to receive(:exist?)
+                         .with(pooler_token_file)
+                         .and_return(true)
+
+      allow(File).to receive(:read)
+                         .with(pooler_token_file)
+                         .and_return(token_value)
+
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      expect(abs_service.token).to eq('decade')
+      expect(abs_service.token_vmpooler).to eq(nil)
+    end
+    it 'takes the second line as vmpooler token' do
+      token_value = "decade\nanddaycade"
+      allow(File).to receive(:exist?)
+                         .with(pooler_token_file)
+                         .and_return(true)
+
+      allow(File).to receive(:read)
+                         .with(pooler_token_file)
+                         .and_return(token_value)
+
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      expect(abs_service.token).to eq('decade')
+      expect(abs_service.token_vmpooler).to eq('anddaycade')
+    end
+  end
+
+  describe '#read_vmfloaty_token' do
+    before :each do
+      allow(File).to receive(:exist?)
+                         .with(pooler_token_file)
+                         .and_return(false)
+
+      allow(File).to receive(:exist?)
+                         .with(floaty_config)
+                         .and_return(true)
+    end
+    token_value = 'decade'
+    it %(reads a token from '~/.vmfloaty.yml at the top level') do
+      allow(YAML).to receive(:load_file)
+                         .with(floaty_config)
+                         .and_return({'token' => token_value})
+
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      expect(abs_service.token).to eq(token_value)
+      expect(abs_service.token_vmpooler).to eq(nil)
+    end
+    it %(reads a token from '~/.vmfloaty.yml in the abs service') do
+      allow(YAML).to receive(:load_file)
+                         .with(floaty_config)
+                         .and_return({'services' =>
+                                          {'MYabs' => {'type'=>'abs', 'token'=>token_value, 'url'=>'foo'}}
+                                     })
+
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      expect(abs_service.token).to eq(token_value)
+      expect(abs_service.token_vmpooler).to eq(nil)
+    end
+    it %(reads a token from '~/.vmfloaty.yml in the abs service and includes the vmpooler token') do
+      vmp_token_value = 'deecade'
+      allow(YAML).to receive(:load_file)
+                         .with(floaty_config)
+                         .and_return({'services' =>
+                                          {'MYabs' => {'type'=>'abs', 'token'=>token_value, 'url'=>'foo', 'vmpooler_fallback' => 'myvmp'},
+                                           'myvmp' => {'token'=>vmp_token_value, 'url'=>'bar'}}
+                                     })
+
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      expect(abs_service.token).to eq(token_value)
+      expect(abs_service.token_vmpooler).to eq(vmp_token_value)
+    end
+  end
+  describe '#select_target_from' do
+    it 'runs successfully' do
+      hostname = 'faint-whirlwind.puppet.com'
+      stub_request(:post, "https://foobar/request").
+          to_return({status: 202, body: "", headers: {}},{status: 200, body: '[{"hostname":"'+hostname+'","type":"aix-6.1-ppc","engine":"nspooler"}]', headers: {}})
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      abs_service.select_target_from("https://foobar")
+      expect(abs_service.target).to eq(hostname)
+    end
+    it 'returns a warning if the first request is not a 202' do
+      hostname = 'fainter-whirlwind.puppet.com'
+      stub_request(:post, "https://foobar/request").
+          to_return({status: 404, body: "", headers: {}},{status: 200, body: '[{"hostname":"'+hostname+'","type":"aix-6.1-ppc","engine":"nspooler"}]', headers: {}})
+      allow_any_instance_of(Object).to receive(:warn)
+      expect_any_instance_of(Object).to receive(:warn).with("failed to request ABS with code 404")
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      pooler = abs_service.select_target_from("https://foobar")
+      expect(pooler).to eq('')
+    end
+    it 'returns a warning and retries until request is a 200' do
+      hostname = 'faintest-whirlwind.puppet.com'
+      stub_request(:post, "https://foobar/request").
+          to_return({status: 202, body: "", headers: {}},
+                    {status: 503, body: "", headers: {}},
+                    {status: 200, body: '[{"hostname":"'+hostname+'","type":"aix-6.1-ppc","engine":"nspooler"}]', headers: {}})
+      allow_any_instance_of(Object).to receive(:warn)
+      expect_any_instance_of(Object).to receive(:warn).with(/Waiting 1 seconds to check if ABS request has been filled/)
+      abs_service = Vanagon::Engine::AlwaysBeScheduling.new(platform, nil)
+      abs_service.select_target_from("https://foobar")
+      expect(abs_service.target).to eq(hostname)
+    end
+  end
+end

data/spec/lib/vanagon/engine/ec2_spec.rb

@@ -27,6 +27,8 @@ if defined? ::Aws
     end
 
     it 'returns "ec2" name' do
+      stub_request(:get, "http://169.254.169.254/latest/meta-data/iam/security-credentials/").
+          to_return(status: 200, body: "", headers: {})
       expect(Vanagon::Engine::Ec2.new(platform_ec2).name).to eq('ec2')
     end
   end

data/spec/lib/vanagon/engine/pooler_spec.rb

@@ -6,7 +6,7 @@ describe 'Vanagon::Engine::Pooler' do
   let (:platform_with_vcloud_name) {
     plat = Vanagon::Platform::DSL.new('debian-6-i386')
     plat.instance_eval("platform 'debian-6-i386' do |plat|
-                      plat.vcloud_name 'debian-6-i386'
+                      plat.vmpooler_template 'debian-6-i386'
                     end")
     plat._platform
   }

data/spec/spec_helper.rb

@@ -19,6 +19,7 @@ end
 
 require 'tmpdir'
 require 'vanagon'
+require 'webmock/rspec'
 
 RSpec.configure do |c|
   c.before do

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: vanagon
 version: !ruby/object:Gem::Version
-  version: 0.16.1
+  version: 0.17.0
 platform: ruby
 authors:
 - Puppet Labs
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-09-19 00:00:00.000000000 Z
+date: 2020-11-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: docopt
@@ -288,41 +288,41 @@ signing_key:
 specification_version: 3
 summary: All of your packages will fit into this van with this one simple trick.
 test_files:
+- spec/lib/vanagon/project/dsl_spec.rb
+- spec/lib/vanagon/extensions/string_spec.rb
+- spec/lib/vanagon/extensions/ostruct/json_spec.rb
+- spec/lib/vanagon/extensions/set/json_spec.rb
+- spec/lib/vanagon/platform/solaris_11_spec.rb
+- spec/lib/vanagon/platform/dsl_spec.rb
 - spec/lib/vanagon/platform/rpm/aix_spec.rb
-- spec/lib/vanagon/platform/osx_spec.rb
 - spec/lib/vanagon/platform/windows_spec.rb
+- spec/lib/vanagon/platform/osx_spec.rb
+- spec/lib/vanagon/platform/solaris_10_spec.rb
 - spec/lib/vanagon/platform/rpm_spec.rb
 - spec/lib/vanagon/platform/deb_spec.rb
-- spec/lib/vanagon/platform/solaris_11_spec.rb
-- spec/lib/vanagon/platform/solaris_10_spec.rb
-- spec/lib/vanagon/platform/dsl_spec.rb
-- spec/lib/vanagon/driver_spec.rb
-- spec/lib/vanagon/project/dsl_spec.rb
-- spec/lib/vanagon/utilities_spec.rb
-- spec/lib/vanagon/cli_spec.rb
-- spec/lib/vanagon/environment_spec.rb
-- spec/lib/vanagon/component/rules_spec.rb
-- spec/lib/vanagon/component/source_spec.rb
-- spec/lib/vanagon/component/source/local_spec.rb
+- spec/lib/vanagon/project_spec.rb
+- spec/lib/vanagon/component/source/rewrite_spec.rb
 - spec/lib/vanagon/component/source/git_spec.rb
+- spec/lib/vanagon/component/source/local_spec.rb
 - spec/lib/vanagon/component/source/http_spec.rb
-- spec/lib/vanagon/component/source/rewrite_spec.rb
+- spec/lib/vanagon/component/rules_spec.rb
 - spec/lib/vanagon/component/dsl_spec.rb
-- spec/lib/vanagon/utilities/shell_utilities_spec.rb
-- spec/lib/vanagon/extensions/set/json_spec.rb
-- spec/lib/vanagon/extensions/string_spec.rb
-- spec/lib/vanagon/extensions/ostruct/json_spec.rb
+- spec/lib/vanagon/component/source_spec.rb
+- spec/lib/vanagon/common/user_spec.rb
+- spec/lib/vanagon/common/pathname_spec.rb
 - spec/lib/vanagon/component_spec.rb
-- spec/lib/vanagon/platform_spec.rb
-- spec/lib/vanagon/engine/local_spec.rb
 - spec/lib/vanagon/engine/hardware_spec.rb
-- spec/lib/vanagon/engine/always_be_scheduling_spec.rb
-- spec/lib/vanagon/engine/pooler_spec.rb
-- spec/lib/vanagon/engine/base_spec.rb
 - spec/lib/vanagon/engine/ec2_spec.rb
+- spec/lib/vanagon/engine/local_spec.rb
 - spec/lib/vanagon/engine/docker_spec.rb
-- spec/lib/vanagon/common/user_spec.rb
-- spec/lib/vanagon/common/pathname_spec.rb
-- spec/lib/vanagon/project_spec.rb
+- spec/lib/vanagon/engine/pooler_spec.rb
+- spec/lib/vanagon/engine/base_spec.rb
+- spec/lib/vanagon/engine/always_be_scheduling_spec.rb
+- spec/lib/vanagon/environment_spec.rb
+- spec/lib/vanagon/cli_spec.rb
+- spec/lib/vanagon/driver_spec.rb
+- spec/lib/vanagon/utilities_spec.rb
+- spec/lib/vanagon/platform_spec.rb
+- spec/lib/vanagon/utilities/shell_utilities_spec.rb
 - spec/lib/makefile_spec.rb
 - spec/lib/git/rev_list_spec.rb