vanagon 0.15.36 → 0.17.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3da7ab177127a9c8735a304f2abbbbcbfa2dc2244b389e3cc4c3c3e440c66e17
-  data.tar.gz: c8b21d1f75c57e10598907a8ca2f5c937f2d96a041d6ad5876f578ab67dca96a
+  metadata.gz: e09a1563ed504da68687d85b7c7a70e62028b7d1ba24568476d5cae338ae1ebf
+  data.tar.gz: 9da1804bfbf40c86ac845ccf60a76c133b217357117dcd22d43d97c35299aa1d
 SHA512:
-  metadata.gz: 3a3d603bacbc6f4588ae915cc9bd36dfcceba404ef75a4da4fb12429956f3aa5def4ab3b7db99eaa88865eda2870bec3d691f882eab1a2e2727a2a203f3b931b
-  data.tar.gz: 54b68ba89cbfe3dcc0d800c248f743258f4da9000696d79687cbe2da38543f02b0cf83dd4f214eaa1ecc90015b4dc5b27a69e64496d2d5100686507b4ce6c8a7
+  metadata.gz: 19591339a372853aa653311214de761ead6dfd173218d68c0086691ede75da8d981079408afaf04e5fd362e2c9250c72ac9b1cb8cc3fdbeb16006a6510707176
+  data.tar.gz: 74e995f55e05778820809bbed4c2dcd501896a8d4894e1a38921bb8057cee049d62c42e136e8929c84c9609790738d77be356ed55b499974b3399c6a032d724c

data/README.md

@@ -45,9 +45,10 @@ Also, Vanagon ships with a number of engines which may include additional option
 
 ### Local Host:
 
-- [Ruby](https://www.ruby-lang.org/en/) (Ruby 2.1.x is the miniumum supported version)
+- [Ruby](https://www.ruby-lang.org/en/) (Ruby 2.3.x is the miniumum supported version)
 - [fustigit](https://github.com/mckern/fustigit)
 - [ruby-git](https://github.com/schacon/ruby-git)
+- [docopt](https://github.com/docopt/docopt.rb)
 - The command line tool `ssh` ([homepage](https://www.openssh.com/)) available on the local `${PATH}` (any modern version should suffice)
 - The command line tool `rsync` ([homepage](https://rsync.samba.org/)) available on the local `${PATH}` (At least rsync 2.6.x)
 - The command line tool `git` ([homepage](https://git-scm.com/)) available on the local `${PATH}` (Vanagon is tested against Git version 1.8.x but should work with any newer version)
@@ -84,10 +85,28 @@ wheezy and build my project against it.
 For more detailed examples of the DSLs available, please see the
 [examples](examples) directory and the YARD documentation for Vanagon.
 
-### `build` usage
+### CLI changes and deprecations (from version 0.16.0)
+
+Prior to 0.16.0, the vanagon command line contained these commands
+
+* `build`
+* `build_host_info`
+* `build_requirements`
+* `inspect`
+* `render`
+* `repo`
+* `ship`
+* `sign`
+
+With the exception of `repo`, which calls `packaging` methods, the remaining commands
+have been moved to a git-like pattern of `vanagon <subcommand>`. `vangon build` replaced `build`, `vanagon ship` replaced `ship` and so forth.
+
+The older calling usage is deprecated.
+
+### `vanagon build` usage
 The build command has positional arguments and position independent flags.
 
-#### Arguments (position dependent)
+#### Positional arguments
 
 ##### project name
 The name of the project to build; a file named `<project_name>.rb` must be
@@ -114,7 +133,8 @@ Build machines should be cleaned between builds.
 
 #### Flagged arguments
 
-**Note:** command flags can be used anywhere in the command.
+**Note:** command flags currently can be used anywhere in the command. Recommended usages
+is to use flagged arguments before the positional arguments.
 
 ##### -w DIR, --workdir DIR
 Specifies a directory on the local host where the sources should be placed and builds performed.
@@ -134,9 +154,10 @@ Currently supported engines are:
 * `base` - Pure ssh backend; no teardown currently defined
 * `local` - Build on the local machine; platform name must match the local machine
 * `docker` - Builds in a docker container
-* `pooler` - Selects a vm from Puppet Labs' vm pooler to build on
+* `pooler` - Selects a vm from Puppet's internal vmpooler to build on
 * `hardware` - Build on a specific taget and lock it in redis
 * `ec2` - Build on a specific AWS instance.
+* `ABS` - Selects a vm from Puppet's internal Always-be-scheduling service to build on
 
 #### Flags
 
@@ -147,9 +168,6 @@ Indicates that the host used for building the project should be left intact
 after the build instead of destroyed. The host is usually destroyed after a
 successful build, or left after a failed build.
 
-##### -v, --verbose
-(Reserved for future implementation) Will increase the verbosity of output, when implemented.
-
 ##### -h, --help
 Display command-line help.
 
@@ -195,16 +213,16 @@ integer value these components to fail after the `VANAGON_TIMEOUT` count is reac
 Note that this value is expected to be in seconds.
 
 #### Example usage
-`build --preserve puppet-agent el-6-i386` will build the puppet-agent project
+`vanagon build --preserve puppet-agent el-6-i386` will build the puppet-agent project
 on the el-6-i386 platform and leave the host intact afterward.
 
-`build --engine=docker puppet-agent el-6-i386` will build the puppet-agent
+`vanagon build --engine=docker puppet-agent el-6-i386` will build the puppet-agent
 project on the el-6-i386 platform using the docker engine (the platform must
 have a docker\_image defined in its config).
 
 ---
 
-### `inspect` usage
+### `vanagon inspect` usage
 
 The `inspect` command has positional arguments and position independent flags. It
 mirrors the `build` command, but exits with success after loading and interpolating
@@ -212,7 +230,7 @@ all of the components in the given project. No attempt is made to actually build
 the given project; instead, a JSON formatted array of hashes is returned and printed
 to `stdout`. This JSON array can be further processed by external tooling, such as `jq`.
 
-#### Arguments (position dependent)
+#### Positional arguments
 
 ##### project name
 The name of the project to build, and a file named \<project\_name\>.rb must be
@@ -227,7 +245,8 @@ Platform can also be a comma separated list of platforms such as platform1,platf
 
 #### Flagged arguments
 
-**Note:** command flags can be used anywhere in the command.
+**Note:** command flags currently can be used anywhere in the command. Recommended usages
+is to use flagged arguments before the positional arguments.
 
 ##### -w DIR, --workdir DIR
 Specifies a directory where the sources should be placed and builds performed.
@@ -239,29 +258,24 @@ Specifies where project configuration is found. Defaults to $pwd/configs.
 ##### -e ENGINE, --engine ENGINE
 Choose a different virtualization engine to use to select the build target.
 Engines are respected, but only insofar as components and projects are
-rendered -- the `inspect` command performs no compilation.
+rendered -- the `vanagon inspect` command performs no compilation.
 
-Supported engines are the same as the `build` command.
+Supported engines are the same as the `vanagon build` command.
 
 #### Flags
 
-**Note:** command flags can be used anywhere in the command.
-
-##### -v, --verbose (not yet implemented)
-Increase verbosity of output.
-
 ##### -h, --help
 Display command-line help.
 
 #### Environment variables
 
 Environment variables are respected, but only insofar as components and projects are
-rendered -- the `inspect` command has no behavior to alter.
+rendered -- the `vanagon inspect` command has no behavior to alter.
 
-Supported environment variables are the same as the `build` command.
+Supported environment variables are the same as the `vanagon build` command.
 
 #### Example usage
-`inspect puppet-agent el-6-i386` will load the puppet-agent project
+`vanagon inspect puppet-agent el-6-i386` will load the puppet-agent project
 on the el-6-i386 platform and print the resulting list of dependencies,
 build-time configuration, environment variables, and expected artifacts.
 
@@ -297,6 +311,16 @@ platform "el-7-x86_64" do |plat|
 end
 ```
 
+### ABS (internal)
+When using the ABS engine, there is a variety of ways you can specify your token: 
+- the environment variable ABS_TOKEN
+- or vanagon token file ~/.vanagon-token (note this is the same file read by the pooler engine)
+- or [vmfloaty](https://github.com/puppetlabs/vmfloaty)'s config file ~/.vmfloaty.yml
+
+In order to modify or track the VM via floaty or bit-bar you can optionally add the vmpooler token (different from the ABS token) via
+- VMPOOLER_TOKEN
+- or as a second line to the file ~/.vanagon-token
+- or by using the existing mechanism in floaty using a vmpooler_fallback
 
 Contributing
 ---
@@ -310,3 +334,4 @@ See [LICENSE](LICENSE) file.
 
 ## Maintainers
 See [MAINTAINERS](MAINTAINERS) file.
+

data/bin/build

@@ -1,29 +1,8 @@
 #!/usr/bin/env ruby
-load File.expand_path(File.join(File.dirname(__FILE__), "..", "lib", "vanagon.rb"))
 
-optparse = Vanagon::OptParse.new("#{File.basename(__FILE__)} <project-name> <platform-name>[,<platform-name>...] [<target>,<target>...] [options]",
-                                 %i[workdir configdir engine preserve verbose skipcheck only_build remote-workdir])
-options = optparse.parse! ARGV
+script = File.basename($0)
 
-project = ARGV[0]
-platforms = ARGV[1]
-targets = ARGV[2]
+warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+     Use: 'vanagon #{script}' instead."
 
-if project.nil? or platforms.nil?
-  warn "project and platform are both required arguments."
-  warn optparse
-  exit 1
-end
-
-platform_list = platforms.split(',')
-if targets
-  target_list = targets.split(',')
-else
-  target_list = []
-end
-
-platform_list.zip(target_list).each do |pair|
-  platform, target = pair
-  artifact = Vanagon::Driver.new(platform, project, options.merge({ :target => target }))
-  artifact.run
-end
+exec "vanagon", script, *ARGV

data/bin/build_host_info

@@ -1,21 +1,8 @@
 #!/usr/bin/env ruby
-require 'json'
 
-load File.expand_path(File.join(File.dirname(__FILE__), "..", "lib", "vanagon.rb"))
+script = File.basename($0)
 
-optparse = Vanagon::OptParse.new("#{File.basename(__FILE__)} <project-name> <platform-name> [options]", %i[workdir configdir engine])
-options = optparse.parse! ARGV
+warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+     Use: 'vanagon #{script}' instead."
 
-project = ARGV[0]
-platforms = ARGV[1]
-
-if project.nil? or platforms.nil?
-  warn "project and platform are both required arguments."
-  warn optparse
-  exit 1
-end
-
-platforms.split(',').each do |platform|
-  driver = Vanagon::Driver.new(platform, project, options)
-  $stdout.puts JSON.generate(driver.build_host_info)
-end
+exec "vanagon", script, *ARGV

data/bin/build_requirements

@@ -1,35 +1,8 @@
 #!/usr/bin/env ruby
-require 'json'
-require 'vanagon/extensions/ostruct/json'
-require 'vanagon/extensions/set/json'
-require 'vanagon/extensions/hashable'
 
-load File.expand_path(File.join(File.dirname(__FILE__), "..", "lib", "vanagon.rb"))
+script = File.basename($0)
 
-optparse = Vanagon::OptParse.new("#{File.basename(__FILE__)} <project-name> <platform-name> [options]", %i[workdir configdir engine])
-optparse.parse! ARGV
-
-project = ARGV[0]
-platform = ARGV[1]
-
-unless project and platform
-  warn "project and platform are both required arguments."
-  warn optparse
-  exit 1
-end
-
-driver = Vanagon::Driver.new(platform, project)
-components = driver.project.components
-component_names = components.map(&:name)
-build_requirements = []
-components.each do |component|
-  build_requirements << component.build_requires.reject do |req|
-    # only include external requirements: i.e. those that do not match
-    # other components in the project
-    component_names.include?(req)
-  end
-end
-$stdout.puts
-$stdout.puts "**** External packages required to build #{project} on #{platform}: ***"
-$stdout.puts JSON.pretty_generate(build_requirements.flatten.uniq.sort)
+warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+     Use: 'vanagon #{script}' instead."
 
+exec "vanagon", script, *ARGV

data/bin/inspect

@@ -1,25 +1,8 @@
 #!/usr/bin/env ruby
-require 'json'
-require 'vanagon/extensions/ostruct/json'
-require 'vanagon/extensions/set/json'
-require 'vanagon/extensions/hashable'
 
-load File.expand_path(File.join(File.dirname(__FILE__), "..", "lib", "vanagon.rb"))
+script = File.basename($0)
 
-optparse = Vanagon::OptParse.new("#{File.basename(__FILE__)} <project-name> <platform-name> [options]", %i[workdir configdir engine])
-options = optparse.parse! ARGV
+warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+     Use: 'vanagon #{script}' instead."
 
-project = ARGV[0]
-platforms = ARGV[1]
-
-unless project or platforms
-  warn "project and platform are both required arguments."
-  warn optparse
-  exit 1
-end
-
-platforms.split(',').each do |platform|
-  driver = Vanagon::Driver.new(platform, project, options)
-  components = driver.project.components.map(&:to_hash)
-  $stdout.puts JSON.pretty_generate(components)
-end
+exec "vanagon", script, *ARGV

data/bin/render

@@ -1,26 +1,8 @@
 #!/usr/bin/env ruby
-load File.expand_path(File.join(File.dirname(__FILE__), "..", "lib", "vanagon.rb"))
 
-optparse = Vanagon::OptParse.new(
-  "#{File.basename(__FILE__)} <project-name> <platform-name> [options]",
-  %i[workdir configdir engine]
-)
-options = optparse.parse! ARGV
+script = File.basename($0)
 
-project = ARGV[0]
-platforms = ARGV[1]
+warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+     Use: 'vanagon #{script}' instead."
 
-if project.nil? or platforms.nil?
-  warn "project and platform are both required arguments."
-  warn optparse
-  exit 1
-end
-
-platform_list = platforms.split(',')
-target_list = []
-
-platform_list.zip(target_list).each do |pair|
-  platform, target = pair
-  artifact = Vanagon::Driver.new(platform, project, options.merge({ :target => target }))
-  artifact.render
-end
+exec "vanagon", script, *ARGV

data/bin/ship

@@ -1,32 +1,8 @@
 #!/usr/bin/env ruby
 
-ENV["PROJECT_ROOT"] = Dir.pwd
+script = File.basename($0)
 
-# Begin warning: This ship script is an internal tool.
-# This is not intended to function outside of Puppet Labs' infrastructure.
-# End of warning.
+warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+     Use: 'vanagon #{script}' instead."
 
-artifactory_warning = <<-DOC
-  Unable to ship packages to artifactory. Please make sure you are pointing to a
-  recent version of packaging in your Gemfile. Please also make sure you include
-  the artifactory gem in your Gemfile.
-
-  i.e.,
-  gem 'packaging', :github => 'puppetlabs/packaging', branch: '1.0.x'
-  gem 'artifactory'
-DOC
-
-if Dir["output/**/*"].select { |entry| File.file?(entry) }.empty?
-  fail "No packages to ship in the output directory. Maybe you want to build some first?"
-else
-  require 'packaging'
-  Pkg::Util::RakeUtils.load_packaging_tasks
-  Pkg::Util::RakeUtils.invoke_task('pl:jenkins:ship', 'artifacts', 'output')
-  begin
-    Pkg::Util::RakeUtils.invoke_task('pl:jenkins:ship_to_artifactory', 'output')
-  rescue LoadError
-    warn artifactory_warning
-  rescue StandardError
-    warn artifactory_warning
-  end
-end
+exec "vanagon", script, *ARGV

data/bin/sign

@@ -1,15 +1,8 @@
 #!/usr/bin/env ruby
 
-ENV["PROJECT_ROOT"] = Dir.pwd
+script = File.basename($0)
 
-# Begin warning: This sign script is an internal tool.
-# This is not intended to function outside of Puppet Labs' infrastructure.
-# End of warning.
+warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+     Use: 'vanagon #{script}' instead."
 
-if Dir["output/**/*"].select { |entry| File.file?(entry) }.empty?
-  fail "No packages to sign in the output directory. Maybe you want to build some first?"
-end
-
-require 'packaging'
-Pkg::Util::RakeUtils.load_packaging_tasks
-Pkg::Util::RakeUtils.invoke_task('pl:jenkins:sign_all', 'output')
+exec "vanagon", script, *ARGV

data/bin/vanagon

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+
+load File.expand_path(File.join(File.dirname(__FILE__), "..", "lib", "vanagon.rb"))
+
+vanagon = Vanagon::CLI.new
+
+vanagon.run(vanagon.parse(ARGV))

data/lib/vanagon.rb

@@ -8,7 +8,7 @@ VANAGON_VERSION = Gem.loaded_specs["vanagon"].version.to_s
 $:.unshift(LIBDIR) unless
   $:.include?(File.dirname(__FILE__)) || $:.include?(LIBDIR)
 
-require 'vanagon/optparse'
+require 'vanagon/cli'
 require 'vanagon/driver'
 
 # The main entry point is {Vanagon::Driver}.

data/lib/vanagon/cli.rb

@@ -0,0 +1,94 @@
+require 'docopt'
+require 'json'
+
+require 'vanagon/extensions/ostruct/json'
+require 'vanagon/extensions/set/json'
+require 'vanagon/extensions/hashable'
+
+require 'vanagon/cli/build'
+require 'vanagon/cli/build_host_info'
+require 'vanagon/cli/build_requirements'
+require 'vanagon/cli/inspect'
+require 'vanagon/cli/render'
+require 'vanagon/cli/ship'
+require 'vanagon/cli/sign'
+
+
+class Vanagon
+  class InvalidArgument < StandardError
+  end
+
+  class CLI
+    DOCUMENTATION = <<~DOCOPT.freeze
+      Usage:
+          vanagon <command> [<args>]...
+
+      Commands are:
+          build               build a package given a project and platform
+          build_host_info     print information about build hosts
+          build_requirements  print external packages required to build project
+          inspect             a build dry-run, printing lots of information about the build
+          render              create local versions of packaging artifacts for project
+          sign                sign a package
+          ship                upload a package to a distribution server
+          help                print this help
+    DOCOPT
+
+    def parse(argv) # rubocop:disable Metrics/AbcSize
+      parsed_options = parse_options(argv)
+      sub_command = parsed_options['<command>']
+      sub_argv = parsed_options['<args>']
+
+      case sub_command
+      when 'build'
+        @sub_parser = Vanagon::CLI::Build.new
+      when 'build_host_info'
+        @sub_parser = Vanagon::CLI::BuildHostInfo.new
+      when 'build_requirements'
+        @sub_parser = Vanagon::CLI::BuildRequirements.new
+      when 'inspect'
+        @sub_parser = Vanagon::CLI::Inspect.new
+      when 'render'
+        @sub_parser = Vanagon::CLI::Render.new
+      when 'sign'
+        @sub_parser = Vanagon::CLI::Sign.new
+      when 'ship'
+        @sub_parser = Vanagon::CLI::Ship.new
+      when 'help'
+        puts DOCUMENTATION
+        exit 0
+      else
+        warn "vanagon: Error: unknown command: \"#{sub_command}\"\n\n#{DOCUMENTATION}"
+        exit 1
+      end
+
+      raw_options = @sub_parser.parse(sub_argv)
+      options = @sub_parser.options_translate(raw_options)
+      @sub_parser.options_validate(options)
+      return options
+    end
+
+    def run(options)
+      @sub_parser.run(options)
+    end
+
+    # Do validation of options
+    def options_validate(options)
+      options
+    end
+
+    # Provide a translation from parsed docopt options to older optparse options
+    def options_translate(docopt_options)
+      docopt_options
+    end
+
+    private
+
+    def parse_options(argv)
+      Docopt.docopt(DOCUMENTATION, { argv: argv, options_first: true })
+    rescue Docopt::Exit => e
+      puts e.message
+      exit 1
+    end
+  end
+end