vanagon 0.16.0 → 0.19.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2fff2c855cb0a9c8555e584f43fe29baa5cc6da9c9d2e938cc077c7049c8e2bf
-  data.tar.gz: 1ee6759c62df9a9eb6c5bf1614c8543ab8a0ff625f444e67471499638745322c
+  metadata.gz: 607d97840532f0417fd81348a7decb7fbc8e86791fee5e11f8ef53350600d5b4
+  data.tar.gz: c03e5996707781aa956b29599e788b55c7c1cde5cfc707a261d4a22eec3c16f2
 SHA512:
-  metadata.gz: 3fe34a4441dd96772e3c9e6b7a824090b27b898b059f7b08996d8ab8dc5ad695c889e91bbe322829fb2b2359a53a191cf78ac8cfba0ac68a0ce938cc12d5b5db
-  data.tar.gz: 996e0265bfb18a929a44348ef287139fb076d9b351b80e4b99e3a26b16624efafb246bff9fb7a90fdc699b004f934df677a0fc0cee8fd71eee4cb8b984f1c353
+  metadata.gz: f59b39738804c83787514f2662a9ba40343d126de298b9b01b4a97cd4d5195ad6c14c0255049fdf002bcb72fa5fa8b27a57983853219a8eddd35dac9cf08030d
+  data.tar.gz: ea3ac00e2beb1919372cde8c74bacc22d68dab5911c0f7cd2c14091153cd0aff3592446bcfac21f77a4c7569299cd2927a3437ee2cfdc3814cb85dc480c42704

data/README.md

@@ -45,9 +45,10 @@ Also, Vanagon ships with a number of engines which may include additional option
 
 ### Local Host:
 
-- [Ruby](https://www.ruby-lang.org/en/) (Ruby 2.1.x is the miniumum supported version)
+- [Ruby](https://www.ruby-lang.org/en/) (Ruby 2.3.x is the miniumum supported version)
 - [fustigit](https://github.com/mckern/fustigit)
 - [ruby-git](https://github.com/schacon/ruby-git)
+- [docopt](https://github.com/docopt/docopt.rb)
 - The command line tool `ssh` ([homepage](https://www.openssh.com/)) available on the local `${PATH}` (any modern version should suffice)
 - The command line tool `rsync` ([homepage](https://rsync.samba.org/)) available on the local `${PATH}` (At least rsync 2.6.x)
 - The command line tool `git` ([homepage](https://git-scm.com/)) available on the local `${PATH}` (Vanagon is tested against Git version 1.8.x but should work with any newer version)
@@ -84,10 +85,28 @@ wheezy and build my project against it.
 For more detailed examples of the DSLs available, please see the
 [examples](examples) directory and the YARD documentation for Vanagon.
 
-### `build` usage
+### CLI changes and deprecations (from version 0.16.0)
+
+Prior to 0.16.0, the vanagon command line contained these commands
+
+* `build`
+* `build_host_info`
+* `build_requirements`
+* `inspect`
+* `render`
+* `repo`
+* `ship`
+* `sign`
+
+With the exception of `repo`, which calls `packaging` methods, the remaining commands
+have been moved to a git-like pattern of `vanagon <subcommand>`. `vangon build` replaced `build`, `vanagon ship` replaced `ship` and so forth.
+
+The older calling usage is deprecated.
+
+### `vanagon build` usage
 The build command has positional arguments and position independent flags.
 
-#### Arguments (position dependent)
+#### Positional arguments
 
 ##### project name
 The name of the project to build; a file named `<project_name>.rb` must be
@@ -114,7 +133,8 @@ Build machines should be cleaned between builds.
 
 #### Flagged arguments
 
-**Note:** command flags can be used anywhere in the command.
+**Note:** command flags currently can be used anywhere in the command. Recommended usages
+is to use flagged arguments before the positional arguments.
 
 ##### -w DIR, --workdir DIR
 Specifies a directory on the local host where the sources should be placed and builds performed.
@@ -134,9 +154,10 @@ Currently supported engines are:
 * `base` - Pure ssh backend; no teardown currently defined
 * `local` - Build on the local machine; platform name must match the local machine
 * `docker` - Builds in a docker container
-* `pooler` - Selects a vm from Puppet Labs' vm pooler to build on
+* `pooler` - Selects a vm from Puppet's internal vmpooler to build on
 * `hardware` - Build on a specific taget and lock it in redis
 * `ec2` - Build on a specific AWS instance.
+* `ABS` - Selects a vm from Puppet's internal Always-be-scheduling service to build on
 
 #### Flags
 
@@ -147,9 +168,6 @@ Indicates that the host used for building the project should be left intact
 after the build instead of destroyed. The host is usually destroyed after a
 successful build, or left after a failed build.
 
-##### -v, --verbose
-(Reserved for future implementation) Will increase the verbosity of output, when implemented.
-
 ##### -h, --help
 Display command-line help.
 
@@ -195,16 +213,16 @@ integer value these components to fail after the `VANAGON_TIMEOUT` count is reac
 Note that this value is expected to be in seconds.
 
 #### Example usage
-`build --preserve puppet-agent el-6-i386` will build the puppet-agent project
+`vanagon build --preserve puppet-agent el-6-i386` will build the puppet-agent project
 on the el-6-i386 platform and leave the host intact afterward.
 
-`build --engine=docker puppet-agent el-6-i386` will build the puppet-agent
+`vanagon build --engine=docker puppet-agent el-6-i386` will build the puppet-agent
 project on the el-6-i386 platform using the docker engine (the platform must
 have a docker\_image defined in its config).
 
 ---
 
-### `inspect` usage
+### `vanagon inspect` usage
 
 The `inspect` command has positional arguments and position independent flags. It
 mirrors the `build` command, but exits with success after loading and interpolating
@@ -212,7 +230,7 @@ all of the components in the given project. No attempt is made to actually build
 the given project; instead, a JSON formatted array of hashes is returned and printed
 to `stdout`. This JSON array can be further processed by external tooling, such as `jq`.
 
-#### Arguments (position dependent)
+#### Positional arguments
 
 ##### project name
 The name of the project to build, and a file named \<project\_name\>.rb must be
@@ -227,7 +245,8 @@ Platform can also be a comma separated list of platforms such as platform1,platf
 
 #### Flagged arguments
 
-**Note:** command flags can be used anywhere in the command.
+**Note:** command flags currently can be used anywhere in the command. Recommended usages
+is to use flagged arguments before the positional arguments.
 
 ##### -w DIR, --workdir DIR
 Specifies a directory where the sources should be placed and builds performed.
@@ -239,29 +258,24 @@ Specifies where project configuration is found. Defaults to $pwd/configs.
 ##### -e ENGINE, --engine ENGINE
 Choose a different virtualization engine to use to select the build target.
 Engines are respected, but only insofar as components and projects are
-rendered -- the `inspect` command performs no compilation.
+rendered -- the `vanagon inspect` command performs no compilation.
 
-Supported engines are the same as the `build` command.
+Supported engines are the same as the `vanagon build` command.
 
 #### Flags
 
-**Note:** command flags can be used anywhere in the command.
-
-##### -v, --verbose (not yet implemented)
-Increase verbosity of output.
-
 ##### -h, --help
 Display command-line help.
 
 #### Environment variables
 
 Environment variables are respected, but only insofar as components and projects are
-rendered -- the `inspect` command has no behavior to alter.
+rendered -- the `vanagon inspect` command has no behavior to alter.
 
-Supported environment variables are the same as the `build` command.
+Supported environment variables are the same as the `vanagon build` command.
 
 #### Example usage
-`inspect puppet-agent el-6-i386` will load the puppet-agent project
+`vanagon inspect puppet-agent el-6-i386` will load the puppet-agent project
 on the el-6-i386 platform and print the resulting list of dependencies,
 build-time configuration, environment variables, and expected artifacts.
 
@@ -297,6 +311,16 @@ platform "el-7-x86_64" do |plat|
 end
 ```
 
+### ABS (internal)
+When using the ABS engine, there is a variety of ways you can specify your token: 
+- the environment variable ABS_TOKEN
+- or vanagon token file ~/.vanagon-token (note this is the same file read by the pooler engine)
+- or [vmfloaty](https://github.com/puppetlabs/vmfloaty)'s config file ~/.vmfloaty.yml
+
+In order to modify or track the VM via floaty or bit-bar you can optionally add the vmpooler token (different from the ABS token) via
+- VMPOOLER_TOKEN
+- or as a second line to the file ~/.vanagon-token
+- or by using the existing mechanism in floaty using a vmpooler_fallback
 
 Contributing
 ---
@@ -310,3 +334,4 @@ See [LICENSE](LICENSE) file.
 
 ## Maintainers
 See [MAINTAINERS](MAINTAINERS) file.
+

data/bin/build

@@ -1,8 +1,10 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 script = File.basename($0)
 
-warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+VanagonLogger.info "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
      Use: 'vanagon #{script}' instead."
 
 exec "vanagon", script, *ARGV

data/bin/build_host_info

@@ -1,8 +1,10 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 script = File.basename($0)
 
-warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+VanagonLogger.info "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
      Use: 'vanagon #{script}' instead."
 
 exec "vanagon", script, *ARGV

data/bin/build_requirements

@@ -1,8 +1,10 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 script = File.basename($0)
 
-warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+VanagonLogger.info "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
      Use: 'vanagon #{script}' instead."
 
 exec "vanagon", script, *ARGV

data/bin/inspect

@@ -1,8 +1,10 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 script = File.basename($0)
 
-warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+VanagonLogger.info "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
      Use: 'vanagon #{script}' instead."
 
 exec "vanagon", script, *ARGV

data/bin/render

@@ -1,8 +1,10 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 script = File.basename($0)
 
-warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+VanagonLogger.info "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
      Use: 'vanagon #{script}' instead."
 
 exec "vanagon", script, *ARGV

data/bin/repo

@@ -1,5 +1,7 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 ENV["PROJECT_ROOT"] = Dir.pwd
 
 # Begin warning: This ship script is an internal tool.
@@ -21,7 +23,7 @@ when 'rpm'
 when 'deb'
   Pkg::Util::RakeUtils.invoke_task('pl:jenkins:deb_repos')
 when 'none'
-  $stdout.puts "Skipping repo generation since repo target is set to 'none'"
+  VanagonLogger.warn "Skipping repo generation since repo target is set to 'none'"
 else
   Pkg::Util::RakeUtils.invoke_task('pl:jenkins:rpm_repos')
   Pkg::Util::RakeUtils.invoke_task('pl:jenkins:deb_repos')

data/bin/ship

@@ -1,8 +1,10 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 script = File.basename($0)
 
-warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+VanagonLogger.info "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
      Use: 'vanagon #{script}' instead."
 
 exec "vanagon", script, *ARGV

data/bin/sign

@@ -1,8 +1,10 @@
 #!/usr/bin/env ruby
 
+require 'vanagon/logger'
+
 script = File.basename($0)
 
-warn "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
+VanagonLogger.info "#{script}: Warning: use of stand alone '#{script}' command is deprecated and may be removed.
      Use: 'vanagon #{script}' instead."
 
 exec "vanagon", script, *ARGV

data/extras/completions/vanagon.bash

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+_vanagon()
+{
+  local cur prev projects commands template_arg_commands 
+
+  # COMREPLY is an array variable used to store completions  
+  # the completion mechanism uses COMPRELY to display its contents as completions
+  # COMP_WORDS is an array of all the words typed after the name of the program 
+  # COMP_CWORD is an index of the COMP_WORDS array pointing to the current word
+  COMPREPLY=()
+  cur="${COMP_WORDS[COMP_CWORD]}"
+  prev="${COMP_WORDS[COMP_CWORD-1]}"
+  projects=($({ vanagon list -r | sed 1d; } 2>/dev/null))
+  commands="build build_host_info build_requirements completion inspect list render sign ship help"
+  template_arg_commands="build build_host_info build_requirements inspect render "
+
+  # completes with a project if the previous word was a command in template_arg_commands
+  if [[ $template_arg_commands =~ (^| )$prev($| ) ]] ; then
+      _vanagon_avail_templates_projects=$({ vanagon list -r | sed 1d; } 2>/dev/null)
+      # compgen generates completions filtered based on what has been typed by the user
+      COMPREPLY=( $(compgen -W "${_vanagon_avail_templates_projects}" -- "${cur}") )
+  fi 
+
+  # allows multiple platforms to be tab completed 
+  if [[ ${#COMP_WORDS[@]} -gt 3 ]] ; then 
+    _vanagon_avail_templates_platforms=$({ vanagon list -l | sed 1d; } 2>/dev/null)
+    COMPREPLY=( $(compgen -W "${_vanagon_avail_templates_platforms}" -- "${cur}") )
+  fi
+
+  if [[ $1 == $prev ]] ; then
+    # only show top level commands we are at root
+    COMPREPLY=( $(compgen -W "${commands}" -- "${cur}") )
+  fi
+}
+
+# assign tab complete function `_vanagon ` to `vanagon` command 
+complete -F _vanagon vanagon

data/extras/completions/vanagon.zsh

@@ -0,0 +1,41 @@
+_vanagon()
+{
+  local line commands template_arg_commands projects
+
+  commands="build build_host_info build_requirements completion inspect list render sign ship help"
+  template_arg_commands=("build" "build_host_info" "build_requirements" "inspect"  "render")
+  projects=($({ vanagon list -r | sed 1d; } 2>/dev/null))
+
+  # '%p:globbed-files:' sets completion to only offer files matching a 
+  # described pattern.
+  zstyle ':completion:*' file-patterns '%p:globbed-files:'
+  
+  # arguments function provides potential completions to zsh 
+  # specs are of the form n:message:action 
+  _arguments -C \
+    ": :(${commands})" \
+    "*::arg:->args" 
+
+  # (Ie)prevents "invalid subscript"
+  if ((template_arg_commands[(Ie)$line[1]])); then
+    _vanagon_template_sub_projects
+  fi
+  if [[ $projects  =~ (^| )$line[2]($| ) ]]; then
+    _vanagon_template_sub_platforms
+  fi
+}
+
+_vanagon_template_sub_projects()
+{
+  # -W look in certain path but don't append path to tab compelte 
+  # -g enables file matching pattern 
+  # (:r) removes the file extension `.rb` from the completion 
+  _arguments "1: :_files -W $(PWD)/configs/projects/ -g '*.rb(:r)'"
+}
+
+_vanagon_template_sub_platforms()
+{
+  _arguments "*: :_files -W $(PWD)/configs/platforms/ -g '*.rb(:r)'"
+}
+# compdef registeres the completion function: compdef <function-name> <program>
+compdef _vanagon vanagon

data/lib/vanagon/cli.rb

@@ -8,11 +8,15 @@ require 'vanagon/extensions/hashable'
 require 'vanagon/cli/build'
 require 'vanagon/cli/build_host_info'
 require 'vanagon/cli/build_requirements'
+require 'vanagon/cli/completion'
 require 'vanagon/cli/inspect'
+require 'vanagon/cli/list'
 require 'vanagon/cli/render'
 require 'vanagon/cli/ship'
 require 'vanagon/cli/sign'
 
+require 'vanagon/logger'
+
 
 class Vanagon
   class InvalidArgument < StandardError
@@ -27,7 +31,9 @@ class Vanagon
           build               build a package given a project and platform
           build_host_info     print information about build hosts
           build_requirements  print external packages required to build project
+          completion          outputs path to tab completion script
           inspect             a build dry-run, printing lots of information about the build
+          list                shows a list of available projects and platforms
           render              create local versions of packaging artifacts for project
           sign                sign a package
           ship                upload a package to a distribution server
@@ -46,10 +52,14 @@ class Vanagon
         @sub_parser = Vanagon::CLI::BuildHostInfo.new
       when 'build_requirements'
         @sub_parser = Vanagon::CLI::BuildRequirements.new
+      when 'completion'
+        @sub_parser = Vanagon::CLI::Completion.new
       when 'inspect'
         @sub_parser = Vanagon::CLI::Inspect.new
       when 'render'
         @sub_parser = Vanagon::CLI::Render.new
+      when 'list'
+        @sub_parser = Vanagon::CLI::List.new
       when 'sign'
         @sub_parser = Vanagon::CLI::Sign.new
       when 'ship'
@@ -58,7 +68,7 @@ class Vanagon
         puts DOCUMENTATION
         exit 0
       else
-        warn "vanagon: Error: unknown command: \"#{sub_command}\"\n\n#{DOCUMENTATION}"
+        VanagonLogger.error "vanagon: Error: unknown command: \"#{sub_command}\"\n\n#{DOCUMENTATION}"
         exit 1
       end
 
@@ -87,7 +97,7 @@ class Vanagon
     def parse_options(argv)
       Docopt.docopt(DOCUMENTATION, { argv: argv, options_first: true })
     rescue Docopt::Exit => e
-      puts e.message
+      VanagonLogger.error e.message
       exit 1
     end
   end

data/lib/vanagon/cli/build.rb

@@ -1,4 +1,5 @@
 require 'docopt'
+require 'vanagon/logger'
 
 class Vanagon
   class CLI
@@ -10,7 +11,7 @@ class Vanagon
         Options:
           -h, --help                       Display help
           -c, --configdir DIRECTORY        Configuration directory [default: #{Dir.pwd}/configs]
-          -e, --engine ENGINE              Custom engine to use [base, local, docker, pooler] [default: pooler]
+          -e, --engine ENGINE              Custom engine to use [default: always_be_scheduling]
           -o, --only-build COMPONENT,COMPONENT,...
                                            Only build listed COMPONENTs
           -p, --preserve [RULE]            Rule for VM preservation: never, on-failure, always
@@ -19,12 +20,20 @@ class Vanagon
           -s, --skipcheck                  Skip the "check" stage when building components
           -w, --workdir DIRECTORY          Working directory on the local host
           -v, --verbose                    Only here for backwards compatibility. Does nothing.
+
+        Engines:
+          always_be_scheduling: default engine using Puppet's ABS infrastructure
+          docker: a docker container on the local host
+          ec2: an Amazon EC2 instance
+          hardware: a dedicated hardware device
+          local: the local machine, cannot be used with a target
+          pooler: [deprecated] Puppet's vmpooler
       DOCOPT
 
       def parse(argv)
         Docopt.docopt(DOCUMENTATION, { argv: argv })
       rescue Docopt::Exit => e
-        puts e.message
+        VanagonLogger.error e.message
         exit 1
       end
 
@@ -38,7 +47,7 @@ class Vanagon
 
         platform_list.zip(target_list).each do |pair|
           platform, target = pair
-          artifact = Vanagon::Driver.new(platform, project, options.merge({ 'target' => target }))
+          artifact = Vanagon::Driver.new(platform, project, options.merge({ :target => target }))
           artifact.run
         end
       end