r10k 1.2.4 → 1.3.0rc1

checksums.yaml

@@ -1,15 +1,15 @@
 ---
 !binary "U0hBMQ==":
   metadata.gz: !binary |-
-    ZDJiODYxYjgwYmRkMTYxY2MzYTI1MTk4MmI4ZmI5MThlNzI5MmRhYw==
+    Mjc5ZmQ5NWQxMmU5NDAwOWE2MzhmNTZhZTE4MDllYjNjNDAzMTA3YQ==
   data.tar.gz: !binary |-
-    YjRjNTJjZTM3N2M4YTM1NzQwZTY4ZTMyNTdlMWI2Y2RiNjkwZWJhMQ==
+    YzA1MmNiMWUxZDI2YzcyMWZiNWQ0NzQ4NjNiNWEzMzBkNDFkMmQwZg==
 SHA512:
   metadata.gz: !binary |-
-    ZWRmNTIxMGU0NTY1MDVkMmJjNjRmODAwMzA0ZDFhYWQ2ZWYxZjBkNGFkZmJi
-    Y2VhMTkwODA4Y2ZhYzIxY2QyNTJiYjc0YmZkYjU5ZWI3Y2YwNTEzYjRhNjc0
-    ODA2YjE0MjBjN2QyZDRiNTY5MmEzNWM3Y2UzMTQxMzM0N2NmNGE=
+    Y2RlZWUzZGM5M2ZiNjQ3ZTQ5ZTlhMThkZTkxM2M1NTk3YmIxNzBiZGQ3YWQx
+    MWQ1NmNkNjVjOGI2NDE4NzEzYTAwZTFmNWQzZmM4OTlhOTY5YjdhNGMzNzg0
+    NzU2NTFkYzAwODUzZjMyZWZmMmQ5ZjIyODRlMzQ2ZmYyNWY2OGY=
   data.tar.gz: !binary |-
-    ODQwYjI0YWFhODQwZWNkODY4N2U1NjE1ZWYxZGJmOTM1NjMzYjlkNTUzNmEw
-    OGQyOGZlMTI4MTUxM2Q2MGI1YWFhZjFlZThjYjBlYWQ2NDZhYzhiNmFjOGMw
-    NTk1N2NlMGQ0ZTIzMWMxMGIzYTEzNzkxOTVjZGZiY2ZmOGU3ZjA=
+    MGEyNTFhYzNiY2YxZDIzNGExNmVlNjEyMTgwYWUzN2NiZDMwMDM0ZjhjMWY4
+    ZjFhM2MxYmUxYWIyYjQ4ZTdlMGY3MDU5NGI4OTdmYjM1MGE2MTBiZjUwM2Nj
+    MTc4NDIwNjJiYmI4MjI5N2ZjZDI5MjY1MjM3ODY1YzRlNTZlYzQ=

data/{CHANGELOG → CHANGELOG.mkd}

@@ -1,58 +1,68 @@
 CHANGELOG
 =========
 
-1.2.4
------
-
-2014/09/14
-
-### User Notes
-
-(GH-188) Call puppet module tool install with --force for downgrades
-
-1.2.3
------
-
-2014/07/27
-
-### User Notes
-
-(GH-173) Fixed a bug with Ruby 1.8.7 with Pathname objects.
-
-1.2.2
------
+1.3.0rc1
+--------
 
-2014/07/16
+2014/06/07
 
-### User Notes
+### User notes
 
-(GH-165) `r10k puppetfile` only consumes handled command line options.
+#### (GH-104) SVN support for environments.
 
-Previously, passing `-v` or other commands when running `r10k puppetfile *`
-could result in this error:
+R10k can now dynamically generate enviroments based on SVN repositories. SVN
+repositories must SVN repositories must conform to the conventional SVN
+repository structure with the directories trunk/, branches/, and optionally
+tags/ in the root of the repository. The trunk/ directory is specifically
+mapped to the production environment, branches are created as environments with
+the name of the given branch.
 
-    r10k puppetfile install --help --trace
-    Error while running: #<RuntimeError: Unrecognized options: help>
+Please note that since SVN support for environments should be considered
+preliminary and may still have some kinks to be worked out, so use it with
+caution in production.
 
-This was due to overly greedy code passing in all options from the command line
-to the TaskRunner. This has been fixed so only known options are passed along,
-and options that aren't relevant (such as :verbose) will be ignored.
+#### (GH-112) Modules can be deployed in a single environment.
 
-(GH-158) Log levels are now documented in the command line --help pages.
+When deploying modules with `r10k deploy module <modules>`, users can specify
+the `-e <environment>` flag to update modules in a single environment.
 
-(GH-137) Git remotes are now correctly updated.
+#### (GH-117) Controllable behavior for invalid Git branches.
 
-A regression in the Git remote handling meant that git remotes would never be
-properly updated when switching Git environments and modules from one remote
-to another, and the git alternates file was never updated properly. This has
-been fixed so that when the Git remote is updated, all references to the
-remotes and alternates will be updated.
+Git sources can now tune how r10k behaves when it encounters a git branch with
+a non-word character. Valid values are 'correct_and_warn' which emits a warning
+and sanitizes the environment name, 'correct' which silently corrects the
+environment, and 'error' which emits an error and ignores the environment.
 
-(GH-163) All Git tags are deleted when switching Git remotes
+### Developer notes
 
-Git tags cannot necessarily be transferred from one Git repository to another,
-so when a Git repo has its remotes changed all tags are deleted to prevent stale
-tags from overwriting tags from the new repo.
+#### IMPORTANT: as of 1.3.0, r10k is relaxing how it implements semantic versioning.
+
+There are a lot of internal APIs in r10k that need to be improved or
+overhauled, and making changes in a backwards compatible manner has been
+impeding development on a number of important features. There's no indication
+that there are any consumers of the r10k internal APIs, and if that's the case
+then r10k doesn't help anyone by maintaining SemVer for its internal APIs.
+
+As of 1.3.0, r10k is dropping guarantees about API compatibility for most of
+the core functionality. The `R10K::Task*` classes were designed to be the
+primary interface for external use and those will remain backwards compatible
+for 1.x. However any code around configuration parsing, deployments, sources,
+environments, and modules may have API changes in minor versions.
+
+That being said, if you are using any of these APIs and you experience breakage,
+you're not out of luck. If an API change actually does affect you please report
+it as a bug and those specific APIs can probably be fixed up. Hopefully this
+will ease development of r10k while not making the lives of external developers
+too painful.
+
+#### (GH-116) Allow alternate implementations of sources and environments
+
+This allows the groundwork for allowing users to implement plugins for sources
+and environments. A real API specification for sources and environments has
+been started, and sources and environments can be defined at runtime. In the
+long run r10k will add a plugin system for loading additional code from
+Rubygems and other sources, so r10k will be extensible without requiring
+modifications to the source.
 
 1.2.1
 -----

data/doc/dynamic-environments/configuration.mkd

@@ -32,7 +32,7 @@ cachedir: '/var/cache/r10k'
 
 [prerun_command](http://docs.puppetlabs.com/references/latest/configuration.html#preruncommand)
 
-The cachedir setting defaults to `~/.r10k'. If the HOME environment variable is
+The cachedir setting defaults to `~/.r10k`. If the HOME environment variable is
 unset r10k will assume that r10k is being run with the Puppet [`prerun_command`][prerun_command]
 setting and will set the cachedir default to `/root/.r10k`.
 

data/doc/dynamic-environments/git-environments.markdown

@@ -40,3 +40,22 @@ appropriate Puppet environment to the latest version. Finally if there are
 Puppet environments that don't have matching Git branches, r10k will assume that
 the branches for those environments were deleted and will remove those
 environments.
+
+Configuration
+-------------
+
+The following configuration options can be specified for Git based environment
+sources.
+
+### invalid_branches:
+
+This setting specifies how Git branch names that cannot be cleanly mapped to
+Puppet environments will be handled.
+
+Valid values:
+
+  * 'correct_and_warn': Non-word characters will be replaced with underscores
+    and a warning will be emitted. (Default)
+  * 'correct': Non-word characters will silently be replaced with underscores.
+  * 'error': Branches with non-word characters will be ignored and an error will
+    be emitted.

data/doc/dynamic-environments/usage.mkd

@@ -73,3 +73,9 @@ This is useful for when you're working on a module specified in a Puppetfile and
 Update multiple modules across all environments
 
     r10k deploy module apache jenkins java
+
+- - -
+
+Update one or more modules in a single environment
+
+    r10k deploy module -e production apache jenkins java

data/lib/r10k/cli/deploy.rb

@@ -51,6 +51,10 @@ scheduled. On subsequent deployments, Puppetfile deployment will default to off.
           DESCRIPTION
 
           flag :p, :puppetfile, 'Deploy modules from a puppetfile'
+          flag :h, :help, 'Show help for this command' do |value, cmd|
+            puts cmd.help
+            exit 0
+          end
 
           run do |opts, args, cmd|
             deploy = R10K::Deployment.load_config(opts[:config])
@@ -86,11 +90,18 @@ It will load the Puppetfile configurations out of all environments, and will
 try to deploy the given module names in all environments.
           DESCRIPTION
 
+          flag :h, :help, 'Show help for this command' do |value, cmd|
+            puts cmd.help
+            exit 0
+          end
+          required :e, :environment, 'Update the modules in the given environment'
+
           run do |opts, args, cmd|
             deploy = R10K::Deployment.load_config(opts[:config])
 
             task = R10K::Task::Deployment::DeployModules.new(deploy)
             task.module_names = args
+            task.environment_names = [opts[:environment]] if opts[:environment]
 
             runner = R10K::TaskRunner.new(:trace => opts[:trace])
             runner.append_task task
@@ -111,6 +122,10 @@ try to deploy the given module names in all environments.
           summary 'Display environments and modules in the deployment'
 
           flag :p, :puppetfile, 'Display Puppetfile modules'
+          flag :h, :help, 'Show help for this command' do |value, cmd|
+            puts cmd.help
+            exit 0
+          end
 
           run do |opts, args, cmd|
             deploy = R10K::Deployment.load_config(opts[:config])

data/lib/r10k/cli/ext/logging.rb

@@ -1,5 +1,4 @@
 require 'cri/command_dsl'
-require 'r10k/logging'
 
 module Cri
   class CommandDSL

data/lib/r10k/cli/module/deploy.rb

@@ -1,6 +1,5 @@
 require 'r10k/cli/module'
 require 'r10k/deployment'
-require 'r10k/logging'
 require 'cri'
 
 require 'fileutils'

data/lib/r10k/cli/puppetfile.rb

@@ -37,7 +37,7 @@ Puppetfile (http://bombasticmonkey.com/librarian-puppet/).
 
             puppetfile = R10K::Puppetfile.new(puppetfile_root, puppetfile_path, puppetfile)
 
-            runner = R10K::TaskRunner.new(:trace => opts[:trace])
+            runner = R10K::TaskRunner.new(opts)
             task   = R10K::Task::Puppetfile::Sync.new(puppetfile)
             runner.append_task task
 
@@ -90,7 +90,7 @@ Puppetfile (http://bombasticmonkey.com/librarian-puppet/).
 
             puppetfile = R10K::Puppetfile.new(puppetfile_root, puppetfile_path, puppetfile)
 
-            runner = R10K::TaskRunner.new(:trace => opts[:trace])
+            runner = R10K::TaskRunner.new(opts)
             task   = R10K::Task::Puppetfile::Purge.new(puppetfile)
             runner.append_task task
 

data/lib/r10k/cli.rb

@@ -1,5 +1,4 @@
 require 'r10k'
-require 'r10k/logging'
 require 'r10k/version'
 require 'r10k/cli/ext/logging'
 
@@ -16,23 +15,10 @@ module R10K::CLI
         complex environments.
       EOD
 
-
-      flag :h, :help, 'Show help for this command' do |value, cmd|
-        # This is evil because we may not necessarily be called from the
-        # command line and have a meaningful ARGV to scan. However the best
-        # way of having a globally useful --help command is to define the
-        # behavior in the block of the option to immediately handle it and exit
-        # and we don't have access to the verbose option, so the simple method
-        # is to simply scan ARGV.
-        verbose = (ARGV.include?('-v') || ARGV.include?('--verbose'))
-        puts cmd.help(:verbose => verbose)
-        exit 0
-      end
-
+      flag :h, :help, 'Show help for this command'
       flag :t, :trace, 'Display stack traces on application crash'
 
-      loglevels = R10K::Logging::LOG_LEVELS.reverse.map(&:downcase).join(", ")
-      optional :v, :verbose, "Set log verbosity. Valid values: #{loglevels}" do |value, cmd|
+      optional :v, :verbose, 'Set verbosity level' do |value, cmd|
         case value
         when true
           R10K::Logging.level = 'INFO'

data/lib/r10k/deployment/environment.rb

@@ -1,90 +1,20 @@
-require 'r10k/module'
-require 'r10k/logging'
+require 'r10k/deployment'
+require 'r10k/environment'
 
-module R10K
-class Deployment
-class Environment
-
-  include R10K::Logging
-
-  # @!attribute [r] ref
-  #   The git ref to instantiate into the basedir
-  attr_reader :ref
-
-  # @!attribute [r] remote
-  #   The location of the remote git repository
-  attr_reader :remote
-
-  # @!attribute [r] basedir
-  #   The basedir to clone the root into
-  attr_reader :basedir
-
-  # @!attribute [r] dirname
-  #   @return [String] The directory name to use for the environment
-  attr_reader :dirname
+class R10K::Deployment::Environment
 
   # @param [String] ref
   # @param [String] remote
   # @param [String] basedir
   # @param [String] dirname The directory to clone the root into, defaults to ref
   # @param [String] source_name An additional string which may be used with ref to build dirname
-  def initialize(ref, remote, basedir, dirname = nil, source_name = "")
-    @ref     = ref
-    @remote  = remote
-    @basedir = basedir
-    alternate_name =  source_name.empty? ? ref : source_name + "_" + ref
-    @dirname = sanitize_dirname(dirname || alternate_name)
-
-    @working_dir = R10K::Git::WorkingDir.new(@ref, @remote, @basedir, @dirname)
-
-    @full_path = File.join(@basedir, @dirname)
-  end
-
-  def sync
-    recursive_needed = !(@working_dir.cloned?)
-    @working_dir.sync
-
-    if recursive_needed
-      logger.debug "Environment #{@full_path} is a fresh clone; automatically updating modules."
-      sync_modules
-    end
-  end
-
-  def sync_modules
-    modules.each do |mod|
-      mod.sync
-    end
-  end
-
-  def puppetfile
-    @puppetfile ||= R10K::Puppetfile.new(@full_path)
-  end
-
-  def modules
-    puppetfile.load
-    puppetfile.modules
-  end
-
-  private
-
-  # Strip out non-word characters in an environment directory name
-  #
-  # Puppet can only use word characters (letter, digit, underscore) in
-  # environment names; this cleans up environment names to avoid traversals
-  # and similar issues.
-  #
-  # @param input [String] The raw branch name
   #
-  # @return [String] The sanitized branch dirname
-  def sanitize_dirname(input)
-    output  = input.dup
-    invalid = %r[\W+]
-    if output.gsub!(invalid, '_')
-      logger.warn "Environment #{input.inspect} contained non-word characters; sanitizing to #{output.inspect}"
-    end
+  # @deprecated
+  # @return [R10K::Environment::Git]
+  def self.new(ref, remote, basedir, dirname = nil, source_name = "")
+    alternate_name =  source_name.empty? ? ref : source_name + "_" + ref
+    dirname = dirname || alternate_name
 
-    output
+    R10K::Environment::Git.new(ref, basedir, dirname, {:remote => remote, :ref => ref})
   end
 end
-end
-end

data/lib/r10k/deployment/source.rb

@@ -1,110 +1,36 @@
-require 'r10k/git/cache'
-require 'r10k/deployment/environment'
-require 'r10k/deployment/basedir'
-require 'r10k/util/purgeable'
+require 'r10k/source'
+require 'r10k/util/core_ext/hash_ext'
 
 module R10K
 class Deployment
 class Source
-  # Represents a git repository to map branches to environments
-  #
-  # This module is backed with a bare git cache that's used to enumerate
-  # branches. The cache isn't used for anything else here, but all environments
-  # using that remote will be able to reuse the cache.
-
-  # @!attribute [r] name
-  #   @return [String] The short name for the deployment source
-  attr_reader :name
-
-  # @!attribute [r] source
-  #   @return [String] The git remote to use for environments
-  attr_reader :remote
-
-  # @!attribute [r] basedir
-  #   @return [String] The base directory to deploy the environments into
-  attr_reader :basedir
-
-  # @!attribute [r] environments
-  #   @return [Array<R10K::Deployment::Environment>] All environments for this source
-  attr_reader :environments
-
   # Create a new source from a hash representation
   #
   # @param name [String] The name of the source
   # @param opts [Hash] The properties to use for the source
-  # @param prefix [true, false] Whether to prefix the source name to created
-  #   environments
   #
   # @option opts [String] :remote The git remote for the given source
   # @option opts [String] :basedir The directory to create environments in
   # @option opts [true, false] :prefix Whether the environment names should
-  #   be prefixed by the source name. Defaults to false. This takes precedence
-  #   over the `prefix` argument
+  #   be prefixed by the source name. Defaults to false.
   #
-  # @return [R10K::Deployment::Source]
-  def self.vivify(name, attrs, prefix = false)
-    remote  = (attrs.delete(:remote) || attrs.delete('remote'))
-    basedir = (attrs.delete(:basedir) || attrs.delete('basedir'))
-    prefix_config = (attrs.delete(:prefix) || attrs.delete('prefix'))
-    prefix_outcome = prefix_config.nil? ? prefix : prefix_config
+  # @deprecated
+  # @return [R10K::Source::Base]
+  def self.vivify(name, attrs)
 
-    raise ArgumentError, "Unrecognized attributes for #{self.name}: #{attrs.inspect}" unless attrs.empty?
-    new(name, remote, basedir, prefix_outcome)
-  end
-
-  def initialize(name, remote, basedir, prefix = nil)
-    @name    = name
-    @remote  = remote
-    @basedir = basedir
-    @prefix = prefix.nil? ? false : prefix
+    attrs.extend R10K::Util::CoreExt::HashExt::SymbolizeKeys
+    attrs.symbolize_keys!
 
-    @cache   = R10K::Git::Cache.generate(@remote)
-
-    load_environments
-  end
+    remote  = attrs.delete(:remote)
+    basedir = attrs.delete(:basedir)
+    prefix  = attrs.delete(:prefix)
 
-  def fetch_remote
-    @cache.sync
-    load_environments
-  end
-
-  include R10K::Util::Purgeable
-
-  def managed_directory
-    @basedir
-  end
-
-  def current_contents
-    dir = self.managed_directory
-    glob_part = @prefix ? @name.to_s() + '_*' : '*'
-    glob_exp = File.join(dir, glob_part)
-
-    Dir.glob(glob_exp).map do |fname|
-      File.basename fname
-    end
-  end
-
-  # List all environments that should exist in the basedir for this source
-  # @note This implements a required method for the Purgeable mixin
-  # @return [Array<String>]
-  def desired_contents
-    @environments.map {|env| env.dirname }
+    raise ArgumentError, "Unrecognized attributes for #{self.name}: #{attrs.inspect}" unless attrs.empty?
+    new(name, remote, basedir, prefix)
   end
 
-  private
-
-  def load_environments
-    if @cache.cached?
-      @environments = @cache.branches.map do |branch|
-        if @prefix
-          R10K::Deployment::Environment.new(branch, @remote, @basedir, nil, @name.to_s())
-        else
-          R10K::Deployment::Environment.new(branch, @remote, @basedir)
-        end
-      end
-    else
-      @environments = []
-    end
+  def self.new(name, remote, basedir, prefix)
+    R10K::Source::Git.new(name, basedir, {:prefix => prefix, :remote => remote})
   end
 end
 end

data/lib/r10k/deployment.rb

@@ -1,13 +1,16 @@
-require 'r10k'
-require 'r10k/deployment/source'
-require 'r10k/deployment/config'
-
 require 'yaml'
+require 'r10k'
+require 'r10k/source'
 
 module R10K
 class Deployment
   # Model a full installation of module directories and modules.
 
+  require 'r10k/deployment/environment'
+  require 'r10k/deployment/basedir'
+  require 'r10k/deployment/source'
+  require 'r10k/deployment/config'
+
   # Generate a deployment object based on a config
   #
   # @param path [String] The path to the deployment config
@@ -19,16 +22,12 @@ class Deployment
 
   def initialize(config)
     @config = config
-
-    load_environments
   end
 
-  def fetch_sources
-    sources.each do |source|
-      source.fetch_remote
-    end
-    load_environments
+  def preload!
+    sources.each(&:preload!)
   end
+  alias fetch_sources preload!
 
   # Lazily load all sources
   #
@@ -36,7 +35,7 @@ class Deployment
   # used directly as it could be legitimately unset if we're doing lazy
   # loading.
   #
-  # @return [Array<R10K::Deployment::Source>] All repository sources
+  # @return [Array<R10K::Source::Base>] All repository sources
   #   specified in the config
   def sources
     load_sources if @_sources.nil?
@@ -49,7 +48,7 @@ class Deployment
   # used directly as it could be legitimately unset if we're doing lazy
   # loading.
   #
-  # @return [Array<R10K::Deployment::Environment>] All enviroments across
+  # @return [Array<R10K::Environment::Base>] All enviroments across
   #   all sources
   def environments
     load_environments if @_environments.nil?
@@ -61,7 +60,7 @@ class Deployment
   def load_sources
     sources = @config.setting(:sources)
     @_sources = sources.map do |(name, hash)|
-      R10K::Deployment::Source.vivify(name, hash)
+      R10K::Source.from_hash(name, hash)
     end
   end
 

data/lib/r10k/environment/base.rb

@@ -0,0 +1,42 @@
+# This class defines a common interface for environment implementations.
+#
+# @since 1.3.0
+class R10K::Environment::Base
+
+  # @!attribute [r] name
+  #   @return [String] A name for this environment that is unique to the given source
+  attr_reader :name
+
+  # @!attribute [r] basedir
+  #   @return [String] The path that this environment will be created in
+  attr_reader :basedir
+
+  # @!attribute [r] dirname
+  #   @return [String] The directory name for the given environment
+  attr_reader :dirname
+
+  # Initialize the given environment.
+  #
+  # @param name [String] The unique name describing this environment.
+  # @param basedir [String] The base directory where this environment will be created.
+  # @param dirname [String] The directory name for this environment.
+  # @param options [Hash] An additional set of options for this environment.
+  #   The semantics of this environment may depend on the environment implementation.
+  def initialize(name, basedir, dirname, options = {})
+    @name    = name
+    @basedir = basedir
+    @dirname = dirname
+    @options = options
+
+    @full_path = File.join(@basedir, @dirname)
+  end
+
+  # Synchronize the given environment.
+  #
+  # @api public
+  # @abstract
+  # @return [void]
+  def sync
+    raise NotImplementedError, "#{self.class} has not implemented method #{__method__}"
+  end
+end