r10k 1.2.1 → 1.2.2

checksums.yaml

@@ -1,15 +1,15 @@
 ---
 !binary "U0hBMQ==":
   metadata.gz: !binary |-
-    NzM2NDQyNzM1Y2I0Zjc5ZmUzNmEyZDA0MTJiMzZjM2E2MWNhNWJhMw==
+    MzBlNGU0MTQxMjk1ODQzM2ZkOGMzNmEyYjFmZTVkMGIzY2RiMzk5OQ==
   data.tar.gz: !binary |-
-    NmRkYWZiZDYxNGEyNTZiZDgzZDQwMjEzYjQ0MjA2MjgxNjE0MjRlMw==
+    NTc0ZWVhNjdiZGZjZWNlZTUyZjQ3OTU5ZGMxZWI1NDAzN2Q4YTQ1ZQ==
 SHA512:
   metadata.gz: !binary |-
-    YWE2ZTg3ZDI1YzA4YTlkNTFlNjQ2MDIzZDU1YWNjZGExMjY3NTY5NGRhN2Fm
-    MTNhNDA4NzY1YTNkODM4ZjQ4NTI4ZmRiOTEwNDIwZWYwOWNiN2VhZWE3M2My
-    NDE4MGY0M2ZiZjcyOGI2Y2UyNWU5ZWRjNjlhZGJkZDc4ZjZlMGY=
+    ODk2YzRlZGZhOGEwNmMyYTgzMzNjMTY0YzI4OTZjZTc5ZjA3MWYwNTViMTA1
+    MmNmMDhjMDUzMDRkYzQ2ZmY5NWRkNmQ3ZWU1YWRkMDZkYTdmOWYxZWYzOGVh
+    ZjU1MzY3MjU1OTgzNTk0ZmIyYzYwYjE0YjA5N2I4YWIwNTNjYjA=
   data.tar.gz: !binary |-
-    NzM5YmU2ODRlNzg2NGEzMzQ3NjBmODIzYzE4MmUzM2NkYmRmYmExOTBhMWNk
-    NGNhMjI5Njc3MGEyNWVjYjg3NTNmMjJkOTY0ZTAzMDM3NzQ5YzhlYjlhZDI0
-    Yzg5MDQ0MDNlNmI5NjlhNGQwZTRiMGRhM2ZkNDI3MWM4Zjk2OGQ=
+    ZDUzYzQ1Yjg3Njk2NjA1YmM2YjVhN2JjNDkyYjY3OTU5YWVkYmE1NGI0OGJl
+    MzQ3OGM2NDY2M2U3OGMwYjVkMGY3OTU4NjI0NTIyMjQyMDlmMzg2YzMyZDg2
+    OTQwMjI4NDVhZGQwYzMyOTkxNWYyZWEzNmQ5ZjYwNzUzNmU1YjI=

data/CHANGELOG

@@ -1,6 +1,41 @@
 CHANGELOG
 =========
 
+1.2.2
+-----
+
+2014/07/16
+
+### User Notes
+
+(GH-165) `r10k puppetfile` only consumes handled command line options.
+
+Previously, passing `-v` or other commands when running `r10k puppetfile *`
+could result in this error:
+
+    r10k puppetfile install --help --trace
+    Error while running: #<RuntimeError: Unrecognized options: help>
+
+This was due to overly greedy code passing in all options from the command line
+to the TaskRunner. This has been fixed so only known options are passed along,
+and options that aren't relevant (such as :verbose) will be ignored.
+
+(GH-158) Log levels are now documented in the command line --help pages.
+
+(GH-137) Git remotes are now correctly updated.
+
+A regression in the Git remote handling meant that git remotes would never be
+properly updated when switching Git environments and modules from one remote
+to another, and the git alternates file was never updated properly. This has
+been fixed so that when the Git remote is updated, all references to the
+remotes and alternates will be updated.
+
+(GH-163) All Git tags are deleted when switching Git remotes
+
+Git tags cannot necessarily be transferred from one Git repository to another,
+so when a Git repo has its remotes changed all tags are deleted to prevent stale
+tags from overwriting tags from the new repo.
+
 1.2.1
 -----
 

data/doc/dynamic-environments.mkd

@@ -0,0 +1,24 @@
+Dynamic Environments
+====================
+
+Dynamic environment deployment is the core functionality of r10k.
+
+Table of contents
+-----------------
+
+  * [Introduction](dynamic-environments/introduction/): A brief description of
+    what dynamic environments are and how they work.
+  * [Git environments](dynamic-environments/git-environments/) How r10k
+    implements dynamic environments using git
+  * [Configuration](dynamic-environments/configuration/) A reference of dynamic
+    environment configuration options and how they're used
+  * [Usage](dynamic-environments/usage/) A reference of r10k commands and how
+    they're used.
+
+- - -
+
+Community guides
+----------------
+
+  * [Building a Functional Puppet Workflow Part 3: Dynamic Environments with R10k](http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/)
+  * [Puppet Infrastructure with r10k](http://terrarum.net/blog/puppet-infrastructure-with-r10k.html)

data/doc/dynamic-environments/configuration.mkd

@@ -0,0 +1,177 @@
+Dynamic Environment Configuration
+=================================
+
+r10k uses a configuration file to determine how dynamic environments should be
+deployed.
+
+Config file location
+--------------------
+
+By default r10k will try to read `/etc/r10k.yaml` for configuration settings.
+You can specify an alternate configuration file by specifying the `--config`
+option, like so:
+
+    r10k deploy -c /srv/puppet/r10k.yaml
+
+General options
+---------------
+
+### cachedir
+
+The 'cachedir' setting specifies where r10k should keep cached information.
+Right now this is predominantly used for caching git repositories but will be
+expanded as other subsystems can take advantage of caching.
+
+For example:
+
+```yaml
+---
+# Store all cache information in /var/cache
+cachedir: '/var/cache/r10k'
+```
+
+[prerun_command](http://docs.puppetlabs.com/references/latest/configuration.html#preruncommand)
+
+The cachedir setting defaults to `~/.r10k'. If the HOME environment variable is
+unset r10k will assume that r10k is being run with the Puppet [`prerun_command`][prerun_command]
+setting and will set the cachedir default to `/root/.r10k`.
+
+Deployment options
+------------------
+
+The following options configure how r10k deploys dynamic environments.
+
+### sources
+
+The `sources` setting specifies what repositories should be used for creating
+dynamic environments.  It is a hash where each key is the short name of a
+specific repository (for instance, "qa" or "web" or "ops") and the value is a
+hash of properties for that source.
+
+```yaml
+---
+sources:
+  main:
+    # Source settings follow
+```
+
+Source options
+--------------
+
+The following options are respected by all source implementations. Sources may
+implement other options in addition to the ones listed below; see the source
+specific documentation for more information.
+
+### remote
+
+The 'remote' setting specifies where the source repository should be fetched
+from. It may be any valid URL that the source may check out or clone. The remote
+must be able to be fetched without any interactive input, eg usernames or
+passwords cannot be prompted for in order to fetch the remote.
+
+```
+---
+sources:
+  mysource:
+    remote: 'git://git-server.site/my-org/main-modules'
+```
+
+### basedir
+
+The 'basedir' setting specifies where environments will be created for this
+source.  This directory will be entirely managed by r10k and any contents that
+r10k did not put there will be _removed_.
+
+```yaml
+---
+sources:
+  mysource:
+    basedir: '/etc/puppet/environments'
+```
+
+R10k will not check to make sure that different sources collide in a single base
+directory; if you are using multiple sources you are responsible for making sure
+that they do not overwrite each other. See also the prefix setting.
+
+### prefix
+
+The prefix setting allows environment names to be prefixed with the short name
+of the given source. This prevents collisions when multiple sources are deployed
+into the same directory.
+
+```yaml
+---
+sources:
+  mysource:
+    basedir: '/etc/puppet/environments'
+    prefix: true # All environments will be prefixed with "mysource_"
+```
+
+Examples
+--------
+
+### Minimal example
+
+The majority of users will only have a single repository where all modules and
+hiera data files are kept. In this case you will specify a single source:
+
+```yaml
+---
+sources:
+  operations:
+    remote: 'git://git-server.site/my-org/org-modules'
+    basedir: '/etc/puppet/environments'
+```
+
+### Separate hiera data
+
+For more complex cases where you want to store hiera data in a different
+repository and your modules in another repository, you can specify two sources:
+
+```yaml
+---
+sources:
+  operations:
+    remote: 'git://git-server.site/my-org/org-modules'
+    basedir: '/etc/puppet/environments'
+  hiera:
+    remote: 'git://git-server.site/my-org/org-hiera-data'
+    basedir: '/etc/puppet/hiera-data'
+```
+
+### Multiple tenancy
+
+Alternately you may want to create separate environments from multiple
+repositories. This is useful when you want two groups to be able to deploy
+Puppet modules but they should only have write access to their own modules and
+not the modules of other groups.
+
+```yaml
+---
+sources:
+  main:
+    remote: 'git://git-server.site/my-org/main-modules'
+    basedir: '/etc/puppet/environments'
+    prefix: false # Prefix defaults to false so this is only here for clarity
+  qa:
+    remote: 'git://git-server.site/my-org/qa-puppet-modules'
+    basedir: '/etc/puppet/environments'
+    prefix: true
+  dev:
+    remote: 'git://git-server.site/my-org/dev-puppet-modules'
+    basedir: '/etc/puppet/environments'
+    prefix: true
+```
+
+This will create the following directory structure:
+
+```
+/etc/puppet/environments
+|-- production       # main-modules repository, production branch
+|-- upgrade_apache   # main-modules repository, upgrade_apache branch
+|-- qa_production    # qa repository, production branch
+|-- qa_jenkins_test  # qa repository, jenkins_test branch
+|-- dev_production   # dev repository, production branch
+`-- dev_loadtest     # dev repository, loadtest branch
+```
+

data/doc/dynamic-environments/git-environments.markdown

@@ -0,0 +1,42 @@
+Git Based Dynamic Environments
+==============================
+
+r10k can use Git repositories to implement dynamic environments. You can create,
+update, and delete Puppet environments automatically as part of your normal Git
+workflow.
+
+Dynamic Environments in a nutshell
+----------------------------------
+
+The core idea of dynamic environments is that you should be able to manage your
+Puppet modules in the same manner that you would manage any other code base. It
+builds on top of Git topic branch model.
+
+[git-topic-branching]: http://git-scm.com/book/en/Git-Branching-Branching-Workflows#Topic-Branches "Git Topic Branches"
+
+One of the most prevalent ways of using Git relies on using [topic branches][git-topic-branching].
+Whenever changes need to be made that need to be reviewed or tested before going
+live, they should be done in a different, short lived branch called a topic
+branch. Work can be freely done on a topic branch in isolation and when the work
+is completed it is merged into a "master" or "production" branch. This is very
+powerful because it allows any number of people to rapidly develop features in
+isolation and merge features in a single operation.
+
+The dynamic environment model extends extends this git branching strategy to
+your live Puppet masters. It creates a mapping between Git branches and Puppet
+environments so that you can use the Git branching model and have that be
+seamlessly reflected in Puppet environments. This means that creating a new Git
+branch creates a new Puppet environment, updating a Git branch will update that
+environment, and deleting a Git branch will remove that environment.
+
+How it works
+------------
+
+r10k works by tracking the state of your git repositories and comparing them the
+the currently deployed environments. If there's a Git branch that doesn't have a
+corresponding Puppet environment then r10k will clone that branch into the
+appropriate directory. When Git branches are updated r10k will update the
+appropriate Puppet environment to the latest version. Finally if there are
+Puppet environments that don't have matching Git branches, r10k will assume that
+the branches for those environments were deleted and will remove those
+environments.

data/doc/dynamic-environments/introduction.mkd

@@ -0,0 +1,70 @@
+Dynamic Environments
+====================
+
+One of the most important functions of r10k is its ability to dynamically manage
+your Puppet environments.
+
+When environments were originally built into Puppet they were meant to be static
+in nature. Each environment had to be defined beforehand in the master's
+puppet.conf file in a section, like so:
+
+```ini
+[master]
+# Environment independent settings
+vardir = '/var/lib/puppet'
+
+[production]
+modulepath = '/etc/puppet/environments/production/modules'
+
+[testing]
+modulepath = '/etc/puppet/environments/testing/modules'
+
+[development]
+modulepath = '/etc/puppet/environments/development/modules'
+```
+
+Static Puppet environments were frequently used to implement a pipeline for
+developing Puppet code. New Puppet code would be developed and deployed to the
+development environment, pushed to testing for validation, and then finally
+pushed to the production for general deployment.
+
+This static nature of Puppet environments turned out to be inflexible in
+practice. With a predefined list of environments it could be very cumbersome to
+develop on different parts of a Puppet codebase in isolation; you could either
+develop multiple features in the same environment and risk cross pollution, or
+manually create new environments every time you needed isolation.
+
+Dynamic environments work by dynamically determining the settings for Puppet
+environment when the environment is used, rather than by defining an explicit
+section in puppet.conf. This works by making the current environment (in the
+'$environment' variable) part of of the path to environment specific settings,
+like modulepath, manifest, and so forth.
+
+```ini
+[master]
+vardir = '/var/lib/puppet'
+modulepath = '/etc/puppet/environments/$environment/modules'
+```
+
+Running `puppet agent -t --environment myenv` will cause $environment to be
+expanded to 'myenv', so the modulepath for that environment will be set to 
+'/etc/puppet/environments/myenv/modules'.
+
+This approach of allowing environments to be defined on the fly is a complete
+reversal of the original architecture of environments. This approach means that
+it can be very easy to create new environments, update existing environments,
+and remove environments that aren't needed anymore. It's common practice
+to create a temporary environment to test an idea and destroy it shortly after.
+R10k is designed to enable this sort of fluid workflow.
+
+R10k predominantly uses version control systems to implement dynamic
+environments. This works by inspecting the VCS repositories containing your
+Puppet code and checking out that code on your masters so, that there's a 1:1
+connection between a branch in your VCS repository and a Puppet environment on
+your masters. This approach allows you to define the way you want to work and
+use that with your chosen VCS, and r10k will make Puppet implement that
+workflow.
+
+Different version control systems will implement dynamic environments in
+slightly different ways; check out the VCS specific documentation for more
+information.

data/doc/dynamic-environments/usage.mkd

@@ -0,0 +1,75 @@
+Usage
+=====
+
+R10k provides fairly fine grained controls over your environments to fit your
+needs. If you want to do a full update of all of your environments and modules
+and don't need it to be done in real time, you can trigger a full update and let
+it run in the background. If you are actively developing code and need to run
+very fast updates of one specific environment, you can do a targeted update of
+that code as well.
+
+All commands that deal with deploying environments are grouped under the `r10k
+deploy` subcommand.
+
+Command line invocation
+-----------------------
+
+### Deploying environments
+
+Recursively update all environments:
+
+    r10k deploy environment --puppetfile
+
+The simplest way to use r10k is by simply updating all environments and modules
+and takes the brute force approach of "update everything, ever." When this
+command is run r10k will update all sources, create new environments and delete
+old environments, and recursively update all environment modules specified in
+environment Puppetfiles. While this is the simplest method for running r10k, is
+is also the slowest by a very large degree because it does the maximum possible
+work. This should not be something you run interactively, or use on a regular
+basis.
+
+- - -
+
+Update environments while avoiding unnecessary recursion
+
+    r10k deploy environment
+
+This will update existing environments and recursively create new environments.
+Note that when an environment is deployed for the first time, it will
+automatically update all modules as well. For subsequent updates only the
+environment itself will be updated.
+
+- - -
+
+Update a single environment:
+
+    r10k deploy environment my_working_environment
+
+When you're actively developing on a given environment, this is the best way to
+deploy your changes. Note that when an environment is deployed for the first
+time, it will automatically update all modules as well. For subsequent updates
+only the environment itself will be updated.
+
+- - -
+
+Update a single environment and force an update of modules:
+
+    r10k deploy environment my_working_environment --puppetfile
+
+This will update the given environment and update all contained modules. This is
+useful if you want to make sure that a given environment is fully up to date.
+
+### Deploying modules
+
+Update a single module across all environments
+
+    r10k deploy module apache
+
+This is useful for when you're working on a module specified in a Puppetfile and want to update it across all environments.
+
+- - -
+
+Update multiple modules across all environments
+
+    r10k deploy module apache jenkins java

data/lib/r10k/cli.rb

@@ -16,10 +16,23 @@ module R10K::CLI
         complex environments.
       EOD
 
-      flag :h, :help, 'Show help for this command'
+
+      flag :h, :help, 'Show help for this command' do |value, cmd|
+        # This is evil because we may not necessarily be called from the
+        # command line and have a meaningful ARGV to scan. However the best
+        # way of having a globally useful --help command is to define the
+        # behavior in the block of the option to immediately handle it and exit
+        # and we don't have access to the verbose option, so the simple method
+        # is to simply scan ARGV.
+        verbose = (ARGV.include?('-v') || ARGV.include?('--verbose'))
+        puts cmd.help(:verbose => verbose)
+        exit 0
+      end
+
       flag :t, :trace, 'Display stack traces on application crash'
 
-      optional :v, :verbose, 'Set verbosity level' do |value, cmd|
+      loglevels = R10K::Logging::LOG_LEVELS.reverse.map(&:downcase).join(", ")
+      optional :v, :verbose, "Set log verbosity. Valid values: #{loglevels}" do |value, cmd|
         case value
         when true
           R10K::Logging.level = 'INFO'

data/lib/r10k/cli/deploy.rb

@@ -51,10 +51,6 @@ scheduled. On subsequent deployments, Puppetfile deployment will default to off.
           DESCRIPTION
 
           flag :p, :puppetfile, 'Deploy modules from a puppetfile'
-          flag :h, :help, 'Show help for this command' do |value, cmd|
-            puts cmd.help
-            exit 0
-          end
 
           run do |opts, args, cmd|
             deploy = R10K::Deployment.load_config(opts[:config])
@@ -90,11 +86,6 @@ It will load the Puppetfile configurations out of all environments, and will
 try to deploy the given module names in all environments.
           DESCRIPTION
 
-          flag :h, :help, 'Show help for this command' do |value, cmd|
-            puts cmd.help
-            exit 0
-          end
-
           run do |opts, args, cmd|
             deploy = R10K::Deployment.load_config(opts[:config])
 
@@ -120,10 +111,6 @@ try to deploy the given module names in all environments.
           summary 'Display environments and modules in the deployment'
 
           flag :p, :puppetfile, 'Display Puppetfile modules'
-          flag :h, :help, 'Show help for this command' do |value, cmd|
-            puts cmd.help
-            exit 0
-          end
 
           run do |opts, args, cmd|
             deploy = R10K::Deployment.load_config(opts[:config])

data/lib/r10k/cli/puppetfile.rb

@@ -37,7 +37,7 @@ Puppetfile (http://bombasticmonkey.com/librarian-puppet/).
 
             puppetfile = R10K::Puppetfile.new(puppetfile_root, puppetfile_path, puppetfile)
 
-            runner = R10K::TaskRunner.new(opts)
+            runner = R10K::TaskRunner.new(:trace => opts[:trace])
             task   = R10K::Task::Puppetfile::Sync.new(puppetfile)
             runner.append_task task
 
@@ -90,7 +90,7 @@ Puppetfile (http://bombasticmonkey.com/librarian-puppet/).
 
             puppetfile = R10K::Puppetfile.new(puppetfile_root, puppetfile_path, puppetfile)
 
-            runner = R10K::TaskRunner.new(opts)
+            runner = R10K::TaskRunner.new(:trace => opts[:trace])
             task   = R10K::Task::Puppetfile::Purge.new(puppetfile)
             runner.append_task task
 

data/lib/r10k/git.rb

@@ -10,6 +10,7 @@ module R10K
 
     require 'r10k/git/repository'
     require 'r10k/git/cache'
+    require 'r10k/git/alternates'
     require 'r10k/git/working_dir'
   end
 end

data/lib/r10k/git/alternates.rb

@@ -0,0 +1,49 @@
+require 'pathname'
+
+# Manage `$GIT_DIR/objects/info/alternates`
+#
+# @see man gitrepository-layout(5)
+class R10K::Git::Alternates
+
+  # @attribute [r] file
+  #   @return [Pathname] The alternates file
+  attr_reader :file
+
+  # @param git_dir [String] The path to the git repository
+  def initialize(git_dir)
+    @file = Pathname.new(File.join(git_dir, 'objects', 'info', 'alternates'))
+  end
+
+  def to_a
+    read()
+  end
+
+  def <<(path)
+    write(to_a << path)
+  end
+
+  def include?(path)
+    to_a.include?(path)
+  end
+
+  private
+
+  def write(entries)
+    if ! @file.parent.directory?
+      raise R10K::Git::GitError, "Cannot write #{@file.to_path}; parent directory does not exist"
+    end
+    @file.open("w") do |fh|
+      entries.each do |entry|
+        fh.puts(entry)
+      end
+    end
+  end
+
+  def read
+    entries = []
+    if @file.file?
+      entries = @file.readlines.map(&:chomp)
+    end
+    entries
+  end
+end

data/lib/r10k/git/repository.rb

@@ -84,7 +84,7 @@ class R10K::Git::Repository
 
     ret = {}
     output.stdout.each_line do |line|
-      next if line.match /\(push\)/
+      next if line.match(/\(push\)/)
       name, url, _ = line.split(/\s+/)
       ret[name] = url
     end
@@ -92,6 +92,13 @@ class R10K::Git::Repository
     ret
   end
 
+  def tags
+    entries = []
+    output = git(['tag', '-l'], :git_dir => @git_dir).stdout
+    output.each_line { |line| entries << line.chomp }
+    entries
+  end
+
   private
 
   # Fetch objects and refs from the given git remote

data/lib/r10k/git/working_dir.rb

@@ -41,7 +41,8 @@ class R10K::Git::WorkingDir < R10K::Git::Repository
     @full_path = File.join(@basedir, @dirname)
     @git_dir   = File.join(@full_path, '.git')
 
-    @cache = R10K::Git::Cache.generate(@remote)
+    @alternates = R10K::Git::Alternates.new(@git_dir)
+    @cache      = R10K::Git::Cache.generate(@remote)
 
     if ref.is_a? String
       @ref = R10K::Git::Ref.new(ref, self)
@@ -61,7 +62,9 @@ class R10K::Git::WorkingDir < R10K::Git::Repository
   end
 
   def update
-    if fetch?
+    update_remotes if update_remotes?
+
+    if ref_needs_fetch?
       fetch_from_cache
       checkout(@ref)
     elsif needs_checkout?
@@ -114,22 +117,17 @@ class R10K::Git::WorkingDir < R10K::Git::Repository
 
   private
 
-  def fetch?
+  # Do we need to fetch additional objects and refs in order to resolve the given ref?
+  # @return [true, false]
+  def ref_needs_fetch?
     @ref.fetch?
   end
 
   def fetch_from_cache
-    set_cache_remote
     @cache.sync
     fetch('cache')
   end
 
-  def set_cache_remote
-    if self.remote != @cache.remote
-      git ["remote", "set-url", "cache", @cache.git_dir], :path => @full_path
-    end
-  end
-
   # Perform a non-bare clone of a git repository.
   def clone
     @cache.sync
@@ -151,6 +149,30 @@ class R10K::Git::WorkingDir < R10K::Git::Repository
     expected = ref.sha1
     actual   = rev_parse('HEAD')
 
-    ! (expected == actual)
+    !(expected == actual)
+  end
+
+  def update_remotes?
+    real_remotes = remotes
+
+    expected_origin = @remote
+    expected_cache  = @cache.git_dir
+
+    !(expected_origin == real_remotes['origin'] and expected_cache == real_remotes['cache'])
+  end
+
+  def update_remotes
+    # todo: remove all existing refs as they may belong to the old remote
+    git ['remote', 'set-url', 'origin', remote], :path => @full_path
+    git ['remote', 'set-url', 'cache', @cache.git_dir], :path => @full_path
+    @alternates << File.join(@cache.git_dir, 'objects')
+    logger.debug("Removing stale git tags from #{@full_path}")
+    remove_tags
+  end
+
+  def remove_tags
+    tags.each do |tag|
+      git ['tag', '-d', tag], :path => @full_path
+    end
   end
 end

data/lib/r10k/version.rb

@@ -1,3 +1,3 @@
 module R10K
-  VERSION = '1.2.1'
+  VERSION = '1.2.2'
 end

data/spec/unit/git/alternates_spec.rb

@@ -0,0 +1,90 @@
+require 'spec_helper'
+require 'stringio'
+require 'r10k/git'
+
+describe R10K::Git::Alternates do
+  subject { described_class.new("/some/nonexistent/path/.git") }
+
+  it "interacts with the alternates file in the given git repository" do
+    expect(subject.file.to_path).to eq("/some/nonexistent/path/.git/objects/info/alternates")
+  end
+
+  describe "reading alternate object entries" do
+    it "reads the alternates file and splits on lines" do
+      expect(subject.file).to receive(:file?).and_return true
+      expect(subject.file).to receive(:readlines).and_return([
+        "/var/cache/r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git\n",
+        "/vagrant/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git\n",
+      ])
+
+      expect(subject.to_a).to eq([
+        "/var/cache/r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git",
+        "/vagrant/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git",
+      ])
+    end
+
+    it "returns an empty array when the file is not present" do
+      expect(subject.file).to receive(:file?).and_return false
+      expect(subject.file).to receive(:readlines).never
+      expect(subject.to_a).to eq([])
+    end
+  end
+
+  describe "determining if an entry is already present" do
+    before do
+      allow(subject).to receive(:to_a).and_return([
+        "/var/cache/r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git",
+        "/vagrant/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git",
+      ])
+    end
+
+    it "is true if the element is in the array of read entries" do
+      expect(subject).to include("/vagrant/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git")
+    end
+
+    it "is false if the element is not in the array of read entries" do
+      expect(subject).to_not include("/tmp/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git")
+    end
+  end
+
+  describe "appending a new alternate object entry" do
+    describe "and the git objects/info directory does not exist" do
+      it "raises an error when the parent directory does not exist" do
+        expect {
+          subject << "/tmp/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git"
+        }.to raise_error(R10K::Git::GitError,"Cannot write /some/nonexistent/path/.git/objects/info/alternates; parent directory does not exist")
+      end
+    end
+
+    describe "and the git objects/info directory exists" do
+      let(:io) { StringIO.new }
+
+      before do
+        expect(subject.file).to receive(:open).with('w').and_yield(io)
+        subject.file.stub_chain(:parent, :directory?).and_return true
+      end
+
+
+      it "creates the alternates file with the new entry when not present" do
+        expect(subject).to receive(:to_a).and_return([])
+        subject << "/tmp/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git"
+
+        expect(io.string).to eq("/tmp/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git\n")
+      end
+
+      it "rewrites the file with all alternate entries" do
+        expect(subject).to receive(:to_a).and_return([
+          "/var/cache/r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git",
+          "/vagrant/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git",
+        ])
+        subject << "/tmp/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git"
+
+        expect(io.string).to eq(<<-EOD)
+/var/cache/r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git
+/vagrant/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git
+/tmp/.r10k/git/git---github.com-puppetlabs-puppetlabs-apache.git
+        EOD
+      end
+    end
+  end
+end

data/spec/unit/git/repository_spec.rb

@@ -21,4 +21,14 @@ describe R10K::Git::Repository do
       })
     end
   end
+
+  describe "tags" do
+    let(:tags) { %w[0.1.1 0.1.2 0.1.3 0.1.4 0.2.0 0.3.0 2.0.0] }
+    let(:output) { tags.map {|x| x + "\n"}.join }
+
+    it "returns a list of tags for this repo" do
+      expect(subject).to receive(:git).with(%w[tag -l], anything).and_return(double(:stdout => output))
+      expect(subject.tags).to eq(tags)
+    end
+  end
 end

data/spec/unit/git/working_dir_spec.rb

@@ -2,11 +2,120 @@ require 'spec_helper'
 require 'r10k/git'
 
 describe R10K::Git::WorkingDir do
+  include_context "fail on execution"
 
   describe "initializing" do
     it "generates a new cache for the remote" do
-      wd = described_class.new('master', 'git://github.com/adrienthebo/r10k-fixture-repo', '/tmp')
+      wd = described_class.new('master', 'git://github.com/adrienthebo/r10k-fixture-repo', '/some/nonexistent/dir')
       wd.cache.should be_kind_of R10K::Git::Cache
     end
+
+    it "uses the provided ref as the dirname when no dirname is given" do
+      wd = described_class.new('master', 'git://github.com/adrienthebo/r10k-fixture-repo', '/some/nonexistent/dir')
+      expect(wd.dirname).to eq('master')
+    end
+
+    it "uses an explicit dirname when given" do
+      wd = described_class.new('master', 'git://github.com/adrienthebo/r10k-fixture-repo', '/some/nonexistent/dir', 'mydir')
+      expect(wd.dirname).to eq('mydir')
+    end
+  end
+
+  describe "synchronizing the working directory" do
+    subject { described_class.new('master', 'git://github.com/adrienthebo/r10k-fixture-repo', '/some/nonexistent/dir') }
+    it "clones the repository when the repository doesn't exist" do
+      expect(subject).to receive(:cloned?).and_return false
+      expect(subject).to receive(:clone)
+      subject.sync
+    end
+
+    it "updates the repository when the repository already exists" do
+      expect(subject).to receive(:cloned?).and_return true
+      expect(subject).to receive(:update)
+      subject.sync
+    end
+  end
+
+  describe "when cloning a new repository" do
+    subject { described_class.new('master', 'git://github.com/adrienthebo/r10k-fixture-repo', '/some/nonexistent/dir') }
+
+    before do
+      allow(subject).to receive(:cloned?).and_return false
+    end
+
+    it "updates the cache before cloning" do
+      expect(subject.cache).to receive(:sync)
+      allow(subject).to receive(:git)
+      allow(subject).to receive(:checkout)
+      subject.sync
+    end
+
+    it "clones the repository and uses the cache git dir as an object reference" do
+      allow(subject.cache).to receive(:sync)
+      expect(subject).to receive(:git).with(['clone', '--reference', subject.cache.git_dir,
+                                             'git://github.com/adrienthebo/r10k-fixture-repo',
+                                             '/some/nonexistent/dir/master'])
+      expect(subject).to receive(:git).with(['remote', 'add', 'cache', subject.cache.git_dir],
+                                            an_instance_of(Hash))
+
+      expect(subject).to receive(:git).with(['fetch', 'cache'], an_instance_of(Hash))
+      allow(subject).to receive(:checkout)
+      subject.sync
+    end
+
+    it 'checks out the specific ref after the clone' do
+      allow(subject.cache).to receive(:sync)
+      allow(subject).to receive(:git)
+      expect(subject).to receive(:checkout)
+      subject.sync
+    end
+  end
+
+  describe "updating an existing repository" do
+    subject { described_class.new('master', 'git://github.com/adrienthebo/r10k-fixture-repo', '/some/nonexistent/dir') }
+
+    before do
+      allow(subject).to receive(:cloned?).and_return true
+    end
+
+    it "updates the remotes when they are out of sync" do
+      allow(subject).to receive(:ref_needs_fetch?).and_return false
+      allow(subject).to receive(:needs_checkout?).and_return false
+
+      expect(subject).to receive(:update_remotes?).and_return true
+      expect(subject).to receive(:update_remotes)
+
+      subject.sync
+    end
+
+    it "updates the cache when the ref requires an update" do
+      allow(subject).to receive(:update_remotes?).and_return false
+
+      expect(subject).to receive(:ref_needs_fetch?).and_return true
+      expect(subject).to receive(:fetch_from_cache)
+      expect(subject).to receive(:checkout).with(an_instance_of(R10K::Git::Ref))
+
+      subject.sync
+    end
+
+    it "checks out the ref when the wrong commit is checked out" do
+      allow(subject).to receive(:update_remotes?).and_return false
+      allow(subject).to receive(:ref_needs_fetch?).and_return false
+
+      expect(subject).to receive(:needs_checkout?).and_return true
+      expect(subject).to receive(:checkout).with(an_instance_of(R10K::Git::Ref))
+
+      subject.sync
+    end
+
+    it "doesn't update the repo when everything is in sync" do
+      allow(subject).to receive(:update_remotes?).and_return false
+      allow(subject).to receive(:ref_needs_fetch?).and_return false
+      allow(subject).to receive(:needs_checkout?).and_return false
+
+      expect(subject).to_not receive(:checkout)
+
+      subject.sync
+    end
   end
 end

data/spec/unit/util/subprocess_spec.rb

@@ -46,6 +46,10 @@ describe R10K::Util::Subprocess do
       end
 
       it "raises an exception if raise_on_fail is true" do
+        if RUBY_VERSION =~ /^2\./
+          pending "Ruby 2.x and RSpec 2.x fail when raising an exception with a custom #to_s method"
+        end
+
         subject.raise_on_fail = true
 
         allow(result).to receive(:exit_code).and_return(255)

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: r10k
 version: !ruby/object:Gem::Version
-  version: 1.2.1
+  version: 1.2.2
 platform: ruby
 authors:
 - Adrien Thebo
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-04-22 00:00:00.000000000 Z
+date: 2014-07-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: colored
@@ -197,7 +197,11 @@ files:
 - README.markdown
 - Rakefile
 - bin/r10k
-- doc/dynamic-environments.markdown
+- doc/dynamic-environments.mkd
+- doc/dynamic-environments/configuration.mkd
+- doc/dynamic-environments/git-environments.markdown
+- doc/dynamic-environments/introduction.mkd
+- doc/dynamic-environments/usage.mkd
 - doc/puppetfile.markdown
 - lib/r10k.rb
 - lib/r10k/cli.rb
@@ -223,6 +227,7 @@ files:
 - lib/r10k/errors.rb
 - lib/r10k/execution.rb
 - lib/r10k/git.rb
+- lib/r10k/git/alternates.rb
 - lib/r10k/git/cache.rb
 - lib/r10k/git/commit.rb
 - lib/r10k/git/errors.rb
@@ -292,6 +297,7 @@ files:
 - spec/system/version_spec.rb
 - spec/unit/deployment/environment_spec.rb
 - spec/unit/deployment/source_spec.rb
+- spec/unit/git/alternates_spec.rb
 - spec/unit/git/cache_spec.rb
 - spec/unit/git/commit_spec.rb
 - spec/unit/git/head_spec.rb
@@ -381,6 +387,7 @@ test_files:
 - spec/unit/git/commit_spec.rb
 - spec/unit/git/repository_spec.rb
 - spec/unit/git/head_spec.rb
+- spec/unit/git/alternates_spec.rb
 - spec/unit/git/tag_spec.rb
 - spec/unit/deployment/environment_spec.rb
 - spec/unit/deployment/source_spec.rb

data/doc/dynamic-environments.markdown

@@ -1,206 +0,0 @@
-Dynamic Environments
-====================
-
-r10k implements the dynamic environment workflow with Puppet. This allows you to
-create, modify, and remove Puppet environments on the fly with Git branches.
-
-Dynamic Environments in a nutshell
-----------------------------------
-
-The core idea of dynamic environments is that you should be able to manage your
-Puppet modules in the same manner that you would manage any other code base. It
-builds on top of Git topic branch model.
-
-[git-topic-branching]: http://git-scm.com/book/en/Git-Branching-Branching-Workflows#Topic-Branches "Git Topic Branches"
-
-One of the most prevalent ways of using Git relies on using [topic branches][git-topic-branching].
-Whenever changes need to be made that need to be reviewed or tested before going
-live, they should be done in a different, short lived branch called a topic
-branch. Work can be freely done on a topic branch in isolation and when the work
-is completed it is merged into a "master" or "production" branch. This is very
-powerful because it allows any number of people to rapidly develop features in
-isolation and merge features in a single operation.
-
-The dynamic environment model extends extends this git branching strategy to
-your live Puppet masters. It creates a mapping between Git branches and Puppet
-environments so that you can use the Git branching model and have that be
-seamlessly reflected in Puppet environments. This means that creating a new Git
-branch creates a new Puppet environment, updating a Git branch will update that
-environment, and deleting a Git branch will remove that environment.
-
-How it works
-------------
-
-r10k works by tracking the state of your git repositories and comparing them the
-the currently deployed environments. If there's a Git branch that doesn't have a
-corresponding Puppet environment then r10k will clone that branch into the
-appropriate directory. When Git branches are updated r10k will update the
-appropriate Puppet environment to the latest version. Finally if there are
-Puppet environments that don't have matching Git branches, r10k will assume that
-the branches for those environments were deleted and will remove those
-environments.
-
-Configuration
--------------
-
-r10k uses a configuration file to determine how dynamic environments should be
-deployed.
-
-### Config file location
-
-By default r10k will try to read `/etc/r10k.yaml` for configuration settings.
-You can specify an alternate configuration file by specifying the `--config`
-option, like so:
-
-    r10k deploy -c /srv/puppet/r10k.yaml
-
-### Configuration format
-
-#### cachedir
-
-The `cachedir` setting specifies where r10k should keep cached information.
-Right now this is predominantly used for caching git repositories but will be
-expanded as other subsystems can take advantage of caching.
-
-For example:
-
-    ---
-    # Store all cache information in /var/cache
-    cachedir: '/var/cache/r10k'
-
-#### sources
-
-The `sources` setting specifies what repositories should be used for creating
-dynamic environments.
-
-The `sources` setting is a hash where each key is the short name of a specific
-repository (for instance, "qa" or "web" or "ops") and the value is a hash of
-properties for that source.
-
-#### source sub-options
-
-##### remote
-
-The remote is the URL of the Git repository to clone. This repository will need
-to be cloned without user intervention so SSH keys will need to be configured
-for the user running r10k.
-
-##### basedir
-
-The basedir is the directory that will be populated with Puppet environments.
-This directory will be entirely managed by r10k and any contents that r10k did
-not put there will be _removed_.
-
-##### prefix
-
-The prefix setting allows environment names to be prefixed with the short name
-of the given source. This prevents collisions when multiple sources are deployed
-into the same directory.
-
-#### source examples
-
-##### Basic examples
-
-The majority of users will only have a single repository where all modules and
-hiera data files are kept. In this case you will specify a single source:
-
-    ---
-    # Specify a single environment source
-    sources:
-      operations:
-        remote: 'git@github.com:my-org/org-modules'
-        basedir: '/etc/puppet/environments'
-
-- - -
-
-##### Advanced examples
-
-For more complex cases where you want to store hiera data in a different
-repository and your modules in another repository, you can specify two sources:
-
-    ---
-    sources:
-      operations:
-        remote: 'git@github.com:my-org/org-modules'
-        basedir: '/etc/puppet/environments'
-      hiera:
-        remote: 'git@github.com:my-org/org-hiera-data'
-        basedir: '/etc/puppet/hiera-data'
-
-- - -
-
-Alternately you may want to create separate environments from multiple
-repositories. This is useful when you want two groups to be able to deploy
-Puppet modules but they should only have write access to their own modules and
-not the modules of other groups.
-
-    ---
-    sources:
-      main:
-        remote: 'git@github.com:my-org/main-modules'
-        basedir: '/etc/puppet/environments'
-        prefix: false # Prefix defaults to false so this is only here for clarity
-      qa:
-        remote: 'git@github.com:my-org/qa-puppet-modules'
-        basedir: '/etc/puppet/environments'
-        prefix: true
-      dev:
-        remote: 'git@github.com:my-org/dev-puppet-modules'
-        basedir: '/etc/puppet/environments'
-        prefix: true
-
-This will create the following directory structure:
-
-
-    /etc/puppet/environments
-    |-- production       # main-modules repository, production branch
-    |-- upgrade_apache   # main-modules repository, upgrade_apache branch
-    |-- qa_production    # qa repository, production branch
-    |-- qa_jenkins_test  # qa repository, jenkins_test branch
-    |-- dev_production   # dev repository, production branch
-    `-- dev_loadtest     # dev repository, loadtest branch
-
-Dynamic environments and Puppetfiles
-------------------------------------
-
-TODO
-
-Interacting with dynamic environments
--------------------------------------
-
-r10k provides fairly fine grained controls over your environments to fit your
-needs. If you want to do a full update of all of your environments and modules
-and don't need it to be done in real time, you can trigger a full update and let
-it run in the background. If you are actively developing code and need to run
-very fast updates of one specific environment, you can do a targeted update of
-that code as well.
-
-All commands that deal with deploying environments are grouped under the `r10k
-deploy` subcommand.
-
-### Examples
-
-#### Deploying environments
-
-    # Update all environments across all sources. This can be slow depending
-    # on the number of environments and modules that you're using.
-    r10k deploy environment
-
-    # Update a single environment. When you're actively working on an
-    # environment this is the best way to deploy your changes.
-    r10k deploy environment my_working_environment
-
-    # This is the brute force approach of "update everything, ever." This can
-    # run for an extremely long time so it should not be something you run
-    # interactively on a regular basis.
-    r10k deploy environment --puppetfile
-
-#### Deploying modules
-
-    # Update a single module across all environments This is useful for when
-    # you're working on a module in an environment and only want to update that
-    # one module.
-    r10k deploy module apache
-
-    # More than one module can be updated at a time.
-    r10k deploy module apache jenkins java