r10k 1.1.4 → 1.2.0rc1

checksums.yaml

@@ -1,15 +1,15 @@
 ---
 !binary "U0hBMQ==":
   metadata.gz: !binary |-
-    YTRiMTQ2ZTY4M2U0YjFkNmU0NTJjMDgzMmQxOTY0NjkxODA4ZGU5ZQ==
+    ZDUwMzg3OTY2MGUxZjA3MmFkMmQyNTAxOTUyMTc2YmJjNDU3ZWIyMg==
   data.tar.gz: !binary |-
-    NDlmZmRjZjljZDUzMzI2YTc5ODVlMThmODY0ZTY2YjEyZTVlNGY4Zg==
+    ODkwYWNjOGRjNzcyMGEwYTJiOTM5ZmE0ZDcwYjdlYzdkYjkwMmE2YQ==
 SHA512:
   metadata.gz: !binary |-
-    MWU3YTljOWYwYjljZDYzMzY3OGQyMzg2YzRmN2ZmNDM3NDM2M2E1MmI1MDA5
-    MmNlMTM1OGNlYTk4Y2E0NWIyNmY4MWE5Yzk4ZjE4MTQ1YWY4NDFkMWYzNGIx
-    YTNhOWIzZTU5YzNkYWFmYzY3OWJlMGNmN2QxOTg0NjI4ZGM3ZjM=
+    ZmRmZmJiMjRkMGM3MjcwZGRlODk2YTAzZDM4ZTYwNTFhZTNhYjA0ZTg4Yzg5
+    M2M3OTI2YjgxYjdjZTE3NjQ5MWU1MmY3ZTcxYTg0ZTA1MDQ0MmI0ODI0NTI3
+    NWJmMDk0M2NhODczNjllNzA4MDU4ODJiNjRmOWQ4NzNjMmQ3YmU=
   data.tar.gz: !binary |-
-    YmRmY2IyOGY0ZGVkZTFiZjVhNzY4M2U2NjdlY2U5MTMwNTliNjE0ZjAxMTIx
-    NDM5ZjY5ZTg2ZDlkY2I4MmI0N2RmYjA3MjQzNTJmY2Y3YWEwMTg2ZDQyZjhj
-    MDU2YmVlZWZjNmU1OGY1M2IxYTBiOWE3YTQzYTI0MTI0ODQ5ZmY=
+    MzExZTMyY2VjZTI4N2E3M2M2NjQ3Yzg4YTZmYzZjMjUwMmE0ZmQ4MjNmMWJk
+    MmY3NTEwNmJlNDVkNWI5MGEyN2MwMjZjMmQ0NjYyYmJmZTQ0ZGNkYTU2MzZi
+    NWRmZTViMTBjZGEzYTAwOTdiMmYzYmNkMmM2NThkM2U1MGQyMjk=

data/.gitignore

@@ -3,3 +3,4 @@ Gemfile.local
 Gemfile.lock
 .bundle
 bundle
+/.rspec_system

data/.nodeset.yml

@@ -0,0 +1,7 @@
+---
+default_set: 'centos-59-x64'
+sets:
+  'centos-59-x64':
+    nodes:
+      "main":
+        prefab: 'centos-59-x64'

data/.rspec

@@ -0,0 +1 @@
+--default_path spec/unit

data/.travis.yml

@@ -1,6 +1,7 @@
 ---
 language: ruby
-script: "bundle exec rspec --color --format documentation"
+bundler_args: "--without system"
+script: "bundle exec rspec --color --format documentation spec/unit"
 notifications:
   email: false
 rvm:

data/CHANGELOG

@@ -1,26 +1,31 @@
 CHANGELOG
 =========
 
-1.1.4
+1.2.0
 -----
 
-2014-02-24
+2014/02/08
 
-This is a backwards compatible bugfix release.
+### User Notes
 
-### Notes
+Preliminary support for Puppetfile modules from SVN sources. SVN repositories
+can track the latest available revision or may be pinned to a specific revision.
 
-  * (GH-90) Multiple environments with the same name but with different sources
-    were previously colliding and some environments were being ignored. This has
-    been fixed and all environments should be deployed when updates are run.
+Forge modules can now track the latest available version. This can be enabled by
+setting the module version to `:latest`.
 
-### Thanks
+Git based Puppetfile modules can now be specified as branches, tags, and
+commits. When tags and commits are specified r10k can perform optimizations
+when updating the given repositories to reduce network accesses.
 
-Thanks to the following contributors for their their extraordinary patience and
-help in for chasing down GH-90:
+Command execution has been greatly improved. The old library for executing
+commands (systemu) had very high overhead and was 50 - 100 times slower than
+%x[] or fork/exec. It's been replaced with a custom process execution
+implementation.
 
-  * Andreas Ntaflos (antaflos)
-  * Igor Galić (igalic)
+Modules can swap out sources. When an existing module is changed from Forge to
+Git, for instance, the existing module will be removed before the new module is
+installed. (GH-30)
 
 1.1.3
 -----

data/Gemfile

@@ -2,6 +2,14 @@ source 'https://rubygems.org'
 
 gemspec
 
+group(:system) do
+  gem 'rake'
+  gem 'rspec-system', '~> 2.8.0'
+  gem 'rspec-system-serverspec', '~> 2.0.1'
+  gem 'net-ssh', '2.7.0'
+  gem 'vagrant', :git => 'git://github.com/mitchellh/vagrant', :tag => 'v1.4.1'
+end
+
 if File.exists? "#{__FILE__}.local"
   eval(File.read("#{__FILE__}.local"), binding)
 end

data/README.markdown

@@ -75,9 +75,11 @@ Common Commands
 Puppetfile support
 ------------------
 
-r10k can operate on a Puppetfile as a drop-in replacement for librarian-puppet.
+r10k can operate on a Puppetfile using the same syntax as librarian-puppet uses.
 Puppetfiles are a simple Ruby based DSL that specifies a list of modules to
-install, what version to install, and where to fetch them from.
+install, what version to install, and where to fetch them from. Unlike
+librarian-puppet dependency resolution is not yet implemented but is on the
+roadmap.
 
 Puppetfile based commands are under the `r10k puppetfile` subcommand.
 

data/Rakefile

@@ -0,0 +1 @@
+require 'rspec-system/rake_task'

data/doc/dynamic-environments.markdown

@@ -0,0 +1,206 @@
+Dynamic Environments
+====================
+
+r10k implements the dynamic environment workflow with Puppet. This allows you to
+create, modify, and remove Puppet environments on the fly with Git branches.
+
+Dynamic Environments in a nutshell
+----------------------------------
+
+The core idea of dynamic environments is that you should be able to manage your
+Puppet modules in the same manner that you would manage any other code base. It
+builds on top of Git topic branch model.
+
+[git-topic-branching]: http://git-scm.com/book/en/Git-Branching-Branching-Workflows#Topic-Branches "Git Topic Branches"
+
+One of the most prevalent ways of using Git relies on using [topic branches][git-topic-branching].
+Whenever changes need to be made that need to be reviewed or tested before going
+live, they should be done in a different, short lived branch called a topic
+branch. Work can be freely done on a topic branch in isolation and when the work
+is completed it is merged into a "master" or "production" branch. This is very
+powerful because it allows any number of people to rapidly develop features in
+isolation and merge features in a single operation.
+
+The dynamic environment model extends extends this git branching strategy to
+your live Puppet masters. It creates a mapping between Git branches and Puppet
+environments so that you can use the Git branching model and have that be
+seamlessly reflected in Puppet environments. This means that creating a new Git
+branch creates a new Puppet environment, updating a Git branch will update that
+environment, and deleting a Git branch will remove that environment.
+
+How it works
+------------
+
+r10k works by tracking the state of your git repositories and comparing them the
+the currently deployed environments. If there's a Git branch that doesn't have a
+corresponding Puppet environment then r10k will clone that branch into the
+appropriate directory. When Git branches are updated r10k will update the
+appropriate Puppet environment to the latest version. Finally if there are
+Puppet environments that don't have matching Git branches, r10k will assume that
+the branches for those environments were deleted and will remove those
+environments.
+
+Configuration
+-------------
+
+r10k uses a configuration file to determine how dynamic environments should be
+deployed.
+
+### Config file location
+
+By default r10k will try to read `/etc/r10k.yaml` for configuration settings.
+You can specify an alternate configuration file by specifying the `--config`
+option, like so:
+
+    r10k deploy -c /srv/puppet/r10k.yaml
+
+### Configuration format
+
+#### cachedir
+
+The `cachedir` setting specifies where r10k should keep cached information.
+Right now this is predominantly used for caching git repositories but will be
+expanded as other subsystems can take advantage of caching.
+
+For example:
+
+    ---
+    # Store all cache information in /var/cache
+    cachedir: '/var/cache/r10k'
+
+#### sources
+
+The `sources` setting specifies what repositories should be used for creating
+dynamic environments.
+
+The `sources` setting is a hash where each key is the short name of a specific
+repository (for instance, "qa" or "web" or "ops") and the value is a hash of
+properties for that source.
+
+#### source sub-options
+
+##### remote
+
+The remote is the URL of the Git repository to clone. This repository will need
+to be cloned without user intervention so SSH keys will need to be configured
+for the user running r10k.
+
+##### basedir
+
+The basedir is the directory that will be populated with Puppet environments.
+This directory will be entirely managed by r10k and any contents that r10k did
+not put there will be _removed_.
+
+##### prefix
+
+The prefix setting allows environment names to be prefixed with the short name
+of the given source. This prevents collisions when multiple sources are deployed
+into the same directory.
+
+#### source examples
+
+##### Basic examples
+
+The majority of users will only have a single repository where all modules and
+hiera data files are kept. In this case you will specify a single source:
+
+    ---
+    # Specify a single environment source
+    sources:
+      operations:
+        remote: 'git@github.com:my-org/org-modules'
+        basedir: '/etc/puppet/environments'
+
+- - -
+
+##### Advanced examples
+
+For more complex cases where you want to store hiera data in a different
+repository and your modules in another repository, you can specify two sources:
+
+    ---
+    sources:
+      operations:
+        remote: 'git@github.com:my-org/org-modules'
+        basedir: '/etc/puppet/environments'
+      hiera:
+        remote: 'git@github.com:my-org/org-hiera-data'
+        basedir: '/etc/puppet/hiera-data'
+
+- - -
+
+Alternately you may want to create separate environments from multiple
+repositories. This is useful when you want two groups to be able to deploy
+Puppet modules but they should only have write access to their own modules and
+not the modules of other groups.
+
+    ---
+    sources:
+      main:
+        remote: 'git@github.com:my-org/main-modules'
+        basedir: '/etc/puppet/environments'
+        prefix: false # Prefix defaults to false so this is only here for clarity
+      qa:
+        remote: 'git@github.com:my-org/qa-puppet-modules'
+        basedir: '/etc/puppet/environments'
+        prefix: true
+      dev:
+        remote: 'git@github.com:my-org/dev-puppet-modules'
+        basedir: '/etc/puppet/environments'
+        prefix: true
+
+This will create the following directory structure:
+
+
+    /etc/puppet/environments
+    |-- production       # main-modules repository, production branch
+    |-- upgrade_apache   # main-modules repository, upgrade_apache branch
+    |-- qa_production    # qa repository, production branch
+    |-- qa_jenkins_test  # qa repository, jenkins_test branch
+    |-- dev_production   # dev repository, production branch
+    `-- dev_loadtest     # dev repository, loadtest branch
+
+Dynamic environments and Puppetfiles
+------------------------------------
+
+TODO
+
+Interacting with dynamic environments
+-------------------------------------
+
+r10k provides fairly fine grained controls over your environments to fit your
+needs. If you want to do a full update of all of your environments and modules
+and don't need it to be done in real time, you can trigger a full update and let
+it run in the background. If you are actively developing code and need to run
+very fast updates of one specific environment, you can do a targeted update of
+that code as well.
+
+All commands that deal with deploying environments are grouped under the `r10k
+deploy` subcommand.
+
+### Examples
+
+#### Deploying environments
+
+    # Update all environments across all sources. This can be slow depending
+    # on the number of environments and modules that you're using.
+    r10k deploy environment
+
+    # Update a single environment. When you're actively working on an
+    # environment this is the best way to deploy your changes.
+    r10k deploy environment my_working_environment
+
+    # This is the brute force approach of "update everything, ever." This can
+    # run for an extremely long time so it should not be something you run
+    # interactively on a regular basis.
+    r10k deploy environment --puppetfile
+
+#### Deploying modules
+
+    # Update a single module across all environments This is useful for when
+    # you're working on a module in an environment and only want to update that
+    # one module.
+    r10k deploy module apache
+
+    # More than one module can be updated at a time.
+    r10k deploy module apache jenkins java

data/doc/puppetfile.markdown

@@ -0,0 +1,87 @@
+Puppetfile
+==========
+
+The Puppetfile is a way to reuse independent Puppet modules in your codebase.
+
+When directly working with Puppetfiles, you can use the `r10k puppetfile`
+subcommand to interact with a Puppetfile.
+
+When using r10k's deploy functionality, interacting with Puppetfiles is handled
+on a case by case basis.
+
+The Puppetfile format is actually implemented using a Ruby DSL so any valid Ruby
+expression can be used. That being said, being a bit too creative in the DSL
+can lead to surprising (read: bad) things happening, so consider keeping it
+simple.
+
+Module types
+------------
+
+### Git
+
+Modules can be installed via git.
+
+#### Examples
+
+    # Install puppetlabs/apache and keep it up to date with 'master'
+    mod 'apache',
+      :git => 'https://github.com/puppetlabs/puppetlabs-apache'
+
+    # Install puppetlabs/apache and track the 'docs_experiment' branch
+    mod 'apache',
+      :git => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :ref => 'docs_experiment'
+
+You can also use the exact object type you want to check out. This may be a
+little bit more work but it has the advantage that r10k make certain
+optimizations based on the object type that you specify.
+
+    mod 'apache',
+      :git => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :tag => '0.9.0'
+
+    mod 'apache',
+      :git    => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :commit => '83401079053dca11d61945bd9beef9ecf7576cbf'
+
+You can also explicitly specify a branch, but this behaves the same as
+specifying :ref and is mainly useful for clarity.
+
+    mod 'apache',
+      :git    => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :branch => 'docs_experiment'
+
+### Forge
+
+Modules can be installed using the Puppet module tool.
+
+If no version is specified the latest version available at the time will be
+installed, and will be kept at that version.
+
+    mod 'puppetlabs/apache'
+
+If a version is specified then that version will be installed.
+
+    mod 'puppetlabs/apache', '0.10.0'
+
+If the version is set to :latest then the module will be always updated to the
+latest version available.
+
+    mod 'puppetlabs/apache', :latest
+
+### SVN
+
+Modules can be installed via SVN.
+
+    mod 'apache',
+      :svn => 'https://github.com/puppetlabs/puppetlabs-apache/trunk'
+
+    mod 'apache',
+      :svn => 'https://github.com/puppetlabs/puppetlabs-apache/trunk',
+      :rev => '154'
+
+Alternately,
+
+    mod 'apache',
+      :svn      => 'https://github.com/puppetlabs/puppetlabs-apache/trunk',
+      :revision => '154'

data/lib/r10k/cli.rb

@@ -28,7 +28,7 @@ module R10K::CLI
         end
       end
 
-      required :c, :config, 'Specify a configuration file' do |value, cmd|
+      required :c, :config, 'Specify a global configuration file (deprecated, use `r10k deploy -c`)' do |value, cmd|
         logger.warn "Calling `r10k --config <action>` as a global option is deprecated; use r10k <action> --config"
       end
 

data/lib/r10k/errors.rb

@@ -1,5 +1,32 @@
 module R10K
-class ExecutionFailure < StandardError
-  attr_accessor :exit_code, :stdout, :stderr
-end
+  class ExecutionFailure < StandardError
+    attr_accessor :exit_code, :stdout, :stderr
+  end
+
+  # An error class that accepts an optional hash.
+  #
+  # @overload initialize(mesg)
+  #   @param mesg [String] The exception mesg
+  #
+  # @overload initialize(mesg, options)
+  #   @param mesg [String] The exception mesg
+  #   @param options [Hash] A set of options to store on the exception
+  #
+  # @overload initialize(options)
+  #   @param options [Hash] A set of options to store on the exception
+  #
+  class R10KError < StandardError
+    def initialize(mesg = nil, options = {})
+      if mesg.is_a? String
+        super(mesg)
+        @mesg    = mesg
+        @options = options
+      elsif mesg.is_a? Hash
+        @mesg    = nil
+        @options = mesg
+      elsif mesg.nil? and options
+        @options = options
+      end
+    end
+  end
 end