r10k 1.4.2 → 1.5.1

data/.travis.yml

@@ -4,7 +4,9 @@ bundler_args: "--without system"
 script: "bundle exec rspec --color --format documentation spec/unit"
 notifications:
   email: false
+sudo: false
 rvm:
+  - "2.2.0"
   - "2.1.0"
   - "2.0.0"
   - "1.9.3"

data/CHANGELOG.mkd

@@ -1,6 +1,198 @@
 CHANGELOG
 =========
 
+1.5.1
+-----
+
+2015/04/09
+
+### Thanks
+
+Thanks to all the users that helped track down RK-86, and Zack Smith in specific
+for tracing the source of the bug.
+
+### User notes
+
+(RK-62) Warn when Rugged is compiled without SSH or HTTPS transports
+
+If Rugged/libgit2 was compiled without libssh2, trying to access a Git
+repository via SSH throws a fairly cryptic error. It's not terribly easy to
+specially handle the error message that's being logged, but in lieu of that r10k
+now checks the Rugged gem to make sure it's compiled with support for SSH and
+HTTPS when the Rugged Git provider is used.
+
+(RK-79) Bump minimum required version of faraday_middlware-multi_json
+
+Faraday 0.9.0 changed the API for middleware plugins, which made it incompatible
+with the faraday_middleware-multi_json plugin. That plugin supported the new API
+in version 0.0.6 but the minimum required version in the r10k gemspec was not
+updated, allowing r10k to be installed with incompatible versions of faraday and
+faraday_middleware-multi_json. This has been fixed by requiring the minimum
+compatible version of faraday_middleware-multi_json.
+
+(RK-86) Git modules don't properly track branches
+
+R10k 1.5.0 added smarter syncing for Git caches to reduce network traffic, but
+it accidentally caused branches to stop tracking changes to the remote branch.
+This has been fixed and the pre-1.5.0 behavior has been restored.
+
+1.5.0
+-----
+
+2015/04/02
+
+### Announcements
+
+Ruby 1.8.7 has had a good run, but it's time for r10k to think about moving on.
+As of r10k 1.5.0, support for Ruby 1.8.7 is officially deprecated. Issues
+affecting Ruby 1.8.7 will still be fixed, but will be of lower priority. Support
+for Ruby 1.8.7 will be dropped entirely in r10k 2.0.0.
+
+### Thanks
+
+Thanks to the following contributors for their work on this release:
+
+  * [Brett Swift](https://github.com/brettswift) for adding custom prefix values
+    for sources (RK-35)
+  * [Eli Young](https://github.com/elyscape) for adding additional debug
+    information to `r10k version` (RK-37)
+  * [Pete Fritchman](https://github.com/fetep) for removing a warning generated
+    under Ruby 2.2 (RK-55)
+  * [Theo Chatzimichos](https://github.com/tampakrap) for updating the
+    dependency versions for r10k
+  * [ETL](https://github.com/etlweather), [Ben S](https://github.com/juniorsysadmin),
+    and [Robert Nelson](https://github.com/rnelson0) for their documentation
+    contributions
+
+### User notes
+
+(GH-57) Git based modules fail to detect presence of git binary
+
+As part of RK-17, r10k now checks to make sure that the `git` binary is present
+and executable before trying to run any commands. If Git is missing, r10k will
+fail hard instead of trying to run and failing on the first failed command.
+
+(GH-195) (RK-35) Allow sources to specify a custom prefix string
+
+For r10k deployments that have separate sources for Puppet code and Hiera data,
+the created environment paths must line up between the code and data repos. With
+basic prefixing where the source name is used as the prefix string, it's not
+possible to use prefixing and commonly named environments. This has been fixed
+so that the source `prefix` option can be given true, false, or a string which
+is a custom value to use for prefixing.
+
+(RK-17) Add rugged/libgit2 based Git implementation
+
+The libgit2 library and Ruby rugged gem provide a native interface to Git for
+Ruby that is faster and more consistent than shelling out to Git. An additional
+rugged based Git implementation has been added along the original 'shellgit'
+implementation, and the implementations can be swapped out as needed.
+
+(RK-24) Add '/etc/puppetlabs/r10k/r10k.yaml' to config search path
+
+In order to be more consistent with the rest of the Puppet ecosystem, r10k will
+now check for configuration in a standard location inside of '/etc/puppetlabs'.
+The old location, '/etc/r10k.yaml' is still respected so no configuration change
+is needed at this point, although the old location will be deprecated in the
+future.
+
+(RK-25) Warn if both '/etc/r10k.yaml' and '/etc/puppetlabs/r10k/r10k.yaml' exist
+
+(RK-31) Remove undocumented search for 'r10k.yaml'
+
+Early versions of r10k would try to search for 'r10k.yaml' in parent
+directories, in the same manner that programs like Git and Bundler recursively
+search for configuration. However this functionality isn't terribly useful, is a
+bit surprising, and adds a lot of complexity. This behavior has been removed in
+1.5.0.
+
+(RK-32, RK-33, RK-38) Provide configuration options for Git providers
+
+The r10k Git providers can now be selected and configured via configuration in
+r10k.yaml. See the [Git specific documentation](doc/git) for more information on
+these settings.
+
+(RK-37) Support higher verbosity levels for `r10k version`
+
+If `r10k version` is run with the `--verbose` flag, diagnostic information about
+r10k will be included along with version information.
+
+(RK-39, RK-66) Improved logging/messaging
+
+R10k now logs more information about what's going on and is more consistent
+about which levels it displays messages.
+
+(RK-74) Improved log formatting
+
+The log formatting that r10k uses was added in some distant point in the past
+where there were few users, and wasn't really designed with users in mind. The
+formatting has been improved so that at lower log levels the formatting will
+include the log level and nothing else, and at higher levels it will include
+timing information.
+
+(RK-75) Optional log coloring
+
+At higher log levels r10k can produce a lot of logging information at a rapid
+rate, and in general errors and warnings are not very visually distinct and easy
+to miss. To help solve these problems r10k 1.5.0 now has optional colored
+logging to help provide more information/context at a glance. Coloring can be
+enabled with the `--color` option.
+
+(GH-265) (RK-11) Unix commands with > 64KiB of output no longer deadlock
+
+Unix pipes have a maximum buffer size of 64KiB, and if the pipe buffer fills
+then subsequent writes will block. Since r10k used to wait for a subprocess to
+finish before reading from the attached pipes, if the process filled the buffers
+then the processes would deadlock.
+
+This has been fixed by continually reading from the subprocess pipes to prevent
+the buffers from filling up.
+
+Unfortunately implementing this correctly requires reading from the pipes until
+EOF, which breaks the ssh ControlPersist functionality. Supporting both the
+ControlPersist functionality while preventing deadlock introduces too many
+potential race conditions, so unfortunately workarounds for this issue can't be
+accepted.
+
+See https://bugzilla.mindrot.org/show_bug.cgi?id=1988 for more information about
+the ssh ControlPersist bug.
+
+### Developer notes
+
+(GH-142) Use Forge v3 API
+
+R10k was using the old and deprecated v1 Forge API to determine the latest
+version of Forge modules when using the `:latest` module version. This has been
+fixed so that the v3 API is used.
+
+(RK-16) Decouple Git platform dependent and independent code
+
+The original code that interfaced r10k and Git was very tightly coupled; there
+was no defined interface and it was expected that r10k would always shell out to
+Git. This has been fixed by extracting the shellout specific code to a library
+and defining a common interface for Git classes so that different Git
+implementations can be used. The existing shellout implementation has been
+renamed to 'shellgit' and should be functionally equivalent to the Git
+implementation in previous versions of r10k.
+
+(RK-55, GH-355) Prevent warnings on Ruby 2.2
+
+Ruby 2.2 generates warnings when comparing values and `#<=>` raises an
+exception; the code triggering this behavior has been cleaned up to prevent a
+warning from being logged.
+
+(RK-65) Switch to using the semantic_puppet gem
+
+R10k is switching from the old vendored copy of 'SemVer' that was stolen from
+Puppet to the 'semantic_puppet' gem, which is the library that Puppet now uses
+for version comparisons and parsing.
+
+(RK-48) Ignore deleted versions when fetching latest module version
+
+When looking up the latest version of a module on the Forge, if the latest
+version had been deleted r10k would try to install that deleted version anyways.
+This has been fixed so that all deleted module releases will be ignored.
+
 1.4.2
 -----
 

data/CONTRIBUTING.mkd

@@ -8,7 +8,7 @@ tracker][github-issue-tracker]. When filing a bug report or feature request,
 please spend a moment to look for existing issue reports and update those with
 additional information if possible.
 
-[github-issue-tracker]: https://github.com/adrienthebo/r10k/issues
+[github-issue-tracker]: https://github.com/puppetlabs/r10k/issues
 
 ### Bug reports
 
@@ -57,7 +57,7 @@ Contributing fixes and improvements to the core documentation is a great way to
 help improve R10K. Improvements to the FAQ, example code, and so forth can go a
 long way towards making R10K more approachable for new users and existing users
 alike. For more information about the submission process please see the section
-on [code contributions][#code-contributions].
+on [code contributions](#code-contributions).
 
 ## Code contributions
 
@@ -75,7 +75,7 @@ guide.
     * To create a new topic branch; `git checkout -b fix/master/my_contribution master`
     * Please avoid working directly on the 'master' branch.
   * Make commits of logical units.
-    * Your commit should include a high level description of your work in HISTORY.textile 
+    * Your commit message should include a high level description of your work.
   * Check for unnecessary whitespace with `git diff --check` before committing.
 
 ```

data/Gemfile

@@ -2,6 +2,10 @@ source 'https://rubygems.org'
 
 gemspec
 
+group :extra do
+  gem 'rugged', '~> 0.21.4' if RUBY_VERSION > '1.8.7'
+end
+
 group :development do
   gem 'simplecov', '~> 0.9.1' if RUBY_VERSION > '1.8.7'
 end

data/README.mkd

@@ -3,16 +3,15 @@ r10k
 
 Puppet environment and module deployment
 
-[![Build Status](https://travis-ci.org/adrienthebo/r10k.png?branch=master)](https://travis-ci.org/adrienthebo/r10k)
+[![Build Status](https://travis-ci.org/puppetlabs/r10k.png?branch=master)](https://travis-ci.org/puppetlabs/r10k)
 
 Description
 -----------
 
-[librarian-puppet]: https://github.com/rodjek/librarian-puppet
 [workflow]: http://puppetlabs.com/blog/git-workflow-and-puppet-environments/
 
 R10k provides a general purpose toolset for deploying Puppet environments and
-modules. It implements the [Puppetfile][librarian-puppet] format and provides a native
+modules. It implements the [Puppetfile](doc/puppetfile.mkd) format and provides a native
 implementation of Puppet [dynamic environments][workflow].
 
 Requirements
@@ -20,7 +19,7 @@ Requirements
 
 R10k supports the following Ruby versions:
 
-  - 1.8.7 (POSIX minimum version)
+  - 1.8.7 (POSIX minimum version, deprecated)
   - 1.9.3 (Windows minimum version)
   - 2.0.0
   - 2.1.0
@@ -33,6 +32,14 @@ environments and modules.
   - Git is required for creating environments and modules from Git
   - SVN is required for creating environments and modules from SVN
 
+### Ruby 1.8.7 is deprecated
+
+As of r10k 1.5.0, support for Ruby 1.8.7 is deprecated. Because of the
+increasing cost of supporting Ruby 1.8.7, issues only affecting 1.8.7 will be
+fixed but will have a lower priority.
+
+Support for 1.8.7 will be dropped entirely in r10k 2.0.0.
+
 Installation
 ------------
 
@@ -57,7 +64,7 @@ installation:
 If you have more specific needs or plan on modifying r10k you can run it out of
 a git repository using Bundler for dependencies:
 
-    git clone git://github.com/adrienthebo/r10k
+    git clone git://github.com/puppetlabs/r10k
     cd r10k
     bundle install
     bundle exec r10k help
@@ -73,7 +80,7 @@ information see the topic specific documentation:
   * [Environment Deployment Documentation](doc/dynamic-environments.mkd)
   * [Quickstart](doc/dynamic-environments/quickstart.mkd)
   * [Common Patterns](doc/common-patterns.mkd)
-  * [Workflow Guide](doc/workflow-guide.mkd)
+  * [Workflow Guide](doc/dynamic-environments/workflow-guide.mkd)
 
 For more general questions, see the [FAQ](doc/faq.mkd).
 

data/doc/dynamic-environments/configuration.mkd

@@ -7,11 +7,27 @@ deployed.
 Config file location
 --------------------
 
-By default r10k will try to read `/etc/r10k.yaml` for configuration settings.
-You can specify an alternate configuration file by specifying the `--config`
-option, like so:
+### Manual configuration
 
-    r10k deploy -c /srv/puppet/r10k.yaml
+An explicit configuration file location be specified by providing the `--config`
+option to `r10k deploy`, like so:
+
+    r10k deploy --config /srv/puppet/r10k.yaml [...]
+
+### Automatic configuration
+
+If an explicit configuration file is not given, r10k will search the following
+locations for a configuration file.
+
+  * `{current working directory}/r10k.yaml`
+  * `/etc/puppetlabs/r10k/r10k.yaml` (1.5.0 and later)
+  * `/etc/r10k.yaml` (deprecated in 1.5.0)
+
+In 1.5.0 r10k added `/etc/puppetlabs/r10k/r10k.yaml` to the configuration search
+path. The old location, `/etc/r10k.yaml` has been deprecated in favor of the new
+location. If both `/etc/puppetlabs/r10k/r10k.yaml` and `/etc/r10k.yaml` exist
+and explicit configuration file has not been given, r10k will log a warning and
+use `/etc/puppetlabs/r10k/r10.yaml`.
 
 General options
 ---------------
@@ -36,6 +52,22 @@ The cachedir setting defaults to `~/.r10k`. If the HOME environment variable is
 unset r10k will assume that r10k is being run with the Puppet [`prerun_command`][prerun_command]
 setting and will set the cachedir default to `/root/.r10k`.
 
+### git
+
+The 'git' setting is a hash that contains Git specific settings.
+
+#### provider
+
+The provider option determines which Git provider should be used.
+
+```yaml
+git:
+  provider: rugged # one of shellgit, rugged
+```
+
+See the [git provider documentation](../git/providers.mkd) for more information
+regarding Git providers.
+
 Deployment options
 ------------------
 
@@ -84,7 +116,7 @@ from. It may be any valid URL that the source may check out or clone. The remote
 must be able to be fetched without any interactive input, eg usernames or
 passwords cannot be prompted for in order to fetch the remote.
 
-```
+```yaml
 ---
 sources:
   mysource:
@@ -104,9 +136,11 @@ sources:
     basedir: '/etc/puppet/environments'
 ```
 
-R10k will not check to make sure that different sources collide in a single base
-directory; if you are using multiple sources you are responsible for making sure
-that they do not overwrite each other. See also the prefix setting.
+If two different sources have the same basedir, it's possible for them to create
+two separate environments with the same name and file path. If this occurs r10k
+will treat this as a fatal error and will abort. To avoid this, use prefixing on one
+or both of the sources to make sure that all environment names are unique. See
+also the [prefix](#prefix) setting.
 
 ### prefix
 
@@ -121,6 +155,10 @@ sources:
     basedir: '/etc/puppet/environments'
     prefix: true # All environments will be prefixed with "mysource_"
 ```
+#### prefix behaviour
+* if `true` environment folder will be prefixed with the name of the source.
+* if `false` (default) environment folder will not be prefixed
+* if `String` environment folder will be prefixed with the `prefix` value.
 
 Examples
 --------
@@ -190,3 +228,34 @@ This will create the following directory structure:
 `-- dev_loadtest     # dev repository, loadtest branch
 ```
 
+#### Multiple tenancy with external hieradata
+
+If hiera data is in a separate repository from your control repository, you
+must override the `prefix` so environment folders line up in both directories:
+
+```yaml
+---
+  sources:
+    main:
+      app1_data:
+        remote: 'git://git-server.site/my-org/app1-hieradata'
+        basedir: '/etc/puppet/hieradata'
+        prefix: "app1"
+      app1_modules:
+        remote: 'git://git-server.site/my-org/app1-puppet-modules'
+        basedir: '/etc/puppet/environments'
+        prefix: "app1"
+```
+
+
+This will create the following directory structure:
+
+```
+/etc/puppet/environments
+|-- app1_production  # app1 modules repository, production branch
+|-- app1_develop     # app1 modules repository, develop branch
+
+/etc/puppet/hieradata
+|-- app1_production  # app1 data repository, production branch
+|-- app1_develop     # app1 data repository, develop branch
+```

data/doc/dynamic-environments/git-environments.mkd

@@ -12,7 +12,7 @@ The core idea of dynamic environments is that you should be able to manage your
 Puppet modules in the same manner that you would manage any other code base. It
 builds on top of Git topic branch model.
 
-[git-topic-branching]: http://git-scm.com/book/en/Git-Branching-Branching-Workflows#Topic-Branches "Git Topic Branches"
+[git-topic-branching]: http://git-scm.com/book/en/Git-Branching-Branching-Workflows#Topic-Branches
 
 One of the most prevalent ways of using Git relies on using [topic branches][git-topic-branching].
 Whenever changes need to be made that need to be reviewed or tested before going
@@ -22,7 +22,7 @@ is completed it is merged into a "master" or "production" branch. This is very
 powerful because it allows any number of people to rapidly develop features in
 isolation and merge features in a single operation.
 
-The dynamic environment model extends extends this git branching strategy to
+The dynamic environment model extends this git branching strategy to
 your live Puppet masters. It creates a mapping between Git branches and Puppet
 environments so that you can use the Git branching model and have that be
 seamlessly reflected in Puppet environments. This means that creating a new Git
@@ -45,6 +45,16 @@ future r10k runs. Finally, if there are directories that do not match existing
 branches, r10k will assume that the branches for those environments were delete
 and will remove those environments.
 
+r10k will need to be be able to authenticate with each repository. Most Git
+systems support authentication with SSH keys. GitHub calls them [deploy
+keys][github-deploy-keys]. Bitbucket calls them [deployment
+keys][bitbucket-deployment-keys]. Stash calls them [SSH access
+keys][stash-access-keys].
+
+[github-deploy-keys]: https://developer.github.com/guides/managing-deploy-keys/#deploy-keys
+[bitbucket-deployment-keys]: https://confluence.atlassian.com/display/BITBUCKET/Use+deployment+keys
+[stash-access-keys]: https://confluence.atlassian.com/display/STASH/SSH+access+keys+for+system+use
+
 Configuration
 -------------