r10k 2.0.3 → 2.1.0

checksums.yaml

@@ -1,15 +1,15 @@
 ---
 !binary "U0hBMQ==":
   metadata.gz: !binary |-
-    ZGIwYzM5NTYxYjdiMGU3ZjgwNzgxYjYyODIwMDVlMWU2MmQ1ZDYyYw==
+    ZWM0YTE3ZTkwYmEyZDkxZTE0ZDdlMzUyOGEzODI0Zjk1ZGU4OWI4MQ==
   data.tar.gz: !binary |-
-    ZmQ2NjM2NTFhNzM0NzM5ZjllNDY3MjFiNWFhM2FjYmFiZGM2YzZlZQ==
+    ZjM0ZGE5NDk2ZjE1YjM0ZDA0NjU4MzE5ZmRmYmI5NzFkYjkxMjViMw==
 SHA512:
   metadata.gz: !binary |-
-    ZjU1Mzc1ODgyMTlmNWRlNzYwMTU1ZGNkM2Y3NjM1ZDgzMTMwODgzNmVmYTc0
-    NTBjMzAwYTg3MzUyYzczNTdkNjJlNDAxZjk2NzFkYzMzMzRlYWQ0N2ZhODFl
-    Mjg3ZWExNTBlMTJlMWQxODJiODQ5ZWQyMDY3MGQ2YmQ5NDA4NTE=
+    MTc2MTFjZjBiNTljNGY0NzQyZDhiNDllOTFmYjEyOTY5OTA1MzViMDlhM2I4
+    MTczZjQwMWE1NjI3MTEwNjllYTlkOTcyZDgyZjFkZWI2MTYxODc5ZDhkOGZl
+    YjQxZDE2YmY4ZTc1YjY3MGJjMjE4ZWMyYzk1YmQ3NGJmZDgyZjk=
   data.tar.gz: !binary |-
-    NjhlNzAxZWJiYzcxMzYwNzIwNDFkODI2MGVkNTg4ZWI1MDAyYjkyMWEzYzJj
-    ZmI1NjJkMjQ2OTdlYjQ2YTIwNTZlMzJmM2ZhMDc0OTlhMDM2MjdmOWZkNmZk
-    ZjcyOTNlN2JhZDYwMjkyNzkyZTE0MzY1MDliNDZmY2NkNDJhYjY=
+    MGJlYWJlZmU0MzczZmNjZmY4MTA1M2RhMGM2YjY3ODM5NjkwNGRhMzE5Y2Zj
+    YjMzZTU5ODk0NGIwYmFiMTVjMWJjMGY0OTIyMTFiMzcwZjUzN2JhNWE3MzY3
+    N2JlNTZlMWQ1ZGE1YTE1YjgxZDQwMWM1NDZkMGMwZDM5NjQzYjI=

data/CHANGELOG.mkd

@@ -1,6 +1,156 @@
 CHANGELOG
 =========
 
+2.1.0
+-----
+
+2015/10/28
+
+### Thanks
+
+Thanks to the following contributors for their work on this release:
+
+  * [Abel Paz](https://github.com/apazga) fixing some broken links in the Git Environments documentation
+  * [Alex Rowley](https://github.com/rowleyaj) for surfacing the --puppetfile and --module options for
+    `r10k puppetfile install`
+  * [Austin Blatt](https://github.com/austb) (our summer intern!) for extracting the vendored Puppetforge
+    code and pushing it into the upstream puppet_forge gem.
+  * [Branan Riley](https://github.com/branan) for updating the quickstart guide for Puppet 4.
+  * [Darrell](https://github.com/darrell) for adding documentation for the :local module type
+  * [David Pashley](https://github.com/dpash) for documenting potential issues with Rugged/SSH on Ubuntu
+  * [Dennis](https://github.com/pskrz) for clarifying path information in the quickstart guide
+  * [E. Dunham](https://github.com/edunham) for adding an entry to the FAQ to explain the name of r10k
+  * [Kirill Kondratenko](https://github.com/cybem) for contributing the :local module type (RK-149)
+  * [Louis Mayorga](https://github.com/lmayorga1980) for finding and reporting RK-143, and testing fixes
+    for that issue.
+  * [Mark McKinstry](https://github.com/mmckinst) for fixing some broken links in the FAQ
+  * [Patrick Robinson](https://github.com/nemski) for submitting GH-516 to catch and handle
+    ArgumentErrors raised when loading Puppetfiles
+  * [Thomas Lapinski](https://github.com/thlapin) for fixing a bug where switching from a Git module to
+    a Forge module would incorrectly leave the Git module version installed.
+  * [Thomas Mueller](https://github.com/vinzent) for updating the r10k homepage link in the gemspec.
+
+### User notes
+
+(RK-70) Expose Forge baseurl setting in r10k.yaml
+
+The Puppet Forge URL can now be changed from the default forgeapi.puppetlabs.com to a user specified value.
+This allows users to use private/testing Puppet Forge instances. See
+https://github.com/puppetlabs/r10k/blob/master/doc/dynamic-environments/configuration.mkd#baseurl
+for documentation on using the baseurl setting.
+
+(RK-96) Correctly switch from Forge modules to Git modules
+
+If a Git version of a module had been installed via a Puppetfile and the Puppetfile entry was updated to
+use the Forge version of the module, r10k would only check the metadata - and if the metadata of the Git
+version matched then r10k would assume that the Forge module was installed. This has been fixed so that
+when checking the status of a Forge module r10k specifically looks for a `.git` directory and considers
+the module mismatched when the directory is present, which means that it will correctly replace a Git
+version of a module with the Forge version.
+
+(RK-142) Add machine readable output for `r10k deploy display`
+
+The output format for `r10k deploy display` was a YAML-like, janky hand rolled version; this has been
+replaced with the ability to specify an actual data format for the output. The default output type is
+now YAML.
+
+(GH-477) Add `--fetch` option for `r10k deploy display`
+
+When `r10k deploy display` is run it might be useful to see which environments don't yet exist; the
+added `--fetch` option allows r10k to update the environment sources to check for missing environments
+when displaying environments.
+
+(RK-143) Use argument vector when executing commands on Windows
+
+R10k used the `Open3.capture3` call, and used to concatenate the string to execute and join it with
+spaces. However Windows loves including spaces in file names, which creates all sort of weird behavior
+especially when running shell commands over SSH to Windows. This has been fixed to use the argument
+vector form of `capture3` to avoid these issues.
+
+(RK-149) Add :local module type
+
+The new :local module type allows users to add modules to the Puppetfile that are included in a control
+repository, so that modules can be kept in the Puppetfile moduledir without them being destroyed.
+
+(CODEMGMT-345) Write deploy signatures to a file
+
+If the code deployed by r10k is copied to another location without the cached repos, it becomes impossible
+to use Git to interact with the repository and see which version of code r10k deployed. R10k now creates a
+`.r10k-deploy.json` file that records the time and SHA of the last code deployment.
+
+(RK-80) Support non SSH key based authentication for rugged provider
+
+An error in how the Rugged Git provider provided SSH credentials prevented HTTPS urls requiring authentication
+to function. This has been fixed so that only SSH urls are provided ssh private key credentials, and HTTPS
+urls can provided a username and password in the URL for authentication.
+
+(RK-90) Don't raise an error when forge modules are duplicated
+
+A bug in how r10k Forge modules created caused r10k to crash when two forge modules tried to create the same
+directory. This has been fixed and restores the old behavior of letting the last module specified win.
+
+In the long term trying to specify two modules with the same name will produce an error, but in the mean time
+this fix restores existing behavior so that r10k won't roll over and die on error.
+
+(RK-120) Enable --config as top level command option
+
+R10k was originally built with environment deployment in mind, but the `r10k puppetfile` subcommand made this
+assumption faulty. However the way the config file was loaded only worked with `r10k deploy` subcommands,
+which prevented users from using a custom cachedir or use other settings when running `r10k puppetfile`. This has
+been corrected so that r10k can always read a config file when running any subcommand.
+
+(RK-175) Don't crash on unresolvable Rugged ref
+
+When r10k was using the rugged provider, checking out an unresolvable ref would throw a TypeError because r10k
+didn't always ensure that the ref was resolvable. This has been fixed so that r10k explicitly ensures that refs
+can be resolved before it checks them out.
+
+(RK-174) Always ensure alternates file is up to date
+
+When r10k created a Git working repository it permanently linked the repository to a cached Git repository via
+the Git alternates file. However if the cachedir setting changed, existing repositories would still reference
+the old path which could very badly break Git. This has been fixed so that r10k always ensures that the alternates
+file is up to date before any Git operations can happen, so that changing the cachedir will not break Git repositories.
+
+(RK-169) Print validation failures for invalid configs
+
+R10k used to have very lax handling of config file input, which would cause crashes during r10k runtime. This
+has been fixed so that r10k validates all configuration before it runs and prints out all configuration
+validation errors when validation fails.
+
+(RK-21) Indicate error when deploying non-existent environment
+
+When r10k was deploying a specific list of environments, it would skip any existing environment that didn't match
+the desired list of environments. However this meant that deploying a non-existent environment would not
+deploy any environments and r10k would silently exit with an exit code of 0. This has been fixed so that
+when r10k deploys a list of environments it makes sure that each environment can be updated, and when requested
+environments are missing it logs an error and exits with a non-zero exit code.
+
+(RK-163) Add deploy/write_lock setting
+
+Users can now prevent `r10k deploy` commands that change environments and modules from running via a config
+option; this allows users to lock out code deployments at certain times (code freezes and times outside of
+maintenance windows.)
+
+(GH-516) Handle ArgumentError when loading Puppetfiles
+
+The `r10k puppetfile check` command didn't output a very useful error when a module was given the wrong number
+of arguments; this has been fixed so that r10k catches ArgumentErrors raised while loading a Puppetfile
+and wraps it so that more information is added to the error message.
+
+(GH-469) Add `--puppetfile` and `--moduledir` options to `r10k puppetfile install` subcommand
+
+The `r10k puppetfile install` subcommand was able to set a custom puppetfile path and moduledir location
+via environment variables to match librarian-puppet, but they accidentally didn't match the librarian-puppet
+semantics - and environment variables aren't very nice to expose for configuring and application. The
+`r10k puppetfile install` subcommand now supports command line flags to set these options.
+
+(CODEMGMT-339) Add command line option to set cachedir
+
+There are some scenarios where r10k needs to be run with a common config file, but on a specific basis
+may need to set a custom cachedir. To make this work `r10k deploy` now supports a `--cachedir` setting
+for these temporary overrides.
+
 2.0.3
 -----
 

data/README.mkd

@@ -41,7 +41,7 @@ For general use, you should install r10k from Ruby gems:
     gem install r10k
     r10k help
 
-### Puppet Enterprise
+### Puppet Enterprise 3.x
 
 Puppet Enterprise bundles a copy of Ruby, so if you do not want to use the
 system version of Ruby with r10k, you need to use the bundled PE gem command for
@@ -50,6 +50,15 @@ installation:
     /opt/puppet/bin/gem install r10k
     r10k help
 
+### Puppet 4
+
+Puppet 4 bundles a copy of Ruby, so if you do not want to use the
+system version of Ruby with r10k, you need to use the bundled puppet gem command
+for installation.
+
+    /opt/puppetlabs/puppet/bin/gem install r10k
+    /opt/puppetlabs/puppet/bin/r10k help
+
 ### Bundler
 
 If you have more specific needs or plan on modifying r10k you can run it out of

data/doc/dynamic-environments/configuration.mkd

@@ -86,6 +86,15 @@ forge:
 If no proxy is given, r10k will check the environment variables 'HTTPS_PROXY',
 'https_proxy', 'HTTP_PROXY', and 'http_proxy' in that order for a proxy URL.
 
+#### baseurl
+
+The 'baseurl' setting indicates where Forge modules should be installed from.
+This defaults to 'https://forgeapi.puppetlabs.com'
+
+```yaml
+forge:
+  baseurl: 'https://private-forge.mysite'
+```
 
 Deployment options
 ------------------
@@ -121,6 +130,28 @@ sources:
     # Source settings follow
 ```
 
+### deploy
+
+The `deploy` setting is a new top level setting for controlling how r10k deploys
+behave. At this point only new settings are included under this setting, but in
+the long term the current top level deploy settings will be moved under
+`deploy`.
+
+#### write_lock
+
+The `write_lock` setting allows administrators to temporarily disallow r10k code
+deploys without having to remove the r10k configuration entirely. This can be
+useful to prevent r10k deploys at certain times or prevent r10k from interfering
+with a common set of code that may be touched by multiple r10k configurations.
+
+```yaml
+---
+
+deploy:
+  write_lock: "Deploying code is disallowed until the next maintenance window (2038-01-19)"
+
+```
+
 Source options
 --------------
 

data/doc/dynamic-environments/quickstart.mkd

@@ -3,22 +3,23 @@
 This intent of this document is to serve as a basic guide for getting started with r10k and a fresh Puppet installation, including the following:
 
 * Installing Puppet and its dependencies
-* Installing Hiera and its dependencies.
 * Installing r10k and its dependencies
 * Configuring all components to support r10k
 * Configuring your git repository and initial files
 
+This document is based around Puppet 4+, specifically using the Puppet Collection repositories provided by Puppet Labs.
+
 # Pre-Requisites
 
-* Clean install of CentOS 6.5 or Debian 7.0 with root access / sudo rights. 
+* Clean install of CentOS 7 or Debian 7.0 with root access / sudo rights.
 * Clean github repository with a deploy key generated by the server above.
 
-# Installing Puppetmaster on Centos 6.5
+# Installing Puppetmaster on Centos 7
 
 Install and enable the official Puppet Labs package repositories.
 
 ```
-rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
+yum localinstall http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
 ```
 
 Clean all yum data and rebuild the metadata cache.
@@ -30,7 +31,7 @@ yum clean all && yum makecache
 Install the Puppet master.
 
 ```
-yum install puppet-server
+yum install puppetserver
 ```
 
 # Installing Puppetmaster on Debian 7
@@ -38,75 +39,48 @@ yum install puppet-server
 Install and enable the official Puppet Labs package repositories
 
 ```
-wget http://apt.puppetlabs.com/puppetlabs-release-wheezy.deb
-dpkg -i puppetlabs-release-wheezy.deb
+wget http://apt.puppetlabs.com/puppetlabs-release-pc1-wheezy.deb
+dpkg -i puppetlabs-release-pc1-wheezy.deb
 apt-get update
 ```
 
 Install the Pupppet master
 ```
-apt-get install puppetmaster-passenger=3.7.1-1puppetlabs1
+apt-get install puppetserver
 ```
-Above command will install 3.7.1-1puppetlabs but there are many versions to choose. To see available versions run: apt-cache show puppetmaster-passenger|grep ^Version
 
-# Configuring Puppet 
+# Configuring Puppet
 
-Configure the Puppet master by editing `/etc/puppet/puppet.conf` and ensuring it has the following contents:
+Configure the Puppet master by editing `/etc/puppetlabs/puppet/puppet.conf` and ensuring it has the following contents:
 
 ```
 [main]
-    # The Puppet log directory.
-    # The default value is '$vardir/log'.
-    logdir = /var/log/puppet
-
-    # Where Puppet PID files are kept.
-    # The default value is '$vardir/run'.
-    rundir = /var/run/puppet
-
-    # Where SSL certificates are kept.
-    # The default value is '$confdir/ssl'.
-    ssldir = $vardir/ssl
-
     dns_alt_names = $_Insert FQDN of Puppet Master Here_$
 
-    environmentpath = $confdir/environments
-
 [agent]
-    # The file in which puppetd stores a list of the classes
-    # associated with the retrieved configuratiion.  Can be loaded in
-    # the separate ``puppet`` executable using the ``--loadclasses``
-    # option.
-    # The default value is '$confdir/classes.txt'.
-    classfile = $vardir/classes.txt
-
-    # Where puppetd caches the local configuration.  An
-    # extension indicating the cache format is added automatically.
-    # The default value is '$confdir/localconfig'.
-    localconfig = $vardir/localconfig
-
     server = $_Insert FQDN of Puppet Master Here_$
 ```
 
 Restart the Puppet master service.
 
 ```
-service puppetmaster restart
+service puppetserver restart
 ```
 
 Ensure the certificate for the Puppet master was created.
 
 ```
-# puppet cert list --all
+# /opt/puppetlabs/bin/puppet cert list --all
 
 + "puppet-master.domain.local" (SHA256) 3F:F3:63:BB:EE:57:46:A4:7B:03:AB:9D:FD:97:0F:8F:73:87:40:3B:6D:E5:DC:FC:C3:49:F5:C9:B6:F4:DE:B8 (alt names: "DNS:puppet-master.domain.local")
 ```
 
-Notice for Debian users: apt post-configure will build the certificate for the server BEFORE you configure it. Therefore you should rebuild your certs after done with /etc/puppet/puppet.conf configuration.
+Notice for Debian users: apt post-configure will build the certificate for the server BEFORE you configure it. Therefore you should rebuild your certs after done with /etc/puppetlabs/puppet/puppet.conf configuration.
 To do so you need to remove old certs and restart puppetmaster:
 ```
-service puppetmaster stop
-find $(puppet master --configprint ssldir) -name "$(puppet master --configprint certname).pem" -delete
-puppet master --no-daemonize --verbose
+service puppetserver stop
+find $(/opt/puppetlabs/bin/puppet master --configprint ssldir) -name "$(/opt/puppetlabs/bin/puppet master --configprint certname).pem" -delete
+service puppetserver start
 ```
 
 # Install and Configure R10k
@@ -114,54 +88,34 @@ puppet master --no-daemonize --verbose
 Install r10k via Ruby Gems.
 
 ```
-gem install r10k
+/opt/puppetlabs/puppet/bin/gem install r10k
 ```
 
-Configure r10k by editing `/etc/puppetlabs/r10k/r10k.yaml` and ensuring it has the following contents:
+Configure r10k by creating the following directory structure and file `/etc/puppetlabs/r10k/r10k.yaml` and ensuring it has the following contents:
 
 ```
 # The location to use for storing cached Git repos
-:cachedir: '/var/cache/r10k'
+:cachedir: '/opt/puppetlabs/r10k/cache'
 
 # A list of git repositories to create
 :sources:
   # This will clone the git repository and instantiate an environment per
-  # branch in /etc/puppet/environments
+  # branch in /etc/puppetlabs/code/environments
   :my-org:
     remote: 'git@github.com:$_Insert GitHub Organization Here_$/$_Insert GitHub Repository That Will Be Used For Your Puppet Code Here_$'
-    basedir: '/etc/puppet/environments'
-```
-# Install and Configure Hiera
-
-Hiera is installed as part of the Puppet master installation.
-
-Configure Hiera by editing `/etc/hiera.yaml` and ensuring it has the following contents:
-
+    basedir: '/etc/puppetlabs/code/environments'
 ```
----
-:backends:
-  - yaml
-:hierarchy:
-  - "nodes/%{fqdn}"
-  - common
-
-:yaml:
-   :datadir: /etc/puppet/environments/%{environment}/hiera
-```
-
-Create a symlink to the Hiera configuration in the Puppet directory.
-
-```
-ln -s /etc/hiera.yaml /etc/puppet/hiera.yaml
-```
-
 # Configure Puppet Code Repository
 
 Populate the repository by cloning it locally and performing each of the following actions within it:
 
+Note that puppet defaults to the `production` environnment. You may wish to change your default git
+branch from `master` to `production` in order to match this. Alternatively, you can set your agents'
+environment to `master`.
+
 ```
-mkdir -p {modules,site/profile/manifests,hiera}
-touch hiera/common.yaml
+mkdir -p {modules,site/profile/manifests,hieradata}
+touch hieradata/common.yaml
 touch site/profile/manifests/base.pp
 touch environment.conf
 touch Puppetfile
@@ -181,7 +135,7 @@ Edit the `site.pp` file and ensure it has the following contents:
 hiera_include('classes')
 ```
 
-Edit the `hiera/common.yaml file and ensure it has the following contents:
+Edit the `hieradata/common.yaml file and ensure it has the following contents:
 
 ```
 ---
@@ -199,7 +153,7 @@ Edit the `Puppetfile` file and ensure it has the following contents:
 forge 'forge.puppetlabs.com'
 
 # Forge Modules
-mod 'puppetlabs/ntp', '3.0.3'
+mod 'puppetlabs/ntp', '4.1.0'
 mod 'puppetlabs/stdlib'
 ```
 
@@ -213,7 +167,7 @@ class profile::base {
 
 Ensure that the user r10k runs as (typically root) can access the git
 repository. See the [git environment guide]
-(doc/dynamic-environments/git-environments.mkd) for more detail.  You can test
+(git-environments.mkd) for more detail.  You can test
 the access by using su/sudo to perform `git clone yourrepoURL` as the correct
 user.
 
@@ -232,13 +186,13 @@ This base will allow us to do all sorts of useful things. Most interesting (to m
 1. Create a new branch of the Puppet code repository
 2. Create your Puppet code in this new branch
 3. Push the new branch up to the repository
-4. Deploy it as a new environment using the `r10k deploy environment -p` command.
+4. Deploy it as a new environment using the `/opt/puppetlabs/puppet/bin/r10k deploy environment -p` command.
 
 From any agent node (including the master), you may run the agent against the new environment by specifying it on the command line. For example, if you create the branch `test`, run puppet as:
 ```
 puppet agent -t --environment test
 ```
-You can also modify the `/etc/puppet/puppet.conf` file on a node and add the environment setting to the agent section to make the change permanent:
+You can also modify the `/etc/puppetlabs/puppet/puppet.conf` file on a node and add the environment setting to the agent section to make the change permanent:
 ```
 ...
 [agent]