r10k 3.5.2

data/doc/dynamic-environments/git-environments.mkd

@@ -0,0 +1,75 @@
+Git Based Dynamic Environments
+==============================
+
+R10k can use Git repositories to implement dynamic environments. You can create,
+update, and delete Puppet environments automatically as part of your normal Git
+workflow.
+
+Dynamic Environments in a nutshell
+----------------------------------
+
+The core idea of dynamic environments is that you should be able to manage your
+Puppet modules in the same manner that you would manage any other code base. It
+builds on top of Git topic branch model.
+
+[git-topic-branching]: http://git-scm.com/book/en/Git-Branching-Branching-Workflows#Topic-Branches
+
+One of the most prevalent ways of using Git relies on using [topic branches][git-topic-branching].
+Whenever changes need to be made that need to be reviewed or tested before going
+live, they should be done in a different, short lived branch called a topic
+branch. Work can be freely done on a topic branch in isolation and when the work
+is completed it is merged into a "master" or "production" branch. This is very
+powerful because it allows any number of people to rapidly develop features in
+isolation and merge features in a single operation.
+
+The dynamic environment model extends this git branching strategy to
+your live Puppet masters. It creates a mapping between Git branches and Puppet
+environments so that you can use the Git branching model and have that be
+seamlessly reflected in Puppet environments. This means that creating a new Git
+branch creates a new Puppet environment, updating a Git branch will update that
+environment, and deleting a Git branch will remove that environment.
+
+R10k supports both [directory and config file environments](https://puppet.com/docs/puppet/latest/env_environments.html).
+Ensure that the basedir for your sources and your puppet config align.
+
+How it works
+------------
+
+R10k works by tracking the state of your Git repository or repositories. Each
+repository's branches will be cloned into a directory with a matching name,
+creating a Puppet environment for the branch. If a repository includes a
+Puppetfile, such as the control repo, the Forge modules and Git/SVN
+repositories described within will be cloned as well, into the same directories.
+Subsequent changes to the branches will be kept in sync on the filesystem by
+future r10k runs. Finally, if there are directories that do not match existing
+branches, r10k will assume that the branches for those environments were delete
+and will remove those environments.
+
+r10k will need to be be able to authenticate with each repository. Most Git
+systems support authentication with SSH keys. GitHub calls them [deploy
+keys][github-deploy-keys]. Bitbucket calls them [deployment
+keys][bitbucket-deployment-keys]. Stash calls them [SSH access
+keys][stash-access-keys].
+
+[github-deploy-keys]: https://developer.github.com/guides/managing-deploy-keys/#deploy-keys
+[bitbucket-deployment-keys]: https://confluence.atlassian.com/display/BITBUCKET/Use+deployment+keys
+[stash-access-keys]: https://confluence.atlassian.com/display/STASH/SSH+access+keys+for+system+use
+
+Configuration
+-------------
+
+The following configuration options can be specified for Git based environment
+sources.
+
+### invalid_branches:
+
+This setting specifies how Git branch names that cannot be cleanly mapped to
+Puppet environments will be handled.
+
+Valid values:
+
+  * 'correct_and_warn': Non-word characters will be replaced with underscores
+    and a warning will be emitted. (Default)
+  * 'correct': Non-word characters will silently be replaced with underscores.
+  * 'error': Branches with non-word characters will be ignored and an error will
+    be emitted.

data/doc/dynamic-environments/introduction.mkd

@@ -0,0 +1,69 @@
+Dynamic Environments
+====================
+
+One of the most important functions of r10k is its ability to dynamically manage
+your Puppet environments.
+
+When environments were originally built into Puppet they were meant to be static
+in nature. Each environment had to be defined beforehand in the master's
+puppet.conf file in a section, like so:
+
+```ini
+[master]
+# Environment independent settings
+vardir = '/var/lib/puppet'
+
+[production]
+modulepath = '/etc/puppet/environments/production/modules'
+
+[testing]
+modulepath = '/etc/puppet/environments/testing/modules'
+
+[development]
+modulepath = '/etc/puppet/environments/development/modules'
+```
+
+Static Puppet environments were frequently used to implement a pipeline for
+developing Puppet code. New Puppet code would be developed and deployed to the
+development environment, pushed to testing for validation, and then finally
+pushed to the production for general deployment.
+
+This static nature of Puppet environments turned out to be inflexible in
+practice. With a predefined list of environments it could be very cumbersome to
+develop on different parts of a Puppet codebase in isolation; you could either
+develop multiple features in the same environment and risk cross pollution, or
+manually create new environments every time you needed isolation.
+
+Dynamic environments work by dynamically determining the settings for Puppet
+environment when the environment is used, rather than by defining an explicit
+section in puppet.conf. This works by making the current environment (in the
+'$environment' variable) part of of the path to environment specific settings,
+like modulepath, manifest, and so forth.
+
+```ini
+[master]
+environmentpath = $confdir/environments'
+```
+
+Running `puppet agent -t --environment myenv` will cause $environment to be
+expanded to 'myenv', so the modulepath for that environment will be set to 
+'/etc/puppet/environments/myenv/modules'.
+
+This approach of allowing environments to be defined on the fly is a complete
+reversal of the original architecture of environments. This approach means that
+it can be very easy to create new environments, update existing environments,
+and remove environments that aren't needed anymore. It's common practice
+to create a temporary environment to test an idea and destroy it shortly after.
+R10k is designed to enable this sort of fluid workflow.
+
+R10k predominantly uses version control systems to implement dynamic
+environments. This works by inspecting the VCS repositories containing your
+Puppet code and checking out that code on your masters so, that there's a 1:1
+connection between a branch in your VCS repository and a Puppet environment on
+your masters. This approach allows you to define the way you want to work and
+use that with your chosen VCS, and r10k will make Puppet implement that
+workflow.
+
+Different version control systems will implement dynamic environments in
+slightly different ways; check out the VCS specific documentation for more
+information.

data/doc/dynamic-environments/master-configuration.mkd

@@ -0,0 +1,40 @@
+Puppet master configuration
+===========================
+
+In order to use environments, your Puppet masters will need to be
+configured to load manifests and modules relative to the requested environment.
+
+This is the default behavior in Puppet 4+
+
+This behavior is controled by the following settings
+(listed with their default values):
+
+```
+codedir = /etc/puppetlabs/code
+environmentpath = $codedir/environments
+basemodulepath = $codedir/modules:/opt/puppetlabs/puppet/share/modules
+```
+
+The environment requested by the agent or assigned to it by an ENC is looked
+for at `$environmentpath/$environment`.
+
+That environment may have an environment.conf file in its base directory that
+specifies its own modulepath. If not, the default computed modulepath for that
+environment is `$environmentpath/$environment/modules:$basemodulepath`.
+
+These configuration variables are documented at
+[Puppet Configuration Reference](https://puppet.com/docs/puppet/latest/configuration.html)
+and their interaction in forming the modulepath is documented at
+[Directories and the Modulepath](https://puppet.com/docs/puppet/latest/dirs_modulepath.html).
+More information can be found about environments in general at
+[Creating Environmnets](https://puppet.com/docs/puppet/latest/environments_creating.html).
+
+The evolution to using environments in this way was a gradual one from community
+conventions to supported feature and there were several intermediate stages that
+had different configuration requirements within the puppet.conf. Some Puppet
+documentation from the 3.x and 4.x series may no longer be applicable to the
+above which solidified in Puppet 4.x.
+
+If you need to upgrade away from any intermediate setup see
+[Environments in Puppet 3.8](https://puppet.com/docs/puppet/3.8/environments.html)
+for examples that may help.

data/doc/dynamic-environments/quickstart.mkd

@@ -0,0 +1,201 @@
+# Overview
+
+This intent of this document is to serve as a basic guide for getting started with r10k and a fresh Puppet installation, including the following:
+
+* Installing Puppet and its dependencies
+* Installing r10k and its dependencies
+* Configuring all components to support r10k
+* Configuring your git repository and initial files
+
+This document is based around Puppet 4+, specifically using the Puppet Collection repositories provided by Puppet Labs.
+
+# Pre-Requisites
+
+* Clean install of CentOS 7 or Debian 7.0 with root access / sudo rights.
+* Clean github repository with a deploy key generated by the server above.
+
+# Installing Puppetmaster on Centos 7
+
+Install and enable the official Puppet Labs package repositories.
+
+```
+yum localinstall http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
+```
+
+Clean all yum data and rebuild the metadata cache.
+
+```
+yum clean all && yum makecache
+```
+
+Install the Puppet master.
+
+```
+yum install puppetserver
+```
+
+# Installing Puppetmaster on Debian 7
+
+Install and enable the official Puppet Labs package repositories
+
+```
+wget http://apt.puppetlabs.com/puppetlabs-release-pc1-wheezy.deb
+dpkg -i puppetlabs-release-pc1-wheezy.deb
+apt-get update
+```
+
+Install the Pupppet master
+```
+apt-get install puppetserver
+```
+
+# Configuring Puppet
+
+Configure the Puppet master by editing `/etc/puppetlabs/puppet/puppet.conf` and ensuring it has the following contents:
+
+```
+[main]
+    dns_alt_names = $_Insert FQDN of Puppet Master Here_$
+
+[agent]
+    server = $_Insert FQDN of Puppet Master Here_$
+```
+
+Restart the Puppet master service.
+
+```
+service puppetserver restart
+```
+
+Ensure the certificate for the Puppet master was created.
+
+```
+# /opt/puppetlabs/bin/puppet cert list --all
+
++ "puppet-master.domain.local" (SHA256) 3F:F3:63:BB:EE:57:46:A4:7B:03:AB:9D:FD:97:0F:8F:73:87:40:3B:6D:E5:DC:FC:C3:49:F5:C9:B6:F4:DE:B8 (alt names: "DNS:puppet-master.domain.local")
+```
+
+Notice for Debian users: apt post-configure will build the certificate for the server BEFORE you configure it. Therefore you should rebuild your certs after done with /etc/puppetlabs/puppet/puppet.conf configuration.
+To do so you need to remove old certs and restart puppetmaster:
+```
+service puppetserver stop
+find $(/opt/puppetlabs/bin/puppet master --configprint ssldir) -name "$(/opt/puppetlabs/bin/puppet master --configprint certname).pem" -delete
+service puppetserver start
+```
+
+# Install and Configure R10k
+
+Install r10k via Ruby Gems.
+
+```
+/opt/puppetlabs/puppet/bin/gem install r10k
+```
+
+Configure r10k by creating the following directory structure and file `/etc/puppetlabs/r10k/r10k.yaml` and ensuring it has the following contents:
+
+```
+# The location to use for storing cached Git repos
+:cachedir: '/var/cache/r10k'
+
+# A list of git repositories to create
+:sources:
+  # This will clone the git repository and instantiate an environment per
+  # branch in /etc/puppetlabs/code/environments
+  :my-org:
+    remote: 'git@github.com:$_Insert GitHub Organization Here_$/$_Insert GitHub Repository That Will Be Used For Your Puppet Code Here_$'
+    basedir: '/etc/puppetlabs/code/environments'
+```
+# Configure Puppet Code Repository
+
+Populate the repository by cloning it locally and performing each of the following actions within it:
+
+Note that puppet defaults to the `production` environment. You may wish to change your default git
+branch from `master` to `production` in order to match this. Alternatively, you can set your agents'
+environment to `master`.
+
+```
+mkdir -p {modules,site/profile/manifests,hieradata}
+touch hieradata/common.yaml
+touch site/profile/manifests/base.pp
+touch environment.conf
+touch Puppetfile
+touch site.pp
+```
+
+Edit the `environment.conf` file and ensure it has the following contents:
+
+```
+manifest = site.pp
+modulepath = modules:site
+```
+
+Edit the `site.pp` file and ensure it has the following contents:
+
+```
+hiera_include('classes')
+```
+
+Edit the `hieradata/common.yaml file and ensure it has the following contents:
+
+```
+---
+classes:
+ - 'profile::base'
+
+ntp::servers:
+  - 0.us.pool.ntp.org
+  - 1.us.pool.ntp.org
+```
+
+Edit the `Puppetfile` file and ensure it has the following contents:
+
+```
+forge 'forge.puppetlabs.com'
+
+# Forge Modules
+mod 'puppetlabs/ntp', '4.1.0'
+mod 'puppetlabs/stdlib'
+```
+
+Edit the `site/profile/manifests/base.pp` file and ensure it has the following contents:
+
+```
+class profile::base {
+  class { '::ntp': }
+}
+```
+
+Ensure that the user r10k runs as (typically root) can access the git
+repository. See the [git environment guide](git-environments.mkd)
+for more detail.  You can test
+the access by using su/sudo to perform `git clone yourrepoURL` as the correct
+user.
+
+# Summary
+
+We now have the following functional pieces:
+
+1. Puppet master
+2. Hiera
+3. r10k
+4. Puppet code repository
+5. Initial 'profile' named 'base' that will configure NTP on our servers.
+
+This base will allow us to do all sorts of useful things. Most interesting (to me and for the purposes of this tutorial) is the ability to now utilize Git branches to help manage infrastructure as part of your software development lifecycle. Now, when you want to test a new profile, you can do the following:
+
+1. Create a new branch of the Puppet code repository
+2. Create your Puppet code in this new branch
+3. Push the new branch up to the repository
+4. Deploy it as a new environment using the `/opt/puppetlabs/puppet/bin/r10k deploy environment -p` command.
+
+From any agent node (including the master), you may run the agent against the new environment by specifying it on the command line. For example, if you create the branch `test`, run puppet as:
+```
+puppet agent -t --environment test
+```
+You can also modify the `/etc/puppetlabs/puppet/puppet.conf` file on a node and add the environment setting to the agent section to make the change permanent:
+```
+...
+[agent]
+environment = test
+```
+Voila - you're testing code without impacting your production environment!

data/doc/dynamic-environments/svn-environments.mkd

@@ -0,0 +1,45 @@
+SVN Based Dynamic Environments
+==============================
+
+R10k can use SVN repositories to implement dynamic environments. You can create,
+update, and delete Puppet environments automatically as part of your normal SVN
+workflow.
+
+How it works
+------------
+
+R10k implements a branching workflow similar to Git by using the SVN concept of
+branches. SVN repositories must conform to the conventional SVN repository
+structure with the directories trunk/, branches/, and optionally tags/ in the
+root of the repository. R10k maps the trunk/ directory to the production
+environment, and production environment, and branches (directories in branches/)
+are created as environments with the name of the given branch.
+
+Configuration
+-------------
+
+In addition to the settings that all sources support, SVN sources can specify
+the following additional options:
+
+### username/password
+
+If the SVN repository requires credentials, you can supply the `username` and
+`password` options.
+
+Both `username` and `password` must be specified in order to SVN authentication.
+
+**Note**: SVN credentials are passed as command line options, so the SVN
+credentials may be visible in the process table when r10k is running. If you
+choose to supply SVN credentials make sure that the system running r10k is
+appropriately secured.
+
+```yaml
+---
+sources:
+  myenvs:
+    type: svn
+    remote: 'svn://my-svn.server/my-svn-repo'
+    basedir: '/etc/puppet/environments'
+    username: 'azurediamond'
+    password: 'hunter2'
+```

data/doc/dynamic-environments/usage.mkd

@@ -0,0 +1,132 @@
+Usage
+=====
+
+R10k provides fairly fine grained controls over your environments to fit your
+needs. If you want to do a full update of all of your environments and modules
+and don't need it to be done in real time, you can trigger a full update and let
+it run in the background. If you are actively developing code and need to run
+very fast updates of one specific environment, you can do a targeted update of
+that code as well.
+
+All commands that deal with deploying environments are grouped under the `r10k
+deploy` subcommand.
+
+Command line invocation
+-----------------------
+
+### Deploying environments
+
+Recursively update all environments:
+
+    r10k deploy environment --puppetfile
+
+The simplest way to use r10k is by simply updating all environments and modules
+and takes the brute force approach of "update everything, ever." When this
+command is run r10k will update all sources, create new environments and delete
+old environments, and recursively update all environment modules specified in
+environment Puppetfiles. While this is the simplest method for running r10k, is
+is also the slowest by a very large degree because it does the maximum possible
+work. This should not be something you run interactively, or use on a regular
+basis.
+
+- - -
+
+Update environments while avoiding unnecessary recursion:
+
+    r10k deploy environment
+
+This will update existing environments and recursively create new environments.
+Note that when an environment is deployed for the first time, it will
+automatically update all modules as well. For subsequent updates only the
+environment itself will be updated.
+
+- - -
+
+Update a single environment:
+
+    r10k deploy environment my_working_environment
+
+When you're actively developing on a given environment, this is the best way to
+deploy your changes. Note that when an environment is deployed for the first
+time, it will automatically update all modules as well. For subsequent updates
+only the environment itself will be updated.
+
+- - -
+
+Update a single environment and force an update of modules:
+
+    r10k deploy environment my_working_environment --puppetfile
+
+This will update the given environment and update all contained modules. This is
+useful if you want to make sure that a given environment is fully up to date.
+
+- - -
+
+Update a single environment and specify a default branch override:
+
+    r10k deploy environment my_working_environment --puppetfile --default-branch-override default_branch_override
+
+This will update the given environment and update all contained modules, overrideing 
+the :default_branch entry in the Puppetfile of each module. This is used primarily to allow
+automated r10k solutions using the control_branch pattern with a temporary branch deployment to 
+ensure the deployment is pushed to the correct module repository branch. Note that the :default_branch
+is only ever utilized if the desired ref cannot be located.
+
+### Deploying modules
+
+Update a single module across all environments:
+
+    r10k deploy module apache
+
+This is useful for when you're working on a module specified in a Puppetfile
+and want to update it across all environments. See
+[Puppetfile documentation](doc/puppetfile.mkd) for details on how this affects
+Forge vs. Git/SVN modules.
+
+- - -
+
+Update multiple modules across all environments:
+
+    r10k deploy module apache jenkins java
+
+- - -
+
+Update one or more modules in a single environment:
+
+    r10k deploy module -e production apache jenkins java
+
+### Viewing environments
+
+Display all environments being managed by r10k:
+
+    r10k deploy display
+
+Display all environments being managed by r10k, and modules specified in the
+Puppetfile:
+
+    r10k deploy display -p
+
+Display all environments being managed by r10k, and modules specified in the
+Puppetfile along with their expected and actual versions:
+
+    r10k deploy display -p --detail
+
+Display an explicit list of environments being managed by r10k and modules
+specified in the Puppetfile along with their expected and actual versions:
+
+    r10k deploy display -p --detail production vmwr webrefactor
+
+User accounts
+-------------
+
+When running commands to deploy code on a master, r10k needs to have write
+access to your Puppet environment path and should create files that are
+readable by the user account running the master. If you're using Puppet
+Enterprise this account is `pe-puppet`, and if you're using Puppet open source
+this account is `puppet`.
+
+This can be done in a few ways. First off, you can run r10k as the puppet user
+itself. You can also create a new user that has write access to the Puppet
+environment path, has the same GID as the puppet user, and has a umask of 0027.
+You can also run r10k as root, which is the simplest solution but does require
+access control to the root user.