r10k 1.3.5 → 1.4.0

data/doc/common-patterns.mkd

@@ -0,0 +1,44 @@
+Common Patterns
+===============
+
+This guide provides common patterns seen in the r10k community. These patterns
+are, of course, simply a guide. Understand why you are or are not using a
+specific pattern before implementing it.
+
+Repository Setup
+----------------
+
+Use a [Control Repo](http://www.jeffmalnick.com/blog/2014/05/16/r10k-control-repos/)
+to store your `Puppetfile`.
+
+Hiera data should be in the Control repo OR as a separate source in
+`r10k.yaml`. Any `hiera.yaml` in the Control repo will be ignored on a per
+environment basis, locating it at `/etc/puppet/hiera.yaml` is prefered.
+
+Each puppet module should be contained in its own independent forge module or
+repository.
+
+Editing modules
+---------------
+
+All environment content is checked out into `$environmentpath/modules` on the
+node r10k is run on, either your puppet master or each masterless node. Edits
+made directly to these files will be lost on the next deploy. It is best
+practice not to edit code on the production system in the production paths.
+
+You may clone upstream repositories in a regular user's directory, on the master
+or on another machine. Create a new feature branch locally, make all required
+edits, and push the new branch upstream when ready for testing. R10k will
+deploy changes from the upstream repositories, eliminating the need for manual
+updates of the `$environmentpath` contents.
+
+Automated deploys
+-----------------
+
+To reduce manual intervention, use a post-receive hook on your control and
+module repos to initiate an r10k deploy. You can develop your own or use a
+publicly available hook. These include:
+
+* [Reaktor](https://github.com/pzim/reaktor)
+* [zack/r10k's Webhooks](https://forge.puppetlabs.com/zack/r10k#webhook-support)
+(Puppet Enterprise only)

data/doc/dynamic-environments.mkd

@@ -6,13 +6,19 @@ Dynamic environment deployment is the core functionality of r10k.
 Table of contents
 -----------------
 
-  * [Introduction](dynamic-environments/introduction/): A brief description of
+  * [Introduction](dynamic-environments/introduction.mkd): A brief description of
     what dynamic environments are and how they work.
-  * [Git environments](dynamic-environments/git-environments/) How r10k
-    implements dynamic environments using git
-  * [Configuration](dynamic-environments/configuration/) A reference of dynamic
+  * [Quickstart](dynamic-environments/quickstart.mkd): Getting started with the
+    essentials of dynamic environments and r10k.
+  * [Configuration](dynamic-environments/configuration.mkd) A reference of dynamic
     environment configuration options and how they're used
-  * [Usage](dynamic-environments/usage/) A reference of r10k commands and how
+  * [Git environments](dynamic-environments/git-environments.mkd) How r10k
+    implements dynamic environments using git
+  * [Master Configuration](dynamic-environments/master-configuration.mkd) How to
+    configure your Puppet masters to use dynamic environments
+  * [Workflow Guide](dynamic-environments/workflow-guide.mkd) A general-purpose
+    workflow guide for using r10k.
+  * [Usage](dynamic-environments/usage.mkd) A reference of r10k commands and how
     they're used.
 
 - - -
@@ -22,3 +28,4 @@ Community guides
 
   * [Building a Functional Puppet Workflow Part 3: Dynamic Environments with R10k](http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/)
   * [Puppet Infrastructure with r10k](http://terrarum.net/blog/puppet-infrastructure-with-r10k.html)
+  * [Puppet for vSphere Admins](http://rnelson0.com/puppet-for-vsphere-admins/) Installing and using r10k in a vSphere environment

data/doc/dynamic-environments/configuration.mkd

@@ -1,7 +1,7 @@
 Dynamic Environment Configuration
 =================================
 
-r10k uses a configuration file to determine how dynamic environments should be
+R10k uses a configuration file to determine how dynamic environments should be
 deployed.
 
 Config file location
@@ -41,6 +41,21 @@ Deployment options
 
 The following options configure how r10k deploys dynamic environments.
 
+### postrun
+
+The `postrun` setting specifies an arbitrary command to run after deploying all
+environments. The command must be an array of strings that will be used as an
+argument vector. The exit code of the command is not currently used, but the
+command should exit with a return code of 0 as the exit code may have semantics
+in the future.
+
+```yaml
+---
+postrun: ['/usr/bin/curl', '-F', 'deploy=done', 'http://my-app.site/endpoint']
+```
+
+The postrun setting can only be set once.
+
 ### sources
 
 The `sources` setting specifies what repositories should be used for creating

data/doc/dynamic-environments/{git-environments.markdown → git-environments.mkd}

@@ -1,7 +1,7 @@
 Git Based Dynamic Environments
 ==============================
 
-r10k can use Git repositories to implement dynamic environments. You can create,
+R10k can use Git repositories to implement dynamic environments. You can create,
 update, and delete Puppet environments automatically as part of your normal Git
 workflow.
 
@@ -29,17 +29,21 @@ seamlessly reflected in Puppet environments. This means that creating a new Git
 branch creates a new Puppet environment, updating a Git branch will update that
 environment, and deleting a Git branch will remove that environment.
 
+R10k supports both [directory and config file environments](https://docs.puppetlabs.com/puppet/latest/reference/environments.html).
+Ensure that the basedir for your sources and your puppet config align.
+
 How it works
 ------------
 
-r10k works by tracking the state of your git repositories and comparing them the
-the currently deployed environments. If there's a Git branch that doesn't have a
-corresponding Puppet environment then r10k will clone that branch into the
-appropriate directory. When Git branches are updated r10k will update the
-appropriate Puppet environment to the latest version. Finally if there are
-Puppet environments that don't have matching Git branches, r10k will assume that
-the branches for those environments were deleted and will remove those
-environments.
+R10k works by tracking the state of your Git repository or repositories. Each
+repository's branches will be cloned into a directory with a matching name,
+creating a Puppet environment for the branch. If a repository includes a
+Puppetfile, such as the control repo, the Forge modules and Git/SVN
+repositories described within will be cloned as well, into the same directories.
+Subsequent changes to the branches will be kept in sync on the filesystem by
+future r10k runs. Finally, if there are directories that do not match existing
+branches, r10k will assume that the branches for those environments were delete
+and will remove those environments.
 
 Configuration
 -------------

data/doc/dynamic-environments/introduction.mkd

@@ -42,8 +42,7 @@ like modulepath, manifest, and so forth.
 
 ```ini
 [master]
-vardir = '/var/lib/puppet'
-modulepath = '/etc/puppet/environments/$environment/modules'
+environmentpath = $confdir/environments'
 ```
 
 Running `puppet agent -t --environment myenv` will cause $environment to be

data/doc/dynamic-environments/master-configuration.mkd

@@ -0,0 +1,70 @@
+Puppet master configuration
+===========================
+
+In order to use dynamic environments, your Puppet masters will need to be
+configured to load manifests and modules relative to the requested environment.
+The following settings should be configured in puppet.conf.
+
+## Puppet <= 3.4.x
+
+In Puppet 3.4 and earlier, dynamic environments are achieved by interpolating
+the `$environment` variable in the modulepath, manifestdir, and manifest
+settings. When environments is loaded the settings are dynamically set based on
+the name of the environment, thus allowing environments to be created on the
+fly.
+
+    [master]
+        modulepath = /etc/puppet/environments/$environment/modules:/etc/puppet/environments/$environment/dist
+
+        # If you use a top level manifest dir
+        manifestdir = /etc/puppet/environments/$environment/manifests
+        # If you use a specific site wide manifest
+        manifest = /etc/puppet/environments/$environment/manifests/nodes.pp
+
+## Puppet = 3.5.x
+
+Puppet 3.5.0 adds initial support for "directory environments", where
+environments are created by enumerating directories in `$environmentpath`.
+This new configuration expect that all modules are in the root of the
+directories in the environment path.
+
+    [master]
+    # None of modulepath, manifestdir, or manifest should be enabled
+    # See http://docs.puppetlabs.com/puppet/latest/reference/environments.html#enabling-directory-environments
+    #   for more information on the changes
+        environmentpath = $confdir/environments
+
+Note that these settings go into the `[master]` section; you don't need to
+explicitly configure an environment section for each environment you want to
+use. (But you can if you want.)
+
+## Puppet >= 3.6.0
+
+[environmentconf]: http://docs.puppetlabs.com/puppet/latest/reference/config_file_environment.html
+
+Puppet 3.6.0 adds more fine grained control over how directory environments are
+configured. Each directory based environment can have an
+[`environment.conf`][environmentconf] file in the root of that environment that
+can specify the manifest, modulepath, config_version, and environment_timeout
+for that specific environment.
+
+    # puppet.conf
+    [master]
+    # None of modulepath, manifestdir, or manifest should be enabled
+    # See http://docs.puppetlabs.com/puppet/latest/reference/environments.html#enabling-directory-environments
+    #   for more information on the changes
+        environmentpath = $confdir/environments
+
+- - -
+
+    # environment.conf
+    manifest  = site.pp
+    moduledir = modules:dist
+
+A deployed environment with a Puppetfile will look something like this:
+
+    .
+    ├── Puppetfile
+    ├── environment.conf
+    ├── dist
+    └── modules

data/doc/dynamic-environments/quickstart.mkd

@@ -0,0 +1,241 @@
+# Overview
+
+This intent of this document is to serve as a basic guide for getting started with r10k and a fresh Puppet installation, including the following:
+
+* Installing Puppet and its dependencies
+* Installing Hiera and its dependencies.
+* Installing r10k and its dependencies
+* Configuring all components to support r10k
+* Configuring your git repository and initial files
+
+# Pre-Requisites
+
+* Clean install of CentOS 6.5 or Debian 7.0 with root access / sudo rights. 
+* Clean github repository with a deploy key generated by the server above.
+
+# Installing Puppetmaster on Centos 6.5
+
+Install and enable the official Puppet Labs package repositories.
+
+```
+rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
+```
+
+Clean all yum data and rebuild the metadata cache.
+
+```
+yum clean all && yum makecache
+```
+
+Install the Puppet master.
+
+```
+yum install puppet-server
+```
+
+# Installing Puppetmaster on Debian 7
+
+Install and enable the official Puppet Labs package repositories
+
+```
+wget http://apt.puppetlabs.com/puppetlabs-release-wheezy.deb
+dpkg -i puppetlabs-release-wheezy.deb
+apt-get update
+```
+
+Install the Pupppet master
+```
+apt-get install puppetmaster-passenger=3.7.1-1puppetlabs1
+```
+Above command will install 3.7.1-1puppetlabs but there are many versions to choose. To see available versions run: apt-cache show puppetmaster-passenger|grep ^Version
+
+# Configuring Puppet 
+
+Configure the Puppet master by editing `/etc/puppet/puppet.conf` and ensuring it has the following contents:
+
+```
+[main]
+    # The Puppet log directory.
+    # The default value is '$vardir/log'.
+    logdir = /var/log/puppet
+
+    # Where Puppet PID files are kept.
+    # The default value is '$vardir/run'.
+    rundir = /var/run/puppet
+
+    # Where SSL certificates are kept.
+    # The default value is '$confdir/ssl'.
+    ssldir = $vardir/ssl
+
+    dns_alt_names = $_Insert FQDN of Puppet Master Here_$
+
+    environmentpath = $confdir/environments
+
+[agent]
+    # The file in which puppetd stores a list of the classes
+    # associated with the retrieved configuratiion.  Can be loaded in
+    # the separate ``puppet`` executable using the ``--loadclasses``
+    # option.
+    # The default value is '$confdir/classes.txt'.
+    classfile = $vardir/classes.txt
+
+    # Where puppetd caches the local configuration.  An
+    # extension indicating the cache format is added automatically.
+    # The default value is '$confdir/localconfig'.
+    localconfig = $vardir/localconfig
+
+    server = $_Insert FQDN of Puppet Master Here_$
+```
+
+Restart the Puppet master service.
+
+```
+service puppetmaster restart
+```
+
+Ensure the certificate for the Puppet master was created.
+
+```
+# puppet cert list --all
+
++ "puppet-master.domain.local" (SHA256) 3F:F3:63:BB:EE:57:46:A4:7B:03:AB:9D:FD:97:0F:8F:73:87:40:3B:6D:E5:DC:FC:C3:49:F5:C9:B6:F4:DE:B8 (alt names: "DNS:puppet-master.domain.local")
+```
+
+Notice for Debian users: apt post-configure will build the certificate for the server BEFORE you configure it. Therefore you should rebuild your certs after done with /etc/puppet/puppet.conf configuration.
+To do so you need to remove old certs and restart puppetmaster:
+```
+service puppetmaster stop
+find $(puppet master --configprint ssldir) -name "$(puppet master --configprint certname).pem" -delete
+puppet master --no-daemonize --verbose
+```
+
+# Install and Configure R10k
+
+Install r10k via Ruby Gems.
+
+```
+gem install r10k
+```
+
+Configure r10k by editing `/etc/r10k.yaml` and ensuring it has the following contents:
+
+```
+# The location to use for storing cached Git repos
+:cachedir: '/var/cache/r10k'
+
+# A list of git repositories to create
+:sources:
+  # This will clone the git repository and instantiate an environment per
+  # branch in /etc/puppet/environments
+  :my-org:
+    remote: 'git@github.com:$_Insert GitHub Organization Here_$/$_Insert GitHub Repository That Will Be Used For Your Puppet Code Here_$'
+    basedir: '/etc/puppet/environments'
+```
+# Install and Configure Hiera
+
+Hiera is installed as part of the Puppet master installation.
+
+Configure Hiera by editing `/etc/hiera.yaml` and ensuring it has the following contents:
+
+```
+---
+:backends:
+  - yaml
+:hierarchy:
+  - "nodes/%{fqdn}"
+  - common
+
+:yaml:
+   :datadir: /etc/puppet/environments/%{environment}/hiera
+```
+
+Create a symlink to the Hiera configuration in the Puppet directory.
+
+```
+ln -s /etc/hiera.yaml /etc/puppet/hiera.yaml
+```
+
+# Configure Puppet Code Repository
+
+Populate the repository by cloning it locally and performing each of the following actions within it:
+
+```
+mkdir -p {modules,site/profile/manifests,hiera}
+touch hiera/common.yaml
+touch site/profile/manifests/base.pp
+touch environment.conf
+touch Puppetfile
+touch site.pp
+```
+
+Edit the `environment.conf` file and ensure it has the following contents:
+
+```
+manifest = site.pp
+modulepath = modules:site
+```
+
+Edit the `site.pp` file and ensure it has the following contents:
+
+```
+hiera_include('classes')
+```
+
+Edit the `hiera/common.yaml file and ensure it has the following contents:
+
+```
+---
+classes:
+ - 'profile::base'
+
+ntp::servers:
+  - 0.us.pool.ntp.org
+  - 1.us.pool.ntp.org
+```
+
+Edit the `Puppetfile` file and ensure it has the following contents:
+
+```
+forge 'forge.puppetlabs.com'
+
+# Forge Modules
+mod 'puppetlabs/ntp', '3.0.3'
+mod 'puppetlabs/stdlib'
+```
+
+Edit the `site/profile/manifests/base.pp` file and ensure it has the following contents:
+
+```
+class profile::base {
+  class { '::ntp': }
+}
+```
+
+# Summary
+
+We now have the following functional pieces:
+
+1. Puppet master
+2. Hiera
+3. r10k
+4. Puppet code repository
+5. Initial 'profile' named 'base' that will configure NTP on our servers.
+
+This base will allow us to do all sorts of useful things. Most interesting (to me and for the purposes of this tutorial) is the ability to now utilize Git branches to help manage infrastructure as part of your software development lifecycle. Now, when you want to test a new profile, you can do the following:
+
+1. Create a new branch of the Puppet code repository
+2. Create your Puppet code in this new branch
+3. Push the new branch up to the repository
+4. Deploy it as a new environment using the `r10k deploy environment -p` command.
+
+From any agent node (including the master), you may run the agent against the new environment by specifying it on the command line. For example, if you create the branch `test`, run puppet as:
+```
+puppet agent -t --environment test
+```
+You can also modify the `/etc/puppet/puppet.conf` file on a node and add the environment setting to the agent section to make the change permanent:
+```
+...
+[agent]
+environment = test
+```
+Voila - you're testing code without impacting your production environment!