r10k 1.3.5 → 1.4.0

data/doc/dynamic-environments/svn-environments.mkd

@@ -0,0 +1,45 @@
+SVN Based Dynamic Environments
+==============================
+
+R10k can use SVN repositories to implement dynamic environments. You can create,
+update, and delete Puppet environments automatically as part of your normal SVN
+workflow.
+
+How it works
+------------
+
+R10k implements a branching workflow similar to Git by using the SVN concept of
+branches. SVN repositories must conform to the conventional SVN repository
+structure with the directories trunk/, branches/, and optionally tags/ in the
+root of the repository. R10k maps the trunk/ directory to the production
+environment, and production environment, and branches (directories in branches/)
+are created as environments with the name of the given branch.
+
+Configuration
+-------------
+
+In addition to the settings that all sources support, SVN sources can specify
+the following additional options:
+
+### username/password
+
+If the SVN repository requires credentials, you can supply the `username` and
+`password` options.
+
+Both `username` and `password` must be specified in order to SVN authentication.
+
+**Note**: SVN credentials are passed as command line options, so the SVN
+credentials may be visible in the process table when r10k is running. If you
+choose to supply SVN credentials make sure that the system running r10k is
+appropriately secured.
+
+```yaml
+---
+sources:
+  myenvs:
+    type: svn
+    remote: 'svn://my-svn.server/my-svn-repo'
+    basedir: '/etc/puppet/environments'
+    username: 'azurediamond'
+    password: 'hunter2'
+```

data/doc/dynamic-environments/usage.mkd

@@ -31,7 +31,7 @@ basis.
 
 - - -
 
-Update environments while avoiding unnecessary recursion
+Update environments while avoiding unnecessary recursion:
 
     r10k deploy environment
 
@@ -62,20 +62,59 @@ useful if you want to make sure that a given environment is fully up to date.
 
 ### Deploying modules
 
-Update a single module across all environments
+Update a single module across all environments:
 
     r10k deploy module apache
 
-This is useful for when you're working on a module specified in a Puppetfile and want to update it across all environments.
+This is useful for when you're working on a module specified in a Puppetfile
+and want to update it across all environments. See
+[Puppetfile documentation](doc/puppetfile.mkd) for details on how this affects
+Forge vs. Git/SVN modules.
 
 - - -
 
-Update multiple modules across all environments
+Update multiple modules across all environments:
 
     r10k deploy module apache jenkins java
 
 - - -
 
-Update one or more modules in a single environment
+Update one or more modules in a single environment:
 
     r10k deploy module -e production apache jenkins java
+
+### Viewing environments
+
+Display all environments being managed by r10k:
+
+    r10k deploy display
+
+Display all environments being managed by r10k, and modules specified in the
+Puppetfile:
+
+    r10k deploy display -p
+
+Display all environments being managed by r10k, and modules specified in the
+Puppetfile along with their expected and actual versions:
+
+    r10k deploy display -p --detail
+
+Display an explicit list of environments being managed by r10k and modules
+specified in the Puppetfile along with their expected and actual versions:
+
+    r10k deploy display -p --detail production vmwr webrefactor
+
+User accounts
+-------------
+
+When running commands to deploy code on a master, r10k needs to have write
+access to your Puppet environment path and should create files that are
+readable by the user account running the master. If you're using Puppet
+Enterprise this account is `pe-puppet`, and if you're using Puppet open source
+this account is `puppet`.
+
+This can be done in a few ways. First off, you can run r10k as the puppet user
+itself. You can also create a new user that has write access to the Puppet
+environment path, has the same GID as the puppet user, and has a umask of 0027.
+You can also run r10k as root, which is the simplest solution but does require
+access control to the root user.

data/doc/dynamic-environments/workflow-guide.mkd

@@ -0,0 +1,247 @@
+R10k's dynamic deployments work best with a workflow that understands and
+respects how r10k works, to prevent automation and manual processes from
+conflicting. Your workflow will need to be customized to meet your team's
+skills, tools, and needs. This guide describes a generic workflow that can
+be customized easily.
+
+This guide assumes that each of your modules is in a separate repository 
+and that the `Puppetfile` is in its own repo called the
+[Control Repo](http://www.jeffmalnick.com/blog/2014/05/16/r10k-control-repos/).
+All module repos have a primary branch of *master* and the Control's primary
+branch is *production*. All changes are made through r10k and no user makes
+manual changes to the environments under **/etc/puppet**.
+
+Adding New Modules
+------------------
+
+This workflow is useful when adding a forge or internally-developed module
+to your puppet environment.
+
+### Create new feature branch
+
+Create a new feature branch in your module repositories. Do this for each
+repository, including the control repository, that will reference the new
+module. You do not need to do so for modules that are not being edited.
+
+```git checkout -b feature```
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, only the Control repo requires a new branch.
+
+### Add new module and branches to control repo
+
+The new module is added to the control repository's `Puppetfile` like so:
+
+```
+# Forge modules:
+mod "puppetlabs/ntp"
+
+# Your modules:
+mod "custom_facts",
+  :git => "git://github.com/user/custom_facts"
+```
+
+For any existing modules that you branched, add a reference to the new branch
+name. Don't forget the comma at the end of the *:git* value.
+
+```
+mod "other_module",
+  :git => "git://github.com/user/other_module",
+  :ref => "feature"
+```
+
+### Reference new module in manifests, modules, and hiera
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, you may skip this step.
+
+Edit your existing manifests, modules, and hiera as needed to make sure of
+the new module.
+
+### Deploy environments
+
+Save all your changes in each module repo. Commit and push the changes upstream:
+
+```
+git commit -a -m ‘Add feature reference to module’
+git push origin feature
+```
+
+Commit and push the change in your control repo upstream:
+
+```
+git commit -a -m ‘Add module puppetlabs/ntp to branch feature'
+git push origin feature
+```
+
+Finally, deploy the environments via r10k. This step must occur on the master:
+
+```r10k deploy environment -p```
+
+Add the **-v** option for verbosity if you need to troubleshoot any errors. The
+new branch should be located at `$environmentpath/feature`.
+
+
+### Test the new module branches
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, you may skip this step.
+
+Run the puppet agent against the new environment from at least two nodes, one
+that should not be impacted by change and one that should be impacted.
+
+```puppet agent -t --environment feature```
+
+Verify that catalog compilation succeeds and that you are satisfied that the
+effective changes match your expected changes. Repeat the steps above until
+you are satisfied with the results.
+
+### Merge changes
+
+In each of the changed modules and the control repo, checkout the main branch,
+merge, and push changes to the master/production branch.
+
+```
+# Module repos
+git checkout master
+git merge feature
+git push origin master
+
+# Control repo
+git checkout production
+git merge feature
+vi Puppetfile
+# Remove all :ref's pointing to 'feature'. Don't forget the trailing commas
+# on the :git statements
+git commit -a -m 'Remove refs to feature branch for module puppetlabs/ntp'
+git push origin production
+```
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, you are now finished.
+
+### Cleanup feature branches
+
+You may skip this step for long-lived branches, however most feature branches
+should be short-lived and can be pruned once testing and merging is complete.
+
+Remove the old branches in each repo:
+
+```
+git branch -D repo
+git push origin :repo
+```
+
+Deploy via r10k on the master and ensure there are no errors. The *feature*
+dynamic environment will no longer exist at `$environmentpath/feature` if you
+deleted the branch in your Control repo.
+
+```r10k deploy environment -p```
+
+Editing existing Modules
+------------------------
+
+When editing your own existing modules, this workflow should be followed.
+
+### Create new feature branches
+
+Create a new feature branch in your module repositories. Do this in the edited
+module, the control repository, and in each module that will reference the
+updated module. You do not need to do so for modules that are not being edited.
+
+```git checkout -b feature```
+
+### Update control repo to reference new branch
+
+For all modules that you branched, add a reference to the new branch name to
+the `Puppetfile` in your Control repo. Don't forget the comma at the end of
+the *:git* value.
+
+```
+mod "other_module",
+  :git => "git://github.com/user/other_module",
+  :ref => "feature"
+```
+
+### Modify existing module, references to module
+
+Make the required changes to your existing module. Edit your existing manifests,
+modules, and hiera as needed to make sure of the updated module.
+
+### Deploy environments
+
+Save all your changes in each modified repo. Commit and push the changes upstream:
+
+```
+git commit -a -m ‘Add feature reference to module’
+git push origin feature
+```
+
+Commit and push the change in your control repo upstream:
+
+```
+git commit -a -m ‘Add module puppetlabs/ntp to branch feature'
+git push origin feature
+```
+
+Finally, deploy the environments via r10k. This step must occur on the master:
+
+```r10k deploy environment -p```
+
+Add the *-v* option for verbosity if you need to troubleshoot any errors. The
+new branch should be located at `$environmentpath/feature`.
+
+### Test the new module branches
+
+Run the puppet agent against the new environment from at least two nodes, one
+that should not be impacted by change and one that should be impacted.
+
+```puppet agent -t --environment feature```
+
+Verify that catalog compilation succeeds and that you are satisfied that the
+effective changes match your expected changes. Repeat the steps above until
+you are satisfied with the results.
+
+### Merge changes
+
+In each of the changed module repos, checkout the main branch and merge.
+
+```
+# Module repos
+git checkout master
+git merge feature
+git push origin master
+```
+
+In the Control repo, check out master. Do NOT merge the feature branch as it
+now references the incorrect branch for each git repo, and no other changes
+were made (unlike a new module, where a new repo is referenced).
+
+```
+# Control repo
+git checkout master
+```
+
+### Cleanup feature branches
+
+You may skip this step for long-lived branches, however most feature branches
+should be short-lived and can be pruned once testing and merging is complete.
+
+Remove the old branches in each repo:
+
+```
+git branch -D repo
+git push origin :repo
+```
+
+Redeploy with r10k on the master and ensure there are no errors. The *feature*
+dynamic environment should no longer exist at `$environmentpath/feature`.
+
+```r10k deploy environment -p```
+
+Customize Your Workflow
+-----------------------
+
+This guide is very generic in nature. Use it as a template and expand and
+modify it to fit your team, your tools, and your company culture. Above all,
+be consistent in your methodology.

data/doc/faq.mkd

@@ -0,0 +1,52 @@
+Frequently Asked Questions
+==========================
+
+### The default Git branch is 'master', while the default Puppet environment is 'production'. How do I reconcile this?
+
+The default Git branch name is 'master', but this is a somewhat arbitrary name
+and doesn't necessarily map to every use case. In the case of R10K it's generally
+easiest to rename 'master' to 'production'. You can rename the master branch
+with the following:
+
+```
+git branch -m master production
+git push --set-upstream origin production
+```
+
+Note that this will only create a new branch called production with a copy of
+master - to change the default branch for all subsequent clones, read on.
+
+#### Changing the default branch for bare Git repositories
+
+When you clone a repository, Git checks out the [currently active branch][git-clone]
+on the remote repository. Changing this for a non-bare repository is simple - just
+check out a different branch and subsequent clones from that repository will
+use that branch.
+
+For bare repositories things are a bit more complex. Bare repositories do not
+have a working directory that can be checked out, but they do have a [symbolic
+ref][git-symbolic-ref] that serves the same role. To change this, run the
+following command:
+
+```
+git --git-dir /path/to/bare/repo symbolic-ref HEAD refs/heads/production
+```
+
+#### Changing the default branch for different Git services
+
+For Git hosting services where you may not cannot directly invoke commands,
+there are usually administrative tools to allow you to change the default branch
+on your remote repositories:
+
+  * [GitHub][github-default-branch]
+  * [Bitbucket][bitbucket-default-branch]
+  * [Gitolite v2][gitolite-v2-default-branch]
+  * [Gitolite v3][gitolite-v3-default-branch]
+
+[git-clone]: https://www.kernel.org/pub/software/scm/git/docs/git-clone.html "git-clone(1)"
+[git-symbolic-ref]: https://www.kernel.org/pub/software/scm/git/docs/git-symbolic-ref.html "git-symbolic-ref(1)"
+
+[github-default-branch]: https://help.github.com/articles/setting-the-default-branch "Setting the default branch"
+[bitbucket-default-branch]: https://answers.atlassian.com/questions/280944/how-to-change-main-branch-in-bitbucket "How to change MAIN branch in BitBucket?"
+[gitolite-v2-default-branch]: http://stackoverflow.com/questions/7091599/git-default-remote-branch-with-gitolite "git default remote branch with gitolite"
+[gitolite-v3-default-branch]: http://stackoverflow.com/questions/13949093/git-change-default-branch-gitolite "git change default branch (gitolite)"

data/doc/puppetfile.mkd

@@ -0,0 +1,203 @@
+Puppetfile
+==========
+
+Puppetfiles are a simple Ruby based DSL that specifies a list of modules to
+install, what version to install, and where to fetch them from. r10k can use a
+Puppetfile to install a set of Puppet modules for local development, or they can
+be used with r10k environment deployments to install additional modules into a
+given environment.
+
+Unlike librarian-puppet, the r10k implementation of Puppetfiles does not include
+dependency resolution, but it is on the roadmap.
+
+When directly working with Puppetfiles, you can use the `r10k puppetfile`
+subcommand to interact with a Puppetfile.
+
+When using r10k's deploy functionality, interacting with Puppetfiles is handled
+on a case by case basis.
+
+Because the Puppetfile format is actually implemented using a Ruby DSL any valid
+Ruby expression can be used. That being said, being a bit too creative in the
+DSL can lead to surprising (read: bad) things happening, so consider keeping it
+simple.
+
+Commands
+--------
+
+Puppetfile subcommands assume that the Puppetfile to operate on is in the
+current working directory and modules should be installed in the 'modules'
+directory relative to the current working directory.
+
+Install or update all modules in a given Puppetfile
+into ./modules)
+
+    r10k puppetfile install
+
+Verify the Puppetfile syntax
+
+    r10k puppetfile check
+
+Remove any modules in the 'modules' directory that are not specified in the
+Puppetfile:
+
+    r10k puppetfile purge
+
+Global settings
+---------------
+
+The following settings can be used to control how the Puppetfile installs and
+handles modules.
+
+### forge
+
+The `forge` setting specifies which server that Forge based modules are fetched
+from. This is currently a noop and is provided for compatibility with
+librarian-puppet, but will be made functional in a future version. See
+[GH-106](https://github.com/adrienthebo/r10k/issues/106) for more information.
+
+### moduledir
+
+The `moduledir` setting specifies where modules from the Puppetfile will be
+installed. This defaults to the `modules` directory relative to the Puppetfile.
+If the path is absolute then the modules will be installed to that absolute
+path, otherwise it's assumed that the `moduledir` setting should be relative and
+the modules will be installed in that directory relative to the Puppetfile.
+
+The moduledir setting should be placed before any modules are declared.
+
+Install modules to an absolute path:
+
+```ruby
+moduledir '/etc/puppet/modules'
+
+mod 'branan/eight_hundred' # will be installed into '/etc/puppet/modules/eight_hundred'
+```
+
+Install modules to a relative path:
+
+```ruby
+moduledir 'thirdparty'
+
+mod 'branan/eight_hundred' # will be installed into `dirname /path/to/Puppetfile`/thirdparty/eight_hundred
+```
+
+**Note**: support for a relative moduledir was added in r10k 1.4.0; the behavior
+of a relative moduledir path is undefined on earlier versions of r10k.
+
+Module types
+------------
+
+r10k can install Puppet modules from a number of different sources. Right now
+modules can be installed via Git, SVN, and from the Puppet Forge.
+
+### Git
+
+Git repositories that contain a Puppet module can be cloned and used as modules.
+When Git is used, the module version can be specified by using `:ref`, `:tag`,
+`:commit`, and `:branch`.
+
+When a module is installed using `:ref`, r10k uses some simple heuristics to
+determine the type of Git object that should be checked out. This can be used
+with a git commit, branch reference, or a tag.
+
+When a module is installed using `:tag` or `:commit`, r10k assumes that the
+given object is a tag or commit and can do some optimizations around fetching
+the object. If the tag or commit is already available r10k will skip network
+operations when updating the repo, which can speed up install times.
+
+Module versions can also be specified using `:branch`. This behaves similarly to
+`:ref`, and is mainly useful for clarity.
+
+#### Examples
+
+    # Install puppetlabs/apache and keep it up to date with 'master'
+    mod 'apache',
+      :git => 'https://github.com/puppetlabs/puppetlabs-apache'
+
+    # Install puppetlabs/apache and track the 'docs_experiment' branch
+    mod 'apache',
+      :git => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :ref => 'docs_experiment'
+
+    # Install puppetlabs/apache and pin to the '0.9.0' tag
+    mod 'apache',
+      :git => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :tag => '0.9.0'
+
+    # Install puppetlabs/apache and pin to the '83401079' commit
+    mod 'apache',
+      :git    => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :commit => '83401079053dca11d61945bd9beef9ecf7576cbf'
+
+    # Install puppetlabs/apache and track the 'docs_experiment' branch
+    mod 'apache',
+      :git    => 'https://github.com/puppetlabs/puppetlabs-apache',
+      :branch => 'docs_experiment'
+
+### Forge
+
+Modules can be installed using the Puppet module tool.
+
+If no version is specified the latest version available at the time will be
+installed, and will be kept at that version.
+
+    mod 'puppetlabs/apache'
+
+If a version is specified then that version will be installed.
+
+    mod 'puppetlabs/apache', '0.10.0'
+
+If the version is set to :latest then the module will be always updated to the
+latest version available.
+
+    mod 'puppetlabs/apache', :latest
+
+### SVN
+
+Modules can be installed via SVN. If no version is given, the module will track
+the latest version available in the main SVN repository.
+
+    mod 'apache',
+      :svn => 'https://github.com/puppetlabs/puppetlabs-apache/trunk'
+
+If an SVN revision number is specified with `:rev` (or `:revision`), that
+SVN revision will be kept checked out.
+
+    mod 'apache',
+      :svn => 'https://github.com/puppetlabs/puppetlabs-apache/trunk',
+      :rev => '154'
+
+    mod 'apache',
+      :svn      => 'https://github.com/puppetlabs/puppetlabs-apache/trunk',
+      :revision => '154'
+
+If the SVN repository requires credentials, you can supply the `:username` and
+`:password` options.
+
+    mod 'apache',
+      :svn      => 'https://github.com/puppetlabs/puppetlabs-apache/trunk',
+      :username => 'azurediamond',
+      :password => 'hunter2'
+
+**Note**: SVN credentials are passed as command line options, so the SVN
+credentials may be visible in the process table when r10k is running. If you
+choose to supply SVN credentials make sure that the system running r10k is
+appropriately secured.
+
+## Environment variables
+
+It is possible to set an alternate name/location for your `Puppetfile` and
+`modules` directory. This is useful if you want to control multiple environments
+and have a single location for your `Puppetfile`.
+
+Example:
+
+    PUPPETFILE=/etc/r10k.d/Puppetfile.production \
+    PUPPETFILE_DIR=/etc/puppet/modules/production \
+    /usr/bin/r10k puppetfile install
+
+NOTE: using these environment variables is not a suggested configuration, and
+have different semantics than librarian-puppet. Specifically, the PUPPETFILE_DIR
+is the environment that r10k will install modules into, and it will take full
+control over that directory and _remove any unmanaged content_. Use these
+variables with caution.