r10k 3.5.2

data/doc/dynamic-environments/workflow-guide.mkd

@@ -0,0 +1,247 @@
+R10k's dynamic deployments work best with a workflow that understands and
+respects how r10k works, to prevent automation and manual processes from
+conflicting. Your workflow will need to be customized to meet your team's
+skills, tools, and needs. This guide describes a generic workflow that can
+be customized easily.
+
+This guide assumes that each of your modules is in a separate repository 
+and that the `Puppetfile` is in its own repo called the
+[Control Repo](http://technoblogic.io/blog/2014/05/16/r10k-control-repos/).
+All module repos have a primary branch of *master* and the Control's primary
+branch is *production*. All changes are made through r10k and no user makes
+manual changes to the environments under **/etc/puppet**.
+
+Adding New Modules
+------------------
+
+This workflow is useful when adding a forge or internally-developed module
+to your puppet environment.
+
+### Create new feature branch
+
+Create a new feature branch in your module repositories. Do this for each
+repository, including the control repository, that will reference the new
+module. You do not need to do so for modules that are not being edited.
+
+```git checkout -b feature```
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, only the Control repo requires a new branch.
+
+### Add new module and branches to control repo
+
+The new module is added to the control repository's `Puppetfile` like so:
+
+```
+# Forge modules:
+mod "puppetlabs/ntp"
+
+# Your modules:
+mod "custom_facts",
+  :git => "git://github.com/user/custom_facts"
+```
+
+For any existing modules that you branched, add a reference to the new branch
+name. Don't forget the comma at the end of the *:git* value.
+
+```
+mod "other_module",
+  :git => "git://github.com/user/other_module",
+  :ref => "feature"
+```
+
+### Reference new module in manifests, modules, and hiera
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, you may skip this step.
+
+Edit your existing manifests, modules, and hiera as needed to make sure of
+the new module.
+
+### Deploy environments
+
+Save all your changes in each module repo. Commit and push the changes upstream:
+
+```
+git commit -a -m ‘Add feature reference to module’
+git push origin feature
+```
+
+Commit and push the change in your control repo upstream:
+
+```
+git commit -a -m ‘Add module puppetlabs/ntp to branch feature'
+git push origin feature
+```
+
+Finally, deploy the environments via r10k. This step must occur on the master:
+
+```r10k deploy environment -p```
+
+Add the **-v** option for verbosity if you need to troubleshoot any errors. The
+new branch should be located at `$environmentpath/feature`.
+
+
+### Test the new module branches
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, you may skip this step.
+
+Run the puppet agent against the new environment from at least two nodes, one
+that should not be impacted by change and one that should be impacted.
+
+```puppet agent -t --environment feature```
+
+Verify that catalog compilation succeeds and that you are satisfied that the
+effective changes match your expected changes. Repeat the steps above until
+you are satisfied with the results.
+
+### Merge changes
+
+In each of the changed modules and the control repo, checkout the main branch,
+merge, and push changes to the master/production branch.
+
+```
+# Module repos
+git checkout master
+git merge feature
+git push origin master
+
+# Control repo
+git checkout production
+git merge feature
+vi Puppetfile
+# Remove all :ref's pointing to 'feature'. Don't forget the trailing commas
+# on the :git statements
+git commit -a -m 'Remove refs to feature branch for module puppetlabs/ntp'
+git push origin production
+```
+
+If you are simply adding the module at this time and not referencing it in
+other modules or manifests, you are now finished.
+
+### Cleanup feature branches
+
+You may skip this step for long-lived branches, however most feature branches
+should be short-lived and can be pruned once testing and merging is complete.
+
+Remove the old branches in each repo:
+
+```
+git branch -D repo
+git push origin :repo
+```
+
+Deploy via r10k on the master and ensure there are no errors. The *feature*
+dynamic environment will no longer exist at `$environmentpath/feature` if you
+deleted the branch in your Control repo.
+
+```r10k deploy environment -p```
+
+Editing existing Modules
+------------------------
+
+When editing your own existing modules, this workflow should be followed.
+
+### Create new feature branches
+
+Create a new feature branch in your module repositories. Do this in the edited
+module, the control repository, and in each module that will reference the
+updated module. You do not need to do so for modules that are not being edited.
+
+```git checkout -b feature```
+
+### Update control repo to reference new branch
+
+For all modules that you branched, add a reference to the new branch name to
+the `Puppetfile` in your Control repo. Don't forget the comma at the end of
+the *:git* value.
+
+```
+mod "other_module",
+  :git => "git://github.com/user/other_module",
+  :ref => "feature"
+```
+
+### Modify existing module, references to module
+
+Make the required changes to your existing module. Edit your existing manifests,
+modules, and hiera as needed to make sure of the updated module.
+
+### Deploy environments
+
+Save all your changes in each modified repo. Commit and push the changes upstream:
+
+```
+git commit -a -m ‘Add feature reference to module’
+git push origin feature
+```
+
+Commit and push the change in your control repo upstream:
+
+```
+git commit -a -m ‘Add module puppetlabs/ntp to branch feature'
+git push origin feature
+```
+
+Finally, deploy the environments via r10k. This step must occur on the master:
+
+```r10k deploy environment -p```
+
+Add the *-v* option for verbosity if you need to troubleshoot any errors. The
+new branch should be located at `$environmentpath/feature`.
+
+### Test the new module branches
+
+Run the puppet agent against the new environment from at least two nodes, one
+that should not be impacted by change and one that should be impacted.
+
+```puppet agent -t --environment feature```
+
+Verify that catalog compilation succeeds and that you are satisfied that the
+effective changes match your expected changes. Repeat the steps above until
+you are satisfied with the results.
+
+### Merge changes
+
+In each of the changed module repos, checkout the main branch and merge.
+
+```
+# Module repos
+git checkout master
+git merge feature
+git push origin master
+```
+
+In the Control repo, check out production. Do NOT merge the feature branch as it
+now references the incorrect branch for each git repo, and no other changes
+were made (unlike a new module, where a new repo is referenced).
+
+```
+# Control repo
+git checkout production
+```
+
+### Cleanup feature branches
+
+You may skip this step for long-lived branches, however most feature branches
+should be short-lived and can be pruned once testing and merging is complete.
+
+Remove the old branches in each repo:
+
+```
+git branch -D repo
+git push origin :repo
+```
+
+Redeploy with r10k on your Puppet Master and ensure there are no errors. The *feature*
+dynamic environment should no longer exist at `$environmentpath/feature`.
+
+```r10k deploy environment -p```
+
+Customize Your Workflow
+-----------------------
+
+This guide is very generic in nature. Use it as a template and expand and
+modify it to fit your team, your tools, and your company culture. Above all,
+be consistent in your methodology.

data/doc/faq.mkd

@@ -0,0 +1,164 @@
+Frequently Asked Questions
+==========================
+
+### How can run I `puppet generate types` for each changed environment during deployment?
+
+The command `puppet generate types` creates Puppet signatures for custom
+types circumventing the need to load their Ruby code during compilation.
+This prevents the type definitions from "leaking" across environments.
+
+Best practice currently is to run this command as part of your Puppet
+code deployment pipeline, and r10k's postrun command ability is a natural
+place to do so.
+
+The postrun command will have any occurance of "$modifiedenvs" replaced
+with a space-seperated list of environments the deploy was configured to
+modify. For example creating the script:
+
+```
+$ cat /usr/local/bin/generate-puppet-types.sh
+!#/bin/bash
+
+for environment in $1; do
+  /opt/puppetlabs/bin/puppet generate types --environment $environment
+done
+```
+
+And configuring r10k such:
+```
+$ tail -2 /etc/puppetlabs/r10k/r10k.yaml
+
+postrun: ["/usr/local/bin/generate-puppet-types.sh", "$modifiedenvs"]
+```
+
+Should provide you with the necessary generate type definitions for
+improved environment isolation.
+
+
+
+### The default Git branch is 'master', while the default Puppet environment is 'production'. How do I reconcile this?
+
+The default Git branch name is 'master', but this is a somewhat arbitrary name
+and doesn't necessarily map to every use case. In the case of R10K it's generally
+easiest to rename 'master' to 'production'. You can rename the master branch
+with the following:
+
+```
+git branch -m master production
+git push --set-upstream origin production
+```
+
+Note that this will only create a new branch called production with a copy of
+master - to change the default branch for all subsequent clones, read on.
+
+#### Changing the default branch for bare Git repositories
+
+When you clone a repository, Git checks out the [currently active branch][git-clone]
+on the remote repository. Changing this for a non-bare repository is simple - just
+check out a different branch and subsequent clones from that repository will
+use that branch.
+
+For bare repositories things are a bit more complex. Bare repositories do not
+have a working directory that can be checked out, but they do have a [symbolic
+ref][git-symbolic-ref] that serves the same role. To change this, run the
+following command:
+
+```
+git --git-dir /path/to/bare/repo symbolic-ref HEAD refs/heads/production
+```
+
+#### Changing the default branch for different Git services
+
+For Git hosting services where you may not cannot directly invoke commands,
+there are usually administrative tools to allow you to change the default branch
+on your remote repositories:
+
+  * [GitHub][github-default-branch]
+  * [Bitbucket][bitbucket-default-branch]
+  * [Gitolite v2][gitolite-v2-default-branch]
+  * [Gitolite v3][gitolite-v3-default-branch]
+
+[git-clone]: https://www.kernel.org/pub/software/scm/git/docs/git-clone.html "Man page for git-clone"
+[git-symbolic-ref]: https://www.kernel.org/pub/software/scm/git/docs/git-symbolic-ref.html "Man page for git-symbolic-ref"
+
+[github-default-branch]: https://help.github.com/articles/setting-the-default-branch "Changing the default branch on GitHub"
+[bitbucket-default-branch]: https://answers.atlassian.com/questions/280944/how-to-change-main-branch-in-bitbucket "Changing the default branch on Bitbucket"
+[gitolite-v2-default-branch]: http://stackoverflow.com/questions/7091599/git-default-remote-branch-with-gitolite "Changing the default branch on Gitolite v2"
+[gitolite-v3-default-branch]: http://stackoverflow.com/questions/13949093/git-change-default-branch-gitolite "Changing the default branch on Gitolite v3"
+
+### How do I prevent r10k from removing modules in the `/modules` directory of my Git repository?
+
+By default, r10k will install modules specified in the Puppetfile into the
+`/modules` directory of each environment, but if you already use that directory
+and keep modules in it, r10k may think those modules are not meant to exist and
+may remove them.
+
+There are three ways of fixing this: including your local modules in the
+Puppetfile, moving the directory where r10k install Puppetfile sourced modules,
+or moving your modules.
+
+#### Including your local modules in the Puppetfile
+
+The Puppetfile has a concept of a "local" module, otherwise known as a module
+that r10k did not directly placed there but should not be removed. If you want
+to continue keep your modules in the `/modules` directory and still install
+external modules from the Puppetfile into that directory, you can add a `mod`
+directive to the Puppetfile for each of your local modules.
+
+```
+
+mod 'my_ntp', :local => true
+mod 'roles', :local => true
+mod 'profiles', :local => true
+
+# Include your external modules as usual
+mod 'puppetlabs/stdlib'
+mod 'puppetlabs/apache'
+```
+
+#### Move where the Puppetfile installs external modules
+
+Instead of having to add a module entry for each of your local modules, you can
+simply move where the Puppetfile installs modules with the `moduledir` setting.
+
+```
+# The moduledir setting must be set before any modules are created
+moduledir "external-modules"
+
+mod 'puppetlabs/stdlib'
+mod 'puppetlabs/apache'
+```
+
+In Puppet 3.6 and later you can create an `environment.conf` in the root of your
+environment to indicate which directories contain modules:
+
+```
+# environment.conf
+modulepath = modules:external-modules
+```
+
+#### Move your local modules
+
+Lastly, you can simply move your locally versioned modules to a separate
+directory to avoid conflicting over the `/modules` directory entirely. With this
+example as well you can use the `environment.conf` file to tell Puppet which
+directories contain modules.
+
+```
+# environment.conf
+modulepath = internal-modules:modules
+```
+
+#### Does R10K support Local/Private Forge?
+
+Yes. Set the Forge to use _globally_ in `r10k.yaml`. see [Configuration](/doc/dynamic-environments/configuration.mkd#baseurl) for details.
+
+
+#### What does the name mean?
+
+It’s called R10K because I’m terrible at names. When I started working on R10K
+I couldn’t come up with a decent name. While trying to come up with something
+clever, I recalled that Randall Munroe wrote a bot for [controlling IRC chatter](http://blog.xkcd.com/2008/01/14/robot9000-and-xkcd-signal-attacking-noise-in-chat/),
+and gave it the most generic name he could think of - Robot 9000. Since I just
+needed a name, any name, I decided to go with an equally generic name by
+incrementing the robot index.

data/doc/git/cloning-and-mirroring.mkd

@@ -0,0 +1,60 @@
+Git Cloning and Mirroring
+=========================
+
+Most Git operations on a repository require having a full clone of that
+repository locally available. Because this clone is a full copy, the initial
+clone is heavyweight and generally time consuming. For a situation like r10k
+where the same repositories are reused often, constantly cloning and deleting
+repositories is very inefficient.
+
+In order to speed up Git operations r10k takes a number of steps to avoid
+cloning and fetching repositories, as well as deduplicating content across
+multiple clones of the same repositories.
+
+Because r10k tends to reuse the same Git repositories in multiple places, r10k
+avoids repeated, full repository clones by mirroring repositories. When r10k
+starts using a new repository, it first clones that repository into a central
+location for later use. This initial clone is the only time that r10k will
+perform a full clone of that repository.
+
+When r10k creates an actual checkout of a Git repository, it uses the
+corresponding mirrored repository as a [reference](#git-alternates). This allows
+the working checkout to borrow objects from the mirrored repository instead of
+cloning all of the Git objects again. This saves a great deal of time and space
+as the number of copies of a repository increases.
+
+Mirrored git repositories are cloned into the directory specified by the
+r10k.yaml 'cachedir' setting.
+
+The name 'cachedir' is a bit of a misnomer; Git repositories are mirrored to
+speed up access to the remote resource like a cache, but unlike a traditional
+cache the mirrored repositories are persistent and should not be deleted.
+
+Because the mirrored repository contains all of the objects for all of the
+referencing repositories, deleting the mirrored repository is akin to deleting
+the `.git/objects` directory. Doing so effectively cripples the repository, so
+removing mirrored repositories should be done with care to avoid deleting
+repositories that are still in use.
+
+Git alternates
+--------------
+
+A standard git repository stores all content in the `.git/objects` directory,
+either as a zlib compressed file in `.git/objects/[0-9a-f]{2}/[0-9a-f]{38}` or
+as part of a packfile in `.git/objects/pack`. Since content can be both stored
+in and retrieved from this location it's treated like a simple database, and is
+generally referred to as the Git object database.
+
+Git allows a single repository to look in more than just `.git/objects` for
+objects; additional object databases are referred to as **alternates**. If a
+repository has alternate object databases set up, it will check `.git/objects`
+and then each alternate object database when looking for an object. Git stores a
+list of alternate object databases in `.git/objects/info/alternates`. Invoking
+`git clone` with the `--reference <repo>` flag will use that repository as an
+alternate object database.
+
+Links
+-----
+
+  * [man git-clone(1) --reference](https://www.kernel.org/pub/software/scm/git/docs/git-clone.html)
+  * [man gitrepository-layout(5) objects/info/alternates](https://www.kernel.org/pub/software/scm/git/docs/gitrepository-layout.html)