runbook 0.13.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5fe4b7ea15d7d74edf1a660fda55c9f8360a16f7e5032acadc427039240b9cf6
-  data.tar.gz: c72e2c91cd75587c87ed4b9e5921e1be050ea6466c59e1d903fefcf7f6770bfd
+  metadata.gz: 1c6c617aa11aeff1ab914c2f6d4d7b4db2cc5f46e6c70acc67a97eda7ba5b3a6
+  data.tar.gz: 3d68b0e53afbddc94ac0fa7c56eab9a8ac78ad104830c551f79c477314a81c4f
 SHA512:
-  metadata.gz: 129f995881810339793273c74d00a872b62cd865f8712fe7db063356c95bdae7f090537d244383b00fbf81cf2de8555e59627248b9e4a6284320aa9594508ef1
-  data.tar.gz: b1c2ef5526b2225d02a246c2d4f5d8e657c599ab04faf96f192084ea4f11d61863f455abc91136a17174dd7645f1112231fa7614ed83862d4444debbc0d2ca07
+  metadata.gz: cd186e2e3d0bde065b1ed0cccecce8c5943ec6ca626dac0cfa38725f4dd4050f5208df8343e770c602f2ec6cf7b38be9b01fee69ac0f304a16a14fe6c610378a
+  data.tar.gz: 4ce8c856c8cbd26cbffdfc1f7c2fe1b1fad5f6d5cdfc511d3be731d8db43db5ddeff1abf1d1ab2738374ac5eac27dbe33151e99f97abd604f3ddebb10394bf8c

data/.dockerignore

@@ -0,0 +1,17 @@
+**/.git
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+
+# rspec failure tracking
+.rspec_status
+
+# recommended:
+# https://github.com/thoughtbot/appraisal#version-control
+gemfiles/*.gemfile.lock

data/.gitignore

@@ -10,3 +10,7 @@
 
 # rspec failure tracking
 .rspec_status
+
+# recommended:
+# https://github.com/thoughtbot/appraisal#version-control
+gemfiles/*.gemfile.lock

data/.ruby-version

@@ -1 +1 @@
-ruby-2.2.3
+ruby-2.5.5

data/.travis.yml

@@ -1,5 +1,28 @@
-sudo: false
+dist: xenial
 language: ruby
+services:
+  - docker
 rvm:
-  - 2.2.3
+  - 2.2.10
+  - 2.3.8
+  - 2.4.6
+  - 2.5.5
+  - 2.6.3
+gemfile:
+  - gemfiles/activesupport_5.gemfile
+  - gemfiles/activesupport_6.gemfile
+matrix:
+  exclude:
+    - rvm: 2.2.3
+      gemfile: gemfiles/activesupport_6.gemfile
+    - rvm: 2.2.10
+      gemfile: gemfiles/activesupport_6.gemfile
+    - rvm: 2.3.8
+      gemfile: gemfiles/activesupport_6.gemfile
+    - rvm: 2.4.6
+      gemfile: gemfiles/activesupport_6.gemfile
+
 before_install: gem install bundler -v 1.15.4
+
+script:
+  - bundle exec rake spec_all

data/Appraisals

@@ -0,0 +1,8 @@
+appraise "activesupport-5" do
+  gem "activesupport", "~> 5.0"
+end
+
+appraise "activesupport-6" do
+  gem "activesupport", "~> 6.0"
+end
+

data/CHANGELOG.md

@@ -4,6 +4,71 @@ This log maintains a list of all substantive changes to Runbook. The log include
 
 ## master
 
+## `v1.0.0` (2020-07-24)
+
+### Breaking Changes:
+
+* Commands and tmux commands that previously escaped single quotes will now require un-escaped single quotes
+
+### Features:
+
+* Add Node#parent_entity to find the containinng entity for a node
+
+### Fixes:
+
+* Fix bugs requiring escaping single quotes in commands and tmux_commands (BACKWARDS INCOMPATIBLE CHANGE)
+* Fix `File.exists?` deprecation warning (Thanks onk!)
+
+### Documentation:
+
+* Add suppress_capture_output.rb runbook example
+
+## `v0.16.1` (2019-11-25)
+
+### Fixes:
+
+* Fix bug preventing skipping of steps not nested in sections (Thanks celeen!)
+
+## `v0.16.0` (2019-11-22)
+
+### Fixes:
+
+* Add better error messages for runtime values accessed at compile time
+
+### New Features
+
+* Add entity tags and labels
+* Add `setup` entity for initial runbook setup code
+* Add `Runbook.views` method for accessing an array of all defined views
+* Add airbrussh context for better ssh_kit output
+* Backtick "into" targets in markdown view output (Thanks fwolfst!)
+
+## `v0.15.0` (2019-09-29)
+
+### Fixes:
+
+* Halt the project generator if gem generation fails
+* Replace timeout_statement with abort_statement for assert statements
+* Make runbook state files only readable by the current user
+* Allow / characters in the title of a runbook (Thanks brafales!)
+
+### New Features
+
+* Allow books to have steps as children
+* Allow "echo: false" for ask statements
+* Expose "run" as an argument to ruby_commands
+
+## `v0.14.0` (2019-08-15)
+
+### Fixes:
+
+* Relax gem dependencies to be compatible with gems up to the next major version
+
+### New Features
+
+* Alias `install` cli command to `init`
+* Add `Runbook.config` alias for `Runbook.configuration`
+
 ## `v0.13.0` (2019-07-10)
 
 ### Potentially Breaking Changes:

data/README.md

@@ -1,6 +1,10 @@
 # Runbook
 
 [![Gem Version](https://badge.fury.io/rb/runbook.svg)](https://badge.fury.io/rb/runbook)
+[![Build Status](https://travis-ci.org/braintree/runbook.svg?branch=master)](https://travis-ci.org/braintree/runbook)
+[![Gitter](https://badges.gitter.im/braintree/runbook.svg)](https://gitter.im/braintree/runbook?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
+
+_See our [blog post](https://medium.com/braintree-product-technology/https-medium-com-braintree-product-technology-runbook-be6f072cfc0d) for the philosophy behind Runbook and an overview of its features._
 
 Runbook provides a DSL for specifying a series of steps to execute an operation. Once your runbook is specified, you can use it to generate a formatted representation of the book or to execute the runbook interactively. For example, you can export your runbook to markdown or use the same runbook to execute commands on remote servers.
 
@@ -26,7 +30,7 @@ Lastly, Runbook provides an extendable interface for augmenting the DSL and defi
 * **Dynamic Control Flow** - Runbooks can start execution at any step and can skip steps based on user input.
 * **Resumable** - Runbooks save their state at each step. If your runbook encounters an error, you can resume your runbook at the previous step after addressing the error.
 * **Noop and Auto Modes** - Runbooks can be executed in noop mode. This allows you to see what a runbook will do before it executes. Runbooks can be run in auto mode to eliminate the need for human interaction.
-* **Execution Lifecycle Hooks** - Runbook provides before, after, around hooks to augment its execution behavior.
+* **Execution Lifecycle Hooks** - Runbook provides before, after, and around hooks to augment its execution behavior.
 * **Tmux Integration** - Runbook integrates with [tmux](https://github.com/tmux/tmux). You can define terminal pane layouts and send commands to terminal panes.
 * **Generators** - Runbook provides commands to generate runbooks, extensions, and runbook projects. You can define your own generators for easy, customized runbook creation.
 * **Extendable DSL** - Runbook's DSL is designed to be extendable. You can extend its DSL to add your own behavior.
@@ -107,18 +111,20 @@ Install the Runbook gem:
 
     $ bundle install
 
-Install Runbook into your project:
+Initialize Runbook in your project:
 
-    $ bundle exec runbook install
+    $ bundle exec runbook init
 
 ## Contents
 
 * [1. Runbook Anatomy](#runbook-anatomy)
   * [1.1 Entities, Statements, and Setters](#entities-statements-and-setters)
-    * [1.1.1 Books, Sections, and Steps](#books-sections-and-steps)
+    * [1.1.1 Books, Sections, Steps, and Setup](#books-sections-steps-and-setup)
       * [1.1.1.1 Books](#books)
       * [1.1.1.2 Sections](#sections)
       * [1.1.1.3 Steps](#steps)
+      * [1.1.1.4 Setup](#setup)
+      * [1.1.1.5 Tags and Labels](#tags-and-labels)
     * [1.1.2 Statements](#statements)
       * [1.1.2.1 Ask](#ask)
       * [1.1.2.2 Assert](#assert)
@@ -137,38 +143,41 @@ Install Runbook into your project:
       * [1.1.2.15 Wait](#wait)
       * [1.1.2.16 Tmux Layouts](#tmux-layouts)
     * [1.1.3 Setters](#setters)
-* [2. Configuration](#configuration)
-  * [2.1 Configuration Files](#configuration-files)
-* [3. Working With Runbooks](#working-with-runbooks)
-  * [3.1 From Within Your Project](#from-within-your-project)
-  * [3.2 Via The Command Line](#via-the-command-line)
-  * [3.3 Self-executable](#self-executable)
+* [2. Working With Runbooks](#working-with-runbooks)
+  * [2.1 Via The Command Line](#via-the-command-line)
+  * [2.2 From Within Your Project](#from-within-your-project)
+  * [2.3 Self-executable](#self-executable)
+* [3. Configuration](#configuration)
+  * [3.1 Configuration Files](#configuration-files)
 * [4. Best Practices](#best-practices)
   * [4.1 Iterative Automation](#iterative-automation)
   * [4.2 Parameterizing Runbooks](#parameterizing-runbooks)
-  * [4.3 Execution Best Practices](#execution-best-practices)
-  * [4.4 Composing Runbooks](#composing-runbooks)
-  * [4.5 Deep Nesting](#deep-nesting)
-  * [4.6 Load Vs. Eval](#load-vs-eval)
-  * [4.7 Passing State](#passing-state)
+  * [4.3 Passing State](#passing-state)
+  * [4.4 Execution Best Practices](#execution-best-practices)
+  * [4.5 Remote Command Execution](#remote-command-execution)
+  * [4.6 Composing Runbooks](#composing-runbooks)
+  * [4.7 Deep Nesting](#deep-nesting)
+  * [4.8 Load Vs. Eval](#load-vs-eval)
 * [5. Generators](#generators)
   * [5.1 Predefined Generators](#predefined-generators)
   * [5.2 Custom Generators](#custom-generators)
 * [6. Extending Runbook](#extending-runbook)
-  * [6.1 Adding Runs and Views](#adding-runs-and-views)
-  * [6.2 DSL Extensions](#dsl-extensions)
-  * [6.3 Adding New Statements](#adding-new-statements)
-  * [6.4 Adding Run and View Functionality](#adding-run-and-view-functionality)
+  * [6.1 Adding New Statements](#adding-new-statements)
+  * [6.2 Adding Run and View Functionality](#adding-run-and-view-functionality)
+  * [6.3 DSL Extensions](#dsl-extensions)
+  * [6.4 Adding Runs and Views](#adding-runs-and-views)
   * [6.5 Augmenting Functionality With Hooks](#augmenting-functionality-with-hooks)
   * [6.6 Adding New Run Behaviors](#adding-new-run-behaviors)
   * [6.7 Adding to Runbook's Run Metadata](#adding-to-runbooks-run-metadata)
   * [6.8 Adding to Runbook's Configuration](#adding-to-runbooks-configuration)
-* [7. Known Issues](#known-issues)
-* [8. Development](#development)
-* [9. Contributing](#contributing)
-* [10. Feature Requests](#feature-requests)
-* [11. License](#license)
-* [12. Code of Conduct](#code-of-conduct)
+* [7. Testing](#testing)
+* [8. Known Issues](#known-issues)
+* [9. FAQ](#faq)
+* [10. Development](#development)
+* [11. Contributing](#contributing)
+* [12. Feature Requests](#feature-requests)
+* [13. License](#license)
+* [14. Code of Conduct](#code-of-conduct)
 
 ## Runbook Anatomy
 
@@ -215,7 +224,7 @@ Hierarchically, a runbook looks like this:
 
 A runbook is composed of entities, statements, and setters. Runbook entities contain either other entities or statements. Examples of entities include Books, Sections, and Steps. They define the structure of the runbook and can be considered the "nodes" of the tree structure. As entities are the nodes of the tree structure, statements are the "leaves" of the structure and comprise the various behaviors or commands of the runbook. Setters, typically referenced from within steps, associate state with the node, which can be accessed by its children.
 
-#### Books, Sections, and Steps
+#### Books, Sections, Steps, and Setup
 
 Entities are composed of a title and a list of items which are their children. Each entity can be rendered with a specific view or executed with a specific run.
 
@@ -228,7 +237,7 @@ Runbook.book "Unbalance node" do
 end
 ```
 
-Every book requires a title. Books can have description, layout, and section children. Descriptions describe the book and are declared with the `description` keyword.
+Every book requires a title. Books can have description, layout, section, and step children. Descriptions describe the book and are declared with the `description` keyword.
 
 ##### Sections
 
@@ -238,18 +247,78 @@ A book is broken up into sections. Every section requires a title. Sections can
 
 Steps hold state and group together a set of statements. Steps do not require titles or children. This allows runbooks to be very flexible. You can fill out steps as needed, or be terse when the behavior of the step is self-evident. Steps without titles will not prompt to continue when running in paranoid mode.
 
+##### Setup
+
+Setup is a special entity that is always executed. It is not skipped when starting or resuming execution in the middle of a runbook. A prompt is never presented to determine if you should or should not execute the setup section. The setup section is similar to the step entity in that it shares the same DSL. In other words, any keywords available within steps are also available within the setup section.
+
+The setup section provides two important use cases. First, it allows you ensure any dependent values are defined when executing your runbook. If skipping the initial steps of your runbook and starting in the middle, you can be sure that any initialization steps have been executed. For example, presume you have a runbook that prompts for a list of servers, stops the servers, and then starts them. It would be advantageous to define the prompting server logic in a setup section, so you can start the runbook at the start server step and know that the list of servers is defined.
+
+Second, if you dynamically define the sections and steps in your runbook based on user input, then doing this in the setup section allows you start the runbook in the middle of the dynamically defined steps.
+
+Because the setup section is always executed, it's execution should be idempotent. In other words, the setup section should be able to be executed multiple times in a row and produce the same result.
+
+It may be ideal to ensure user input is only asked for once when executing a setup section.
+
+```ruby
+Runbook.book "Make Pizza" do
+  setup do
+    ruby_command do
+      @toppings ||= ENV["TOPPINGS"]
+      ask "What toppings would you like?", into: :toppings, default: "cheese" unless @toppings
+    end
+  end
+end
+```
+
+The above example will set `@toppings` from a passed-in environment variable if present, otherwise it will ask the user to set `@toppings`. If toppings have already has been defined from a previous execution, it will not prompt the user for the value again. Because this logic references a value that is defined at runtime (`@toppings`), it must be wrapped in a `ruby_command`.
+
+##### Tags and Labels
+
+Any entity can be associated with arbitrary tags or labels. Once tags or labels are assigned, entity behavior can be modified using [hooks](#augmenting-functionality-with-hooks).
+
+```ruby
+Runbook.book "Bounce Nodes", :untested do
+  step "Disable monitoring", :skip do
+    confirm "Have you disabled health monitoring?"
+  end
+
+  step "Restart nodes", :aws_only, :mutator, labels: {rails_env: :production} do
+    confirm "Have you restarted the nodes?"
+  end
+end
+```
+
+```ruby
+Runbook::Runs::SSHKit.register_hook(:warn_for_untested_runbook, :before, Runbook::Entities::Book) do |object, metadata|
+  warning = "This runbook has not yet been tested. Beware of bugs!"
+  metadata[:toolbox].warn(warning) if object.tags.include?(:untested)
+end
+
+Runbook::Runs::SSHKit.register_hook(:skip_skippable_entities, :around, Runbook::Entity) do |object, metadata, block|
+  next if object.tags.include?(:skip)
+  next if object.labels[:rails_env] && object.labels[:rails_env] != ENV["RAILS_ENV"]
+  block.call
+end
+```
+
 #### Statements
 
 Statements are the workhorses of runbooks. They comprise all the behavior runbooks execute. Runbook comes with the following statements:
 
 ##### Ask
 
-Prompts the user for a string and stores its value on the containing step entity. Once this statement is executed, its value is accessed as an instance variable under the `into` parameter. This value can be referenced in later statements such as the `ruby_command` statement.
+Prompts the user for a string and stores its value on the containing step entity. Once this statement is executed, its value is accessed as an instance variable under the `into` parameter. This value can be referenced in later statements such as the `ruby_command` statement. An optional `default` value can be specified. An optional `echo` parameter can be specified to indicate whether typed input should be echoed to the screen.
 
 ```ruby
-ask "What percentage of requests are failing?", into: :failing_request_percentage, default: "100"
+ask "What percentage of requests are failing?", into: :failing_request_percentage, default: "100", echo: true
+
+ruby_command do
+  note "Failing request percentage: #{@failing_request_percentage}"
+end
 ```
 
+In the above example, the `note` statement must be wrapped in a `ruby_command` statement. Without wrapping `note` in a `ruby_command`, it would be evaluated at compile time but the user will only be asked for input when the runbook is executed (so `@failing_request_percentage` would not have a value). If you find yourself wrapping many or all runbook statements in ruby commands it may make sense to set these values at compile time using environment variables.
+
 ##### Assert
 
 Runs the provided `cmd` repeatedly until it returns true. A timeout and maximum number of attempts can be set. You can specify a command to be run if a timeout occurs or the maximum number of attempts is hit. Commands can optionally be specified as `raw`. This tells SSHKit to not perform auto-wrapping of the commands, but execute the exact string on the remote server. See SSHKit's documentation for more details.
@@ -262,7 +331,7 @@ assert(
   interval: 3, # seconds
   timeout: 300, # seconds
   attempts: 3,
-  timeout_statement: Runbook::Statements::Command.new(
+  abort_statement: Runbook::Statements::Command.new(
     "echo 'help' | mail -s 'need help' page-me@page-me.com",
     ssh_config: {servers: [:local], parallelization: {strategy: :parallel}},
     raw: false
@@ -272,7 +341,7 @@ assert(
 
 ##### Capture
 
-Runs the provided `cmd` and captures its output into `into`. An optional `ssh_config` can be specified to configure how the capture command gets run. Capture commands take an optional `strip` parameter that indicates if the returned output should have leading and trailing whitespace removed. Capture commands also take an optional `raw` parameter that tells SSHKit whether the command should be executed as is, or to include the auto-wrapping of the ssh_config.
+Runs the provided `cmd` and captures its output into `into`. Once captured, this value can be referenced in later statements such as the `ruby_command` statement. An optional `ssh_config` can be specified to configure how the capture command gets run. Capture commands take an optional `strip` parameter that indicates if the returned output should have leading and trailing whitespace removed. Capture commands also take an optional `raw` parameter that tells SSHKit whether the command should be executed as is, or to include the auto-wrapping of the ssh_config.
 
 ```ruby
 capture %Q{wc -l file.txt | cut -d " " -f 1}, into: :num_lines, strip: true, ssh_config: {user: "root"}
@@ -318,7 +387,7 @@ DESC
 Downloads the specified file to `to`. An optional `ssh_config` can be specified to configure how the download command gets run, for example specifying the remote host and remote directory to download from. Optional `options` can be specified that get passed down to the underlying sshkit implementation
 
 ```ruby
-download /home/pblesi/rad_file.txt, to: my_rad_file.txt, ssh_config: {servers: ["host1.prod"]}, options: {log_percent: 10}
+download '/home/pblesi/rad_file.txt', to: my_rad_file.txt, ssh_config: {servers: ["host1.prod"]}, options: {log_percent: 10}
 ```
 
 ##### Layout
@@ -350,10 +419,10 @@ notice "There be dragons!"
 
 ##### Ruby Command
 
-Executes its block in the context of the parent step. The block is passed the ruby_command statement and the execution metadata as arguments.
+Executes its block in the context of the parent step. The block is passed the ruby_command statement, the execution metadata, and the run as arguments.
 
 ```ruby
-ruby_command do |rb_cmd, metadata|
+ruby_command do |rb_cmd, metadata, run|
   if (failure_rate = rb_cmd.parent.failing_request_percentage) > 25
     `echo 'Help! failure rate at #{failure_rate}' | mail -s 'High failure rate!' page-me@page-me.com`
   else
@@ -539,51 +608,10 @@ step do
 end
 ```
 
-## Configuration
-
-Runbook is configured using its configuration object. Below is an example of how to configure Runbook.
-
-```ruby
-Runbook.configure do |config|
-  config.ssh_kit.umask = "077"
-  config.ssh_kit.default_runner_config = {in: :groups, limit: 5}
-  config.ssh_kit.default_env = {rails_env: :staging}
-
-  config.enable_sudo_prompt = true
-  config.use_same_sudo_password = true
-end
-```
-
-If the `ssh_kit` configuration looks familiar, that's because it's an SSHKit Configuration object. Any configuration options set on `SSHKit.config` can be set on `config.ssh_kit`.
-
-### Configuration Files
-
-Runbook loads configuration from a number of predefined files. Runbook will attempt to load configuration from the following locations on startup: `/etc/runbook.conf`, a `Runbookfile` in a parent directory from the current directory, a `.runbook.conf` file in the current user's home directory, a file specified with `--config` on the command line, any configuration specified in a runbook. Runbook will also load configuration from these files in this order of preference, respectively. That is, configuration values specified at the project level (`Runbookfile`) will override configuration values set at the global level (`/etc/runbook.conf`), etc.
-
 ## Working With Runbooks
 
 You can integrate with Runbook in several different ways. You can create your own project or incorporate Runbook into your existing projects. You can use Runbook via the command line. And you can even create self-executing runbooks.
 
-### From Within Your Project
-
-Runbooks can be executed using the `Runbook::Viewer` and `Runbook::Runner` classes.
-
-#### Executing a runbook using `Runbook::Viewer`
-
-```ruby
-Runbook::Viewer.new(book).generate(view: :markdown)
-```
-
-In this case book is a `Runbook::Entities::Book` and `:markdown` refers to the specific view type (`Runbook::Views::Markdown`).
-
-#### Executing a runbook using `Runbook::Runner`
-
-```ruby
-Runbook::Runner.new(book).run(run: :ssh_kit, noop: false, auto: false, paranoid: true, start_at: "0")
-```
-
-This will execute `book` using the `Runbook::Runs::SSHKit` run type. It will not run the book in `noop` mode. It will not run the book in `auto` mode. It will run the book in `paranoid` mode. And it will start at the beginning of the book. Noop mode runs the book without side-effects outside of printing what it will execute. Auto mode will skip any prompts in the runbook. If there are any required prompts in the runbook (such as the `ask` statement), then the run will fail. Paranoid mode will prompt the user for whether they should continue at every step. Finally `start_at` can be used to skip parts of the runbook or to restart at a certain point in the event of failures, stopping and starting the runbook, etc.
-
 ### Via The Command Line
 
 Runbook can be used to write stand-alone runbook files that can be executed via the command line. Below is a list of examples of how to use Runbook via the command line.
@@ -637,6 +665,26 @@ the runbook at runtime.
 $ HOSTS="appbox{01..30}.prod" ENV="production" runbook exec --start-at 1.2.1 my_runbook.rb
 ```
 
+### From Within Your Project
+
+Runbooks can be executed using the `Runbook::Viewer` and `Runbook::Runner` classes. Using these classes, you can invoke runbooks from within your existing codebase. This could be ideal for several reasons. It allows you to maintain a consistent interface with other system tasks such as rake tasks or cap tasks. It allows you to perform setup or sanity check functionality before executing your runbooks. And it allows you to load an environment to be accessed within your runbooks, such as providing access to a canonical list of servers or shared business logic.
+
+#### Executing a runbook using `Runbook::Viewer`
+
+```ruby
+Runbook::Viewer.new(book).generate(view: :markdown)
+```
+
+In this case book is a `Runbook::Entities::Book` and `:markdown` refers to the specific view type (`Runbook::Views::Markdown`).
+
+#### Executing a runbook using `Runbook::Runner`
+
+```ruby
+Runbook::Runner.new(book).run(run: :ssh_kit, noop: false, auto: false, paranoid: true, start_at: "0")
+```
+
+This will execute `book` using the `Runbook::Runs::SSHKit` run type. It will not run the book in `noop` mode. It will not run the book in `auto` mode. It will run the book in `paranoid` mode. And it will start at the beginning of the book. Noop mode runs the book without side-effects outside of printing what it will execute. Auto mode will skip any prompts in the runbook. If there are any required prompts in the runbook (such as the `ask` statement), then the run will fail. Paranoid mode will prompt the user for whether they should continue at every step. Finally `start_at` can be used to skip parts of the runbook or to restart at a certain point in the event of failures, stopping and starting the runbook, etc.
+
 ### Self-executable
 
 Runbooks can be written to be self-executable
@@ -673,6 +721,27 @@ runbook = Runbook.books.last # Runbooks register themselves to Runbook.books whe
 Runbook::Runner.new(runbook).run(auto: true)
 ```
 
+## Configuration
+
+Runbook is configured using its configuration object. Below is an example of how to configure Runbook.
+
+```ruby
+Runbook.configure do |config|
+  config.ssh_kit.umask = "077"
+  config.ssh_kit.default_runner_config = {in: :groups, limit: 5}
+  config.ssh_kit.default_env = {rails_env: :staging}
+
+  config.enable_sudo_prompt = true
+  config.use_same_sudo_password = true
+end
+```
+
+If the `ssh_kit` configuration looks familiar, that's because it's an SSHKit Configuration object. Any configuration options set on `SSHKit.config` can be set on `config.ssh_kit`.
+
+### Configuration Files
+
+Runbook loads configuration from a number of predefined files. Runbook will attempt to load configuration from the following locations on startup: `/etc/runbook.conf`, a `Runbookfile` in a parent directory from the current directory, a `.runbook.conf` file in the current user's home directory, a file specified with `--config` on the command line, any configuration specified in a runbook. Runbook will also load configuration from these files in this order of preference, respectively. That is, configuration values specified at the project level (`Runbookfile`) will override configuration values set at the global level (`/etc/runbook.conf`), etc.
+
 ## Best Practices
 
 The following are best practices when developing your own runbooks.
@@ -695,12 +764,60 @@ rails_env = `facter rails_env`
 customer_list = File.read("/tmp/customer_list.txt")
 ```
 
+### Passing State
+
+Runbook provides a number of different mechanisms for passing state throughout a runbook. For any data that is known at compile time, local variables can be used because Runbooks are lexically scoped.
+
+```ruby
+home_planet = "Krypton"
+Runbook.book "Book Using Local Variables" do
+  hometown = "Smallville"
+
+  section "My Biography" do
+    step do
+      note "Home Planet: #{home_planet}"
+      note "Home Town: #{hometown}"
+    end
+  end
+end
+```
+
+When looking to pass data generated at runtime, for example data from `ruby_command`, `ask`, or `capture` statements, Runbook persists and synchronizes instance variables for these commands.
+
+```ruby
+Runbook.book "Book Using Instance Variables" do
+  section "The Transported Man" do
+    step do
+      ask "Who's the greatest magician?", into: :greatest, default: "Alfred Borden"
+      ruby_command { @magician = "Robert Angier" }
+    end
+
+    step do
+      ruby_command {
+        note "Magician: #{@magician}"
+        note "Greatest Magician: #{@greatest}"
+      }
+    end
+  end
+end
+```
+
+Instance variables are only passed between statements such as `ruby_command`. They should not be set on entities such as steps, sections, or books. Instance variables are persisted using `metadata[:repo]`. They are copied to the repo after each statement finishes executing and copied from the repo before each statement starts executing. Because instance variables utilize the repo, they are persisted if the runbook is stopped and restarted at the same step.
+
+Be careful with your naming of instance variables as it is possible to clobber the step's DSL methods because they share the same namespace.
+
 ### Execution Best Practices
 
 As a best practice, Runbooks should always be nooped before they are run. This will allow you to catch runtime errors such as using the ask statement when running in auto mode, typos in your runbooks, and to visually confirm what will be executed.
 
 Additionally, it can be nice to have a generated view of the runbook you are executing to have a good high-level overview of the steps in the runbook.
 
+### Remote Command Execution
+
+Runbook uses [SSHKit](https://github.com/capistrano/sshkit) for remote command execution. When specifying `servers`, you are specifying the target host to execute the command. If you want to use a non-standard port or login using a different user than your current user, then you can specify the `server` as `lucy@host1.prod:2345`. Alternatively, you can use an ssh config file such as `~/.ssh/config` to specify the user and port used to ssh to a given host. See [Capistrano's SSH setup instructions](https://capistranorb.com/documentation/getting-started/installation/#ssh) for further support on setting up SSH to execute commands on remote hosts.
+
+The `user` setter designates the user you will sudo as once sshed to the remote host. Runbook supports password-protected sudo execution. That is, if your server requires a password to execute commands as another user, Runbook will allow you to enter your password when prompted. The `enable_sudo_prompt` configuration value controls this behavior. Enabling the sudo password prompt requires that your commands execute using a tty, which can lead to unexpected behavior when executing certain commands. Enabling `use_same_sudo_password` will use the same password accross different hosts and users instead of re-prompting for each unique user/host combo.
+
 ### Composing Runbooks
 
 Runbooks can be composed using the `add` keyword. Below is an example of composing a runbook from smaller, reusable components.
@@ -720,6 +837,8 @@ Runbook.book "Update configuration" do
 end
 ```
 
+If you want to parameterize these runbook snippets, you can place them in a ruby function that takes arguments and generates the desired entity or statement. If these snippets set information that is used by the runbook, such as with `capture` statements, it is a good practice to parameterize where the result is stored. This lets the snippet fit different contexts and makes clear what data is being returned from the snippet.
+
 ### Deep Nesting
 
 Because the Runbook DSL is declarative, it is generally discouraged to develop elaborate nested decision trees. For example, it is discouraged to use the `ask` statement to gather user feedback, branch on this information in a `ruby_command`, and follow completely separate sets of steps. This is because deep nesting eliminates the benefits of the declarative DSL. You can no longer noop the deeply nested structure for example.
@@ -734,7 +853,7 @@ step "Inspect plate" do
     when "carrots"
       add carrots_book
     when "peas"
-      system("runbook exec samples/print_peas.rb")
+      system("runbook exec examples/print_peas.rb")
     else
       metadata[:toolbox].warn("Found #{veggie}!")
     end
@@ -759,48 +878,6 @@ runbook = eval(File.read("my_runbook.rb"))
 
 Loading your runbook file is more ideal, but adds slight complexity. This method is prefered because the Ruby mechanism for retrieving source code does not work for code that has been `eval`ed. This means that you will not see `ruby_command` code blocks in view and noop output when using the `eval` method. You will see an "Unable to retrieve source code" message instead.
 
-### Passing State
-
-Runbook provides a number of different mechanisms for passing state throughout a runbook. For any data that is known at compile time, local variables can be used because Runbooks are lexically scoped.
-
-```ruby
-home_planet = "Krypton"
-Runbook.book "Book Using Local Variables" do
-  hometown = "Smallville"
-
-  section "My Biography" do
-    step do
-      note "Home Planet: #{home_planet}"
-      note "Home Town: #{hometown}"
-    end
-  end
-end
-```
-
-When looking to pass data generated at runtime, for example data from `ruby_command`, `ask`, or `capture` statements, Runbook persists and synchronizes instance variables for these commands.
-
-```ruby
-Runbook.book "Book Using Instance Variables" do
-  section "The Transported Man" do
-    step do
-      ask "Who's the greatest magician?", into: :greatest, default: "Alfred Borden"
-      ruby_command { @magician = "Robert Angier" }
-    end
-
-    step do
-      ruby_command {
-        note "Magician: #{@magician}"
-        note "Greatest Magician: #{@greatest}"
-      }
-    end
-  end
-end
-```
-
-Instance variables are only passed between statements such as `ruby_command`. They should not be set on entities such as steps, sections, or books. Instance variables are persisted using `metadata[:repo]`. They are copied to the repo after each statement finishes executing and copied from the repo before each statement starts executing. Because instance variables utilize the repo, they are persisted if the runbook is stopped and restarted at the same step.
-
-Be careful with your naming of instance variables as it is possible to clobber the step's DSL methods because they share the same namespace.
-
 ## Generators
 
 Runbook provides a number of generators accessible via the command line that can be used to generate code for new runbooks, Runbook projects, and Runbook extensions. Additionally, Runbook provides a generator generator so you can define your own custom generators.
@@ -849,28 +926,47 @@ Generate your own generator using the `generate generator` command
 
 Runbook can be extended to add custom functionality.
 
-### Adding Runs and Views
+### Adding New Statements
 
-You can add new run and view types by defining modules under `Runbook:::Runs` and `Runbook::Views` respectively. They will automatically be accessible from the command line or via the `Runner` and `Viewer` classes. See `lib/runbook/runs/ssh_kit.rb` or `lib/runbook/views/markdown.rb` for examples of how to implement runs and views.
+In order to add a new statement to your DSL, create a class under `Runbook::Statements` that inherits from `Runbook::Statement`. This statement will be initialized with all arguments passed to the corresponding keyword in the DSL. Remember to also add a corresponding method to runs and views so your new statement can be interpreted in each context.
 
 ```ruby
-module Runbook::Views
-  module Yaml
-    include Runbook::View
+module Runbook::Statements
+  class Diagram < Runbook::Statement
+    attr_reader :alt_text, :url
 
-    # handler names correspond to the entity or statement class name
-    # Everything is underscored and "::" is replaced by "__"
-    def self.runbook__entities__book(object, output, metadata)
-      output << "---\n"
-      output << "book:\n"
-      output << "  title: #{object.title}\n"
+    def initialize(alt_text, url)
+      @alt_text = alt_text
+      @url = url
     end
+  end
+end
+```
 
-    # Add other handlers here
+In the above example a keyword `diagram` will be added to the step dsl and its arguments will be used to initialize the Diagram object.
+
+New statements can be generated using the statement generator.
+
+    $ runbook generate statement diagram --root lib/runbook/extensions
+
+### Adding Run and View Functionality
+
+You can add handlers for new statements and entities to your runs and views by prepending the modules with the new desired functionality.
+
+```ruby
+module MyRunbook::Extensions
+  module Diagram
+    def self.runbook__entities__diagram(object, output, metadata)
+      output << "![#{object.alt_text}](#{object.url})"
+    end
   end
+
+  Runbook::Views::Markdown.prepend(Diagram)
 end
 ```
 
+If you are not modifying existing methods, you can simply re-open the module to add new methods.
+
 ### DSL Extensions
 
 You can add arbitrary keywords to your entity DSLs. For example, you could add an alias to Runbook's Book DSL as follows:
@@ -889,43 +985,32 @@ module MyRunbook::Extensions
 end
 ```
 
-### Adding New Statements
-
-In order to add a new statement to your DSL, create a class under `Runbook::Statements` that inherits from `Runbook::Statement`. This statement will be initialized with all arguments passed to the corresponding keyword in the DSL. Remember to also add a corresponding method to runs and views so your new statement can be interpretted in each context.
-
-```ruby
-module Runbook::Statements
-  class Diagram < Runbook::Statement
-    attr_reader :alt_text, :url
-
-    def initialize(alt_text, url)
-      @alt_text = alt_text
-      @url = url
-    end
-  end
-end
-```
+DSL extensions can be generated using the dsl_extension generator.
 
-In the above example a keyword `diagram` will be added to the step dsl and its arguments will be used to initialize the Diagram object.
+    $ runbook generate dsl_extension aliases --root lib/runbook/extensions
 
-### Adding Run and View Functionality
+### Adding Runs and Views
 
-You can add handlers for new statements and entities to your runs and views by prepending the modules with the new desired functionality.
+You can add new run and view types by defining modules under `Runbook:::Runs` and `Runbook::Views` respectively. They will automatically be accessible from the command line or via the `Runner` and `Viewer` classes. See `lib/runbook/runs/ssh_kit.rb` or `lib/runbook/views/markdown.rb` for examples of how to implement runs and views.
 
 ```ruby
-module MyRunbook::Extensions
-  module Diagram
-    def self.runbook__entities__diagram(object, output, metadata)
-      output << "![#{object.alt_text}](#{object.url})"
+module Runbook::Views
+  module Yaml
+    include Runbook::View
+
+    # handler names correspond to the entity or statement class name
+    # Everything is underscored and "::" is replaced by "__"
+    def self.runbook__entities__book(object, output, metadata)
+      output << "---\n"
+      output << "book:\n"
+      output << "  title: #{object.title}\n"
     end
-  end
 
-  Runbook::Views::Markdown.prepend(Diagram)
+    # Add other handlers here
+  end
 end
 ```
 
-If you are not modifying existing methods, you can simply re-open the module to add new methods.
-
 ### Augmenting Functionality With Hooks
 
 You can add `before`, `after`, or `around` hooks to any statement or entity by defining a hook on a `Run` or `View`.
@@ -948,6 +1033,22 @@ end
 
 When registering a hook, you specify the name of the hook, the type, and the statement or entity to add the hook to. `before` and `after` hooks execute the block before and after executing the entity or statement, respectively. `around` hooks take a block which executes the specified entity or statement. When specifying the class that the hook applies to, you can have the hook apply to all entities by specifying `Runbook::Entity`, all statements by specifying `Runbook::Statement`, or all items by specifying `Object`. Additionally, you can specify any specific entity or statement you would like the hook to apply to.
 
+Hooks are defined on the run or view objects themselves. For example, you would register a hook with `Runbook::Runs::SSHKit` to have the hook be applied to the `SSHKit` run. You would register a hook with the `Runbook::Views::Markdown` view to have hooks apply to this view. If you want to apply a hook to all runs or views, you can use the `Runbook.runs` method or `Runbook.views` method to iterate through the runs or views respectively.
+
+```ruby
+Runbook.runs.each do |run|
+  run.register_hook(
+    :give_words_of_encouragement,
+    :before,
+    Runbook::Entities::Book
+  ) do |object, metadata|
+    metadata[:toolbox].output("You've got this!")
+  end
+end
+```
+
+Hooks can be defined anywhere prior to runbook execution. If defining a hook for only a single runbook, it makes sense to define the hook immediately prior to the runbook definition. If you want a hook to apply to all runbooks in your project, it can be defined in a config file such as the `Runbookfile`. If you want to selectively apply the hook to certain runbooks, it may make sense to define it in a file that can be required by runbooks when it is needed.
+
 When starting at a certain position in the runbook, hooks for any preceding sections and steps will be skipped. After hooks will be run for a parent when starting at a child entity of a parent.
 
 ### Adding New Run Behaviors
@@ -1024,11 +1125,24 @@ module MyRunbook::Extensions
 end
 ```
 
-This will add a `log_level` attribute to Runbook's configuration with a default value of `:info`.
+This will add a `log_level` attribute to Runbook's configuration with a default value of `:info`. This configuration value can be accessed via `Runbook.config.log_level`.
+
+## Testing
+
+Runbooks are inherently difficult to test because they are primarily composed of side-effects. That being said, there are a number of strategies you can employ to test your runbooks.
+
+1. Push complex logic to stand-alone Ruby objects that can be tested in isolation
+2. Use `TEST` or `DEBUG` environment variables to conditionally disable side-effects during execution
+3. Execute your runbooks in staging environments
+4. Noop your runbooks to understand what they will be executing before executing them
+
+See Runbook's test suite for more ideas on how to test your runbooks. For example, Runbook uses [aruba](https://github.com/cucumber/aruba) to test Runbook at the CLI level.
+
+Additionally, runbooks should contain their own assertions, sanity checks, monitoring, and alerting to mitigate errors and alert you if intervention is required.
 
 ## Known Issues
 
-### Command Quoting
+### Command Quoting (Prior to v1.0)
 
 Because ssh_config declarations such as `user`, `group`, `path`, `env`, and `umask` are implemented as wrappers around your provided commands, you must be aware that issues can arise if your commands contain characters such as single quotes that are not properly escaped.
 
@@ -1046,6 +1160,8 @@ command "echo '\\''I love cheese'\\''"
 
 Alternatively, if you wish to avoid issues with SSHKit command wrapping, you can specify that your commands be executed in raw form, passed directly as written to the specified host.
 
+`tmux_command` wraps the input passed to it in single quotes. Therefore any single quotes passed to the `tmux_command` should be escaped using `'\\''`. This issue can manifest itself as part of the command not being echoed to the tmux pane.
+
 ### Specifying env values
 
 When specifying the `env` for running commands, if you place curly braces `{}` around the env values, it is required to enclose the arguments in parenthesis `()`, otherwise the following syntax error will result:
@@ -1072,21 +1188,72 @@ not as
 env {rails_env: :production}
 ```
 
+## FAQ
+
+### Are runbooks compiled?
+
+Yes they are. When you define a runbook, a tree data structure is constructed much like an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree). This is important because you do not have to worry about any side-effects such as executing server commands when this data structure is compiled. Once compiled, choosing either the run or view to execute the runbook object determines what behavior is executed at each node.
+
+### Why are runbooks compiled?
+
+Runbook is designed to minimize and mitigate issues that arise when running operations in production enviroments. One way this is accomplished is by compiling all statements in the runbook before execution is started. Validations and assertions can be made to reduce the likelihood that a runbook will encounter an error in the middle of an operation. In other words, Runbook provides some guarantees about proper formatting of a runbook before any commands execute that could affect live systems.
+
+### Why is my variable/method not set?
+
+Because runbooks are compiled, statements that set values such as `ask`, `capture`, and `capture_all` statements (using the `:into` keyword) only expose their values at runtime. This means any references to these methods or variables (specified with `:into`) can only happen within `ruby_command` blocks which are evaluated at runtime. If an argument to a statement references the values set by these statements, then the statement must be wrapped in a `ruby_command` block. See [Passing State](#passing-state) for specific examples.
+
+### How do I define and call methods within a runbook?
+
+When defining and referencing your own functions in a runbook, functions should be wrapped in a module so they can be referenced globally. For example:
+
+```ruby
+module Adder
+  def add(x, y)
+    x + y
+  end
+end
+
+Runbook.book "Add Two Numbers" do
+  step "Add numbers" do
+    ask "X?", into: :x
+    ask "Y?", into: :y
+
+    ruby_command do |rb_cmd, metadata, run|
+      metadata[:toolbox].output("Result: #{Adder.add(x, y)}")
+    end
+  end
+end
+```
+
+### Why does my command work on the command line but not in runbook?
+
+There are a number of reasons why a command may work directly on your command line, but not when executed using the `command` statement. Some possible things to try include:
+
+* Print the command with any variables substituted
+* Ensure the command works outside of runbook
+* Use full paths. The `PATH` environment variable may not be set.
+* Check for aliases. Aliases are usually not set for non-interactive shells.
+* Check for environment variables. Differences between your shell environment variables and those set for the executing shell may modify command behavior.
+* Check for differing behavior between the bourne shell (`sh`) and your shell (usually bash).
+* Check that quotes are properly being escaped.
+* Simplify the command you are executing and then slowly build it back up
+* Check for permissions issues that might cause different execution behavior.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`.
 
-To execute runbook using this repo, run `bundle exec exe/runbook exec samples/layout_runbook.rb`.
+To execute runbook using this repo, run `bundle exec exe/runbook exec examples/layout_runbook.rb`.
 
 To release a new version:
 
 1. Update the version number in `version.rb`.
 2. Update the changelog in `CHANGELOG.rb`.
 3. Commit changes with commit messsage: "Bump runbook version to X.Y.Z"
-4. Run `bundle exec rake release`, which will create a git tag for the version and push git commits and tags.
-5. Push the `.gem` file in `pkg` to your gem repository
+4. Run `gem signin` to ensure you can push the new version to rubygems.org
+5. Run `bundle exec rake release`, which will create a git tag for the version and push git commits and tags.
 
 ## Contributing
 
@@ -1094,7 +1261,7 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/braint
 
 ## Feature Requests
 
-Any feature requests are always welcome and will be considered in accordance with time and need. Additionally, existing feature requests are tracked in TODO.md. If you choose to contribute, your contributions will be greatly appreciated.
+Any feature requests are always welcome and will be considered in accordance with time and need. Additionally, existing feature requests are tracked in TODO.md. If you choose to contribute, your contributions will be greatly appreciated. Please reach out before creating any substantial pull requests. A bit of discussion can save a lot of time and increase the chances that your pull request will be accepted.
 
 ## License