linecook 0.6.2 → 1.0.0

data/History

@@ -1,3 +1,142 @@
+== 1.0.0 2011/04/26
+
+First major release. Significant work on documentation and various bug fixes.
+Simplified run command to only run one script at a time.
+
+== 0.20.0 2011/03/17
+
+* added callback support
+* changed capture_block to accept capture target, and to return
+  the target (not the target output)
+* added capture_str to replace capture_block functionality
+
+== 0.19.1 2011/03/15
+
+* updated the functionality of Recipe#rewrite
+* removed Recipe#rewriteln
+
+== 0.19.0 2011/03/15
+
+* optimizations for helpers
+* removed _erbout tricks from Recipe
+* added Recipe#rewrite
+
+== 0.18.0 2011/03/10
+
+* enforce word method names for helpers
+* linecook prints backtrace on error with $DEBUG
+* added proxy chaining to Recipe
+
+== 0.17.0 2011/03/10
+
+* added hooks for TESTCASE and NAME to test helpers
+* changed helper sections to be identified by leading dash
+
+== 0.16.0 2011/03/09
+
+* added mode support to packages
+* removed preview command
+* added outdent to recipes
+* made run allocate a pseudo-tty
+* updates to allow multitests
+* various patches
+
+== 0.15.1 2011/02/23
+
+* made linecook use Gemfile if available
+* updated scaffold for new project
+* added package file to package dependencies
+
+== 0.15.0 2011/02/23
+
+Continued overhaul of testing methods and structure.
+
+== 0.14.0 2011/02/16
+
+Significant overhaul of testing methods and structure. Cleanup of internal
+structure and documentation.
+
+== 0.13.1 2011/02/03
+
+* added back close methods to template
+* fixed test bug
+
+== 0.13.0 2011/02/03
+
+* documentation, testing, cleanup
+
+== 0.12.0 2011/02/01
+
+* added support for multiple vms
+* simplified test methods
+
+== 0.11.0 2011/01/28
+
+* reworked test methods again
+
+== 0.10.0 2011/01/27
+
+* updated test methods, adding script/vbox test methods
+* reworked helper generation to inline ERB during generation
+* misc bug fixes
+
+== 0.9.5 2011/01/13
+
+* removed double-print of preview
+* added more configs to filter preview results
+* moved helper template docs into the method
+* added selection of a range of content from packages
+* added FORCE to scaffold Rakefile to force helper/package creation
+
+== 0.9.4 2011/01/13
+
+* improved preview functionality
+* made script_test run scripts in export dir
+* made recipe add attrs to template locals
+
+== 0.9.3 2011/01/13
+
+* bug fixes in commands
+* added configuration of default vmname
+
+== 0.9.2 2011/01/13
+
+* added package reset to script tests
+* added preview command
+
+== 0.9.1 2011/01/12
+
+* added script_test test method
+* added auto-incrementing variables to recipe
+
+== 0.9.0 2011/01/12
+
+* added 'share' command to setup a vbox share
+* converted 'scripts' to 'packages' in scaffold/rake tasks
+* removed helpers from manifest, now require directly
+* corrected documentation
+
+== 0.8.3 2011/01/11
+
+* bug fixes
+
+== 0.8.1 2011/01/11
+
+* added sorted hashes to env
+* bugfix - env did not display cookbook config
+
+== 0.8.0 2011/01/11
+
+* now detect helpers using lazydoc (::helper)
+* bugfix env command didn't work
+* bugfix for using nested capture path
+* now nest resources in manifest by type
+
+== 0.7.0 2011/01/10
+
+* removed shell helpers (to linebook)
+* reworked internals for saner testing
+
 == 0.6.2 2011/01/06
 
 * added helper command to build a single helper

data/HowTo/Control Virtual Machines

@@ -0,0 +1,106 @@
+= Control Virtual Machines
+
+Linecook provides these commands to control VirtualBox VMs:
+
+  # ex: 'linecook start'
+  start              start a vm
+  stop               stop a vm
+  state              print the vm state
+  save               take a vm snapshop
+  ssh                ssh to a vm
+
+Most of these commands wrap the VirtualBox command line tool VBoxManage in
+very simple ways. With these commands it's straightforward to make an
+iterative script development workflow to build a server component, save the
+server state, build another component, and then at any time reset and rebuild
+the whole thing from scratch.
+
+Linecook uses a standard ssh config file to run scripts; the same config file
+is also used to configure the VM control commands. More details are presented
+in the help to {run scripts}[link:files/HowTo/Run%20Scripts.html], but suffice
+these configs will access the 'abox' VM built by the {VM Setup
+procedure}[link:files/HowTo/Setup%20Virtual%20Machines.html]:
+
+  [config/ssh]
+  Host abox
+  HostName localhost
+  User linecook
+  Port 2220
+
+Now you can control the VM like this:
+
+  # start the VM, resetting to a snapshot
+  linecook start --snapshot base
+  
+  # ssh to the VM
+  linecook ssh
+  
+  # take a snapshot
+  linecook snapshot modified
+  
+  # reset a snapshot, removing all children
+  linecook snapshot --reset base
+  
+  # stop the VM and go home
+  linecook stop
+
+== Multiple VMs
+
+To control multiple VMs, build and configure ssh access to each. For example
+make a 'bbox' VM as before but configure ssh access on port 2221:
+
+  - name: bbox
+  - hostname: bbox-ubuntu
+  VBoxManage modifyvm bbox --natpf1 'bbox-ssh,tcp,,2221,,22'
+
+Now add an entry for bbox in the ssh config file (note the * Host can be used
+to declare default settings):
+
+  [config/ssh]
+  Host abox
+  Port 2220
+  
+  Host bbox
+  Port 2221
+  
+  Host *
+  HostName localhost
+  User linecook
+
+Now same linecook commands will control both VMs:
+
+  # start all (or a subset) the VMs to a known snapshot
+  linecook start --snapshot base
+  linecook start --snapshot base abox
+  
+  # ssh to the VMs one at a time
+  linecook ssh abox
+  linecook ssh bbox
+  
+  # move the current snapshot forward for all (or a subset) the VMs
+  linecook snapshot modified
+  linecook snapshot modified abox
+  
+  # reset a snapshot, removing all children for all (or a subset) the VMs
+  linecook snapshot --reset base
+  linecook snapshot --reset base abox
+  
+  # stop all (or a subset) the VMs
+  linecook stop
+  linecook stop abox
+
+== Host and VM Names
+
+The Host config corresponds to the package name that will be run on the VM; ie
+the 'abox.yml' package goes to the 'abox' Host. Implicitly the VM will also be
+named 'abox' in VirtualBox (such that if you opened the VirtualBox application
+you'd see 'abox'). If you name it something else, then declare the actual VM
+name next to the Host config using a linecook-specific comment like this:
+
+  [config/ssh]
+  ...
+  Host abox    # [the_vm_name]
+  ...
+
+The Host config, 'abox' in this case, is the input for all linecook commands;
+linecook resolves the VM name internally when interacting with VirtualBox.

data/HowTo/Generate Scripts

@@ -0,0 +1,263 @@
+= Generate Scripts
+
+Linecook generates scripts and puts them into a directory. Once you have the
+scripts you can compress them, share them, run them, or do whatever you please
+- at that point they're ordinary files sitting in a directory. Linecook refers
+to the directories containing generated scripts as packages.
+
+The best way to understand how Linecook generates scripts is to start by
+making a package with a known script, and then rewrite the script using
+Linecook. At each step of the rewrite you can rebuild the package and verify
+the script is reproduced.
+
+The command to rebuild a package is:
+
+  linecook build
+
+This tutorial is designed such that if you make the files as specified, you
+can use it to rebuild the same package (functionally or literally) at the end
+of each section.
+
+== Packages
+
+Packages are located in the packages directory by default. The simplest
+package is a directory containing a single executable script. Assuming a bare
+Ubuntu VM and a bash shell, start with this script to setup a minimal
+development environment:
+
+  [packages/demo/run]
+  sudo apt-get -y install git
+  git config --global user.name "John Doe"
+  git config --global user.email "john.doe@example.com"
+  sudo apt-get -y install ruby
+
+This package represents the end result of 'linecook build'. To actually build
+it, a package file need to be made. The package file describes what goes into
+the package and is written in YAML; by default the package file is named like
+the final package, but with a .yml extension.
+
+As an example, this package file generates the 'run' script from the 'demo' recipe and puts it into the package during build.
+
+  [packages/demo.yml]
+  linecook:
+    package:
+      recipes:
+        run: demo     # package/path: recipe_name
+
+However this package file is overly verbose; the recipe with the same name as
+the package is used to generate the run script by default, so this equivalent:
+
+  [packages/demo.yml]
+  {}
+
+At this point a build (ie 'linecook build') raises an error; To see the
+package file in action we need the demo recipe.
+
+== Recipes
+
+Recipes are ruby code that generates text during a build. Typically the text
+is shell script, and therefore the result of a recipe is a script file, but
+the text could be a config file, SQL, or whatever.
+
+Specifically recipes are code that gets executed in the context of a
+Linecook::Recipe instance. These instances have a 'target' IO object (usually
+a Tempfile) to which the generated text is written. The most basic recipe
+simply writes the script content to the target.
+
+  [recipes/demo.rb]
+  target.puts <<-SCRIPT
+  sudo apt-get -y install ruby1.8
+  sudo apt-get -y install git
+  git config --global user.name "John Doe"
+  git config --global user.email "john.doe@example.com"
+  SCRIPT
+
+Now a build will write the script to the target, then move the corresponding
+Tempfile to become 'packages/demo/run'. As code, recipes work like this:
+
+  class Recipe
+    attr_accessor :target
+    def initialize
+      @target = ""
+    end
+  end
+  
+  recipe = Recipe.new
+  recipe.instance_eval File.read('recipes/demo.rb')
+  recipe.target[0, 31]  # => "sudo apt-get -y install ruby1.8"
+
+This exercise illustrates the great and powerful truth of recipes - they are
+simply a context to generate text and write it to a file. A direct consequence
+of this design is that commands (for example debugging command) can always be
+inserted into a script in a trivial manner.
+
+Obviously recipes can do more.
+
+== Attributes
+
+Attributes allow variables to be separated from the recipe code, such that
+they may be overridden on a per-package basis, in the package file. Recipes
+have a nested 'attrs' hash (which is literally a Hash) that merges together
+the defaults and overrides to provide attributes access.
+
+Using attributes:
+
+  [attributes/git.rb]
+  attrs['git']['package'] = 'git'
+  attrs['git']['config']['user.name'] = 'John Doe'
+  attrs['git']['config']['user.email'] = 'john.doe@example.com'
+  
+  [attributes/ruby.rb]
+  attrs['ruby']['package'] = 'ruby1.9.1'
+  
+  [packages/demo.yml]
+  ruby:
+    package: ruby1.8
+  
+  [recipes/demo.rb]
+  attributes 'ruby'
+  attributes 'git'
+  #########################################################################
+  target.puts <<-SCRIPT
+  sudo apt-get -y install #{attrs['ruby']['package']}
+  sudo apt-get -y install #{attrs['git']['package']}
+  git config --global user.name "#{attrs['git']['config']['user.name']}"
+  git config --global user.email "#{attrs['git']['config']['user.email']}"
+  SCRIPT
+
+Attribute files must be included using the Recipe#attributes method; no
+attributes are included by default. This ensures that attributes will be
+deterministic in cases where two attribute files (unwisely) use the same
+namespace.
+
+The overrides specified in the package file are always available, however.
+They function as a kind of environment for recipes; since they are overrides,
+they take precedence over any values set by attribute files.
+
+Take note of two details. First, there is no special code needed to set nested
+attributes in an attributes file. Attributes files are executed in the context
+of a Linecook::Attributes instance which provides this auto-nesting behavior.
+Second, as a convention, attrs are accessed using string keys because it's
+cleaner to use string keys in the package file.
+
+== Helpers
+
+Helpers allow you to define methods that generate text. Helper methods are
+defined as ERB files under the 'helpers' directory and compile into an
+ordinary module under the 'lib' directory. For example (assuming the
+attribute and package files from above):
+
+  [helpers/demo/install.erb]
+  Installs a package using apt-get.
+  (package)
+  --
+  sudo apt-get -y install <%= package %>
+  
+  
+  [helpers/demo/set_git_config.erb]
+  Sets a global git config.
+  (key, value)
+  --
+  git config --global <%= key %> <%= value %>
+  
+  
+  [recipes/demo.rb]
+  attributes 'ruby'
+  attributes 'git'
+  helpers 'demo'
+  #########################################################################
+  install attrs['ruby']['package']
+  install attrs['git']['package']
+  attrs['git']['config'].each_pair do |key, value|
+    set_git_config key, value
+  end
+
+When you define a helper, you're literally defining a method in a module that
+you can use in your recipe to generate text. This example generates the 'Demo'
+module which adds 'install' and 'set_git_config' into the recipe context. In
+fact the 'helpers' method is equivalent to:
+
+  require 'demo'
+  extend Demo
+
+The exact details are elegant but unimportant to the workflow, which can be
+summed up like this: put an ERB template into a file, include the directory
+using Recipe#helpers, and now the template is available as a method with
+inputs. Whenever you call the method, you make text that gets written to
+target.
+
+To capture the output of a template without writing it to target, prefix the
+method with an underscore. The output can then be used as an input to another
+helper... see the {README}[link:files/README.html] for an example.
+
+== Files
+
+Files are files of any sort that you might want to include in a package. They
+could be an archive of some sort, a binary, or a stock script you don't need a
+recipe to reproduce.
+
+Files are typically included via a recipe like this:
+
+  [files/gitconfig]
+  [user]
+  	name = John Doe
+  	email = john.doe@example.com
+  
+  [recipes/demo.rb]
+  target.puts <<-SCRIPT
+  sudo apt-get -y install ruby1.8
+  sudo apt-get -y install git
+  cp "#{ file_path "gitconfig" }" ~/.gitconfig
+  SCRIPT
+
+The only 'trick' here is that the return value of file_path is a path that
+will be correct at runtime, when the script is on the remote server
+(specifically it will be like "${0%/run}/gitconfig", which works because $0
+will be the full path to the recipe).
+
+== Templates
+
+Templates work the same as files except, as you may imagine, they are ERB
+templates that evaluate with whatever locals you provide them. The attrs hash
+is local by default.
+
+  [attributes/git.rb]
+  attrs['git']['package'] = 'git'
+  attrs['git']['config']['user.name'] = 'John Doe'
+  attrs['git']['config']['user.email'] = 'john.doe@example.com'
+  
+  [templates/gitconfig.erb]
+  [user]
+  	name = <%= attrs['git']['config']['user.name'] %>
+  	email = <%= attrs['git']['config']['user.email'] %>
+  
+  [recipes/demo.rb]
+  attributes 'git'
+  #########################################################################
+  target.puts <<-SCRIPT
+  sudo apt-get -y install ruby1.8
+  sudo apt-get -y install git
+  cp "#{ template_path "gitconfig" }" ~/.gitconfig
+  SCRIPT
+
+== {Linebook}[http://rubygems.org/gems/linebook/]
+
+The techniques presented here are sufficient to work with many scripts in many
+situations but they are quite bare. Eventually, or perhaps immediately, you
+will want a suite of standard helpers. The canonical helper library is
+Linebook (Linecook's version of a Cookbook).
+
+See the {Linebook documentation}[http://rubygems.org/gems/linebook/] to learn
+helpers for flow control, file system tests, commands, chaining, redirection,
+heredocs, and other convenience methods.
+
+  [recipes/demo.rb]
+  helpers 'linebook/shell'
+  
+  unless_ _file?('/tmp/message') do
+    cat.to('/tmp/message').heredoc do
+      writeln 'hello world!'
+    end
+  end
+
+Enjoy!