linecook 0.6.2 → 1.0.0

data/HowTo/Run Scripts

@@ -0,0 +1,87 @@
+= Run Scripts
+
+Linecook runs scripts using the 'run' command. Internally run copies scripts
+to a server using scp and then executes the script using ssh. More
+specifically run copies a *package* to the server and executes a script within
+the package using ssh. A package is simply a directory containing scripts and
+any files the scripts use.
+
+To manually recreate what run does, make a directory with an script, copy it,
+and execute as below (assuming you set up the 'abox' server {as described
+earlier}[link:files/Tutorial/1%20-%20VM%20Setup.html]). Note that the script
+is made executable locally such that scp will make the script executable on
+the server.
+
+  mkdir demo
+  cat > demo/script.sh <<"DOC"
+  echo "# $(whoami)@$(hostname): hello world!"
+  DOC
+  chmod +x demo/script.sh
+  
+  scp -r -P 2220 demo linecook@localhost:/tmp/demo
+  ssh -p 2220 linecook@localhost -- "/tmp/demo/script.sh"
+  # linecook@abox-ubuntu: hello world!
+
+To simplify this a little, move the redundant information used by scp and ssh
+can into a standard ssh config file. Linecook looks for an ssh config file in
+'config/ssh' by default:
+
+  mkdir config
+  cat > config/ssh <<DOC
+  Host abox
+  User linecook
+  HostName localhost
+  Port 2220
+  DOC
+
+Now run will copy the package under the 'packages' directory with the same
+name as the Host config, and execute the specified script:
+
+  mkdir packages
+  mv demo packages/abox
+  linecook run --script script.sh abox
+  # linecook@abox-ubuntu: hello world!
+
+To run multiple packages at once, simply add more hosts to the ssh config
+file. Run copies each package to the appropriate host and then runs the same
+script on each (in alphabetic order by host). So say you also set up a 'bbox'
+same as 'abox':
+
+  cat >> config/ssh <<DOC
+  Host abox
+  Port 2220
+  
+  Host bbox
+  Port 2221
+  
+  Host *
+  User linecook
+  HostName localhost
+  DOC
+  cp packages/abox packages/bbox
+
+Then:
+
+  linecook run --script script.sh abox bbox
+  # linecook@abox-ubuntu: hello world!
+  # linecook@bbox-ubuntu: hello world!
+
+Note that by default run will execute the 'run' script on all hosts configured
+in 'config/ssh'. Leveraging these defaults simplifies the last command:
+
+  mv packages/abox/script.sh packages/abox/run
+  mv packages/bbox/script.sh packages/bbox/run
+  linecook run
+  # linecook@abox-ubuntu: hello world!
+  # linecook@bbox-ubuntu: hello world!
+
+If you've never used scp/ssh with a config file then this may seem unfamiliar
+but hopefully quite graceful. For scp/ssh masters, these are equivalent for
+each host:
+
+  linecook run --remote-dir=/tmp/linecook SCRIPT ARGS...
+  
+  scp -q -r -p -F config/ssh packages/host "host:/tmp/linecook"
+  ssh -q -t -t -F config/ssh host -- "/tmp/linecook/SCRIPT ARGS..."
+
+Now that you can run scripts, onward to generating scripts!

data/HowTo/Setup Virtual Machines

@@ -0,0 +1,76 @@
+= Setup Virtual Machines
+
+{VirtualBox}[http://www.virtualbox.org] runs virtual machines on your local
+box. Local VMs help minimize development cycles because they are quick to
+access and reset. In addition, VirtualBox allows snapshots which can save and
+restore a particular server state - this functionality enables iterative
+development.
+
+To set up a Ubuntu VM using VirtualBox:
+
+1. Download and Install VirtualBox (http://www.virtualbox.org)
+2. Download a Ubuntu ISO (http://www.ubuntu.com/server/get-ubuntu/download)
+3. Build the Box
+
+When done, see the documentation for {controlling
+VMs}[link:files/HowTo/Control%20Virtual%20Machines.html]
+
+== Building a Ubuntu Box
+
+Use the VirtualBox wizard to get started. The name and other settings (ex
+user, ssh port) may be changed but be sure to propagate changes throughout the
+setup process.
+
+  - name: abox
+  - Linux/Ubuntu
+  - 512 MB memory
+  - 8 GB dynamically resizing drive
+
+Add the Ubuntu iso to the cd/dvd device under Settings > Storage. Now start
+the server and install Ubuntu (use default settings unless specified):
+
+  - hostname: abox-ubuntu
+  - user/password: linecook
+  - select 'OpenSSH server' in packages to install
+
+When the server has rebooted and is at the login screen, remove the install
+iso, take a snapshot, and setup port forwarding. Port forwarding allows you to
+access the ssh port (22) on the VM via a port on your local box (2220).
+
+  (Devices > CD/DVD Devices > Remove disk from virtual drive)
+  VBoxManage snapshot abox take RAW
+  VBoxManage controlvm abox poweroff
+  # wait to fully power off
+  VBoxManage modifyvm abox --natpf1 'abox-ssh,tcp,,2220,,22'
+  VBoxManage -q snapshot abox restore RAW
+  VBoxManage startvm abox
+
+Transfer your ssh key to the VM (notice port 2220). Help to generate ssh keys
+can be found on {GitHub}[http://help.github.com/key-setup-redirect]:
+
+  scp -P 2220 -o UserKnownHostsFile=/dev/null ~/.ssh/id_rsa.pub linecook@localhost:id_rsa.pub
+  
+Login as linecook and setup SSH access:
+
+  vm: mkdir .ssh
+  vm: mv id_rsa.pub .ssh/authorized_keys
+  vm: chmod 0700 .ssh
+  vm: chmod 0600 .ssh/authorized_keys
+
+Remove the login banner (as a convenience) and exit:
+
+  vm: sudo rm /etc/motd
+  vm: exit
+
+Now take a base snapshot:
+
+  VBoxManage snapshot abox take BASE
+  VBoxManage controlvm abox poweroff
+
+To cleanup port forwarding (run later, if ever):
+
+  VBoxManage modifyvm abox --natpf1 delete 'abox-ssh'
+
+The same procedure can be repeated to build other VMs. Nothing is special
+about this setup. The os, name, port, and other settings may be changed to
+your liking - all that Linecook requires is ssh access.

data/License.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2010 Pinnacol Assurance
+Copyright (c) 2010-2011 Pinnacol Assurance
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README

@@ -4,95 +4,114 @@ A shell script generator.
 
 == Description
 
-Linecook lets you generate shell scripts using an extensible set of tiny ERB
-helpers. The helpers output a shell script that you can use to (for example)
-provision a server.
+Linecook generates shell scripts using an extensible set of ERB helpers. The
+shell scripts and associated resources (files, subscripts, etc) make up
+packages that can be used, for example, to provision servers.
 
-The workflow is like this:
+Linecook provides a command line tool to manage development servers, generate
+scripts, and run scripts on servers. Helpers and recipes can be shared as
+gems; the canonical shell helpers are available through
+{Linebook}[http://rubygems.org/gems/linebook/].
 
-* define helpers
-* write a recipe using the helpers
-* generate a script from the recipe
+See the documentation for help to:
 
-Linecook provides very basic helpers for POSIX shells and a command-line tool
-for generating scripts and running them on a test vm.
+* {Setup}[link:files/HowTo/Setup%20Virtual%20Machines.html] and {Control}[link:files/HowTo/Control%20Virtual%20Machines.html] Virtual Machines
+* {Generate Scripts}[link:files/HowTo/Generate%20Scripts.html]
+* {Run Scripts}[link:files/HowTo/Run%20Scripts.html]
 
 == Usage
 
-Generate a cookbook.
+Generate a project.
 
-  % linecook init example
-  % cd example
+  % linecook init chalkboard
+  % cd chalkboard
 
-Make some helpers.
+Define default attributes.
 
-  [helpers/example/_head.rb]
-  COLOR_CODES = Hash[*%W{
-    black       0;30   red         0;31
-    white       1;37   green       0;32
-    light_gray  0;37   blue        0;34
-  }]
-  
-  [helpers/example/color.erb]
-  Adds color to a string.
+  [attributes/chalkboard.rb]
+  attrs['chalkboard']['n'] = 3
+  attrs['chalkboard']['color'] = 'white'
+  attrs['chalkboard']['message'] = 'I will not manually configure my server'
+
+Define a helper.
+
+  [helpers/chalkboard/echo_in_color.erb]
+  Echo a string in color.
   (color, str)
+    color_codes = Hash[*%W{
+      black       0;30   red         0;31
+      white       1;37   green       0;32
+      light_gray  0;37   blue        0;34
+    }]
   --
-  \033[<%= COLOR_CODES[color.to_s] %>m<%= str %>\033[0m
+  echo -e '\033[<%= color_codes[color.to_s] %>m<%= str %>\033[0m'
+
+Use both in a recipe.
+
+  [recipes/chalkboard.rb]
+  attributes "chalkboard"
+  helpers "chalkboard"
   
-  [helpers/example/echo.erb]
-  Echo a string in color
-  (str, options={})
-  color = options[:color]
-  --
-  echo -e '<%= color ? _color(color, str) : str %>'
+  attrs['chalkboard']['n'].times do
+    echo_in_color attrs['chalkboard']['color'], attrs['chalkboard']['message']
+  end
 
-Define default attributes.
+Define a package to use the recipe; set non-default attributes as needed.
 
-  [recipes/example.rb]
-  attrs['example']['n'] = 3
-  attrs['example']['color'] = 'blue'
+  [packages/chalkboard.yml]
+  chalkboard:
+    n: 5
+    color: blue
 
-Use them in a recipe.
+Generate the package from the attributes, helpers, and recipe.
 
-  [recipes/example.rb]
-  helpers "example"
-  attributes "example"
-  
-  attrs['example']['n'].times do
-    echo "I will not manually configure my server", :color => attrs['example']['color']
-  end
+  % linecook build
 
-Define a script using the recipe.
+Check the packages directory to see the resulting script.
 
-  [scripts/example.yml]
-  linecook:
-    recipe_name: example
-    script_name: example.sh
-  example:
-    n: 100
+  % cat packages/chalkboard/run
+  echo -e '\033[0;34mI will not manually configure my server\033[0m'
+  echo -e '\033[0;34mI will not manually configure my server\033[0m'
+  echo -e '\033[0;34mI will not manually configure my server\033[0m'
+  echo -e '\033[0;34mI will not manually configure my server\033[0m'
+  echo -e '\033[0;34mI will not manually configure my server\033[0m'
 
-Generate a script from the helpers, attributes, and recipe.
+Now run the package.
 
-  % rake scripts
+  % linecook run
 
-Start the VM and run the scripts (see the docs for setting up a VM).
+The package can be run on any server accessible via ssh. See the generated
+'config/ssh' file as well as the documentation for {Running Scripts on
+Servers}[link:files/HowTo/Run%20Scripts.html].
 
-  % rake vbox:start
-  % rake vbox:ssh 
-  vm: bash /vbox/scripts/example.sh
-  vm: exit
+== Composition
 
-Stop the VM.
+Linecook consists of the following.
+
+  = Components =
+  attributes
+  helpers
+  |-----> recipes -> packages -a-> (scripts) -b-> (servers) <-c- server setup
+  files
+  templates
+  
+  = Commands =
+  ex: 'linecook build' on the command line
+  
+  a) env, helper, package, build        # build scripts
+  b) run                                # run scripts
+  c) start, stop, ssh, snapshot, reset  # server setup/control
 
-  % rake vbox:stop
+Everything to the left of 'scripts' has to do with generating scripts;
+everything to the right with running scripts on servers.
 
 == Installation
 
-Linecook is available as a gem on {Gemcutter}[http://gemcutter.org/gems/linecook]
+Linecook is available as a {gem}[http://rubygems.org/gems/linecook].
 
   % gem install linecook
 
 == Info 
 
-Developer:: {Simon Chiang}[http://bahuvrihi.wordpress.com]
+Developer:: {Simon Chiang}[http://github.com/thinkerbot]
 License:: {MIT-Style}[link:files/License_txt.html]

data/bin/linecook

@@ -1,5 +1,11 @@
 #!/usr/bin/env ruby
 
+if File.exists?('Gemfile')
+  require 'rubygems'
+  require 'bundler'
+  Bundler.setup
+end
+
 require 'linecook/version'
 require 'linecook/commands'
 registry = Linecook::Commands::Command.registry
@@ -24,7 +30,7 @@ if cmd_class.nil?
     
     exit
   when '-v', '--version'
-    puts "version #{Linecook::VERSION} -- #{Linecook::WEBSITE}"
+    puts "linecook version #{Linecook::VERSION} -- #{Linecook::WEBSITE}"
     exit
   else
     puts "unknown command: #{cmd.inspect}"
@@ -37,10 +43,10 @@ parser.add cmd_class.configurations
 parser.on '-h', '--help', 'print this help' do
   puts "usage: linecook [options] #{cmd} #{cmd_class.args}"
   
-  desc = cmd_class.desc.wrap
+  desc = cmd_class.desc.wrap(76, 2, "\n  ")
   unless desc.empty?
     puts
-    puts cmd_class.desc.wrap
+    puts "  #{desc}"
     puts
   end
   
@@ -53,6 +59,7 @@ parser.parse! ARGV
 begin
   cmd_class.new(parser.config).call ARGV
 rescue Linecook::Commands::CommandError
-  puts $!.message
-  exit 1
+  puts $!.message unless $!.message.strip.empty?
+  puts $!.backtrace if $DEBUG
+  exit $!.exitstatus
 end

data/bin/linecook_run

@@ -0,0 +1,45 @@
+#! /bin/sh
+############################################################################
+
+ssh_config_file=${SSH_CONFIG_FILE:-config/ssh}
+remote_dir=${REMOTE_DIR:-$(pwd)/linecook}
+remote_script=${REMOTE_SCRIPT:-run}
+
+usage="usage: %s [-F SSH_CONFIG_FILE] [-D REMOTE_DIR] [-S REMOTE_SCRIPT] [-h] PACKAGE_DIRS...\n"
+option="       %s   %s\n"
+while getopts "F:D:S:h" opt
+do
+  case $opt in
+  F  )  ssh_config_file=$OPTARG ;;
+  D  )  remote_dir=$OPTARG ;;
+  S  )  remote_script=$OPTARG ;;
+  h  )  printf "$usage" $0
+        printf "$option" "-F" "the ssh config file"
+        printf "$option" "-D" "the remote package dir"
+        printf "$option" "-S" "the remote script"
+        printf "$option" "-h" "prints this help"
+        exit 0 ;;
+  \? )  printf "$usage" $0
+        exit 2 ;;
+  esac
+done
+shift $(($OPTIND - 1))
+
+################################### run ####################################
+
+for package_dir in "$@"
+do
+host=$(basename -- "$package_dir")
+
+ssh -q -t -t -F "$ssh_config_file" "$host" -- "$remote_dir/$remote_script" 2>/dev/null </dev/null
+
+status=$?
+if [ $status -ne 0 ]
+then
+  echo "[$status] $remote_dir/$remote_script" >&2
+  exit 1
+fi
+
+done
+
+################################## (run) ###################################

data/bin/linecook_scp

@@ -0,0 +1,50 @@
+#! /bin/sh
+############################################################################
+
+ssh_config_file=${SSH_CONFIG_FILE:-config/ssh}
+remote_dir=${REMOTE_DIR:-\$(pwd)/linecook}
+
+usage="usage: %s [-D REMOTE_DIR] [-F SSH_CONFIG_FILE] [-h] PACKAGE_DIRS...\n"
+option="       %s   %s\n"
+while getopts "F:D:h" opt
+do
+  case $opt in
+  F  )  ssh_config_file=$OPTARG ;;
+  D  )  remote_dir=$OPTARG ;;
+  h  )  printf "$usage" $0
+        printf "$option" "-F" "the ssh config file"
+        printf "$option" "-D" "the remote package dir"
+        printf "$option" "-h" "prints this help"
+        exit 0 ;;
+  \? )  printf "$usage" $0
+        exit 2 ;;
+  esac
+done
+shift $(($OPTIND - 1))
+
+################################### scp ####################################
+
+for package_dir in "$@"
+do
+host=$(basename -- "$package_dir")
+
+ssh -q -T -F "$ssh_config_file" "$host" -- <<SCRIPT
+rm -rf "$remote_dir"
+if [ "\$(dirname "$remote_dir")" != "" ]
+then
+  mkdir -p "\$(dirname "$remote_dir")"
+fi
+SCRIPT
+
+scp -q -r -p -F "$ssh_config_file" "$package_dir" "$host:$remote_dir"
+
+status=$?
+if [ $status -ne 0 ]
+then
+  echo "[$status] $remote_dir/$script_name " >&2
+  exit 1
+fi
+done
+
+################################## (scp) ###################################
+