sunzi 0.4.4 → 0.5.0

data/README.md

@@ -16,6 +16,10 @@ Its design goals are:
 * **Always use the root user.** Think twice before blindly assuming you need a regular user - it doesn't add any security benefit for server provisioning, it just adds extra verbosity for nothing. However, it doesn't mean that you shouldn't create regular users with Sunzi - feel free to write your own recipes.
 * **Minimum dependencies.** No configuration server required. You don't even need a Ruby runtime on the remote server.
 
+### What's new:
+
+* v0.5: Role-based configuration supported. Reworked directory structure. (Incompatible with previous versions)
+
 Quickstart
 ----------
 
@@ -31,7 +35,7 @@ Go into your project directory (if it's a Rails project, `config` would be a goo
 $ sunzi create
 ```
 
-It generates a `sunzi` folder along with subdirectories and templates. Inside `sunzi`, there's `sunzi.yml`, which defines your own dynamic attributes to be used from scripts. Also there's the `remote` folder, which will be transferred to the remote server, that contains recipes and dynamic variables compiled from `sunzi.yml`.
+It generates a `sunzi` folder along with subdirectories and templates. Inside `sunzi`, there are `sunzi.yml` and `install.sh`. Those two are the most important files that you mainly work on.
 
 Go into the `sunzi` directory, then run `sunzi deploy`:
 
@@ -42,25 +46,25 @@ $ sunzi deploy example.com
 
 Now, what it actually does is:
 
-1. Compile `sunzi.yml` to generate attributes and retrieve remote recipes
+1. Compile `sunzi.yml` to generate attributes and retrieve remote recipes, then copy files into the `compiled` directory
 1. SSH to `example.com` and login as `root`
-1. Transfer the content of the `remote` directory to the remote server and extract in `$HOME/sunzi`
+1. Transfer the content of the `compiled` directory to the remote server and extract in `$HOME/sunzi`
 1. Run `install.sh` on the remote server
 
 As you can see, all you need to do is edit `install.sh` and add some shell commands. That's it.
 
-A Sunzi project with no recipes is totally fine, so that you can start small, go big as you get along.
+A Sunzi project without any recipes or roles is totally fine, so that you can start small, go big as you get along.
 
 Commands
 --------
 
 ```bash
-$ sunzi                               # Show command help
-$ sunzi compile                       # Compile Sunzi project
-$ sunzi create                        # Create a new Sunzi project
-$ sunzi deploy [user@host:port]       # Deploy Sunzi project
-$ sunzi setup [linode|ec2]            # Setup a new VM on the Cloud services
-$ sunzi teardown [linode|ec2] [name]  # Teardown an existing VM on the Cloud services
+$ sunzi                                 # Show command help
+$ sunzi compile                         # Compile Sunzi project
+$ sunzi create                          # Create a new Sunzi project
+$ sunzi deploy [user@host:port] [role]  # Deploy Sunzi project
+$ sunzi setup [linode|ec2]              # Setup a new VM on the Cloud services
+$ sunzi teardown [linode|ec2] [name]    # Teardown an existing VM on the Cloud services
 ```
 
 Directory structure
@@ -68,21 +72,25 @@ Directory structure
 
 Here's the directory structure that `sunzi create` automatically generates:
 
-```
+```bash
 sunzi/
-  sunzi.yml         ---- add custom attributes and remote recipes here
-  remote/           ---- everything under this folder will be transferred to the remote server
-    attributes/     ---- compiled attributes from sunzi.yml at deploy (do not edit directly)
-      ssh_key
-    recipes/        ---- put commonly used scripts here, referred from install.sh
-      ssh_key.sh
-    install.sh      ---- main scripts that gets run on the remote server
+  install.sh      # main script
+  sunzi.yml       # add custom attributes and remote recipes here
+
+  recipes/        # put commonly used scripts here, referred from install.sh
+    ssh_key.sh
+  roles/          # when role is specified, scripts here will be concatenated
+    app.sh        # to install.sh in the compile phase
+    db.sh
+    web.sh
+  compiled/       # everything under this folder will be transferred to the
+                  # remote server (do not edit directly)
 ```
 
 How do you pass dynamic values to a recipe?
 -------------------------------------------
 
-In the compile phase, attributes defined in `sunzi.yml` are split into multiple files, one per attribute. We use filesystem as a sort of key-value storage so that it's easy to use from shell scripts.
+In the compile phase, attributes defined in `sunzi.yml` are split into multiple files in `compiled/attributes`, one per attribute. We use filesystem as a sort of key-value storage so that it's easy to use from shell scripts.
 
 The convention for argument passing to a recipe is to use `$1`, `$2`, etc. and put a comment line for each argument.
 
@@ -119,7 +127,7 @@ Goodbye Chef, Hello Sunzi!
 Remote Recipes
 --------------
 
-Recipes can be retrieved remotely via HTTP. Put a URL in the recipes section of `sunzi.yml`, and Sunzi will automatically load the content and put it into the `remote/recipes` folder in the compile phase.
+Recipes can be retrieved remotely via HTTP. Put a URL in the recipes section of `sunzi.yml`, and Sunzi will automatically load the content and put it into the `compiled/recipes` folder in the compile phase.
 
 For instance, if you have the following line in `sunzi.yml`,
 
@@ -132,6 +140,23 @@ recipes:
 
 You may find sample recipes in this repository useful: https://github.com/kenn/sunzi-recipes
 
+Role-based configuration
+------------------------
+
+You probably have different configurations between **web servers** and **database servers**.
+
+No problem - how Sunzi handles role-based configuration is refreshingly simple.
+
+Shell scripts under the `roles` directory, such as `web.sh` or `db.sh`, are automatically recognized as a role. The role script will be appended to `install.sh` at deploy, so you should put common configurations in `install.sh` and role specific procedures in the role script.
+
+For instance, when you set up a new web server, deploy with a role name:
+
+```bash
+sunzi deploy example.com web
+```
+
+It is equivalent to running `install.sh`, followed by `web.sh`.
+
 Cloud Support
 -------------
 

data/lib/sunzi/cli.rb

@@ -9,14 +9,14 @@ module Sunzi
       do_create(project)
     end
 
-    desc "deploy example.com (or user@example.com:2222)", "Deploy sunzi project"
-    def deploy(target)
-      do_deploy(target)
+    desc "deploy [user@host:port] [role]", "Deploy sunzi project"
+    def deploy(target, role = nil)
+      do_deploy(target, role)
     end
 
     desc "compile", "Compile sunzi project"
-    def compile
-      do_compile
+    def compile(role = nil)
+      do_compile(role)
     end
 
     desc "setup [linode|ec2]", "Setup a new VM"
@@ -38,66 +38,84 @@ module Sunzi
 
       def do_create(project)
         empty_directory project
-        empty_directory "#{project}/remote"
-        empty_directory "#{project}/remote/recipes"
-        template "templates/create/sunzi.yml",                 "#{project}/sunzi.yml"
-        template "templates/create/remote/install.sh",         "#{project}/remote/install.sh"
-        template "templates/create/remote/recipes/ssh_key.sh", "#{project}/remote/recipes/ssh_key.sh"
+        empty_directory "#{project}/recipes"
+        empty_directory "#{project}/roles"
+        # template "templates/create/.gitignore",         "#{project}/.gitignore"
+        template "templates/create/sunzi.yml",          "#{project}/sunzi.yml"
+        template "templates/create/install.sh",         "#{project}/install.sh"
+        template "templates/create/recipes/ssh_key.sh", "#{project}/recipes/ssh_key.sh"
+        template "templates/create/roles/app.sh",       "#{project}/roles/app.sh"
+        template "templates/create/roles/db.sh",        "#{project}/roles/db.sh"
+        template "templates/create/roles/web.sh",       "#{project}/roles/web.sh"
       end
 
-      def do_deploy(target)
+      def do_deploy(target, role)
         user, host, port = parse_target(target)
         endpoint = "#{user}@#{host}"
 
         # compile attributes and recipes
-        compile
+        compile(role)
 
         # The host key might change when we instantiate a new VM, so
         # we remove (-R) the old host key from known_hosts.
         `ssh-keygen -R #{host} 2> /dev/null`
 
-        commands = <<-EOS
-        cd remote
-        tar cz . | ssh -o 'StrictHostKeyChecking no' #{endpoint} -p #{port} '
+        remote_commands = <<-EOS
         rm -rf ~/sunzi &&
         mkdir ~/sunzi &&
         cd ~/sunzi &&
         tar xz &&
-        bash install.sh'
+        bash install.sh
         EOS
 
-        Open3.popen3(commands) do |stdin, stdout, stderr|
+        local_commands = <<-EOS
+        cd compiled
+        tar cz . | ssh -o 'StrictHostKeyChecking no' #{endpoint} -p #{port} '#{remote_commands}'
+        EOS
+
+        Open3.popen3(local_commands) do |stdin, stdout, stderr|
           stdin.close
-          t = Thread.new(stderr) do |terr|
-            while (line = terr.gets)
-              print shell.set_color(line, :red, true)
+          t = Thread.new do
+            while (line = stderr.gets)
+              print line.color(:red).bright
             end
           end
           while (line = stdout.gets)
-            print print shell.set_color(line, :green, true)
+            print line.color(:green).bright
           end
           t.join
         end
       end
 
-      def do_compile
+      def do_compile(role)
         # Check if you're in the sunzi directory
-        unless File.exists?('sunzi.yml')
-          abort_with "You must be in the sunzi folder"
-        end
+        abort_with "You must be in the sunzi folder" unless File.exists?('sunzi.yml')
+        # Check if role exists
+        abort_with "#{role} doesn't exist!" if role and !File.exists?("roles/#{role}.sh")
 
         # Load sunzi.yml
         hash = YAML.load(File.read('sunzi.yml'))
-        empty_directory 'remote/attributes'
-        empty_directory 'remote/recipes'
-
-        # Compile attributes.yml
-        hash['attributes'].each do |key, value|
-          File.open("remote/attributes/#{key}", 'w'){|file| file.write(value) }
-        end
-        # Compile recipes.yml
-        hash['recipes'].each do |key, value|
-          get value, "remote/recipes/#{key}.sh"
+        empty_directory 'compiled'
+        empty_directory 'compiled/attributes'
+        empty_directory 'compiled/recipes'
+        empty_directory 'compiled/files'
+
+        # Break down attributes into individual files
+        hash['attributes'].each {|key, value| create_file "compiled/attributes/#{key}", value }
+
+        # Retrieve remote recipes via HTTP
+        hash['recipes'].each {|key, value| get value, "compiled/recipes/#{key}.sh" }
+
+        # Copy local files
+        Dir['recipes/*'].each {|file| copy_file File.expand_path(file), "compiled/recipes/#{File.basename(file)}" }
+        Dir['roles/*'].each   {|file| copy_file File.expand_path(file), "compiled/roles/#{File.basename(file)}" }
+        hash['files'].each    {|file| copy_file File.expand_path(file), "compiled/files/#{File.basename(file)}" }
+
+        # Build install.sh
+        if role
+          create_file 'compiled/install.sh', File.binread("install.sh") << "\n" << File.binread("roles/#{role}.sh")
+        else
+          copy_file File.expand_path('install.sh'), 'compiled/install.sh'
         end
       end
 

data/lib/sunzi/version.rb

@@ -1,3 +1,3 @@
 module Sunzi
-  VERSION = "0.4.4"
+  VERSION = "0.5.0"
 end

data/lib/templates/create/.gitignore

@@ -0,0 +1 @@
+/compiled

data/lib/templates/create/{remote/install.sh → install.sh}

@@ -6,9 +6,9 @@ export DEBIAN_FRONTEND=noninteractive
 # SSH key
 source recipes/ssh_key.sh $(cat attributes/ssh_key)
 
+# Add Dotdeb repository
+source recipes/dotdeb.sh
+
 # Update apt catalog
 aptitude update
 aptitude -y safe-upgrade
-
-# Install RVM - rvm.sh will be retrieved from Github in the compile phase
-source recipes/rvm.sh

data/lib/templates/create/recipes/ssh_key.sh

@@ -0,0 +1,17 @@
+# SSH key
+# $1: SSH key filename
+
+if [ -f ~/.ssh/authorized_keys ]; then
+  echo 'authorized_keys already created'
+else
+  if [ -f "files/$1" ]; then
+    mkdir -p ~/.ssh
+    cat "files/$1" > ~/.ssh/authorized_keys
+    rm "files/$1"
+    chmod 700 ~/.ssh
+    chmod 600 ~/.ssh/authorized_keys
+  else
+    echo "The public key file \"$1\" is not found! Look into files section in sunzi.yml."
+    exit 1
+  fi
+fi

data/lib/templates/create/roles/app.sh

@@ -0,0 +1,30 @@
+# Install Application Server
+
+# Required attributes: env, ruby_version
+export ENVIRONMENT=$(cat attributes/environment)
+export RUBY_VERSION=$(cat attributes/ruby_version)
+
+# Install RVM
+source recipes/rvm.sh
+
+# Set RAILS_ENV
+if grep -Fq "RAILS_ENV" ~/.bash_profile; then
+  echo 'RAILS_ENV entry already exists'
+else
+  echo "export RAILS_ENV=$ENVIRONMENT" >> ~/.bash_profile
+  source ~/.bash_profile
+fi
+
+# Install Ruby
+if [[ "$(which ruby)" == /usr/local/rvm/rubies/ruby-$RUBY_VERSION* ]]; then
+  echo 'ruby-$RUBY_VERSION already installed'
+else
+  aptitude -y install curl build-essential libssl-dev libreadline6-dev
+  rvm install $RUBY_VERSION
+  rvm $RUBY_VERSION --default
+  echo 'gem: --no-ri --no-rdoc' > ~/.gemrc
+
+  # Install RubyGems
+  gem update --system
+  gem install bundler
+fi

data/lib/templates/create/roles/db.sh

@@ -0,0 +1,5 @@
+# Install Database Server
+
+source recipes/mongodb-10gen.sh       # MongoDB
+# source recipes/mysql-server.sh 5.5  # MySQL 5.5
+# source recipes/redis-server.sh      # Redis

data/lib/templates/create/roles/web.sh

@@ -0,0 +1,4 @@
+# Install Web server
+
+source recipes/nginx.sh       # Nginx
+# source recipes/apache2.sh   # Apache

data/lib/templates/create/sunzi.yml

@@ -1,7 +1,20 @@
 ---
+# Dynamic variables here will be compiled to individual files in compiled/attributes.
 attributes:
-  # Dynamic variables here will be compiled to individual files in remote/attributes.
   ssh_key: id_rsa.pub
+  environment: development
+  ruby_version: 1.9.3
+
+# Remote recipes here will be downloaded to compiled/recipes.
 recipes:
-  # Remote recipes here will be loaded to individual files in remote/recipes.
-  rvm: https://raw.github.com/kenn/sunzi-recipes/master/ruby/rvm.sh
+  rvm: https://raw.github.com/kenn/sunzi-recipes/master/ruby/rvm.sh
+  dotdeb: https://raw.github.com/kenn/sunzi-recipes/master/debian/dotdeb.sh
+  # nginx: https://raw.github.com/kenn/sunzi-recipes/master/debian/nginx.sh
+  # apache2: https://raw.github.com/kenn/sunzi-recipes/master/debian/apache2.sh
+  # mongodb-10gen: https://raw.github.com/kenn/sunzi-recipes/master/debian/mongodb-10gen.sh
+  # mysql-server: https://raw.github.com/kenn/sunzi-recipes/master/debian/mysql-server.sh
+  # redis-server: https://raw.github.com/kenn/sunzi-recipes/master/debian/redis-server.sh
+
+# Files specified here will be copied to compiled/.
+files:
+  - ~/.ssh/id_rsa.pub

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sunzi
 version: !ruby/object:Gem::Version
-  version: 0.4.4
+  version: 0.5.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-03-02 00:00:00.000000000 Z
+date: 2012-03-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
-  requirement: &2160582840 !ruby/object:Gem::Requirement
+  requirement: &2164772780 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *2160582840
+  version_requirements: *2164772780
 - !ruby/object:Gem::Dependency
   name: rainbow
-  requirement: &2160582300 !ruby/object:Gem::Requirement
+  requirement: &2164772220 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,10 +32,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *2160582300
+  version_requirements: *2164772220
 - !ruby/object:Gem::Dependency
   name: rake
-  requirement: &2160581480 !ruby/object:Gem::Requirement
+  requirement: &2164771480 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -43,7 +43,7 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *2160581480
+  version_requirements: *2164771480
 description: Server provisioning utility for minimalists
 email:
 - kenn.ejima@gmail.com
@@ -66,8 +66,12 @@ files:
 - lib/sunzi/logger.rb
 - lib/sunzi/utility.rb
 - lib/sunzi/version.rb
-- lib/templates/create/remote/install.sh
-- lib/templates/create/remote/recipes/ssh_key.sh
+- lib/templates/create/.gitignore
+- lib/templates/create/install.sh
+- lib/templates/create/recipes/ssh_key.sh
+- lib/templates/create/roles/app.sh
+- lib/templates/create/roles/db.sh
+- lib/templates/create/roles/web.sh
 - lib/templates/create/sunzi.yml
 - lib/templates/setup/linode/linode.yml
 - sunzi.gemspec

data/lib/templates/create/remote/recipes/ssh_key.sh

@@ -1,19 +0,0 @@
-# SSH key
-# $1: SSH key filename
-
-if [ -f ~/.ssh/authorized_keys ]; then
-  echo 'authorized_keys already created'
-else
-  if [ -f "$1" ]; then
-    mkdir -p ~/.ssh
-    cat $1 > ~/.ssh/authorized_keys
-    rm $1
-    chmod 700 ~/.ssh
-    chmod 600 ~/.ssh/authorized_keys
-  else
-    echo "The public key file is not found! Try the following command:"
-    echo "cp ~/.ssh/$1 remote"
-    echo "If the file name found in ~/.ssh is different from \"$1\", edit sunzi.yml as appropriate."
-    exit 1
-  fi
-fi