necktie 0.2.1 → 0.3.0

data/README.rdoc

@@ -0,0 +1,174 @@
+I use Necktie to setup and upgrade multiple servers. 
+
+
+== Dress to impress
+
+Necktie runs a set of scripts (or configuration tasks) that can create and edit
+files, install and setup services, mount volumes, and basically anything you
+can script with Ruby and a shell.
+
+To use Necktie:
+- Write your Necktie tasks using Ruby
+- Include support files (config, binaries, etc)
+- git push
+- cap necktie
+
+I use it to:
+
+- gem install, apt-get
+- Configure Nginx, install and setup Unicorn
+- Change memcached configuration from default
+- Setup MySQL to use a mounted (EBS) volume
+- Install and update cron scripts
+
+
+== Tasking
+
+Necktie is based on Rake, so if you know Rake you already know most of Necktie.
+At the base of your Necktie project there's a file called necktie (or Necktie,
+or necktie.rb), with a list of task. For example:
+
+  file "/var/myapp" do
+    append "/etc/fstab", "\n/mnt/myapp /var/myapp none bind" unless read("/etc/fstab")["/mnt/myapp"]
+    mkdir_p "/var/myapp/"
+    sh "mount /var/myapp"
+  end
+
+  task :rubygems do
+    FileList["gems/*.gem"].each do |gem|
+      install_gem gem
+    end
+  end
+
+  task :nginx do
+    rm_rf "/etc/nginx/sites-enabled/*"
+    cp "etc/nginx/unicorn.conf", "/etc/nginx/sites-available/"
+    ln_sf "/etc/nginx/sites-available/unicorn.conf", "/etc/nginx/sites-enabled/"
+    services.start "nginx"
+  end
+
+  task :unicorn=>[:rubygems] do
+    cp "etc/init.d/unicorn", "/etc/init.d/"
+    chmod 0750, "/etc/init.d/unicorn"
+  end
+
+  task :app=>["/var/myapp", :nginx, :unicorn]
+
+You then run the app task, which runs all the dependencies necessary to setup
+your application directory, system gems, Nginx and Unicorn configuration. Ready
+for deployment.
+
+If you changed one of the configuration files, you can run Necktie again: it
+will pull updates from the Git repository and run these tasks with the new
+configuration files.
+
+Note: The nginx and unicorn files will run every time you run the app task, so
+you can use them to push out new configuration.  In contrast, the /var/myapp
+directory is mounted by a file task that will only run once (if the directory
+doesn't exist).
+
+
+== cap necktie
+
+I use Capistrano to setup new instances and upgrade existing ones.  I then run
+deploy:cold on the new instances, or deploy to upgrade running instances.
+
+The Capistrano tasks looks something like this (see also:
+extra/capistrano/necktie.rb):
+
+  # Copy Gem over so not depending on gem server being online, and all servers get
+  # to use the same version of Necktie.
+  gem_spec = Gem::SourceIndex.from_installed_gems.find_name("necktie").last
+  gem_file = File.join(Gem.dir, "cache", spec.file_name)
+  upload gem_file, File.basename(gem_file), :via=>:scp
+  sudo "gem install #{File.basename(gem_file)}"
+  # Run Necktie as sudo.
+  sudo "necktie #{necktie_url} #{ENV["ROLES"].to_s.gsub(',', ' ')}"
+
+To setup a new server:
+
+  cap necktie HOSTS=ec2-75-101-239-12.compute-1.amazonaws.com
+
+To upgrade instances:
+
+  git push && cap necktie
+
+
+== Feel the rush
+
+Necktie includes Rush, so you can write tasks like this:
+
+  unless processes.find { |p| p.cmdline[/memcached\s.*-l\s0.0.0.0/] }
+    box["/etc/memcached.conf"].replace_contents! /^-l 127.0.0.1/, "-l 0.0.0.0"
+    Services.start "memcached"
+  end
+
+You can learn more about Rush here: http://rush.heroku.com
+
+Of course, there's also FileUtils, system and sh, so you can just:
+
+  cp "etc/init.d/unicorn", "/etc/init.d"
+  chmod 0755, "/etc/init.d/unicorn"
+  sh "service start unicorn"
+
+Note: The current directory (launch_dir and Dir.pwd) is the root directory
+of your Necktie repository. You can use relative paths to access files from your
+Necktie repository.
+
+
+== Role play
+
+If you have different setups, split them into roles and give each role its own
+head task. You can then run Necktie for one or several roles, in the order they
+are listed on the command line.
+
+For example:
+
+  task :app=>["rubygems", "nginx", "unicorn", "/var/myapp"]
+  task :db=>["mount", "master", "backup"]
+
+
+I recommend using the same roles for Necktie and Capistrano, then you can:
+
+  cap necktie ROLES=app,db
+
+Note: Command line arguments are either task names (run in order), or
+name=value pairs that set environment variables (e.g.  RAILS_ENV=production).
+
+
+== gem install
+
+You can use install_gem in one of two ways. You can pass it a gem name and
+(optional, but highly recommended) version requirement. For example:
+
+  install_gem "unicorn", "~= 0.93"
+
+You can store the gem file in your Necktie repository and install it from there:
+
+  install_gem "gems/unicorn-0.93.3.gem"
+
+I prefer the later, so I'm not affected when gem sources go offline just as I
+decide to run my Necktie tasks.
+
+Since install_gem will only install the same gem/version once, a run-always
+rubygems.rb task is all you need:
+
+  launch_dir["gems/*.gem"].each do |gem|
+    install_gem gem.to_s
+  end
+
+
+== License
+
+Necktie, copyright (C) 2009 Assaf Arkin, released under the "Use for good, not
+evil" license (http://www.json.org/license.html)
+
+Includes Rake, created by Jim Weirich and released under the MIT license
+http://rake.rubyforge.org/ http://github.com/jimweirich/rake
+
+Includes Rush, created by Adam Wiggins and released under the MIT License
+http://rush.heroku.com http://github.com/adamwiggins/rush
+
+Includes Session, created by Ara T. Howard and released under the Ruby License
+http://raa.ruby-lang.org/project/session
+http://www.codeforpeople.com/lib/ruby/session

data/bin/necktie

@@ -1,5 +1,6 @@
 #!/usr/bin/env ruby
 $LOAD_PATH.unshift File.expand_path("../lib", File.dirname(__FILE__)),
-  File.expand_path("../rush/lib", File.dirname(__FILE__)), File.expand_path("../session/lib", File.dirname(__FILE__))
+  File.expand_path("../vendor/rake/lib", File.dirname(__FILE__)), File.expand_path("../vendor/rush/lib", File.dirname(__FILE__)),
+  File.expand_path("../vendor/session/lib", File.dirname(__FILE__))
 require "necktie"
 necktie ARGV

data/lib/necktie/files.rb

@@ -0,0 +1,29 @@
+def read(name)
+  File.read(name)
+end
+
+def write(name, contents = nil)
+  contents ||= yield
+  File.open name, "w" do |f|
+    f.write contents
+  end
+end
+
+def append(name, contents = nil)
+  contents ||= yield
+  File.open name, "a" do |f|
+    f.write contents
+  end
+end
+
+def update(name, from = nil, to = nil)
+  contents = File.read(name)
+  if from && to
+    contents = contents.sub(from, to)
+  else
+    contents = yield(contents)
+  end
+  File.open name, "w" do |f|
+    f.write contents
+  end
+end

data/lib/necktie/rake.rb

@@ -0,0 +1,127 @@
+require "rake"
+
+module Necktie
+  class Application < Rake::Application
+
+    def initialize
+      super
+      @name = "necktie"
+      @rakefiles = ["Necktie", "necktie", "Necktie.rb", "necktie.rb"]
+      options.nosearch = true
+    end
+
+    def run
+      standard_exception_handling do
+        handle_options
+        Dir.chdir clone_repo do
+          collect_tasks
+          load_rakefile
+          top_level
+        end
+      end
+    end
+
+    def clone_repo
+      @git_url = ARGV.shift or fail "expecting first argument to be a Git repository URL"
+      repo = File.expand_path(".necktie")
+      if File.exist?(repo)
+        puts "Pulling latest updates to #{repo}"
+        system "cd #{repo.inspect} && git pull origin #{ENV["BRANCH"] || "master"}" or fail
+      else
+        puts "Cloning #{@git_url} to #{repo}"
+        system "git clone #{@git_url} #{repo.inspect}" or fail
+      end
+      repo
+    end
+
+    def necktie_options
+      [
+        ['--describe', '-D [PATTERN]', "Describe the tasks (matching optional PATTERN), then exit.",
+          lambda { |value|
+            options.show_tasks = :describe
+            options.show_task_pattern = Regexp.new(value || '')
+            TaskManager.record_task_metadata = true
+          }
+        ],
+        ['--execute-print',  '-p CODE', "Execute some Ruby code, print the result, then exit.",
+          lambda { |value|
+            puts eval(value)
+            exit
+          }
+        ],
+        ['--execute-continue',  '-E CODE',
+          "Execute some Ruby code, then continue with normal task processing.",
+          lambda { |value| eval(value) }            
+        ],
+        ['--prereqs', '-P', "Display the tasks and dependencies, then exit.",
+          lambda { |value| options.show_prereqs = true }
+        ],
+        ['--tasks', '-T [PATTERN]', "Display the tasks (matching optional PATTERN) with descriptions, then exit.",
+          lambda { |value|
+            options.show_tasks = :tasks
+            options.show_task_pattern = Regexp.new(value || '')
+            Rake::TaskManager.record_task_metadata = true
+          }
+        ],
+        ['--trace', '-t', "Turn on invoke/execute tracing, enable full backtrace.",
+          lambda { |value|
+            options.trace = true
+            verbose(true)
+          }
+        ],
+        ['--verbose', '-v', "Log message to standard output.",
+          lambda { |value| verbose(true) }
+        ],
+        ['--version', '-V', "Display the program version.",
+          lambda { |value|
+            spec = Gem::Specification.load(File.expand_path("../necktie.gemspec", File.dirname(__FILE__)))
+            puts "Necktie, version #{spec.version}"
+            exit
+          }
+        ],
+        ['--where', '-W [PATTERN]', "Describe the tasks (matching optional PATTERN), then exit.",
+          lambda { |value|
+            options.show_tasks = :lines
+            options.show_task_pattern = Regexp.new(value || '')
+            Rake::TaskManager.record_task_metadata = true
+          }
+        ],
+      ]
+    end
+
+    # Read and handle the command line options.
+    def handle_options
+      options.rakelib = ['necktie']
+      options.top_level_dsl = true
+
+      OptionParser.new do |opts|
+        opts.banner = "necktie git_url {options} tasks..."
+        opts.separator ""
+        opts.separator "Options are ..."
+
+        opts.on_tail("-h", "--help", "-H", "Display this help message.") do
+          puts opts
+          exit
+        end
+
+        necktie_options.each { |args| opts.on(*args) }
+        opts.environment('RAKEOPT')
+      end.parse!
+
+      Rake::DSL.include_in_top_scope
+    end
+
+    def raw_load_rakefile # :nodoc:
+      @rakefile = have_rakefile
+      fail "No Necktie file found (looking for: #{@rakefiles.join(', ')})" if @rakefile.nil?
+      Rake::Environment.load_rakefile(File.expand_path(@rakefile)) if @rakefile && @rakefile != ''
+      options.rakelib.each do |rlib|
+        glob("necktie/*.rb") do |name|
+          add_import name
+        end
+      end
+      load_imports
+    end
+
+  end
+end

data/lib/necktie/services.rb

@@ -1,33 +1,37 @@
-module Services
-  def self.start(name)
+class Services
+  def start(name)
     puts " ** Starting service #{name}"
     system "update-rc.d #{name} defaults" and
       system "service #{name} start" or
       fail "failed to start #{name}"
   end
 
-  def self.enable(name)
+  def enable(name)
     system "update-rc.d #{name} defaults" or "cannot enable #{name}"
   end
 
-  def self.stop(name)
+  def stop(name)
     puts " ** Stopping service #{name}"
     system "service #{name} stop" and
     system "update-rc.d -f #{name} remove" or
       fail "failed to stop #{name}"
   end
 
-  def self.disable(name)
+  def disable(name)
     system "update-rc.d -f #{name} remove" or fail "cannot disable #{name}"
   end
 
-  def self.restart(name)
+  def restart(name)
     puts " ** Stopping service #{name}"
     system "service #{name} restart" or fail "failed to restart #{name}"
   end
 
-  def self.status(name)
+  def running?(name)
     status = File.read("|service --status-all 2>&1")[/^ \[ (.) \]  #{Regexp.escape name}$/,1]
     status == "+" ? true : status == "-" ? false : nil
   end
 end
+
+def services
+  @services ||= Services.new
+end

data/lib/necktie.rb

@@ -1,59 +1,9 @@
-require "fileutils"
-extend FileUtils
+require "necktie/rake"
+require "necktie/files"
 require "necktie/gems"
 require "necktie/services"
 require "necktie/rush"
 
 
-def necktie(args)
-  etc = "/etc/necktie"
-  File.writable?(etc) or File.writable?(File.dirname(etc)) or fail "Can't write to #{etc}. You should be running necktie as sudo."
-
-  envs, args = args.partition { |a| a[/=/] }
-  git_url = args.shift or fail "Usage: necktie git-url role* name=value*"
-  roles = args
-  # Setup environment variable from name=value arguments.
-  envs.each do |env|
-    name, value = env.split("=")
-    ENV[name] = value
-  end
-
-  # Default role is app. For all rules, execute setup scripts.
-  if roles.empty?
-    puts "    Default role: app"
-    roles = ["app"]
-  end
-
-  repo = File.expand_path(".necktie")
-  if File.exist?(repo)
-    puts "  * Pulling latest updates to #{repo}"
-    system "cd #{repo.inspect} && git pull origin #{ENV["branch"] || "master"}" or fail
-  else
-    puts "  * Cloning #{git_url} to #{repo}"
-    system "git clone #{git_url} #{repo.inspect}" or fail
-  end
-
-  Dir.chdir repo do
-    executed = File.exist?(etc) ? File.read(etc).split("\n") : []
-    roles.each do |role|
-      skip = executed.map { |t| t[/^#{Regexp.escape role}\/(.*)$/, 1] }.reject(&:nil?)
-      tasks = Dir["tasks/#{role}/*.rb"].sort.map { |name| File.basename(name).gsub(/\.rb$/, "") }
-      todo = tasks - skip
-      if tasks.empty?
-        puts "  * No tasks for role #{role}"
-      elsif todo.empty?
-        puts "  * All tasks completed for role #{role}"
-      else
-        puts "  * Now in role: #{role}"
-        File.open etc, "a" do |f|
-          todo.each do |task|
-            puts " ** Executing #{task}"
-            load "tasks/#{role}/#{task}.rb"
-            f.puts task if task[/^\d+_/]
-          end
-        end
-      end
-    end
-  end
-
-end
+Rake.application = Necktie::Application.new
+Rake.application.run

data/necktie.gemspec

@@ -1,12 +1,12 @@
 Gem::Specification.new do |spec|
   spec.name           = "necktie"
-  spec.version        = "0.2.1"
+  spec.version        = "0.3.0"
   spec.author         = "Assaf Arkin"
   spec.email          = "assaf@labnotes.org"
   spec.homepage       = ""
   spec.summary        = "Dress to impress"
-  spec.description    = ""
+  spec.description    = "Configure your servers remotely using Ruby and Git"
 
-  spec.files          = Dir["{bin,lib,rush,session}/**/*", "*.{gemspec,rdoc}"]
+  spec.files          = Dir["{bin,lib,vendor}/**/*", "*.{gemspec,rdoc}"]
   spec.executable     = "necktie"
 end