Empact-deprec 1.99.21

data/docs/README.svn

@@ -0,0 +1,31 @@
+== svn
+
+# Add the following to your deploy.rb (or ~/.caprc if always the same)
+	set :scm, 'your.svn.server'
+
+# The run the following from the root the project you wish to import	
+cap deprec:svn:install
+cap deprec:svn:setup
+
+
+# XXX Not tested on deprec2
+# 
+# 
+# Installs subversion and trac software on server.
+# Creates subversion repository on server and imports project.
+# Sets up trac installation for project.
+# 
+# role :scm, 'deptest.deprecated.org' 
+# 
+# # install packages
+# cap apache_install svn_install trac_install
+# 
+# # import project
+# cap svn_import_project
+# cap trac_init
+# cap trac_start
+# 
+# # your project will be viewable at http://yourdomain:8000/
+# # you can add other users with 'cap trac_user_add'
+
+

data/docs/ROADMAP.txt

@@ -0,0 +1,74 @@
+deprec 2.0 roadmap
+
+# expectations/constraints
+
+- all tasks should be re-runnable
+- all install_ tasks should install required dependancies 
+- users should expect that standard cap commands will work
+
+
+# interactive tools
+
+deprec should make it easy to perform commonly use sysadmin tasks.
+While automation brings great power it's still often quicker to 
+run 'semi automated' tasks that require some interactive input.
+
+For example, adding a user account to a number of servers could be
+made easier with a general purpose interactive recipe:
+
+cap add_user
+> username for new user? : fred
+> password for new user? : ******
+> retype password        : ******
+> user type for new user?
+1. default
+2. admin
+?: 2
+creating...
+server01 - created user 'fred' of type 'admin'
+server02 - created user 'fred' of type 'admin'
+server03 - created user 'fred' of type 'admin'
+server04 - user 'fred' exists. Set type to 'admin'
+
+* Note that the servers above were all servers mentioned in deploy.rb
+  for a project however there are many ways to get this list.
+
+
+Plugin support for different distro's was too ambitious. I've tried other 
+distros and didn't like them much. I'm happy with Ubuntu at the moment and 
+# # plugin based support different distros, versions (and possibly OS's)
+# 
+# While deprec was built to work with Ubuntu 6.06, some people have shown interest
+# in using it with other distro's. I plan to move all of the Ubuntu specific code
+# into a separate gem 'deprec_ubuntu' which deprec will load by default. This will
+# mean anyone else can create and maintain plugins that allow the use of other linux
+# distros (and perhaps other OS's).
+# 
+# All platform dependent functions will be:
+# 
+# - cap extensions, not tasks
+# - plugins, not part of the deprec itself
+# - loaded using gemplugins http://mongrel.rubyforge.org/gem_plugin_rdoc/index.html
+# 
+# I'd like 'cap -T' to show the distros supported for each command when not arch independant.
+
+
+# add lots more useful tasks
+
+I'd like to extend deprec beyond just getting your rails app up and running.
+Anything I need that takes some time to do right will go in. If you've got 
+something you want added then write it and let me know. 
+
+install and configure the following package groups: 
+- source control (svn, trac)
+- mail (postfix, imap, mutt, mailx)
+
+
+# don't rely on third party servers to be available when installing software
+
+As I write, www.apache.org is serving at a crawling pace. I'd like to make it easy for 
+users to download and cache all required third party apps, gems, etc and have deprec
+use these.
+
+Currently you can drop the tarballs into /usr/local/src on the target server and
+they'll get picked up automatically. I haven't looked at a simple way to cache the gems yet.

data/docs/deprec-1.x/deprec-1.x.quickstart

@@ -0,0 +1,50 @@
+#
+# Old notes - YMMV
+#
+
+== deprec quickstart (with svn, trac and deployment to apache/mongrel/mysql)
+
+Here are instructions that will take a fresh install of Ubuntu Dapper (6.06.1 server), create a working rails app, create an SVN repository and trac installation for it and deploy it using apache, mongrel and mysql.
+
+All commands below are run on your local host. You will *never* be requested 
+to log into the remote server manually. Capistrano does all the work.
+
+- Mike
+
+# Install deprec on workstation
+sudo gem install deprec --include-dependencies
+echo 'require "deprec/recipes"'               >> ~/.caprc
+echo "ssh_options[:keys] = %w(${HOME}/.ssh/id_dsa)" >> ~/.caprc
+echo 'ssh_options[:paranoid] = false'         >> ~/.caprc
+echo 'ssh_options[:forward_agent] = true'     >> ~/.caprc
+
+# Create rails project on workstation and configure for deprec
+# (alternatively use an existing project)
+rails example
+cd example
+./script/generate scaffold_resource person name:string age:integer
+deprec --apply-to . --name example --domain www.example.com
+
+# NOTE! Use following two commands if you only have 'root' account on server
+# Some VPS services provide you with this when you sign up
+cap change_root_password_as_root # '_as_root' means run this as 'root' user
+cap setup_admin_account_as_root 
+
+# Copy your ssh keys to remote server to avoid having to type passwords
+cap setup_ssh_keys
+
+# Install all required software on remote server
+cap install_rails_stack svn_install trac_install
+
+# Import application into subversion respository and setup trac
+cap svn_setup
+cap trac_setup
+cap trac_user_add # this command allows you to create other trac users
+cap trac_start # trac is now available on http://www.example.com:9000/
+cap trac_stop  # if you had the need
+
+# Deploy application
+cap setup
+cap deploy_with_migrations
+cap apache_restart 
+# application is now running on http://www.example.com/people

data/docs/deprec-1.x/notes.txt

@@ -0,0 +1,12 @@
+# update timezone on gutsy
+# dpkg-reconfigure tzdata
+
+
+# Building Edge Capistrano
+You can build an edge gem by cd'ing to that directory and doing:
+
+1. svn info. Find the line that says "Revision:" and mark the revision number.
+2. rake PKG_BUILD=<rev> gem. Replace <rev> with the revision number.
+3. Look in the pkg subdirectory. Install that gem file via "gem install pkg/<gem file>"
+
+- Jamis

data/lib/deprec.rb

@@ -0,0 +1,8 @@
+unless Capistrano::Configuration.respond_to?(:instance)
+  abort "deprec2 requires Capistrano 2"
+end
+
+require "#{File.dirname(__FILE__)}/deprec/capistrano_extensions"
+require "#{File.dirname(__FILE__)}/vmbuilder_plugins/all"
+require "#{File.dirname(__FILE__)}/deprec/recipes"
+

data/lib/deprec/capistrano_extensions.rb

@@ -0,0 +1,378 @@
+# Copyright 2006-2008 by Mike Bailey. All rights reserved.
+require 'capistrano'
+require 'fileutils'
+
+module Deprec2
+  DEPREC_TEMPLATES_BASE = File.join(File.dirname(__FILE__), 'templates')
+
+  # Render template (usually a config file) 
+  # 
+  # Usually we render it to a file on the local filesystem.
+  # This way, we keep a copy of the config file under source control.
+  # We can make manual changes if required and push to new hosts.
+  #
+  # If the options hash contains :path then it's written to that path.
+  # If it contains :remote => true, the file will instead be written to remote targets
+  # If options[:path] and options[:remote] are missing, it just returns the rendered
+  # template as a string (good for debugging).
+  #
+  #  XXX I would like to get rid of :render_template_to_file
+  #  XXX Perhaps pass an option to this function to write to remote
+  #
+  def render_template(app, options={})
+    template = options[:template]
+    path = options[:path] || nil
+    remote = options[:remote] || false
+    mode = options[:mode] || 0755
+    owner = options[:owner] || nil
+  
+    # replace this with a check for the file
+    if ! template
+      puts "render_template() requires a value for the template!"
+      return false 
+    end
+  
+    # If local copies of deprec templates exist they will be used 
+    # If you don't specify the location with the local_template_dir option
+    # it defaults to config/templates.
+    # e.g. config/templates/nginx/nginx.conf.erb
+    local_template = File.join(local_template_dir, app.to_s, template)
+    if File.exists?(local_template)
+      puts
+      puts "Using local template (#{local_template})"
+      template = ERB.new(IO.read(local_template), nil, '-')
+    else
+      template = ERB.new(IO.read(File.join(DEPREC_TEMPLATES_BASE, app.to_s, template)), nil, '-')
+    end
+    rendered_template = template.result(binding)
+  
+    if remote 
+      # render to remote machine
+      puts 'You need to specify a path to render the template to!' unless path
+      exit unless path
+      sudo "test -d #{File.dirname(path)} || sudo mkdir -p #{File.dirname(path)}"
+      std.su_put rendered_template, path, '/tmp/', :mode => mode
+      sudo "chown #{owner} #{path}" if defined?(owner)
+    elsif path 
+      # render to local file
+      full_path = File.join('config', app.to_s, path)
+      path_dir = File.dirname(full_path)
+      if File.exists?(full_path)
+        if IO.read(full_path) == rendered_template
+          puts "[skip] File exists and is identical (#{full_path})."
+          return false
+        elsif overwrite?(full_path, rendered_template)
+          File.delete(full_path)
+        else
+          puts "[skip] Not overwriting #{full_path}"
+          return false
+        end
+      end
+      FileUtils.mkdir_p "#{path_dir}" if ! File.directory?(path_dir)
+      # added line above to make windows compatible
+      # system "mkdir -p #{path_dir}" if ! File.directory?(path_dir) 
+      File.open(full_path, 'w'){|f| f.write rendered_template }
+      puts "[done] #{full_path} written"
+    else
+      # render to string
+      return rendered_template
+    end
+  end
+  
+  def overwrite?(full_path, rendered_template)
+    if defined?(overwrite_all)      
+      if overwrite_all == true
+        return true
+      else
+        return false
+      end
+    end
+    
+    # XXX add :always and :never later - not sure how to set persistent value from here
+    # response = Capistrano::CLI.ui.ask "File exists. Overwrite? ([y]es, [n]o, [a]lways, n[e]ver)" do |q|
+    puts
+    response = Capistrano::CLI.ui.ask "File exists (#{full_path}). 
+    Overwrite? ([y]es, [n]o, [d]iff)" do |q|
+      q.default = 'n'
+    end
+    
+    case response
+    when 'y'
+      return true
+    when 'n'
+      return false
+    when 'd'
+      require 'tempfile'
+      tf = Tempfile.new("deprec_diff") 
+      tf.puts(rendered_template)
+      tf.close
+      puts
+      puts "Running diff -u current_file new_file_if_you_overwrite"
+      puts
+      system "diff -u #{full_path} #{tf.path} | less"
+      puts
+      overwrite?(full_path, rendered_template)
+    # XXX add :always and :never later - not sure how to set persistent value from here  
+    # when 'a'
+    #   set :overwrite_all, true
+    # when 'e'
+    #   set :overwrite_all, false
+    end
+    
+  end
+
+  def render_template_to_file(template_name, destination_file_name, templates_dir = DEPREC_TEMPLATES_BASE)
+    template_name += '.conf' if File.extname(template_name) == '' # XXX this to be removed
+
+    file = File.join(templates_dir, template_name)
+    buffer = render :template => File.read(file)
+
+    temporary_location = "/tmp/#{template_name}"
+    put buffer, temporary_location
+    sudo "cp #{temporary_location} #{destination_file_name}"
+    delete temporary_location
+  end
+  
+  # Copy configs to server(s). Note there is no :pull task. No changes should 
+  # be made to configs on the servers so why would you need to pull them back?
+  def push_configs(app, files)   
+    app = app.to_s
+    files.each do |file|
+      # If the file path is relative we will prepend a path to this projects
+      # own config directory for this service.
+      if file[:path][0,1] != '/'
+        full_remote_path = File.join(deploy_to, app, file[:path]) 
+      else
+        full_remote_path = file[:path]
+      end
+      full_local_path = File.join('config', app, file[:path])
+      sudo "test -d #{File.dirname(full_remote_path)} || sudo mkdir -p #{File.dirname(full_remote_path)}"
+      #
+      # XXX work this in to check for per-host variants of config files
+      #
+      # if any variants of this file exist for this host (they have -hostname at end)
+      #   servers = find_servers_for_task(current_task)
+      #   servers.each do |server|
+      #     puts server(..., :hosts => server) # XXX find a way to restrict su_put to one host
+      #   end
+      # else
+      #   # just send them the normal way, it's quicker in parallel
+        std.su_put File.read(full_local_path), full_remote_path, '/tmp/', :mode=>file[:mode]
+      # end
+      #
+      sudo "chown #{file[:owner]} #{full_remote_path}"
+    end
+  end
+  
+  def teardown_connections
+    sessions.keys.each do |server|
+         sessions[server].close
+         sessions.delete(server)
+    end
+  end
+
+
+  def append_to_file_if_missing(filename, value, options={})
+    # XXX sort out single quotes in 'value' - they'l break command!
+    # XXX if options[:requires_sudo] and :use_sudo then use sudo
+    sudo <<-END
+    sh -c '
+    grep -F "#{value}" #{filename} > /dev/null 2>&1 || 
+    test ! -f #{filename} ||
+    echo "#{value}" >> #{filename}
+    '
+    END
+  end
+
+  # create new user account on target system
+  def useradd(user, options={})
+    options[:shell] ||= '/bin/bash' # new accounts on ubuntu 6.06.1 have been getting /bin/sh
+    switches = ''
+    switches += " --shell=#{options[:shell]} " if options[:shell]
+    switches += ' --create-home ' unless options[:homedir] == false
+    switches += " --gid #{options[:group]} " unless options[:group].nil?
+    invoke_command "grep '^#{user}:' /etc/passwd || sudo /usr/sbin/useradd #{switches} #{user}", 
+    :via => run_method
+  end
+
+  # create a new group on target system
+  def groupadd(group, options={})
+    via = options.delete(:via) || run_method
+    # XXX I don't like specifying the path to groupadd - need to sort out paths before long
+    invoke_command "grep '#{group}:' /etc/group || sudo /usr/sbin/groupadd #{group}", :via => via
+  end
+
+  # add group to the list of groups this user belongs to
+  def add_user_to_group(user, group)
+    invoke_command "groups #{user} | grep ' #{group} ' || sudo /usr/sbin/usermod -G #{group} -a #{user}",
+    :via => run_method
+  end
+
+  # create directory if it doesn't already exist
+  # set permissions and ownership
+  # XXX move mode, path and
+  def mkdir(path, options={})
+    via = options.delete(:via) || :run
+    # XXX need to make sudo commands wrap the whole command (sh -c ?)
+    # XXX removed the extra 'sudo' from after the '||' - need something else
+    invoke_command "sh -c 'test -d #{path} || mkdir -p #{path}'", :via => via
+    invoke_command "chmod #{sprintf("%3o",options[:mode]||0755)} #{path}", :via => via if options[:mode]
+    invoke_command "chown -R #{options[:owner]} #{path}", :via => via if options[:owner]
+    groupadd(options[:group], :via => via) if options[:group]
+    invoke_command "chgrp -R #{options[:group]} #{path}", :via => via if options[:group]
+  end
+  
+  def create_src_dir
+    mkdir(src_dir, :mode => 0775, :group => group_src, :via => :sudo)
+  end
+  
+  # download source package if we don't already have it
+  def download_src(src_package, src_dir)
+    create_src_dir
+    # check if file exists and if we have an MD5 hash or bytecount to compare 
+    # against if so, compare and decide if we need to download again
+    if defined?(src_package[:md5sum])
+      md5_clause = " && echo '#{src_package[:md5sum]}' | md5sum -c - "
+    end
+    apt.install( {:base => %w(wget)}, :stable )
+    # XXX replace with invoke_command
+    sudo <<-SUDO
+    sh -c "cd #{src_dir} && test -f #{src_package[:filename]} #{md5_clause} || wget --quiet --timestamping #{src_package[:url]}"
+    SUDO
+  end
+
+  # unpack src and make it writable by the group
+  def unpack_src(src_package, src_dir)
+    package_dir = File.join(src_dir, src_package[:dir])
+    # XXX replace with invoke_command
+    sudo <<-SUDO
+    sh -c '
+    cd #{src_dir};
+    test -d #{package_dir}.old && rm -fr #{package_dir}.old;
+    test -d #{package_dir} && mv #{package_dir} #{package_dir}.old;
+    #{src_package[:unpack]}
+    chgrp -R #{group} #{package_dir};  
+    chmod -R g+w #{package_dir};
+    '
+    SUDO
+  end
+
+  # install package from source
+  def install_from_src(src_package, src_dir)
+    package_dir = File.join(src_dir, src_package[:dir])
+    unpack_src(src_package, src_dir)
+    apt.install( {:base => %w(build-essential)}, :stable )
+    # XXX replace with invoke_command
+    sudo <<-SUDO
+    sh -c '
+    cd #{package_dir};
+    #{src_package[:configure]}
+    #{src_package[:make]}
+    #{src_package[:install]}
+    #{src_package[:post_install]}
+    '
+    SUDO
+  end
+
+
+  ##
+  # Run a command and ask for input when input_query is seen.
+  # Sends the response back to the server.
+  #
+  # +input_query+ is a regular expression that defaults to /^Password/.
+  #
+  # Can be used where +run+ would otherwise be used.
+  #
+  #  run_with_input 'ssh-keygen ...', /^Are you sure you want to overwrite\?/
+
+  def run_with_input(shell_command, input_query=/^Password/, response=nil)
+    handle_command_with_input(:run, shell_command, input_query, response)
+  end
+
+  ##
+  # Run a command using sudo and ask for input when a regular expression is seen.
+  # Sends the response back to the server.
+  #
+  # See also +run_with_input+
+  #
+  # +input_query+ is a regular expression
+
+  def sudo_with_input(shell_command, input_query=/^Password/, response=nil)
+    handle_command_with_input(:sudo, shell_command, input_query, response)
+  end
+
+  def invoke_with_input(shell_command, input_query=/^Password/, response=nil)
+    handle_command_with_input(run_method, shell_command, input_query, response)
+  end
+
+  ##
+  # Run a command using sudo and continuously pipe the results back to the console.
+  #
+  # Similar to the built-in +stream+, but for privileged users.
+
+  def sudo_stream(command)
+    sudo(command) do |ch, stream, out|
+      puts out if stream == :out
+      if stream == :err
+        puts "[err : #{ch[:host]}] #{out}"
+        break
+      end
+    end
+  end
+
+  # We don't need this. Put 'USER=root' on the command line instead.
+  #
+  # XXX Not working in deprec2
+  # ##
+  # # Run a command using the root account.
+  # #
+  # # Some linux distros/VPS providers only give you a root login when you install.
+  # 
+  # def run_as_root(shell_command)
+  #   std.connect_as_root do |tempuser|
+  #     run shell_command
+  #   end
+  # end
+  # 
+  # ##
+  # # Run a task using root account.
+  # #
+  # # Some linux distros/VPS providers only give you a root login when you install.
+  # #
+  # # tempuser: contains the value replaced by 'root' for the duration of this call
+  # 
+  # def as_root()
+  #   std.connect_as_root do |tempuser|
+  #     yield tempuser
+  #   end
+  # end
+  
+
+
+  private
+
+  ##
+  # Does the actual capturing of the input and streaming of the output.
+  #
+  # local_run_method: run or sudo
+  # shell_command: The command to run
+  # input_query: A regular expression matching a request for input: /^Please enter your password/
+
+  def handle_command_with_input(local_run_method, shell_command, input_query, response=nil)
+    send(local_run_method, shell_command) do |channel, stream, data|
+      logger.info data, channel[:host]
+      if data =~ input_query
+        if response
+          channel.send_data "#{response}\n"
+        else 
+          response = ::Capistrano::CLI.password_prompt "#{data}"
+          channel.send_data "#{response}\n"
+        end
+      end
+    end
+  end
+  
+  
+end
+
+Capistrano.plugin :deprec2, Deprec2