salted-rails 0.0.4 → 0.0.5

data/README.md

@@ -1,10 +1,15 @@
 # Salted-Rails
 
-Salted-Rails: Provision rails using salt to vagrant or capistrano controlled systems
+Salted-Rails: Provision a vagrant machine for rails using salt.
 
-This gem inspects .ruby-version, config/database.yml, Gemfile and Gemfile.lock to generate salt pillar files to control the configuration of the system. 
+This gem inspects your rails app configuration to work out what needs to be installed in the virtual machine.
 
-THIS GEM IS IN THE EXPERIMENTAL STAGE (pre pre alpha)! EXPECT THINGS TO CHANGE AND BREAK WITHOUT NOTICE!
+THIS GEM IS IN ALPHA STAGE! THINGS MAY CHANGE AND BREAK WITHOUT NOTICE!
+
+It Inspects:
+  * .ruby-version / .rvmrc to control the version of ruby installed (using rbenv)
+  * config/database.yml to create users and databases
+  * Gemfile and Gemfile.lock to preload gems into the system, and trigger the installation of packages required by gems
 
 This configures vagrant in the way that I personally like:
 * ubunutu 12.04 (LTS) 32bit from cloud-images.ubuntu.com (up to date packages and more memory free for systems < 4GB memory)
@@ -23,18 +28,62 @@ Add as a vagrant plugin
 
     vagrant plugin add salted-rails
 
-And then adjust your Vagrantfile as follows:
+And then adjust your Vagrantfile as follows (accepting all the defaults, one machine):
+
+  Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
+
+    if defined? SaltedRails::Config
+      salted_config = SaltedRails::Config.new(File.dirname(__FILE__))
+      salted_config.configure_vagrant(config)
+    end
+
+    # .... etc ....
+  end
+
+Or for a more complicated example:
 
-    require 'salted_rails/vagrant_helper'
-    vagrant_helper = SaltedRails::VagrantHelper.new(File.dirname(__FILE__))
     Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
-      vagrant_helper.configure_vagrant(config)
-      vagrant_helper.configure_ubuntu_mirror(config, 'mirror')  # best (ping not bandwidth?) mirror
-      vagrant_helper.configure_digital_ocean(config)
-      vagrant_helper.configure_salt(config)
-      vagrant_helper.configure_ports(config)
-      # example - override default key
-      # config.ssh.private_key_path = '~/.ssh/id_rsa_Another'
+
+      if defined? SaltedRails::Config
+        salted_config = SaltedRails::Config.new(File.dirname(__FILE__))
+
+        # On the guest run the following to copy all these files/firectories to vagrant home directory
+        # cd /srv/salt/generated/home ; cp -R .[a-z]* ~vagrant
+        # note .ssh/known_hosts and .ssh/authorized_hosts are renamed with .copy_from_home appended
+        salted_config.copy_from_home = %w{ .vim .vimrc .gitconfig .ssh .tmux .tmux.conf }
+
+        # if you have multiple ssh keys, you can select which one
+        salted_config.private_key_path = '~/.ssh/id_rsa_project' if File.exist? '~/.ssh/id_rsa_project'
+
+        # override default domain
+        salted_config.domain = 'mydomain.com'
+
+        # Define a machine  dev
+        salted_config.define('dev') do |machine_config|
+        end
+        
+        salted_config.define('qa') do |machine_config|
+          # use my ISP's mirror 
+          salted_config.mirror = 'internode'
+
+          # explicitly specify memory
+          machine_config.memory = 1024
+
+          # add extra roles, which will install extra packages and increase teh defsault memory allocation
+
+          # two continuous integration packages
+          # browse http://localhost:8111
+          machine_config.roles <<= 'teamcity'
+
+          # browse http://localhost:3333
+          machine_config.roles <<= 'curisecontrolrb'
+
+          # gui also configures virtualbox for standard rather than headless mode
+          machine_config.roles <<= 'gui'
+        end
+          
+        salted_config.configure_vagrant(config)
+      end
     end
 
 You can add configuration that applies to all your projects to `~/.vagrant.d/Vagrantfile`, eg:
@@ -46,7 +95,7 @@ You can add configuration that applies to all your projects to `~/.vagrant.d/Vag
       end
     end
 
-The ubuntu_mirror value can also be:
+The mirror value can also be:
 * 'mirror' - Configures mirror: option to auto select from http://mirrors.ubuntu.com/mirrors.txt
 * 'internode' - an australian ISP (mine ;)
 * a country code - eg 'au', 'uk', 'us' etc
@@ -72,12 +121,8 @@ TODO: write the helper and then configure capistrano to use it ...
 
 TODO: Write usage instructions here
 
-Salt rules are copied into RAILS_ROOT/tmp/salt and pillar info into RAILS_ROOT/tmp/pillar by the `configure_salt` method.
-It also (re)creates `pillar/application.sls` based on `.ruby-version`, `config/database.yml` and `Gemfile` ehenever they change.
-The configuration files are also copied to `salt/railsapp/files/`.
-
-If you move `RAILS_ROOT/tmp/salt` and `RAILS_ROOT/tmp/pillar` into a RAILS_ROOT/salt directory then you can adjust the other files as desired.
-In that case only the `pillar/application.sls` and `salt/railsapp/files/*` files will be refreshed when they become stale (rather than all files).
+This gem generates pillar and salt files in RAILS_ROOT/tmp/salt and RAILS_ROOT/tmp/pillar respectively from your application configuration files.
+The files it generates are based on `.ruby-version`, `config/database.yml` and `Gemfile` as well 
 
 ## Contributing
 

data/init.sh

@@ -0,0 +1,64 @@
+#!/bin/sh
+
+msg='OK: Vagrant has been setup: ready for vagrant up [--provider=digital_ocean]'
+
+if which VirtualBox; then
+  echo Found Virtualbox
+else
+  echo ACTION REQUIRED: Please install Virtualbox 4.2.18 or later from https://www.virtualbox.org/wiki/Downloads
+  msg='(PLEASE RERUN SCRIPT AFTERWARDS)'
+fi
+
+if which vagrant; then
+  echo Found vagrant
+  echo Checking vagrant plugins are installed ...
+
+  vagrant plugin list > /tmp/t$$
+  for plugin in deep_merge vagrant-digitalocean vagrant-vbguest # salted-rails
+  do
+    if grep $plugin < /tmp/t$$; then
+      vagrant plugin update $plugin
+    else
+      vagrant plugin install $plugin
+    fi
+  done
+
+else
+  echo ACTION REQUIRED: Please install vagrant 1.3.4 or later from http://www.vagrantup.com/
+  msg='(PLEASE RERUN SCRIPT AFTERWARDS)'
+fi
+
+if [ -f Vagrantfile.example ]; then
+  if [ -f Vagrantfile ]; then
+    echo Found Vagrantfile '(previously copied from example)'
+  else
+    echo Copying Vagrantfile.example to Vagrantfile '(so you can customize it)'
+    cp Vagrantfile.example Vagrantfile
+  fi
+else
+  echo ERROR: setup_vagrant Must be run in rails root directory '(Vagrantfile.example not found)'
+  msg='(PLEASE RERUN SCRIPT FROM CORRECT DIRECTORY)'
+fi
+
+mkdir -p $HOME/.vagrant.d
+
+if [ -f "$HOME/.vagrant.d/Vagrantfile" ]; then
+  echo Found global Vagrantfile
+else
+  echo "About to set up ~/.vagrant.d/Vagrantfile"
+  echo -n "Enter your digital ocean client key (defaul none): "
+  read client_id
+  echo -n "Enter your digital ocean API key (defaul none): "
+  read api_key
+  echo "
+Vagrant.configure('2') do |config|
+  config.vm.provider :digital_ocean do |provider|
+    provider.client_id = '$client_id'
+    provider.api_key = '$api_key'
+  end
+end
+" > $HOME/.vagrant.d/Vagrantfile
+fi
+echo
+echo "$msg"
+exit 0

data/lib/salted-rails.rb

@@ -1,8 +1,6 @@
-require "salted_rails_base"
-require "salted_rails/version"
+require "salted-rails/version"
+require "salted-rails/config"
+require "salted-rails/vagrant_helper"
 
-class SaltedRails < SaltedRailsBase
-  name "Salted Rails"
+module SaltedRails
 end
-
-

data/lib/salted-rails/capistrano_helper.rb

@@ -0,0 +1,106 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+#
+
+require 'erb'
+require 'yaml'
+require 'fileutils'
+require 'salted-rails/helper_base'
+
+module SaltedRails
+  class CapistranoHelper < HelperBase
+
+    def initialize(rails_root, logger = Log4r::Logger.new("salted_rails::capistrano_helper"))
+      super(rail_root, logger)
+    end
+
+#    def configure_vagrant(config)
+#      config.vm.box = 'UbuntuCloud_12.04_32bit'
+#      config.vm.box_url = 'http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-vagrant-i386-disk1.box'
+#    end
+#
+#    def configure_ubuntu_mirror(config, mirror = 'mirror://mirrors.ubuntu.com/mirrors.txt')
+#      config.vm.provision "shell" do |s|
+#        s.path = @salt_root + 'salt/bin/change_mirror.sh'
+#        s.args = "'#{mirror}'"
+#      end
+#    end
+#
+#    def configure_digital_ocean(config, private_key_path = '~/.ssh/id_rsa', disable_vagrant_sync = true)
+#      @logger.info 'Configuring digital ocean provider'  if @logger
+#      @has_digital_ocean = 1
+#      config.vm.provider :digital_ocean do |provider, override|
+#        override.ssh.username = 'vagrant'
+#        override.vm.box = 'digital_ocean'
+#        override.vm.box_url = 'https://github.com/smdahlen/vagrant-digitalocean/raw/master/box/digital_ocean.box'
+#        provider.image = 'Ubuntu 12.04 x32'
+#        provider.region = 'San Francisco 1'
+#        provider.ca_path = '/etc/ssl/certs/ca-certificates.crt' if File.exist?('/etc/ssl/certs/ca-certificates.crt')
+#        override.vm.synced_folder '.', '/vagrant', :disabled => true if disable_vagrant_sync
+#        override.ssh.private_key_path = private_key_path
+#        @private_key_name = 'Vagrant ' + private_key_path.sub(/~\//, '').sub(/\.ssh\//, '').sub(/^id_/, '').gsub(/\W+/, ' ')
+#        provider.ssh_key_name = @private_key_name if @private_key_name
+#        override.ssh.forward_agent = true
+#      end
+#    end
+#
+#    def configure_salt(config)
+#      pillarize_application_configuration
+#      config.vm.synced_folder @salt_root + 'config/salt/', '/srv/salt/config/'
+#      config.vm.synced_folder @salt_root + 'config/pillar/', '/srv/pillar/config/'
+#      config.vm.synced_folder @salt_root + 'salt/', '/srv/salt/salted-rails/'
+#      config.vm.synced_folder @salt_root + 'pillar/', '/srv/pillar/salted-rails/'
+#      # Bootstrap salt
+#      ## config.vm.provision :shell, :inline => 'salt-call --version || wget -O - http://bootstrap.saltstack.org | sudo sh'
+#      # Provisioning #2: masterless highstate call
+#      config.vm.provision :salt do |salt|
+#        @logger.info 'Configuring salt provisioner'  if @logger
+#        minion_file = @rails_root + 'config/salt/vagrant/minion'
+#        minion_file = @salt_root + 'salt/vagrant/minion' unless File.exists? minion_file
+#        salt.minion_config = minion_file
+#        salt.run_highstate = true
+#        salt.verbose = !! @logger
+#      end
+#    end
+#
+#    def configure_ports(vm_config, port_offset=0)
+#      vm_config.vm.network :forwarded_port, :guest => 3000 + port_offset, :host => 3000, auto_correct: true
+#      [ 80, 443, 880 ].each do |port|
+#        vm_config.vm.network :forwarded_port, :guest => port, :host => 3000+port + port_offset, auto_correct: true
+#      end
+#    end
+#
+#    def configure_gui(vm_config, memory = 1024)
+#      @gui = true
+#      configure_memory(memory)
+#      vm_config.vm.boot_mode == :gui 
+#      vm_config.vm.provision :salt do |salt|
+#        minion_file = @rails_root + 'config/salt/vagrant/gui_minion'
+#        minion_file = @salt_root + 'salt/vagrant/gui_minion' unless File.exists? minion_file
+#        salt.minion_config = minion_file
+#      end
+#    end
+#
+#    def configure_memory(memory = 1024)
+#      config.vm.provider :virtualbox do |vb|
+#       vb.customize ["modifyvm", :id, "--memory", memory]
+#      end
+#      config.vm.provider :digital_ocean do |provider, override|
+#        if memory >= 8000
+#          provider.size = '8GB'
+#        elsif memory >= 4000
+#          provider.size = '4GB'
+#        elsif memory >= 2000
+#          provider.size = '2GB'
+#        elsif memory >= 1000
+#          provider.size = '1GB'
+#        else
+#          provider.size = '512MB'
+#        end
+#      end
+#    end
+
+
+  end
+
+end

data/lib/salted-rails/cloud_vbguest_installer.rb

@@ -0,0 +1,14 @@
+module SaltedRails
+  class CloudVbguestInstaller < VagrantVbguest::Installers::Ubuntu
+    def install(opts=nil, &block)
+      communicate.sudo("apt-get -y -q purge virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11", opts, &block)
+      @vb_uninstalled = true
+      super
+    end
+
+    def running?(opts=nil, &block)
+      return false if @vb_uninstalled
+      super
+    end
+  end
+end

data/lib/salted-rails/config.rb

@@ -0,0 +1,212 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+#
+
+require 'log4r'
+
+module SaltedRails
+  class Config
+
+    attr_accessor :private_key_path
+    attr_accessor :machine
+    attr_accessor :hostname
+    attr_accessor :domain
+    attr_accessor :rails_root
+    attr_accessor :salt_root
+    attr_accessor :mirror
+    attr_accessor :logger
+    attr_accessor :memory
+    attr_accessor :ports
+    attr_accessor :disable_vagrant_sync
+    attr_accessor :ca_path
+    attr_accessor :region
+    attr_accessor :forward_agent
+    attr_accessor :files
+    attr_accessor :databases
+    attr_accessor :roles
+    attr_accessor :gems
+    attr_accessor :ruby_version
+    attr_accessor :java_version
+    attr_accessor :php_version
+    attr_accessor :machines
+    attr_accessor :copy_from_home
+
+    def sanitize_dns_name(name)
+      dns_name = name.downcase.gsub(/[^-0-9a-z]+/,'-').sub(/^-+/, '').sub(/-+$/, '')
+    end
+
+    # pass vm.ui for the logger if you want debugging info
+    def initialize(rails_root, machine = 'default')
+      @logger = Log4r::Logger.new("vagrant::salted-rails")
+      @machine = machine
+      @rails_root = rails_root
+      @rails_root += '/' unless @rails_root =~ /\/$/
+      @salt_root = nil
+      # see salt/vagrant/top.sls for other roles
+      @roles = %w{ app web db }
+      @domain = nil
+      @private_key_path = nil
+      @mirror = nil
+      @memory = nil
+      @ports = [ 80, 443, 880, 3000 ]
+      @disable_vagrant_sync = true
+      @ca_path = nil
+      @region = nil
+      @forward_agent = true
+      @files = [ '.ruby-version', '.java-version', '.php-version', '.rvmrc', 'config/database.yml', 'Gemfile', 'Gemfile.lock' ].select{ |f| File.exist?(@rails_root + f) }
+      @copy_from_home = [ ]
+
+      ENV['REMOTE_MACHINE'] = 'true'
+      database_file = @rails_root + 'config/database.yml'
+      @databases = YAML.load(ERB.new(IO.read(database_file)).result) rescue { }
+      ENV['REMOTE_MACHINE'] = nil
+
+      @gems = { }
+      if File.exists? @rails_root + 'Gemfile'
+        File.foreach(@rails_root + 'Gemfile') do |line|
+          if line =~ /^\s*gem\s*['"]([^'"]+)['"][,\s]*(['"]([^'"]+)['"])?/
+            gem = $1
+            version = $3.to_s
+            version = true if version == '' or version !~ /\d/
+            @gems[gem] = version
+          end
+        end
+      end
+      @databases.each do |key, details|
+        @gems[details['adapter']] = true
+      end
+
+      @ruby_version = nil
+      @java_version = nil
+      @php_version = nil
+
+      @machines = [ ]
+      @hostname = nil
+    end
+
+    def gui?
+      @roles.include? 'gui'
+    end
+
+    def normalize
+      unless @memory
+        @memory = 512
+        {
+                'gui' => 1536,
+                'teamcity' => 1536,
+                'cruisecontrolrb' => 512
+        }.each do |role, extra|
+          @memory += extra if @roles.include?(role)
+        end
+      end
+      unless @domain
+        if @hostname
+          @domain = @hostname.sub(/^[^.]*\.?/, '')
+        else
+          @domain = sanitize_dns_name(File.basename(File.expand_path(@rails_root).sub(/\/$/, '').sub(/\/(app|site|web|www|website)\d*$/, ''))) + '.test'
+          @domain = 'railsapp.test' if @domain == '.test'
+        end
+      end
+      if @hostname.nil? or @hostname == ''
+        if @machine == 'default'
+          @hostname = @domain
+        else
+          @hostname = sanitize_dns_name(@machine.to_s)
+          if @hostname == ''
+            @hostname = @domain
+          else
+            @hostname <<= '.' + @domain
+          end
+        end
+      end
+      @ruby_version ||= File.open(@rails_root + '.ruby-version', 'r') do |f_in|
+        f_in.gets.gsub(/\s/,'')
+      end rescue nil
+      @ruby_version ||= File.open(@rails_root + '.rvmrc', 'r') do |f_in|
+        while !ruby_version && (line = f_in.gets) 
+          ruby_version = $1 if line =~ /^\s*environment_id=['"]([^"'@]+)/
+        end
+      end rescue nil
+      @ruby_version ||= '1.9.3'
+
+      @java_version ||= File.open(@rails_root + '.java-version', 'r') do |f_in|
+        f_in.gets.gsub(/\s/,'')
+      end rescue nil
+
+      @php_version ||= File.open(@rails_root + '.php-version', 'r') do |f_in|
+        f_in.gets.gsub(/\s/,'')
+      end rescue nil
+
+      @private_key_path ||= '~/.ssh/id_rsa'
+      @mirror ||= 'auto'
+      @salt_root ||= File.dirname(__FILE__) + '/../../'
+      @ca_path ||= '/etc/ssl/certs/ca-certificates.crt'
+      @ca_path = nil unless File.exist?(@ca_path)
+      @region ||= 'San Francisco 1'
+
+      @machines.each {|m| m.normalize}
+
+      {
+              'teamcity' => 8111,
+              'cruisecontrolrb' => 3333
+      }.each do |role, port|
+        @ports << port if @roles.include?(role) and not @ports.include?(port)
+      end
+
+    end
+
+    def define(machine, &block)
+      obj = self.clone
+      obj.machine = machine
+      obj.machines = [ ]
+      obj.logger = @logger
+      @machines << obj
+      yield(obj) if block_given?
+    end
+
+    # Clone
+    def clone
+      obj = self.dup
+      obj.roles = @roles.dup
+      obj.ports = @ports.dup
+      obj.files = @files.dup
+      obj.copy_from_home = @copy_from_home.dup
+      obj.databases = @databases.dup
+      obj.gems = @gems.dup
+      obj
+    end
+
+    def to_hash
+      {
+        'domain' => @domain,
+        'mirror' => @mirror,
+        'machine' => @machine,
+        'hostname' => @hostname,
+        'memory' => @memory,
+        'disable_vagrant_sync' => @disable_vagrant_sync,
+        'region' => @region,
+        'forward_agent' => @forward_agent,
+        'files' => @files,
+        'databases' => @databases,
+        'gems' => @gems,
+        'ruby_version' => @ruby_version,
+        'java_version' => @java_version,
+        'php_version' => @php_version,
+        'roles' => @roles
+      }
+    end
+
+    def to_yaml
+      normalize
+      self.to_hash.to_yaml
+    end
+
+    def configure_vagrant(config)
+      require 'salted-rails/vagrant_helper'
+      normalize
+      helper = SaltedRails::VagrantHelper.new(self)
+      helper.configure_vagrant(config)
+    end
+
+  end
+end