recap 0.2.0 → 1.0.0

data/.gitignore

@@ -4,4 +4,5 @@
 Gemfile.lock
 pkg/*
 /test-vm/share/*
-capistrano.log
+capistrano.log
+doc

data/README.md

@@ -1,14 +1,59 @@
-[recap](http://github.com/freerange/recap) is an opinionated set of capistrano deployment recipes, designed to use git's strengths to deploy applications and websites in a fast and simple manner.
+# Recap
 
-Recap's core features are:
+[Recap](https://github.com/freerange/recap) is an opinionated set of [Capistrano](https://github.com/capistrano/capistrano) deployment recipes, designed to use git's strengths to deploy applications and websites in a fast and simple manner.
 
-  * Release versions are managed with git.  There's no need for `releases` or `current` folders, and no symlinking.
-  * Intelligently decides whether tasks need to execute.  e.g. The `bundle:install` task will only run if a `Gemfile.lock` exists, and if it has changed since the last deployment.
-  * A dedicated user account and group owns all an application's associated files and processes.
-  * Deployments are run using personal logins.  The right to deploy is granted by adding a user to the application group.
-  * Environment variables are used for application specific configuration.  These can easily be read and set using the `env` and `env:set` tasks.
-  * Out of the box support for `bundler` and `foreman`
 
-For more information, the main documentation can be found at [http://code.gofreerange.com/recap](http://code.gofreerange.com/recap), while the code is available [on github](https://github.com/freerange/recap).
+## Features & Aims
 
-Recap was written by [Tom Ward](http://tomafro.net) and the other members of [Go Free Range](http://gofreerange.com), and is released under the [MIT License](https://github.com/freerange/recap/blob/master/LICENSE).
+  * Releases are managed using git.  All code is deployed to a single directory, and git tags are used to manage different released versions.  No `releases`, `current` or `shared` directories are created, avoiding unnecessary sym-linking.
+  * Deployments do the minimum work possible, using git to determine whether tasks need to run.  e.g. the `bundle:install` task only runs if the app contains a `Gemfile.lock` file and it has changed since the last deployment.
+  * Applications have their own user account and group, owning all of that application's associated files and processes.  This gives them a dedicated environment, allowing environment variables to be used for application specific configuration.  Tasks such as `env`, `env:set` and `env:edit` make setting and changing these variables easy.
+  * Personal accounts are used to deploy to the server, distinct from the application user.  The right to deploy an application is granted simply by adding a user to the application group.
+
+
+## Documentation
+
+For more information, the main documentation can be found at [http://gofreerange.com/recap/docs](http://http://gofreerange.com/recap/docs).
+
+
+## Prerequistes
+
+* Recap's built-in tasks only support deploying to Ubuntu
+* Your user account (as opposed to the application account) must be able to `sudo`
+* Your user account should be able to connect to the remote git repository from your deployment server(s)
+
+
+## Source
+
+The source code is available [on Github](https://github.com/freerange/recap).
+
+
+## Running Tests
+
+- Run the following commands from the checked out project directory.
+- Install dependencies (assumes the bundler gem is installed).
+
+    `$ bundle install`
+
+- Run specs
+
+    `$ bundle exec rake`
+
+- Install [VirtualBox](https://www.virtualbox.org/) - only necessary if you want to run [Cucumber](https://github.com/cucumber/cucumber) features.
+- Install and provision a test VM based on the [Vagrantfile](https://github.com/freerange/recap/blob/master/Vagrantfile) (assumes VirtualBox is installed)
+
+    `$ bundle exec vagrant up`
+
+- Run features
+
+    `$ bundle exec cucumber`
+
+
+## Credits
+
+Recap was written by [Tom Ward](http://tomafro.net) and the other members of [Go Free Range](http://gofreerange.com).
+
+
+## License
+
+Recap is released under the [MIT License](https://github.com/freerange/recap/blob/master/LICENSE).

data/Rakefile

@@ -3,13 +3,27 @@ require 'rocco/tasks'
 require 'rspec/core/rake_task'
 
 desc 'build docs'
-Rocco::Task.new :rocco, 'doc/', ['index.rb', 'lib/**/*.rb']
+task :doc do
+  FileUtils.cd('lib') do
+    files = Dir['**/*.rb']
+    files.each do |source_file|
+      rocco = Rocco.new(source_file, files.to_a, {})
+      dest_file = '../doc/' + source_file.sub(Regexp.new("#{File.extname(source_file)}$"), '.html')
+      FileUtils.mkdir_p(File.dirname(dest_file))
+      File.open(dest_file, 'wb') { |fd| fd.write(rocco.to_html) }
+    end
+  end
+  File.open('doc/index.html', 'w') do |f|
+    f.write <<-EOS
+      <html><meta http-equiv="refresh" content="0; url=recap.html">
+    EOS
+  end
+end
 
 desc 'publish docs'
 task :publish do
-  sha = `git ls-tree -d HEAD doc | awk '{print $3}'`.strip
-  commit = `echo "Publishing docs from master branch" | git commit-tree #{sha} -p refs/heads/gh-pages`.strip
-  `git update-ref refs/heads/gh-pages #{commit}`
+  path = "/home/freerange/docs/recap"
+  system %{ssh gofreerange.com "sudo rm -fr #{path} && mkdir -p #{path}" && scp -r doc/* gofreerange.com:#{path}}
 end
 
 RSpec::Core::RakeTask.new(:spec) do |t|
@@ -17,4 +31,4 @@ RSpec::Core::RakeTask.new(:spec) do |t|
   t.rspec_opts = "-fn --color"
 end
 
-task :default => :spec
+task :default => :spec

data/bin/recap

@@ -1,3 +1,3 @@
-require 'recap/cli'
-Recap::CLI.start
+require 'recap/support/cli'
+Recap::Support::CLI.start
 

data/features/managing-processes.feature

@@ -1,4 +1,3 @@
-
 Feature: Managing processes with foreman
 
   Scenario: Running an application process
@@ -7,6 +6,7 @@ Feature: Managing processes with foreman
     When I run "cap deploy:setup deploy"
     Then the project should own the running application process
 
+  @wip
   Scenario: Running processes can read environment variables
     Given a new ruby project and a bootstrapped server
     And the project has an application process defined in a Procfile

data/features/setting-environment-variables.feature

@@ -6,13 +6,38 @@ Feature: Setting and unsetting environment config variables
     When I run "cap env:set SECRET=very-secure"
     Then the variable "SECRET" should be set to "very-secure"
 
-  @wip
   Scenario: Setting an environment variable based on an existing variable
 
     Given a new project and a bootstrapped server
     When I run "cap env:set SUPER_PATH=\$PATH"
     Then the variable "SUPER_PATH" should be set to the application's PATH
 
+  Scenario: Setting default environment variable values
+
+    Given a new project and a bootstrapped server
+    When I add a default environment variable "PASSWORD" with the value "sup3r-s3cr3t" to the project
+    And I run "cap env:set"
+    Then the variable "PASSWORD" should be set to "sup3r-s3cr3t"
+
+    When I run "cap env:set PASSWORD=anoth3r-passw0rd"
+    Then the variable "PASSWORD" should be set to "anoth3r-passw0rd"
+
+    When I run "cap env:set PASSWORD="
+    Then the variable "PASSWORD" should be set back to "sup3r-s3cr3t"
+
+  Scenario: Resetting back to default values
+
+    Given a new project and a bootstrapped server
+    And I add a default environment variable "PASSWORD" with the value "sup3r-s3cr3t" to the project
+
+    When I run "cap env:set SECRET=something PASSWORD=anoth3r-passw0rd"
+    Then the variable "SECRET" should be set to "something"
+    And the variable "PASSWORD" should be set to "anoth3r-passw0rd"
+
+    When I run "cap env:reset"
+    Then the variable "PASSWORD" should be set back to "sup3r-s3cr3t"
+    And the variable "SECRET" should have no value
+
   Scenario: Unsetting a variable
 
     Given a new project and a bootstrapped server

data/features/steps/capistrano_steps.rb

@@ -14,7 +14,7 @@ end
 
 Given /^a new (ruby )?project and a bootstrapped server$/ do |project_type|
   type = (project_type || 'static').strip
-  start_project server: server, capfile: { recap_require: "recap/#{type}" }
+  start_project server: server, capfile: { recap_require: "recap/recipes/#{type}" }
   project.run_cap 'bootstrap'
 end
 
@@ -60,6 +60,10 @@ When /^I wait for the server to start$/ do
   sleep(5)
 end
 
+When /^I add a default environment variable "([^"]*)" with the value "([^"]*)" to the project$/ do |name, value|
+  project.add_default_env_value_to_capfile(name, value)
+end
+
 Then /^the project should be deployed$/ do
   project.deployed_version.should eql(project.latest_version)
 end
@@ -76,17 +80,17 @@ Then /^the deployed project should include version "([^"]*)" of "([^"]*)"$/ do |
   project.run_on_server("bin/#{gem} --version").strip.should eql(version)
 end
 
-Then /^the variable "([^"]*)" should be set to "([^"]*)"$/ do |name, value|
-  project.run_on_server("sudo su - #{project.name} -c 'env | grep #{name}'").strip.should eql("#{name}=#{value}")
+Then /^the variable "([^"]*)" should be set (?:back )?to "([^"]*)"$/ do |name, value|
+  project.run_on_server("sudo su - #{project.name} -c 'env | grep #{name}'", ".").strip.should eql("#{name}=#{value}")
 end
 
 Then /^the variable "([^"]*)" should be set to the application's PATH$/ do |name|
-  path = project.run_on_server("echo $PATH").strip
-  project.run_on_server("sudo su - #{project.name} -c 'env | grep #{name}'").strip.should eql("#{name}=#{path}")
+  path = project.run_on_server("echo $PATH", ".").strip
+  project.run_on_server("sudo su - #{project.name} -c 'env | grep #{name}'", ".").strip.should eql("#{name}=#{path}")
 end
 
 Then /^the variable "([^"]*)" should have no value$/ do |name|
-  project.run_on_server("sudo su - #{project.name} -c 'env'").include?("#{name}=").should be_false
+  project.run_on_server("sudo su - #{project.name} -c 'env'", ".").include?("#{name}=").should be_false
 end
 
 Then /^the project should own the running application process$/ do

data/features/support/project.rb

@@ -42,7 +42,11 @@ module ProjectSupport
     def initialize(project, options = {})
       super('project/Capfile.erb')
       @project = project
-      @recap_require = options[:recap_require] || 'recap/static'
+      @recap_require = options[:recap_require] || 'recap/recipes/static'
+    end
+
+    def default_environment_values
+      @default_environment_values ||= {}
     end
   end
 
@@ -137,10 +141,19 @@ module ProjectSupport
     end
 
     def write_and_commit_file(path, content = "")
+      write_file(path, content)
+      commit_files(path)
+    end
+
+    def read_file(path)
+      full_path = File.join(repository_path, path)
+      File.read(full_path)
+    end
+
+    def write_file(path, content = "")
       full_path = File.join(repository_path, path)
       FileUtils.mkdir_p File.dirname(full_path)
       File.write(full_path, content)
-      commit_files(path)
     end
 
     def commit_files(*paths)
@@ -153,7 +166,7 @@ module ProjectSupport
     end
 
     def deployment_path(path = "")
-      File.join("/home/#{name}/apps/#{name}", path)
+      File.join("/home/#{name}/app", path)
     end
 
     def deployed_version
@@ -165,8 +178,8 @@ module ProjectSupport
       raise "Exit code returned running 'cap #{command}'" if $?.exitstatus != 0
     end
 
-    def run_on_server(cmd)
-      @server.run("cd #{deployment_path} && #{cmd}")
+    def run_on_server(cmd, path = deployment_path)
+      @server.run("cd #{path} && #{cmd}")
     end
 
     def git(command)
@@ -202,6 +215,12 @@ module ProjectSupport
       write_and_commit_file 'Procfile', Procfile.new(name, command)
     end
 
+    def add_default_env_value_to_capfile(name, value)
+      content = read_file('Capfile')
+      content << "\nset_default_env '#{name}', '#{value}'"
+      write_and_commit_file('Capfile', content)
+    end
+
     def commit_changes
       write_and_commit_file 'project-file', Faker::Lorem.sentence
     end

data/features/templates/project/Capfile.erb

@@ -8,7 +8,7 @@ set :user, 'vagrant'
 ssh_options[:port] = 2222
 ssh_options[:keys] = ['<%= project.private_key_path %>']
 
-server '127.0.0.1', :web
+server '127.0.0.1', :app
 
 # Each project has its own location shared between the host machine and the VM
 

data/lib/recap/recipes/rails.rb

@@ -0,0 +1,6 @@
+# The `recap/recipes/rails` builds on the [ruby](ruby.html)
+# recipe, which provides support for both `bundler` and `foreman`.
+require 'recap/recipes/ruby'
+
+# It adds to this with a number of rails specific tasks.
+require 'recap/tasks/rails'

data/lib/recap/recipes/ruby.rb

@@ -0,0 +1,11 @@
+# Require `recap/recipes/ruby` in your `Capfile` to use the default recap recipies for deploying a
+# Ruby application.
+require 'recap/tasks/deploy'
+
+# If your application uses Bundler, `bundle install` will be run automatically when deploying
+# any changes to your `Gemfile`.
+require 'recap/tasks/bundler'
+
+# If your application uses Foreman, recap will use that to stop, start and restart your
+# application processes.
+require 'recap/tasks/foreman'

data/lib/recap/recipes/static.rb

@@ -0,0 +1,3 @@
+# Require `recap/recipes/static` if you are deploying a simple application which doesn't use
+# Rails, Bundler or Foreman.
+require 'recap/tasks/deploy'

data/lib/recap/recipes.rb

@@ -0,0 +1,18 @@
+# There are three main recipes, defined in [recap/recipes/static.rb](recipes/static.html),
+# [recap/recipes/ruby.rb](recipes/ruby.html) and [recap/recipes/rails.rb](recipes/rails.html)
+# that include tasks for static, ruby-based and rails sites respectively.  One of these should be
+# required at the top of your `Capfile`.
+#
+# The static recipe includes all the main deployment behaviour.  It provides everything you
+# should need to push static content up to one or more servers, as well as the ability to
+# rollback to a previous release if you make a mistake.
+#
+# The ruby recipe builds on this with support for `bundler`, to automatically install any
+# bundled gems.  It also includes `foreman` support, starting and restarting processes
+# defined in a `Procfile`.
+#
+# The rails recipe includes all the above, and adds automatic database migration and
+# asset compilation to each deploy.
+#
+# To swap between each of these, simply change the top line of your `Capfile` to require
+# the one you want.

data/lib/recap/support/capistrano_extensions.rb

@@ -0,0 +1,85 @@
+require 'tempfile'
+
+# These methods are used by recap tasks to run commands and detect when files have changed
+# as part of a deployments
+
+module Recap::Support::CapistranoExtensions
+  # Run a command as the given user
+  def as_user(user, command, pwd = deploy_to)
+    sudo "su - #{user} -c 'cd #{pwd} && #{command}'"
+  end
+
+  # Run a command as root
+  def as_root(command, pwd = deploy_to)
+    as_user 'root', command, pwd
+  end
+
+  # Run a command as the application user
+  def as_app(command, pwd = deploy_to)
+    as_user application_user, command, pwd
+  end
+
+  # Put a string into a file as the application user
+  def put_as_app(string, path)
+    put string, "/tmp/recap-put-as-app"
+    as_app "cp /tmp/recap-put-as-app #{path} && chmod g+rw #{path}", "/"
+  ensure
+    run "rm /tmp/recap-put-as-app"
+  end
+
+  def editor
+    ENV['DEPLOY_EDITOR'] || ENV['EDITOR']
+  end
+
+  # Edit a file on the remote server, using a local editor
+  def edit_file(path)
+    if editor
+      as_app "touch #{path} && chmod g+rw #{path}"
+      local_path = Tempfile.new('deploy-edit').path
+      get(path, local_path)
+      Recap::Support::ShellCommand.execute_interactive("#{editor} #{local_path}")
+      File.read(local_path)
+    else
+      abort "To edit a remote file, either the EDITOR or DEPLOY_EDITOR environment variables must be set"
+    end
+  end
+
+  # Run a git command in the `deploy_to` directory
+  def git(command)
+    run "cd #{deploy_to} && umask 002 && sg #{application_group} -c \"git #{command}\""
+  end
+
+  # Capture the result of a git command run within the `deploy_to` directory
+  def capture_git(command)
+    capture "cd #{deploy_to} && umask 002 && sg #{application_group} -c 'git #{command}'"
+  end
+
+  def exit_code(command)
+    capture("#{command} > /dev/null 2>&1; echo $?").strip
+  end
+
+  def exit_code_as_app(command, pwd = deploy_to)
+    capture(%|sudo -p 'sudo password: ' su - #{application_user} -c 'cd #{pwd} && #{command} > /dev/null 2>&1'; echo $?|).strip
+  end
+
+  # Find the latest tag from the repository.  As `git tag` returns tags in order, and our release
+  # tags are timestamps, the latest tag will always be the last in the list.
+  def latest_tag_from_repository
+    result = capture_git("tag | tail -n1").strip
+    result.empty? ? nil : result
+  end
+
+  # Does the given file exist within the deployment directory?
+  def deployed_file_exists?(path, root_path = deploy_to)
+    exit_code("cd #{root_path} && [ -f #{path} ]") == "0"
+  end
+
+  # Has the given path been created or changed since the previous deployment?  During the first
+  # successful deployment this will always return true.
+  def deployed_file_changed?(path)
+    return true unless latest_tag
+    exit_code("cd #{deploy_to} && git diff --exit-code #{latest_tag} origin/#{branch} #{path}") == "1"
+  end
+
+  Capistrano::Configuration.send :include, self
+end

data/lib/recap/support/cli.rb

@@ -0,0 +1,57 @@
+require 'thor'
+require 'recap/support/shell_command'
+
+module Recap::Support
+
+  # Recap provides a simple command-line tool (`recap`) to generate a `Capfile` in your
+  # project.
+
+  class CLI < Thor
+    include Thor::Actions
+
+    attr_accessor :name, :repository, :recipe, :server
+
+    def self.source_root
+      File.expand_path("../templates", __FILE__)
+    end
+
+    desc 'setup', 'Setup basic capistrano recipes, e.g: recap setup'
+    method_option :name
+    method_option :repository
+    method_option :server
+    method_option :recipe, :type => 'string', :banner => 'static|ruby|rails'
+
+    def setup
+      self.name = options["name"] || guess_name
+      self.repository = options["repo"] || guess_repository
+      self.recipe = options["recipe"] || guess_recipe
+      self.server = options["server"] || 'your-server-address'
+      template 'Capfile.erb', 'Capfile'
+    end
+
+    private
+
+    def guess_name
+      Dir.pwd.split(File::SEPARATOR).last
+    end
+
+    def guess_repository
+      ShellCommand.execute('git remote -v').split[1]
+    rescue
+      warn "Unable to determine git repository.  Setting to <unknown>."
+      "<unknown>"
+    end
+
+    def guess_recipe
+      if File.exist?('Gemfile.lock')
+        if File.read('Gemfile.lock') =~ / rails /
+          'rails'
+        else
+          'ruby'
+        end
+      else
+        'static'
+      end
+    end
+  end
+end

data/lib/recap/{compatibility.rb → support/compatibility.rb}

@@ -5,8 +5,8 @@
 # Including this recipe adds these legacy settings, but provides no guarantee that original tasks
 # will work.  Many are based on assumptions about the deployment layout that no longer hold true.
 
-module Recap::Compatibility
-  extend Recap::Namespace
+module Recap::Support::Compatibility
+  extend Recap::Support::Namespace
 
   # As `git` to manages releases, all deployments are placed directly in the `deploy_to` folder.  The
   # `current_path` is always this directory (no symlinking required).

data/lib/recap/support/environment.rb

@@ -0,0 +1,61 @@
+module Recap::Support
+
+  # This class is used to manipulate environment variables on the remote server.
+  # You should not need to use it directly; you are probably looking for the
+  # [env](../tasks/env.html) tasks instead.
+
+  class Environment
+    def initialize(variables = {})
+      @variables = variables
+    end
+
+    def get(name)
+      @variables[name]
+    end
+
+    def set(name, value)
+      if value.nil? || value.empty?
+        @variables.delete(name)
+      else
+        @variables[name] = value
+      end
+    end
+
+    def set_string(string)
+      if string =~ /\A([A-Za-z0-9_]+)=(.*)\z/
+        set $1, $2
+      end
+    end
+
+    def empty?
+      @variables.empty?
+    end
+
+    def merge(hash)
+      hash.each {|k, v| set(k, v)}
+    end
+
+    def each(&block)
+      @variables.sort.each(&block)
+    end
+
+    def include?(key)
+      @variables.include?(key)
+    end
+
+    def to_s
+      @variables.keys.sort.map do |key|
+        key + "=" + @variables[key] + "\n" if @variables[key]
+      end.compact.join
+    end
+
+    class << self
+      def from_string(string)
+        string.split(/[\n\r]/).inject(new) do |env, line|
+          env.set_string(line)
+          env
+        end
+      end
+    end
+  end
+end

data/lib/recap/support/namespace.rb

@@ -0,0 +1,47 @@
+require 'capistrano'
+require 'recap/support/capistrano_extensions'
+
+# This module is used to capture the definition of capistrano tasks, which makes it
+# easier to test the behaviour of specific tasks without loading everything. If you
+# are writing tests for a collection of tasks, you should put those tasks in a module
+# and extend that module with `Recap::Support::Namespace.
+#
+# You can look at some of the existing tasks (such as [env](../tasks/env.html)) and
+# its corresponding specs for an example of this in practice.
+#
+# You should not need to use this module directly when using recap to deploy.
+
+module Recap::Support::Namespace
+  def self.default_config
+    @default_config
+  end
+
+  def self.default_config=(config)
+    @default_config = config
+  end
+
+  if Capistrano::Configuration.instance
+    self.default_config = Capistrano::Configuration.instance(:must_exist)
+  end
+
+  def capistrano_definitions
+    @capistrano_definitions ||= []
+  end
+
+  def namespace(name, &block)
+    capistrano_definitions << Proc.new do
+      namespace name do
+        instance_eval(&block)
+      end
+    end
+
+    load_into(Recap::Support::Namespace.default_config) if Recap::Support::Namespace.default_config
+  end
+
+  def load_into(configuration)
+    configuration.extend(self)
+    capistrano_definitions.each do |definition|
+      configuration.load(&definition)
+    end
+  end
+end

data/lib/recap/support/shell_command.rb

@@ -0,0 +1,35 @@
+require 'open4'
+
+module Recap::Support
+  class ShellCommand
+    def self.execute(*commands)
+      output, error = "", ""
+      commands.each do |command|
+        status = Open4::popen4(command) do |pid, stdin, stdout, stderr|
+          output, error = stdout.read, stderr.read
+        end
+        unless status.success?
+          message = [
+            "Executing shell command failed.",
+            "  Command: #{command}",
+            "  Status:  #{status.exitstatus}",
+            "  Message: #{error}"
+          ].join("\n")
+          raise message
+        end
+      end
+      output
+    end
+
+    def self.execute_interactive(command)
+      unless system(command)
+        message = [
+          "Executing shell command failed.",
+          "  Command: #{command}",
+          "  Status:  #{$?.exitstatus}"
+        ].join("\n")
+        raise message
+      end
+    end
+  end
+end

data/lib/recap/support/templates/Capfile.erb

@@ -0,0 +1,6 @@
+require 'recap/recipes/<%= recipe %>'
+
+set :application, '<%= name %>'
+set :repository, '<%= repository %>'
+
+server '<%= server %>', :app