carafe 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: a4a7e5d60093d18eadcdf5151550524ba2811152
+  data.tar.gz: 041a8c124e42db6e2526bc1ab6b2b581870f9f40
+SHA512:
+  metadata.gz: 7571338becfac1a55944ca258178190c3fd15039a2d59b8762a4f95b11253e3affc1e19c197f4b9d4f21f8e93d8f21f0b821514c46070cbe3f3c23ebb16aabb8
+  data.tar.gz: 5eb5e9fa216d2e7c1344d33881057153f1570dc8d198c1c62db93579bd1333eba0981417bb882e2a036cda8f5b510531af7689aeb23eeed2e75c1d31e7c384ce

data/LICENSE.md

@@ -0,0 +1,22 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 Thomas Stratmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

data/README.md

@@ -0,0 +1,140 @@
+# Carafe
+
+This is a tool for deploying Elixir applications, built upon [capistrano](http://capistranorb.com/).
+
+Carafe requires git for hosting the source repository. It depends on
+[Edeliver](https://github.com/boldpoker/edeliver) for a few parts that are not handled in
+Carafe yet. Release building requires [Distillery](https://github.com/bitwalker/distillery).
+
+Currently, only full releases, not upgrades, are supported, and archives are
+kept locally.
+
+## Installation
+
+### Prerequisites, Ruby side
+
+You need ruby >= 2.0 installed in your development environment. The recommended way of installing dependencies on the ruby side is via bundler. Create a `Gemfile` at
+the project root containing:
+
+```
+source "https://rubygems.org"
+
+group :development do
+  gem "carafe"
+end
+```
+
+Then run `bundle install --path vendor/bundle`, followed by `bundle exec cap install`. This gives you
+these additional files:
+
+```
+o .bundle/config
++ Capfile
++ Gemfile
++ Gemfile.lock
+o vendor/bundle
++ config/deploy.rb
++ config/deploy/staging.rb
++ config/deploy/production.rb
+```
+
+Files behind `o` should be gitignored, the others checked in. Add the following line to the `Capfile`:
+
+```
+require "carafe"
+```
+
+### Prerequisites, Elixir side
+
+Add these deps to your `mix.exs`:
+
+```
+  defp deps do
+    [
+      {:edeliver, "~> 1.4.2"},
+      {:distillery, "~> 0.9"},
+      {:carafe, "0.2.0"}
+    ]
+  end
+```
+
+and run `mix deps.get`. Add `:edeliver` to your `:extra_applications` AS LAST:
+
+```
+  def application do
+    [extra_applications: [:logger, :edeliver]]
+  end
+```
+
+## Configuration
+
+First, configure your application for [distillery](https://github.com/bitwalker/distillery/).
+
+The [Capistrano documentation on configuration](http://capistranorb.com/documentation/getting-started/configuration/)
+gives you all the technical details on configuration in general. Note that Carafe does not use most of the
+variables listed there. Below you find a short introduction that gets you started.
+
+General configuration goes into `config/deploy.rb`. Capistrano has the concept of "stages", and stage specific
+configuration goes into separate files. For the `production` stage this would be `config/deploy/production.rb`.
+Stage specific configuration has precedence over general one.
+
+To configure the deployment process, we mostly set variables and declare servers, as in this example snippet:
+
+```
+set :application, "my_app"
+set :repo_url, "https://github.com/...."
+
+set :repo_path, "dummy1_repo"
+set :build_path, "build_path"
+server "buildhost1", user: "user", roles: ["build"]
+```
+
+Note we are declaring a host (and how to connect to via ssh) to be a build host by giving it the "build" role.
+There obviously must be only one buildhost. In `config/deploy/production.rb`, we might write
+
+```
+server "main", user: "user", roles: ["app"]
+```
+
+to declare a server as a node to deploy our app to.
+Documentation on the `server`
+options can be found [here](http://capistranorb.com/documentation/advanced-features/properties/).
+
+Here are the config variables honored by carafe:
+
+|Varible|Used for/as...|
+|---|---|
+|`:branch`| git branch to build the release from, or :current for current branch|
+|`:repo_url`| cloning the repo on the build host|
+|`:repo_path`| path of repository cache on build host|
+|`:mix_env`| MIX_ENV environment when running `release` mix task from distillery|
+|`:application`| name of the OTP application|
+|`:distillery_environment`| name of the distillery environment, defaulting to the value of :mix_env|
+|`:build_path`| path to build release on build host|
+|`:app_path`| path on application server where releases are unpacked and operated|
+
+### Umbrella project deployments
+
+A demonstration for an umbrella project is in the `dummies/dummy2` test dummy project.
+
+## Usage
+
+Currently, only deploying releases is supported. Every deploy scenario is a bit different, so
+you need to tell how a deploy is to be done. In `deploy.rb`, add the following line:
+
+```
+task "deploy" => [
+  "buildhost:generate_release",
+  "buildhost:archive:download",
+  "node:archive:upload_and_unpack",
+  "node:full_restart"
+]
+```
+
+You should now be able to perform a production deploy with the command `bundle exec cap production deploy`.
+
+## Development & Contributing
+
+Coming soon.
+
+

data/carafe.gemspec

@@ -0,0 +1,25 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'carafe/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "carafe"
+  spec.version       = Carafe::VERSION
+  spec.authors       = ["Thomas Stratmann"]
+  spec.email         = ["thomas.stratmann@9elements.com"]
+
+  spec.summary       = %q{Deployment for Elixir applications, using capistrano}
+  spec.description   = %q{Deployment for Elixir applications, using capistrano}
+  spec.homepage      = "https://github.com/schnittchen/carafe"
+
+  spec.files         = Dir.glob("{bin,lib}/**/*.rb") +
+                       Dir.glob("{bin,lib}/**/*.rake") +
+                       %w(README.md LICENSE.md carafe.gemspec)
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.13"
+  spec.add_development_dependency "rake", "~> 10.0"
+end

data/lib/capistrano/tasks/buildhost.rake

@@ -0,0 +1,141 @@
+task "buildhost:git:check_reachable" do
+  on Carafe.build_host do |host|
+    Carafe::Buildhost.git.check_repo_is_reachable
+  end
+end
+
+desc "Creates or updates the repo cache on the build host"
+task "buildhost:repo:update" => "buildhost:git:check_reachable" do
+  on Carafe.build_host do |host|
+    unless Carafe::Buildhost.git.repo_mirror_exists?
+      Carafe::Buildhost.git.clone_repo
+    else
+      # .clone_repo respects the repo_path, .update_mirror not.
+      within repo_path do
+        Carafe::Buildhost.git.update_mirror
+      end
+    end
+  end
+end
+
+desc "Deletes the build path on the build host"
+task "buildhost:clean" do
+  on Carafe.build_host do |host|
+    execute :rm, "-rf", Carafe::Buildhost.build_path
+  end
+end
+
+desc "Deletes everything in the build path on the build host, except deps/"
+task "buildhost:clean:keepdeps" do
+  on Carafe.build_host do |host|
+    execute :mkdir, "-p", Carafe::Buildhost.build_path
+    within Carafe::Buildhost.build_path do
+      execute :find,  %w{\( -path './deps/*' -or -path ./deps \) -or -delete}
+    end
+  end
+end
+
+desc "Checks if the needed source revision is available on the build host"
+task "buildhost:check_rev_available" => ["local:gather-rev", "buildhost:repo:update"] do
+  on Carafe.build_host do |host|
+    within repo_path do
+      rev = fetch(:rev)
+      execute :git, "cat-file -e #{rev}^{commit}"
+    end
+  end
+end
+
+desc "Prepare the build path on the build host for building a release"
+task "buildhost:prepare_build_path" => ["buildhost:clean:keepdeps", "buildhost:check_rev_available"] do
+  on Carafe.build_host do |host|
+    rev= fetch(:rev)
+
+    execute :sh, "-c", "git -C #{repo_path} archive #{rev} | tar xfC - #{Carafe::Buildhost.build_path}".shellescape
+  end
+end
+
+desc "Execute `mix deps.get` on the build host"
+task "buildhost:mix:deps.get" do
+  on Carafe.build_host do |host|
+    within Carafe::Buildhost.build_path do
+      with Carafe::Buildhost.mix_env_with_arg do
+        execute :mix, "local.hex", "--force"
+        execute :mix, "local.rebar", "--force"
+        execute :mix, "deps.get"
+      end
+    end
+  end
+end
+
+desc "Execute `mix compile` on the build host"
+task "buildhost:mix:compile" do
+  on Carafe.build_host do |host|
+    within Carafe::Buildhost.build_path do
+      with Carafe::Buildhost.mix_env_with_arg do
+        execute :mix, "compile"
+      end
+    end
+  end
+end
+
+desc "Compile entire project at the build path on the build host"
+task "buildhost:compile" => [
+  "buildhost:mix:deps.get",
+  "buildhost:mix:compile"
+]
+
+desc "Execute `mix release` on the build host"
+task "buildhost:mix:release" do
+  on Carafe.build_host do |host|
+    within Carafe::Buildhost.build_path do
+      with Carafe::Buildhost.mix_env_with_arg do
+        execute :mix, "release", "--env=#{Carafe.distillery_environment}"
+      end
+    end
+  end
+end
+
+desc "Generate release on the build host"
+task "buildhost:generate_release" => [
+  "buildhost:prepare_build_path",
+  "buildhost:compile",
+  "buildhost:mix:release"
+]
+
+task "buildhost:gather-vsn" do
+  on Carafe.build_host do |host|
+    within Carafe::Buildhost.build_path do
+      with Carafe::Buildhost.mix_env_with_arg do
+        # Pull the version out of rel/config.exs
+        arg =
+          %Q{IO.puts Mix.Releases.Config.read!("rel/config.exs").releases[:#{Carafe.distillery_release}].version}.shellescape
+
+        vsn = capture(:mix, "run", "--no-start", "-e", arg).chomp
+        raise ArgumentError if vsn.empty?
+
+        set :vsn, vsn
+      end
+    end
+  end
+end
+
+task "buildhost:archive_path" => "buildhost:gather-vsn" do
+  vsn = fetch(:vsn)
+
+  archive_path =
+    Carafe::Buildhost.build_path.join(
+      "_build", Carafe.distillery_environment,
+      "rel", Carafe.distillery_release,
+      "releases", vsn, "#{Carafe.distillery_release}.tar.gz")
+
+  set :buildhost_archive_path, archive_path
+end
+
+task "buildhost:archive:download" => ["buildhost:archive_path", "local:archive_path"] do
+  buildhost_archive_path = fetch(:buildhost_archive_path)
+  local_archive_path = fetch(:local_archive_path)
+
+  on Carafe.build_host do |host|
+    download! buildhost_archive_path, local_archive_path
+  end
+end

data/lib/capistrano/tasks/local.rake

@@ -0,0 +1,16 @@
+task "local:gather-rev" do
+  rev =
+    run_locally do
+      capture :git, "rev-parse", Carafe.rev_param
+    end
+
+  set :rev, rev
+  set :abbrev_rev, rev[0..9]
+end
+
+# Determines and saves the local path for an archive
+task "local:archive_path" do
+  # TODO
+  archive_path = "archive.tar.gz"
+  set :local_archive_path, archive_path
+end

data/lib/capistrano/tasks/node.rake

@@ -0,0 +1,120 @@
+task "node:archive:upload_and_unpack" => "local:archive_path" do
+  local_archive_path = fetch(:local_archive_path)
+
+  Rake::Task["node:archive:upload"].invoke(local_archive_path)
+  Rake::Task["node:archive:unpack"].invoke
+end
+
+task "node:archive:upload", [:archive_path] => "local:archive_path" do |t, args|
+  on Carafe::Node.hosts do |host|
+    execute :mkdir, "-p", Carafe::Node.app_path
+    upload! args[:archive_path], Carafe::Node.app_path.join("archive.tar.gz")
+  end
+end
+
+task "node:archive:unpack"  do
+  on Carafe::Node.hosts do |host|
+    within Carafe::Node.app_path do
+      execute :tar, "-xzvf", "archive.tar.gz"
+    end
+  end
+end
+
+#task "node:archive:unlink"  do
+#  on Carafe::Node.hosts do |host|
+#    within Carafe::Node.app_path do
+#      execute :rm, archive_path
+#    end
+#  end
+#end
+
+
+desc "Pings the node on each server, fails if one is not responding"
+task "node:ping" do
+  # TODO in case of failure, we should collect all failing nodes
+  # and report eventually.
+  script = Carafe.distillery_release
+  on Carafe::Node.hosts do |host|
+    within Carafe::Node.app_path do
+      execute "bin/#{script}", "ping"
+      info "Host #{host}: pong"
+    end
+  end
+end
+
+desc "Starts all nodes. Has no effect on servers where a node is already running"
+task "node:start" do
+  script = Carafe.distillery_release
+  on Carafe::Node.hosts do |host|
+    within Carafe::Node.app_path do
+      execute "bin/#{script}", "start"
+    end
+  end
+end
+
+desc "Stops all nodes."
+task "node:stop" do
+  # FIXME we should attempt to stop all nodes before potentially failing
+  script = Carafe.distillery_release
+  on Carafe::Node.hosts do |host|
+    within Carafe::Node.app_path do
+      execute "bin/#{script}", "stop"
+    end
+  end
+end
+
+task "node:stop-if-running" do
+  script = Carafe.distillery_release
+  on Carafe::Node.hosts do |host|
+    within Carafe::Node.app_path do
+      if test("bin/#{script}", "ping")
+        execute "bin/#{script}", "stop"
+      end
+    end
+  end
+end
+
+desc "Restarts the node, making sure a new version (including ERTS) is booted."
+task "node:full_restart" => ["node:stop-if-running", "node:start"] do
+  script = Carafe.distillery_release
+  app = Carafe::Node.app_name
+
+  # see https://github.com/boldpoker/edeliver/blob/0582a32546edca8e6b047c956e3dd4ef74b09ac1/libexec/erlang#L856
+  on Carafe::Node.hosts do |host|
+    within Carafe::Node.app_path do
+      execute <<-EOS
+        for i in {1..10}; do bin/#{script} ping && break || true; sleep 1; done
+      EOS
+
+      execute "bin/#{script}", <<-EOS
+      rpcterms Elixir.Edeliver run_command '[monitor_startup_progress, \"#{app}\", verbose].' | grep -e 'Started\\|^ok'
+      EOS
+    end
+  end
+end
+
+desc "Attach to a running node, for introspection"
+task "node:attach" do
+  script = Carafe.distillery_release
+  on Carafe::Node.hosts do |host|
+
+    puts "Attaching to node. When you are done, make sure to detach with Ctrl-D (otherwise node goes down)"
+
+    # This does not respect the configured host's ssh_options,
+    # and depends on the ssh binary of the local host.
+
+    args =
+      [
+        "-l#{host.user}",
+        "-t",
+        ("-p#{host.port}" if host.port),
+        host.hostname,
+        "sh -c 'cd #{Carafe::Node.app_path} && bin/#{script} attach'"
+      ].compact
+
+    pid = spawn("ssh", *args)
+    Process.waitpid(pid, Process::WUNTRACED)
+    raise "Error, node probably not running" unless $?.success?
+  end
+end
+

data/lib/carafe/version.rb

@@ -0,0 +1,3 @@
+module Carafe
+  VERSION = "0.1.0"
+end

data/lib/carafe.rb

@@ -0,0 +1,75 @@
+require "carafe/version"
+
+load File.expand_path("../capistrano/tasks/local.rake", __FILE__)
+load File.expand_path("../capistrano/tasks/buildhost.rake", __FILE__)
+load File.expand_path("../capistrano/tasks/node.rake", __FILE__)
+
+module Carafe
+  def self.build_host
+    hosts = roles(:build)
+
+    if hosts.none?
+      raise "No build host available."
+    end
+
+    if hosts.length > 1
+      raise "There can only be one build host."
+    end
+
+    hosts.first
+  end
+
+  def self.rev_param
+    branch = fetch(:branch) { raise "you need to set :branch to a branch name or :current" }
+
+    if branch == :current
+      "@"
+    else
+      branch
+    end
+  end
+
+  def self.mix_env
+    fetch(:mix_env) { raise "set :mix_env in stage config!" }.to_s
+  end
+
+  def self.distillery_release
+    fetch(:application) { raise }
+  end
+
+  def self.distillery_environment
+    fetch(:distillery_environment) { mix_env }
+  end
+
+  module Buildhost
+    def self.git
+      Capistrano::SCM::Git.new
+    end
+
+    def self.build_path
+      Pathname(fetch(:build_path) { raise "no build_path configured" })
+    end
+
+    def self.mix_env_with_arg
+      { mix_env: Carafe.mix_env }
+    end
+  end
+
+  module Node
+    def self.app_path
+      Pathname(fetch(:app_path) { raise "set :app_path node path where the release is unpacked an run" })
+    end
+
+    def self.app_name
+      fetch(:application) { raise }
+    end
+
+    def self.hosts
+      hosts = roles(:app)
+      if hosts.none?
+        raise "No hosts have been configured with role 'app'"
+      end
+      hosts
+    end
+  end
+end

metadata

@@ -0,0 +1,80 @@
+--- !ruby/object:Gem::Specification
+name: carafe
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Thomas Stratmann
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2017-05-11 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.13'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.13'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+description: Deployment for Elixir applications, using capistrano
+email:
+- thomas.stratmann@9elements.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- LICENSE.md
+- README.md
+- carafe.gemspec
+- lib/capistrano/tasks/buildhost.rake
+- lib/capistrano/tasks/local.rake
+- lib/capistrano/tasks/node.rake
+- lib/carafe.rb
+- lib/carafe/version.rb
+homepage: https://github.com/schnittchen/carafe
+licenses: []
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.5.2
+signing_key: 
+specification_version: 4
+summary: Deployment for Elixir applications, using capistrano
+test_files: []
+has_rdoc: