mamiya 0.0.1.beta3 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: feb2301bfc6bf520112e3606038d9c5387a3bd94
-  data.tar.gz: b4f96200bd9d5d2495cfd62076f4a9e804f6ef61
+  metadata.gz: 8fa28dadde7c00c1165fb66de3fceee10b564577
+  data.tar.gz: 614971d0864612144dab18e4db4aee2c3c7a1126
 SHA512:
-  metadata.gz: d18e32e3a5ee9ec3a46560337b810c254f1f51008eb1e3ec0be4ab8a3df3b774d73397e21bb6b341b317602b14ec47db5b3ef1281f9300c09f2b40faf81ba47f
-  data.tar.gz: 2004514ee647cec743bcce5a0230f0ceb012157bcae4662a32ba82efda376e8307c0cbdd142c03334e836969ef5fc1feb3217a0f298370b315972a4a0c9ab8c8
+  metadata.gz: 494a8f6d5d1e171ec5f2a154a81c534780eb464e6bc30adff4df5e9393788a43eb57a6f2ffaea7e11ce0208c34b9f5a64547fed531149da478a083380ec18a96
+  data.tar.gz: 3cc97e2941f5a10f5ce67552ec1066431f08c1b6ffe4748a2563fa1f085229bc68074e702649529fa3492a682f36f516795b6b963c4f2dbebcf81136be52fd86

data/.travis.yml

@@ -1,7 +1,8 @@
 language: ruby
 cache: bundler
 rvm:
-  - "2.1.1"
+  - "2.2"
+  - "2.1"
   - "2.0.0"
   - "ruby-head"
 

data/README.md

@@ -6,28 +6,22 @@ Mamiya allows you to deploy using without ssh -- using tarballs, some storages (
 
 ## Installation
 
-    $ gem install mamiya
-
-or bundle it:
-
-``` ruby
-gem 'mamiya'
-```
-
-## Usage
-
 (Detailed documentation coming soon)
 
-1. Prepare master node for `mamiya master` process
+1. Install gem (`gem install mamiya`, or bundling by `gem 'mamiya'`)
+2. Prepare master node for `mamiya master` process
   - master has HTTP API to allow control the cluster via HTTP
   - master watches agents in the cluster
-2. Then install `mamiya agent` in your deployment target hosts
-3. Write deploy script for your own
+3. Then install `mamiya agent` in your deployment target hosts
+4. Write deploy script for your own
   - how to build package, how to prepare release, how to release, etc.
-4. Build then push package
-5. Time to deploy
+5. Build then push package
+6. Time to deploy
 
-[example](./example) directory contains configuration files that work out-of-the-box. Try Mamiya in your local machine: `foreman start`
+### Example configuration
+
+[example](./example) directory contains configuration files that work out-of-the-box.
+Try Mamiya in your local machine: `foreman start`
 
 ## Problems in existing deploy tool
 
@@ -39,30 +33,13 @@ This solves such problem by using [Serf](http://www.serfdom.io/) and tarball on
 Also, I'm planning to allow to distribute files before the deploy command. I guess this can skip or shorten
 file transferring phase in deploy.
 
+## Misc.
 
-## Upgrade Notes
-
-### 0.0.1.alpha21
-
-#### Configuration now written in Ruby
+- [Scalable Deployments - How we deploy Rails app to 150+ hosts in a minute // Speaker Deck](https://speakerdeck.com/sorah/scalable-deployments-how-we-deploy-rails-app-to-150-plus-hosts-in-a-minute)
 
-You should rewrite your configuration yaml in Ruby. See examples/config.rb for example.
-
-### 0.0.1.alpha20
-
-_tl;dr_ Don't mix alpha19 and alpha20.
-
-#### Internal component for distribution has been replaced completely
-
-alpha20 introduces new class `TaskQueue` and removes `Fetcher`. This changes way to distribute packages -- including internal serf events, job tracking that Distribution API does, etc.
-So basically there's no compatibility for distribution, between alpha19 and alpha20 and later. Distribute task from alpha20 doesn't effect to alpha19, and vice versa.
-
-Good new: There's no change in Distribution API.
-
-#### Agent status has changed
+## Upgrade Notes
 
-- Due to removal of `Fetcher`, alpha20 removes `.fetcher` object from agent status.
-- Added `.queues`, represents task queues that managed by `TaskQueue` class.
+See [docs/upgrading.md](./docs/upgrading.md).
 
 ## Contributing
 

data/docs/upgrading.md

@@ -0,0 +1,25 @@
+# Upgrade Notes
+
+## 0.0.1.alpha21
+
+### Configuration now written in Ruby
+
+You should rewrite your configuration yaml in Ruby. See examples/config.rb for example.
+
+## 0.0.1.alpha20
+
+_tl;dr_ Don't mix alpha19 and alpha20.
+
+### Internal component for distribution has been replaced completely
+
+alpha20 introduces new class `TaskQueue` and removes `Fetcher`. This changes way to distribute packages -- including internal serf events, job tracking that Distribution API does, etc.
+So basically there's no compatibility for distribution, between alpha19 and alpha20 and later. Distribute task from alpha20 doesn't effect to alpha19, and vice versa.
+
+Good new: There's no change in Distribution API.
+
+### Agent status has changed
+
+- Due to removal of `Fetcher`, alpha20 removes `.fetcher` object from agent status.
+- Added `.queues`, represents task queues that managed by `TaskQueue` class.
+
+

data/example/config.agent.rb

@@ -42,6 +42,7 @@ set :keep_packages, 3
 set :keep_prereleases, 3
 set :fetch_sleep, 2
 
+_labels = [rand(2) == 0 ? :a : :b]
 labels do
-  [:"p#{ENV['PORT']}"]
+  [:"p#{ENV['PORT']}", *_labels]
 end

data/example/deploy.rb

@@ -16,6 +16,28 @@ set :build_to, "#{File.dirname(__FILE__)}/builds"
 # to test it
 Dir.mkdir build_to unless File.exist?(build_to)
 
+build("test") do
+  # build something...
+  File.write('built_at', "#{Time.now}\n")
+end
+
+prepare 'test' do
+  # run 'bundle', 'install'
+  sleep 5
+  File.write release_path.join('prepared_at'), Time.now
+end
+
+release 'test' do
+  # run '/etc/init.d/app', 'reload'
+  i = rand(10)
+  logger.info "SLEEP #{i} sec"
+  sleep i
+  File.write release_path.join('released_at'), Time.now
+end
+
+
+
+
 #set :bundle_without, [:development, :test]
 #set :bundle_dir, "#{deploy_to}/shared/bundle"
 
@@ -25,10 +47,6 @@ Dir.mkdir build_to unless File.exist?(build_to)
 #  run "bundle", "install"
 #end
 
-build("test") do
-  File.write('built_at', "#{Time.now}\n")
-end
-
 #build("include assets and .bundle") do
 #  exclude_from_package.reject! { |_| _ == 'public/assets/' }
 #  exclude_from_package.reject! { |_| _ == '.bundle/' }
@@ -38,21 +56,4 @@ end
 #  run "bundle", "exec", "rake", "assets:precompile"
 #end
 
-prepare 'test' do
-  # run 'bundle', 'install'
-  logger.info "- prep/deploy_to: #{deploy_to}"
-  logger.info "- prep/release_path: #{release_path}"
-  sleep 5
-  File.write release_path.join('prepared_at'), Time.now
-end
-
-release 'test' do
-  logger.info "- prep/deploy_to: #{deploy_to}"
-  logger.info "- prep/release_path: #{release_path}"
-  sl = rand(80)
-  logger.info "!!!!!!!!!!!!!!! sleep #{sl} sec"
-  sleep sl
-  File.write release_path.join('released_at'), Time.now
-end
-
 

data/lib/mamiya/agent/task_queue.rb

@@ -117,6 +117,8 @@ module Mamiya
           begin
             status[:lock].synchronize do
               status[:pending].delete task
+
+              task['start'] = Time.now.to_i
               status[:working] = task
             end
             task_logger = @logger.with_clean_progname

data/lib/mamiya/agent/tasks/switch.rb

@@ -52,8 +52,10 @@ module Mamiya
             raise PrereleaseMissing, "Existing release is not prepared but prerelease doesn't exist"
           end
 
-          logger.info "Copying #{prerelease_path} -> #{release_path}"
-          FileUtils.cp_r prerelease_path, release_path
+          unless release_path.exist?
+            logger.info "Copying #{prerelease_path} -> #{release_path}"
+            FileUtils.cp_r prerelease_path, release_path
+          end
 
           logger.info "Switching"
           switch_step.run!

data/lib/mamiya/steps/switch.rb

@@ -5,28 +5,18 @@ module Mamiya
     class Switch < Abstract
       def run!
         @exception = nil
-        logger.info "Switching to #{target}"
-
-        script.before_switch(labels)[]
-
-        # TODO: link with relative if available?
-        # TODO: Restore this if FAILED
-        File.unlink script.current_path if script.current_path.symlink?
-        script.current_path.make_symlink(target.realpath)
+        @switched = false
 
-        if do_release?
-          begin
-            old_pwd = Dir.pwd
-            Dir.chdir(target)
-
-            logger.info "Releasing..."
+        if current_targets_release?
+          logger.info "Already switched"
+        else
+          switch
+        end
 
-            script.release(labels)[@exception]
-          ensure
-            Dir.chdir old_pwd if old_pwd
-          end
+        if @switched ? do_release? : force_release?
+          release
         else
-          logger.warn "Skipping release (:no_release is set)"
+          logger.warn "Skipping release"
         end
 
       rescue Exception => e
@@ -35,11 +25,34 @@ module Mamiya
       ensure
         logger.warn "Exception occured, cleaning up..." if @exception
 
-        script.after_switch(labels)[@exception]
+        script.after_switch(labels)[@exception] if @switched
 
         logger.info "DONE!" unless @exception
       end
 
+      def switch
+        logger.info "Switching to #{target}"
+        @switched = true
+        script.before_switch(labels)[]
+
+        File.unlink script.current_path if script.current_path.symlink?
+        script.current_path.make_symlink(target.realpath)
+      end
+
+      def release
+        # TODO: link with relative if available?
+        # TODO: Restore this if FAILED
+
+        old_pwd = Dir.pwd
+        Dir.chdir(target)
+
+        logger.info "Releasing..."
+
+        script.release(labels)[@exception]
+      ensure
+        Dir.chdir old_pwd if old_pwd
+      end
+
       # XXX: dupe with prepare step. modulize?
 
       # This class see target_dir's script
@@ -57,8 +70,20 @@ module Mamiya
 
       private
 
+      def current_targets_release?
+        script.current_path.exist? && script.current_path.realpath == target.realpath
+      end
+
       def do_release?
-        !options[:no_release]
+        force_release? ? true : !no_release?
+      end
+
+      def force_release?
+        !!options[:do_release]
+      end
+
+      def no_release?
+        !!options[:no_release]
       end
 
       def target

data/lib/mamiya/version.rb

@@ -1,3 +1,3 @@
 module Mamiya
-  VERSION = "0.0.1.beta3"
+  VERSION = "0.1.0"
 end

data/spec/agent/task_queue_spec.rb

@@ -41,7 +41,6 @@ describe Mamiya::Agent::TaskQueue do
     end
   end
 
-
   let(:task_class_a) do
     Class.new(task_class_root) do
       def self.identifier
@@ -62,6 +61,10 @@ describe Mamiya::Agent::TaskQueue do
     described_class.new(agent, task_classes: [task_class_a, task_class_b])
   end
 
+  before do
+    allow(Time).to receive(:now).and_return(Time.now)
+  end
+
   describe "lifecycle (#start!, #stop!)" do
     it "can start and stop" do
       expect(queue).not_to be_running
@@ -170,7 +173,7 @@ describe Mamiya::Agent::TaskQueue do
 
         100.times { break unless task_class_a.locks.empty?; sleep 0.01 }
         expect(task_class_a.locks).not_to be_empty
-        expect(queue.status[:a][:working]).to eq('wait' => true, 'id' => 1, 'task' => :a)
+        expect(queue.status[:a][:working]).to eq('wait' => true, 'id' => 1, 'task' => :a, 'start' => Time.now.to_i)
 
         queue.enqueue(:a, 'id' => 2)
         100.times { break unless queue.status[:a][:queue].empty?; sleep 0.01 }
@@ -236,8 +239,8 @@ describe Mamiya::Agent::TaskQueue do
           expect(task_class_a.locks).not_to be_empty
           expect(task_class_b.locks).not_to be_empty
 
-          expect(queue.status[:a][:working]).to eq('wait' => true, 'id' => 1, 'task' => :a)
-          expect(queue.status[:b][:working]).to eq('wait' => true, 'id' => 2, 'task' => :b)
+          expect(queue.status[:a][:working]).to eq('wait' => true, 'id' => 1, 'task' => :a, 'start' => Time.now.to_i)
+          expect(queue.status[:b][:working]).to eq('wait' => true, 'id' => 2, 'task' => :b, 'start' => Time.now.to_i)
 
           queue.enqueue(:a, 'id' => 3)
           queue.enqueue(:b, 'id' => 4)

data/spec/steps/switch_spec.rb

@@ -119,6 +119,42 @@ describe Mamiya::Steps::Switch do
       }.from(nil).to(release_path.realpath)
     end
 
+    context "when current targets the target" do
+      before do
+        current_path.make_symlink(release_path.realpath)
+      end
+
+      it "doesn't call hooks" do
+        called = false
+        allow(script).to receive(:release).and_return(proc { called = true })
+        allow(script).to receive(:before_switch).and_return(proc { called = true })
+        allow(script).to receive(:after_switch).and_return(proc { called = true })
+        step.run!
+
+        expect(called).to be_false
+      end
+
+      context "with do_release" do
+        let(:options) do
+          {target: release_path, do_release: true}
+        end
+
+        it "calls hooks" do
+          hooks = %i(release)
+
+          flags = []
+          hooks.each do |sym|
+            allow(script).to receive(sym).and_return(proc { flags << sym })
+          end
+
+          expect { step.run! }.
+            to change { flags }.
+            from([]).
+            to(hooks)
+        end
+      end
+    end
+
     context "when current already exists" do
       before do
           current_path.make_symlink('releases/20140515000707')

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: mamiya
 version: !ruby/object:Gem::Version
-  version: 0.0.1.beta3
+  version: 0.1.0
 platform: ruby
 authors:
 - Shota Fukumori (sora_h)
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-09-17 00:00:00.000000000 Z
+date: 2014-09-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
@@ -180,9 +180,7 @@ files:
 - README.md
 - Rakefile
 - bin/mamiya
-- docs/cli.md
-- docs/sequences/deploy.png
-- docs/sequences/deploy.uml
+- docs/upgrading.md
 - example/.gitignore
 - example/Procfile
 - example/README.md
@@ -289,9 +287,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">"
+  - - ">="
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubyforge_project: 
 rubygems_version: 2.2.2

data/docs/cli.md

@@ -1,78 +0,0 @@
-# Commands
-
-## Global options
-
-- `-C`, `--config`: path to configuration file (default: `./config.yml`)
-  - config file won't be required if a command don't need it.
-- `-S`, `--script`: path to script file (default: `./deploy.rb`)
-  - script file won't be required if a command don't need it.
-- `-a`, `--app`: Specify application name to operate
-- `-d`, `--debug`: Enable debug mode
-- `--color`, `--no-color`: Enable or disable colored output. When the stdout is a terminal, it'll be enabled by default.
-
-## Storage related commands
-
-### `list-applications` - list application in storage
-
-```
-$ mamiya list-applications -C ./config.yml
-```
-
-### `list-packages` - list packages for specified app in storage
-
-```
-$ mamiya list-packages -C ./config.yml -a myapp
-```
-
-- __Requires:__ configuration file, application name
-  - application name will be retrieved from deploy script when not specified.
-- __Options:__
-  - `-n`, `--name-only`: Show only names (without heading text)
-
-### `show` - show package information
-
-```
-$ mamiya list-packages -C ./config.yml -a myapp PACKAGE_NAME
-```
-
-- __Requires:__ configuration file, application name, package name
-  - application name will be retrieved from deploy script when not specified.
-- __Options:__
-  - `-f`, `--format`: Choose output format from `pp`, `json`, or `yaml`. Default: `pp`.
-
-## Build, pushing and fetching packages
-
-### `build` - build package using script
-
-```
-$ mamiya build --script ./deploy.rb --source source_dir --destination dest_dir
-```
-
-- __Requires:__ application name, source, destination, deploy script
-  - source, description, and app name will be taken from deploy script, if omitted
-- __Options:__
-  - `-f`, `--source`, `--build-from`: directory for package source.
-  - `-t`, `--destination`, `--build-to`: directory to save built packages.
-  - `-P`, `--skip-prepare-build`: Skip prepare build phase of deploy script.
-
-### `push` - push built package to the storage
-
-### `fetch`
-
-## Package related commands
-
-### `extract`
-
-## `client` - API client for `mamiya master`
-
-### `list-applications`
-
-### `list-packages`
-
-### `show-package`
-
-### `list-agents`
-
-### `show-agent`
-
-### `show-distribution`

data/docs/sequences/deploy.uml

@@ -1,58 +0,0 @@
-@startuml
-== Building ==
-Master --> Master: Build package
-note right of Master: e.g. assets precompile
-Master --> Storage: Upload package
-
-== Distributing ==
-Master --> Client: Download request
-activate Master
-Master --> Master: Wait all clients to have package to deploy
-
-alt if doesn't have package
-    activate Client
-    Client --> Client: Set status serf tag
-    Client --> Storage: HTTP request
-    Storage --> Client: Response package
-    Client --> Client: Set packages serf tag
-    Client --> Master: Report download finish
-    deactivate Client
-end
-
-deactivate Master
-
-== Preparing ==
-
-Master --> Client: Request to prepare package
-activate Master
-Master --> Master: Wait all clients to have package to deploy
-activate Client
-Client --> Client: Set status serf tag
-Client --> Client: Unpack package
-Client --> Client: Run prepare phase
-note left of Client: e.g. bundle install, config files
-Client --> Client: Set prepared serf tag
-Client --> Client: Set status serf tag
-Client --> Master: Report preparation finish
-deactivate Client
-deactivate Master
-
-== Finalize ==
-
-Master --> Client: Request finalize
-activate Master
-Master --> Master: Wait all clients restarted
-activate Client
-Client --> Client: Set status serf tag
-Client --> Client: Run finalize phase
-note left of Client: e.g. restart web server
-Client --> Master: Report finalize complete
-Client --> Client: Set status serf tag
-Client --> Client: Wait to be reloaded
-Client --> Client: Set status serf tag
-Client --> Client: Set deployed serf tag
-Client --> Master: Report deploy complete
-deactivate Client
-deactivate Master
-
-@enduml