capistrano-s3-copy 0.0.1 → 0.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore CHANGED
@@ -3,7 +3,6 @@
3
3
  .bundle
4
4
  .config
5
5
  .yardoc
6
- Gemfile.lock
7
6
  InstalledFiles
8
7
  _yardoc
9
8
  coverage
@@ -16,4 +15,5 @@ test/tmp
16
15
  test/version_tmp
17
16
  tmp
18
17
 
18
+ .idea
19
19
  .rvmrc
data/Gemfile.lock ADDED
@@ -0,0 +1,31 @@
1
+ PATH
2
+ remote: .
3
+ specs:
4
+ capistrano-s3-copy (0.0.1)
5
+ capistrano (>= 2.12.0)
6
+ s3sync
7
+
8
+ GEM
9
+ remote: https://rubygems.org/
10
+ specs:
11
+ capistrano (2.12.0)
12
+ highline
13
+ net-scp (>= 1.0.0)
14
+ net-sftp (>= 2.0.0)
15
+ net-ssh (>= 2.0.14)
16
+ net-ssh-gateway (>= 1.1.0)
17
+ highline (1.6.11)
18
+ net-scp (1.0.4)
19
+ net-ssh (>= 1.99.1)
20
+ net-sftp (2.0.5)
21
+ net-ssh (>= 2.0.9)
22
+ net-ssh (2.3.0)
23
+ net-ssh-gateway (1.1.0)
24
+ net-ssh (>= 1.99.1)
25
+ s3sync (1.2.5)
26
+
27
+ PLATFORMS
28
+ ruby
29
+
30
+ DEPENDENCIES
31
+ capistrano-s3-copy!
data/README.md CHANGED
@@ -2,16 +2,12 @@
2
2
 
3
3
  This is a revised implementation of the ideas in Bill Kirtleys capistrano-s3 gem.
4
4
 
5
- I have a requirement to push new deployments via capistrano, but also to retain the last
6
- deployed package in S3 for the purposes of auto-scaling.
5
+ I have a requirement to push new deployments via capistrano, but also to retain the last deployed
6
+ package in S3 for the purposes of auto-scaling.
7
7
 
8
- This gem use Capistrano's own code to build the tarball package, but instead of
9
- deploying it to each machine, we deploy it to a configured S3 bucket (using s3cmd), then
10
- deploy it from there to the known nodes from the capistrano script.
11
-
12
- At some point, I aim to persist a shell script to accompany the package. This will be used
13
- to instruct a fresh AWS instance how to locate, download and install he S3 package as if
14
- it was deployed via capistrano.
8
+ This gem use Capistrano's own code to package the tarball, but instead of deploying it to each
9
+ machine, we deploy it to a configured S3 bucket (using s3cmd provided by the rahugo-s3sync gem https://github.com/frahugo/s3sync),
10
+ then deploy it from there to the known nodes from the capistrano script.
15
11
 
16
12
  ## Installation
17
13
 
@@ -43,7 +39,7 @@ package to S3
43
39
 
44
40
  Finally, we need to indicate which bucket to store the packages in:
45
41
 
46
- set :aws_s3_copy_bucket, 'mybucket-deployments'
42
+ set :aws_releases_bucket, 'mybucket-deployments'
47
43
 
48
44
  The package will be stored in S3 prefixed with a rails_env that was set in capistrano:
49
45
 
@@ -51,6 +47,105 @@ e.g.
51
47
 
52
48
  S3://mybucket-deployment/production/201204212007.tar.gz
53
49
 
50
+ If the deployment succeeds, another file is written to S3:
51
+
52
+ S3://mybucket-deployment/production/aws_install.sh
53
+
54
+ The intention is that auto-scaled instances started after the deploy could download this well-known script
55
+ to an AMI, and executing it would bring down the latest tarball, and extract it in a manner similar to
56
+ someone running:
57
+
58
+ cap deploy:setup
59
+ cap deploy
60
+
61
+ Of course, everyone has tweaks that they make to the standard capistrano recipe. For this reason, the script
62
+ thats executed is generated from an ERB template.
63
+
64
+ #!/bin/sh
65
+
66
+ # Auto-scaling capistrano like deployment script Rails3 specific.
67
+
68
+ echo "AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}"
69
+ echo "AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}"
70
+ AWS_RELEASES_BUCKET=<%= configuration[:aws_releases_bucket] %>
71
+
72
+ RAILS_ENV=<%= configuration[:rails_env] %> # e.g. production
73
+ DEPLOY_TO=<%= configuration[:deploy_to] %> # e.g. /u/apps/myapp
74
+ RELEASES_PATH=<%= configuration[:releases_path] %> # e.g. /u/apps/myapp/releases
75
+ RELEASE_PATH=<%= configuration[:release_path] %> # e.g. /u/apps/myapp/releases/20120428210958
76
+ SHARED_PATH=<%= configuration[:shared_path] %> # e.g. /u/apps/myapp/shared
77
+ CURRENT_PATH=<%= configuration[:current_path] %> # e.g. /u/apps/myapp/current
78
+
79
+ PACKAGE_NAME=<%= File.basename(filename) %> # e.g. 20120428210958.tar.gz
80
+ S3_PACKAGE_PATH=${RAILS_ENV}/${PACKAGE_NAME} # e.g. production/20120428210958.tar.gz
81
+ DOWNLOADED_PACKAGE_PATH=<%= remote_filename %> # e.g. /tmp/20120428210958.tar.gz
82
+ DECOMPRESS_CMD="<%= decompress(remote_filename).join(" ") %>" # e.g. tar xfz /tmp/20120428210958.tar.gz
83
+
84
+ mkdir -p $DEPLOY_TO
85
+ mkdir -p $RELEASES_PATH
86
+ mkdir -p ${SHARED_PATH}
87
+ mkdir -p ${SHARED_PATH}/system
88
+ mkdir -p ${SHARED_PATH}/log
89
+ mkdir -p ${SHARED_PATH}/pids
90
+
91
+ touch ${SHARED_PATH}/log/${RAILS_ENV}.log
92
+ chmod 0666 ${SHARED_PATH}/log/${RAILS_ENV}.log
93
+ chmod -R g+w ${DEPLOY_TO}
94
+
95
+ # AFTER: cap deploy:setup
96
+ # Project specific shared directories
97
+ # mkdir -p ${SHARED_PATH}/content
98
+ # mkdir -p ${SHARED_PATH}/uploads
99
+
100
+ # cap deploy:update_code
101
+ s3cmd get ${AWS_RELEASE_BUCKET}:${S3_PACKAGE_PATH} ${DOWNLOADED_PACKAGE_PATH} 2>&1
102
+ cd ${RELEASES_PATH} && ${DECOMPRESS_CMD} && rm ${DOWNLOADED_PACKAGE_PATH}
103
+
104
+ # cap deploy:assets_symlink (Rails 3.x specific)
105
+ rm -rf ${RELEASE_PATH}/public/assets
106
+ mkdir -p ${RELEASE_PATH}/public
107
+ mkdir -p ${DEPLOY_TO}/shared/assets
108
+ ln -s ${SHARED_PATH}/assets ${RELEASE_PATH}/public/assets
109
+
110
+ # cap deploy:finalize_update
111
+ chmod -R g+w ${RELEASE_PATH}
112
+ rm -rf ${RELEASE_PATH}/log
113
+ rm -rf ${RELEASE_PATH}/public/system
114
+ rm -rf ${RELEASE_PATH}/tmp/pids
115
+ mkdir -p ${RELEASE_PATH}/public
116
+ mkdir -p ${RELEASE_PATH}/tmp
117
+ ln -s ${SHARED_PATH}/system ${RELEASE_PATH}/public/system
118
+ ln -s ${SHARED_PATH}/log ${RELEASE_PATH}/log
119
+ ln -s ${SHARED_PATH}/pids ${RELEASE_PATH}/tmp/pids
120
+
121
+ # AFTER: cap deploy:finalize_update
122
+ cd ${RELEASE_PATH}
123
+ bundle install --gemfile ${RELEASE_PATH}/Gemfile --path ${SHARED_PATH}/bundle --deployment --quiet --without development test
124
+
125
+ # AFTER: cap deploy:update_code
126
+ # cap deploy:assets:precompile
127
+ cd ${RELEASE_PATH}
128
+ bundle exec rake RAILS_ENV=${RAILS_ENV} RAILS_GROUPS=assets assets:precompile
129
+
130
+ # Project specific shared symlinking
131
+ #ln -nfs ${SHARED_PATH}/content ${RELEASE_PATH}/public/content
132
+ #ln -nfs ${SHARED_PATH}/uploads ${RELEASE_PATH}/public/uploads
133
+
134
+ # cap deploy:create_symlink
135
+ rm -f ${CURRENT_PATH}
136
+ ln -s ${RELEASE_PATH} ${CURRENT_PATH}
137
+
138
+ # cap deploy:restart
139
+ # touch ${CURRENT_PATH}/tmp/restart.txt
140
+
141
+ # AFTER: cap deploy:restart
142
+ # cd ${CURRENT_PATH};RAILS_ENV=${RAILS_ENV} script/delayed_job restart
143
+
144
+ An alternative ERB script can be configured via something like this:
145
+
146
+ set :aws_install_script, File.read(File.join(File.dirname(__FILE__), "custom_aws_install.sh.erb")
147
+
148
+
54
149
  ## Contributing
55
150
 
56
151
  1. Fork it
@@ -4,13 +4,14 @@ require File.expand_path('../lib/capistrano-s3-copy/version', __FILE__)
4
4
  Gem::Specification.new do |gem|
5
5
  gem.authors = ["Richie McMahon"]
6
6
  gem.email = ["richie.mcmahon@gmail.com"]
7
- gem.description = %q{Capistrano deployment strategy that creates and pushes a tarball
7
+ gem.description = %q{Capistrano deployment strategy that creates and pushes a tarball
8
8
  into S3, for both pushed deployments and pulled auto-scaling.}
9
9
  gem.summary = %q{Capistrano deployment strategy that transfers the release on S3}
10
10
  gem.homepage = "http://github.com/richie/capistrano-s3-copy"
11
-
11
+
12
12
  gem.add_dependency 'capistrano', ">= 2.12.0"
13
-
13
+ gem.add_dependency 'frahugo-s3sync'
14
+
14
15
  gem.files = `git ls-files`.split($\)
15
16
  gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
16
17
  gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
@@ -0,0 +1,79 @@
1
+ #!/bin/sh
2
+
3
+ # Auto-scaling capistrano like deployment script Rails3 specific.
4
+
5
+ echo "AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}"
6
+ echo "AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}"
7
+ AWS_RELEASES_BUCKET=<%= configuration[:aws_releases_bucket] %>
8
+
9
+ RAILS_ENV=<%= configuration[:rails_env] %> # e.g. production
10
+ DEPLOY_TO=<%= configuration[:deploy_to] %> # e.g. /u/apps/myapp
11
+ RELEASES_PATH=<%= configuration[:releases_path] %> # e.g. /u/apps/myapp/releases
12
+ RELEASE_PATH=<%= configuration[:release_path] %> # e.g. /u/apps/myapp/releases/20120428210958
13
+ SHARED_PATH=<%= configuration[:shared_path] %> # e.g. /u/apps/myapp/shared
14
+ CURRENT_PATH=<%= configuration[:current_path] %> # e.g. /u/apps/myapp/current
15
+
16
+ PACKAGE_NAME=<%= File.basename(filename) %> # e.g. 20120428210958.tar.gz
17
+ S3_PACKAGE_PATH=${RAILS_ENV}/${PACKAGE_NAME} # e.g. production/20120428210958.tar.gz
18
+ DOWNLOADED_PACKAGE_PATH=<%= remote_filename %> # e.g. /tmp/20120428210958.tar.gz
19
+ DECOMPRESS_CMD="<%= decompress(remote_filename).join(" ") %>" # e.g. tar xfz /tmp/20120428210958.tar.gz
20
+
21
+ mkdir -p $DEPLOY_TO
22
+ mkdir -p $RELEASES_PATH
23
+ mkdir -p ${SHARED_PATH}
24
+ mkdir -p ${SHARED_PATH}/system
25
+ mkdir -p ${SHARED_PATH}/log
26
+ mkdir -p ${SHARED_PATH}/pids
27
+
28
+ touch ${SHARED_PATH}/log/${RAILS_ENV}.log
29
+ chmod 0666 ${SHARED_PATH}/log/${RAILS_ENV}.log
30
+ chmod -R g+w ${DEPLOY_TO}
31
+
32
+ # AFTER: cap deploy:setup
33
+ # Project specific shared directories
34
+ # mkdir -p ${SHARED_PATH}/content
35
+ # mkdir -p ${SHARED_PATH}/uploads
36
+
37
+ # cap deploy:update_code
38
+ s3cmd get ${AWS_RELEASE_BUCKET}:${S3_PACKAGE_PATH} ${DOWNLOADED_PACKAGE_PATH} 2>&1
39
+ cd ${RELEASES_PATH} && ${DECOMPRESS_CMD} && rm ${DOWNLOADED_PACKAGE_PATH}
40
+
41
+ # cap deploy:assets_symlink (Rails 3.x specific)
42
+ rm -rf ${RELEASE_PATH}/public/assets
43
+ mkdir -p ${RELEASE_PATH}/public
44
+ mkdir -p ${DEPLOY_TO}/shared/assets
45
+ ln -s ${SHARED_PATH}/assets ${RELEASE_PATH}/public/assets
46
+
47
+ # cap deploy:finalize_update
48
+ chmod -R g+w ${RELEASE_PATH}
49
+ rm -rf ${RELEASE_PATH}/log
50
+ rm -rf ${RELEASE_PATH}/public/system
51
+ rm -rf ${RELEASE_PATH}/tmp/pids
52
+ mkdir -p ${RELEASE_PATH}/public
53
+ mkdir -p ${RELEASE_PATH}/tmp
54
+ ln -s ${SHARED_PATH}/system ${RELEASE_PATH}/public/system
55
+ ln -s ${SHARED_PATH}/log ${RELEASE_PATH}/log
56
+ ln -s ${SHARED_PATH}/pids ${RELEASE_PATH}/tmp/pids
57
+
58
+ # AFTER: cap deploy:finalize_update
59
+ cd ${RELEASE_PATH}
60
+ bundle install --gemfile ${RELEASE_PATH}/Gemfile --path ${SHARED_PATH}/bundle --deployment --quiet --without development test
61
+
62
+ # AFTER: cap deploy:update_code
63
+ # cap deploy:assets:precompile
64
+ cd ${RELEASE_PATH}
65
+ bundle exec rake RAILS_ENV=${RAILS_ENV} RAILS_GROUPS=assets assets:precompile
66
+
67
+ # Project specific shared symlinking
68
+ #ln -nfs ${SHARED_PATH}/content ${RELEASE_PATH}/public/content
69
+ #ln -nfs ${SHARED_PATH}/uploads ${RELEASE_PATH}/public/uploads
70
+
71
+ # cap deploy:create_symlink
72
+ rm -f ${CURRENT_PATH}
73
+ ln -s ${RELEASE_PATH} ${CURRENT_PATH}
74
+
75
+ # cap deploy:restart
76
+ # touch ${CURRENT_PATH}/tmp/restart.txt
77
+
78
+ # AFTER: cap deploy:restart
79
+ # cd ${CURRENT_PATH};RAILS_ENV=${RAILS_ENV} script/delayed_job restart
@@ -1,57 +1,78 @@
1
1
  require 'capistrano/recipes/deploy/strategy/copy'
2
+ require 'erb'
3
+
2
4
 
3
5
  module Capistrano
4
6
  module Deploy
5
7
  module Strategy
6
8
  class S3Copy < Copy
7
-
8
- S3CMD_VARS = ["aws_access_key_id", "aws_secret_access_key"]
9
-
9
+
10
10
  def initialize(config={})
11
- super(config)
12
-
11
+ super(config)
12
+
13
13
  s3cmd_vars = []
14
- S3CMD_VARS.each do |var|
14
+ ["aws_access_key_id", "aws_secret_access_key"].each do |var|
15
15
  value = configuration[var.to_sym]
16
16
  raise Capistrano::Error, "Missing configuration[:#{var}] setting" if value.nil?
17
17
  s3cmd_vars << "#{var.upcase}=#{value}"
18
- end
19
- @aws_environment = s3cmd_vars.join(" ")
20
-
21
- @bucket_name = configuration[:aws_s3_copy_bucket]
22
- raise Capistrano::Error, "Missing configuration[:aws_s3_copy_bucket]" if @bucket_name.nil?
23
- end
24
-
18
+ end
19
+ @aws_environment = s3cmd_vars.join(" ")
20
+
21
+ @bucket_name = configuration[:aws_releases_bucket]
22
+ raise Capistrano::Error, "Missing configuration[:aws_releases_bucket]" if @bucket_name.nil?
23
+ end
24
+
25
25
  def check!
26
26
  super.check do |d|
27
- d.local.command("s3cmd")
27
+ d.local.command("s3cmd")
28
28
  d.remote.command("s3cmd")
29
29
  end
30
- end
31
-
32
- # Distributes the file to the remote servers
30
+ end
31
+
32
+ # Distributes the file to the remote servers
33
33
  def distribute!
34
34
  package_path = filename
35
35
  package_name = File.basename(package_path)
36
- s3_package = "s3://#{bucket_name}/#{rails_env}/#{package_name}"
37
- system("#{aws_environment} s3cmd --no-progress put #{package_path} #{s3_package} 2>&1")
38
-
39
- if $? != 0
40
- raise Capistrano::Error, "shell command failed with return code #{$?}"
41
- end
42
-
36
+ s3_push_cmd = "#{aws_environment} s3cmd put #{bucket_name}:#{rails_env}/#{package_name} #{package_path} x-amz-server-side-encryption:AES256"
37
+
38
+ if configuration.dry_run
39
+ logger.debug s3_push_cmd
40
+ else
41
+ system(s3_push_cmd)
42
+ raise Capistrano::Error, "shell command failed with return code #{$?}" if $? != 0
43
+ end
44
+
43
45
  run "#{aws_environment} s3cmd get #{bucket_name}:#{rails_env}/#{package_name} #{remote_filename} 2>&1"
44
- run "cd #{configuration[:releases_path]} && #{decompress(remote_filename).join(" ")} && rm #{remote_filename}"
46
+ run "cd #{configuration[:releases_path]} && #{decompress(remote_filename).join(" ")} && rm #{remote_filename}"
45
47
  logger.debug "done!"
48
+
49
+ build_aws_install_script
50
+ end
51
+
52
+ def build_aws_install_script
53
+ template_text = configuration[:aws_install_script]
54
+ template_text = File.read(File.join(File.dirname(__FILE__), "aws_install.sh.erb")) if template_text.nil?
55
+ template_text = template_text.gsub("\r\n?", "\n")
56
+ template = ERB.new(template_text, nil, '<>-')
57
+ output = template.result(self.binding)
58
+ local_output_file = File.join(copy_dir, "aws_install.sh")
59
+ File.open(local_output_file, "w") do |f|
60
+ f.write(output)
61
+ end
62
+ configuration[:s3_copy_aws_install_cmd] = "#{aws_environment} s3cmd put #{bucket_name}:#{rails_env}/aws_install.sh #{local_output_file} x-amz-server-side-encryption:AES256"
46
63
  end
47
-
48
- def aws_environment
64
+
65
+ def binding
66
+ super
67
+ end
68
+
69
+ def aws_environment
49
70
  @aws_environment
50
71
  end
51
-
72
+
52
73
  def bucket_name
53
74
  @bucket_name
54
- end
75
+ end
55
76
  end
56
77
  end
57
78
  end
@@ -0,0 +1,15 @@
1
+ Capistrano::Configuration.instance(false).load do
2
+
3
+ namespace :s3_copy do
4
+
5
+ desc "Internal hook that updates the aws_install.sh script to latest if the deploy completed"
6
+ task :store_aws_install_script_on_success do
7
+ cmd = fetch(:s3_copy_aws_install_cmd)
8
+ if cmd
9
+ run_locally(cmd)
10
+ end
11
+ end
12
+ end
13
+
14
+ after 'deploy', 's3_copy:store_aws_install_script_on_success'
15
+ end
@@ -1,7 +1,7 @@
1
1
  module Capistrano
2
2
  module S3
3
3
  module Copy
4
- VERSION = "0.0.1"
4
+ VERSION = "0.0.2"
5
5
  end
6
6
  end
7
7
  end
@@ -1,9 +1,2 @@
1
1
  require "capistrano-s3-copy/version"
2
-
3
- module Capistrano
4
- module S3
5
- module Copy
6
- # Your code goes here...
7
- end
8
- end
9
- end
2
+ require 'capistrano-s3-copy/recipes/s3_copy'
metadata CHANGED
@@ -2,7 +2,7 @@
2
2
  name: capistrano-s3-copy
3
3
  version: !ruby/object:Gem::Version
4
4
  prerelease:
5
- version: 0.0.1
5
+ version: 0.0.2
6
6
  platform: ruby
7
7
  authors:
8
8
  - Richie McMahon
@@ -10,7 +10,7 @@ autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
12
 
13
- date: 2012-04-21 00:00:00 Z
13
+ date: 2012-04-28 00:00:00 Z
14
14
  dependencies:
15
15
  - !ruby/object:Gem::Dependency
16
16
  name: capistrano
@@ -23,8 +23,19 @@ dependencies:
23
23
  version: 2.12.0
24
24
  type: :runtime
25
25
  version_requirements: *id001
26
+ - !ruby/object:Gem::Dependency
27
+ name: frahugo-s3sync
28
+ prerelease: false
29
+ requirement: &id002 !ruby/object:Gem::Requirement
30
+ none: false
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: "0"
35
+ type: :runtime
36
+ version_requirements: *id002
26
37
  description: |-
27
- Capistrano deployment strategy that creates and pushes a tarball
38
+ Capistrano deployment strategy that creates and pushes a tarball
28
39
  into S3, for both pushed deployments and pulled auto-scaling.
29
40
  email:
30
41
  - richie.mcmahon@gmail.com
@@ -37,12 +48,15 @@ extra_rdoc_files: []
37
48
  files:
38
49
  - .gitignore
39
50
  - Gemfile
51
+ - Gemfile.lock
40
52
  - LICENSE
41
53
  - README.md
42
54
  - Rakefile
43
55
  - capistrano-s3-copy.gemspec
44
56
  - lib/capistrano-s3-copy.rb
57
+ - lib/capistrano-s3-copy/recipes/s3_copy.rb
45
58
  - lib/capistrano-s3-copy/version.rb
59
+ - lib/capistrano/recipes/deploy/strategy/aws_install.sh.erb
46
60
  - lib/capistrano/recipes/deploy/strategy/s3_copy.rb
47
61
  homepage: http://github.com/richie/capistrano-s3-copy
48
62
  licenses: []