matross 0.2.6 → 0.2.7

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,15 +1,15 @@
1
1
  ---
2
2
  !binary "U0hBMQ==":
3
3
  metadata.gz: !binary |-
4
- MGJkNDFkNjgyNTMwMjVhZWE4ZWUzZGQzYjg2YjY0MzI2ODJhNzZlMg==
4
+ NzRhOTg3OGUwZDgyZGU2OTQ4ZWYwMjU4ZGY1YTljM2JiYzgwN2IyMA==
5
5
  data.tar.gz: !binary |-
6
- MDFhYTAwZmQxYTgxMDg2NGU4Njc5MTA5MzNjMjAyYTczYmJkMTEwNA==
6
+ NjRlYjllYzc3ODZlNjBmMmIyZjQxMGM1NDAxMjUyY2I0YTJlZjQ1MA==
7
7
  SHA512:
8
8
  metadata.gz: !binary |-
9
- ZWFiMDdjMWY4MWFkMGZjYzhiMzI5MDMyYTZkNzNiNTg4Mjk3ZTRhM2Y0NjZi
10
- NjZiMjIwZWRjNzRjMDFmM2ZkOGJmODAzMWFiN2JkNWNjZDAyNmM2NjEzY2Ux
11
- ODU5MGI3ZDkzNzRmZjllNDgzN2UyNTA0YTUyOTJhZmFmNDlkOGM=
9
+ NjhjMDg0NjFiYjk2YWRlMjRlMGIyODE1NmNmZGM1MjFkZWZjYjI2MDUxMDNl
10
+ NWFiY2E1MDY3ZGJiNWQxMWQzMTI5MDFiYmM4Y2FmZGNhNzRmYTUwZDZkNjcw
11
+ NjQ5NGM1OWU5MDM0NTQzZDc5NDRmMGVhN2M3NGUwMjlmN2MwZjA=
12
12
  data.tar.gz: !binary |-
13
- MWNkNmFiYzE1MDYyNWZhMDQxNTJhZmI0MzA2OGZlNjMxOWNhMWJlOTUyMzI1
14
- MTM1NzliYzA0NzQ3ZGZlNDJlY2ZjOGRiZTNkYmFkNDgzMjM4ODcwZWE3Nzli
15
- MzUyM2E0M2M5NDM5YTdhNWY5MDJhNjQwY2RkYWViODE1M2E4ZWU=
13
+ ZTk0ZDY5ZDE2NjYwYzBkODBhMTQ5N2UwZDJmNmMwNzNiZjc3OGJkMTBlOWMz
14
+ MTIxYjZiZTNlNWY1NWRlNzdlOWZlNTUyMjk4YjFjMmU1MjZiNmRhMmI5NzA1
15
+ MTgzYjBkOTBiMzUzMDczNTQxM2YxMmJhZGMxMjM3OTk1NmUzNDA=
data/README.md CHANGED
@@ -1,12 +1,17 @@
1
1
  # matross
2
2
 
3
+ Matross is our collection of opinionated Capistrano recipes. We made a bunch of additions and customizations. Below we list the most relevant ones.
4
+
5
+ * **Foreman by default**: we use [`foreman`](http://ddollar.github.io/foreman/) to environment variables, init scripts, task definitions and more.
6
+ * **Custom foreman upstart template**: we also leverage `foreman`'s templates to build a custom upstart template that enables `console log`, allowing `logrotate` to work properly.
7
+
3
8
  ## Usage
4
9
 
5
- Put matross in the `:development` group of your `Gemfile`:
10
+ Put `matross` in the `:development` group of your `Gemfile`:
6
11
 
7
12
  ```ruby
8
13
  group :development do
9
- gem 'matross', :git => 'git://github.com/innvent/matross.git'
14
+ gem 'matross'
10
15
  end
11
16
  ```
12
17
 
@@ -16,32 +21,327 @@ Run `bundle exec capify .` in the project root folder:
16
21
  $ bundle exec capify .
17
22
  ```
18
23
 
24
+ Find a full example [down this `README`](#full-example).
25
+
26
+ ### Overriding default templates
27
+
28
+ We have our opinions, but don't know everything. What works for us, may not fit your needs since each app is a unique snowflake. To take care of that, `matross` allows you to define your own templates instead of the built in ones. Look at the included ones in `lib/matross/templates` to see how we think things should go.
29
+
30
+ ### Managing application daemons with Foreman
31
+
32
+ Foreman has freed us of the tedious task of writing `init` and Upstart scripts. Some of our `matross` recipes automatically add processes - such as the `unicorn` server - to the `Procfile`.
33
+
34
+ If you have an application Procfile with custom daemons defined, such as Rake task, they will be concatenated with all the processes defined in `matross`, resulting in one final `Procfile-matross` file that will be used to start your application and export init scrips.
35
+
36
+ You can specify the number of each instance defined in Procfile-matross using the `foreman_procs` variable.
37
+ Suppose you have a process called `dj` and want to export 3 instances of it:
38
+
39
+ ```ruby
40
+ set :foreman_procs, {
41
+ dj: 3
42
+ }
43
+ ```
44
+
45
+ We also modified the default upstart template to log through upstart instead of just piping stdout and stderr into files. Goodbye nocturnal logexplosion. (Like all templates you can override it!).
46
+
47
+ If you have custom tasks that should also be started, simply list them in the `Procfile` in the root of your application. They will be appended to the recipe's task definitions (eg.: `unicorn`).
48
+
49
+ ```
50
+ custom_task: bundle exec rake custom_task
51
+ ```
52
+
53
+ If there are any environment variables that you want to use, just set them in a `.env` file in the root of your application. Please note that `RAILS_ENV` is properly set during `foreman` tasks.
54
+
55
+ ```
56
+ CUSTOM_TASK_ENV=boost
57
+ ```
58
+
59
+ ## Recipes
60
+
61
+ ### Foreman
62
+
63
+ Requires having [`foreman`](http://rubygems.org/gems/foreman) available in the application. As mentioned before, we use `foreman` in production to save us from generating upstart init scripts. As a bonus we get sane definition of environment variables.
64
+
65
+ Overwritable template: [`process.conf.erb`](lib/matross/templates/foreman/process.conf.erb)
66
+
67
+ > Variables
68
+
69
+ | Variable | Default value | Description |
70
+ | --- | --- | --- |
71
+ | `:foreman_user` | `{ user }` - The user defined in Capistrano | The user which should run the tasks defined in the `Procfile` |
72
+ | `:foreman_bin` | `'bundle exec foreman'` | The `foreman` command |
73
+ | `:foreman_procs` | `{}` - Defaults to one per task definition | Number of processes for each task definition in the `Procfile` |
74
+
75
+ > Tasks
76
+
77
+ | Task | Description |
78
+ | --- | --- |
79
+ | `foreman:pre_setup` | Creates the `upstart` folder in the `shared_path` |
80
+ | `foreman:setup` | Merges all partial `Procfile`s and `.env`s, including the appropriate `RAILS_ENV` |
81
+ | `foreman:export` | Export the task definitions as Upstart scripts |
82
+ | `foreman:symlink` | Symlink `.env-matross` and `Procfile-matross` to `current_path` |
83
+ | `foreman:log` | Symlink Upstart logs to the log folder in `shared_path` |
84
+ | `foreman:stop` | Stop all of the application tasks |
85
+ | `foreman:restart` | Restart or start all of the application tasks |
86
+ | `foreman:remove` | Remove all of the application tasks from Upstart |
87
+
88
+ ### Unicorn
89
+
90
+ Requires having [`unicorn`](http://unicorn.bogomips.org/index.html) available in the application. By loading our `unicorn` recipe, you get [our default configuration](lib/matross/templates/unicorn/unicorn.rb.erb).
91
+
92
+ Overwritable template: [`unicorn.rb.erb`](lib/matross/templates/unicorn/unicorn.rb.erb)
93
+ Procfile task: `web: bundle exec unicorn -c <%= unicorn_config %> -E <%= rails_env %>`
94
+
95
+ > Variables
96
+
97
+ | Variable | Default value | Description |
98
+ | --- | --- | --- |
99
+ | `:unicorn_config` | `"#{shared_path}/config/unicorn.rb"` | Location of the configuration file |
100
+ | `:unicorn_log` | `"#{shared_path}/log/unicorn.log"` | Location of unicorn log |
101
+ | `:unicorn_workers` | `1` | Number of unicorn workers |
102
+
103
+ > Tasks
104
+
105
+ | Task | Description |
106
+ | --- | --- |
107
+ | `unicorn:setup` | Creates the `unicorn.rb` configuration file in the `shared_path` |
108
+ | `unicorn:procfile` | Defines how `unicorn` should be run in a temporary `Procfile` |
109
+
110
+
111
+ ### Nginx
112
+
113
+ This recipes creates and configures the virtual_host for the application. [This virtual host] has some sane defaults, suitable for most of our deployments (non-SSL). The file is created at `/etc/nginx/sites-available` and symlinked to `/etc/nginx/sites-enabled`. These are the defaults for the Nginx installation in Ubuntu. You can take a look at [our general `nginx.conf`](https://github.com/innvent/parcelles/blob/puppet/puppet/modules/nginx/files/nginx.conf).
114
+
115
+ > Variables
116
+
117
+ | Variable | Default value | Description |
118
+ | --- | --- | --- |
119
+ | `:htpasswd` | None | `htpasswd` user:password format |
120
+
121
+ > Tasks
122
+
123
+ | Task | Description |
124
+ | --- | --- |
125
+ | `nginx:setup` | Creates the virtual host file |
126
+ | `nginx:reload` | Reloads the Nginx configuration |
127
+ | `nginx:lock` | Sets up the a basic http auth on the virtual host |
128
+ | `nginx:unlock` | Removes the basic http auth |
129
+
130
+
131
+ ### MySQL
132
+
133
+ Requires having [`mysql2`](http://rubygems.org/gems/mysql2) available in the application. In our MySQL recipe we dynamically generate a `database.yml` based on the variables that should be set globally or per-stage.
134
+
135
+ Overwritable template: [`database.yml.erb`](lib/matross/templates/mysql/database.yml.erb)
136
+
137
+ > Variables
138
+
139
+ | Variable | Default value | Description |
140
+ | --- | --- | --- |
141
+ | `:database_config` | `"#{shared_path}/config/database.yml"` | Location of the configuration file |
142
+ | `:mysql_host` | None | MySQL host address |
143
+ | `:mysql_database` | None | MySQL database name. We automatically substitute dashes `-` for underscores `_` |
144
+ | `:mysql_user` | None | MySQL user |
145
+ | `:mysql_passwd` | None | MySQL password |
146
+
147
+ > Tasks
148
+
149
+ | Task | Description |
150
+ | --- | --- |
151
+ | `mysql:setup` | Creates the `database.yml` in the `shared_path` |
152
+ | `mysql:symlink` | Creates a symlink for the `database.yml` file in the `current_path` |
153
+ | `mysql:create` | Creates the database if it hasn't been created |
154
+ | `mysql:schema_load` | Loads the schema if there are no tables in the DB |
155
+
156
+ ## Mongoid
157
+
158
+ Requires having [`mongoid`](http://rubygems.org/gems/mongoid) available in the application. In our Mongoid recipe we dynamically generate a `mongoid.yml` based on the variables that should be set globally or per-stage.
159
+
160
+ Overwritable template: [`mongoid.yml.erb`](lib/matross/templates/mongoid/mongoid.yml.erb)
161
+
162
+ > Variables
163
+
164
+ | Variable | Default value | Description |
165
+ | --- | --- | --- |
166
+ | `:mongoid_config` | `"#{shared_path}/config/mongoid.yml"` | Location of the mongoid configuration file |
167
+ | `:mongo_hosts` | None | **List** of MongoDB hosts |
168
+ | `:mongo_database` | None | MongoDB database name |
169
+ | `:mongo_user` | None | MongoDB user |
170
+ | `:mongo_passwd` | None | MongoDB password |
171
+
172
+ > Tasks
173
+
174
+ | Task | Description |
175
+ | --- | --- |
176
+ | `mongoid:setup` | Creates the `mongoid.yml` in the `shared_path` |
177
+ | `mongoid:symlink` | Creates a symlink for the `mongoid.yml` file in the `current_path` |
178
+
179
+ ### Delayed Job
180
+
181
+ Requires having [`delayed_job`](http://rubygems.org/gems/delayed_job) available in the application.
182
+
183
+ Procfile task: `dj: bundle exec rake jobs:work` or `dj_<%= queue_name %>: bundle exec rake jobs:work QUEUE=<%= queue_name %>`
184
+
185
+ > Variables
186
+
187
+ | Variable | Default value | Description |
188
+ | --- | --- | --- |
189
+ | `:dj_queues` | None | List of queues |
190
+
191
+
192
+ > Tasks
193
+
194
+ | Task | Description |
195
+ | --- | --- |
196
+ | `delayed_job:procfile` | Defines how `delayed_job` should be run in a temporary `Procfile` |
197
+
198
+
199
+ ### Fog (AWS)
200
+
201
+ Requires having [`fog`](http://rubygems.org/gems/fog) available in the application. When we use `fog`, it is for interacting with Amazon services, once again very opinionated.
202
+
203
+ Overwritable template: [`fog_config.yml.erb`](lib/matross/templates/fog/fog_config.yml.erb)
204
+
205
+ The configuration generated may be used by other gems, such as [`carrierwave`](http://rubygems.org/gems/carrierwave). Here is an example of how we use it:
206
+
207
+ ```ruby
208
+ # config/initializers/carrierwave.rb
209
+ CarrierWave.configure do |config|
210
+ fog_config = YAML.load(File.read(File.join(Rails.root, 'config', 'fog_config.yml')))
211
+ config.fog_credentials = {
212
+ :provider => 'AWS',
213
+ :aws_access_key_id => fog_config['aws_access_key_id'],
214
+ :aws_secret_access_key => fog_config['aws_secret_access_key'],
215
+ :region => fog_config['region']
216
+ }
217
+ config.fog_directory = fog_config['directory']
218
+ config.fog_public = fog_config['public']
219
+ end
220
+ ```
221
+
222
+ > Variables
223
+
224
+ | Variable | Default value | Description |
225
+ | --- | --- | --- |
226
+ | `:fog_config` | `"#{shared_path}/config/fog_config.yml"` | Location of the fog configuration file |
227
+ | `:fog_region` | `'us-east-1'` | AWS Region |
228
+ | `:fog_public` | `false` | Bucket policy |
229
+ | `:fog_aws_access_key_id` | None | AWS Access Key Id |
230
+ | `:fog_aws_secret_access_key` | None | AWS Secret Access Key |
231
+
232
+ > Tasks
19
233
 
20
- ## What's inside?
234
+ | Task | Description |
235
+ | --- | --- |
236
+ | `fog:setup` | Creates the `fog_config.yml` in the `shared_path` |
237
+ | `fog:symlink` | Creates a symlink for the `fog_config.yml` file in the `current_path` |
21
238
 
22
- We made a bunch of additions and customizations. Below we list the most relevant ones.
239
+ ### Faye
23
240
 
24
- * **Foreman by default**:
25
- * **Custom foreman upstart template**: we use a custom upstart template, that enables `console log`, allowing `logrotate` to work properly.
241
+ Requires having [`faye`](http://rubygems.org/gems/faye) available in the application.
26
242
 
27
- ## Overriding default templates
28
- We have our opinions, but don't know everything. What works for us, may not fit your needs since each app is a unique snowflake. To take care of that `matross` allows you to define your own templates to use instead of the built in ones. Look at the included ones in `lib/matross/templates` to see how we think things should go.
243
+ Overwritable templates: [`faye.ru.erb`](lib/matross/templates/faye/faye.ru.erb) and [`faye_server.yml`](lib/matross/templates/faye/faye_server.yml)
244
+ Procfile task: `faye: bundle exec rackup <%= faye_ru %> -s thin -E <%= rails_env %> -p <%= faye_port %>`
29
245
 
30
- For example to override the default `lib/matross/templates/unicorn.rb.erb` simply put your template in your applications `config/matross/unicorn/unicorn.rb.erb`. The general idea is `config/matross/#{recipe_name}/#{erb_template}`
246
+ > Variables
31
247
 
32
- ## Managing application daemons with Foreman
248
+ | Variable | Default value | Description |
249
+ | --- | --- | --- |
250
+ | `:faye_config` | `"#{shared_path}/config/faye_config.yml"` | Location of the `faye` parameters configuration file |
251
+ | `:faye_ru` | `"#{shared_path}/config/faye.ru"` | Location of the `faye` configuration file |
252
+ | `:faye_port` | None | Which port `faye` should listen on |
33
253
 
34
- Foreman has freed us of the tedious task of writing `init` and `upstart` scripts. Some of our `matross` recipes automatically add processes - such as the `unicorn` server - to the Procfile.
254
+ > Tasks
35
255
 
36
- If you have an application Procfile with custom daemons defined, such as `delayed_job`, they will be concantenated with all the processes defined in `matross`, resulting in one final Procfile-matross file that will be used to start your app and export init scrips.
256
+ | Task | Description |
257
+ | --- | --- |
258
+ | `faye:setup` | Creates `faye_config.yml` and `faye.ru` in the `shared_path` |
259
+ | `faye:symlink` | Creates a symlink for the `faye_config.yml` file in the `current_path` |
37
260
 
38
- You can specify the number of each instance defined in Procfile-matross using the `foreman_procs` capistrano variable.
39
- Supose you have a process called `dj` and want to export 3 instances of it:
261
+
262
+ ### Local Assets
263
+
264
+ This recipe overwrites the default assets precompilation by compiling them locally and then uploading the result to the server.
265
+
266
+ ## Full Example
267
+
268
+ Below is a full example of how to use `matross` exhaustively. **Do note** that this would be an edge case as, for example, you don't normally run `mongoid` with `mysql`.
269
+
270
+ > `config/deploy.rb`
40
271
 
41
272
  ```ruby
42
- set :foreman_procs, {
43
- dj: 3
273
+ set :stages, %w(production staging)
274
+ set :default_stage, 'staging'
275
+ require 'capistrano/ext/multistage'
276
+ require 'bundler/capistrano'
277
+ require 'matross'
278
+ load 'matross/local_assets'
279
+ load 'matross/nginx'
280
+ load 'matross/unicorn'
281
+ load 'matross/faye'
282
+ load 'matross/delayed_job'
283
+ load 'matross/fog'
284
+ load 'matross/mongoid'
285
+ load 'matross/mysql'
286
+ load 'matross/foreman'
287
+
288
+ set :application, 'awesome_application'
289
+ set :repository, 'git@github.com:innvent/awesome_application.git'
290
+ set :ssh_options, { :forward_agent => true }
291
+ set :scm, :git
292
+ set :scm_verbose, true
293
+ set :deploy_via, :remote_cache
294
+ set :shared_children, %w(public/system log tmp/pids public/uploads)
295
+ default_run_options[:pty] = true
296
+
297
+ logger.level = Capistrano::Logger::DEBUG
298
+
299
+ after 'deploy:update', 'deploy:cleanup'
300
+ ```
301
+
302
+ > `config/deploy/production.rb`
303
+
304
+ ```ruby
305
+ set :user, 'ubuntu'
306
+ set :group, 'ubuntu'
307
+ set :use_sudo, false
308
+ set :branch, 'master'
309
+ set :rails_env, 'production'
310
+ set :deploy_to, "/home/#{user}/#{application}"
311
+ set :server_name, 'example.com'
312
+
313
+ set :mongo_hosts, [ 'localhost' ]
314
+ set :mongo_database, "#{application}_#{rails_env}"
315
+
316
+ set :mysql_host, 'localhost'
317
+ set :mysql_database, "#{application}_#{rails_env}"
318
+ set :mysql_user, "#{user}"
319
+ set :mysql_passwd, ''
320
+
321
+ set :faye_port, '9292'
322
+ set :faye_local, true
323
+
324
+ set :htpasswd, 'admin:$apr1$twOdKBdh$okL.giy91y9LzXsD5swUb0'
325
+
326
+ set :dj_queues, [ 'queue1', 'queue2' ]
327
+
328
+ set :fog_aws_access_key_id, 'AKIACS5E2GU9NND0NMD4'
329
+ set :fog_aws_secret_access_key, 'rNB38h5Y4ysUM3r10F3oehrnp2ZaBcUPtiOnJyLn'
330
+ set :fog_directory, 'awesome_application_production'
331
+
332
+ set :foreman_procs, { 'dj_queue1' => 2, 'dj_queue2' => 3 }
333
+
334
+ server '192.168.1.1', :app, :web, :dj, :faye, :db, :primary => true
335
+
336
+ set :default_environment, {
337
+ 'PATH' => "/home/#{user}/.rbenv/shims:/home/#{user}/.rbenv/bin:$PATH"
44
338
  }
45
339
  ```
46
340
 
47
- We also modified the default upstart template to log through upstart instead of just piping stdout and stderr into files. Goodbye nocturnal logexplosion. (Like all templates you can override it)
341
+ > `Capfile`
342
+
343
+ ```ruby
344
+ load 'deploy'
345
+ load 'deploy/assets'
346
+ load 'config/deploy'
347
+ ```
@@ -5,7 +5,6 @@ namespace :nginx do
5
5
  template "nginx/nginx_virtual_host_conf.erb", "/tmp/#{application}"
6
6
  run "#{sudo} mv /tmp/#{application} /etc/nginx/sites-available/#{application}"
7
7
  run "#{sudo} ln -fs /etc/nginx/sites-available/#{application} /etc/nginx/sites-enabled/#{application}"
8
- run "mkdir -p #{shared_path}/sockets"
9
8
  end
10
9
  after "deploy:setup", "nginx:setup"
11
10
 
@@ -1,5 +1,5 @@
1
1
  upstream <%= application %> {
2
- server unix:/tmp/unicorn.sock fail_timeout=0;
2
+ server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0;
3
3
  }
4
4
 
5
5
  server {
@@ -3,7 +3,7 @@ app_path = "<%= current_path %>"
3
3
  working_directory "#{app_path}"
4
4
  pid "#{app_path}/tmp/pids/unicorn.pid"
5
5
 
6
- listen "/tmp/unicorn.sock"
6
+ listen "/tmp/unicorn.<%= application %>.sock"
7
7
  worker_processes <%= unicorn_workers %>
8
8
  timeout 60
9
9
  preload_app true
@@ -1,3 +1,3 @@
1
1
  module Matross
2
- VERSION = "0.2.6"
2
+ VERSION = "0.2.7"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: matross
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.6
4
+ version: 0.2.7
5
5
  platform: ruby
6
6
  authors:
7
7
  - Artur Rodrigues
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2013-11-05 00:00:00.000000000 Z
12
+ date: 2013-11-18 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: bundler
@@ -110,7 +110,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
110
110
  version: '0'
111
111
  requirements: []
112
112
  rubyforge_project:
113
- rubygems_version: 2.1.10
113
+ rubygems_version: 2.1.11
114
114
  signing_key:
115
115
  specification_version: 4
116
116
  summary: Our collection of opnionated Capistrano recipes