matross 0.2.6 → 0.2.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,15 +1,15 @@
1
1
  ---
2
2
  !binary "U0hBMQ==":
3
3
  metadata.gz: !binary |-
4
- MGJkNDFkNjgyNTMwMjVhZWE4ZWUzZGQzYjg2YjY0MzI2ODJhNzZlMg==
4
+ NzRhOTg3OGUwZDgyZGU2OTQ4ZWYwMjU4ZGY1YTljM2JiYzgwN2IyMA==
5
5
  data.tar.gz: !binary |-
6
- MDFhYTAwZmQxYTgxMDg2NGU4Njc5MTA5MzNjMjAyYTczYmJkMTEwNA==
6
+ NjRlYjllYzc3ODZlNjBmMmIyZjQxMGM1NDAxMjUyY2I0YTJlZjQ1MA==
7
7
  SHA512:
8
8
  metadata.gz: !binary |-
9
- ZWFiMDdjMWY4MWFkMGZjYzhiMzI5MDMyYTZkNzNiNTg4Mjk3ZTRhM2Y0NjZi
10
- NjZiMjIwZWRjNzRjMDFmM2ZkOGJmODAzMWFiN2JkNWNjZDAyNmM2NjEzY2Ux
11
- ODU5MGI3ZDkzNzRmZjllNDgzN2UyNTA0YTUyOTJhZmFmNDlkOGM=
9
+ NjhjMDg0NjFiYjk2YWRlMjRlMGIyODE1NmNmZGM1MjFkZWZjYjI2MDUxMDNl
10
+ NWFiY2E1MDY3ZGJiNWQxMWQzMTI5MDFiYmM4Y2FmZGNhNzRmYTUwZDZkNjcw
11
+ NjQ5NGM1OWU5MDM0NTQzZDc5NDRmMGVhN2M3NGUwMjlmN2MwZjA=
12
12
  data.tar.gz: !binary |-
13
- MWNkNmFiYzE1MDYyNWZhMDQxNTJhZmI0MzA2OGZlNjMxOWNhMWJlOTUyMzI1
14
- MTM1NzliYzA0NzQ3ZGZlNDJlY2ZjOGRiZTNkYmFkNDgzMjM4ODcwZWE3Nzli
15
- MzUyM2E0M2M5NDM5YTdhNWY5MDJhNjQwY2RkYWViODE1M2E4ZWU=
13
+ ZTk0ZDY5ZDE2NjYwYzBkODBhMTQ5N2UwZDJmNmMwNzNiZjc3OGJkMTBlOWMz
14
+ MTIxYjZiZTNlNWY1NWRlNzdlOWZlNTUyMjk4YjFjMmU1MjZiNmRhMmI5NzA1
15
+ MTgzYjBkOTBiMzUzMDczNTQxM2YxMmJhZGMxMjM3OTk1NmUzNDA=
data/README.md CHANGED
@@ -1,12 +1,17 @@
1
1
  # matross
2
2
 
3
+ Matross is our collection of opinionated Capistrano recipes. We made a bunch of additions and customizations. Below we list the most relevant ones.
4
+
5
+ * **Foreman by default**: we use [`foreman`](http://ddollar.github.io/foreman/) to environment variables, init scripts, task definitions and more.
6
+ * **Custom foreman upstart template**: we also leverage `foreman`'s templates to build a custom upstart template that enables `console log`, allowing `logrotate` to work properly.
7
+
3
8
  ## Usage
4
9
 
5
- Put matross in the `:development` group of your `Gemfile`:
10
+ Put `matross` in the `:development` group of your `Gemfile`:
6
11
 
7
12
  ```ruby
8
13
  group :development do
9
- gem 'matross', :git => 'git://github.com/innvent/matross.git'
14
+ gem 'matross'
10
15
  end
11
16
  ```
12
17
 
@@ -16,32 +21,327 @@ Run `bundle exec capify .` in the project root folder:
16
21
  $ bundle exec capify .
17
22
  ```
18
23
 
24
+ Find a full example [down this `README`](#full-example).
25
+
26
+ ### Overriding default templates
27
+
28
+ We have our opinions, but don't know everything. What works for us, may not fit your needs since each app is a unique snowflake. To take care of that, `matross` allows you to define your own templates instead of the built in ones. Look at the included ones in `lib/matross/templates` to see how we think things should go.
29
+
30
+ ### Managing application daemons with Foreman
31
+
32
+ Foreman has freed us of the tedious task of writing `init` and Upstart scripts. Some of our `matross` recipes automatically add processes - such as the `unicorn` server - to the `Procfile`.
33
+
34
+ If you have an application Procfile with custom daemons defined, such as Rake task, they will be concatenated with all the processes defined in `matross`, resulting in one final `Procfile-matross` file that will be used to start your application and export init scrips.
35
+
36
+ You can specify the number of each instance defined in Procfile-matross using the `foreman_procs` variable.
37
+ Suppose you have a process called `dj` and want to export 3 instances of it:
38
+
39
+ ```ruby
40
+ set :foreman_procs, {
41
+ dj: 3
42
+ }
43
+ ```
44
+
45
+ We also modified the default upstart template to log through upstart instead of just piping stdout and stderr into files. Goodbye nocturnal logexplosion. (Like all templates you can override it!).
46
+
47
+ If you have custom tasks that should also be started, simply list them in the `Procfile` in the root of your application. They will be appended to the recipe's task definitions (eg.: `unicorn`).
48
+
49
+ ```
50
+ custom_task: bundle exec rake custom_task
51
+ ```
52
+
53
+ If there are any environment variables that you want to use, just set them in a `.env` file in the root of your application. Please note that `RAILS_ENV` is properly set during `foreman` tasks.
54
+
55
+ ```
56
+ CUSTOM_TASK_ENV=boost
57
+ ```
58
+
59
+ ## Recipes
60
+
61
+ ### Foreman
62
+
63
+ Requires having [`foreman`](http://rubygems.org/gems/foreman) available in the application. As mentioned before, we use `foreman` in production to save us from generating upstart init scripts. As a bonus we get sane definition of environment variables.
64
+
65
+ Overwritable template: [`process.conf.erb`](lib/matross/templates/foreman/process.conf.erb)
66
+
67
+ > Variables
68
+
69
+ | Variable | Default value | Description |
70
+ | --- | --- | --- |
71
+ | `:foreman_user` | `{ user }` - The user defined in Capistrano | The user which should run the tasks defined in the `Procfile` |
72
+ | `:foreman_bin` | `'bundle exec foreman'` | The `foreman` command |
73
+ | `:foreman_procs` | `{}` - Defaults to one per task definition | Number of processes for each task definition in the `Procfile` |
74
+
75
+ > Tasks
76
+
77
+ | Task | Description |
78
+ | --- | --- |
79
+ | `foreman:pre_setup` | Creates the `upstart` folder in the `shared_path` |
80
+ | `foreman:setup` | Merges all partial `Procfile`s and `.env`s, including the appropriate `RAILS_ENV` |
81
+ | `foreman:export` | Export the task definitions as Upstart scripts |
82
+ | `foreman:symlink` | Symlink `.env-matross` and `Procfile-matross` to `current_path` |
83
+ | `foreman:log` | Symlink Upstart logs to the log folder in `shared_path` |
84
+ | `foreman:stop` | Stop all of the application tasks |
85
+ | `foreman:restart` | Restart or start all of the application tasks |
86
+ | `foreman:remove` | Remove all of the application tasks from Upstart |
87
+
88
+ ### Unicorn
89
+
90
+ Requires having [`unicorn`](http://unicorn.bogomips.org/index.html) available in the application. By loading our `unicorn` recipe, you get [our default configuration](lib/matross/templates/unicorn/unicorn.rb.erb).
91
+
92
+ Overwritable template: [`unicorn.rb.erb`](lib/matross/templates/unicorn/unicorn.rb.erb)
93
+ Procfile task: `web: bundle exec unicorn -c <%= unicorn_config %> -E <%= rails_env %>`
94
+
95
+ > Variables
96
+
97
+ | Variable | Default value | Description |
98
+ | --- | --- | --- |
99
+ | `:unicorn_config` | `"#{shared_path}/config/unicorn.rb"` | Location of the configuration file |
100
+ | `:unicorn_log` | `"#{shared_path}/log/unicorn.log"` | Location of unicorn log |
101
+ | `:unicorn_workers` | `1` | Number of unicorn workers |
102
+
103
+ > Tasks
104
+
105
+ | Task | Description |
106
+ | --- | --- |
107
+ | `unicorn:setup` | Creates the `unicorn.rb` configuration file in the `shared_path` |
108
+ | `unicorn:procfile` | Defines how `unicorn` should be run in a temporary `Procfile` |
109
+
110
+
111
+ ### Nginx
112
+
113
+ This recipes creates and configures the virtual_host for the application. [This virtual host] has some sane defaults, suitable for most of our deployments (non-SSL). The file is created at `/etc/nginx/sites-available` and symlinked to `/etc/nginx/sites-enabled`. These are the defaults for the Nginx installation in Ubuntu. You can take a look at [our general `nginx.conf`](https://github.com/innvent/parcelles/blob/puppet/puppet/modules/nginx/files/nginx.conf).
114
+
115
+ > Variables
116
+
117
+ | Variable | Default value | Description |
118
+ | --- | --- | --- |
119
+ | `:htpasswd` | None | `htpasswd` user:password format |
120
+
121
+ > Tasks
122
+
123
+ | Task | Description |
124
+ | --- | --- |
125
+ | `nginx:setup` | Creates the virtual host file |
126
+ | `nginx:reload` | Reloads the Nginx configuration |
127
+ | `nginx:lock` | Sets up the a basic http auth on the virtual host |
128
+ | `nginx:unlock` | Removes the basic http auth |
129
+
130
+
131
+ ### MySQL
132
+
133
+ Requires having [`mysql2`](http://rubygems.org/gems/mysql2) available in the application. In our MySQL recipe we dynamically generate a `database.yml` based on the variables that should be set globally or per-stage.
134
+
135
+ Overwritable template: [`database.yml.erb`](lib/matross/templates/mysql/database.yml.erb)
136
+
137
+ > Variables
138
+
139
+ | Variable | Default value | Description |
140
+ | --- | --- | --- |
141
+ | `:database_config` | `"#{shared_path}/config/database.yml"` | Location of the configuration file |
142
+ | `:mysql_host` | None | MySQL host address |
143
+ | `:mysql_database` | None | MySQL database name. We automatically substitute dashes `-` for underscores `_` |
144
+ | `:mysql_user` | None | MySQL user |
145
+ | `:mysql_passwd` | None | MySQL password |
146
+
147
+ > Tasks
148
+
149
+ | Task | Description |
150
+ | --- | --- |
151
+ | `mysql:setup` | Creates the `database.yml` in the `shared_path` |
152
+ | `mysql:symlink` | Creates a symlink for the `database.yml` file in the `current_path` |
153
+ | `mysql:create` | Creates the database if it hasn't been created |
154
+ | `mysql:schema_load` | Loads the schema if there are no tables in the DB |
155
+
156
+ ## Mongoid
157
+
158
+ Requires having [`mongoid`](http://rubygems.org/gems/mongoid) available in the application. In our Mongoid recipe we dynamically generate a `mongoid.yml` based on the variables that should be set globally or per-stage.
159
+
160
+ Overwritable template: [`mongoid.yml.erb`](lib/matross/templates/mongoid/mongoid.yml.erb)
161
+
162
+ > Variables
163
+
164
+ | Variable | Default value | Description |
165
+ | --- | --- | --- |
166
+ | `:mongoid_config` | `"#{shared_path}/config/mongoid.yml"` | Location of the mongoid configuration file |
167
+ | `:mongo_hosts` | None | **List** of MongoDB hosts |
168
+ | `:mongo_database` | None | MongoDB database name |
169
+ | `:mongo_user` | None | MongoDB user |
170
+ | `:mongo_passwd` | None | MongoDB password |
171
+
172
+ > Tasks
173
+
174
+ | Task | Description |
175
+ | --- | --- |
176
+ | `mongoid:setup` | Creates the `mongoid.yml` in the `shared_path` |
177
+ | `mongoid:symlink` | Creates a symlink for the `mongoid.yml` file in the `current_path` |
178
+
179
+ ### Delayed Job
180
+
181
+ Requires having [`delayed_job`](http://rubygems.org/gems/delayed_job) available in the application.
182
+
183
+ Procfile task: `dj: bundle exec rake jobs:work` or `dj_<%= queue_name %>: bundle exec rake jobs:work QUEUE=<%= queue_name %>`
184
+
185
+ > Variables
186
+
187
+ | Variable | Default value | Description |
188
+ | --- | --- | --- |
189
+ | `:dj_queues` | None | List of queues |
190
+
191
+
192
+ > Tasks
193
+
194
+ | Task | Description |
195
+ | --- | --- |
196
+ | `delayed_job:procfile` | Defines how `delayed_job` should be run in a temporary `Procfile` |
197
+
198
+
199
+ ### Fog (AWS)
200
+
201
+ Requires having [`fog`](http://rubygems.org/gems/fog) available in the application. When we use `fog`, it is for interacting with Amazon services, once again very opinionated.
202
+
203
+ Overwritable template: [`fog_config.yml.erb`](lib/matross/templates/fog/fog_config.yml.erb)
204
+
205
+ The configuration generated may be used by other gems, such as [`carrierwave`](http://rubygems.org/gems/carrierwave). Here is an example of how we use it:
206
+
207
+ ```ruby
208
+ # config/initializers/carrierwave.rb
209
+ CarrierWave.configure do |config|
210
+ fog_config = YAML.load(File.read(File.join(Rails.root, 'config', 'fog_config.yml')))
211
+ config.fog_credentials = {
212
+ :provider => 'AWS',
213
+ :aws_access_key_id => fog_config['aws_access_key_id'],
214
+ :aws_secret_access_key => fog_config['aws_secret_access_key'],
215
+ :region => fog_config['region']
216
+ }
217
+ config.fog_directory = fog_config['directory']
218
+ config.fog_public = fog_config['public']
219
+ end
220
+ ```
221
+
222
+ > Variables
223
+
224
+ | Variable | Default value | Description |
225
+ | --- | --- | --- |
226
+ | `:fog_config` | `"#{shared_path}/config/fog_config.yml"` | Location of the fog configuration file |
227
+ | `:fog_region` | `'us-east-1'` | AWS Region |
228
+ | `:fog_public` | `false` | Bucket policy |
229
+ | `:fog_aws_access_key_id` | None | AWS Access Key Id |
230
+ | `:fog_aws_secret_access_key` | None | AWS Secret Access Key |
231
+
232
+ > Tasks
19
233
 
20
- ## What's inside?
234
+ | Task | Description |
235
+ | --- | --- |
236
+ | `fog:setup` | Creates the `fog_config.yml` in the `shared_path` |
237
+ | `fog:symlink` | Creates a symlink for the `fog_config.yml` file in the `current_path` |
21
238
 
22
- We made a bunch of additions and customizations. Below we list the most relevant ones.
239
+ ### Faye
23
240
 
24
- * **Foreman by default**:
25
- * **Custom foreman upstart template**: we use a custom upstart template, that enables `console log`, allowing `logrotate` to work properly.
241
+ Requires having [`faye`](http://rubygems.org/gems/faye) available in the application.
26
242
 
27
- ## Overriding default templates
28
- We have our opinions, but don't know everything. What works for us, may not fit your needs since each app is a unique snowflake. To take care of that `matross` allows you to define your own templates to use instead of the built in ones. Look at the included ones in `lib/matross/templates` to see how we think things should go.
243
+ Overwritable templates: [`faye.ru.erb`](lib/matross/templates/faye/faye.ru.erb) and [`faye_server.yml`](lib/matross/templates/faye/faye_server.yml)
244
+ Procfile task: `faye: bundle exec rackup <%= faye_ru %> -s thin -E <%= rails_env %> -p <%= faye_port %>`
29
245
 
30
- For example to override the default `lib/matross/templates/unicorn.rb.erb` simply put your template in your applications `config/matross/unicorn/unicorn.rb.erb`. The general idea is `config/matross/#{recipe_name}/#{erb_template}`
246
+ > Variables
31
247
 
32
- ## Managing application daemons with Foreman
248
+ | Variable | Default value | Description |
249
+ | --- | --- | --- |
250
+ | `:faye_config` | `"#{shared_path}/config/faye_config.yml"` | Location of the `faye` parameters configuration file |
251
+ | `:faye_ru` | `"#{shared_path}/config/faye.ru"` | Location of the `faye` configuration file |
252
+ | `:faye_port` | None | Which port `faye` should listen on |
33
253
 
34
- Foreman has freed us of the tedious task of writing `init` and `upstart` scripts. Some of our `matross` recipes automatically add processes - such as the `unicorn` server - to the Procfile.
254
+ > Tasks
35
255
 
36
- If you have an application Procfile with custom daemons defined, such as `delayed_job`, they will be concantenated with all the processes defined in `matross`, resulting in one final Procfile-matross file that will be used to start your app and export init scrips.
256
+ | Task | Description |
257
+ | --- | --- |
258
+ | `faye:setup` | Creates `faye_config.yml` and `faye.ru` in the `shared_path` |
259
+ | `faye:symlink` | Creates a symlink for the `faye_config.yml` file in the `current_path` |
37
260
 
38
- You can specify the number of each instance defined in Procfile-matross using the `foreman_procs` capistrano variable.
39
- Supose you have a process called `dj` and want to export 3 instances of it:
261
+
262
+ ### Local Assets
263
+
264
+ This recipe overwrites the default assets precompilation by compiling them locally and then uploading the result to the server.
265
+
266
+ ## Full Example
267
+
268
+ Below is a full example of how to use `matross` exhaustively. **Do note** that this would be an edge case as, for example, you don't normally run `mongoid` with `mysql`.
269
+
270
+ > `config/deploy.rb`
40
271
 
41
272
  ```ruby
42
- set :foreman_procs, {
43
- dj: 3
273
+ set :stages, %w(production staging)
274
+ set :default_stage, 'staging'
275
+ require 'capistrano/ext/multistage'
276
+ require 'bundler/capistrano'
277
+ require 'matross'
278
+ load 'matross/local_assets'
279
+ load 'matross/nginx'
280
+ load 'matross/unicorn'
281
+ load 'matross/faye'
282
+ load 'matross/delayed_job'
283
+ load 'matross/fog'
284
+ load 'matross/mongoid'
285
+ load 'matross/mysql'
286
+ load 'matross/foreman'
287
+
288
+ set :application, 'awesome_application'
289
+ set :repository, 'git@github.com:innvent/awesome_application.git'
290
+ set :ssh_options, { :forward_agent => true }
291
+ set :scm, :git
292
+ set :scm_verbose, true
293
+ set :deploy_via, :remote_cache
294
+ set :shared_children, %w(public/system log tmp/pids public/uploads)
295
+ default_run_options[:pty] = true
296
+
297
+ logger.level = Capistrano::Logger::DEBUG
298
+
299
+ after 'deploy:update', 'deploy:cleanup'
300
+ ```
301
+
302
+ > `config/deploy/production.rb`
303
+
304
+ ```ruby
305
+ set :user, 'ubuntu'
306
+ set :group, 'ubuntu'
307
+ set :use_sudo, false
308
+ set :branch, 'master'
309
+ set :rails_env, 'production'
310
+ set :deploy_to, "/home/#{user}/#{application}"
311
+ set :server_name, 'example.com'
312
+
313
+ set :mongo_hosts, [ 'localhost' ]
314
+ set :mongo_database, "#{application}_#{rails_env}"
315
+
316
+ set :mysql_host, 'localhost'
317
+ set :mysql_database, "#{application}_#{rails_env}"
318
+ set :mysql_user, "#{user}"
319
+ set :mysql_passwd, ''
320
+
321
+ set :faye_port, '9292'
322
+ set :faye_local, true
323
+
324
+ set :htpasswd, 'admin:$apr1$twOdKBdh$okL.giy91y9LzXsD5swUb0'
325
+
326
+ set :dj_queues, [ 'queue1', 'queue2' ]
327
+
328
+ set :fog_aws_access_key_id, 'AKIACS5E2GU9NND0NMD4'
329
+ set :fog_aws_secret_access_key, 'rNB38h5Y4ysUM3r10F3oehrnp2ZaBcUPtiOnJyLn'
330
+ set :fog_directory, 'awesome_application_production'
331
+
332
+ set :foreman_procs, { 'dj_queue1' => 2, 'dj_queue2' => 3 }
333
+
334
+ server '192.168.1.1', :app, :web, :dj, :faye, :db, :primary => true
335
+
336
+ set :default_environment, {
337
+ 'PATH' => "/home/#{user}/.rbenv/shims:/home/#{user}/.rbenv/bin:$PATH"
44
338
  }
45
339
  ```
46
340
 
47
- We also modified the default upstart template to log through upstart instead of just piping stdout and stderr into files. Goodbye nocturnal logexplosion. (Like all templates you can override it)
341
+ > `Capfile`
342
+
343
+ ```ruby
344
+ load 'deploy'
345
+ load 'deploy/assets'
346
+ load 'config/deploy'
347
+ ```
@@ -5,7 +5,6 @@ namespace :nginx do
5
5
  template "nginx/nginx_virtual_host_conf.erb", "/tmp/#{application}"
6
6
  run "#{sudo} mv /tmp/#{application} /etc/nginx/sites-available/#{application}"
7
7
  run "#{sudo} ln -fs /etc/nginx/sites-available/#{application} /etc/nginx/sites-enabled/#{application}"
8
- run "mkdir -p #{shared_path}/sockets"
9
8
  end
10
9
  after "deploy:setup", "nginx:setup"
11
10
 
@@ -1,5 +1,5 @@
1
1
  upstream <%= application %> {
2
- server unix:/tmp/unicorn.sock fail_timeout=0;
2
+ server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0;
3
3
  }
4
4
 
5
5
  server {
@@ -3,7 +3,7 @@ app_path = "<%= current_path %>"
3
3
  working_directory "#{app_path}"
4
4
  pid "#{app_path}/tmp/pids/unicorn.pid"
5
5
 
6
- listen "/tmp/unicorn.sock"
6
+ listen "/tmp/unicorn.<%= application %>.sock"
7
7
  worker_processes <%= unicorn_workers %>
8
8
  timeout 60
9
9
  preload_app true
@@ -1,3 +1,3 @@
1
1
  module Matross
2
- VERSION = "0.2.6"
2
+ VERSION = "0.2.7"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: matross
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.6
4
+ version: 0.2.7
5
5
  platform: ruby
6
6
  authors:
7
7
  - Artur Rodrigues
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2013-11-05 00:00:00.000000000 Z
12
+ date: 2013-11-18 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: bundler
@@ -110,7 +110,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
110
110
  version: '0'
111
111
  requirements: []
112
112
  rubyforge_project:
113
- rubygems_version: 2.1.10
113
+ rubygems_version: 2.1.11
114
114
  signing_key:
115
115
  specification_version: 4
116
116
  summary: Our collection of opnionated Capistrano recipes