cl-mina 0.1.1 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2683e634441d8c2c052c3b43044906f8c547210b
-  data.tar.gz: 255bcf0ff8fdc4b0e60db407d19f55be5c00eb88
+  metadata.gz: 412ef0bb993ac60314015c3888243284ee29daf1
+  data.tar.gz: f4f06e24d646e0c998874cac2d1531346d4f8a2e
 SHA512:
-  metadata.gz: 5bc7538493cb9b0f70219ddc3fbc44625da5a1a395994ec42adf1620e6b8ca7682cba78c293ec92718e2a2cc48c28dc88f766b021ee6afc1405ca9e40de70d57
-  data.tar.gz: c396182235ae69892b9ca194d545d863d3818198116f9fec317f36696802bfb2f5239ca5779ff2167edd61084e3eb7e498cc917eebe2d2bd986b67d3fd347f24
+  metadata.gz: 41c9ecd1a50f1f58bee90ff287f03f58b6bb3d2c8c4ad7185e20861dd0f8541078e584bd6567b02784cdd0572cfd1fd53ae7097f8e358693cdbfb8dd2ade74ea
+  data.tar.gz: af3599738061a9c7c581e713869422924b36ef912e05dd69bbfa497db521a608c3dbd795bad7b85288ad5e8e9afaa4141412967c2586a1c17617a320a4e7e05d

data/README.md

@@ -1,3 +1,7 @@
+# What is mina?
+
+Mina is a gem that allows you to easily write rake tasks to automate deployment tasks. Mina works by generating bash scripts that it then runs on a remote server. This gem includes tasks to configure and deploy to servers with nginx
+
 # How to use
 
 1. Add to Gemfile: `gem "cl-mina"`
@@ -5,4 +9,82 @@
 3. ???
 4. Profit
 
-Hint: `mina tasks` will get you a long way toward understanding what ??? is.
+In addition to the documentation here, `config/deploy.rb`, `config/deploy/production.rb`, and `config/deploy/staging.rb` are reasonably well-documented.
+
+# Install
+
+## Run generator
+
+To create the necessary files to run the cl-mina tasks, run `rails generate cl-mina:install`. The install generator creates several files:
+1. `config/deploy.rb` contains application-specific configuration for all environments as well as the basic deploy script.
+2. `config/deploy/defaults.rb` contains defaults for configuration variables. Most of the time you won't need to modify this. This task is called by the specific environment tasks, and it will only set variables that have not been set for the environment.
+3. `config/deploy/staging.rb`, `config/deploy/production.rb` contain configuration variables that are likely to be different for staging and production environments.
+4. `config/environments/staging.rb` is a base environment configuration for a staging environment since Rails does not include one by default. Generally you will want to keep yours if you already have one.
+
+## Customize configuration
+
+For most applications, you'll want to edit three files. All the documentation you need should be in the files themselves.
+1. `config/deploy.rb`
+2. `config/deploy/production.rb`
+3. `config/deploy/staging.rb`
+
+# Tasks
+
+To list all available tasks with descriptions, `mina tasks`.
+
+To run a task, `mina namespace:task to={server}`. So for example, to deploy to production, you would execute `mina deploy to=production`. The default server is set in `config/deploy.rb`. It's a good idea to have the default set to staging to force yourself to be intentional in deploying to production. But that's up to you.
+
+All of the tasks are in `lib/cl-mina/tasks`, so if you want to see what exactly a task is doing, look there.
+
+Tasks are listed here by namespace. E.g., to set up a postgresql db and configuration, you'd execute `mina postgresql:setup` (or `mina postgresql:setup to=production` for production server).
+
+## Variables
+
+Tasks use variables set in the files in the `config/deploy/` directory. `config/deploy/defaults.rb` includes default variables for all servers/environments, and `config/deploy/production.rb` or `config/deploy/staging.rb` include values specific to the server/environment. If you want to change a variable, you should almost always do so in one of the files in the `config/deploy/` directory.
+
+## Task descriptions by namespace
+
+### db
+
+`db:seed`  
+**Run rake db:seed.**
+
+### log
+
+`log:rails`, `log:unicorn`, `log:sidekiq`, `log:nginx`, `log:nginx_access`  
+**Display tail of log.** Gets log locations from variables. Number of lines shown determined by `log_tail` variable (default 50).
+
+### logrotate
+
+`logrotate:setup`  
+**Sets up logrotate.** I bet you'd never have guessed that.
+
+### nginx
+
+`nginx:setup`  
+**Creates nginx configuration and init script.** Overwrites any existing configuration and restarts nginx.
+
+`nginx:remove`  
+**Removes nginx configuration.** Removes config from `/etc/nginx/sites-enabled`, restarts nginx.
+
+`nginx:start`, `nginx:stop`, `nginx:restart`  
+**Start, stop, or restart nginx.**
+
+### postgresql
+
+`postgresql:setup`  
+**Creates user and database along with matching database.yml file**. Asks for password, uses it for database user and database.yml template. If user or database already exists, the existing database/user is unchanged. Among other things, this means that if you run this after a user has been created, make sure you enter the same password, or the password in your database.yml file will not match the password of your db user. This is a bug, and it'll be fixed...eventually.
+
+`postgresql:create_user`  
+**Prompts for password, creates postgresql user with that password**. Generally, you should just run the :setup command above rather than running this directly. The username is determined by `postgresql_user` and by default is the the application name.
+
+`postgresql:create_database`  
+**Creates a database for the application.** As with :create_user, you should usually just use the :setup command instead of using this directly. The database name is determined by `postgresql_user` and is by default the application name along with the server name, e.g. "your_app_production".
+
+### unicorn  
+
+`unicorn:setup`  
+**Create unicorn config and init script.**
+
+`unicorn:start`, `unicorn:stop`, `unicorn:restart`  
+**Start, stop, restart unicorn.**

data/cl-mina.gemspec

@@ -24,7 +24,6 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency "generator_spec"
 
   spec.add_runtime_dependency "mina"
-  spec.add_runtime_dependency "pg"
   spec.add_runtime_dependency "unicorn"
   spec.add_runtime_dependency "carrier-pigeon"
   spec.add_runtime_dependency "highline"

data/lib/cl_mina/tasks/all.rb

@@ -1,9 +1,11 @@
 require 'cl_mina/tasks/check'
+require 'cl_mina/tasks/clouddb'
 require 'cl_mina/tasks/db'
 require 'cl_mina/tasks/irc'
 require 'cl_mina/tasks/log'
 require 'cl_mina/tasks/logrotate'
 require 'cl_mina/tasks/nginx'
 require 'cl_mina/tasks/postgresql'
+require 'cl_mina/tasks/sidekiq'
 require 'cl_mina/tasks/unicorn'
 require 'cl_mina/tasks/utility'

data/lib/cl_mina/tasks/check.rb

@@ -1,3 +1,5 @@
+# Checks commit hash of local branch against remote. If local does not match remote, says so and stops execution.
+
 namespace :check do
   desc "Make sure local git is in sync with remote."
   task :revision do

data/lib/cl_mina/tasks/clouddb.rb

@@ -0,0 +1,18 @@
+require "highline/import"
+
+namespace :clouddb do
+  desc "Generate the database.yml configuration file"
+  task :setup do
+    get_clouddb_password
+    template "clouddb.yml.erb", "#{config_path}/database.yml"
+  end
+end
+
+def get_clouddb_password
+  if !clouddb_password
+    pw = ask "Database password: " do |p|
+      p.echo = '*'
+    end
+    set :clouddb_password, pw
+  end
+end

data/lib/cl_mina/tasks/log.rb

@@ -1,26 +1,26 @@
 namespace :log do
   desc "View tail of rails log"
   task :rails do
-    queue "tail #{logs_path}/#{rails_env}.log -n 50"
+    queue "tail #{logs_path}/#{rails_env}.log -n #{log_tail}"
   end
 
   desc "View tail of unicorn log"
   task :unicorn do
-    queue "tail #{unicorn_log} -n 50"
+    queue "tail #{unicorn_log} -n #{log_tail}"
   end
 
   desc "View tail of sidekiq log"
   task :sidekiq do
-    queue "tail #{sidekiq} -n 50"
+    queue "tail #{sidekiq_log} -n #{log_tail}"
   end
 
   desc "View tail of nginx error log"
   task :nginx do
-    queue "tail /var/log/nginx/error.log -n 50"
+    queue "tail /var/log/nginx/error.log -n #{log_tail}"
   end
 
   desc "View tail of nginx access log"
   task :nginx_access do
-    queue "tail /var/log/nginx/access.log -n 50"
+    queue "tail /var/log/nginx/access.log -n #{log_tail}"
   end
 end

data/lib/cl_mina/tasks/sidekiq.rb

@@ -0,0 +1,28 @@
+namespace :sidekiq do
+  desc "Start sidekiq"
+  task :start do
+    queue %{cd #{deploy_to}/#{current_path}}
+
+    queue %{echo "-----> Start sidekiq"}
+    queue echo_cmd "bundle exec sidekiq -d -c #{sidekiq_concurrency} -e #{rails_env} -L #{sidekiq_log} -P #{sidekiq_pid}"
+  end
+
+  desc "Quiet sidekiq"
+  task :quiet do
+    queue %{cd #{deploy_to}/#{current_path}}
+    queue echo_cmd "bundle exec sidekiqctl quiet #{sidekiq_pid}"
+  end
+
+  desc "Stop sidekiq"
+  task :stop do
+    queue %{cd #{deploy_to}/#{current_path}}
+    queue echo_cmd "bundle exec sidekiqctl stop #{sidekiq_pid} 60"
+  end
+
+  desc "Restart sidekiq"
+  task :restart do
+    queue %{cd #{deploy_to}/#{current_path}}
+    invoke :'sidekiq:stop'
+    invoke :'sidekiq:start'
+  end
+end

data/lib/cl_mina/tasks/utility.rb

@@ -1,18 +1,12 @@
 task :setup do
-  invoke :'postgresql:setup'
+  invoke :'postgresql:setup' if setup_postgresql
+  invoke :'clouddb:setup' if setup_clouddb
   invoke :'nginx:setup'
   invoke :'unicorn:setup'
   invoke :'logrotate:setup'
   invoke :create_extra_paths
 end
 
-task :echo_config do
-  queue %{echo #{rails_env}}
-  queue %{echo #{deploy_to}}
-  queue %{echo #{shared_path}}
-  queue %{echo #{shared_paths.first}}
-end
-
 task :create_extra_paths do
   queue 'echo "-----> Create configs path"'
   queue echo_cmd "mkdir -p #{config_path}"

data/lib/cl_mina/templates/clouddb.yml.erb

@@ -0,0 +1,9 @@
+<%= rails_env %>:
+  adapter: <%= clouddb_adapter %>
+  encoding: utf8
+  database: <%= clouddb_database %>
+  pool: 10
+  username: <%= clouddb_user %>
+  password: <%= clouddb_password %>
+  host: <%= clouddb_host %>
+  reconnect: true

data/lib/cl_mina/templates/logrotate.erb

@@ -1,5 +1,6 @@
 <%= logs_path %>/*.log {
     weekly
+    size 200M
     missingok
     rotate 52
     compress

data/lib/cl_mina/templates/unicorn.rb.erb

@@ -13,7 +13,9 @@ before_exec do |server|
   if defined? ActiveRecord::Base
     ActiveRecord::Base.connection.disconnect!
   end
+end
 
+before_fork do |server, worker|
   old_pid = "#{server.config[:pid]}.oldbin"
 
   if File.exists?(old_pid) && server.pid != old_pid

data/lib/cl_mina/version.rb

@@ -1,3 +1,3 @@
 module ClMina
-  VERSION = "0.1.1"
+  VERSION = "0.2.2"
 end

data/lib/generators/cl_mina/install_generator.rb

@@ -19,6 +19,10 @@ module ClMina
       def copy_defaults
         template "defaults.rb", "config/deploy/defaults.rb"
       end
+
+      def create_staging_environment
+        copy_file "config/environments/production.rb", "config/environments/staging.rb"
+      end
     end
   end
 end

data/lib/generators/templates/defaults.rb

@@ -9,6 +9,7 @@ task :defaults do
   set_default :shared_paths,    ['tmp', 'log', 'public/uploads', 'config/database.yml']
   set_default :branch,          'staging'
   set_default :deploy_to,       "/home/#{user}/#{application}/#{deploy_server}"
+  set_default :log_tail,        50
 
   set_default :sockets_path,    "#{deploy_to}/#{shared_path}/tmp/sockets"
   set_default :pids_path,       "#{deploy_to}/#{shared_path}/tmp/pids"
@@ -26,13 +27,21 @@ task :defaults do
   set_default :sidekiq_log,     "#{logs_path}/sidekiq.log"
   set_default :sidekiq_pid,     "#{pids_path}/sidekiq.pid"
   set_default :sidekiq_script,  "/etc/init/sidekiq_#{application}_#{deploy_server}.conf"
+  set_default :sidekiq_concurrency, 5
 
   set_default :nginx_config,    "#{nginx_path}/sites-enabled/#{application}_#{deploy_server}"
   set_default :nginx_log_path,  "#{deploy_to}/#{shared_path}/log/nginx"
   set_default :nginx_server_name, domain || server_ip
 
+  set_default :setup_postgresql, true
   set_default :postgresql_host, "localhost"
   set_default :postgresql_user, application
   set_default :postgresql_database, "#{application}_#{deploy_server}"
   set_default :postgresql_pid,  "/var/run/postgresql/9.1-main.pid"
+
+  set_default :setup_clouddb,   false
+  set_default :clouddb_host,    'localhost'
+  set_default :clouddb_adapter, 'mysql2'
+  set_default :clouddb_database, "#{application}_#{rails_env}"
+  set_default :clouddb_user,    application
 end

data/lib/generators/templates/deploy.rb

@@ -1,7 +1,6 @@
 ###########################################################################
-
 #### Setup ####
-# 1. Set "Common settings for all servers" in this file
+# 1. Set common settings for all servers in this file
 # 2. Set environment-specific settings in config/deploy/staging.rb and config/deploy/production.rb
 # 3. Verify defaults in config/deploy/defaults.rb
 
@@ -12,14 +11,18 @@
 # mina setup to=staging
 # mina nginx:setup to=production
 # mina log:rails to=production
+# mina deploy
 ###########################################################################
 
+# Include built-in mina tasks
 require 'mina/bundler'
 require 'mina/rails'
 require 'mina/git'
 
+# Load default configuration from deploy/defaults
 load 'config/deploy/defaults.rb'
 
+# Load all tasks in cl-mina gem
 require 'cl_mina/tasks/all'
 
 # Load environments from config/deploy
@@ -29,35 +32,103 @@ Dir['config/deploy/*.rb'].each { |f| load f }
 # Common settings for all servers
 ###########################################################################
 
+# application
+# The...unix-safe name of the application.
+# It will be used when generating directories, files, usernames, etc.
+# Guidelines for naming:
+# - No spaces. Things will break.
+# - For the sake of standardization, go for all lowercase unless you have a
+#   really good reason not to.
 set :application,       "your_app"
+
+# application_name
+# The name of the application.
+# This is used any time a script broadcasts messages, like upon deployment.
+# Spaces are fine, even encouraged, here.
 set :application_name,  "Human Name for Your App"
+
+# domain_name
+# The domain name where the application will live.
+# Use the generic domain name that will apply across environments.
+# You can customize it for specific environments in environment-specific configuration.
 set :domain_name,       "example.com"
+
+# keep_releases
+# Number of releases to keep when deploying.
+# If there are more releases than this on the server, the oldest will be deleted.
+# TODO: actually implement this.
 set :keep_releases,     5
-set :repository,        "user@git.example.com:/your-app.git"
+
+# respository
+# The repository where the application lives.
+# Only supports git.
+# When you deploy, the server will pull from this repository to create a new release.
+# SSH agent forwarding is enabled by default, so the server will use the key
+# you're using to ssh into the server to fetch the respository. If your key works
+# when using the git server directly but not when you're deploying, it's probably
+# an ssh-agent issue. Github has pretty good documentation about that.
+set :repository,        "user@git.example.com:your-app.git"
+
+# default_server
+# The server to run a task on if one is not specified.
+# If you don't specify a server with to=whatever, this is the server where the task(s)
+# will run. Change to production at your own risk.
 set :default_server,    "staging"
 
-# IRC options
+##### IRC options #####
+# If you want to use IRC, set these.
+
+# send_irc_messages
+# Whether to send irc announcements.
+# If you set this to false, you can ignore the other irc settings.
 set :send_irc_messages, true
+
+# irc_uri
+# Format: irc://{nick}:{password}@{server name}:{port}/#{channel}
 set :irc_uri,           "irc://MinaBot:password@git.example.com:6697/#log"
+
+# irc_ssl
+# Whether to use ssl when connecting to the irc server
 set :irc_ssl,           true
 
-# SSL Options
+##### End IRC options #####
+
+##### SSL options ####
+# If you're using https, you can use these options combined with the environment
+# deploy configuration to set up nginx config.
 set :ssl_cert_path,     "/etc/nginx/ssl/your_app.crt"
 set :ssl_cert_key_path, "/etc/nginx/ssl/your_app.key"
 
 ###########################################################################
 
+# Set server based on "to" environment variable.
+# E.g., mina deploy to=production
 set :server, ENV['to'] || default_server
+
+# Load environment-specific deploy config from config/deploy/
 invoke :"env:#{server}"
 
+# This is what happens when you run mina deploy
 desc "Deploys the current version to the server."
 task :deploy => :environment do
   deploy do
+    # Make sure local git branch is synced with remote
     invoke :'check:revision'
+
+    # Clone git repository to create a new release
     invoke :'git:clone'
+
+    # Link configuration you want shared across releases into new release
     invoke :'deploy:link_shared_paths'
+
+    # Install gems for new release
+    # Gems are shared across releases, so they won't need to be installed on every deploy.
     invoke :'bundle:install'
+
+    # Migrate database. If there are no new migrations, does nothing.
     invoke :'rails:db_migrate'
+
+    # Precompile assets. If there are no changed assets, does nothing.
     invoke :'rails:assets_precompile'
 
     to :launch do

data/lib/generators/templates/production.rb

@@ -1,15 +1,50 @@
 namespace :env do
   task :production => [:environment] do
+    # server_ip
+    # Ip address of server to deploy staging app to
     set :server_ip,       '1.1.1.1'
+
+    # domain
+    # Domain of staging app.
+    # Feel free to leave as server_ip.
     set :domain,          server_ip
+
+    # deploy_server
     set :deploy_server,   'production'
+
+    # rails_env
+    # Rails environment to use when setting up and running the app.
     set :rails_env,       'production'
+
+    # branch
+    # git branch that is used to create new releases.
     set :branch,          'master'
+
+    # rewrite_www
+    # Whether to include logic in nginx to rewrite www.example.com to example.com.
     set :rewrite_www,     false
+
+    # include_ssl
+    # Whether to include ssl configuration in nginx config.
     set :include_ssl,     false
+
+    # default_host
+    # Whether the app should be set as the default host in nginx config.
+    # Generally, you'll want true if it's the only app running on the server.
+    # If there are multiple apps running on the server, make sure only one of them
+    # is set as the default.
     set :default_host,    true
+
+    # unicorn_workers
+    # Number of unicorn workers to run for this server. Keep in mind that workers
+    # will briefly overlap when transitioning to new release during unicorn:restart.
     set :unicorn_workers, 2
 
+    # sidekiq_concurrency
+    # Number of threads created by one sidekiq process
+    set :sidekiq_concurrency, 5
+
+    # Set all other configuration using config/deploy/defaults.rb
     invoke :defaults
   end
 end

data/lib/generators/templates/staging.rb

@@ -1,15 +1,52 @@
 namespace :env do
   task :staging => [:environment] do
+    # server_ip
+    # Ip address of server to deploy staging app to
     set :server_ip,       '1.1.1.1'
+
+    # domain
+    # Domain of staging app.
+    # Feel free to leave as server_ip.
     set :domain,          server_ip
+
+    # deploy_server
+    # Used often in naming files, database, etc. to allow multiple versions of the
+    # same application to run on the same server.
     set :deploy_server,   'staging'
+
+    # rails_env
+    # Rails environment to use when setting up and running the app.
     set :rails_env,       'staging'
+
+    # branch
+    # git branch that is used to create new releases.
     set :branch,          'staging'
+
+    # rewrite_www
+    # Whether to include logic in nginx to rewrite www.example.com to example.com.
     set :rewrite_www,     false
+
+    # include_ssl
+    # Whether to include ssl configuration in nginx config.
     set :include_ssl,     false
+
+    # default_host
+    # Whether the app should be set as the default host in nginx config.
+    # Generally, you'll want true if it's the only app running on the server.
+    # If there are multiple apps running on the server, make sure only one of them
+    # is set as the default.
     set :default_host,    true
+
+    # unicorn_workers
+    # Number of unicorn workers to run for this server. Keep in mind that workers
+    # will overlap when transitioning to new release during unicorn:restart.
     set :unicorn_workers, 1
 
+    # sidekiq_concurrency
+    # Number of threads created by one sidekiq process
+    set :sidekiq_concurrency, 5
+
+    # Set all other configuration using config/deploy/defaults.rb
     invoke :defaults
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: cl-mina
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.2.2
 platform: ruby
 authors:
 - Nathan Sharpe
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-07-10 00:00:00.000000000 Z
+date: 2014-12-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -80,20 +80,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: pg
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
 - !ruby/object:Gem::Dependency
   name: unicorn
   requirement: !ruby/object:Gem::Requirement
@@ -153,14 +139,17 @@ files:
 - lib/cl_mina.rb
 - lib/cl_mina/tasks/all.rb
 - lib/cl_mina/tasks/check.rb
+- lib/cl_mina/tasks/clouddb.rb
 - lib/cl_mina/tasks/db.rb
 - lib/cl_mina/tasks/irc.rb
 - lib/cl_mina/tasks/log.rb
 - lib/cl_mina/tasks/logrotate.rb
 - lib/cl_mina/tasks/nginx.rb
 - lib/cl_mina/tasks/postgresql.rb
+- lib/cl_mina/tasks/sidekiq.rb
 - lib/cl_mina/tasks/unicorn.rb
 - lib/cl_mina/tasks/utility.rb
+- lib/cl_mina/templates/clouddb.yml.erb
 - lib/cl_mina/templates/logrotate.erb
 - lib/cl_mina/templates/nginx_unicorn.erb
 - lib/cl_mina/templates/postgresql.yml.erb
@@ -195,7 +184,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.2.1
+rubygems_version: 2.2.2
 signing_key: 
 specification_version: 4
 summary: Common deploy tasks and settings.