RubyGems - terra_boi - Versions diffs - 0.0.13 → 1.0.2 - Mend

terra_boi 0.0.13 → 1.0.2

Files changed (58) hide show

checksums.yaml CHANGED

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a3778d98d44fb92002f085027ce5e14b51d77d4dfee1ef11c97a4474191c200c
-  data.tar.gz: 9f8786b9e4c67d19d366cf6a0822ae5f72694480dac17bdf36cde7720d8ee8bf
+  metadata.gz: 3f586f964ace1990c78dd488bec9045c589493bb6a8d3bb76bd1184535d19927
+  data.tar.gz: 2d04e5da26bee200183935911f7b5a60820099750bd79ff17326378a52f05212
 SHA512:
-  metadata.gz: 954ea0480e5c6dc1272983f665d5a8e3b46f2e3093df85b9acf4e41c18b8e27cb8a70bfb4f55cab8021a452c11de75598330ee48b482afd96f7a75e1345fbc92
-  data.tar.gz: b93f8776203b459762dee8311880630afe61096ee8921343d9a40e9fdca86f9ce2d0bb3f7bb0a54fe2d9a56988bb41d6c1d6a9372a3c68b3e4e8c2b33a7acd62
+  metadata.gz: dd77679206ecc88ff7f9816494da339d7bdc5b6bf09a9ea3597f67fac2d1550723c5a857ead2b6c3a8265ce582e6cd716619e55c539d43a7f041aa9a1818f315
+  data.tar.gz: 283ca1f89860f934937a2f85c1389904936fd5512c11fa26a5dc0b17794227d4f7419e952423d7e5c01af3658be55197801808997e87bca116f22dc93f94836a

data/README.md CHANGED

@@ -1,234 +1,163 @@
 # TerraBoi
-This ruby gem was created by [Charlie Reese](https://charliereese.ca/about) at [Clientelify](https://clientelify.com) to get rails applications deployed into production as quickly and easily as possible.
+## Introduction
-**Raison d'etre**: creating basic infrastructure to house production SaaS applications on AWS is tedious and boring. It's often a similar process every time, and every time it sucks.
+Deploy your rails application to AWS with `rake deploy`.
-List of items created by this gem's generators:
-- Dockerfile
-- Rails initializer file (for setting up config.hosts)
-- Packer repository (for creating AMIs)
-- Terraform repository (for creating infrastructure as code to immediately deploy staging / prod infrastructure as well as rolling out application updates)
+This ruby / rails gem was created by [Charlie Reese](https://charliereese.ca/about) for [Clientelify](https://clientelify.com). It creates AWS infrastructure for your rails application and deploys it in 5 steps (3 installation steps and 2 rake tasks). Subsequent deployment is 1 step (`rake deploy`).
-**Note**: generated Terraform files create / support remote state locking, load-balancing, auto-scaling, zero-downtime web app deployments, DBs, and S3 buckets.
+It is free to use.
-**Note**: after infrastructure files are generated, you will be ready to deploy your application to staging / production on AWS. If you have more advanced infrastructure needs (e.g. Redis / Solr instances), you may add to the generated Terraform files to support this.
+Out of the box, terra_boi provides remote state locking, load-balancing, simple scaling, zero-downtime deployments, CloudWatch logging, DBs, and S3 buckets for multiple infrastructure environments: by default, terra_boi creates staging and prod environments for your web app.
+## Installation
-## Pre-requisites
+#### Installation A: pre-requisites
 * [Terraform](https://www.terraform.io/) installed on your computer
-* [Packer](https://www.packer.io/downloads.html) installed on your computer
 * [Amazon Web Services (AWS) account](http://aws.amazon.com/)
+* [AWS access key and secret key](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) (to be used below shortly)
+* Rails ~> 6.0
+#### Installation B: install ruby gems
-## Installation
-**Note**: below installation steps should be completed in order.
-### Installation - gem
-Add this line to your (Rails) application's Gemfile:
+Add these lines to your (Rails) application's Gemfile:
 ```ruby
-gem 'terra_boi'
+group :development do
+	gem 'terra_boi'
+end
+gem 'pg' # Postgresql
+gem 'whenever', require: false # For cron jobs set in config/schedule.rb
 ```
 And then execute:
 ```bash
-$ bundle
+$ bundle && bundle exec wheneverize .
 ```
-### Installation - AWS access
+#### Installation C: set environment variables
-Set up your [AWS access / secret access
-keys](http://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) in `~/.zprofile` (or equivalent file for your shell if not using .zsh) as environment variables:
+Set and export terraform AWS and data-related environment variables in ~/.zprofile (or your respective shell dotfile e.g. ~/.bashrc)
 ```
-export AWS_ACCESS_KEY_ID=your_access_key_id
-export AWS_SECRET_ACCESS_KEY=your_secret_access_key
-```
-Then run `source ~/.zprofile` (or equivalent command for your shell if not using .zsh)
-Note: SSH access to created EC2 instances is granted using SSH keys existing at `~/.ssh/id_rsa.pub` and `~/.ssh/id_rsa` on your local machine. If they do not exist, create them.
-### Installation - generate infrastructure code
-To generate boilerplate infrastructure code (config.host initializer filer, Dockerfile, Packer repository, and terraform repository):
-`rails generate terra_boi:boilerplate --domain_name DOMAIN.COM --ruby_version 2.5.1`
-### Installation - Packer (creating web server AMIs)
-**A. Create private DockerHub repository**
+TF_VAR_db_password=your_password # whatever you want it to be
+TF_VAR_db_username=your_username # whatever you want it to be
+export TF_VAR_db_password TF_VAR_db_username
-Create private DockerHub repository for your rails application (if possible, use the exact same name as your rails application).
+AWS_ACCESS_KEY=your_access_key
+TF_VAR_aws_access_key=$AWS_ACCESS_KEY
+export AWS_ACCESS_KEY TF_VAR_aws_access_key
-**Note**: packer generator will assume your DockerHub repository has the same name as your rails application folder. If this isn't true, update generated Packer `ami_build.json` file after it is generated.
-**B. Setup DockerHub access:**
-Add DockerHub username and access key to `~/.zprofile` (or equivalent file for your shell if not using .zsh) as environment variables (if your image is in a private repository):
-```
-DOCKERHUB_USERNAME=myname
-DOCKERHUB_ACCESS_TOKEN=myAccessToken
-export DOCKERHUB_USERNAME DOCKERHUB_ACCESS_TOKEN
-```
-Then run `source ~/.zprofile` (or equivalent command for your shell if not using .zsh)
-**Note**: DockerHub access key can be found at https://hub.docker.com/settings/security
-### Installation - Terraform (deploying DBs + web server AMIs)
-**A. Set up remote state:**
-`cd terraform/state`
-Run `terraform init` and then `terraform apply` to set up s3 bucket and dynamoDB for remote state and locking (this will work for both prod and staging).
-**B. Set up DB / S3:**
-`cd terraform/[ENV]/data`
-Set terraform data-related environment variables in .zprofile (or your respective shell dotfile)
-```
-TF_VAR_db_password=your_password
-TF_VAR_db_username=your_username
+AWS_SECRET_KEY=your_secret_access_key
+TF_VAR_aws_secret_key=$AWS_SECRET_KEY
+export AWS_SECRET_KEY TF_VAR_aws_secret_key
 ```
-To deploy infrastructure to AWS:
+After you've set and export the above environment variables, run `source ~/.zprofile` (or source your respective shell dotfile).
-```
-terraform init # IF NOT ALREADY RUN
-terraform apply
-```
-**C. Set up web servers:**
-`cd terraform/[ENV]/web_servers`
+## Usage
-To deploy infrastructure to AWS:
+#### Usage A: generate infrastructure rake task
 ```
-terraform init # IF NOT ALREADY RUN
-terraform apply
+rake terra_boi:generate_infra
 ```
-While aws_acm_certificate_validation.cert is creating (it will hang if you don't add CNAME verification record in ACM):
-i. Log into AWS console, go to certificate management, and add the created CNAME record specified to the DNS configuration for your domain
-ii. Redirect domain name to Application load balancer:
-	- Go to your domain registrar of choice
-	- Create alias record that points to the dns name of the application load balancer (use subdomain in alias record like STAGING.example.com for staging)
-	- Create URL redirect record for prod (redirect www.site.com to site.com)
+The above rake task will ask you some interactive questions, so stay close to your terminal!
-After these changes propogate (should take about an hour or two locally), your webservers should be set up, https should be working, and you should be good to go!
-## Usage
+Expected rake task duration: ~20 minutes (provisioning AWS infrastructure takes a while).
-**Note**: below usage steps should be completed in order
+Note: the above rake task takes an optional argument for environments (e.g. `rake \"terra_boi:generate_infra[dev staging prod]\"`). Infrastructure environments default to staging and prod.
-### Usage - Packer (creating web server AMIs)
+Note: this command will also deploy your application after infrastructure has been created. If you only wish to deploy an update, use the below command.
-**A. Push latest application image to DockerHub**
-You can automatically trigger DockerHub image builds when new code is pushed to a repository's master branch using DockerHub's free Github integration.
-Otherwise, `docker build . && docker container create [IMAGE_ID] && docker commit [CONTAINER_ID] [DOCKER_USERNAME]/[APPLICATION_NAME]:latest && docker push [DOCKER_USERNAME]/[APPLICATION_NAME]:latest`. Make sure you are pushing to a private repository.
-**B. Create Packer AMI:**
+#### Usage B: deploy application updates rake task
 ```
-cd packer
+rake deploy
-packer build -var DOCKERHUB_ACCESS_TOKEN=$DOCKERHUB_ACCESS_TOKEN -var DOCKERHUB_USERNAME=$DOCKERHUB_USERNAME -var DB_PASSWORD=$TF_VAR_db_password -var AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -var AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY application.json
+# Alternatively: rake "deploy[staging]", rake "deploy[staging prod]", etc.
 ```
-**C. Clean up:**
-Every so often you'll want to remove old AMIs created by Packer (unless you want to be charged a couple cents a month).
+The above rake task will ask you an interactive question, so stay close to your terminal!
-To remove them, deregister them on the [AWS AMI management page](https://us-east-2.console.aws.amazon.com/ec2/v2/home?region=us-east-2#Images:sort=name), then delete the associated snapshot on the [AWS snapshot management page](https://us-east-2.console.aws.amazon.com/ec2/home?region=us-east-2#Snapshots:sort=snapshotId).
+Expected rake task duration: 5 - 10 minutes.
-### Usage - Terraform (update web server AMIs)
+Note: the above rake task takes an optional argument for environments (e.g. `rake \"deploy[staging prod]\"`). By default, it will deploy to staging then prod infrastructure.
-**A. Update Terraform web server AMIs:**
+#### Usage C: recommendations
-`cd terraform/[ENV]/web_servers`
+- Uncomment `config.force_ssl = true` in `config/environments/production.rb` file.
-To deploy infrastructure to AWS:
-```
-terraform init # IF NOT ALREADY RUN
-terraform apply
-```
+## Infrastructure customization
-### Usage - ssh access
+Because terra_boi uses Terraform modules, small customizations can be made by changing variables in .tf files.
-ssh into ec2 instance and run bash in container:
+For example, if you wanted to run 10 web app instances behind the load balancer in your prod environment (instead of the default 2), you could change the web_app_task => desired_count argument from 2 to 10 in the file `terraform/prod/web_app/ecs.tf`. Similarly, if your web app task container requires more memory, you could change the web_app_task => memory argument in the file `terraform/prod/web_app/ecs.tf`.
-```
-ssh ubuntu@PUBLICIPOFINSTANCE
+**Note**: if your rails application has more advanced infrastructure customization needs (e.g. Redis / Solr instances), you may add the resources required to the Terraform files created by terra_boi.
-docker container exec -it APP_NAME bash
-```
+## Appendix
+#### Appendix - infrastructure / code generated by terra_boi
+- AWS Elastic Container Registry (ECR)
+- AWS Application Load Balancer (ALB)
+- AWS Elastic Container Services (for your web app and worker)
+- AWS Fargate tasks (for your web app and worker)
+- AWS CloudWatch logging
+- AWS Security Groups
+- AWS DynamoDB for remote state locking
+- AWS RDS Postgresql DBs
+- Rails DB config file
+- Rails data config file
+- Rails initializer file (for setting up config.hosts)
+- Dockerfile
+- HTTPS / SSL certificate
-## Infrastructure created
-The aforementioned generators create a `terraform` directory with `state`, `prod`, and `staging` subdirectories.
-The `state` directory contains an S3 bucket and a DynamoDB table to store and lock state (for both prod and staging).
-The `prod` and `staging` subdirectories contain `data` (DB + S3) and `web_servers` (SSL cert, load balancing, autoscaling, EC2) directories.
+#### Appendix - destroying your infrastructure
+To destroy infrastructure created by terra_boi, run `terraform destroy` in the following directories in the following order:
-## Running tests
+- terraform/ENV/head_worker
+- terraform/ENV/web_app
+- terraform/ENV/ecs_cluster
+- terraform/ENV/data
+- terraform/ecr
+- terraform/cert
+- terraform/state
-From the root directory:
+OR
 ```
-rake test
+rake terra_boi:destroy_infra
 ```
+Note: the above rake task takes an optional argument for environments (e.g. `rake \"terra_boi:destroy_infra[dev staging prod]\"`). Infrastructure environments default to staging and prod.
+#### Appendix - license
-## Other tips
-Clean up terraform infrastructure when no longer planning to use (DANGER FOR PROD, WILL DESTROY INFRASTRUCTURE):
-`terraform destroy`
-**For extra security in staging:** update Terraform web_servers `main.tf` file to only allow ingress web_server connections from your IP / your team's IPs
-## Contributing
-This gem is currently not actively accepting contributions.
-With that in mind, if you'd like to make a fix / change, please create a pull request (and when I have a moment - probably in a couple weeks time - I'll have a look)!
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+#### Appendix - contributing
+If you'd like to make a fix / change, please create a pull request! When I have a moment, I'll have a look!
-## License
+Recommended contribution: terraform infrastructure generation testing
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+#### Appendix - other tips
+**Something didn't work and you aren't sure why?** If you're running `rake terra_boi:generate_infra`, try running it again. Sometimes Terraform / the AWS API will throw sporadic errors without reason!
+**For extra security in staging / internal applications:** update Terraform security groups to only allow ingress web_app connections from your team's IP addresses
-## Updating gem version (for maintainers)
+#### Appendix - updating gem version (for maintainers)
 **1. Update version**

data/lib/generators/extensions.rb CHANGED

@@ -3,16 +3,16 @@ module TerraBoi
 		# https://api.rubyonrails.org/classes/Rails/Generators/NamedBase.html#method-i-application_name
 		def generate_application_name
 			if defined?(Rails) && Rails.application
-		    Rails.application.class.name.split("::").first.underscore
-		  else
-		    "application"
-		  end
+				Rails.application.class.name.split("::").first.underscore
+			else
+				"application"
+			end
 		end
 		def generate_terraform_files(args)
 			args[:env].each do |env|
 				template(args[:template],
-					"terraform/#{env}/#{args[:file_path]}",
+					"terraform_v2/#{env}/#{args[:file_path]}",
 					{
 						env: env,
 						domain_name: (class_options && class_options[:domain_name]) || generate_application_name + '.com'

data/lib/generators/terra_boi/boilerplate_generator.rb CHANGED

@@ -1,38 +1,45 @@
 module TerraBoi
 	class BoilerplateGenerator < Rails::Generators::Base
 		attr_accessor :application_name, :class_options
-		class_option :ruby_version, type: :string, default: "2.5.1"
-	  class_option :domain_name, type: :string, default: 'example.com'
-	  source_root File.expand_path('templates', __dir__)
-	  desc (<<-EOF
-		  This generator runs the terra_boi web_servers, data,
-		  and state generators. It sets up all boilerplate
-		  terraform infrastructure needed for a standard web app:
-		  DB + load balancer + zero downtime, scalable web app
-		  launch config + S3 bucket.
-		  To execute, run rails generate terra_boi:boilerplate --domain_name example.com
-	 		EOF
-	 		.gsub(/[\t]/, '')
-	 	)
-	 	def init
-	  	self.class_options = options
-	  	puts application_name
-	  end
-	  def run_other_generators
-	  	generate "terra_boi:web_servers --domain_name #{class_options[:domain_name]}"
-	  	generate "terra_boi:data"
-	  	generate "terra_boi:state"
-	  	generate "terra_boi:dockerfile --ruby_version #{class_options[:ruby_version]}"
-	  	generate "terra_boi:host_initializer --domain_name #{class_options[:domain_name]}"
-	  	generate "terra_boi:db_config"
-	  	generate "terra_boi:data_config"
-	  	generate "terra_boi:packer"
-	  	generate "terra_boi:master_worker"
-	  end
+		class_option :ruby_version, type: :string, default: "2.7.1", aliases: ["r"]
+		class_option :domain_name, type: :string, default: 'example.com', aliases: ["d"]
+		class_option :envs, type: :array, default: ['staging', 'prod'], aliases: ["e"]
+		source_root File.expand_path('templates', __dir__)
+		desc (<<-EOF
+			This generator runs the terra_boi web_servers, data,
+			and state generators. It sets up all boilerplate
+			terraform infrastructure needed for a standard web app:
+			DB + load balancer + zero downtime, scalable web app
+			launch config + S3 bucket.
+			To execute, run rails g terra_boi:boilerplate --domain_name example.com --ruby_version 2.7.1 --envs staging prod
+			OR
+			rails g terra_boi:boilerplate --d example.com --r 2.7.1 -e staging prod
+			Note: --ruby_version option is for base Dockerfile ruby image. Defaults to 2.7.1.
+			EOF
+			.gsub(/[\t]/, '')
+		)
+		def init
+			self.class_options = options
+			puts application_name
+		end
+		def run_other_generators
+			generate "terra_boi:tf_cert -d #{class_options[:domain_name]}"
+			generate "terra_boi:tf_ecr"
+			generate "terra_boi:tf_lib -d #{class_options[:domain_name]}"
+			generate "terra_boi:tf_env -e #{class_options[:envs].join(' ')} -d #{class_options[:domain_name]}"
+			generate "terra_boi:tf_state"
+			generate "terra_boi:dockerfile --ruby_version #{class_options[:ruby_version]}"
+			generate "terra_boi:host_initializer -d #{class_options[:domain_name]}"
+			generate "terra_boi:db_config"
+			generate "terra_boi:data_config"
+		end
 	end
 end

data/lib/generators/terra_boi/dockerfile_generator.rb CHANGED

@@ -2,27 +2,28 @@ require "generators/extensions"
 module TerraBoi
 	class DockerfileGenerator < Rails::Generators::Base
-	  attr_accessor :application_name, :class_options
-	  class_option :ruby_version, type: :string, default: "2.5.1"
-	  source_root File.expand_path('templates', __dir__)
+		attr_accessor :application_name, :class_options
+		class_option :ruby_version, type: :string, default: "2.7"
+		source_root File.expand_path('templates', __dir__)
-	  desc (<<-EOF
-		  Generate Dockerfile for rails app
-		  To execute, run rails generate terra_boi:dockerfile
+		desc (<<-EOF
+			Generate Dockerfile for rails app
+			To execute, run rails generate terra_boi:dockerfile
-		  Can pass in --ruby_version command line argument. Defaults to 2.5.1.
-		  EOF
-	  	.gsub(/\t/, '')
-	  )
+			Can pass in --ruby_version command line argument. Defaults to 2.5.1.
+			EOF
+			.gsub(/\t/, '')
+		)
-	  def init
-	  	self.application_name = generate_application_name
-	  	self.class_options = options
-	  end
+		def init
+			self.application_name = generate_application_name
+			self.class_options = options
+		end
-	  def create_dockerfile
-	  	template "Dockerfile.erb", "Dockerfile"
-	  end
+		def create_dockerfile
+			template "Dockerfile.erb", "Dockerfile"
+			template ".dockerignore.erb", ".dockerignore"
+		end
 	end
 end