prepd 0.1.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 7da5fcedfd696ae8ed02c72983c3580aa81ebdb2
4
+ data.tar.gz: 5ff634d081c083161b62fef9b55506d7ffe0c0d8
5
+ SHA512:
6
+ metadata.gz: eed08a0948cc105d1c88bc4ef489e35fd0dce5f5429c731c3f708689cc18ea14aee2f9413fdc85fce2c5fa926c93eb4b50203f80a221dc01c9ac4a15984fe770
7
+ data.tar.gz: 5d687fbcbc00fcb87c2d0c1f25e619ac0532f640220daea216be2f7fcb7d6243010a42328fe90e2249cb579130427c762325aedcd1103eb57851b565b22c6d62
data/.gitignore ADDED
@@ -0,0 +1,9 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /Gemfile.lock
4
+ /_yardoc/
5
+ /coverage/
6
+ /doc/
7
+ /pkg/
8
+ /spec/reports/
9
+ /tmp/
@@ -0,0 +1,49 @@
1
+ # Contributor Code of Conduct
2
+
3
+ As contributors and maintainers of this project, and in the interest of
4
+ fostering an open and welcoming community, we pledge to respect all people who
5
+ contribute through reporting issues, posting feature requests, updating
6
+ documentation, submitting pull requests or patches, and other activities.
7
+
8
+ We are committed to making participation in this project a harassment-free
9
+ experience for everyone, regardless of level of experience, gender, gender
10
+ identity and expression, sexual orientation, disability, personal appearance,
11
+ body size, race, ethnicity, age, religion, or nationality.
12
+
13
+ Examples of unacceptable behavior by participants include:
14
+
15
+ * The use of sexualized language or imagery
16
+ * Personal attacks
17
+ * Trolling or insulting/derogatory comments
18
+ * Public or private harassment
19
+ * Publishing other's private information, such as physical or electronic
20
+ addresses, without explicit permission
21
+ * Other unethical or unprofessional conduct
22
+
23
+ Project maintainers have the right and responsibility to remove, edit, or
24
+ reject comments, commits, code, wiki edits, issues, and other contributions
25
+ that are not aligned to this Code of Conduct, or to ban temporarily or
26
+ permanently any contributor for other behaviors that they deem inappropriate,
27
+ threatening, offensive, or harmful.
28
+
29
+ By adopting this Code of Conduct, project maintainers commit themselves to
30
+ fairly and consistently applying these principles to every aspect of managing
31
+ this project. Project maintainers who do not follow or enforce the Code of
32
+ Conduct may be permanently removed from the project team.
33
+
34
+ This code of conduct applies both within project spaces and in public spaces
35
+ when an individual is representing the project or its community.
36
+
37
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
38
+ reported by contacting a project maintainer at rjayroach@gmail.com. All
39
+ complaints will be reviewed and investigated and will result in a response that
40
+ is deemed necessary and appropriate to the circumstances. Maintainers are
41
+ obligated to maintain confidentiality with regard to the reporter of an
42
+ incident.
43
+
44
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage],
45
+ version 1.3.0, available at
46
+ [http://contributor-covenant.org/version/1/3/0/][version]
47
+
48
+ [homepage]: http://contributor-covenant.org
49
+ [version]: http://contributor-covenant.org/version/1/3/0/
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in prepd.gemspec
4
+ gemspec
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2016 Robert Roach
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,335 @@
1
+ # Prepd
2
+
3
+ Prepd - A Production Ready Environment for Project Development
4
+
5
+ One of the core principles of Agile Development is delivering viewable results
6
+ to the business from Week 1. Too often product developement begins with the
7
+ application software, while the infrastructure to deploy into is addressed as
8
+ and when it is needed.
9
+
10
+ Thankfully, many web application products get to market on similar,
11
+ if not identical, infrastructure. However setting up this infastructure takes time,
12
+ is error prone and typically is non-repeatable ending up as a unique snowflake.
13
+
14
+ To avoid this, many development teams turn to a PaaS service such as Heroku.
15
+ This has limitations and only addresses the final deployment infrastructure.
16
+
17
+ Prepd aims to address this by providing a 'convention over configruation' approach
18
+ to provisioning infrastructure. From local developer machines (vagrant running linux
19
+ on the developer's laptop) to staging and production running a docker swarm cluster.
20
+
21
+ With microservices becoming a common application development strategy, prepd
22
+ aims to make it dead simple to build and deploy a microservice based application.
23
+ Beginning with the end in mind, Prepd offers a simple, conventional way to provision
24
+ all this infrastructure, including CI workflow, secrets managment, 12-factor apps
25
+
26
+ Agile Development requires 'near production' infrastructure to be in place from Day 1.
27
+ Using Prepd, makes that possible quickly and easily without resorting to a PaaS provider.
28
+
29
+ ## Focus
30
+
31
+ The focus of Prepd is on enabling developers to build and deploy applications following current
32
+ industry best practices with as little effort as possible. Being flexible and configurable
33
+ for the wide variety of application deployment strategies is currently a secondary goal to
34
+ getting something up and running. Therefore, choices are made:
35
+
36
+ 1. Infrastructure is provisioned via:
37
+ ..* Vagrantfile on local machines for development and a local cluster
38
+ ..* Terraform plans for clutser infrastructure exclusively on AWS
39
+ 2. Ansible is the automation tool used to configure the infrastructure for application deployment
40
+ 3. Docker conatainer deployment is currently the only method for deploying applications
41
+ 4. The development environment currently supports:
42
+ ..* Postgres and Redis for data storage
43
+ ..* Rails and Ember for application development
44
+
45
+ A future goal for Prepd is to enable more application types and tool support
46
+
47
+ # What is a Production Ready Environment?
48
+
49
+ It takes a lot of services tuned to work together to make smoothly running infrastructure
50
+
51
+ ## Networking
52
+ - Domain names figured out and DNS running on Route53 etc
53
+ - Ability to programatically change and update DNS
54
+ - SSL certs are already installed so we do TLS from the beginning on all publicly available infrastructure
55
+ - Load Balancing is setup, configured and running in at least staging and production, but also possible in development
56
+
57
+ ## Development Pipeline Required Services
58
+
59
+ Prepd provisions and configures the infrastructure and provides a tool to deploy applications into the infrastructure.
60
+ However, certain aspects of the pipeline are expected to be provided outside of Prepd, which are:
61
+
62
+ - Continuous Integration
63
+ - Container Build and Store
64
+
65
+ ### Continuous Integration
66
+
67
+ CI is expected to be setup and configured as part of an automated deploy process from the outset of the project.
68
+ Here is an example overview of using CircleCI to test a Rails API application
69
+
70
+ - Create an account on CircleCI and link it to your GitHub account. Authorize CircleCI to access the account
71
+ - Add the Rails API repository as a project on CircleCI. If using rails-templates a circle.yml project already exists
72
+ - Configure slack notifications for when a build completes
73
+
74
+ ### Container Build and Store
75
+
76
+ A container repository that also builds containers is expected to be provided.
77
+ Here is an example overview of using quay.io to build a Rails API application container
78
+
79
+ - Create an account on quay.io and link it to your GitHub account. Authorize quay.io to access the account
80
+ - Add the Rails API repository as a docker repository on quay.io
81
+ - Create a trigger to build the container when there is a push on a certain branch of the GitHub repository
82
+
83
+ Prepd provides ansible playbooks that invoke docker compose to deploy the container from quay.io to the target infrastructure
84
+
85
+ ## Application Services (TODO)
86
+
87
+ Prepd will be augmented to provide playbooks for the default Application Group as well as Terraform plans that provide:
88
+
89
+ - Communication Services, e.g. SMTP, SNS (Push), Slack webhooks, Twilio, etc
90
+ - Logging in both local/development and in staging/production with ELK
91
+ - Monitoring/alert service (Prometheus)
92
+ - Additional common 3rd party services as needed
93
+
94
+ ## Swarm Load Balancing
95
+ - network overlays
96
+ - load balancing between micro services
97
+ - manage cluster scaling with compose/swarm mode/ansible or some combination thereof
98
+
99
+
100
+ # Installation
101
+
102
+ Prepd is a ruby gem. It also requires software on the local laptop, including VirtualBox, Vagrant and Ansible
103
+
104
+ ```bash
105
+ gem install prepd
106
+ ```
107
+
108
+ ## Automated Installation of Dependencies (TODO)
109
+
110
+ With the gem installed, navigate to it's directory and run bootstrap.sh to install dependencies
111
+
112
+ ```bash
113
+ bundle cd prepd
114
+ ./bootstrap.sh
115
+ ```
116
+
117
+ This will:
118
+
119
+ - Install ansible
120
+ - Clone the ansible-roles repository
121
+ - Run ansible to install Virtualbox and Vagrant
122
+
123
+ ## Manual Installation of Dependencies
124
+
125
+ ### Ansible
126
+
127
+ Tested with version 2.2.0
128
+
129
+ #### Install on MacOS
130
+
131
+ If planning to install on a clean machine:
132
+ 1. Wipe Mac: http://support.apple.com/kb/PH13871 OR http://support.apple.com/en-us/HT201376
133
+ 2. Create New User with Admin rights
134
+
135
+ Install Homebrew:
136
+
137
+ ```bash
138
+ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
139
+ ```
140
+
141
+ Install python with zlib and ssl support
142
+
143
+ ```bash
144
+ xcode-select --install
145
+ brew install openssl
146
+ brew link openssl --force
147
+ brew uninstall python
148
+ brew install python --with-brewed-openssl
149
+ sudo easy_install pip
150
+ sudo pip install -U ansible
151
+ sudo pip install -U setuptools cryptography markupsafe
152
+ sudo pip install -U ansible boto
153
+ ```
154
+
155
+ #### Install on Ubuntu
156
+
157
+ ```bash
158
+ apt-get install ansible
159
+ ```
160
+
161
+ ### VirtualBox
162
+
163
+ Install VirtualBox from [here](https://www.virtualbox.org/wiki/Downloads)
164
+
165
+ ### Vagrant
166
+
167
+ Install Vagrant from [here](https://www.vagrantup.com/docs/installation/)
168
+
169
+ ```bash
170
+ vagrant plugin install vagrant-vbguest # keep your VirtualBox Guest Additions up to date
171
+ vagrant plugin install vagrant-cachier # caches guest packages
172
+ vagrant plugin install vagrant-hostmanager # updates /etc/hosts file when machines go up/down
173
+ ```
174
+
175
+ #### vagrant-hostmanager
176
+ This plugin automatically updates the host's /etc/hosts file when vagrant machines go up/down
177
+
178
+ In order to do that it needs sudo password or sudo priviledges.
179
+ To avoid being asked for the password every time the hosts file is updated,
180
+ [enable passwordless sudo](https://github.com/devopsgroup-io/vagrant-hostmanager#passwordless-sudo)
181
+ for the specific command that hostmanager uses to update the hosts file
182
+
183
+
184
+ # Prepd Actors
185
+
186
+ A Client may have multiples projects. Applications share common infrastructure that is defined by the Project
187
+
188
+ - Client: An organization with one or more projects, e.g Acme Corp
189
+ - Project: A definition of infrastructure provided for one or more applications
190
+ - Application: A logical group of deployable repositories, e.g. a Rails API server and an Ember web client
191
+
192
+
193
+ ## Projects
194
+
195
+ - A project is comprised of Infrastructure Environments (IE) and Application Groups (AG)
196
+ - Infrastructure Environemnts are defined separately for each environment
197
+ - Application Groups are deployed into one or more Infrastructure EnvironmentS
198
+
199
+ ## Infrastructure Environments
200
+
201
+ Infrastructure is either Vagrant machines for development and local environments or EC2 instances for staging and production
202
+
203
+ Local, Staging and Production Environments use a Docker swarm network to manage applicaiton groups
204
+
205
+ - local: virtual machines running on laptop via vagrant whose primary purpose is application development
206
+ - development: primary purpose is also application development, but the infrastructure is deployed in the cloud (AWS)
207
+ - staging: a mirror of production in every way with the possible exception of reduced or part-time resources
208
+ - production: production ;-)
209
+
210
+ ## Applications
211
+
212
+ Applications are the content that actually gets deployed. The entire purpose of prepd is to provide a consistent
213
+ and easy to manage infrastructure for each environment into which the application will be deployed.
214
+
215
+
216
+ # Usage
217
+
218
+ ## New Client
219
+
220
+ This overview assumes a complete greenfield, e.g. that no infrastructure exists, no applications exist or even 3rd
221
+ party service have been setup. To start from zero, then:
222
+
223
+ - Create a new GH Organization
224
+ - Create an AWS Account and two IAM Groups: Administrators and ReadOnlyAdministrators
225
+ - Create a CI Account and give it access to the GH Organization
226
+ - Create a Docker Private Repository account and give it access to the GH Organization
227
+ - Create the project in prepd
228
+
229
+ The first four items are outside the scope of this document.
230
+
231
+ ```ruby
232
+ prepd
233
+ c = Client.create(name: 'Acme')
234
+ ```
235
+
236
+ ## New Project
237
+ - create a GH repo for the project
238
+ - create an IAM user for project_name-terraform and download the AWS credentials CSV
239
+ - create an IAM user for project_name-ansible and download the AWS credentials CSV
240
+ - use prepd to create the project using the repo_url and path names (tf_creds and ansible_creds) to CSV files
241
+
242
+ ```ruby
243
+ c = Client.find_by(name: 'Acme')
244
+ c.projects.new(name: 'widget', repo_url: 'git@github.com:my_git_hub_account/widget.git')
245
+ c.tf_creds = 'Users/dude/aws/widget-terraform.csv'
246
+ c.ansible_creds = 'Users/dude/aws/widget-ansible.csv'
247
+ c.save
248
+ ```
249
+
250
+ ## New Application
251
+
252
+ View the [lego README.md](https://github.com/rjayroach/lego) on creating micro serivce applications with Rails and Ember
253
+
254
+ ## Bring Up the Machine
255
+
256
+ ```ruby
257
+ cd ~/prepd/acme/widget
258
+ vagrant up
259
+ vagrant ssh
260
+ ```
261
+
262
+
263
+ # Credentials
264
+
265
+ ## Project Credentials
266
+ Prepd will create the following credential (hidden) files in project_root:
267
+
268
+ - .boto: AWS IAM credentials that give read only access to Ansible
269
+ - .developer.yml: Developer’s git account (and other account) details
270
+ - .terraform-vars.txt: AWS IAM credentials that give full access to CRUD AWS resources
271
+ - .vault-password.txt: a UUID used to encrypt and decrypt ansible vault files
272
+ - .id_rsa.pub: the public key uploaded to AWS as the primary key pair for accessing EC2 instances
273
+ - .id_rsa: the private key
274
+
275
+ - terraform will use project_root/id_rsa.pub to upload key_material to AWS for the machine key
276
+ - config-development.yml checks the project_root and: 1) if .boto exists link it, 2) if id_rsa and id_rsa.pub exist then link them
277
+ - the developer can then do ssh-add which will auto load ~/.ssh/id_rsa to login or run ansible
278
+
279
+
280
+ ## Transfer Credentials to New Machine
281
+
282
+ The prepd gem can encrypt the credentials using gpg which must be installed on the host machine
283
+
284
+ The encrypted credentials are written to and read from the user's home directory so that they are not accidentally
285
+ committed to the project repository
286
+
287
+ ### Encrypt
288
+
289
+ ```ruby
290
+ prepd
291
+ c = Client.find_by(name: 'Acme')
292
+ p = c.projects.find_by(name: 'widget')
293
+ p.encrypt
294
+ ```
295
+
296
+ This will create a tar file containing the various project credentials. It will then invoke gpg to encrypt the archive.
297
+ The credentials will be placed in the project's data directory
298
+
299
+ You will be prompted for a passphrase to enter twice. After doing that send the file by email or other mechanism
300
+
301
+ ### Decrypt
302
+
303
+ On the target machine, use prepd to decrypt the file and place it in the correct directory
304
+
305
+ - Clone the project repository
306
+ - Place the gpg tar file in the project's data directory
307
+ - Run prepd. It will expect to find the credentials file in the project's data directory
308
+
309
+ ```ruby
310
+ prepd
311
+ c = Client.find_by(name: 'Acme')
312
+ p = c.projects.find_by(name: 'widget')
313
+ p.decrypt
314
+ ```
315
+
316
+ ## Authorization
317
+
318
+ If giving a developer access to the machine for development only (not terraform or ansible) then add their public key to the
319
+ instance’s ~/.ssh/authorized_keys. The developer uses ssh-agent forwarding to access the machine from the VM
320
+
321
+
322
+ # Development
323
+
324
+ After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
325
+
326
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
327
+
328
+ # Contributing
329
+
330
+ Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/prepd. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
331
+
332
+
333
+ # License
334
+
335
+ The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,2 @@
1
+ require "bundler/gem_tasks"
2
+ task :default => :spec
data/TODO.md ADDED
@@ -0,0 +1,17 @@
1
+ # TODO
2
+
3
+ - option to just gpg encrypt ansible-vault.txt rather than the full set of credentials
4
+ - move gpg files to the project's data dir instead of user's home dir
5
+ - add an option to tar, gizp and gpg the data directory as well
6
+
7
+ - update prepd-project readme when cloning existing to also pull in ansible-roles
8
+
9
+
10
+ - create a bootstrap.sh script that installs Ansible and dependencies, e.g. Homebrew, python, etc
11
+ - add a playbook that provisions a mac or ubuntu laptop with android, packer, extras, etc
12
+
13
+ - update README to document provisioning a mac (and ubuntu) from brand new:
14
+ 1. git clone prepd
15
+ 2. run bootstrap.sh
16
+ 3. run ./laptop.yml
17
+ 4. bin/console to create a client and project OR git clone a project created with prepd
data/bin/console ADDED
@@ -0,0 +1,4 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'bundler/setup'
4
+ require 'prepd/cli'
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
data/exe/prepd ADDED
@@ -0,0 +1,6 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "prepd"
4
+
5
+ require 'bundler/setup'
6
+ require 'prepd/cli'
@@ -0,0 +1,37 @@
1
+ module Prepd
2
+ def self.options=(options)
3
+ @options = options
4
+ end
5
+ def self.options; @options; end
6
+
7
+ def self.commands
8
+ puts (methods(false) - %i(:options= :options :commands default_settings)).join("\n")
9
+ end
10
+
11
+ def self.new(name)
12
+ Client.create(name: name)
13
+ end
14
+
15
+ def self.rm
16
+ FileUtils.rm_rf(work_dir)
17
+ FileUtils.rm_rf(data_dir)
18
+ end
19
+
20
+ def self.clients; Client.pluck(:name); end
21
+
22
+ def self.projects; Project.pluck(:name); end
23
+
24
+ def self.current_client
25
+ @client
26
+ end
27
+
28
+ def self.current_client=(client)
29
+ STDOUT.puts 'duh'
30
+ @client = client
31
+ Dir.chdir(client.path) do
32
+ Pry.start(client, prompt: [proc { "prepd(#{client.name}) > " }])
33
+ end
34
+ STDOUT.puts 'duh2'
35
+ nil
36
+ end
37
+ end
@@ -0,0 +1,42 @@
1
+ require 'optparse'
2
+
3
+ module Prepd::Cli
4
+ class OptionsParser
5
+ attr_accessor :options
6
+
7
+ def initialize(options = nil)
8
+ self.options = options || {}
9
+ end
10
+
11
+ def parse
12
+ optparse = OptionParser.new do |opts|
13
+ opts.on('-c', '--client [OPT]', 'Client') do |value|
14
+ options['CLIENT'] = value
15
+ end
16
+
17
+ opts.on( '-d', '--data_dir [OPT]', 'Data directory' ) do |value|
18
+ options['DATA_DIR'] = value
19
+ end
20
+
21
+ opts.on( '-p', '--project [OPT]', 'Project' ) do |value|
22
+ options['PROJECT'] = value
23
+ end
24
+
25
+ opts.on('-h', '--help', 'Display this screen') do
26
+ puts opts
27
+ exit
28
+ end
29
+
30
+ opts.on('-n', '--no-op', 'Show what would happen but do not execute') do
31
+ options.no_op = true
32
+ end
33
+
34
+ opts.on('-v', '--verbose', 'Display additional information') do
35
+ options.verbose = true
36
+ end
37
+ end
38
+ optparse.parse!
39
+ options
40
+ end
41
+ end
42
+ end
data/lib/prepd/cli.rb ADDED
@@ -0,0 +1,8 @@
1
+ require 'pry'
2
+ require 'prepd'
3
+ require 'prepd/cli/options_parser'
4
+ require 'prepd/cli/commands'
5
+
6
+ Prepd.options = Prepd.default_settings
7
+ Prepd.options.merge!(Prepd::Cli::OptionsParser.new.parse)
8
+ Pry.start(Prepd, prompt: [proc { 'prepd> '}])
@@ -0,0 +1,261 @@
1
+ module Prepd
2
+ class Client < ActiveRecord::Base
3
+ attr_accessor :data_dir
4
+
5
+ has_many :projects, dependent: :destroy
6
+ has_many :applications, through: :projects
7
+
8
+ before_validation :set_defaults
9
+ validates :name, :path, presence: true
10
+
11
+ after_create :setup
12
+ after_destroy :destroy_client
13
+
14
+ def set_defaults
15
+ self.path = "#{Prepd.options['DATA_DIR']}/#{name}"
16
+ end
17
+
18
+ def setup
19
+ FileUtils.mkdir_p(path) unless Dir.exists?(path)
20
+ end
21
+
22
+ def destroy_client
23
+ FileUtils.rm_rf("#{path}")
24
+ end
25
+ end
26
+
27
+
28
+ class Project < ActiveRecord::Base
29
+ attr_accessor :tf_creds, :tf_key, :tf_secret, :ansible_creds, :ansible_key, :ansible_secret
30
+
31
+ belongs_to :client, required: true
32
+ has_many :applications, dependent: :destroy
33
+
34
+ validates :name, presence: true, uniqueness: { scope: :client }
35
+
36
+ after_create :create_project
37
+ after_destroy :destroy_project
38
+
39
+ #
40
+ # Initialize the prepd-project or just copy in developer credentials if the project already exists
41
+ #
42
+ def create_project
43
+ if Dir.exists?(path)
44
+ copy_developer_yml
45
+ return
46
+ end
47
+ setup_git
48
+ clone_submodules
49
+ copy_developer_yml
50
+ generate_credentials
51
+ encrypt_vault_files
52
+ end
53
+
54
+ #
55
+ # Destory the VM and remove the project from the file system
56
+ #
57
+ def destroy_project
58
+ Dir.chdir(path) { system('vagrant destroy') }
59
+ FileUtils.rm_rf(path)
60
+ end
61
+
62
+ #
63
+ # Clone prepd-project, remove the git history and start with a clean repository
64
+ #
65
+ def setup_git
66
+ Dir.chdir(client.path) { system("git clone git@github.com:rjayroach/prepd-project.git #{name}") }
67
+ Dir.chdir(path) do
68
+ FileUtils.rm_rf("#{path}/.git")
69
+ system('git init')
70
+ system('git add .')
71
+ system("git commit -m 'First commit from Prepd'")
72
+ system("git remote add origin #{repo_url}") if repo_url
73
+ end
74
+ end
75
+
76
+ #
77
+ # Clone ansible roles and terraform modules
78
+ #
79
+ def clone_submodules
80
+ Dir.chdir("#{path}/ansible") do
81
+ system('git submodule add git@github.com:rjayroach/ansible-roles.git roles')
82
+ end
83
+ Dir.chdir("#{path}/terraform") do
84
+ system('git submodule add git@github.com:rjayroach/terraform-modules.git modules')
85
+ end
86
+ end
87
+
88
+ #
89
+ # Copy developer credentials or create them if the file doesn't already exists
90
+ # TODO: Maybe the creation of developer creds should be done at startup of prepd
91
+ #
92
+ def copy_developer_yml
93
+ return if File.exists?("#{path}/.developer.yml")
94
+ Dir.chdir(path) do
95
+ if File.exists?("#{Prepd.work_dir}/developer.yml")
96
+ FileUtils.cp("#{Prepd.work_dir}/developer.yml", '.developer.yml')
97
+ elsif File.exists?("#{Dir.home}/.prepd-developer.yml")
98
+ FileUtils.cp("#{Dir.home}/.prepd-developer.yml", '.developer.yml')
99
+ else
100
+ File.open('.developer.yml', 'w') do |f|
101
+ f.puts('---')
102
+ f.puts("git_username: #{`git config --get user.name`.chomp}")
103
+ f.puts("git_email: #{`git config --get user.email`.chomp}")
104
+ f.puts("docker_username: ")
105
+ f.puts("docker_password: ")
106
+ end
107
+ end
108
+ end
109
+ end
110
+
111
+ #
112
+ # Create AWS credential files for Terraform and Ansible, ssh keys and and ansible-vault encryption key
113
+ # NOTE: The path to credentials is used in the ansible-role prepd
114
+ #
115
+ def generate_credentials
116
+ # self.tf_creds = '/Users/rjayroach/Documents/c2p4/aws/legos-terraform.csv'
117
+ # self.ansible_creds = '/Users/rjayroach/Documents/c2p4/aws/legos-ansible.csv'
118
+ generate_tf_creds
119
+ generate_ansible_creds
120
+ generate_ssh_keys
121
+ generate_vault_password
122
+ end
123
+
124
+ def generate_tf_creds
125
+ self.tf_key, self.tf_secret = CSV.read(tf_creds).last.slice(2,2) if tf_creds
126
+ unless tf_key and tf_secret
127
+ STDOUT.puts 'tf_key and tf_secret need to be set (or set tf_creds to path to CSV file)'
128
+ return
129
+ end
130
+ require 'csv'
131
+ Dir.chdir(path) do
132
+ File.open('.terraform-vars.txt', 'w') do |f|
133
+ f.puts("aws_access_key_id = \"#{tf_key}\"")
134
+ f.puts("aws_secret_access_key = \"#{tf_secret}\"")
135
+ end
136
+ end
137
+ end
138
+
139
+ def generate_ansible_creds
140
+ self.ansible_key, self.ansible_secret = CSV.read(ansible_creds).last.slice(2,2) if ansible_creds
141
+ unless ansible_key and ansible_secret
142
+ STDOUT.puts 'ansible_key and ansible_secret need to be set (or set ansible_creds to path to CSV file)'
143
+ return
144
+ end
145
+ Dir.chdir(path) do
146
+ File.open('.boto', 'w') do |f|
147
+ f.puts('[Credentials]')
148
+ f.puts("aws_access_key_id = #{ansible_key}")
149
+ f.puts("aws_secret_access_key = #{ansible_secret}")
150
+ end
151
+ end
152
+ end
153
+
154
+ #
155
+ # Generate a key pair to be used as the EC2 key pair
156
+ #
157
+ def generate_ssh_keys(file_name = '.id_rsa')
158
+ Dir.chdir(path) { system("ssh-keygen -b 2048 -t rsa -f #{file_name} -q -N '' -C 'ansible@#{name}.#{client.name}.local'") }
159
+ end
160
+
161
+ #
162
+ # Generate the key to encrypt ansible-vault files
163
+ #
164
+ def generate_vault_password(file_name = '.vault-password.txt')
165
+ require 'securerandom'
166
+ Dir.chdir(path) { File.open(file_name, 'w') { |f| f.puts(SecureRandom.uuid) } }
167
+ end
168
+
169
+ #
170
+ # Use ansible-vault to encrypt the inventory group_vars
171
+ #
172
+ def encrypt_vault_files
173
+ Dir.chdir("#{path}/ansible") do
174
+ %w(all development local production staging).each do |env|
175
+ system("ansible-vault encrypt inventory/group_vars/#{env}/vault")
176
+ end
177
+ end
178
+ end
179
+
180
+ def encrypt(mode = :vault)
181
+ return unless executable?('gpg')
182
+ Dir.chdir(path) do
183
+ system "tar cf #{archive(:credentials)} #{file_list(mode)}"
184
+ end
185
+ system "gpg -c #{archive(:credentials)}"
186
+ FileUtils.rm(archive(:credentials))
187
+ "File created: #{archive(:credentials)}.gpg"
188
+ end
189
+
190
+ def encrypt_data
191
+ return unless executable?('gpg')
192
+ archive_path = "#{path}/#{client.name}-#{name}-data.tar"
193
+ Dir.chdir(path) do
194
+ system "tar cf #{archive_path} data"
195
+ end
196
+ system "gpg -c #{archive_path}"
197
+ FileUtils.rm(archive_path)
198
+ FileUtils.mv("#{archive_path}.gpg", "#{archive(:data)}.gpg")
199
+ "File created: #{archive(:data)}.gpg"
200
+ end
201
+
202
+ def decrypt(type = :credentials)
203
+ return unless %i(credentials data).include? type
204
+ return unless executable?('gpg')
205
+ unless File.exists?("#{archive(type)}.gpg")
206
+ STDOUT.puts "File not found: #{archive(type)}.gpg"
207
+ return
208
+ end
209
+ system "gpg #{archive(type)}.gpg"
210
+ Dir.chdir(path) do
211
+ system "tar xf #{archive(type)}"
212
+ end
213
+ FileUtils.rm(archive(type))
214
+ "File processed: #{archive(type)}.gpg"
215
+ end
216
+
217
+ def executable?(name = 'gpg')
218
+ require 'mkmf'
219
+ rv = find_executable(name)
220
+ STDOUT.puts "#{name} executable not found" unless rv
221
+ FileUtils.rm('mkmf.log')
222
+ rv
223
+ end
224
+
225
+ def file_list(mode)
226
+ return ".boto .id_rsa .id_rsa.pub .terraform-vars.txt .vault-password.txt" if mode.eql?(:all)
227
+ ".vault-password.txt"
228
+ end
229
+
230
+ def archive(type = :credentials)
231
+ "#{data_path}/#{client.name}-#{name}-#{type}.tar"
232
+ end
233
+
234
+ def data_path
235
+ "#{path}/data"
236
+ end
237
+
238
+ def path
239
+ "#{client.path}/#{name}"
240
+ end
241
+ end
242
+
243
+
244
+ class Application < ActiveRecord::Base
245
+ belongs_to :project, required: true
246
+
247
+ validates :name, presence: true, uniqueness: { scope: :project }
248
+
249
+ after_create :setup
250
+
251
+ def setup
252
+ Dir.chdir("#{project.path}/ansible") do
253
+ FileUtils.cp_r('application', name)
254
+ end
255
+ end
256
+
257
+ def path
258
+ "#{project.path}/ansible/#{name}"
259
+ end
260
+ end
261
+ end
@@ -0,0 +1,23 @@
1
+ ActiveRecord::Schema.define do
2
+ unless ActiveRecord::Base.connection.data_sources.include?('clients')
3
+ create_table :clients do |table|
4
+ table.column :name, :string
5
+ table.column :path , :string
6
+ end
7
+ end
8
+
9
+ unless ActiveRecord::Base.connection.data_sources.include?('projects')
10
+ create_table :projects do |table|
11
+ table.column :client_id, :integer # foreign key <table-name-singular>_id
12
+ table.column :name, :string
13
+ table.column :repo_url, :string
14
+ end
15
+ end
16
+
17
+ unless ActiveRecord::Base.connection.data_sources.include?('applications')
18
+ create_table :applications do |table|
19
+ table.column :project_id, :integer # foreign key <table-name-singular>_id
20
+ table.column :name, :string
21
+ end
22
+ end
23
+ end
@@ -0,0 +1,3 @@
1
+ module Prepd
2
+ VERSION = '0.1.1'
3
+ end
data/lib/prepd.rb ADDED
@@ -0,0 +1,47 @@
1
+ require 'prepd/version'
2
+ require 'dotenv'
3
+ require 'active_record'
4
+ require 'sqlite3'
5
+ require 'fileutils'
6
+
7
+ module Prepd
8
+ def self.work_dir; "#{Dir.home}/.prepd"; end
9
+ def self.data_dir; ENV['DATA_DIR']; end
10
+
11
+ def self.files; Dir.glob("#{work_dir}/*"); end
12
+
13
+ def self.config; "#{work_dir}/config"; end
14
+
15
+ def self.default_settings
16
+ {
17
+ 'VERSION' => '1',
18
+ 'DATA_DIR' => "#{Dir.home}/prepd",
19
+ 'VAGRANT_BASE_BOX' => 'debian/contrib-jessie64'
20
+ }
21
+ end
22
+
23
+ # Create records for exisitng directories in the DATA_DIR
24
+ def self.scan
25
+ clients = Dir.entries(ENV['DATA_DIR'])
26
+ clients.select { |entry| !entry.starts_with?('.') }.each do |client_name|
27
+ c = Client.find_or_create_by(name: client_name)
28
+ projects = Dir.entries("#{ENV['DATA_DIR']}/#{client_name}")
29
+ projects.select { |entry| !entry.starts_with?('.') }.each do |project_name|
30
+ c.projects.find_or_create_by(name: project_name)
31
+ end
32
+ end
33
+ end
34
+
35
+ FileUtils.mkdir_p work_dir
36
+ ActiveRecord::Base.logger = Logger.new(File.open("#{work_dir}/database.log", 'w'))
37
+ ActiveRecord::Base.establish_connection(adapter: :sqlite3, database: "#{work_dir}/sqlite.db")
38
+ unless File.exists?(config)
39
+ File.open(config, 'a') do |f|
40
+ default_settings.each { |key, value| f.puts("#{key}=#{value}") }
41
+ end
42
+ end
43
+ Dotenv.load(config)
44
+ end
45
+
46
+ require 'prepd/schema'
47
+ require 'prepd/models'
data/prepd.gemspec ADDED
@@ -0,0 +1,31 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'prepd/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = 'prepd'
8
+ spec.version = Prepd::VERSION
9
+ spec.authors = ['Robert Roach']
10
+ spec.email = ['rjayroach@gmail.com']
11
+
12
+ spec.summary = %q{An easy to use tool to create Production Ready Environments for Project Development}
13
+ spec.description = %q{Prepd assists builders of web application products to start with the end in mind by making it easy to stand up all required infrastructure
14
+ *before* starting to code the application}
15
+ spec.homepage = 'https://github.com/rjayroach/prepd-gem'
16
+ spec.license = 'MIT'
17
+
18
+ spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
19
+ spec.bindir = 'exe'
20
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
21
+ spec.require_paths = ['lib']
22
+
23
+ spec.add_development_dependency 'bundler', '~> 1.12'
24
+ spec.add_development_dependency 'rake', '~> 10.0'
25
+
26
+ spec.add_dependency 'dotenv'
27
+ spec.add_dependency 'activerecord'
28
+ spec.add_dependency 'sqlite3'
29
+ spec.add_dependency 'pry'
30
+ spec.add_dependency 'awesome_print'
31
+ end
metadata ADDED
@@ -0,0 +1,163 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: prepd
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.1
5
+ platform: ruby
6
+ authors:
7
+ - Robert Roach
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2017-11-02 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: bundler
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.12'
20
+ type: :development
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.12'
27
+ - !ruby/object:Gem::Dependency
28
+ name: rake
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '10.0'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '10.0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: dotenv
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :runtime
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: activerecord
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: sqlite3
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :runtime
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: pry
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :runtime
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: awesome_print
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ type: :runtime
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
111
+ description: |-
112
+ Prepd assists builders of web application products to start with the end in mind by making it easy to stand up all required infrastructure
113
+ *before* starting to code the application
114
+ email:
115
+ - rjayroach@gmail.com
116
+ executables:
117
+ - prepd
118
+ extensions: []
119
+ extra_rdoc_files: []
120
+ files:
121
+ - ".gitignore"
122
+ - CODE_OF_CONDUCT.md
123
+ - Gemfile
124
+ - LICENSE.txt
125
+ - README.md
126
+ - Rakefile
127
+ - TODO.md
128
+ - bin/console
129
+ - bin/setup
130
+ - exe/prepd
131
+ - lib/prepd.rb
132
+ - lib/prepd/cli.rb
133
+ - lib/prepd/cli/commands.rb
134
+ - lib/prepd/cli/options_parser.rb
135
+ - lib/prepd/models.rb
136
+ - lib/prepd/schema.rb
137
+ - lib/prepd/version.rb
138
+ - prepd.gemspec
139
+ homepage: https://github.com/rjayroach/prepd-gem
140
+ licenses:
141
+ - MIT
142
+ metadata: {}
143
+ post_install_message:
144
+ rdoc_options: []
145
+ require_paths:
146
+ - lib
147
+ required_ruby_version: !ruby/object:Gem::Requirement
148
+ requirements:
149
+ - - ">="
150
+ - !ruby/object:Gem::Version
151
+ version: '0'
152
+ required_rubygems_version: !ruby/object:Gem::Requirement
153
+ requirements:
154
+ - - ">="
155
+ - !ruby/object:Gem::Version
156
+ version: '0'
157
+ requirements: []
158
+ rubyforge_project:
159
+ rubygems_version: 2.5.2.1
160
+ signing_key:
161
+ specification_version: 4
162
+ summary: An easy to use tool to create Production Ready Environments for Project Development
163
+ test_files: []