yle_tf 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: b756a2e77feec940d0faac15ac402b1b624e0cbc
+  data.tar.gz: 6202a1cbe6e3aba3337b908bbb0c72cbd56eaef2
+SHA512:
+  metadata.gz: 3d417c9827c142b9d38d08e82f2574771edd50f664c4a4d4e8a5dec0edf7c3d9facca02efad74a56a2813e1b168b71c202743f961b13d27c42b3807f5e10ddc2
+  data.tar.gz: '09842437220a7b72909c19b951219b798b30def5c41fa59e06502ccac760740f75f9e90a0c03951babd5db7b2c9598fa14fa00a94672e7877bf716c2c08657b2'

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.rspec

@@ -0,0 +1 @@
+--color

data/.rubocop.yml

@@ -0,0 +1,26 @@
+Metrics/BlockLength:
+  Exclude:
+    - 'spec/**/*'
+
+Metrics/ClassLength:
+  Severity: warning
+
+Metrics/LineLength:
+  Max: 100
+  Severity: warning
+
+Metrics/MethodLength:
+  Max: 12
+  Severity: warning
+
+Style/Documentation:
+  Enabled: false
+
+Style/GuardClause:
+  MinBodyLength: 3
+
+Style/NegatedIf:
+  Enabled: false
+
+Style/TrailingCommaInLiteral:
+  Enabled: false

data/.travis.yml

@@ -0,0 +1,7 @@
+language: ruby
+cache: bundler
+
+rvm:
+  - 2.4.1
+  - 2.3.4
+  - 2.2.7

data/CHANGELOG.md

@@ -0,0 +1,3 @@
+## 0.1.0 / 2017-08-29
+
+- Initial public release

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at teemu.matilainen@reaktor.fi. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in yle_tf.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016-2017 Yleisradio Oy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,442 @@
+# YleTf
+
+A wrapper for [Terraform](https://www.terraform.io/) with support for hooks and environments.
+
+Offers a command-line helper tool, `tf`
+
+## Getting started
+
+YleTf requires Terraform which can be installed according to their [instructions](https://www.terraform.io/intro/getting-started/install.html). On MacOS (and OSX) you can use [Homebrew](https://brew.sh/). You can also easily install and manage multiple versions of Terraform with [homebrew-terraforms](https://github.com/Yleisradio/homebrew-terraforms).
+
+## Installation
+
+YleTf can be installed as a gem:
+
+```sh
+$ gem install yle_tf
+```
+
+## Usage
+
+The syntax to run YleTf is much like vanilla [Terraform](https://terraform.io). The main difference is that you must include desired environment as the first argument:
+
+```
+tf <environment> <command> [<args>]
+```
+
+For example:
+
+```
+$ tf test plan
+
+$ tf prod apply
+
+$ tf stage destroy
+```
+
+For a full list of available options, run `tf --help`.
+
+When `tf` is executed, it creates a temporary directory where your project is copied and initialized.
+
+### Example project
+
+The following is a really basic example on what you need in order to use YleTf. The project is pretty much the same as the [example project](https://www.terraform.io/intro/getting-started/build.html) in Terraform documentation but with the added support for environments. Introduction to [hooks](#hooks) is in it's own section.
+
+#### Project Config
+
+The root of your project directory will look like this:
+
+```
+.
+├── envs
+│   ├── prod.tfvars
+│   └── test.tfvars
+├── main.tf
+├── tf.yaml
+└── variables.tf
+```
+
+And here are the contents of the files:
+
+##### main.tf
+
+```hcl
+provider "aws" {
+  access_key = "${var.aws_access_key}"
+  secret_key = "${var.aws_secret_key}"
+  region     = "${var.region}"
+}
+
+resource "aws_instance" "example" {
+  ami           = "ami-2757f631"
+  instance_type = "t2.micro"
+}
+```
+
+_Please note that the AMI identifier changes based on your desired OS and region. For more information see [AWS documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html)._
+
+##### variables.tf
+
+```hcl
+variable "region" {
+  description = "The AWS region for the resources"
+  default     = "eu-west-1"
+}
+
+variable "env" {
+  # passed by `tf`
+  description = "The environment"
+}
+
+variable "aws_access_key" {
+  description = "Your AWS access key id"
+}
+
+variable "aws_secret_key" {
+  description = "Your AWS secret key"
+}
+
+variable "instance_type" {
+  description = "Instance type to use"
+  default = "t2.micro"
+}
+```
+
+##### envs/test.tfvars
+
+```ini
+aws_access_key = "TEST_ACCOUNT_ACCESS_KEY_HERE"
+aws_secret_key = "TEST_ACCOUNT_SECRET_KEY_HERE"
+```
+
+##### tf.yaml
+
+```yaml
+backend:
+  type: file
+terraform:
+  version_requirement: "~> 0.9.11"
+```
+
+#### Execution
+
+With all the above in order, there's nothing more to do than to try and run the commands:
+
+First plan:
+
+```
+$ tf test plan
+INFO: Symlinking state to '/usr/local/src/yle_tf/examples/examples_test.tfstate'
+
+Terraform has been successfully initialized!
+
+You may now begin working with Terraform. Try running "terraform plan" to see
+any changes that are required for your infrastructure. All Terraform commands
+should now work.
+
+If you ever set or change modules or backend configuration for Terraform,
+rerun this command to reinitialize your environment. If you forget, other
+commands will detect it and remind you to do so if necessary.
+Refreshing Terraform state in-memory prior to plan...
+The refreshed state will be used to calculate this plan, but will not be
+persisted to local or remote state storage.
+
+The Terraform execution plan has been generated and is shown below.
+Resources are shown in alphabetical order for quick scanning. Green resources
+will be created (or destroyed and then created if an existing resource
+exists), yellow resources are being changed in-place, and red resources
+will be destroyed. Cyan entries are data sources to be read.
+
+Note: You didn't specify an "-out" parameter to save this plan, so when
+"apply" is called, Terraform can't guarantee this is what will execute.
+
++ aws_instance.example
+    ami:                          "ami-d7b9a2b1"
+    associate_public_ip_address:  "<computed>"
+    availability_zone:            "<computed>"
+    ebs_block_device.#:           "<computed>"
+    ephemeral_block_device.#:     "<computed>"
+    instance_state:               "<computed>"
+    instance_type:                "t2.micro"
+    ipv6_address_count:           "<computed>"
+    ipv6_addresses.#:             "<computed>"
+    key_name:                     "<computed>"
+    network_interface.#:          "<computed>"
+    network_interface_id:         "<computed>"
+    placement_group:              "<computed>"
+    primary_network_interface_id: "<computed>"
+    private_dns:                  "<computed>"
+    private_ip:                   "<computed>"
+    public_dns:                   "<computed>"
+    public_ip:                    "<computed>"
+    root_block_device.#:          "<computed>"
+    security_groups.#:            "<computed>"
+    source_dest_check:            "true"
+    subnet_id:                    "<computed>"
+    tenancy:                      "<computed>"
+    volume_tags.%:                "<computed>"
+    vpc_security_group_ids.#:     "<computed>"
+
+
+Plan: 1 to add, 0 to change, 0 to destroy.
+```
+
+Then apply:
+
+```
+$ tf test apply
+
+< SNIP >
+
+aws_instance.example: Creating...
+  ami:                          "" => "ami-d7b9a2b1"
+  associate_public_ip_address:  "" => "<computed>"
+  availability_zone:            "" => "<computed>"
+  ebs_block_device.#:           "" => "<computed>"
+  ephemeral_block_device.#:     "" => "<computed>"
+  instance_state:               "" => "<computed>"
+  instance_type:                "" => "t2.micro"
+  ipv6_address_count:           "" => "<computed>"
+  ipv6_addresses.#:             "" => "<computed>"
+  key_name:                     "" => "<computed>"
+  network_interface.#:          "" => "<computed>"
+  network_interface_id:         "" => "<computed>"
+  placement_group:              "" => "<computed>"
+  primary_network_interface_id: "" => "<computed>"
+  private_dns:                  "" => "<computed>"
+  private_ip:                   "" => "<computed>"
+  public_dns:                   "" => "<computed>"
+  public_ip:                    "" => "<computed>"
+  root_block_device.#:          "" => "<computed>"
+  security_groups.#:            "" => "<computed>"
+  source_dest_check:            "" => "true"
+  subnet_id:                    "" => "<computed>"
+  tenancy:                      "" => "<computed>"
+  volume_tags.%:                "" => "<computed>"
+  vpc_security_group_ids.#:     "" => "<computed>"
+aws_instance.example: Still creating... (10s elapsed)
+aws_instance.example: Still creating... (20s elapsed)
+aws_instance.example: Creation complete (ID: i-0f1327bbc53fd6bab)
+
+Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
+
+< SNIP >
+```
+
+And when you're done using the infrastructure, destroy:
+
+```
+$ tf test destroy
+
+< SNIP >
+
+Do you really want to destroy?
+  Terraform will delete all your managed infrastructure.
+  There is no undo. Only 'yes' will be accepted to confirm.
+
+  Enter a value: yes
+
+aws_instance.example: Refreshing state... (ID: i-0f1327bbc53fd6bab)
+aws_instance.example: Destroying... (ID: i-0f1327bbc53fd6bab)
+aws_instance.example: Still destroying... (ID: i-0f1327bbc53fd6bab, 10s elapsed)
+aws_instance.example: Still destroying... (ID: i-0f1327bbc53fd6bab, 20s elapsed)
+aws_instance.example: Still destroying... (ID: i-0f1327bbc53fd6bab, 30s elapsed)
+aws_instance.example: Still destroying... (ID: i-0f1327bbc53fd6bab, 40s elapsed)
+aws_instance.example: Still destroying... (ID: i-0f1327bbc53fd6bab, 50s elapsed)
+aws_instance.example: Still destroying... (ID: i-0f1327bbc53fd6bab, 1m0s elapsed)
+aws_instance.example: Still destroying... (ID: i-0f1327bbc53fd6bab, 1m10s elapsed)
+aws_instance.example: Destruction complete
+
+Destroy complete! Resources: 1 destroyed.
+```
+
+#### Production environment
+
+Now that we've tried that everything works in the test environment, let's do the same in production. Note that you can override variables if needed:
+
+##### envs/prod.tfvars
+
+```ini
+aws_access_key = "PRODUCTION_ACCOUNT_ACCESS_KEY_HERE"
+aws_secret_key = "PRODUCTION_ACCOUNT_SECRET_KEY_HERE"
+
+# let's use a bigger instance type in prod
+instance_type = "t2.medium"
+```
+
+And run the commands just like with _test_:
+
+```
+$ tf prod plan
+
+< SNIP >
+
+$ tf prod apply
+
+< SNIP >
+
+$ tf prod destroy
+
+< SNIP >
+```
+
+_Note that the instance type is now t2.medium_
+
+## Default components
+
+### tf.yaml
+
+Here be dragons that configure your project. The tools serches for `tf.yaml`'s up your path all the way to root `/`. Configs made in subdirectories override those made upper in the path. This makes it easy to define common settings without having to edit `tf.yaml` in every project.
+
+By defalt the following configuration options are supported:
+* [`hooks`](#hooks) - Pre and Post hooks.
+* [`backend`](#backend) - Configuration of remote state location.
+* [`terraform`](#terraform) - Terraform specific configuration.
+
+#### Hooks
+
+There are cases when it would be beneficial to run a task at the same time as terrafrom, but building native support for that would be quite cumbersome. The support for hooks was build into YleTf having those cases in mind.
+
+Essentially hooks are just scripts and small applications to extend the functionality of Terraform. Hooks can be either
+* [local](#local-hooks) or
+* [remote](#remote-hooks) i.e. stored into git.
+
+Real world usecases for hooks include at least the following:
+* Automatically register ACM certificates and link them to desired resources
+* Automatically generate dns record resources that are managed by separate configuration
+* Package lambda applications for deployment via Terraform
+* Manage SES authorisations and rules
+* Modify parameters not yet supported by Terraform
+
+Currently two kinds of hooks are supported:
+* `pre` - Hooks that run before terraform execution.
+* `post` - Hooks that run after terraform execution.
+
+#### Local hooks
+
+For local hooks, add a directory called `tf_hooks` to your tf project root. You also need a folder to determine wether the hook is run before or after the execution of terraform. The folders are `pre` and/or `post`:
+
+```
+.
+├── envs
+│   ├── prod.tfvars
+│   └── test.tfvars
+├── main.tf
+├── tf.yaml
+├── tf_hooks
+│   ├── pre
+│   │   └── pre-hook.sh
+│   └── post
+│   │   └── post-hook.rb
+└── variables.tf
+```
+
+For example, let's say you wish to have a variable `current_git_hash` and you want to populate it with a value of the latest git commit hash. The hook could be something like this:
+
+##### tf_hooks/pre/get_current_hash.sh
+
+```bash
+#!/bin/bash
+
+set -eu
+
+current_hash() {
+  CURRENT_GIT_HASH="$(git rev-parse HEAD)"
+
+  cat <<EOF > current_git_hash.tf.json
+{
+  "variable": {
+    "current_git_hash": {
+      "value": "$CURRENT_GIT_HASH"
+    }
+  }
+}
+EOF
+}
+
+# Execute only for commands `plan` and `apply`
+case "$TF_COMMAND" in
+  plan|apply)
+    current_hash
+    ;;
+  *)
+    ;;
+esac
+```
+
+_Please note that the script in the [examples](examples/) directory is intentionnally without the execute bit._
+
+Once you set the hook script executable (`chmod +x tf_hooks/pre/get_current_hash.sh`) and run `tf` with either `plan` or `apply`, you'll have the following file during runtime:
+
+##### current_git_hash.tf.json
+
+```json
+{
+  "variable": {
+    "current_git_hash": {
+      "value": "c6a02baf0597e55d7698f78d70299ca4a65776cd"
+    }
+  }
+}
+```
+
+_Naturraly the actual value varies according to your repository._
+
+_**Please note that files generated into the temporary working directory while running hooks are removed once the execution has ended.**_
+
+#### Remote hooks
+
+Hooks can also be stored in common git repositories. This is a handy way to avoid duplication in codebase.
+To use a hook from git just add your script to a suitable repository (making sure it's still executable) and add following config to your `tf.yaml`:
+
+##### tf.yaml
+
+```yaml
+hooks:
+  pre:
+    - description: "Get your current git commit hash"
+      source: "git@github.com:your-org/tf-hooks.git//git-hash/get_current_hash.sh"
+      vars:
+        defaults:
+          MESSAGE: "You can also define environment variables to your hooks"
+        test:
+          MESSAGE: "And the variables can also be environment spesific"
+```
+
+#### Backend
+
+Configure where Terraform [remote state](https://www.terraform.io/docs/state/remote.html) is stored
+
+* `type` - Backend type where the Terraform state is stored.
+* `bucket` - The name of the S3 bucket.
+* `file` - The name of the state file.
+* `region` - The region of the S3 bucket.
+* `encrypt` - Whether to enable server side encryption of the state file.
+
+```yaml
+backend:
+  type: s3
+```
+
+#### Terraform
+
+* `version_requirement` - The version requirement of Terraform in ruby gem syntax.
+
+```yaml
+terraform:
+  version_requirement: "~> 0.9.11"
+```
+
+## Development
+
+After checking out the repo, run `bundle update` to install and update the dependencies. Then, run `bundle exec rake spec` to run the tests.
+
+To install this gem onto your local machine, run `bundle exec rake install`.
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/Yleisradio/yle_tf. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).