inspec-iggy 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 3dc2db85eac33aa51e03baf7f0df30858a7f1815
+  data.tar.gz: e44f8c04a3998030335b6a2dcf2ff225c7b18d4f
+SHA512:
+  metadata.gz: 9c8086684b141e0590b9a55f1b135922a4c9f4f7df3c3c32ec6422c890c7fe24a61443b16eac2e35bedb02f3a031a7efc7586a81d378addcd4c0d52d5d6de037
+  data.tar.gz: cda93559f8d2c47aaa4447d51d6f2151a006e3249e1297f379d041402c0f168f209acfda6003e42e30859ad901063e1aa0c5973127306ef7fdfec29ef7c058de

data/Gemfile

@@ -0,0 +1,15 @@
+# encoding: utf-8
+source "http://rubygems.org"
+
+gemspec
+
+group :test do
+  gem "bundler"
+  gem "rake"
+  gem "chefstyle"
+end
+
+group :tools do
+  gem "github_changelog_generator"
+  gem "rb-readline"
+end

data/README.md

@@ -0,0 +1,114 @@
+# Description #
+
+InSpec-Iggy (InSpec Generate -> "IG" -> "Iggy") is an [InSpec](https://inspec.io) plugin for generating compliance controls and profiles from [Terraform](https://terraform.io) ```tfstate``` files. Iggy generates InSpec AWS controls by mapping Terraform resources to InSpec resources. You may also use tags to annotate your Terraform scripts to specify which compliance profiles to be used and Iggy will create a profile including those dependencies.
+
+Iggy was originally a stand-alone CLI inspired by Christoph Hartmann's [inspec-verify-provision](https://github.com/chris-rock/inspec-verify-provision) and the blog post on testing [Terraform with InSpec](http://lollyrock.com/articles/inspec-terraform/).
+
+The [CHANGELOG.md](https://github.com/mattray/iggy/blob/master/CHANGELOG.md) covers current, previous and future development milestones and contains the features backlog.
+
+# Requirements #
+
+Iggy generates compliance profiles for InSpec 2, which includes the AWS and Azure resources. Because resources are continuing to be added to InSpec, you may want the latest version to support as many resource coverage as possible.
+
+Written and tested with Ruby 2.4.4 (or whatever InSpec 2.0 supports).
+
+# Installation #
+
+`inspec-iggy` is a plugin for InSpec and may be installed as follows
+
+```bash
+# install InSpec
+gem install inspec
+gem install inspec-iggy
+inspec terraform version
+```
+
+## * for development: ##
+
+```bash
+# Install `inspec-iggy` via a symlink:
+git clone git@github.com:inspec/inspec-iggy ~/inspec-iggy
+mkdir -p ~/.inspec/plugins
+ln -s ~/inspec-iggy/ ~/.inspec/plugins/inspec-iggy
+inspec terraform version
+```
+
+## * or build a gem: ##
+
+```bash
+# Build the `inspec-iggy` then install:
+git clone https://github.com/inspec/inspec-iggy && cd inspec-iggy && gem build *gemspec && gem install *gem
+inspec terraform version
+```
+
+# InSpec Terraform Generate #
+
+     inspec terraform generate --tfstate terraform.tfstate
+
+Iggy dynamically pulls the available AWS resources from InSpec and attempts to map them to the Terraform resources. Newer versions of InSpec may provide additional coverage.
+
+# InSpec Terraform Extract (EXPERIMENTAL)#
+
+    inspec terraform extract --tfstate terraform.tfstate
+
+## Tagging Profiles for Extract ##
+
+Compliance profiles are added to the Terraform Resource to be tested. The current 2 options are the ```aws_vpc``` or the ```aws_instance```. By tagging the ```aws_vpc``` you are specifying that the test is against the AWS API rather than individual machines. AWS instances tagged with compliance profiles will attempt to form command lines for ```inspec exec``` against them.
+
+### Tagging Format ###
+
+Given there is not support for lists within AWS tags, we use the convention of starting our tag names with ```inspec_name_``` and ```inspec_url_```. These are extracted and split to identify the relevant compliance profiles to run.
+
+```
+tags {
+    iggy_name_apache_baseline = "apache-baseline",
+    iggy_url_apache_baseline = "https://github.com/dev-sec/apache-baseline",
+    iggy_name_linux_baseline = "linux-baseline",
+    iggy_url_linux_baseline = "https://github.com/dev-sec/linux-baseline"
+}
+```
+
+### Potential Enhancements ###
+
+The current tagging for extraction implementation is directly tied to AWS. Other platforms such as Azure undoubtedly behave differently. Longterm this functionality should probably be turned into a Terraform Provider with predefined outputs.
+
+Subnet might be a better choice for tagging than VPCs, given they list the AZ.
+
+Currently it only supports URL-based compliance profiles. InSpec supports other formats (git, path, supermarket, compliance).
+
+inspec exec https://github.com/dev-sec/linux-baseline -t ssh://clckwrk@52.33.203.34 -i ~/.ssh/mattray-apac
+
+# CloudFormation Support #
+
+**CloudFormation support has been started, but it is incomplete while focusing on Terraform.** Here is an example of the current output, note that it's not tied to an actual deployed CloudFormation Stack, so that will need to be provided for the entry point of testing.
+https://gist.github.com/c4d6eda82dfb25502ef381cc631a1edd
+
+# Testing #
+
+Iggy uses [RSpec](http://rspec.info/) for testing. You should run the following before committing.
+
+    $ rspec
+
+For style
+
+    $ chefstyle .
+
+# License and Author #
+
+|                |                                           |
+|:---------------|:------------------------------------------|
+| **Author**     | Matt Ray (<matt@chef.io>)                 |
+| **Copyright:** | Copyright (c) 2017 Chef Software Inc.     |
+| **License:**   | Apache License, Version 2.0               |
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/inspec-iggy.gemspec

@@ -0,0 +1,26 @@
+# coding: utf-8
+lib = File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+require "inspec-iggy/version"
+
+Gem::Specification.new do |spec|
+  spec.name        = "inspec-iggy"
+  spec.version     = Iggy::VERSION
+  spec.authors     = ["Matt Ray"]
+  spec.email       = ["matt@chef.io"]
+  spec.summary     = "InSpec plugin to generate InSpec compliance profiles from Terraform."
+  spec.description = "Generate InSpec compliance profiles from Terraform by tagging instances and mapping Terraform to InSpec."
+  spec.homepage    = "https://github.com/inspec/inspec-iggy"
+  spec.license     = "Apache-2.0"
+
+  spec.files = %w{
+    README.md inspec-iggy.gemspec Gemfile
+  } + Dir.glob(
+    "{bin,docs,examples,lib,tasks,test}/**/*", File::FNM_DOTMATCH
+  ).reject { |f| File.directory?(f) }
+
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "inspec", ">=2.0", "<3.0.0"
+end

data/lib/inspec-iggy.rb

@@ -0,0 +1,7 @@
+# encoding: utf-8
+
+libdir = File.dirname(__FILE__)
+$LOAD_PATH.unshift(libdir) unless $LOAD_PATH.include?(libdir)
+
+require "inspec-iggy/cli"
+require "inspec-iggy/version"

data/lib/inspec-iggy/cli.rb

@@ -0,0 +1,53 @@
+# encoding: utf-8
+#
+# Author:: Matt Ray (<matt@chef.io>)
+#
+# Copyright:: 2018, Chef Software, Inc <legal@chef.io>
+#
+
+require "inspec/plugins"
+require "thor"
+
+require "inspec-iggy/terraform"
+
+module Iggy
+  class CLI < Thor
+    namespace "terraform"
+
+    map %w{-v --version} => "version"
+
+    desc "version", "Display version information", hide: true
+    def version
+      say("Iggy v#{Iggy::VERSION}")
+    end
+
+    class_option :tfstate,
+      :aliases => "-t",
+      :desc    => "Specify path to the input terraform.tfstate",
+      :default => "terraform.tfstate"
+
+    class_option :debug,
+      :desc    => "Verbose debugging messages",
+      :type    => :boolean,
+      :default => false
+
+    desc "generate [options]", "Generate InSpec compliance controls from terraform.tfstate"
+    def generate
+      Inspec::Log.level = :debug if options[:debug]
+      generated_controls = Iggy::Terraform.parse_generate(options[:tfstate])
+      # let's just generate a control file with a set of controls for now
+      Iggy::InspecHelper.print_controls(options[:tfstate], generated_controls)
+      exit 0
+    end
+
+    desc "extract [options]", "Extract tagged InSpec profiles from terraform.tfstate"
+    def extract
+      Inspec::Log.level = :debug if options[:debug]
+      extracted_profiles = Iggy::Terraform.parse_extract(options[:tfstate])
+      Iggy::InspecHelper.print_commands(extracted_profiles)
+      exit 0
+    end
+  end
+
+  Inspec::Plugins::CLI.add_subcommand(CLI, "terraform", "terraform SUBCOMMAND ...", "Extract or generate InSpec from Terraform", {})
+end

data/lib/inspec-iggy/cloudformation.rb

@@ -0,0 +1,104 @@
+#
+# Author:: Matt Ray (<matt@chef.io>)
+#
+# Copyright:: 2018, Chef Software, Inc <legal@chef.io>
+#
+
+require "iggy"
+
+require "json"
+require "thor"
+
+module Iggy
+  class CloudFormation < Thor
+    option :template,
+      :aliases => "-t",
+      :desc    => "Specify path to the input CloudFormation template"
+
+    option :debug,
+      :desc    => "Verbose debugging messages",
+      :type    => :boolean,
+      :default => false
+
+    desc "generate [options]", "Generate InSpec compliance controls from CloudFormation template"
+    long_desc <<-LONGDESC
+Reads in a CloudFormation JSON file and generates matching InSpec compliance controls.
+    LONGDESC
+    def generate
+      Iggy::Log.level = :debug if options[:debug]
+      Iggy::Log.debug "CloudFormation.generate file = #{options[:template]}"
+      # hash of generated controls
+      generated_controls = parse_generate(options[:template])
+      Iggy::Log.debug "CloudFormation.generate generated_controls = #{generated_controls}"
+      # let's just generate a control file with a set of controls for now
+      Iggy::Inspec.print_controls(options[:template], generated_controls)
+      exit 0
+    end
+
+    private
+
+    def parse_generate(file)
+      Iggy::Log.debug "CloudFormation.parse_generate file = #{file}"
+      begin
+        unless File.file?(file)
+          STDERR.puts "ERROR: #{file} is an invalid file, please check your path."
+          exit(-1)
+        end
+        template = JSON.parse(File.read(file))
+      rescue JSON::ParserError => e
+        STDERR.puts e.message
+        STDERR.puts "ERROR: Parsing error in #{file}."
+        exit(-1)
+      end
+      basename = File.basename(file)
+      absolutename = File.absolute_path(file)
+
+      # InSpec controls generated
+      generated_controls = {}
+
+      # iterate over the resources
+      cfn_resources = template["Resources"]
+      # iterate over the Resources, use these as IDs?
+      cfn_resources.keys.each do |cfn_res|
+        # split out the last ::, these are all AWS
+        cfn_res_type = "aws_" + cfn_resources[cfn_res]["Type"].split("::").last.downcase
+
+        # does this match an InSpec resource?
+        if Inspec::RESOURCES.include?(cfn_res_type)
+          Iggy::Log.debug "CloudFormation.parse_generate cfn_res_type = #{cfn_res_type} MATCH"
+          # insert new control based off the resource's ID
+          generated_controls[cfn_res] = {}
+          generated_controls[cfn_res]["name"] = "#{cfn_res_type}::#{cfn_res}"
+          generated_controls[cfn_res]["title"] = "Iggy #{basename} #{cfn_res_type}::#{cfn_res}"
+          generated_controls[cfn_res]["desc"] = "#{cfn_res_type}::#{cfn_res} from the source file #{absolutename}\nGenerated by Iggy v#{Iggy::VERSION}"
+          generated_controls[cfn_res]["impact"] = "1.0"
+          generated_controls[cfn_res]["resource"] = cfn_res_type
+          generated_controls[cfn_res]["parameter"] = cfn_res
+          generated_controls[cfn_res]["tests"] = []
+          generated_controls[cfn_res]["tests"][0] = "it { should exist }"
+
+          # if there's a match, see if there are matching InSpec properties
+          inspec_properties = Iggy::Inspec.resource_properties(cfn_res_type)
+
+          cfn_resources[cfn_res]["Properties"].keys.each do |attr|
+            # insert '_' on the CamelCase to get camel_case
+            attr_split = attr.split /(?=[A-Z])/
+            property = attr_split.join("_").downcase
+            if inspec_properties.member?(property)
+              Iggy::Log.debug "CloudFormation.parse_generate #{cfn_res_type} inspec_property = #{property} MATCH"
+              value = cfn_resources[cfn_res]["Properties"][attr]
+              # skip the {"Ref"=>"VPC"} for now
+              generated_controls[cfn_res]["tests"].push("its('#{property}') { should cmp '#{value}' }") unless value.is_a? Hash
+            else
+              Iggy::Log.debug "CloudFormation.parse_generate #{cfn_res_type} inspec_property = #{property} SKIP"
+            end
+          end
+        else
+          Iggy::Log.debug "CloudFormation.parse_generate cfn_res_type = #{cfn_res_type} SKIP"
+        end
+      end
+      generated_controls
+    end
+
+  end
+end

data/lib/inspec-iggy/inspec_helper.rb

@@ -0,0 +1,59 @@
+#
+# Author:: Matt Ray (<matt@chef.io>)
+#
+# Copyright:: 2018, Chef Software, Inc <legal@chef.io>
+#
+
+require "inspec"
+
+module Iggy
+  class InspecHelper
+
+    # constants for the InSpec resources
+    RESOURCES = Inspec::Resource.registry.keys
+
+    # translate Terraform resource name to InSpec
+    TERRAFORM_RESOURCES = {
+      "aws_instance" => "aws_ec2_instance",
+      # 'aws_route' => 'aws_route_table' # needs route_table_id instead of id
+    }
+
+    # # there really should be some way to get this directly from InSpec's resources
+    def self.resource_properties(resource)
+      # remove the common methods, in theory only leaving only unique InSpec properties
+      inspec_properties = Inspec::Resource.registry[resource].instance_methods - COMMON_PROPERTIES
+      # get InSpec properties by method names
+      inspec_properties.collect! { |x| x.to_s }
+      Inspec::Log.debug "Iggy::InspecHelper.resource_properties #{resource} properties = #{inspec_properties}"
+
+      inspec_properties
+    end
+
+    def self.print_commands(extracted_profiles)
+      extracted_profiles.keys.each do |cmd|
+        type = extracted_profiles[cmd]["type"]
+        url = extracted_profiles[cmd]["url"]
+        key_name = extracted_profiles[cmd]["key_name"]
+        if type == "aws_instance"
+          ip = extracted_profiles[cmd]["public_ip"]
+          puts "inspec exec #{url} -t ssh://#{ip} -i #{key_name}"
+        else
+          puts "inspec exec #{url} -t aws://us-west-2"
+        end
+      end
+    end
+
+    def self.print_controls(file, generated_controls)
+      puts "# encoding: utf-8\n#"
+
+      puts "\ntitle '#{File.absolute_path(file)} controls generated by Iggy v#{Iggy::VERSION}'"
+
+      # write all controls
+      puts generated_controls.flatten.map(&:to_ruby).join("\n\n")
+    end
+
+    # a hack for sure, finds common methods as proxy for InSpec properties
+    COMMON_PROPERTIES = Inspec::Resource.registry["aws_subnet"].instance_methods &
+      Inspec::Resource.registry["directory"].instance_methods
+  end
+end

data/lib/inspec-iggy/terraform.rb

@@ -0,0 +1,146 @@
+#
+# Author:: Matt Ray (<matt@chef.io>)
+#
+# Copyright:: 2018, Chef Software, Inc <legal@chef.io>
+#
+
+require "inspec/objects/control"
+require "inspec/objects/ruby_helper"
+require "inspec/objects/describe"
+
+require "inspec-iggy/inspec_helper"
+
+module Iggy
+  class Terraform
+
+    # makes it easier to change out later
+    TAG_NAME = "iggy_name_"
+    TAG_URL = "iggy_url_"
+
+    # boilerplate tfstate parsing
+    def self.parse_tfstate(file)
+      Inspec::Log.debug "Iggy::Terraform.parse_tfstate file = #{file}"
+      begin
+        unless File.file?(file)
+          STDERR.puts "ERROR: #{file} is an invalid file, please check your path."
+          exit(-1)
+        end
+        tfstate = JSON.parse(File.read(file))
+      rescue JSON::ParserError => e
+        STDERR.puts e.message
+        STDERR.puts "ERROR: Parsing error in #{file}."
+        exit(-1)
+      end
+    end
+
+    # parse through the JSON for the tagged Resources
+    def self.parse_extract(file)
+      tfstate = parse_tfstate(file)
+      # InSpec profiles extracted
+      extracted_profiles = {}
+
+      # iterate over the resources
+      tf_resources = tfstate["modules"][0]["resources"]
+      tf_resources.keys.each do |tf_res|
+        tf_res_id = tf_resources[tf_res]["primary"]["id"]
+
+        # get the attributes, see if any of them have a tagged profile attached
+        tf_resources[tf_res]["primary"]["attributes"].keys.each do |attr|
+          next unless attr.start_with?("tags." + TAG_NAME)
+          Inspec::Log.debug "Iggy::Terraform.parse_extract tf_res = #{tf_res} attr = #{attr} MATCHED TAG"
+          # get the URL and the name of the profiles
+          name = attr.split(TAG_NAME)[1]
+          url = tf_resources[tf_res]["primary"]["attributes"]["tags.#{TAG_URL}#{name}"]
+          if tf_res.start_with?("aws_vpc") # should this be VPC or subnet?
+            # if it's a VPC, store it as the VPC id + name
+            key = tf_res_id + ":" + name
+            Inspec::Log.debug "Iggy::Terraform.parse_extract aws_vpc tagged with InSpec #{key}"
+            extracted_profiles[key] = {
+              "type" => "aws_vpc",
+              "az" => "us-west-2",
+              "url" => url,
+            }
+          elsif tf_res.start_with?("aws_instance")
+            # if it's a node, get information about the IP and SSH/WinRM
+            key = tf_res_id + ":" + name
+            Inspec::Log.debug "Iggy::Terraform.parse_extract aws_instance tagged with InSpec #{key}"
+            extracted_profiles[key] = {
+              "type" => "aws_instance",
+              "public_ip" => tf_resources[tf_res]["primary"]["attributes"]["public_ip"],
+              "key_name" => tf_resources[tf_res]["primary"]["attributes"]["key_name"],
+              "url" => url,
+            }
+          else
+            # should generic AWS just be the default except for instances?
+            STDERR.puts "ERROR: #{file} #{tf_res_id} has an InSpec-tagged resource but #{tf_res} is currently unsupported."
+            exit(-1)
+          end
+        end
+      end
+      Inspec::Log.debug "Iggy::Terraform.parse_extract extracted_profiles = #{extracted_profiles}"
+      extracted_profiles
+    end
+
+    # parse through the JSON and generate InSpec controls
+    def self.parse_generate(file)
+      tfstate = parse_tfstate(file)
+      basename = File.basename(file)
+      absolutename = File.absolute_path(file)
+
+      # InSpec controls generated
+      generated_controls = []
+
+      # iterate over the resources
+      tf_resources = tfstate["modules"][0]["resources"]
+      tf_resources.keys.each do |tf_res|
+        tf_res_type = tf_resources[tf_res]["type"]
+
+        # add translation layer
+        if InspecHelper::TERRAFORM_RESOURCES.keys.include?(tf_res_type)
+          Inspec::Log.debug "Iggy::Terraform.parse_generate tf_res_type = #{tf_res_type} #{InspecHelper::TERRAFORM_RESOURCES[tf_res_type]} TRANSLATED"
+          tf_res_type = InspecHelper::TERRAFORM_RESOURCES[tf_res_type]
+        end
+
+        # does this match an InSpec resource?
+        if InspecHelper::RESOURCES.include?(tf_res_type)
+          Inspec::Log.debug "Iggy::Terraform.parse_generate tf_res_type = #{tf_res_type} MATCH"
+          tf_res_id = tf_resources[tf_res]["primary"]["id"]
+
+          # insert new control based off the resource's ID
+          ctrl = Inspec::Control.new
+          ctrl.id = "#{tf_res_type}::#{tf_res_id}"
+          ctrl.title = "Iggy #{basename} #{tf_res_type}::#{tf_res_id}"
+          ctrl.desc = "#{tf_res_type}::#{tf_res_id} from the source file #{absolutename}\nGenerated by Iggy v#{Iggy::VERSION}"
+          ctrl.impact = "1.0"
+
+          describe = Inspec::Describe.new
+          # describes the resourde with the id as argument
+          describe.qualifier.push([tf_res_type, tf_res_id])
+
+          # ensure the resource exists
+          describe.add_test(nil, "exist", nil)
+
+          # if there's a match, see if there are matching InSpec properties
+          inspec_properties = Iggy::InspecHelper.resource_properties(tf_res_type)
+          tf_resources[tf_res]["primary"]["attributes"].keys.each do |attr|
+            if inspec_properties.member?(attr)
+              Inspec::Log.debug "Iggy::Terraform.parse_generate #{tf_res_type} inspec_property = #{attr} MATCH"
+              value = tf_resources[tf_res]["primary"]["attributes"][attr]
+              describe.add_test(attr, "cmp", value)
+            else
+              Inspec::Log.debug "Iggy::Terraform.parse_generate #{tf_res_type} inspec_property = #{attr} SKIP"
+            end
+          end
+
+          ctrl.add_test(describe)
+          generated_controls.push(ctrl)
+        else
+          Inspec::Log.debug "Iggy::Terraform.parse_generate tf_res_type = #{tf_res_type} SKIP"
+        end
+      end
+      Inspec::Log.debug "Iggy::Terraform.parse_generate generated_controls = #{generated_controls}"
+      generated_controls
+    end
+
+  end
+end