inspec 0.34.1 → 1.0.0

data/docs/ruby_usage.md

@@ -0,0 +1,154 @@
+# Using Ruby in InSpec
+
+The InSpec DSL is a Ruby based DSL for writing audit controls, which
+includes audit resources that you can invoke. Core and custom resources
+are written as regular Ruby classes which inherit from
+`Inspec.resource`.
+
+Assuming we have a JSON file like this on the node to be tested:
+
+```json
+{
+  "keys":[
+    {"username":"john", "key":"/opt/keys/johnd.key"},
+    {"username":"jane", "key":"/opt/keys/janed.key"},
+    {"username":"sunny ", "key":"/opt/keys/sunnym.key"}
+  ]
+}
+```
+
+The following example shows how you can use pure Ruby code(variables,
+loops, conditionals, regular expressions, etc) to run a few tests
+against the above JSON file:
+
+```ruby
+control 'check-interns' do
+  # use the json inspec resource to get the file
+  json_obj = json('/opt/keys/interns.json')
+  describe json_obj do
+    its('keys') { should_not eq nil }
+  end
+  if json_obj['keys']
+    # loop over the keys array
+    json_obj['keys'].each do |intern|
+      username = intern['username'].strip
+      # check for white spaces chars in usernames
+      describe username do
+        it { should_not match(/\s/) }
+      end
+      # check key file owners and permissions
+      describe file(intern['key']) do
+        it { should be_owned_by username }
+        its('mode') { should cmp '0600' }
+      end
+    end
+  end
+end
+```
+
+## Execution
+
+It's important to understand that Ruby code used in custom resources and
+controls DSL is executed on the system that runs InSpec. This allows
+InSpec to work without Ruby and rubygems being required on remote
+targets(servers or containers).
+
+For example, using `` `ls ``\` or `system('ls')` will result in the `ls`
+command being run locally and not on the target(remote) system. In order
+to process the output of `ls` executed on the target system, use
+`inspec.command('ls')` or `inspec.powershell('ls')`
+
+Similarly, use `inspec.file(PATH)` to access files or directories from
+remote systems in your tests or custom resources.
+
+
+## Using rubygems
+
+Ruby gems are self-contained programs and libraries. If you create a custom
+resource please vendor gems into the library. This ensures that all resources
+are self-contained and complete and don't need any resolution at runtime. We
+vendor resources and requirements through dependency resolution, which is
+independent of programming languages and their resolver mechanisms.
+
+## Interactive Debugging with Pry
+
+Here's a sample InSpec control that users Ruby variables to instantiate
+an InSpec resource once and use the content in multipLe tests.
+
+```ruby
+control 'check-perl' do
+  impact 0.3
+  title 'Check perl compiled options and permissions'
+  perl_out = command('perl -V')
+  #require 'pry'; binding.pry;
+  describe perl_out do
+    its('exit_status') { should eq 0 }
+    its('stdout') { should match (/USE_64_BIT_ALL/) }
+    its('stdout') { should match (/useposix=true/) }
+    its('stdout') { should match (/-fstack-protector/) }
+  end
+
+  # extract an array of include directories
+  perl_inc = perl_out.stdout.partition('@INC:').last.strip.split("\n")
+  # ensure include directories are only writable by 'owner'
+  perl_inc.each do |path|
+    describe directory(path.strip) do
+      it { should_not be_writable.by('group') }
+      it { should_not be_writable.by('other') }
+    end
+  end
+end
+```
+
+An **advanced** but very useful Ruby tip. In the previous example, I
+commented out the `require 'pry'; binding.pry;` line. If you remove the
+`#` prefix and run the control, the execution will stop at that line and
+give you a `pry` shell. Use that to troubleshoot, print variables, see
+methods available, etc. For the above example:
+
+```ruby
+[1] pry> perl_out.exit_status
+=> 0
+[2] pry> perl_out.stderr
+=> ""
+[3] pry> ls perl_out
+Inspec::Plugins::Resource#methods: inspect
+Inspec::Resources::Cmd#methods: command  exist?  exit_status  result  stderr  stdout  to_s
+Inspec::Plugins::ResourceCommon#methods: resource_skipped  skip_resource
+Inspec::Resource::Registry::Command#methods: inspec
+instance variables: @__backend_runner__  @__resource_name__  @command  @result
+[4] pry> perl_out.stdout.partition('@INC:').last.strip.split("\n")
+=> ["/Library/Perl/5.18/darwin-thread-multi-2level",
+ "    /Library/Perl/5.18",
+...REDACTED...
+[5] pry> exit    # or abort
+```
+
+You can use `pry` inside both the controls DSL and resources. Similarly,
+for dev and test, you can use `inspec shell` which is based on `pry`,
+for example:
+
+```ruby
+$ inspec shell
+Welcome to the interactive InSpec Shell
+To find out how to use it, type: help
+
+inspec> command('ls /home/gordon/git/inspec/docs').stdout
+=> "ctl_inspec.rst\ndsl_inspec.rst\ndsl_resource.rst\n"
+inspec> command('ls').stdout.split("\n")
+=> ["ctl_inspec.rst", "dsl_inspec.rst", "dsl_resource.rst"]
+
+inspec> help command
+Name: command
+
+Description:
+Use the command InSpec audit resource to test an arbitrary command that is run on the system.
+
+Example:
+describe command('ls -al /') do
+  it { should exist }
+  its('stdout') { should match /bin/ }
+  its('stderr') { should eq '' }
+  its('exit_status') { should eq 0 }
+end
+```

data/docs/shared/matcher_be.md.erb

@@ -0,0 +1 @@
+Use the `be` matcher to use a comparison operator---`=` (equal to), `>` (greater than), `<` (less than), `>=` (greater than or equal to), and `<=` (less than or equal to)---to compare two values: `its('value') { should be >= value }`, `its('value') { should be < value }`, and so on.

data/docs/shared/matcher_cmp.md.erb

@@ -0,0 +1,45 @@
+Use the `cmp` matcher compare two values, such as comparing strings to numbers, comparing a single value to an array of values, comparing an array of strings to a regular expression, improving the printing of octal values, and comparing while ignoring case sensitivity.
+
+Compare a single value to an array:
+
+    describe some_resource do
+      its('users') { should cmp 'root' }
+      its('users') { should cmp ['root'] }
+    end
+
+Compare strings and regular expressions:
+
+    describe some_resource do
+      its('setting') { should cmp /raw/i }
+    end
+
+Compare strings and numbers:
+
+    describe some_resource do
+      its('setting') { should eq '2' }
+    end
+
+vs:
+
+    describe some_resource do
+      its('setting') { should cmp '2' }
+      its('setting') { should cmp 2 }
+    end
+
+Ignoring case sensitivity:
+
+.. code-block:: ruby
+
+   describe some_resource do
+     its('setting') { should cmp 'raw' }
+     its('setting') { should cmp 'RAW' }
+   end
+
+Printing octal values:
+
+    describe some_resource('/proc/cpuinfo') do
+      its('mode') { should cmp '0345' }
+    end
+
+    expected: 0345
+    got: 0444

data/docs/shared/matcher_eq.md.erb

@@ -0,0 +1,3 @@
+Use the `eq` matcher to test the equality of two values: `its('Port') { should eq '22' }`.
+
+Using `its('Port') { should eq 22 }` will fail because `22` is not a string value! Use the `cmp` matcher for less restrictive value comparisons.

data/docs/shared/matcher_include.md.erb

@@ -0,0 +1 @@
+Use the `include` matcher to verify that a string value is included in a list: `its('list') { should include 'string' }`.

data/docs/shared/matcher_match.md.erb

@@ -0,0 +1 @@
+Use the `match` matcher to check if a string matches a regular expression: `its('string') { should_not match /regex/ }`.

data/docs/shell.md

@@ -0,0 +1,150 @@
+---
+title: InSpec Shell
+---
+
+# InSpec Shell
+
+The InSpec interactive shell is a pry based REPL that can be used to
+quickly run InSpec controls and tests without having to write it to a
+file. Its functionality is similar to `chef shell` - it provides a way
+to exercise the InSpec DSL, its resources, tests and plugins without
+having to create a profile or write a test file. See
+[http://pryrepl.org/](http://pryrepl.org/) for an introduction to what pry is and what it can
+do.
+
+## Launching the shell
+
+If you are using InSpec from a platform-specific package (rpm, msi,
+etc.) or from a chef prepared shell in ChefDK, you can directly launch
+InSpec shell against your local machine using the following. See
+<https://docs.chef.io/install_dk.html#set-system-ruby> for details.
+
+```bash
+$ inspec shell
+$ inspec help shell # This will describe inspec shell usage
+```
+
+If you wish to connect to a remote machine (called a target within
+InSpec), you can use the `-t` flag. We support connecting using ssh,
+WinRm and docker. If no target is provided, we implicitly support the
+"local" target - i.e. tests running on the current machine running
+InSpec. For an ssh connection, use `-i` for specifying ssh key files,
+and the `--sudo*` commands for requesting a privelege escalation after
+logging in. For a WinRM connection, use `--path` to change the login
+path, `--ssl` to use SSL for transport layer encryption.
+
+```bash
+$ inspec shell -t ssh://root@192.168.64.2:11022  # Login to remote machine using ssh as root.
+$ inspec shell -t ssh://user@hostname:1234 -i /path/to/user_key  # Login to hostname on port 1234 as user using given ssh key.
+$ inspec shell -t winrm://UserName:Password@windowsmachine:1234  # Login to windowsmachine over WinRM as UserName.
+$ inspec shell -t docker://container_id # Login to a docker container.
+```
+
+## Using Ruby in InSpec shell
+
+Since InSpec shell is pry based, you may treat the shell as an
+interactive Ruby session. You may write Ruby expressions and evaluate
+them. Source high-lighting, automatic indentation and command history
+(using the up and down arrow keys) are available to make your experience
+more delightful. You can exit the shell using `exit`.
+
+```bash
+$ inspec shell
+Welcome to the interactive InSpec Shell
+To find out how to use it, type: help
+
+inspec> 1 + 2
+=> 3
+inspec> exit
+```
+
+## Using InSpec DSL in InSpec shell
+
+InSpec shell will automatically evaluate the result of every command as
+if it were a test file. If you type in a Ruby command that is not an
+InSpec control or test, the shell will evaluate it as if it were a
+regular ruby command.
+
+Bare InSpec resources are instantiated and their help text is presented.
+You may also access the resource contents or other matchers that they
+define. Run `help <resource>` to get more help on using a particular
+resource or see the InSpec resources documentation online.
+
+```bash
+$ inspec shell
+Welcome to the interactive InSpec Shell
+To find out how to use it, type: help
+
+inspec> file('/Users/ksubramanian').directory?
+=> true
+inspec> os_env('HOME')
+=> Environment variable HOME
+inspec> os_env('HOME').content
+=> /Users/ksubramanian
+inspec> exit
+```
+
+InSpec tests are immediately executed.
+
+```bash
+inspec> describe file('/Users')     # Empty test.
+Summary: 0 successful, 0 failures, 0 skipped
+inspec> describe file('/Users') do  # Test with one check.
+inspec>   it { should exist }
+inspec> end
+  ✔  File /Users should exist
+
+Summary: 1 successful, 0 failures, 0 skipped
+```
+
+All tests in a control are immediately executed as well. If a control is
+redefined in the shell, the old control's tests are destroyed and
+replaced with the redefinition and the control is re-run.
+
+```bash
+inspec> control 'my_control' do
+inspec>   describe os_env('HOME') do
+inspec>     its('content') { should eq '/Users/ksubramanian' }
+inspec>   end
+inspec> end
+  ✔  my_control: Environment variable HOME content should eq "/Users/ksubramanian"
+
+  Summary: 1 successful, 0 failures, 0 skipped
+```
+
+Syntax errors are illegal tests are also detected and reported.
+
+```bash
+inspec> control 'foo' do
+inspec>   thisisnonsense
+inspec> end
+NameError: undefined local variable or method `thisisnonsense' for #<#<Class:0x007fd63b571f98>:0x007fd639825cc8>
+from /usr/local/lib/ruby/gems/2.3.0/gems/rspec-expectations-3.5.0/lib/rspec/matchers.rb:967:in `method_missing'
+inspec> control 'foo' do
+inspec>   describe file('wut') do
+inspec>     its('thismakesnosense') { should cmp 'fail' }
+inspec>   end
+inspec> end
+  ✖  foo: File wut thismakesnosense  (undefined method `thismakesnosense' for File wut:Inspec::Resource::Registry::File)
+
+  Summary: 0 successful, 1 failures, 0 skipped
+```
+
+## Running a single InSpec command
+
+If you wish to run a single InSpec command and fetch its results, you
+may use the `-c` flag. This is similar to using `bash -c`.
+
+```bash
+$ inspec shell -c 'describe file("/Users/ksubramanian") do it { should exist } end'}
+Target:  local://
+
+  ✔  File /Users/ksubramanian should exist
+
+Summary: 1 successful, 0 failures, 0 skipped
+```
+
+```bash
+$ inspec shell --format json -c 'describe file("/Users/ksubramanian") do it { should exist } end'
+{"version":"0.30.0","profiles":{"":{"supports":[],"controls":{"(generated from in_memory.rb:1 5aab65c33fb1f133d9244017958eef64)":{"title":null,"desc":null,"impact":0.5,"refs":[],"tags":{},"code":"          rule = rule_class.new(id, profile_id, {}) do\n            res = describe(*args, &block)\n          end\n","source_location":{"ref":"/Users/ksubramanian/repo/chef/inspec/lib/inspec/profile_context.rb","line":184},"results":[{"status":"passed","code_desc":"File /Users/ksubramanian should exist","run_time":0.000747,"start_time":"2016-08-16 11:41:40 -0400"}]}},"groups":{"in_memory.rb":{"title":null,"controls":["(generated from in_memory.rb:1 5aab65c33fb1f133d9244017958eef64)"]}},"attributes":[]}},"other_checks":[],"summary":{"duration":0.001078,"example_count":1,"failure_count":0,"skip_count":0}}}
+```

data/examples/meta-profile/inspec.yml

@@ -7,12 +7,10 @@ license: Apache 2
 summary: InSpec Profile that is only consuming dependencies
 version: 0.2.0
 depends:
-  - name: ssh-hardening
-    supermarket: hardening/ssh-hardening
+  - name: hardening/ssh-hardening  # defaults to supermarket
   - name: os-hardening
     url: https://github.com/dev-sec/tests-os-hardening/archive/master.zip
-  - name: ssl-benchmark
-    git: https://github.com/dev-sec/ssl-benchmark.git
+  - git: https://github.com/dev-sec/ssl-benchmark.git
   - name: windows-patch-benchmark
     git: https://github.com/chris-rock/windows-patch-benchmark.git
   - name: linux

data/inspec.gemspec

@@ -24,7 +24,7 @@ Gem::Specification.new do |spec|
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ['lib']
 
-  spec.add_dependency 'train', '>=0.19.0', '<1.0'
+  spec.add_dependency 'train', '>=0.19.1', '<1.0'
   spec.add_dependency 'thor', '~> 0.19'
   spec.add_dependency 'json', '>= 1.8', '< 3.0'
   spec.add_dependency 'rainbow', '~> 2'

data/lib/bundles/inspec-compliance/api.rb

@@ -9,18 +9,6 @@ module Compliance
   # API Implementation does not hold any state by itself,
   # everything will be stored in local Configuration store
   class API
-    # login method for pre-1.0 compliance server
-    def self.legacy_login_post(url, username, password, insecure)
-      # form request
-      # TODO: reuse post function
-      uri = URI.parse(url)
-      req = Net::HTTP::Post.new(uri.path)
-      req.basic_auth(username, password)
-      req.form_data={}
-
-      send_request(uri, req, insecure)
-    end
-
     # return all compliance profiles available for the user
     def self.profiles(config)
       url = "#{config['server']}/user/compliance"
@@ -86,19 +74,19 @@ Please login using `inspec compliance login https://compliance.test --user admin
       [res.is_a?(Net::HTTPSuccess), res.body]
     end
 
-    def self.post_refresh_token(url, token, insecure)
+    # Use username and refresh_toke to get an API access token
+    def self.get_token_via_refresh_token(url, refresh_token, insecure)
       uri = URI.parse("#{url}/login")
       req = Net::HTTP::Post.new(uri.path)
-      # req['Authorization'] = "Bearer #{token}"
-      req.body = { token: token }.to_json
+      req.body = { token: refresh_token }.to_json
       access_token = nil
       response = Compliance::HTTP.send_request(uri, req, insecure)
       data = response.body
-      if !data.nil?
+      if response.code == '200'
         begin
           tokendata = JSON.parse(data)
           access_token = tokendata['access_token']
-          msg = 'Successfully fetched access token'
+          msg = 'Successfully fetched API access token'
           success = true
         rescue JSON::ParserError => e
           success = false
@@ -106,7 +94,29 @@ Please login using `inspec compliance login https://compliance.test --user admin
         end
       else
         success = false
-        msg = 'Invalid refresh_token'
+        msg = "Failed to authenticate to #{url} \n\
+  Response code: #{response.code}\n  Body: #{response.body}"
+      end
+
+      [success, msg, access_token]
+    end
+
+    # Use username and password to get an API access token
+    def self.get_token_via_password(url, username, password, insecure)
+      uri = URI.parse("#{url}/login")
+      req = Net::HTTP::Post.new(uri.path)
+      req.body = { userid: username, password: password }.to_json
+      access_token = nil
+      response = Compliance::HTTP.send_request(uri, req, insecure)
+      data = response.body
+      if response.code == '200'
+        access_token = data
+        msg = 'Successfully fetched an API access token valid for 12 hours'
+        success = true
+      else
+        success = false
+        msg = "Failed to authenticate to #{url} \n\
+  Response code: #{response.code}\n  Body: #{response.body}"
       end
 
       [success, msg, access_token]

data/lib/bundles/inspec-compliance/cli.rb

@@ -22,9 +22,9 @@ module Compliance
     option :insecure, aliases: :k, type: :boolean,
       desc: 'Explicitly allows InSpec to perform "insecure" SSL connections and transfers'
     option :user, type: :string, required: false,
-      desc: 'Chef Compliance Username (for legacy auth)'
+      desc: 'Chef Compliance Username'
     option :password, type: :string, required: false,
-      desc: 'Chef Compliance Password (for legacy auth)'
+      desc: 'Chef Compliance Password'
     option :apipath, type: :string, default: '/api',
       desc: 'Set the path to the API, defaults to /api'
     option :token, type: :string, required: false,
@@ -38,7 +38,7 @@ module Compliance
       url = options['server'] + options['apipath']
       if !options['user'].nil? && !options['password'].nil?
         # username / password
-        _success, msg = login_legacy(url, options['user'], options['password'], options['insecure'])
+        _success, msg = login_username_password(url, options['user'], options['password'], options['insecure'])
       elsif !options['user'].nil? && !options['token'].nil?
         # access token
         _success, msg = store_access_token(url, options['user'], options['token'], options['insecure'])
@@ -199,7 +199,7 @@ module Compliance
     private
 
     def login_refreshtoken(url, options)
-      success, msg, access_token = Compliance::API.post_refresh_token(url, options['refresh_token'], options['insecure'])
+      success, msg, access_token = Compliance::API.get_token_via_refresh_token(url, options['refresh_token'], options['insecure'])
       if success
         config = Compliance::Configuration.new
         config['server'] = url
@@ -212,25 +212,17 @@ module Compliance
       [success, msg]
     end
 
-    def login_legacy(url, username, password, insecure)
+    def login_username_password(url, username, password, insecure)
       config = Compliance::Configuration.new
-      success, data = Compliance::API.legacy_login_post(url+'/oauth/token', username, password, insecure)
-      if !data.nil?
-        tokendata = JSON.parse(data)
-        if tokendata['access_token']
-          config['server'] = url
-          config['user'] = username
-          config['token'] = tokendata['access_token']
-          config['insecure'] = insecure
-          config['version'] = Compliance::API.version(url, insecure)
-          config.store
-          success = true
-          msg = 'Successfully authenticated'
-        else
-          msg = 'Response does not include a token'
-        end
-      else
-        msg = "Authentication failed for Server: #{url}"
+      success, msg, api_token = Compliance::API.get_token_via_password(url, username, password, insecure)
+      if success
+        config['server'] = url
+        config['user'] = username
+        config['token'] = api_token
+        config['insecure'] = insecure
+        config['version'] = Compliance::API.version(url, insecure)
+        config.store
+        success = true
       end
       [success, msg]
     end
@@ -245,10 +237,10 @@ module Compliance
       config['version'] = Compliance::API.version(url, insecure)
       config.store
 
-      [true, 'access token stored']
+      [true, 'API access token stored']
     end
 
-    # saves the a user refresh token supplied by the user
+    # saves a refresh token supplied by the user
     def store_refresh_token(url, refresh_token, verify, user, insecure)
       config = Compliance::Configuration.new
       config['server'] = url
@@ -260,13 +252,13 @@ module Compliance
       if !verify
         config.store
         success = true
-        msg = 'refresh token stored'
+        msg = 'API refresh token stored'
       else
-        success, msg, access_token = Compliance::API.post_refresh_token(url, refresh_token, insecure)
+        success, msg, access_token = Compliance::API.get_token_via_refresh_token(url, refresh_token, insecure)
         if success
           config['token'] = access_token
           config.store
-          msg = 'token verified and stored'
+          msg = 'API access token verified and stored'
         end
       end
 

data/lib/bundles/inspec-compliance/target.rb

@@ -4,6 +4,7 @@
 
 require 'uri'
 require 'inspec/fetcher'
+require 'inspec/errors'
 
 # InSpec Target Helper for Chef Compliance
 # reuses UrlHelper, but it knows the target server and the access token already
@@ -24,11 +25,23 @@ module Compliance
 
       # check if we have a compliance token
       config = Compliance::Configuration.new
-      return nil if config['token'].nil?
+      if config['token'].nil?
+        fail Inspec::FetcherFailure, <<EOF
+
+Cannot fetch #{uri} because your compliance token has not been
+configured.
+
+Please login using
+
+    inspec compliance login https://your_compliance_server --user admin --insecure --token 'PASTE TOKEN HERE'
+EOF
+      end
 
       # verifies that the target e.g base/ssh exists
       profile = uri.host + uri.path
-      Compliance::API.exist?(config, profile)
+      if !Compliance::API.exist?(config, profile)
+        fail Inpsec::FetcherFailure, "The compliance profile #{profile} was not found on the configured compliance server"
+      end
       new(target_url(profile, config), config)
     rescue URI::Error => _e
       nil

data/lib/fetchers/git.rb

@@ -56,6 +56,10 @@ module Fetchers
       @repo_directory
     end
 
+    def cache_key
+      resolved_ref
+    end
+
     def archive_path
       @repo_directory
     end

data/lib/fetchers/local.rb

@@ -55,8 +55,23 @@ module Fetchers
       @target
     end
 
+    def writable?
+      File.directory?(@target)
+    end
+
+    def cache_key
+      sha256.to_s
+    end
+
+    def sha256
+      return nil if File.directory?(@target)
+      @archive_shasum ||= Digest::SHA256.hexdigest File.read(@target)
+    end
+
     def resolved_source
-      { path: @target }
+      h = { path: @target }
+      h[:sha256] = sha256 if sha256
+      h
     end
   end
 end

data/lib/fetchers/mock.rb

@@ -27,5 +27,9 @@ module Fetchers
     def resolved_source
       { mock_fetcher: true }
     end
+
+    def cache_key
+      ''
+    end
   end
 end