inspec 0.35.0 → 1.0.0.beta2

data/docs/resources.rst

@@ -2195,7 +2195,7 @@ This file can be queried via:
 
 .. code-block:: ruby
 
-   describe json('/paht/to/name.json') do
+   describe json('/path/to/name.json') do
       its('name') { should eq 'hello' }
       its(['meta','creator']) { should eq 'John Doe' }
       its(['array', 1]) { should eq 'one' }

data/docs/shell.md

@@ -0,0 +1,150 @@
+---
+title: InSpec Shell
+---
+
+# InSpec Shell
+
+The InSpec interactive shell is a pry based REPL that can be used to
+quickly run InSpec controls and tests without having to write it to a
+file. Its functionality is similar to `chef shell` - it provides a way
+to exercise the InSpec DSL, its resources, tests and plugins without
+having to create a profile or write a test file. See
+[http://pryrepl.org/](http://pryrepl.org/) for an introduction to what pry is and what it can
+do.
+
+## Launching the shell
+
+If you are using InSpec from a platform-specific package (rpm, msi,
+etc.) or from a chef prepared shell in ChefDK, you can directly launch
+InSpec shell against your local machine using the following. See
+<https://docs.chef.io/install_dk.html#set-system-ruby> for details.
+
+```bash
+$ inspec shell
+$ inspec help shell # This will describe inspec shell usage
+```
+
+If you wish to connect to a remote machine (called a target within
+InSpec), you can use the `-t` flag. We support connecting using ssh,
+WinRm and docker. If no target is provided, we implicitly support the
+"local" target - i.e. tests running on the current machine running
+InSpec. For an ssh connection, use `-i` for specifying ssh key files,
+and the `--sudo*` commands for requesting a privelege escalation after
+logging in. For a WinRM connection, use `--path` to change the login
+path, `--ssl` to use SSL for transport layer encryption.
+
+```bash
+$ inspec shell -t ssh://root@192.168.64.2:11022  # Login to remote machine using ssh as root.
+$ inspec shell -t ssh://user@hostname:1234 -i /path/to/user_key  # Login to hostname on port 1234 as user using given ssh key.
+$ inspec shell -t winrm://UserName:Password@windowsmachine:1234  # Login to windowsmachine over WinRM as UserName.
+$ inspec shell -t docker://container_id # Login to a docker container.
+```
+
+## Using Ruby in InSpec shell
+
+Since InSpec shell is pry based, you may treat the shell as an
+interactive Ruby session. You may write Ruby expressions and evaluate
+them. Source high-lighting, automatic indentation and command history
+(using the up and down arrow keys) are available to make your experience
+more delightful. You can exit the shell using `exit`.
+
+```bash
+$ inspec shell
+Welcome to the interactive InSpec Shell
+To find out how to use it, type: help
+
+inspec> 1 + 2
+=> 3
+inspec> exit
+```
+
+## Using InSpec DSL in InSpec shell
+
+InSpec shell will automatically evaluate the result of every command as
+if it were a test file. If you type in a Ruby command that is not an
+InSpec control or test, the shell will evaluate it as if it were a
+regular ruby command.
+
+Bare InSpec resources are instantiated and their help text is presented.
+You may also access the resource contents or other matchers that they
+define. Run `help <resource>` to get more help on using a particular
+resource or see the InSpec resources documentation online.
+
+```bash
+$ inspec shell
+Welcome to the interactive InSpec Shell
+To find out how to use it, type: help
+
+inspec> file('/Users/ksubramanian').directory?
+=> true
+inspec> os_env('HOME')
+=> Environment variable HOME
+inspec> os_env('HOME').content
+=> /Users/ksubramanian
+inspec> exit
+```
+
+InSpec tests are immediately executed.
+
+```bash
+inspec> describe file('/Users')     # Empty test.
+Summary: 0 successful, 0 failures, 0 skipped
+inspec> describe file('/Users') do  # Test with one check.
+inspec>   it { should exist }
+inspec> end
+  ✔  File /Users should exist
+
+Summary: 1 successful, 0 failures, 0 skipped
+```
+
+All tests in a control are immediately executed as well. If a control is
+redefined in the shell, the old control's tests are destroyed and
+replaced with the redefinition and the control is re-run.
+
+```bash
+inspec> control 'my_control' do
+inspec>   describe os_env('HOME') do
+inspec>     its('content') { should eq '/Users/ksubramanian' }
+inspec>   end
+inspec> end
+  ✔  my_control: Environment variable HOME content should eq "/Users/ksubramanian"
+
+  Summary: 1 successful, 0 failures, 0 skipped
+```
+
+Syntax errors are illegal tests are also detected and reported.
+
+```bash
+inspec> control 'foo' do
+inspec>   thisisnonsense
+inspec> end
+NameError: undefined local variable or method `thisisnonsense' for #<#<Class:0x007fd63b571f98>:0x007fd639825cc8>
+from /usr/local/lib/ruby/gems/2.3.0/gems/rspec-expectations-3.5.0/lib/rspec/matchers.rb:967:in `method_missing'
+inspec> control 'foo' do
+inspec>   describe file('wut') do
+inspec>     its('thismakesnosense') { should cmp 'fail' }
+inspec>   end
+inspec> end
+  ✖  foo: File wut thismakesnosense  (undefined method `thismakesnosense' for File wut:Inspec::Resource::Registry::File)
+
+  Summary: 0 successful, 1 failures, 0 skipped
+```
+
+## Running a single InSpec command
+
+If you wish to run a single InSpec command and fetch its results, you
+may use the `-c` flag. This is similar to using `bash -c`.
+
+```bash
+$ inspec shell -c 'describe file("/Users/ksubramanian") do it { should exist } end'}
+Target:  local://
+
+  ✔  File /Users/ksubramanian should exist
+
+Summary: 1 successful, 0 failures, 0 skipped
+```
+
+```bash
+$ inspec shell --format json -c 'describe file("/Users/ksubramanian") do it { should exist } end'
+{"version":"0.30.0","profiles":{"":{"supports":[],"controls":{"(generated from in_memory.rb:1 5aab65c33fb1f133d9244017958eef64)":{"title":null,"desc":null,"impact":0.5,"refs":[],"tags":{},"code":"          rule = rule_class.new(id, profile_id, {}) do\n            res = describe(*args, &block)\n          end\n","source_location":{"ref":"/Users/ksubramanian/repo/chef/inspec/lib/inspec/profile_context.rb","line":184},"results":[{"status":"passed","code_desc":"File /Users/ksubramanian should exist","run_time":0.000747,"start_time":"2016-08-16 11:41:40 -0400"}]}},"groups":{"in_memory.rb":{"title":null,"controls":["(generated from in_memory.rb:1 5aab65c33fb1f133d9244017958eef64)"]}},"attributes":[]}},"other_checks":[],"summary":{"duration":0.001078,"example_count":1,"failure_count":0,"skip_count":0}}}
+```

data/inspec.gemspec

@@ -24,7 +24,7 @@ Gem::Specification.new do |spec|
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ['lib']
 
-  spec.add_dependency 'train', '>=0.19.0', '<1.0'
+  spec.add_dependency 'train', '>=0.19.1', '<1.0'
   spec.add_dependency 'thor', '~> 0.19'
   spec.add_dependency 'json', '>= 1.8', '< 3.0'
   spec.add_dependency 'rainbow', '~> 2'

data/lib/bundles/inspec-compliance/api.rb

@@ -9,18 +9,6 @@ module Compliance
   # API Implementation does not hold any state by itself,
   # everything will be stored in local Configuration store
   class API
-    # login method for pre-1.0 compliance server
-    def self.legacy_login_post(url, username, password, insecure)
-      # form request
-      # TODO: reuse post function
-      uri = URI.parse(url)
-      req = Net::HTTP::Post.new(uri.path)
-      req.basic_auth(username, password)
-      req.form_data={}
-
-      send_request(uri, req, insecure)
-    end
-
     # return all compliance profiles available for the user
     def self.profiles(config)
       url = "#{config['server']}/user/compliance"
@@ -86,19 +74,19 @@ Please login using `inspec compliance login https://compliance.test --user admin
       [res.is_a?(Net::HTTPSuccess), res.body]
     end
 
-    def self.post_refresh_token(url, token, insecure)
+    # Use username and refresh_toke to get an API access token
+    def self.get_token_via_refresh_token(url, refresh_token, insecure)
       uri = URI.parse("#{url}/login")
       req = Net::HTTP::Post.new(uri.path)
-      # req['Authorization'] = "Bearer #{token}"
-      req.body = { token: token }.to_json
+      req.body = { token: refresh_token }.to_json
       access_token = nil
       response = Compliance::HTTP.send_request(uri, req, insecure)
       data = response.body
-      if !data.nil?
+      if response.code == '200'
         begin
           tokendata = JSON.parse(data)
           access_token = tokendata['access_token']
-          msg = 'Successfully fetched access token'
+          msg = 'Successfully fetched API access token'
           success = true
         rescue JSON::ParserError => e
           success = false
@@ -106,7 +94,29 @@ Please login using `inspec compliance login https://compliance.test --user admin
         end
       else
         success = false
-        msg = 'Invalid refresh_token'
+        msg = "Failed to authenticate to #{url} \n\
+  Response code: #{response.code}\n  Body: #{response.body}"
+      end
+
+      [success, msg, access_token]
+    end
+
+    # Use username and password to get an API access token
+    def self.get_token_via_password(url, username, password, insecure)
+      uri = URI.parse("#{url}/login")
+      req = Net::HTTP::Post.new(uri.path)
+      req.body = { userid: username, password: password }.to_json
+      access_token = nil
+      response = Compliance::HTTP.send_request(uri, req, insecure)
+      data = response.body
+      if response.code == '200'
+        access_token = data
+        msg = 'Successfully fetched an API access token valid for 12 hours'
+        success = true
+      else
+        success = false
+        msg = "Failed to authenticate to #{url} \n\
+  Response code: #{response.code}\n  Body: #{response.body}"
       end
 
       [success, msg, access_token]

data/lib/bundles/inspec-compliance/cli.rb

@@ -22,9 +22,9 @@ module Compliance
     option :insecure, aliases: :k, type: :boolean,
       desc: 'Explicitly allows InSpec to perform "insecure" SSL connections and transfers'
     option :user, type: :string, required: false,
-      desc: 'Chef Compliance Username (for legacy auth)'
+      desc: 'Chef Compliance Username'
     option :password, type: :string, required: false,
-      desc: 'Chef Compliance Password (for legacy auth)'
+      desc: 'Chef Compliance Password'
     option :apipath, type: :string, default: '/api',
       desc: 'Set the path to the API, defaults to /api'
     option :token, type: :string, required: false,
@@ -38,7 +38,7 @@ module Compliance
       url = options['server'] + options['apipath']
       if !options['user'].nil? && !options['password'].nil?
         # username / password
-        _success, msg = login_legacy(url, options['user'], options['password'], options['insecure'])
+        _success, msg = login_username_password(url, options['user'], options['password'], options['insecure'])
       elsif !options['user'].nil? && !options['token'].nil?
         # access token
         _success, msg = store_access_token(url, options['user'], options['token'], options['insecure'])
@@ -199,7 +199,7 @@ module Compliance
     private
 
     def login_refreshtoken(url, options)
-      success, msg, access_token = Compliance::API.post_refresh_token(url, options['refresh_token'], options['insecure'])
+      success, msg, access_token = Compliance::API.get_token_via_refresh_token(url, options['refresh_token'], options['insecure'])
       if success
         config = Compliance::Configuration.new
         config['server'] = url
@@ -212,25 +212,17 @@ module Compliance
       [success, msg]
     end
 
-    def login_legacy(url, username, password, insecure)
+    def login_username_password(url, username, password, insecure)
       config = Compliance::Configuration.new
-      success, data = Compliance::API.legacy_login_post(url+'/oauth/token', username, password, insecure)
-      if !data.nil?
-        tokendata = JSON.parse(data)
-        if tokendata['access_token']
-          config['server'] = url
-          config['user'] = username
-          config['token'] = tokendata['access_token']
-          config['insecure'] = insecure
-          config['version'] = Compliance::API.version(url, insecure)
-          config.store
-          success = true
-          msg = 'Successfully authenticated'
-        else
-          msg = 'Response does not include a token'
-        end
-      else
-        msg = "Authentication failed for Server: #{url}"
+      success, msg, api_token = Compliance::API.get_token_via_password(url, username, password, insecure)
+      if success
+        config['server'] = url
+        config['user'] = username
+        config['token'] = api_token
+        config['insecure'] = insecure
+        config['version'] = Compliance::API.version(url, insecure)
+        config.store
+        success = true
       end
       [success, msg]
     end
@@ -245,10 +237,10 @@ module Compliance
       config['version'] = Compliance::API.version(url, insecure)
       config.store
 
-      [true, 'access token stored']
+      [true, 'API access token stored']
     end
 
-    # saves the a user refresh token supplied by the user
+    # saves a refresh token supplied by the user
     def store_refresh_token(url, refresh_token, verify, user, insecure)
       config = Compliance::Configuration.new
       config['server'] = url
@@ -260,13 +252,13 @@ module Compliance
       if !verify
         config.store
         success = true
-        msg = 'refresh token stored'
+        msg = 'API refresh token stored'
       else
-        success, msg, access_token = Compliance::API.post_refresh_token(url, refresh_token, insecure)
+        success, msg, access_token = Compliance::API.get_token_via_refresh_token(url, refresh_token, insecure)
         if success
           config['token'] = access_token
           config.store
-          msg = 'token verified and stored'
+          msg = 'API access token verified and stored'
         end
       end
 

data/lib/fetchers/git.rb

@@ -56,6 +56,10 @@ module Fetchers
       @repo_directory
     end
 
+    def cache_key
+      resolved_ref
+    end
+
     def archive_path
       @repo_directory
     end

data/lib/fetchers/local.rb

@@ -55,8 +55,23 @@ module Fetchers
       @target
     end
 
+    def writable?
+      File.directory?(@target)
+    end
+
+    def cache_key
+      sha256.to_s
+    end
+
+    def sha256
+      return nil if File.directory?(@target)
+      @archive_shasum ||= Digest::SHA256.hexdigest File.read(@target)
+    end
+
     def resolved_source
-      { path: @target }
+      h = { path: @target }
+      h[:sha256] = sha256 if sha256
+      h
     end
   end
 end

data/lib/fetchers/mock.rb

@@ -27,5 +27,9 @@ module Fetchers
     def resolved_source
       { mock_fetcher: true }
     end
+
+    def cache_key
+      ''
+    end
   end
 end

data/lib/fetchers/url.rb

@@ -3,6 +3,7 @@
 # author: Christoph Hartmann
 
 require 'uri'
+require 'digest'
 require 'tempfile'
 require 'open-uri'
 
@@ -81,34 +82,61 @@ module Fetchers
     end
 
     def fetch(path)
-      Inspec::Log.debug("Fetching URL: #{@target}")
-      @archive_path = download_archive(path)
+      @archive_path ||= download_archive(path)
+    end
+
+    def sha256
+      c = if @archive_path
+            File.read(@archive_path)
+          else
+            content
+          end
+      Digest::SHA256.hexdigest c
     end
 
     def resolved_source
-      { url: @target }
+      @resolved_source ||= { url: @target, sha256: sha256 }
+    end
+
+    def cache_key
+      sha256
+    end
+
+    def to_s
+      @target
     end
 
     private
 
-    # download url into archive using opts,
-    # returns File object and content-type from HTTP headers
-    def download_archive(path)
+    def open_target
+      Inspec::Log.debug("Fetching URL: #{@target}")
       http_opts = {}
       http_opts['ssl_verify_mode'.to_sym] = OpenSSL::SSL::VERIFY_NONE if @insecure
       http_opts['Authorization'] = "Bearer #{@token}" if @token
+      open(@target, http_opts)
+    end
 
-      remote = open(@target, http_opts)
+    def content
+      open_target.read
+    end
 
+    def file_type_from_remote(remote)
       content_type = remote.meta['content-type']
-      file_type = MIME_TYPES[content_type] ||
-                  throw(RuntimeError, 'Failed to resolve URL target, its '\
-                  "metadata did not match ZIP or TAR: #{content_type}")
+      file_type = MIME_TYPES[content_type]
 
-      # fall back to tar
       if file_type.nil?
-        fail "Could not determine file type for content type #{content_type}."
+        Inspec::Log.warn("Unrecognized content type: #{content_type}. Assuming tar.gz")
+        file_type = '.tar.gz'
       end
+
+      file_type
+    end
+
+    # download url into archive using opts,
+    # returns File object and content-type from HTTP headers
+    def download_archive(path)
+      remote = open_target
+      file_type = file_type_from_remote(remote)
       final_path = "#{path}#{file_type}"
       # download content
       archive = Tempfile.new(['inspec-dl-', file_type])