chef 13.12.14-universal-mingw32 → 14.0.190-universal-mingw32

data/acceptance/README.md

@@ -0,0 +1,137 @@
+# Acceptance Testing for Chef Client
+
+This folder contains acceptance tests that are required for Chef client release readiness.
+
+## Getting started
+
+The tests use the _chef-acceptance_ gem as the high level framework. All the gems needed to run these tests can be installed with Bundler from this directory.
+
+### Important Note!
+
+Before running chef-acceptance, you _MUST_ do the following on your current session:
+
+```shell
+export APPBUNDLER_ALLOW_RVM=true
+```
+
+## Pre-requisites / One time set up
+
+### Set up for local VM (Vagrant)
+
+If you intend to run the acceptance tests on a local VM, the supported solution is to use Vagrant. Ensure that Vagrant is installed on the machine that tests will run from, along with a virtualization driver (E.g.: VirtualBox).
+
+Set up the KITCHEN_DRIVER environment variable appropriately (value should be "vagrant"). E.g.:
+
+```shell
+export KITCHEN_DRIVER=vagrant
+```
+
+Add this to your shell profile or startup script as needed.
+
+### Set up for cloud VM (EC2)
+
+If you intend to run the acceptance tests on a cloud VM, the supported solution is to use EC2.
+
+The steps you will need to do are:
+
+1. Add your AWS credentials to the machine - e.g., to the ~/.aws/credentials directory.
+
+  - The easiest way to do this is to download the aws command line (`brew install awscli` on OS/X) and run `aws configure`.
+
+2. Create or import a SSH key to AWS. (If you already have one, you can skip the import/create)
+
+  - In the AWS console, click Key Pairs, then Create Key Pair or Import Key Pair.
+
+3. Copy or move the private key file (USERNAME.pem) to the SSH folder (e.g. `~/.ssh/USERNAME.pem`).
+
+  - If you Created a key pair in step 2, download the private key and move it to `~/.ssh`.
+  - If you Importd a key pair in step 2, just ensure your private key is in `~/.ssh` and has the same name as the key pair (`~/.ssh/USERNAME` or `~/.ssh/USERNAME.pem`).
+
+4. Set AWS_SSH_KEY_ID to the SSH key name.
+
+  - This is **optional** if your AWS SSH key name is your local username.
+  - You may want to set this in your shell `.profile`.
+
+    ```shell
+    export AWS_SSH_KEY_ID=name-of-private-key
+    ```
+
+5. Set the private key to only be readable by root
+
+  ```shell
+  chmod 0400 ~/.ssh/USERNAME.pem
+  ```
+
+6. Set up the KITCHEN_DRIVER environment variable appropriately (value should be "ec2"). (This is optional, as ec2 is the default.) E.g.:
+
+  ```shell
+  export KITCHEN_DRIVER=ec2
+  ```
+
+  Add this to your shell profile or startup script as needed.
+
+7. **Connect to Chef VPN**. The instances you create will not have public IPs!
+
+## Setting up and running a test suite
+
+To get started, do a bundle install from the acceptance directory:
+
+```shell
+chef/acceptance$ bundle install --binstubs
+```
+
+To get some basic info and ensure chef-acceptance can be run, do:
+
+```shell
+chef/acceptance$ bin/chef-acceptance info
+```
+
+To run a particular test suite, do the following:
+
+```shell
+chef/acceptance$ bin/chef-acceptance test TEST_SUITE
+```
+
+To restrict which OS's will run, use the KITCHEN_INSTANCES environment variable:
+
+```shell
+chef/acceptance$ export KITCHEN_INSTANCES=*-ubuntu-1404
+chef/acceptance$ bin/chef-acceptance test cookbook-git
+```
+
+If KITCHEN_INSTANCES is not specified, the default instances are default-ubuntu-1404 and default-windows-windows-2012r2\. All selected instances will be run in _parallel_ if the driver supports it (ec2 does, vagrant doesn't).
+
+## Optional Settings
+
+In addition to the environment settings above, there are a number of key values that are available to set for changing the way the acceptance tests are run.
+
+### KITCHEN_CHEF_CHANNEL
+
+Use this setting to specify which channel we will pull the chef build from. The default is to use the "current" channel, unless the ARTIFACTORY_USERNAME is set (which normally happens when running under Jenkins), in which case the default is changed to "unstable".
+
+```shell
+export KITCHEN_CHEF_CHANNEL=name-of-channel
+```
+
+### KITCHEN_CHEF_VERSION
+
+Use this setting to override the version of the Chef client that is installed. The default is to get the latest version in the desired channel.
+
+```shell
+export KITCHEN_CHEF_VERSION=version-of-chef-client
+```
+
+### ARTIFACTORY_USERNAME / ARTIFACTORY_PASSWORD
+
+If the desired channel to get the Chef client from is "unstable", the following settings need to be exported:
+
+```shell
+export ARTIFACTORY_USERNAME=username
+export ARTIFACTORY_PASSWORD=password
+```
+
+## Future Work
+
+Currently, there is no simple mechanism for chef-acceptance to build an Omnibus package of the developers local chef instance and run acceptance tests on it - the only packages that can be exercised are ones that come from one of the pipeline channels (unstable, current or stable).
+
+This is not an issue when adding acceptance tests for pre-existing functionality (as that functionality is presumed to already be in a build in one of the pipeline channels).

data/acceptance/basics/.acceptance/acceptance-cookbook/.gitignore

@@ -0,0 +1,2 @@
+nodes/
+tmp/

data/acceptance/basics/.acceptance/acceptance-cookbook/metadata.rb

@@ -0,0 +1,3 @@
+name 'acceptance-cookbook'
+
+depends "kitchen_acceptance"

data/acceptance/basics/.acceptance/acceptance-cookbook/recipes/destroy.rb

@@ -0,0 +1 @@
+kitchen "destroy"

data/acceptance/basics/.acceptance/acceptance-cookbook/recipes/provision.rb

@@ -0,0 +1 @@
+kitchen "converge"

data/acceptance/basics/.acceptance/acceptance-cookbook/recipes/verify.rb

@@ -0,0 +1 @@
+kitchen "verify"

data/acceptance/basics/.kitchen.yml

@@ -0,0 +1,4 @@
+suites:
+  - name: chef-current-install
+    includes: [ubuntu-14.04, windows-server-2012r2]
+    run_list:

data/acceptance/basics/test/integration/chef-current-install/serverspec/chef_client_spec.rb

@@ -0,0 +1,19 @@
+
+require "spec_helper"
+
+gem_path = "/opt/chef/embedded/bin/gem"
+white_list = %w{addressable chef-config json minitest rake}
+
+describe "gem list" do
+  it "should not have non-whitelisted duplicate gems" do
+    gems = command("#{gem_path} list").stdout
+
+    duplicate_gems = gems.lines().select { |l| l.include?(",") }.collect { |l| l.split(" ").first }
+    puts "Duplicate gems found: #{duplicate_gems}" if duplicate_gems.length > 0
+
+    non_whitelisted_duplicates = duplicate_gems.select { |l| !white_list.include?(l) }
+    puts "Non white listed duplicates: #{non_whitelisted_duplicates}" if non_whitelisted_duplicates.length > 0
+
+    (non_whitelisted_duplicates.length).should be == 0
+  end
+end

data/acceptance/basics/test/integration/chef-current-install/serverspec/spec_helper.rb

@@ -0,0 +1,6 @@
+require "serverspec"
+require "pathname"
+
+set :backend, :exec
+
+set :path, "/bin:/usr/local/bin:$PATH"

data/acceptance/basics/test/integration/helpers/serverspec/Gemfile

@@ -0,0 +1,8 @@
+source "https://rubygems.org"
+
+# Until https://github.com/test-kitchen/busser-serverspec/pull/42 is merged and
+# released, we need to include rake and rspec-core in the Gemfile
+gem "rake"
+gem "rspec-core"
+gem "busser-serverspec"
+gem "serverspec"

data/acceptance/data-collector/.acceptance/acceptance-cookbook/.gitignore

@@ -0,0 +1,2 @@
+nodes/
+tmp/

data/acceptance/data-collector/.acceptance/acceptance-cookbook/metadata.rb

@@ -0,0 +1,3 @@
+name 'acceptance-cookbook'
+depends "kitchen_acceptance"
+depends "data-collector-test"

data/acceptance/data-collector/.acceptance/acceptance-cookbook/recipes/destroy.rb

@@ -0,0 +1,2 @@
+log "Running 'destroy' recipe from the acceptance-cookbook in directory '#{node['chef-acceptance']['suite-dir']}'"
+kitchen "destroy"

data/acceptance/data-collector/.acceptance/acceptance-cookbook/recipes/provision.rb

@@ -0,0 +1,2 @@
+log "Running 'provision' recipe from the acceptance-cookbook in directory '#{node['chef-acceptance']['suite-dir']}'"
+kitchen "converge"

data/acceptance/data-collector/.acceptance/acceptance-cookbook/recipes/verify.rb

@@ -0,0 +1,2 @@
+log "Running 'verify' recipe from the acceptance-cookbook in directory '#{node['chef-acceptance']['suite-dir']}'"
+kitchen "verify"

data/acceptance/data-collector/.acceptance/data-collector-test/.gitignore

@@ -0,0 +1,16 @@
+.vagrant
+Berksfile.lock
+*~
+*#
+.#*
+\#*#
+.*.sw[a-z]
+*.un~
+
+# Bundler
+Gemfile.lock
+bin/*
+.bundle/*
+
+.kitchen/
+.kitchen.local.yml

data/acceptance/data-collector/.acceptance/data-collector-test/Berksfile

@@ -0,0 +1,3 @@
+source 'https://supermarket.chef.io'
+
+metadata

data/acceptance/data-collector/.acceptance/data-collector-test/files/default/api.rb

@@ -0,0 +1,101 @@
+require "json"
+require "sinatra"
+
+class Chef
+  class Node
+    # dummy class for JSON parsing
+  end
+end
+
+module ApiHelpers
+  def self.payload_type(payload)
+    message_type = payload["message_type"]
+    status       = payload["status"]
+
+    message_type == "run_converge" ? "#{message_type}.#{status}" : message_type
+  end
+end
+
+class Counter
+  class << self
+    attr_accessor :counters
+
+    def counters
+      @counters ||= Hash.new { |h, k| h[k] = 0 }
+    end
+
+    def reset
+      @counters = nil
+    end
+
+    def increment(payload)
+      counter_name = ApiHelpers.payload_type(payload)
+      counters[counter_name] += 1
+    end
+
+    def to_json
+      counters.to_json
+    end
+  end
+end
+
+class MessageCache
+  include ApiHelpers
+
+  class << self
+    attr_accessor :message_cache
+
+    def message_cache
+      @message_cache ||= {}
+    end
+
+    def reset
+      @message_cache = nil
+    end
+
+    def store(payload)
+      cache_key = ApiHelpers.payload_type(payload)
+
+      message_cache[cache_key] = payload
+    end
+
+    def fetch(cache_key)
+      message_cache[cache_key].to_json
+    end
+  end
+end
+
+Counter.reset
+
+get "/" do
+  "Data Collector API server"
+end
+
+get "/reset-counters" do
+  Counter.reset
+  "counters reset"
+end
+
+get "/counters" do
+  Counter.to_json
+end
+
+get "/cache/:key" do |cache_key|
+  MessageCache.fetch(cache_key)
+end
+
+get "/reset-cache" do
+  MessageCache.reset
+  "cache reset"
+end
+
+post "/data-collector/v0" do
+  body = request.body.read
+  payload = JSON.load(body)
+
+  Counter.increment(payload)
+  MessageCache.store(payload)
+
+  status 201
+  "message received"
+end

data/acceptance/data-collector/.acceptance/data-collector-test/files/default/apigemfile

@@ -0,0 +1,3 @@
+source "https://rubygems.org"
+
+gem "sinatra"

data/acceptance/data-collector/.acceptance/data-collector-test/files/default/client-rb-both-mode.rb

@@ -0,0 +1,4 @@
+chef_server_url           "http://localhost:8889"
+node_name                 "data-collector-test"
+data_collector.server_url "http://localhost:9292/data-collector/v0"
+data_collector.mode       :both

data/acceptance/data-collector/.acceptance/data-collector-test/files/default/client-rb-client-mode.rb

@@ -0,0 +1,4 @@
+chef_server_url           "http://localhost:8889"
+node_name                 "data-collector-test"
+data_collector.server_url "http://localhost:9292/data-collector/v0"
+data_collector.mode       :client

data/acceptance/data-collector/.acceptance/data-collector-test/files/default/client-rb-no-endpoint.rb

@@ -0,0 +1,2 @@
+chef_server_url "http://localhost:8889"
+node_name       "data-collector-test"

data/acceptance/data-collector/.acceptance/data-collector-test/files/default/client-rb-solo-mode.rb

@@ -0,0 +1,4 @@
+chef_server_url           "http://localhost:8889"
+node_name                 "data-collector-test"
+data_collector.server_url "http://localhost:9292/data-collector/v0"
+data_collector.mode       :solo

data/acceptance/data-collector/.acceptance/data-collector-test/files/default/config.ru

@@ -0,0 +1,2 @@
+require_relative "./api"
+run Sinatra::Application

data/acceptance/data-collector/.acceptance/data-collector-test/metadata.rb

@@ -0,0 +1,7 @@
+name 'data-collector-test'
+maintainer 'Adam Leff'
+maintainer_email 'adamleff@chef.io'
+license 'Apache-2.0'
+description 'Installs/Configures data-collector-test'
+long_description 'Installs/Configures data-collector-test'
+version '0.1.0'

data/acceptance/data-collector/.acceptance/data-collector-test/recipes/default.rb

@@ -0,0 +1,44 @@
+
+embedded_path = "/opt/chef/embedded/bin:#{ENV["PATH"]}"
+
+api_root_dir = "/var/opt/data_collector_api"
+
+directory api_root_dir do
+  recursive true
+end
+
+cookbook_file ::File.join(api_root_dir, "Gemfile") do
+  source "apigemfile"
+end
+
+cookbook_file ::File.join(api_root_dir, "config.ru")
+
+cookbook_file ::File.join(api_root_dir, "api.rb")
+
+execute "bundle install --binstubs" do
+  cwd api_root_dir
+  environment({ "PATH" => embedded_path })
+end
+
+pid_file    = "/var/run/api.pid"
+running_pid = ::File.exist?(pid_file) ? ::File.read(pid_file).strip : nil
+
+execute "kill existing API process" do
+  command "kill #{running_pid}"
+  environment({ "PATH" => embedded_path })
+  not_if { running_pid.nil? }
+end
+
+execute "start API" do
+  command "bin/rackup -D -P #{pid_file}"
+  environment({ "PATH" => embedded_path })
+  cwd api_root_dir
+end
+
+directory "/etc/chef"
+
+["both-mode", "client-mode", "no-endpoint", "solo-mode"].each do |config_file|
+  cookbook_file "/etc/chef/#{config_file}.rb" do
+    source "client-rb-#{config_file}.rb"
+  end
+end

data/acceptance/data-collector/.kitchen.yml

@@ -0,0 +1,9 @@
+verifier:
+  name: busser
+
+suites:
+  - name: default
+    includes: 
+      - ubuntu-14.04
+    run_list:
+      - recipe[data-collector-test::default]

data/acceptance/data-collector/Berksfile

@@ -0,0 +1,3 @@
+source "https://supermarket.chef.io"
+
+cookbook "data-collector-test", path: File.join(File.expand_path(File.dirname(__FILE__)), ".acceptance", "data-collector-test")

data/acceptance/data-collector/test/integration/default/serverspec/default_spec.rb

@@ -0,0 +1,208 @@
+#
+# Author:: Adam Leff (<adamleff@chef.io)
+#
+# Copyright:: Copyright 2012-2016, Chef Software Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require "json"
+require "serverspec"
+
+set :backend, :exec
+
+class Chef
+  class Node
+    # dummy class for parsing JSON action messages
+  end
+end
+
+shared_examples_for "reset counters" do
+  it "resets the counters" do
+    expect(command("curl http://localhost:9292/reset-counters").exit_status).to eq(0)
+  end
+end
+
+shared_examples_for "reset cache" do
+  it "resets the message cache" do
+    expect(command("curl http://localhost:9292/reset-cache").exit_status).to eq(0)
+  end
+end
+
+shared_examples_for "successful chef run" do |cmd|
+  include_examples "reset counters"
+  include_examples "reset cache"
+
+  it "runs chef and expects a zero exit status" do
+    expect(command(cmd).exit_status).to eq(0)
+  end
+end
+
+shared_examples_for "unsuccessful chef run" do |cmd|
+  include_examples "reset counters"
+  include_examples "reset cache"
+
+  it "runs chef and expects a non-zero exit status" do
+    expect(command(cmd).exit_status).not_to eq(0)
+  end
+end
+
+shared_examples_for "counter checks" do |counters_to_check|
+  counters_to_check.each do |counter, value|
+    it "counter #{counter} should equal #{value.inspect}" do
+      counter_values = JSON.load(command("curl http://localhost:9292/counters").stdout)
+      expect(counter_values[counter]).to eq(value)
+    end
+  end
+end
+
+shared_examples_for "run_start payload check" do
+  describe "run_start message" do
+    let(:required_fields) do
+      %w{
+        chef_server_fqdn
+        entity_uuid
+        id
+        message_version
+        message_type
+        node_name
+        organization_name
+        run_id
+        source
+        start_time
+      }
+    end
+    let(:optional_fields) { [] }
+
+    it "is not missing any required fields" do
+      payload = JSON.load(command("curl http://localhost:9292/cache/run_start").stdout)
+      missing_fields = required_fields.select { |key| !payload.key?(key) }
+      expect(missing_fields).to eq([])
+    end
+
+  end
+end
+
+shared_examples_for "run_converge.success payload check" do
+  describe "run_converge success message" do
+    let(:required_fields) do
+      %w{
+        chef_server_fqdn
+        entity_uuid
+        id
+        end_time
+        expanded_run_list
+        message_type
+        message_version
+        node
+        node_name
+        organization_name
+        policy_name
+        policy_group
+        resources
+        run_id
+        run_list
+        source
+        start_time
+        status
+        total_resource_count
+        updated_resource_count
+        deprecations
+      }
+    end
+    let(:optional_fields) { [] }
+
+    it "is not missing any required fields" do
+      payload = JSON.load(command("curl http://localhost:9292/cache/run_converge.success").stdout)
+      missing_fields = required_fields.select { |key| !payload.key?(key) }
+      expect(missing_fields).to eq([])
+    end
+
+  end
+end
+
+shared_examples_for "run_converge.failure payload check" do
+  describe "run_converge failure message" do
+    let(:required_fields) do
+      %w{
+        chef_server_fqdn
+        entity_uuid
+        error
+        id
+        end_time
+        expanded_run_list
+        message_type
+        message_version
+        node
+        node_name
+        organization_name
+        policy_name
+        policy_group
+        resources
+        run_id
+        run_list
+        source
+        start_time
+        status
+        total_resource_count
+        updated_resource_count
+        deprecations
+      }
+    end
+    let(:optional_fields) { [] }
+
+    it "is not missing any required fields" do
+      payload = JSON.load(command("curl http://localhost:9292/cache/run_converge.failure").stdout)
+      missing_fields = required_fields.select { |key| !payload.key?(key) }
+      expect(missing_fields).to eq([])
+    end
+
+    it "does not have any extra fields" do
+      payload = JSON.load(command("curl http://localhost:9292/cache/run_converge.failure").stdout)
+      extra_fields = payload.keys.select { |key| !required_fields.include?(key) && !optional_fields.include?(key) }
+      expect(extra_fields).to eq([])
+    end
+  end
+end
+
+describe "CCR with no data collector URL configured" do
+  include_examples "successful chef run", "chef-client -z -c /etc/chef/no-endpoint.rb"
+  include_examples "counter checks", { "run_start" => nil, "run_converge.success" => nil, "run_converge.failure" => nil }
+end
+
+describe "CCR, local mode, config in solo mode" do
+  include_examples "successful chef run", "chef-client -z -c /etc/chef/solo-mode.rb"
+  include_examples "counter checks", { "run_start" => 1, "run_converge.success" => 1, "run_converge.failure" => nil }
+  include_examples "run_start payload check"
+  include_examples "run_converge.success payload check"
+end
+
+describe "CCR, local mode, config in client mode" do
+  include_examples "successful chef run", "chef-client -z -c /etc/chef/client-mode.rb"
+  include_examples "counter checks", { "run_start" => nil, "run_converge.success" => nil, "run_converge.failure" => nil }
+end
+
+describe "CCR, local mode, config in both mode" do
+  include_examples "successful chef run", "chef-client -z -c /etc/chef/both-mode.rb"
+  include_examples "counter checks", { "run_start" => 1, "run_converge.success" => 1, "run_converge.failure" => nil }
+  include_examples "run_start payload check"
+  include_examples "run_converge.success payload check"
+end
+
+describe "CCR, local mode, config in solo mode, failed run" do
+  include_examples "unsuccessful chef run", "chef-client -z -c /etc/chef/solo-mode.rb -r 'recipe[cookbook-that-does-not-exist::default]'"
+  include_examples "counter checks", { "run_start" => 1, "run_converge.success" => nil, "run_converge.failure" => 1 }
+  include_examples "run_start payload check"
+  include_examples "run_converge.failure payload check"
+end