leibniz 0.1.0 → 0.2.0

data/.gitignore

@@ -15,3 +15,4 @@ spec/reports
 test/tmp
 test/version_tmp
 tmp
+.idea/

data/Gemfile

@@ -1,4 +1,7 @@
 source 'https://rubygems.org'
 
+#ruby=ruby-2.0.0
+#ruby-gemset=leibniz
+
 # Specify your gem's dependencies in leibniz.gemspec
 gemspec

data/README.md

@@ -1,6 +1,9 @@
 # Leibniz
 
-Leibniz is an integration testing tool aimed at the Chef community.
+Leibniz is simple utility which provides the ability to launch
+infrastructure using Test Kitchen, and run acceptance tests against
+that infrastructure.  It is designed to be used as part of a set of
+Cucumber / Gherkin features.
 
 ## Installation
 
@@ -18,7 +21,246 @@ Or install it yourself as:
 
 ## Usage
 
-TODO: Write usage instructions here
+### The Leibniz CLI
+
+Since version 0.2.0, Leibniz provides a basic CLI which will create and populate an example Cucumber feature to get you started.  You can call this using:
+
+    leibniz init
+
+### Getting Started
+
+Leibniz takes the view that acceptance testing of infrastructure
+should be performed from the outside in, and make assertions from the
+perspective of an external user consuming the delivered
+infrastructure.
+
+To get started, you will need to write some features and some steps.
+Depending on how you build your infrastructure (at present the assumed
+approach is Berkshelf and wrapper cookbooks, but there's no reason why
+it wouldn't work with Librarian-chef, or some other approach).
+
+#### Using Berkshelf
+
+Assuming you have Berkshelf installed, you can use the in-built
+cookbook generator to create your wrapper cookbook.  The alternative
+is to create a cookbook directory, or use knife to create a cookbook,
+and then add 'berkshelf' to a Gemfile, and run bundle install, followed by berks init.
+
+Either way, once you have a cookbook which has been 'berksified' you
+will have something that looks like this:
+
+```
+.
+├── Berksfile
+├── Gemfile
+├── LICENSE
+├── README.md
+├── Thorfile
+├── Vagrantfile
+├── attributes
+├── chefignore
+├── definitions
+├── files
+│   └── default
+├── libraries
+├── metadata.rb
+├── providers
+├── recipes
+│   └── default.rb
+├── resources
+└── templates
+    └── default
+```
+
+#### Writing your first feature
+
+We are going to take you through the process of iterating on the creation of a feature and its step definitions. In reality you might merge some of these steps together and not run cucumber as often as we do here, but this illustrates every step in the process.
+
+First we need to create a directory to contain our features, then write our first feature:
+
+```
+mkdir features
+cd features
+vi generic_webpage.feature
+```
+
+The feature you write needs to have a `Background` section like this:
+
+```
+Background:
+
+  Given I have provisioned the following infrastructure:
+  | Server Name     | Operating System | Version | Chef Version | Run List                 |
+  | generic_webpage | ubuntu           |   12.04 |       11.8.0 | generic_webpage::default |
+  And I have run Chef
+```
+
+This background section is responsible for provisioning infrastructure and getting it into a state whereupon we can run some acceptance tests against it.  The title of each column is defined by Leibiniz:
+
+- `Server Name`  - this is the name of the machine you will be provisioning.  Leibniz will prepend `leibniz` to the name and will launch a machine with this name.
+- `Operating System` - this translates to the base OS of a Vagrant box which is downloaded on demand.  The boxes used are Opscode's 'Bento' boxes, and have nothing other than a base OS installed.  At present `ubuntu`, `debian`, `centos` and `fedora` are supported.
+- `Version` - this is version of the Operating System.  See the Bento website for an up-to-date specification of the available versions.
+- `Chef Version` - this is the version of the Chef 'client' software to be installed.
+- `Run List` - this is the Chef run list which will be used when the node is converged.
+
+These two steps are satisfied by Leibniz.  We need to ensure the Leibniz library is available to Chef.  To do this, add `leibniz` to your Gemfile, and run `bundle install`.  Then create a `support` directory under your `features` directory, and within the `support` directory, create an `env.rb` file.  This should read:
+
+```
+require 'leibniz'
+```
+
+Now create your step definitions:
+
+```
+mkdir features/step_definitions
+vi features/step_definitions/generic_webpage_steps.rb
+``` 
+
+The following steps will build and converge the infrastructure described in the table:
+
+```
+Given(/^I have provisioned the following infrastructure:$/) do |specification|
+  @infrastructure = Leibniz.build(specification)
+end
+
+Given(/^I have run Chef$/) do
+  @infrastructure.destroy
+  @infrastructure.converge
+end
+```
+
+At present, Leibniz only knows how to provision infrastructure using
+the Vagrant driver.  By default this will assume you have Virtualbox
+on the system where you are running Cucumber.  A top priority is to
+support other Kitchen drivers, which will enable infrastructure to be
+provisioned on cloud platforms, via LXC or Docker, or just with
+Vagrant.
+
+Once you have your feature, env.rb and steps in place, you can run
+`cucumber`.  This will build the infrastructure you described using
+Chef.
+
+You may find it useful to tail the logs during this process:
+
+```
+tail -f .kitchen/logs/leibniz-generic-webpage.log
+```
+
+If all goes well, you should see something like:
+
+```
+Feature: Serve a generic webpage
+
+  In order to demonstrate how Leibniz works
+  As an infrastructure developer
+  I want to be able to serve a generic webpage and test it
+
+  Background:                                              # features/crap_webpage.feature:7
+    Given I have provisioned the following infrastructure: # features/step_definitions/generic_webpage_steps.rb:1
+      | Server Name     | Operating System | Version | Chef Version | Run List                 |
+      | generic_webpage | ubuntu           | 12.04   | 11.8.0       | generic_webpage::default |
+Using generic_webpage (0.1.0) from metadata
+    And I have run Chef                                    # features/step_definitions/generic_webpage_steps.rb:5
+
+0 scenarios
+2 steps (2 passed)
+0m58.613s
+```
+
+At this stage we have only provisioned the machine per the table we provided in the feature.  We now need to describe an example of what the infrastructure does.  Open the feature and add an example:
+
+```
+  Scenario: Infrastructure developer can see generic webpage
+    Given a URL "http://generic-webpage.com"
+    When I browse to the URL
+    Then I should see "This is a generic webpage"
+```
+
+When we run cucumber again, we should be told that our steps are undefined. Cucumber will suggest some snippets we can use:
+
+```
+You can implement step definitions for undefined steps with these snippets:
+
+Given(/^a URL "(.*?)"$/) do |arg1|
+  pending # express the regexp above with the code you wish you had
+end
+
+When(/^I browse to the URL$/) do
+  pending # express the regexp above with the code you wish you had
+end
+
+Then(/^I should see "(.*?)"$/) do |arg1|
+  pending # express the regexp above with the code you wish you had
+end
+```
+Copy and paste these into `features/step_definitions/generic_webpage_steps.rb`, then run cucumber again.  We should now see that the step runs, but is marked as pending - that is we haven't implemented the acceptance test.
+
+The idea of Leibniz is to make the provisioning and converging of infrastructure nodes as painless and flexible as possible, enabling the infrastructure developer to dive into writing the acceptance tests right away.  Future versions of the library may ship some useful features to help with common acceptance test types.
+
+We now need to write the implementation of our acceptance tests:
+
+- Given a URL "http://generic-webpage.com"
+- When I browse to the URL
+- Then I should see "This is a generic webpage"
+
+The first step simply requires us to capture a host header that we can use as part of an http client:
+
+```
+Given(/^a URL "(.*?)"$/) do |url|
+  @host_header = url.split("/").last
+end
+```
+
+The second step requires us to use an http client. There are many options available for this, Faraday is a simple one:
+
+```
+When(/^I browse to the URL$/) do
+  connection = Faraday.new(url: "http://#{@infrastructure['generic-webpage'].ip}", headers: { 'Host' => @host_header }) do |faraday|
+    faraday.adapter Faraday.default_adapter
+  end
+  @page = connection.get('/').body
+end
+```
+
+Note: the ip of the infrastructure we have built is available as part of the @infrastructure object returned by Leibniz.
+
+The final step is a simple rspec expectation:
+
+```
+Then(/^I should see "(.*?)"$/) do |content|
+  expect (@page).to match /#{content}/
+end
+```
+
+When we run cucumber this time, the tests will fail because we've not implemented anything to actually serve a website, nor have we deployed the website for the web server to serve.
+
+Making the tests pass (i.e. writing the Chef code to deploy a web server and serve a static web page) is left as an exercise for the reader. One the code is written, running cucumber again will converge the node and run the acceptance tests, resulting in the tests passing, like this:
+
+```
+Feature: Serve a generic webpage
+
+  In order to demonstrate how Leibniz works
+  As an infrastructure developer
+  I want to be able to serve a generic webpage and test it
+
+  Background:                                              # features/generic_webpage.feature:7
+    Given I have provisioned the following infrastructure: # features/step_definitions/generic_webpage_steps.rb:3
+      | Server Name     | Operating System | Version | Chef Version | Run List                 |
+      | generic_webpage | ubuntu           | 12.04   | 11.8.0       | generic_webpage::default |
+Using generic_webpage (0.1.0)
+Using apt (2.3.0)
+Using apache2 (1.8.4)
+    And I have run Chef                                    # features/step_definitions/generic_webpage_steps.rb:7
+
+  Scenario: Infrastructure developer can see generic webpage # features/generic_webpage.feature:14
+    Given a URL "http://generic-webpage.com"                 # features/step_definitions/generic_webpage_steps.rb:12
+    When I browse to the URL                                 # features/step_definitions/generic_webpage_steps.rb:16
+    Then I should see "This is a generic webpage"            # features/step_definitions/generic_webpage_steps.rb:23
+
+1 scenario (1 passed)
+5 steps (5 passed)
+1m25.227s
+```
 
 ## Contributing
 

data/TODO.md

@@ -0,0 +1,11 @@
+# Things to do on Leibniz
+
+- High level documentation
+- Remove some nasty magic numbers
+- On a physical box, prove that we can run a Leibniz job from Jenkins
+- Try to get Leibniz to with TK to with the EC2 driver (or LXC / Docker)
+- Demonstrate that this can work with Librarian?
+- Demonstrate (and document) how to make Leibniz use the chef-zero provisioner
+- Provide a generator which creates the outline stuff
+- It'd be nice to be able to run the cucumber test and skip the teardown and setup process
+- It'd be ace if we could not have to install chef each time (but that might mean a backed image)

data/WISHLIST.md

@@ -0,0 +1,85 @@
+# Things I'd like this tool to have
+## in no particular order
+
+### Jenkins
+
+Or some other other CI workflow. At present we have a pretty slick Continuous Deployment pipeline for our code. It would be great if we could run our infrastructure code through the same sort of system, because at the moment, although we have tools like Etsy's [knife-spork](https://github.com/jonlives/knife-spork) to guard against idiocy, it's basically gated by me. Fallible, error-prone me.
+
+I know Zach mentioned something about a [Jenkins workflow](https://github.com/Atalanta/cucumber-chef/issues/101) a while ago, but I never heard anything more.
+
+#### Current Status
+
+- A project could be checked out and cucumber run by Jenkins or Travis
+- However at present Leibniz only knows how to build machines via TK with the Vagrant driver
+- This would mean we couldn't run it on Travis or on a 'cloud' Jenkins box; ie the Jenkins box would need to run Vagrant
+  - that said I'm not sure whether Vagrant itself would be able to use the EC2 'provider'?
+
+#### Next Steps
+
+- On a physical box, prove that we can run a Leibniz job from Jenkins
+- Try to get Leibniz to with TK to with the EC2 driver (or LXC / Docker)
+
+### Test configuration per-project
+
+[I mooted this back in April](https://github.com/Atalanta/cucumber-chef/pull/117) but then I went on holiday and kind of forgot about it. The somewhat lashed-together solution I came up with there looks a bit clunky now, but I think the idea still holds. Right now, I'm having to keep several different Labfiles around and symlink the correct one each time.
+
+#### Current Status
+
+- Leibniz simply provides an interface to Test Kitchen, and passes over a run list.
+- This means we can have features / steps per project or per cookbook - whatever works
+
+#### Next Steps
+
+- Demonstrate that this can work with Librarian?
+
+
+### Looser coupling of moving parts
+
+Here's the thing: the most complex project I'm currently managing with cuke-chef has 5 different types of node. In order to test things like Chef-search correctly, I need to spin up one of each from the Labfile, which incurs a huge first-run penalty. Subsequent runs are better, but still incredibly time-consuming as it seems to provision all the nodes on each run (which is not unreasonable, I guess).
+
+Maybe there are already clever things I could with mocking and so on but I've never really looked into that. But whatever, being able to exercise only the required node(s) on a given test run (without commenting-out whole blocks from the Labfile, which is my current anti-pattern) would be splendid.
+
+#### Current Status
+
+- Acceptance tests shouldn't mock - we want to exercise all the pieces
+- TK allows us to use Chef Zero - which gives us an in memory Chef server, including search, which is really really quick
+- At lower levels (chefspec) we can stub the search and return data
+
+#### Next Steps
+
+- Demonstrate (and document) how to make Leibniz use the chef-zero provisioner
+
+### Full-stack testing
+
+Right now, the way I'm using cuke-chef isn't at all BDD. My steps say things like
+
+```
+Scenario: www vhost is correct
+    * file "/var/www/www/current/vhost" should contain
+  """
+  upstream www {
+    server 127.0.0.1:3020;
+  }
+
+```
+
+whereas what I *should* care about would be more like
+
+```
+  Scenario: Home page
+    Given I am on "the home page"
+    Then I should see "All work and no play makes Jack a dull boy"
+```
+
+Now of course this sort of stuff is tested in the apps themselves so maybe I'm looking at this from the wrong end, or maybe others are already doing this with cuke-chef and and I'm just Doing It Wrong.
+
+But this, combined with the *Test configuration per-project* idea from above, would maybe let us test the entire stack from base OS to working app, which has a certain appeal.
+
+#### Current Status
+
+- The lower-level 'vhost is correct' steps should be run post-converge on the node under test using TK + whatever you want to write tests with (I recommend serverspec)
+- The high-level BDD-style acceptance tests (the homepage does what it should) is precisely what Leibniz & Cucumber are for
+
+# Annoyances with current cucumber-chef
+
+* Occasional failure of packet-passing

data/bin/leibniz

@@ -0,0 +1,52 @@
+#!/usr/bin/env ruby
+
+$:.unshift File.join(File.dirname(__FILE__), %w{.. lib})
+require 'thor'
+require 'leibniz'
+require 'pathname'
+
+# Trap interrupts to quit cleanly. See
+# https://twitter.com/mitchellh/status/283014103189053442
+Signal.trap("INT") { exit 1 }
+
+class LeibnizCli < Thor
+  include Thor::Actions
+
+  desc 'init', 'Set up a project ready for Leibniz Acceptance Testing'
+  def init
+    set_template_root
+    create_leibniz_yaml
+    create_skeleton_feature
+    append_to_gitignore(".leibniz/")
+  end
+  
+  desc "version", "Print Leibniz's version information"
+    def version
+      say "Leibniz version #{Leibniz::VERSION}"
+    end
+    map %w(-v --version) => :version
+
+    private
+
+    def set_template_root
+      LeibnizCli.source_root Pathname.new(File.expand_path('../../templates', __FILE__))
+    end
+
+    def create_leibniz_yaml
+      empty_directory '.leibniz'
+    end
+
+    def create_skeleton_feature
+      directory 'features'
+    end
+
+    def append_to_gitignore(line)
+      create_file(".gitignore") unless File.exists?(File.join(destination_root, ".gitignore"))
+      if IO.readlines(File.join(destination_root, ".gitignore")).grep(%r{^#{line}}).empty?
+        append_to_file(".gitignore", "#{line}\n")
+      end
+    end
+    
+end
+
+LeibnizCli.start

data/features/leibniz_command.feature

@@ -0,0 +1,15 @@
+Feature: A command line interface for Leibniz 
+  In order to simplify the process of getting started with Leibniz Acceptance Testing 
+  As a Leibniz user
+  I want a command line interface that has sane defaults and built in help
+
+  Scenario: Displaying help
+    When I run `leibniz help`
+    Then the exit status should be 0
+    And the output should contain "leibniz init"
+    And the output should contain "leibniz version"
+
+  Scenario: Displaying the version of Leibniz 
+    When I run `leibniz version`
+    Then the exit status should be 0
+    And the output should contain "Leibniz version"

data/features/leibniz_init.feature

@@ -0,0 +1,29 @@
+Feature: Add Leibniz testing to an existing project
+  In order to get started with Leibniz
+  As an infrastructure developer
+  I want to run a command to initialize my project
+
+  Scenario: Displaying help
+    When I run `leibniz help init`
+    Then the output should contain:
+    """
+    Usage:
+      leibniz init
+    """
+    And the exit status should be 0
+
+  @announce
+  Scenario: Running leibniz init within a project
+    When I run `leibniz init`
+    Then the exit status should be 0
+    And a directory named ".leibniz" should exist
+    And a directory named "features/support" should exist
+    And the file "features/support/env.rb" should contain "require 'leibniz'"
+    And the file "features/learning_leibniz.feature" should contain:
+    """
+    Given I have provisioned the following infrastructure:
+    """
+    And a directory named "features/step_definitions" should exist
+    And the file "features/step_definitions/learning_steps.rb" should contain "@infrastructure.converge"
+    And the file ".gitignore" should contain ".leibniz/"
+    

data/features/support/env.rb

@@ -0,0 +1,6 @@
+require 'aruba/cucumber'
+require 'leibniz'
+
+Before do
+  @aruba_timeout_seconds = 15
+end

data/leibniz.gemspec

@@ -20,7 +20,9 @@ Gem::Specification.new do |spec|
 
   spec.add_dependency "test-kitchen", "~> 1.0.0.alpha"
   spec.add_dependency "kitchen-vagrant"
+  spec.add_dependency "thor"
 
   spec.add_development_dependency "bundler", "~> 1.3"
+  spec.add_development_dependency 'aruba',     '~> 0.5'
   spec.add_development_dependency "rake"
 end

data/lib/leibniz/version.rb

@@ -1,3 +1,3 @@
 module Leibniz
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

data/lib/leibniz.rb

@@ -1,5 +1,4 @@
-require "leibniz/version"
-
+require 'leibniz/version'
 require 'kitchen'
 require 'forwardable'
 
@@ -22,6 +21,7 @@ module Leibniz
 
   def self.build(specification)
     loader = KitchenLoader.new(specification)
+
     config = Kitchen::Config.new(:loader => loader)
     Infrastructure.new(config.instances)
   end
@@ -75,19 +75,31 @@ module Leibniz
       @platforms = specification.hashes.map do |spec|
         create_platform(spec)
       end
+      @suites = specification.hashes.map do |spec|
+        create_suite(spec)
+      end
     end
 
     def read
       {
         :driver_plugin => "vagrant",
         :platforms => platforms,
-        :suites => [ { :name => "leibniz", :run_list => [] } ]
+        :suites => suites
       }
     end
 
     private
 
-    attr_reader :platforms
+    attr_reader :platforms, :suites
+
+    def create_suite(spec)
+      suite = Hash.new
+      suite[:name] = 'leibniz'
+      suite[:run_list] = []
+      suite[:data_bags_path] = 'test/integration/default/data_bags'
+      suite
+    end
+
 
     def create_platform(spec)
       distro = "#{spec['Operating System']}-#{spec['Version']}"

data/templates/features/learning_leibniz.feature

@@ -0,0 +1,16 @@
+Feature: Learn to use Leibniz
+
+  In order to learn how to use Leibniz
+  As an infrastructure developer
+  I want to be able to have a skeleton feature
+
+  Background:
+
+    Given I have provisioned the following infrastructure:
+    | Server Name     | Operating System | Version | Chef Version | Run List          |
+    | learning        | centos           | 6.4     |       11.8.0 | learning::default |
+    And I have run Chef
+    
+    Scenario: Infrastructure developer can learn Leibniz
+      pending
+      # Write your features here!

data/templates/features/step_definitions/learning_steps.rb

@@ -0,0 +1,8 @@
+Given(/^I have provisioned the following infrastructure:$/) do |specification|
+  @infrastructure = Leibniz.build(specification)
+end
+
+Given(/^I have run Chef$/) do
+  @infrastructure.destroy
+  @infrastructure.converge
+end

data/templates/features/support/env.rb

@@ -0,0 +1 @@
+require 'leibniz'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: leibniz
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-06-06 00:00:00.000000000 Z
+date: 2013-11-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: test-kitchen
@@ -43,6 +43,22 @@ dependencies:
     - - ! '>='
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: thor
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -59,6 +75,22 @@ dependencies:
     - - ~>
       - !ruby/object:Gem::Version
         version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: aruba
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '0.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '0.5'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -78,7 +110,8 @@ dependencies:
 description: Integration Testing for Chef
 email:
 - stephen@atalanta-systems.com
-executables: []
+executables:
+- leibniz
 extensions: []
 extra_rdoc_files: []
 files:
@@ -87,9 +120,18 @@ files:
 - LICENSE.txt
 - README.md
 - Rakefile
+- TODO.md
+- WISHLIST.md
+- bin/leibniz
+- features/leibniz_command.feature
+- features/leibniz_init.feature
+- features/support/env.rb
 - leibniz.gemspec
 - lib/leibniz.rb
 - lib/leibniz/version.rb
+- templates/features/learning_leibniz.feature
+- templates/features/step_definitions/learning_steps.rb
+- templates/features/support/env.rb
 homepage: ''
 licenses:
 - Apache 2.0
@@ -111,8 +153,11 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.23
+rubygems_version: 1.8.24
 signing_key: 
 specification_version: 3
 summary: Arguably Leibniz independently invented integration.
-test_files: []
+test_files:
+- features/leibniz_command.feature
+- features/leibniz_init.feature
+- features/support/env.rb