omnitest 0.2.1

data/doc-src/usage/crosstask.md.tt

@@ -0,0 +1,86 @@
+## Crosstasking (via Psychic)
+
+Omnitest needs to be able to run tasks in any of the projects before it can run tests. Omnitest uses [psychic](https://github.com/omnitest/psychic), to run tasks. Psychic creates a uniform interface for running similar tasks in different projects, delegating to project specific task runners (like Rake, Make, npm run, or gradle) when necessary.
+
+The first task you probably want to run is `bootstrap` in order to make sure the projects project is ready to test. Generally the `bootstrap` task will invoke a dependency manager like Bundler, npm, or pip.
+
+```sh
+$ bundle exec omnitest bootstrap
+<%= scenario_output_snippet 'omnitest', 'bootstrap', include_command: false, format: :raw %>
+```
+
+### Custom tasks
+
+There are a few default tasks like `bootstrap` that are built into omnitest (and psychic). The default tasks exist to match common test workflows (like the Travis-CI stages or Maven lifecycle), but you can also have omnitest invoke custom tasks.
+
+So you could tell omnitest to invoke custom tasks like `documentation`, `metrics`, `lint` or `gitstats`:
+
+```sh
+$ bundle exec omnitest task lint
+-----> Starting Omnitest (v0.2.0)
+-----> Running task lint for ruby
+       Executing bundle exec rubocop -D
+       warning: parser/current is loading parser/ruby21, which recognizes
+       warning: 2.1.5-compliant syntax, but you are running 2.1.4.
+       Inspecting 2 files
+       ..
+
+       2 files inspected, no offenses detected
+-----> Running task lint for java
+       Executing gradle checkstyleMain
+       :compileJava UP-TO-DATE
+       :processResources UP-TO-DATE
+       :classes UP-TO-DATE
+       :checkstyleMain[ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/HelloWorld.java:0: Missing package-info.java file.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/HelloWorld.java:1: Line is longer than 100 characters (found 101).
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/HelloWorld.java:3: Missing a Javadoc comment.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:1: Missing a Javadoc comment.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:2:1: warning: '{' should be on the previous line.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:4:3: warning: '{' should be on the previous line.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:24: warning: 'for' construct must use '{}'s.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:24:8: 'for' is not followed by whitespace.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:24:30: warning: ')' is preceded with whitespace.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:26: warning: 'for' construct must use '{}'s.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:26:8: 'for' is not followed by whitespace.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:27:28: warning: '(' is followed by whitespace.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:27:54: warning: ')' is preceded with whitespace.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:28: warning: 'for' construct must use '{}'s.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:28:8: 'for' is not followed by whitespace.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:29:28: warning: '(' is followed by whitespace.
+       [ant:checkstyle] /Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/src/main/java/Quine.java:29:33: warning: ')' is preceded with whitespace.
+        FAILED
+
+       FAILURE: Build failed with an exception.
+
+       * What went wrong:
+       Execution failed for task ':checkstyleMain'.
+       > Checkstyle rule violations were found. See the report at: file:///Users/Thoughtworker/repos/rackspace/polytrix/samples/sdks/java/build/reports/checkstyle/main.xml
+
+       * Try:
+       Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
+
+       BUILD FAILED
+
+       Total time: 4.904 secs
+-----> Running task lint for python
+       Executing ./scripts/lint.sh
+       New python executable in omnitest_python/bin/python
+       Installing setuptools, pip...done.
+       katas/hello_world.py:2:22: W292 no newline at end of file
+       katas/quine.py:2:8: E228 missing whitespace around modulo operator
+-----> Omnitest is finished. (0m8.49s)
+```
+
+This is equivalent to running `psychic task lint` in each directory. See [psychic](https://github.com/omnitest/psychic) for more details about how psychic decides what command to invoke for any given task.
+
+### Workflows
+
+Coming soon....
+
+A workflow is a group of tasks that you want to run together on each project.
+
+The "commit test" workflow is the most common. This is basically a workflow that runs all the tests and checks that should be run before commiting a change. The [Travis-CI lifecycle](http://docs.travis-ci.com/user/build-lifecycle/) and [Maven Lifecycle](http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html) (excluding the "deploy" stage in both cases) are examples.
+
+Another workflow I've seen is the "morning" workflow. This workflow makes sure each of the projects are ready to start development. This is often similar to a "commit test" workflow, but it will also make sure you have a clean development environment, have fetched the latest upstream changes (from both version control and dependency management systems).
+
+Another possibility would be a "end of sprint" or "pre-release" workflow. This could be very different than the two examples above. It may focus on collecting metrics or building and publishing release notes.

data/doc-src/usage/omnitest.md.tt

@@ -0,0 +1,87 @@
+## Omnitesting (via Skeptic)
+
+The `omnitest test` command will the same tests in each project, using spies to capture data and validate the behavior. This is for running tests that are shared across all projects to test for shared behavior. You can use `omnitask` to run each projects own test.
+
+Currently this is used for testing a code sample for each scenario. Omnitest will run the sample and capture the process's exit code, stdout and stderr. You can register additional "spies" with skeptic in order to capture additional information or perform additional validation. For example, there are spies that use the [Pacto](https://github.com/thoughtworks/pacto) project to capture HTTP requests and compare them with the RESTful services that were expected to be called for the scenario.
+
+### Defining test scenarios
+
+Omnitest uses [Skeptic](https://github.com/omnitest/skeptic) to run tests, so the test scenarios are listed in a file called `skeptic.yaml`. The `suites` section of skeptic.yaml defines the tests you want to run. The suites contain scenarios ("samples"). Here is a simple file that defines a few scenarios:
+
+<%= file_snippet 'samples/skeptic_simple.yaml' %>
+
+You can also define properties for running samples with the "global_env" and "env" on suites. Here's a more complete example that shows defines some properties:
+
+<%= file_snippet 'samples/skeptic.yaml' %>
+
+### Available Commands
+
+You can always see the available commands by running:
+<%= exec_snippet 'bundle exec omnitest help' %>
+
+You can see that most omnitest commands take two optional arguments: `[PROJECT|REGEXP|all] [SCENARIO|REGEXP|all]`. The first selects which projects a command applies to and the second selects which scenarios. See the documentation on [Project Sets](project_sets) for more information about selecting projects with the first parameter. The second argument, for selecting scenarios, has similar behavior:
+- If omitted the command will apply to all scenarios.
+- If a value is provided:
+  - If the value is "all" then the command applies to all scenarios. This only exists to make it possible to pass a third parameter, if necessary.
+  - If the value exactly matches the name of a scenario then only that scenario will be selected.
+  - If it does not match a scenario then it will be used as a regular expression to match other scenarios.
+
+### Listing Tests
+
+Omnitest keeps track of the test results so you don't need to test all projects or scenarios in a single run. You can use the command `omnitest list` to get quick summary of the latest results on the command line.
+
+<%= exec_snippet 'bundle exec omnitest list', cwd: 'samples' %>
+
+
+The list command can also illustrate the behavior of the two parameters for narrowing the scope of any omnitest command. For example you could list the results for the ruby project:
+
+<%= exec_snippet 'bundle exec omnitest list ruby', cwd: 'samples' %>
+
+Or you could list only the quine results for ruby and java:
+
+<%= exec_snippet 'bundle exec omnitest list "(ruby|java)" quine', cwd: 'samples' %>
+
+### Running Tests
+
+The command `omnitest test` is for testing scenarios. If you run the command `omnitest test` in each project it will test every scenario in all projects. You can narrow the scope of what you want to test by selecting projects with the first parameter and tests with the second parameter.
+
+For example, you could just run test quine scenario for the python project:
+
+<%= exec_snippet 'bundle exec omnitest test python quine', cwd: 'samples' %>
+
+Or, you could run all of the ruby tests:
+
+<%= exec_snippet 'bundle exec omnitest test ruby', cwd: 'samples' %>
+
+### Running individual phases
+
+The test command actually runs a series of phases. You can run these phases individually, which may be useful if you're still developing the samples or tests and just want to check that they work up to a certain point. If you run any of these phases they will run all of the previous phases but will stop at your target phase.
+
+There are two phases that apply at the project level:
+- clone: Make sure the project is available, if it is missing attempt to get it from version control.
+- bootstrap: Run the bootstrap task for the project to install dependenices.
+
+And three phases for each scenario:
+- detect: Find the code samples associated with the scenario.
+- exec: Run the code sample and capture the result.
+- verify: Validate the results captured by exec with any validators setup for that scenario.
+
+So if you only want to make sure that all of the samples execute, without validating the results to make sure they performed as expected, you could run `omnitest exec`.
+
+### Viewing Detailed Results
+
+#### On the Console
+
+The command `omnitest show` will display detailed results for the scenarios as text in your console.
+
+<%= exec_snippet 'bundle exec omnitest show ruby hello', cwd: 'samples' %>
+
+You can get even more information (including the actual code sample that was executed, with syntax highlighting if your console supports color) with the `--verbose` tag:
+
+<%= exec_snippet 'bundle exec omnitest show ruby hello --verbose', cwd: 'samples' %>
+
+#### Generating HTML reports
+
+You can also generate a variety of reports with the `omnidoc` command. The most useful is `omnidoc dashboard`, which will generate a dashboard showing a grid of test results that looks similar to the `omnidoc list` command, where each item is linked to additional details about the result that is similar to the `omnidoc show` command for that scenario.
+
+See the [omnidoc](omnidoc) documentation for more details.

data/features/bootstrapping.feature

@@ -0,0 +1,25 @@
+Feature: Boostrapping
+
+  Omnitest uses the [script/bootstrap](http://wynnnetherland.com/linked/2013012801/bootstrapping-consistency) pattern to prepare projects for testing. You can hook into any package manager, compiler, build tool, or any other toolchain to prepare to build and run samples.
+
+  Scenario: Bootstrapping all projects
+    Given the ruby project
+    And the java project
+    And the python project
+    And the sample omnitest config
+    And the hello_world skeptic config
+    When I run `bundle exec omnitest bootstrap`
+    Then the output should contain "-----> Bootstrapping java"
+    Then the output should contain "-----> Bootstrapping python"
+    Then the output should contain "-----> Bootstrapping ruby"
+
+  Scenario: Bootstrapping selected projects
+    Given the ruby project
+    And the java project
+    And the python project
+    And the sample omnitest config
+    And the hello_world skeptic config
+    When I run `bundle exec omnitest bootstrap "(java|ruby)"`
+    Then the output should contain "-----> Bootstrapping java"
+    Then the output should not contain "-----> Bootstrapping python"
+    Then the output should contain "-----> Bootstrapping ruby"

data/features/cloning.feature

@@ -0,0 +1,32 @@
+@wip
+Feature: Cloning
+
+  Omnitest can clone projects from git.
+
+  Scenario: Cloning all projects
+    Given the sample omnitest config
+    When I run `bundle exec omnitest clone`
+    Then the output should contain "-----> Cloning java"
+    Then the output should contain "-----> Cloning python"
+    Then the output should contain "-----> Cloning ruby"
+
+  Scenario: Cloning selected projects
+    Given the ruby project
+    And the java project
+    And the python project
+    And the sample omnitest config
+    When I run `bundle exec omnitest clone "(java|ruby)"`
+    Then the output should contain "-----> Cloning java"
+    Then the output should not contain "-----> Cloning python"
+    Then the output should contain "-----> Cloning ruby"
+
+  Scenario: Cloning by scenario
+    Given the ruby project
+    And the java project
+    And the python project
+    And the sample omnitest config
+    And the hello_world skeptic config
+    When I run `bundle exec omnitest clone hello`
+    Then the output should contain "-----> Cloning java"
+    Then the output should contain "-----> Cloning python"
+    Then the output should contain "-----> Cloning ruby"

data/features/fixtures/configs/omnitest_sample.yaml

@@ -0,0 +1,11 @@
+---
+  projects:
+    ruby:
+      language: 'ruby'
+      basedir: 'sdks/ruby'
+    java:
+      language: 'java'
+      basedir: 'sdks/java'
+    python:
+      language: 'python'
+      basedir: 'sdks/python'

data/features/fixtures/configs/skeptic_empty.yaml

@@ -0,0 +1,12 @@
+---
+  global_env:
+    LOCALE: <%= ENV['LANG'] %>
+    FAVORITE_NUMBER: 5
+  suites:
+    Katas:
+      env:
+        NAME: 'Max'
+      samples:
+    Tutorials:
+      env:
+      samples:

data/features/fixtures/configs/skeptic_hello_world.yaml

@@ -0,0 +1,10 @@
+---
+  global_env:
+    LOCALE: <%= ENV['LANG'] %>
+    FAVORITE_NUMBER: 5
+  suites:
+    Katas:
+      env:
+        NAME: 'Max'
+      samples:
+        - hello world

data/features/show.feature

@@ -0,0 +1,38 @@
+Feature: Show
+
+  Scenario: Initial state
+    Given the ruby project
+    And the java project
+    And the python project
+    And the sample omnitest config
+    And the hello_world skeptic config
+    When I run `bundle exec omnitest show ruby 'hello world'`
+    Then the output should contain:
+    """
+    katas-hello_world-ruby:                            <Not Found>
+      Test suite:                                        Katas
+      Test scenario:                                     hello world
+      Project:                                           ruby
+      Source:                                            sdks/ruby/katas/hello_world.rb
+    """
+
+  @no-clobber
+  Scenario: State after testing
+    Given I run `bundle exec omnitest test ruby`
+    When I run `bundle exec omnitest show ruby 'hello world'`
+    Then the output should contain:
+    """
+    katas-hello_world-ruby:                            Fully Verified (1 of 1)
+      Test suite:                                        Katas
+      Test scenario:                                     hello world
+      Project:                                           ruby
+      Source:                                            sdks/ruby/katas/hello_world.rb
+      Execution result:
+        Exit Status:                                       0
+        Stdout:
+          Hello, world!
+        Stderr:
+      Validations:
+        default validator:                                 ✓ Passed
+      Data from spies:
+    """

data/features/states.feature

@@ -0,0 +1,40 @@
+Feature: States
+
+  Scenario: Initial state
+    Given the ruby project
+    And the java project
+    And the python project
+    And the sample omnitest config
+    And the hello_world skeptic config
+    When I run `bundle exec omnitest list`
+    Then the output should contain:
+    """
+    Suite  Scenario     Project  Status
+    Katas  hello world  ruby     <Not Found>
+    Katas  hello world  java     <Not Found>
+    Katas  hello world  python   <Not Found>
+    """
+
+  @no-clobber
+  Scenario: State after execution
+    Given I run `bundle exec omnitest exec python`
+    When I run `bundle exec omnitest list`
+    Then the output should contain:
+    """
+    Suite  Scenario     Project  Status
+    Katas  hello world  ruby     <Not Found>
+    Katas  hello world  java     <Not Found>
+    Katas  hello world  python   Executed
+    """
+
+  @no-clobber
+  Scenario: State after verification
+    Given I run `bundle exec omnitest verify ruby`
+    When I run `bundle exec omnitest list`
+    Then the output should contain:
+    """
+    Suite  Scenario     Project  Status
+    Katas  hello world  ruby     Fully Verified (1 of 1)
+    Katas  hello world  java     <Not Found>
+    Katas  hello world  python   Executed
+    """

data/features/step_definitions/sdk_steps.rb

@@ -0,0 +1,22 @@
+require 'yaml'
+
+Given(/^the (\w+) project$/) do |sdk|
+  FileUtils.mkdir_p "#{current_dir}/sdks"
+  FileUtils.cp_r "samples/sdks/#{sdk}", "#{current_dir}/sdks"
+end
+
+Given(/^the (\w+) omnitest config$/) do |config|
+  FileUtils.cp_r "features/fixtures/configs/omnitest_#{config}.yaml", "#{current_dir}/omnitest.yaml"
+end
+
+Given(/^the (\w+) skeptic config$/) do |config|
+  FileUtils.cp_r "features/fixtures/configs/skeptic_#{config}.yaml", "#{current_dir}/skeptic.yaml"
+end
+
+Then(/^the file "(.*?)" should contain yaml matching:$/) do |file, content|
+  in_current_dir do
+    actual_content = YAML.load(File.read(file))
+    expected_content = YAML.load(content)
+    expect(actual_content).to eq(expected_content)
+  end
+end

data/features/support/env.rb

@@ -0,0 +1,9 @@
+require 'simplecov'
+SimpleCov.start
+
+require 'aruba/cucumber'
+require 'aruba/jruby'
+
+Before do
+  @aruba_timeout_seconds = RUBY_PLATFORM == 'java' ? 60 : 30
+end

data/lib/omnitest.rb

@@ -0,0 +1,211 @@
+require 'omnitest/version'
+require 'omnitest/core'
+require 'omnitest/psychic'
+require 'omnitest/skeptic'
+require 'omnitest/run_action'
+require 'omnitest/project'
+require 'omnitest/workflow'
+require 'omnitest/project_set'
+require 'omnitest/project_logger'
+require 'omnitest/configuration'
+require 'omnitest/documentation_generator'
+
+module Omnitest
+  include Omnitest::Core::Logger
+  include Omnitest::Core::Logging
+
+  # File extensions that Omnitest can automatically detect/execute
+  SUPPORTED_EXTENSIONS = %w(py rb js)
+
+  module Delegators
+    # @api private
+    # @!macro delegate_to_skeptic
+    #   @method $1()
+    def delegate_to_skeptic_class(meth)
+      define_method(meth) { |*args, &block| Skeptic.public_send(meth, *args, &block) }
+    end
+
+    # @api private
+    # @!macro delegate_to_psychic
+    #   @method $1()
+    def delegate_to_psychic_class(meth)
+      define_method(meth) { |*args, &block| Psychic.public_send(meth, *args, &block) }
+    end
+
+    # @api private
+    # @!macro delegate_to_psychic
+    #   @method $1()
+    def delegate_to_psychic_instance(meth)
+      define_method(meth) { |*args, &block| psychic.public_send(meth, *args, &block) }
+    end
+
+    # @api private
+    # @!macro delegate_to_projects
+    #   @method $1()
+    def delegate_to_projects(meth)
+      define_method(meth) do |*args, &block|
+        project_regex = args.shift
+        run_action(filter_projects(project_regex), meth, configuration.concurrency, *args, &block)
+      end
+    end
+
+    # @api private
+    # @!macro delegate_to_scenarios
+    #   @method $1()
+    def delegate_to_scenarios(meth)
+      define_method(meth) do |*args, &block|
+        project_regex = args.shift
+        scenario_regex = args.shift
+        scenarios(project_regex, scenario_regex).map { |s| s.public_send(meth, *args, &block) }
+      end
+    end
+  end
+
+  class << self
+    include Core::Configurable
+    include RunAction
+    extend Delegators
+
+    DEFAULT_PROJECT_SET_FILE = 'omnitest.yaml'
+    DEFAULT_TEST_MANIFEST_FILE = 'skeptic.yaml'
+
+    # @return [Logger] the common Omnitest logger
+    attr_accessor :logger
+
+    attr_accessor :psychic
+
+    attr_accessor :wants_to_quit
+
+    def new_logger(project) # (test, project, index)
+      name = project.name # instance_name(test, project)
+      index = Omnitest.projects.index(project) || 0
+      ProjectLogger.new(
+        stdout: STDOUT,
+        color: Core::Color::COLORS[index % Core::Color::COLORS.size].to_sym,
+        logdev: File.join(Omnitest.configuration.log_root, "#{name}.log"),
+        level: Omnitest::Core::Util.to_logger_level(Omnitest.configuration.log_level),
+        progname: name
+      )
+    end
+
+    def basedir
+      @basedir ||= psychic.basedir
+    end
+
+    # @private
+    def trap_interrupt
+      trap('INT') do
+        exit!(1) if Omnitest.wants_to_quit
+        Omnitest.wants_to_quit = true
+        STDERR.puts "\nInterrupt detected... Interrupt again to force quit."
+      end
+    end
+
+    def update_config!(options)
+      trap_interrupt
+      @logger = Omnitest.default_file_logger
+      project_set_file = options.file || DEFAULT_PROJECT_SET_FILE
+      @basedir = File.dirname project_set_file
+      skeptic_file = options.skeptic || DEFAULT_TEST_MANIFEST_FILE
+
+      Omnitest.configure do | config |
+        config.concurrency = options.concurrency
+        config.log_level = options.log_level || :info
+        config.project_set = project_set_file
+        config.skeptic.manifest_file = skeptic_file
+        config.travis = options.travis if options.respond_to? :travis
+      end
+      @test_dir = options.test_dir || File.expand_path('tests/omnitest/', @basedir)
+    end
+
+    delegate_to_skeptic_class :validate
+
+    delegate_to_projects :task
+
+    delegate_to_projects :workflow
+
+    delegate_to_projects :clone
+
+    delegate_to_projects :bootstrap
+
+    delegate_to_scenarios :test
+
+    delegate_to_scenarios :clear
+
+    delegate_to_scenarios :code2doc
+
+    Skeptic::Scenario::FSM::TRANSITIONS.each do | transition |
+      delegate_to_scenarios transition
+    end
+
+    def setup
+      # This autoload should probably be in Skeptic's initializer...
+      autoload_omnitest_files(@test_dir) unless @test_dir.nil? || !File.directory?(@test_dir)
+    end
+
+    def scenarios(project_regexp = 'all', scenario_regexp = 'all', options = {})
+      filtered_projects = filter_projects(project_regexp, options)
+      filtered_scenarios = filtered_projects.map(&:skeptic).flat_map do | skeptic |
+        skeptic.scenarios scenario_regexp, options
+      end
+
+      fail UserError, "No scenarios for regex `#{scenario_regexp}', try running `omnitest list'" if filtered_scenarios.empty?
+      filtered_scenarios
+    end
+
+    def filter_projects(regexp = 'all', _options = {})
+      return Omnitest.projects if regexp.nil? || regexp == 'all'
+
+      filtered_projects = Omnitest.projects.find { |s| s.name == regexp }
+      return [filtered_projects] if filtered_projects
+
+      filtered_projects ||= Omnitest.projects.select { |s| s.name =~ /#{regexp}/i }
+      fail UserError, "No projects matching regex `#{regexp}', known projects: #{Omnitest.projects.map(&:name)}" if filtered_projects.empty?
+
+      filtered_projects
+    end
+
+    # Returns a default file logger which emits on standard output and to a
+    # log file.
+    #
+    # @return [Logger] a logger
+    def default_file_logger
+      logfile = File.expand_path(File.join('.omnitest', 'logs', 'omnitest.log'))
+      ProjectLogger.new(stdout: $stdout, logdev: logfile, level: Core::Util.to_logger_level(configuration.log_level))
+    end
+
+    # The {Omnitest::TestManifest} that describes the test scenarios known to Omnitest.
+    def manifest
+      configuration.skeptic.manifest
+    end
+
+    # The set of {Omnitest::Project}s registered with Omnitest.
+    def projects
+      configuration.project_set.projects.values
+    end
+
+    # @api private
+    def psychic
+      @psychic ||= Omnitest::Psychic.new(cwd: Omnitest.basedir, logger: logger, travis: Omnitest.configuration.travis)
+    end
+
+    # Returns whether or not standard output is associated with a terminal
+    # device (tty).
+    #
+    # @return [true,false] is there a tty?
+    def tty?
+      $stdout.tty?
+    end
+
+    protected
+
+    def autoload_omnitest_files(dir)
+      $LOAD_PATH.unshift dir
+      Dir["#{dir}/**/*.rb"].each do | file_to_require |
+        # TODO: Need a better way to skip generators or only load validators
+        next if file_to_require.match %r{generators/.*/files/}
+        require Omnitest::Core::FileSystem.relativize(file_to_require, dir).to_s.gsub('.rb', '')
+      end
+    end
+  end
+end