heroku_hatchet 1.4.3 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c80bac9dc11b95cdcbfb5156c7b90c4975afbf82
-  data.tar.gz: 7700ae73439de98a7043e1fdebdde5e80a80462e
+  metadata.gz: a9f803b82d8440304dca2283cdcb3f1f9d689094
+  data.tar.gz: bfd8d1b84f7fdd80f53535905f474bfdc82d5e59
 SHA512:
-  metadata.gz: 131ba4fd7860bc306214a9e6f05136c67637380b9fac1ff0591e3a4584895b43c5aecec3d23c8cce28b2d3df2786e32e3f6a07f5f0a932205b169c444b10d466
-  data.tar.gz: 48000503d40b2c7f2cacfcca0dc3b0c59baa501b61563be38172d0ea8d95776d0bbb24468cadfc9bc1fdf43cada27e4c43f111c163e6eb4ce8c7a6af05e1c0c5
+  metadata.gz: 7f7cfcda96ee5f0222afc0cb70b2119007a8efadd6f72dd389b25df6754a46c7bb4336e88591e6678066f681ae4381af3f124221fb3b3ebdff24ec1435d2176a
+  data.tar.gz: cf49871c076c769bf1bcd5f3cc24f4c1607de040ef684b3dd493fb5073dde7af70e3baff264406801aade89c423ce0378de8f7e43877af969798be29856abfa1

data/.travis.yml

@@ -9,4 +9,4 @@ env:
   - HATCHET_BUILDPACK_BRANCH=master
   - HATCHET_RETRIES=3
   - HATCHET_APP_LIMIT=80
-  - secure: IuwT8MLmcdWIZvuD+XgHE9jh/49NcZ5Q6Wh7M2ROIQPsNjQ9jPHFpTTdL6p0/63eq948WUKrrPqDFRm/jh/JKyJPmQfDgWq4ScoPdurhgLlGg1CH81gi22t9VwO5QeoYLufMyueqazGbQCAl+xsa+hsKcs6nb836yDHsmBud6z0=
+  - secure: TvpZ0CrIe0FqjyTUOAtVqjHHrtF1esMroa00bYYRBas050/y7ygVpAn9utFZZChgt1PUbM48I01UaQglGxtmXVl3ahQXtPpXlzlwJOlDS09dlZFfLenkv530/pxIlpRtqk4q18gCoLBblXX7RZu3TGt0qds+o8dQrBzL6QifAHs=

data/CHANGELOG.md

@@ -1,5 +1,10 @@
 ## HEAD
 
+## 2.0.0
+
+- Support for Heroku CI added
+- Removed support for Anvil
+
 ## 1.4.1
 
 - Shell commands now escaped automatically

data/README.md

@@ -38,13 +38,25 @@ To prevent regressions and to make pushing out new features faster and easier.
 
 ## What can Hatchet Test?
 
-Hatchet can easily test deployment of buildpacks, getting the build output, and running arbitrary interactive processes such as `heroku run bash`.
+Hatchet can easily test deployment of buildpacks, getting the build output, and running arbitrary interactive processes such as `heroku run bash`. It can also test running CI against an app.
+
+## Writing Tests
+
+Hatchet is test framework agnostic. [This project](https://github.com/heroku/hatchet) uses `Test::Unit` to run it's own tests. While the [heroku-ruby-buildpack](https://github.com/heroku/heroku-buildpack-ruby) uses rspec.
+
+Rspec has a number of advantages, the ability to run `focused: true` to only run the exact test you want as well as the ability to tag tests. Rspec also has a number of useful plugins, one especialy useful one is `gem 'rspec-retry'` which will re-run any failed tests a given number of times (I recommend setting this to at least 2) this decrease the number of false negatives your tests will have.
+
+Whatever testing framework you chose, we recommend using a parallel test runner when running the full suite [parallel_tests](https://github.com/grosser/parallel_tests) works with rspec and test::unit and is amazing.
+
+If you're unfamiliar with the ruby testing eco-system or want some help, looking at existing projects is a good place to get started.
+
+There is a section below on getting Hatchet to work on Travis.
 
 ## Testing a Buildpack
 
 Hatchet was built for testing the Ruby buildpack, but you can use it to test any buildpack you desire provided you don't mind writing your tests written in Ruby.
 
-You will need copies of applications that can be deployed by your buildpack. You can see the ones for the Hatchet unit tests (and the Ruby buildpack) https://github.com/sharpstone. Hatchet does not require that you keep these apps checked into your git repo which would make fetching your buildpack slow instead declare them in a `hatchet.json` file (see below).
+You will need copies of applications that can be deployed by your buildpack. You can see the ones for the Hatchet unit tests (and the Ruby buildpack) https://github.com/sharpstone. Hatchet does not require that you keep these apps checked into your git repo which would make fetching your buildpack slow, instead declare them in a `hatchet.json` file (see below).
 
 Hatchet will automate retrieving these files `$ hatchet install`, as well as deploying them using your local copy of the buildpack, retrieving the build output and running commands against deploying applications.
 
@@ -100,38 +112,19 @@ spontaneously fail. In the future we may create a hatchet.lockfile or
 something to declare the commit
 
 
-## Deployments: Anvil vs. Git
-
-Before you start testing a buildpack, understand that there are two different ways to deploy to Heroku. The first you are likely familiar with Git, requires a `git push heroku master`. You can configure the buildpack of an app being deployed in this way through the `BUILDPACK_URL` environment variable of the app. The buildpack url must be publicaly available.
-
-The second method is by [Anvil](https://github.com/ddollar/anvil-cli) . In this method the build is performed as a service. This service takes an app to be built as well as a buildpack. When using Anvil, you do not need to put your buildpack anywhere publicly available.
-
-When developing local features you will likely wish to use Anvil since it does not require a publicly available URL and you can iterate faster. When testing for regression you will almost always want to use Git, since it is the closest approximation to real world deployment. For this reason Hatchet provides a globally configurable way to toggle between the two deployment modes `Hatchet::Runner`
-
-
 ## Deploying apps
 
-Now that you've got your apps locally you can have hatchet deploy them for you. Hatchet can deploy using one of two ways Anvil and Git. To specify one or the other set your `HATCHET_DEPLOY_STRATEGY` environment variable to `anvil` or `git`. The default is `anvil`. In production, you should always test against `git`
-
-
-A `Hatchet::GitApp` will deploy using the standard `git push heroku master` and is not configurable, if you use this option you need to have a publicly accessible copy of your buildpack. Using Git to test your buildpack may be slow and require you to frequently push your buildpack to a public git repo. For this reason we recommend using Anvil to run your tests locally:
-
-```ruby
-Hatchet::Runner.new("rails3_mri_193").deploy do |app|
-
-end
-```
 
-If you are using GIT, you can specify the location of your public buildpack url in an environment variable:
+You can specify the location of your public buildpack url in an environment variable:
 
 ```sh
 HATCHET_BUILDPACK_BASE=https://github.com/heroku/heroku-buildpack-ruby.git
 HATCHET_BUILDPACK_BRANCH=master
 ```
 
-If you do not specify `HATCHET_BUILDPACK_URL` the default Ruby buildpack will be used. If you do not specify a `HATCHET_BUILDPACK_BRANCH` the current branch you are on will be used.
+If you do not specify `HATCHET_BUILDPACK_URL` the default Ruby buildpack will be used. If you do not specify a `HATCHET_BUILDPACK_BRANCH` the current branch you are on will be used. This is how the Ruby buildpack runs tests on branches on travis (by leaving `HATCHET_BUILDPACK_BRANCH` blank).
 
-Deploys are expected to work, if the `ENV['HATCHET_RETRIES']` is set, then deploys will be automatically retried that number of times. Due to testing using a network and random Anvil failures, setting this value to `3` retries seems to work well. If an app cannot be deployed within its allotted number of retries an error will be raised.
+Deploys are expected to work, if the `ENV['HATCHET_RETRIES']` is set, then deploys will be automatically retried that number of times. Due to testing using a network and random failures, setting this value to `3` retries seems to work well. If an app cannot be deployed within its allotted number of retries an error will be raised.
 
 If you are testing an app that is supposed to fail deployment you can set the `allow_failure: true` flag when creating the app:
 
@@ -139,7 +132,7 @@ If you are testing an app that is supposed to fail deployment you can set the `a
 Hatchet::Runner.new("no_lockfile", allow_failure: true).deploy do |app|
 ```
 
-After the block finishes your app will be removed from heroku. If you are investigating a deploy, you can add the `debug: true` flag to your app:
+After the block finishes your app will be queued to be removed from heroku. If you are investigating a deploy, you can add the `debug: true` flag to your app:
 
 ```ruby
 Hatchet::Runner.new("rails3_mri_193", debug: true).deploy do |app|
@@ -155,21 +148,15 @@ app = Hatchet::Runner.new("rails3_mri_193", name: "testapp")
 
 Deploying the app takes a few minutes, so you may want to skip that part to make debugging a problem easier since you're iterating much faster.
 
-
-If you need to deploy using a buildpack that is not in the root of your directory you can specify a path in the `buildpack` option:
-
+If you need to deploy using a different buildpack you can specify one manually:
 
 ```ruby
-buildpack_path = File.expand_path 'test/fixtures/buildpacks/heroku-buildpack-ruby'
 
 def test_deploy
-  Hatchet::GitApp.new("rails3_mri_193", buildpack: buildpack_path).deploy do |app|
+  Hatchet::Runner.new("rails3_mri_193", buildpack: "https://github.com/heroku/heroku-buildpack-ruby.git").deploy do |app|
   # ...
 ```
 
-If you are using a `Hatchet::GitApp` this is where you specify the publicly avaialble location of your buildpack, such as `https://github.com/heroku/heroku-buildpack-ruby.git#mybranch`
-
-
 ## Getting Deploy Output
 
 After Hatchet deploys your app you can get the output by using `app.output`
@@ -201,6 +188,15 @@ Hatchet::Runner.new("rails3_mri_193").deploy do |app|
 
 This is useful for checking the existence of generated files such as assets. If you need to run an interactive session such as `heroku run bash` or `heroku run rails console` you can use the run command and pass a block:
 
+```
+Hatchet::Runner.new("rails3_mri_193").deploy do |app|
+  app.run("cat Procfile")
+end
+```
+
+This is the prefered way to run commands against the app. You can also string together commands in a session, however due to difficulties in driving a REPL programatically via [repl_runner](http://github.com/schneems/repl_runner) it's less deterministic.
+
+
 ```ruby
 Hatchet::Runner.new("rails3_mri_193").deploy do |app|
   app.run("bash") do |bash|
@@ -210,38 +206,56 @@ Hatchet::Runner.new("rails3_mri_193").deploy do |app|
 end
 ```
 
-or
+Please read the docs on [repl_runner](http://github.com/schneems/repl_runner) for more info. The only interactive commands that are supported out of the box are `rails console`, `bash`, and `irb` it is fairly easy to add your own though:
+
+## Heroku CI
+
+Hatchet supports testing Heroku CI.
 
 ```ruby
-Hatchet::Runner.new("rails3_mri_193").deploy do |app|
-  app.run("rails console") do |console|
-    console.run("a = 1 + 2")  {|result| assert_match "3", result }
-    console.run("'foo' * a")  {|result| assert_match "foofoofoo", result }
-  end
+Hatchet::Runner.new("rails5_ruby_schema_format").run_ci do |test_run|
+  assert_match "Ruby buildpack tests completed successfully", test_run.output
 end
 ```
 
-This functionality is provided by [repl_runner](http://github.com/schneems/repl_runner). Please read the docs on that readme for more info. The only interactive commands that are supported out of the box are `rails console`, `bash`, and `irb` it is fairly easy to add your own though:
+Call the `run_ci` method on the hatchet `Runner`. The object passed to the block is a `Hatchet::TestRun` object. You can call:
 
-```
-ReplRunner.register_commands(:python)  do |config|
-  config.terminate_command "exit()"        # the command you use to end the 'python' console
-  config.startup_timeout 60                # seconds to boot
-  config.return_char "\n"                  # the character that submits the command
-end
-```
+- `test_run.output` this will have the setup and test output of your tests.
+- `test_run.app` this has a reference to the "app" you're testing against, however currently no `heroku create` is run (as it's not needed to run tests, only a pipeline and a blob of code).
 
-If you have questions on setting running other interactive commands message [@schneems](http://twitter.com/schneems)
+An exception will be raised if either the test times out or a status of `:errored` or `:failed` is returned. If you expect your test to fail, you can pass in `allow_failure: true` when creating your hatchet runner. If you do that, you'll also get access to different statuses:
 
-## Writing Tests
+- `test_run.status` this will return a symbol of the status of your test. Statuses include, but are not limited to `:pending`, `:building`, `:errored`, `:creating`, `:succeeded`, and `:failed`
 
-Hatchet is test framework agnostic. [This project](https://github.com/heroku/hatchet) uses `Test::Unit` to run it's own tests. While the [heroku-ruby-buildpack](https://github.com/heroku/heroku-buildpack-ruby) uses rspec.
+You can pass in a different timeout to the `run_ci` method `run_ci(timeout: 300)`.
 
-Rspec has a number of advantages, the ability to run `focused: true` to only run the exact test you want as well as the ability to tag tests. Rspec also has a number of useful plugins, one especialy useful one is `gem 'rspec-retry'` which will re-run any failed tests a given number of times (I recommend setting this to at least 2) this decrease the number of false negatives your tests will have.
+You will likely need an `app.json` in the root directory of the app you're deploying. For example:
 
-Whatever testing framework you chose, we recommend using a parallel test runner when running the full suite [parallel_tests](https://github.com/grosser/parallel_tests) works with rspec and test::unit and is amazing.
+```json
+{
+  "environments": {
+    "test": {
+      "addons":[
+         "heroku-postgresql"
+      ]
+    }
+  }
+}
+```
+
+This is on [a Rails5 test app](https://github.com/sharpstone/rails5_ruby_schema_format/blob/master/app.json) that needs the database to run.
 
-If you're unfamiliar with the ruby testing eco-system or want some help with boilerplate and work for Heroku: [@schneems](http://twitter.com/schneems) can help you get started. Looking at existing projects is a good place to get started
+Do **NOT** specify a `buildpacks` key in the `app.json` as Hatchet will do this for you automatically. If you need to set buildpacks you can pass them into the `buildpacks:` keword argument:
+
+```
+buildpacks = []
+buildpacks << "https://github.com/heroku/heroku-buildpack-pgbouncer.git"
+buildpacks << [HATCHET_BUILDPACK_BASE, HATCHET_BUILDPACK_BRANCH.call].join("#")
+
+Hatchet::Runner.new("rails5_ruby_schema_format", buildpacks: buildpacks).run_ci do |test_run|
+  # ...
+end
+```
 
 ## Testing on Travis
 

data/hatchet.gemspec

@@ -18,6 +18,7 @@ Gem::Specification.new do |gem|
   gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
   gem.require_paths = ["lib"]
 
+  gem.add_dependency "platform-api",  "~> 1"
   gem.add_dependency "heroku-api",    "~> 0"
   gem.add_dependency "activesupport", "~> 4"
   gem.add_dependency "rrrretry",      "~> 1"

data/hatchet.json

@@ -3,5 +3,9 @@
   "rails3":  ["sharpstone/rails3_mri_193"],
   "rails2":  ["sharpstone/rails2blog"],
   "bundler": ["sharpstone/no_lockfile"],
-  "default": ["sharpstone/default_ruby"]
+  "default": ["sharpstone/default_ruby"],
+  "ci":      [
+              "sharpstone/rails5_ruby_schema_format",
+              "sharpstone/rails5_ci_fails_no_database"
+              ]
 }

data/lib/hatchet.rb

@@ -9,39 +9,27 @@ require 'stringio'
 require 'fileutils'
 require 'stringio'
 
-
 module Hatchet
-  RETRIES   = Integer(ENV['HATCHET_RETRIES']   || 1)
+end
 
-  class App
-  end
 
-  def self.git_branch
-    return ENV['TRAVIS_BRANCH'] if ENV['TRAVIS_BRANCH']
-    out = `git describe --contains --all HEAD`.strip
-    raise "Attempting to find current branch name. Error: Cannot describe git: #{out}" unless $?.success?
-    out
-  end
-
-  def self.set_deploy_strategy!
-    deploy_strat = (ENV['HATCHET_DEPLOY_STRATEGY'] || :git).to_sym
-    case Hatchet::const_set("DEPLOY_STRATEGY", deploy_strat)
-    when :anvil
-      Hatchet.const_set("Runner", Hatchet::AnvilApp)
-    when :git
-      Hatchet.const_set("Runner", Hatchet::GitApp)
-    else
-      raise "unknown deploy strategy #{Hatchet::DEPLOY_STRATEGY}, expected 'anvil', 'git'"
-    end
-  end
-end
 
 require 'hatchet/version'
 require 'hatchet/reaper'
+require 'hatchet/test_run'
 require 'hatchet/app'
 require 'hatchet/anvil_app'
 require 'hatchet/git_app'
 require 'hatchet/config'
 
+module Hatchet
+  RETRIES = Integer(ENV['HATCHET_RETRIES']   || 1)
+  Runner  = Hatchet::GitApp
 
-Hatchet.set_deploy_strategy!
+  def self.git_branch
+    return ENV['TRAVIS_BRANCH'] if ENV['TRAVIS_BRANCH']
+    out = `git describe --contains --all HEAD`.strip
+    raise "Attempting to find current branch name. Error: Cannot describe git: #{out}" unless $?.success?
+    out
+  end
+end

data/lib/hatchet/anvil_app.rb

@@ -3,65 +3,8 @@ require 'stringio'
 
 module Hatchet
   class AnvilApp < App
-
     def initialize(directory, options = {})
-      @buildpack = options[:buildpack]
-      @buildpack ||= File.expand_path('.')
-      super
-    end
-
-    def push_without_retry!
-      out, err = wrap_stdout_and_rescue(Anvil::Builder::BuildError) do
-        slug_url  = Anvil::Engine.build(".", :buildpack => @buildpack, :pipeline => true)
-        puts "Releasing to http://#{@name}.herokuapp.com"
-        response = release(@name, slug_url)
-        while response.status == 202
-          response = Excon.get("#{release_host}#{response.headers["Location"]}")
-        end
-      end
-
-      err.string
-    end
-
-    def wrap_stdout_and_rescue(error, &block)
-      wrap_stdout do |orig_out, orig_err|
-        begin
-          yield orig_out, orig_err
-        rescue error => e
-          return [$stdout.dup, $stderr.dup] if @allow_failure
-          orig_out.puts $stderr.dup.string # print the errors to the test output
-          raise e
-        end
-      end
-    end
-
-    def wrap_stdout(orig_out = $stdout, orig_err = $stderr, &block)
-      $stderr  = StringIO.new
-      $stdout  = StringIO.new
-      yield orig_out, orig_err
-      puts [$stdout.dup, $stderr.dup].inspect
-      return $stdout.dup, $stderr.dup
-    ensure
-      $stdout = orig_out
-      $stderr = orig_err
-    end
-
-   def teardown!
-      super
-      FileUtils.rm_rf("#{directory}/.anvil")
-    end
-
-    private
-    def release(name, slug_url)
-      headers         = {"Content-Type" => "application/json", accept: :json}
-      release_options = {description: "Anvil Build", slug_url: slug_url }
-      Excon.post("#{release_host}/v1/apps/#{name}/release",
-                 headers: headers,
-                 body:    release_options.to_json)
-    end
-
-    def release_host
-      "https://:#{api_key}@cisaurus.heroku.com"
+      raise "Anvil no longer works"
     end
   end
 end

data/lib/hatchet/app.rb

@@ -1,7 +1,12 @@
 require 'shellwords'
+require 'platform-api'
 
 module Hatchet
   class App
+    HATCHET_BUILDPACK_BASE   = (ENV['HATCHET_BUILDPACK_BASE'] || "https://github.com/heroku/heroku-buildpack-ruby.git")
+    HATCHET_BUILDPACK_BRANCH = -> { ENV['HATCHET_BUILDPACK_BRANCH'] || Hatchet.git_branch }
+    BUILDPACK_URL = "https://github.com/heroku/heroku-buildpack-ruby.git"
+
     attr_reader :name, :stack, :directory, :repo_name
 
     class FailedDeploy < StandardError
@@ -22,9 +27,14 @@ module Hatchet
       @debug         = options[:debug]         || options[:debugging]
       @allow_failure = options[:allow_failure] || false
       @labs          = ([] << options[:labs]).flatten.compact
+      @buildpack     = options[:buildpack] || options[:buildpack_url] || [HATCHET_BUILDPACK_BASE, HATCHET_BUILDPACK_BRANCH.call].join("#")
       @reaper        = Reaper.new(heroku)
     end
 
+    def allow_failure?
+      @allow_failure
+    end
+
     # config is read only, should be threadsafe
     def self.config
       @config ||= Config.new
@@ -194,6 +204,58 @@ module Hatchet
       @heroku ||= Heroku::API.new(api_key: api_key)
     end
 
+    def run_ci(timeout: 300, &block)
+      Hatchet::RETRIES.times.retry do
+        result       = create_pipeline
+        @pipeline_id = result["id"]
+      end
+
+      # create_app
+      # platform_api.pipeline_coupling.create(app: name, pipeline: @pipeline_id, stage: "development")
+      test_run = TestRun.new(token:      api_key,
+                             buildpacks: @buildpack,
+                             timeout:    timeout,
+                             app:        self,
+                             pipeline:   @pipeline_id)
+
+      Hatchet::RETRIES.times.retry do
+        test_run.create_test_run
+      end
+      test_run.wait!(&block)
+    ensure
+      delete_pipeline(@pipeline_id) if @pipeline_id
+    end
+
+    def pipeline_id
+      @pipeline_id
+    end
+
+    def create_pipeline
+      platform_api.pipeline.create(name: @name)
+    end
+
+    def source_get_url
+      create_source
+      @source_get_url
+    end
+
+    def create_source
+      @create_source ||= begin
+        result = platform_api.source.create
+        @source_get_url = result["source_blob"]["get_url"]
+        @source_put_url = result["source_blob"]["put_url"]
+        @source_put_url
+      end
+    end
+
+    def delete_pipeline(pipeline_id)
+      platform_api.pipeline.delete(pipeline_id)
+    end
+
+    def platform_api
+      @platform_api ||= PlatformAPI.connect_oauth(api_key)
+    end
+
     private
     # if someone uses bundle exec
     def bundle_exec

data/lib/hatchet/git_app.rb

@@ -1,15 +1,6 @@
 module Hatchet
   # used for deploying a test app to heroku via git
   class GitApp < App
-    HATCHET_BUILDPACK_BASE   = (ENV['HATCHET_BUILDPACK_BASE'] || "https://github.com/heroku/heroku-buildpack-ruby.git")
-    HATCHET_BUILDPACK_BRANCH = -> { ENV['HATCHET_BUILDPACK_BRANCH'] || Hatchet.git_branch }
-    BUILDPACK_URL = "https://github.com/heroku/heroku-buildpack-ruby.git"
-
-    def initialize(directory, options = {})
-      @buildpack = options[:buildpack] || options[:buildpack_url] || [HATCHET_BUILDPACK_BASE, HATCHET_BUILDPACK_BRANCH.call].join("#")
-      super
-    end
-
     def setup!
       super
       heroku.put_config_vars(name, 'BUILDPACK_URL' => @buildpack)

data/lib/hatchet/reaper.rb

@@ -25,7 +25,7 @@ module Hatchet
 
     def cycle(apps = get_apps)
       if over_limit?
-        if @hatchet_apps.count > 0
+        if @hatchet_apps.count > 1
           destroy_oldest
           cycle
         else

data/lib/hatchet/test_run.rb

@@ -0,0 +1,192 @@
+module Hatchet
+  class FailedTestError < StandardError
+    def initialize(app, output)
+      msg = "Could run tests on pipeline id: '#{app.pipeline_id}' (#{app.repo_name}) at path: '#{app.directory}'\n" <<
+            " if this was expected add `allow_failure: true` to your hatchet initialization hash.\n" <<
+            "output:\n" <<
+            "#{output}"
+      super(msg)
+    end
+  end
+
+  class TestRun
+
+    # Hatchet::GitApp.new("rails3_mri_193").run_ci do |test_run|
+    #   assert :succeeded, test_run.status
+    # end
+    #
+    # TestRun.new(token: , buildpacks: , test_dir: )
+    #
+    def initialize(
+      token:,
+      buildpacks:,
+      app:,
+      pipeline:,
+      timeout:        10,
+      pause:          5,
+      commit_sha:     "sha",
+      commit_branch:  "master",
+      commit_message: "commit",
+      organization:    nil
+    )
+      @pipeline        = pipeline || "hatchet-t-#{SecureRandom.hex(10)}"
+      @timeout         = timeout
+      @pause           = pause
+      @organization    = organization
+      @token           = token
+      @commit_sha      = commit_sha
+      @commit_branch   = commit_branch
+      @commit_message  = commit_message
+      @buildpacks      = Array(buildpacks)
+      @app             = app
+      @mutex           = Mutex.new
+      @status          = false
+    end
+    attr_reader :app
+
+    def create_test_run
+      @mutex.synchronize do
+        raise "Test is already running" if @status
+        @status = :building
+
+        body = {
+          source_blob_url: source_blob_url,
+          pipeline:        @pipeline,
+          organization:    @organization,
+          commit_sha:      @commit_sha,
+          commit_branch:   @commit_branch,
+          commit_message:  @commit_message,
+        }
+
+        # https://github.com/heroku/api/blob/master/schema/variants/3.ci/platform-api-reference.md#test-run-create
+        attributes = excon_request(
+          method:  :post,
+          path:    "/test-runs",
+          version: "3.ci",
+          body:    body,
+          expects: [201]
+        )
+        @test_run_id = attributes["id"]
+      end
+      info
+    end
+
+    def info
+     # GET /test-runs/{test_run_id}
+      response = excon_request(
+        method:  :get,
+        path:    "/test-runs/#{@test_run_id}",
+        version: "3.ci",
+        expects: [201, 200]
+      )
+
+      @setup_stream_url  = response["setup_stream_url"]
+      @output_stream_url = response["output_stream_url"]
+
+      @status = response["status"].to_sym
+    end
+
+    def status
+      @status # :pending, :building, :creating, :succeeded, :failed, :errored
+    end
+
+    def output
+      get_contents_or_whatever(@setup_stream_url) +
+      get_contents_or_whatever(@output_stream_url)
+    end
+
+    def wait!
+      Timeout::timeout(@timeout) do
+        while true do
+          info
+          case @status
+          when :succeeded
+            yield self
+            return self
+          when :failed, :errored
+            raise FailedTestError.new(self.app, self.output) unless app.allow_failure?
+            yield self
+            return self
+          else
+            # keep looping
+          end
+          sleep @pause
+        end
+      end
+    rescue Timeout::Error
+      puts "Timed out status: #{@status}, timeout: #{@timeout}"
+      raise FailedTestError.new(self.app, self.output) unless app.allow_failure?
+      yield self
+      return self
+    end
+
+    # Here's where the magic happens folks
+    #
+    # == Set the buildpack
+    #
+    # We take the current directory structure and see if it has an `app.json`
+    # This is how Heroku CI knows what buildpacks to use to run your tests
+    # Hatchet will inject whatever buildpack you pass to it, by default
+    # it uses the same buildpack you have specified in your Hatchet constants
+    # and uses the same branch your tests are using
+    #
+    # == Generate source blob url
+    #
+    # The CI endpoint takes a url that has a `.tgz` file to execute your tests.
+    # We pull down the app you're testing against, inject an `app.json` (or modify
+    # if it already exists). We the use the heroku "source" api to generate a
+    # url that we can put our newly generated `.tgz` file. It also returns a "get"
+    # url where those contents can be downloaded. We pass this url back to CI
+    #
+    def source_blob_url
+      @app.in_directory do
+        app_json = JSON.parse(File.read("app.json")) if File.exist?("app.json")
+        app_json ||= {}
+        app_json["environments"]                       ||= {}
+        app_json["environments"]["test"]               ||= {}
+        app_json["environments"]["test"]["buildpacks"] = @buildpacks.map {|b| { url: b } }
+        File.open("app.json", "w") {|f| f.write(JSON.generate(app_json)) }
+
+        `tar c . | gzip -9 > slug.tgz`
+
+        source_put_url = @app.create_source
+        Hatchet::RETRIES.times.retry do
+          Excon.put(source_put_url,
+                    expects: [200],
+                    body:    File.read('slug.tgz'))
+        end
+      end
+      return @app.source_get_url
+    end
+
+  private
+    def get_contents_or_whatever(url)
+      Excon.get(url, read_timeout: @pause).body
+    rescue Excon::Error::Timeout
+      ""
+    end
+
+    def excon_request(options)
+      JSON.parse(raw_excon_request(options).body)
+    end
+
+    def version
+      "3"
+    end
+
+    def raw_excon_request(options)
+      version = options[:version] || 3
+      options[:headers] = {
+        "Authorization" => "Bearer #{@token}",
+        "Accept"        => "application/vnd.heroku+json; version=#{version}",
+        "Content-Type"  => "application/json"
+      }.merge(options[:headers] || {})
+      options[:body] = JSON.generate(options[:body]) if options[:body]
+
+      Hatchet::RETRIES.times.retry do
+        connection = Excon.new("https://api.heroku.com")
+        return connection.request(options)
+      end
+    end
+  end
+end

data/lib/hatchet/version.rb

@@ -1,3 +1,3 @@
 module Hatchet
-  VERSION = "1.4.3"
+  VERSION = "2.0.0"
 end

data/test/hatchet/ci_test.rb

@@ -0,0 +1,24 @@
+require 'test_helper'
+
+
+class CITest < Test::Unit::TestCase
+
+  def test_ci_create_app_with_stack
+    Hatchet::GitApp.new("rails5_ruby_schema_format").run_ci do |test_run|
+      assert_match "Ruby buildpack tests completed successfully", test_run.output
+      assert_equal :succeeded, test_run.status
+    end
+  end
+
+  def test_error_with_bad_app
+    error = assert_raise(Hatchet::FailedTestError) do
+      Hatchet::GitApp.new("rails5_ci_fails_no_database").run_ci {}
+    end
+
+    assert_match "PG::ConnectionBad: could not connect to server" ,error.message
+
+    Hatchet::GitApp.new("rails5_ci_fails_no_database", allow_failure: true).run_ci do |test_run|
+      assert_equal :errored, test_run.status
+    end
+  end
+end

data/test/hatchet/config_test.rb

@@ -11,20 +11,26 @@ class ConfigTest < Test::Unit::TestCase
   end
 
   def test_config_dirs
-    expected_dirs = {"test/fixtures/repos/bundler/no_lockfile"   => "git://github.com/sharpstone/no_lockfile.git",
-                      "test/fixtures/repos/default/default_ruby" => "git://github.com/sharpstone/default_ruby.git",
-                     "test/fixtures/repos/default/default_ruby"  => "git://github.com/sharpstone/default_ruby.git",
-                     "test/fixtures/repos/rails2/rails2blog"     => "git://github.com/sharpstone/rails2blog.git",
-                     "test/fixtures/repos/rails3/rails3_mri_193" => "git://github.com/sharpstone/rails3_mri_193.git"}
-    assert_equal expected_dirs, @config.dirs
-  end
+    {
+      "test/fixtures/repos/bundler/no_lockfile"   => "git://github.com/sharpstone/no_lockfile.git",
+      "test/fixtures/repos/default/default_ruby"  => "git://github.com/sharpstone/default_ruby.git",
+      "test/fixtures/repos/default/default_ruby"  => "git://github.com/sharpstone/default_ruby.git",
+      "test/fixtures/repos/rails2/rails2blog"     => "git://github.com/sharpstone/rails2blog.git",
+      "test/fixtures/repos/rails3/rails3_mri_193" => "git://github.com/sharpstone/rails3_mri_193.git"
+     }.each do |key, value|
+       assert_include(key, value, @config.dirs)
+     end
+   end
 
   def test_config_repos
-    expected_repos = {"default_ruby"   => "test/fixtures/repos/default/default_ruby",
-                      "no_lockfile"    => "test/fixtures/repos/bundler/no_lockfile",
-                      "rails2blog"     => "test/fixtures/repos/rails2/rails2blog",
-                      "rails3_mri_193" => "test/fixtures/repos/rails3/rails3_mri_193"}
-    assert_equal expected_repos, @config.repos
+    {
+      "default_ruby"   => "test/fixtures/repos/default/default_ruby",
+      "no_lockfile"    => "test/fixtures/repos/bundler/no_lockfile",
+      "rails2blog"     => "test/fixtures/repos/rails2/rails2blog",
+      "rails3_mri_193" => "test/fixtures/repos/rails3/rails3_mri_193"
+     }.each do |key, value|
+       assert_include(key, value, @config.repos)
+     end
   end
 
   def test_no_internal_config_raises_no_errors
@@ -37,5 +43,10 @@ class ConfigTest < Test::Unit::TestCase
     @config.send :init_config!, {"foo" => ["schneems/sextant"]}
     assert_equal("git://github.com/schneems/sextant.git", @config.dirs["./repos/foo/sextant"])
   end
+  private
+
+    def assert_include(key, value, actual)
+      assert_equal value, actual[key], "Expected #{actual.inspect} to include #{ {key => value } } but it did not"
+    end
 end
 

data/test/test_helper.rb

@@ -1,11 +1,11 @@
-Bundler.require
+$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
+require 'hatchet'
 
+Bundler.require
 
-require 'hatchet'
 require 'test/unit'
 require "mocha/setup"
 
-
 def assert_tests_run
 end
 

metadata

@@ -1,15 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: heroku_hatchet
 version: !ruby/object:Gem::Version
-  version: 1.4.3
+  version: 2.0.0
 platform: ruby
 authors:
 - Richard Schneeman
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-11-01 00:00:00.000000000 Z
+date: 2017-03-30 00:00:00.000000000 Z
 dependencies:
+- !ruby/object:Gem::Dependency
+  name: platform-api
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
 - !ruby/object:Gem::Dependency
   name: heroku-api
   requirement: !ruby/object:Gem::Requirement
@@ -231,6 +245,7 @@ files:
 - lib/hatchet/git_app.rb
 - lib/hatchet/reaper.rb
 - lib/hatchet/tasks.rb
+- lib/hatchet/test_run.rb
 - lib/hatchet/version.rb
 - test/fixtures/buildpacks/null-buildpack/bin/compile
 - test/fixtures/buildpacks/null-buildpack/bin/detect
@@ -239,8 +254,8 @@ files:
 - test/fixtures/buildpacks/null-buildpack/readme.md
 - test/hatchet/allow_failure_anvil_test.rb
 - test/hatchet/allow_failure_git_test.rb
-- test/hatchet/anvil_test.rb
 - test/hatchet/app_test.rb
+- test/hatchet/ci_test.rb
 - test/hatchet/config_test.rb
 - test/hatchet/edit_repo_test.rb
 - test/hatchet/git_test.rb
@@ -248,7 +263,6 @@ files:
 - test/hatchet/labs_test.rb
 - test/hatchet/multi_cmd_runner_test.rb
 - test/hatchet/reaper_test.rb
-- test/hatchet/runner_test.rb
 - test/test_helper.rb
 homepage: https://github.com/heroku/hatchet
 licenses:
@@ -270,7 +284,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.1
+rubygems_version: 2.6.11
 signing_key: 
 specification_version: 4
 summary: Hatchet is a an integration testing library for developing Heroku buildpacks.
@@ -282,8 +296,8 @@ test_files:
 - test/fixtures/buildpacks/null-buildpack/readme.md
 - test/hatchet/allow_failure_anvil_test.rb
 - test/hatchet/allow_failure_git_test.rb
-- test/hatchet/anvil_test.rb
 - test/hatchet/app_test.rb
+- test/hatchet/ci_test.rb
 - test/hatchet/config_test.rb
 - test/hatchet/edit_repo_test.rb
 - test/hatchet/git_test.rb
@@ -291,5 +305,4 @@ test_files:
 - test/hatchet/labs_test.rb
 - test/hatchet/multi_cmd_runner_test.rb
 - test/hatchet/reaper_test.rb
-- test/hatchet/runner_test.rb
 - test/test_helper.rb

data/test/hatchet/anvil_test.rb

@@ -1,7 +0,0 @@
-require 'test_helper'
-
-class AnvilTest < Test::Unit::TestCase
-  def setup
-    @buildpack_path = File.expand_path 'test/fixtures/buildpacks/heroku-buildpack-ruby'
-  end
-end

data/test/hatchet/runner_test.rb

@@ -1,31 +0,0 @@
-require 'test_helper'
-
-class HatchetRunnerTest < Test::Unit::TestCase
-
-  def setup
-    @default = ENV['HATCHET_DEPLOY_STRATEGY']
-  end
-
-  def teardown
-    ENV['HATCHET_DEPLOY_STRATEGY'] = @default
-    Hatchet.set_deploy_strategy!
-  end
-
-  def test_defaults
-    assert_equal nil, ENV['HATCHET_DEPLOY_STRATEGY']
-    assert_equal :git, Hatchet::DEPLOY_STRATEGY
-    assert_equal Hatchet::GitApp, Hatchet::Runner
-  end
-
-  def test_change_deploy_strat
-    ENV['HATCHET_DEPLOY_STRATEGY'] = "git"
-    Hatchet.set_deploy_strategy!
-    assert_equal :git, Hatchet::DEPLOY_STRATEGY
-    assert_equal Hatchet::GitApp, Hatchet::Runner
-
-    ENV['HATCHET_DEPLOY_STRATEGY'] = "anvil"
-    Hatchet.set_deploy_strategy!
-    assert_equal :anvil, Hatchet::DEPLOY_STRATEGY
-    assert_equal Hatchet::AnvilApp, Hatchet::Runner
-  end
-end