kitchen_hooks 1.5.2 → 1.5.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: cf6b444d973aa69f4eaf3bf79ccacca65e717149
-  data.tar.gz: fcdb939b53fc35a04eb18eaad4cb52ce81166304
+  metadata.gz: bc604b1b2abab2073d9e568aaa74126476eca24e
+  data.tar.gz: 6f8dab7e189d19fd33a4fa15c4e590b15c831b44
 SHA512:
-  metadata.gz: 935caae8b512c4e1607840cf428778fdf94914c70107e73b5f7df50e23d00d44c5430656e0013ed081c32dbaf320881d4625b413b3c1a9e5c9844885fb37179c
-  data.tar.gz: 7ebdc2ebc6cf361a74305547cb455313da6a1c67bfe202af2524a90e65820387800ff5212a3cb86500ad4b98146447e2de3ce537064a9eeecb112aa1095de274
+  metadata.gz: e31f2ac0f8bb6d0e152ace39709ba420b56484f8e3ce22de3ea1d03e9697a352100eb29ff8853d1ccce9e92bc001b1403505df716cc2e8f8527c3048e25ff5cf
+  data.tar.gz: e1ed91030b09ec8dee1a399f316c46c193dd8d6cf9670528ed4f376dc2426cbfea227dcf08e4fd00a95cc568f0b7235e94394a547bb0e611d7c46d5e5e2f6728

data/Readme.md

@@ -4,9 +4,68 @@
 WebHoook for automated Chef Server uploads following Kitchen standards.
 
 
+## What?
+
+So [the Kitchen](http://wiki.bluejeansnet.com/operations/kitchen) is this thing
+we have now in Operations to help guide cookbook development. The [Workflow](http://wiki.bluejeansnet.com/operations/kitchen#workflow)
+sction of the docs prescribes how to go about your day in Chef land including
+how to version, release, and deploy changes to a Chef server.
+
+But that's only _one_ Chef server. What if we want redundancy?
+
+A distributed/highly-available Chef would do the job. It comes in a few flavors:
+
+1. Pay for whatever they're calling [Hosted Chef](https://manage.chef.io/signup) nowadays
+2. Manually deploy a [highly-available setup in AWS](https://docs.getchef.com/install_server_ha_aws.html)
+3. Manually deploy a [highly-available setup with DRBD](https://docs.getchef.com/server_high_availability.html#drbd)
+
+The third option is basically a recipe for disaster. DRBD is known to lose data
+in split-brain scenarios. So that's actually two flavors.
+
+Hosted Chef seems like a great option. Until you start using it. They give you
+whatever bleeding-edge version of Chef they're testing so you don't have time
+to breathe between incompatible versions. Chef is notorious for yanking gems
+and botching releases. They've also got some odd permissions you probably have
+never seen before, because vanilla Chef server doesn't have organizations.
+
+That leaves AWS as an option. But it's a lot of work, and weirdly completely
+not automated.
+
+All I care about is having my Chef data replicated, why does it have to be hard?
+I mean, we've been using `git` hooks to fan out to Chef servers for years now!
+
+### So let's do that.
+
+The old post-receive hook was pretty basic, but it worked. By default, it would
+look up the `knife` and `berks` configuration files (`.rb` and `.json`) in
+`~/knives` then loop over them when uploading Chef objects found in the
+checkout specified by the hook.
+
+The new Kitchen Hooks works much the same, except we're using GitLab WebHooks
+rather than traditional `git` hooks. This makes the Hooks easier to maintain,
+as installing `git` hooks has not been automated, and WebHooks are opt-in.
+
+The new Hooks cover three scenarios:
+
+1. Pushes to the `master` branch of the Kitchen trigger an upload of all roles,
+   environments, and data bags. This will overwrite any changes made on the Chef
+   server, but you shouldn't be doing that anyway. Importantly, this has no
+   effect on cookbook versions pinned in any environment.
+2. Version tag pushes (e.g. `v1.0.0`) to cookbooks trigger a `knife`-style
+   cookbook upload with `freeze` set to true. A `Berksfile.lock` will also
+   trigger the equivalent of `berks install` and `berks upload`.
+3. Environment tag pushes (e.g. `bjn_logger_prod`) to realm cookbooks trigger
+   `berks apply` behavior. Remember to delete both the local and remote tags
+   when you want to update an environment with a later version of the realm.
+
+Actions performed by Kitchen Hooks are stored in a [Daybreak](http://propublica.github.io/daybreak/)
+database and presented as a timeline in the [Web UI](http://git.bluejeansnet.com:4567).
+Notifcations can are sent to HipChat.
+
 ## Installation
 
-Clone the repo and `rake build`.
+Try `gem install kitchen_hooks`. Or add it to your `Gemfile`. Or clone the repo
+and `rake build`. Do all three, I don't care!
 
 
 ## Usage
@@ -29,21 +88,42 @@ Use the `server` command to start the WebHook receiver:
     Listening on 0.0.0.0:80, CTRL+C to stop
     ...
 
+## Configuration
+
+The configuration file is just JSON. Hopefully it's obvious:
+
+    {
+      "hipchat": {
+        "nick": "name",
+        "room": "test",
+        "token": "your_v1_api_token"
+      },
+      "knives": {
+        "user": "~/.chef/knife.rb",
+        "system": "/etc/chef/knife.rb",
+        "another": "/path/to/knife.rb"
+      }
+    }
+
+The `server` command also exposes some options for Sinatra configuration. See
+`kitchen_hooks help server`.
+
+## Development
 
-## TODO
+### TODO
 
 * Use Ridley for data bag, role, and environment uploads to remove Chef dependency
 
-## Changelog
+### Changelog
 
-### 1.5
+#### 1.5
 
 * Add release notifications
 * Add indication of success or failure
 * Only upload on commits to Kitchen `master` branch
 * Add custom timeline icons to distinguish event types
 
-### 1.4
+#### 1.4
 
 * Simplified configuration format (breaking!)
 * Added support for HipChat notifications when configured
@@ -51,19 +131,19 @@ Use the `server` command to start the WebHook receiver:
 * Replaced "modified" with more appropriate verb where appropriate
 * Corrected `berks upload` functionality (`berks install` first)
 
-### 1.3
+#### 1.3
 
 * Added local database to store history (Daybreak), visualized on homepage
 * Added `database` option to `server` command
 * Corrected GitLab link for tagged commits
 * Process events in the background to avoid duplicate entries [INF-6040]
 
-### 1.2
+#### 1.2
 
 * Added `bind` option to `server` command
 * Added `berks upload` functionality when tagging realm versions
 
-### 1.1
+#### 1.1
 
 * Moved `examples` to `etc`
 * Started checking in `Gemfile.lock` for future reference
@@ -73,6 +153,6 @@ Use the `server` command to start the WebHook receiver:
 * Tagging a cookbook with a version triggers a cookbook upload (frozen)
 * Tagging a realm with the name of an environment applies version constraints
 
-### 1.0
+#### 1.0
 
 * Initial release. Gem structure in place, but lacking functionaily

data/VERSION

@@ -1 +1 @@
-1.5.2
+1.5.3

data/etc/commit.json

@@ -1,24 +1,24 @@
 {
   "before": "08f20548e398f634979047d0c8114260bcc2949a",
-  "after": "fb38bcc884a0579cb52836b00589bca653a9c229",
-  "ref": "refs/heads/master",
+  "after": "7b238ce9c6cff250db5a0b727b9661b618b82e1d",
+  "ref": "refs/tags/v1.5.1",
   "user_id": 645,
   "user_name": "Sean Clemmer",
   "repository": {
-    "name": "Kitchen Hooks",
-    "url": "git@git.bluejeansnet.com:chef/kitchen.git",
+    "name": "realm_logger",
+    "url": "git@git.bluejeansnet.com:chef/realm_logger.git",
     "description": "",
-    "homepage": "http://git.bluejeansnet.com/kitchen"
+    "homepage": "http://git.bluejeansnet.com/chef/realm_logger"
   },
   "commits": [
     {
-      "id": "fb38bcc884a0579cb52836b00589bca653a9c229",
-      "message": "Update readme",
+      "id": "7b238ce9c6cff250db5a0b727b9661b618b82e1d",
+      "message": "XYZ",
       "timestamp": "2014-11-25T18:30:57-08:00",
-      "url": "http://git.bluejeansnet.com/kitchen-hooks/commit/fb38bcc884a0579cb52836b00589bca653a9c229",
+      "url": "http://git.bluejeansnet.com/chef/realm_logger/commit/7b238ce9c6cff250db5a0b727b9661b618b82e1d",
       "author": {
-        "name": "Kelly Wong",
-        "email": "kelly@bluejeans.com"
+        "name": "Sean Clemmer",
+        "email": "sclemmer@bluejeans.com"
       }
     }
   ],

data/etc/config.json

@@ -1,4 +1,6 @@
 {
   "hipchat": null,
-  "knives": {}
+  "knives": {
+    "user": "~/.chef/knife.rb"
+  }
 }

data/lib/kitchen_hooks/app.rb

@@ -22,6 +22,10 @@ module KitchenHooks
       @@db = Daybreak::DB.new path
     end
 
+    def self.close!
+      @@db.close
+    end
+
     def self.config! config
       @@hipchat = nil
       if config['hipchat']
@@ -39,7 +43,7 @@ module KitchenHooks
       process_release
       db_entries = {}
       db.each do |k, v|
-        db_entries[k] = v unless k == 'meta'
+        db_entries[k] = v unless k =~ /^meta/
       end
       erb :app, locals: {
         database: db_entries.sort_by { |stamp, _| stamp }
@@ -90,10 +94,12 @@ module KitchenHooks
 
 
     # error == nil   => success
+    # error == true  => success
     # error == false => nop
     # otherwise      => failure
     def mark event, type, error=nil
       return if error == false
+      error = nil if error == true
       entry = { type: type, event: event }
       entry.merge!(error: error, type: 'failure') if error
       db.synchronize do
@@ -105,15 +111,20 @@ module KitchenHooks
 
 
     def process_release version=KitchenHooks::VERSION
-      db['meta'] ||= {}
-      return if db['meta']['version'] == version
+      return if db['meta_version'] == version
+      db.set! 'meta_version', version
       mark version, 'release'
-      db.synchronize do
-        db['meta']['version'] = version
-      end
-      db.flush
     end
 
+
+    def err_out e, msg
+      logger.error msg
+      logger.error e.message
+      logger.error e.backtrace.inspect
+      msg
+    end
+
+
     def process event
       if event.nil? # JSON parse failed
         mark event, 'failure', 'Could not parse WebHook payload'        
@@ -121,22 +132,31 @@ module KitchenHooks
       end
 
       if commit_to_kitchen?(event)
-        possible_error = perform_kitchen_upload(event, knives) \
-          rescue 'Unable to perform kitchen upload'
+        possible_error = begin
+          perform_kitchen_upload(event, knives)
+        rescue Exception => e
+          err_out e, 'Could not perform kitchen upload'
+        end
         mark event, 'kitchen upload', possible_error
       end
 
       if tagged_commit_to_cookbook?(event) &&
          tag_name(event) =~ /^v\d+/ # Cookbooks tagged with a version
-        possible_error = perform_cookbook_upload(event, knives) \
-          rescue 'Unable to perform cookbook upload'
+        possible_error = begin
+          perform_cookbook_upload(event, knives)
+        rescue Exception => e
+          err_out e, 'Could not perform cookbook upload'
+        end
         mark event, 'cookbook upload', possible_error
       end
 
       if tagged_commit_to_realm?(event) &&
          tag_name(event) =~ /^bjn_/ # Realms tagged with an environment
-        possible_error = perform_constraint_application(event, knives) \
-          rescue 'Unable to apply constraints'
+        possible_error = begin
+          perform_constraint_application(event, knives)
+        rescue Exception => e
+          err_out e, 'Could not apply constraints'
+        end
         mark event, 'constraint application', possible_error
       end
     end

data/lib/kitchen_hooks/helpers.rb

@@ -1,4 +1,5 @@
 require 'shellwords'
+require 'fileutils'
 require 'tempfile'
 require 'json'
 
@@ -24,7 +25,8 @@ module KitchenHooks
         end
       end
 
-      return nil # no error
+      logger.info "finished perform_constraint_application: #{event['after']}"
+      return true # no error
     end
 
 
@@ -48,7 +50,8 @@ module KitchenHooks
         end
       end
 
-      return nil # no error
+      logger.info "finished perform_kitchen_upload: #{event['after']}"
+      return true # no error
     end
 
 
@@ -57,7 +60,9 @@ module KitchenHooks
         Dir.chdir clone do
           tagged_version = tag_name(event).delete('v')
           cookbook_version = File.read('VERSION').strip
-          raise unless tagged_version == cookbook_version
+          unless tagged_version == cookbook_version
+            raise 'Tagged version does not match cookbook version'
+          end
 
           logger.info 'Uploading cookbook'
           with_each_knife "cookbook upload #{cookbook_name event} -o .. --freeze", knives
@@ -74,7 +79,8 @@ module KitchenHooks
         end
       end
 
-      return nil # no error
+      logger.info "finished cookbook_upload: #{event['after']}"
+      return true # no error
     end
 
 
@@ -98,8 +104,10 @@ module KitchenHooks
       Dir.mktmpdir do |tmp|
         dir = File::join tmp, cookbook_name(event)
         repo = Git.clone git_daemon_style_url(event), dir, log: $stdout
-        repo.checkout self.send(commit_method, event)
+        commit = self.send(commit_method, event)
+        repo.checkout commit
         yield dir
+        FileUtils.rm_rf dir
       end
     end
 

data/lib/kitchen_hooks/main.rb

@@ -60,7 +60,13 @@ module KitchenHooks
       App.set :port, options.port
       App.set :bind, options.bind
       App.set :raise_errors, true
+      App.set :dump_errors, true
+      App.set :show_exceptions, true
       App.run!
+
+      at_exit do
+        App.close!
+      end
     end
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: kitchen_hooks
 version: !ruby/object:Gem::Version
-  version: 1.5.2
+  version: 1.5.3
 platform: ruby
 authors:
 - Kelly Wong
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-12-06 00:00:00.000000000 Z
+date: 2014-12-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: hipchat