knife-server 0.2.2 → 0.3.0

data/CHANGELOG.md

@@ -1,4 +1,13 @@
-## 0.2.3.dev (unreleased)
+## 0.3.1.dev (unreleased)
+
+
+## 0.3.0 (July 7, 2012)
+
+### New features
+
+* Add `knife server restore` subcommand to restore data components (nodes,
+  roles, environments, data bags) from the workstation's file system.
+  ([@fnichol][])
 
 
 ## 0.2.2 (July 4, 2012)

data/README.md

@@ -1,7 +1,8 @@
 # <a name="title"></a> Knife::Server [![Build Status](https://secure.travis-ci.org/fnichol/knife-server.png?branch=master)](http://travis-ci.org/fnichol/knife-server) [![Dependency Status](https://gemnasium.com/fnichol/knife-server.png)](https://gemnasium.com/fnichol/knife-server) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/fnichol/knife-server)
 
-An Opscode Chef knife plugin to manage Chef Servers. Bootstrapping new Chef
-Servers (currently on Amazon's EC2) and node data backup is supported.
+An Opscode Chef knife plugin to manage Chef Servers. Bootstrap a new Chef
+Server on Amazon's EC2 or a standalone server. Backup and restore your Chef
+Server or Hosted Chef's node, role, data bag, and environment JSON data.
 
 ## <a name="usage"></a> Usage
 
@@ -36,6 +37,12 @@ snap:
 $ knife server backup
 ```
 
+Restoring all that data isn't too hard either:
+
+```bash
+$ knife server restore
+```
+
 See [below](#subcommands) for more details.
 
 ## <a name="installation"></a> Installation
@@ -282,6 +289,14 @@ Host name or IP address of the host to bootstrap.
 
 This option is **required**.
 
+##### --ssh-password PASSWORD (-P)
+
+The SSH password used when bootstrapping the Chef Server node. If no password
+is provided but an SSH key-based authentication fails, then you will be
+prompted interactively for a password. In other words, if your server
+requires password authentication you can skip this option and type it in after
+the plugin starts.
+
 ### <a name="knife-server-backup"></a> knife server backup
 
 Pulls Chef data primitives from a Chef Server as JSON for backup. Backups can
@@ -335,22 +350,52 @@ chef_server_url   = "https://api.opscode.com/organizations/coolinc"
 then a backup directory of
 `/var/chef/backups/api.opscode.com_20120401T084711-0000` would be created.
 
-##### --ssh-password PASSWORD (-P)
+### <a name="knife-server-restore"></a> knife server restore
+
+Restores Chef data primitives from JSON backups to a Chef Server. You can
+restore some or all of:
+
+* nodes
+* roles
+* environments
+* data bags
+
+A big thanks to [Steven Danna][stevendanna] and [Joshua Timberman][jtimberman]
+for the [BackupRestore][backup_restore] knife plugin which was the inspiration
+for this implementation.
+
+#### Configuration
 
-The SSH password used (if needed) when bootstrapping the Chef Server node. If
-this option is not explicitly set and key based authentication fails, you will
-be prompted to enter a password in an interactive prompt. In other words,
-you may omit typing your password on the command line and defer to a prompt.
+##### COMPONENT[ COMPONENT ...]
+
+The following component types are valid:
+
+* `nodes`
+* `roles`
+* `environments`
+* `data_bags` (note the underscore character)
+
+When no component types are specified, all will be selected for restore.
+This is equivalent to invoking:
+
+```bash
+$ knife server restore nodes roles environments data_bags
+```
+
+##### --backup-dir DIR (-D)
+
+The directory to containing backup JSON files. A sub-directory for each data
+primitive type is expected (the `knife server backup` subcommand provides
+this format).
+
+This option is **required**.
 
 ## <a name="roadmap"></a> Roadmap
 
 * Support for other platforms (alternative bootstrap templates)
 * Support for Rackspace provisioning (use knife-rackspace gem)
-* Support for standalone server provisioning
 * knife server backup backed by s3 (fog api)
-* knife server restore {nodes,roles,environments,data bags,all}
 * knife server restore from s3 archive (fog api)
-* knife server restore from local filesystem
 
 ## <a name="development"></a> Development
 
@@ -382,6 +427,7 @@ Apache License, Version 2.0 (see [LICENSE][license])
 [contributors]: https://github.com/fnichol/knife-server/contributors
 
 [backup_export]:            https://github.com/stevendanna/knife-hacks/blob/master/plugins/backup_export.rb
+[backup_restore]:           https://github.com/stevendanna/knife-hacks/blob/master/plugins/backup_restore.rb
 [chef_bootstrap_knife_rb]:  https://github.com/fnichol/chef-bootstrap-repo/blob/master/.chef/knife.rb
 [chef_bootstrap_repo]:      https://github.com/fnichol/chef-bootstrap-repo/
 [jtimberman]:               https://github.com/jtimberman

data/lib/chef/knife/server_restore.rb

@@ -0,0 +1,101 @@
+#
+# Author:: Fletcher Nichol (<fnichol@nichol.ca>)
+# Copyright:: Copyright (c) 2012 Fletcher Nichol
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'chef/knife'
+
+class Chef
+  class Knife
+    class ServerRestore < Knife
+
+      deps do
+        require 'chef/json_compat'
+      end
+
+      banner "knife server restore COMPONENT[ COMPONENT ...] (options)"
+
+      option :backup_dir,
+        :short => "-D DIR",
+        :long => "--backup-dir DIR",
+        :description => "The directory containing backup files"
+
+      def run
+        validate!
+        components = name_args.empty? ? COMPONENTS.keys : name_args
+
+        Array(components).each { |type| restore_components(type) }
+      end
+
+      private
+
+      COMPONENTS = {
+        "nodes" => { :singular => "node", :klass => Chef::Node },
+        "roles" => { :singular => "role", :klass => Chef::Role },
+        "environments" => { :singular => "environment", :klass => Chef::Environment },
+        "data_bags" => { :singular => "data_bag", :klass => Chef::DataBag },
+      }
+
+      def validate!
+        bad_names = name_args.reject { |c| COMPONENTS.keys.include?(c) }
+        unless bad_names.empty?
+          ui.error "Component types #{bad_names.inspect} are not valid."
+          exit 1
+        end
+        if config[:backup_dir].nil?
+          ui.error "You did not provide a valid --backup-dir value."
+          exit 1
+        end
+      end
+
+      def restore_components(type)
+        c = COMPONENTS[type]
+        dir_path = ::File.join(config[:backup_dir], type)
+
+        Array(Dir.glob(::File.join(dir_path, "**/*.json"))).each do |json_file|
+          restore_component(c, json_file)
+        end
+      end
+
+      def restore_component(c, json_file)
+        obj = Chef::JSONCompat.from_json(
+          ::File.open(json_file, "rb") { |f| f.read }
+        )
+
+        if c[:klass] == Chef::DataBag
+          create_data_bag(::File.basename(::File.dirname(json_file)))
+          msg = "Restoring #{c[:singular]}" +
+            "[#{obj.data_bag}][#{obj.raw_data[:id]}]"
+        else
+          msg = "Restoring #{c[:singular]}[#{obj.name}]"
+        end
+
+        ui.msg msg
+        obj.save
+      end
+
+      def create_data_bag(name)
+        @created_data_bags ||= []
+
+        unless @created_data_bags.include?(name)
+          ui.msg "Restoring data_bag[#{name}]"
+          rest.post_rest("data", { "name" => name })
+          @created_data_bags << name
+        end
+      end
+    end
+  end
+end

data/lib/knife/server/version.rb

@@ -18,6 +18,6 @@
 
 module Knife
   module Server
-    VERSION = "0.2.2"
+    VERSION = "0.3.0"
   end
 end

data/spec/chef/knife/server_restore_spec.rb

@@ -0,0 +1,230 @@
+#
+# Author:: Fletcher Nichol (<fnichol@nichol.ca>)
+# Copyright:: Copyright (c) 2012 Fletcher Nichol
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'chef/knife/server_restore'
+require 'fakefs/spec_helpers'
+require 'fileutils'
+Chef::Knife::ServerRestore.load_deps
+
+describe Chef::Knife::ServerRestore do
+  include FakeFS::SpecHelpers
+
+  before do
+    Chef::Log.logger = Logger.new(StringIO.new)
+    @knife = Chef::Knife::ServerRestore.new
+    @stdout = StringIO.new
+    @knife.ui.stub!(:stdout).and_return(@stdout)
+    @knife.ui.stub(:msg)
+    @stderr = StringIO.new
+    @knife.ui.stub!(:stderr).and_return(@stderr)
+    @knife.config[:backup_dir] = "/baks"
+  end
+
+  describe "#run" do
+    let(:rest_client) { stub(:post_rest => true) }
+
+    before do
+      Chef::Node.any_instance.stub(:save) { true }
+      Chef::Role.any_instance.stub(:save) { true }
+      Chef::Environment.any_instance.stub(:save) { true }
+      Chef::DataBagItem.any_instance.stub(:save) { true }
+      @knife.stub(:rest) { rest_client }
+    end
+
+    it "exists if component type is invalid" do
+      @knife.name_args = %w{nodes hovercraft}
+
+      lambda { @knife.run }.should raise_error SystemExit
+    end
+
+    it "exists if backup_dir is missing" do
+      @knife.config.delete(:backup_dir)
+
+      lambda { @knife.run }.should raise_error SystemExit
+    end
+
+    context "for nodes" do
+      before do
+        @knife.name_args = %w{nodes}
+
+        stub_json_node!("mynode")
+      end
+
+      it "sends a message to the ui" do
+        @knife.ui.should_receive(:msg).with(/mynode/)
+
+        @knife.run
+      end
+
+      it "saves the node" do
+        Chef::Node.any_instance.should_receive(:save).once
+
+        @knife.run
+      end
+    end
+
+    context "for roles" do
+      before do
+        @knife.name_args = %w{roles}
+
+        stub_json_role!("myrole")
+      end
+
+      it "sends a message to the ui" do
+        @knife.ui.should_receive(:msg).with(/myrole/)
+
+        @knife.run
+      end
+
+      it "saves the role" do
+        Chef::Role.any_instance.should_receive(:save).once
+
+        @knife.run
+      end
+    end
+
+    context "for environments" do
+      before do
+        @knife.name_args = %w{environments}
+
+        stub_json_env!("myenv")
+      end
+
+      it "sends a message to the ui" do
+        @knife.ui.should_receive(:msg).with(/myenv/)
+
+        @knife.run
+      end
+
+      it "saves the environment" do
+        Chef::Environment.any_instance.should_receive(:save).once
+
+        @knife.run
+      end
+    end
+
+    context "for data_bags" do
+      before do
+        @knife.name_args = %w{data_bags}
+
+        stub_json_data_bag_item!("mybag", "myitem")
+      end
+
+      it "sends a message to the ui" do
+        @knife.ui.should_receive(:msg).with(/myitem/)
+
+        @knife.run
+      end
+
+      it "creates the data bag" do
+        rest_client.should_receive(:post_rest).
+          with("data", { "name" => "mybag" })
+
+        @knife.run
+      end
+
+      it "only creates the data bag once for multiple items" do
+        stub_json_data_bag_item!("mybag", "anotheritem")
+        rest_client.should_receive(:post_rest).
+          with("data", { "name" => "mybag" }).once
+
+        @knife.run
+      end
+
+      it "saves the data bag item" do
+        Chef::DataBagItem.any_instance.should_receive(:save).once
+
+        @knife.run
+      end
+    end
+
+    context "for all" do
+      before do
+        stub_json_node!("nodey")
+        stub_json_role!("roley")
+        stub_json_env!("envey")
+        stub_json_data_bag_item!("bagey", "itemey")
+      end
+
+      it "saves nodes" do
+        Chef::Node.any_instance.should_receive(:save)
+
+        @knife.run
+      end
+
+      it "saves roles" do
+        Chef::Role.any_instance.should_receive(:save)
+
+        @knife.run
+      end
+
+      it "saves environments" do
+        Chef::Environment.any_instance.should_receive(:save)
+
+        @knife.run
+      end
+
+      it "creates data bags" do
+        rest_client.should_receive(:post_rest).
+          with("data", { "name" => "bagey" })
+
+        @knife.run
+      end
+
+      it "saves data bag items" do
+        Chef::DataBagItem.any_instance.should_receive(:save)
+
+        @knife.run
+      end
+    end
+  end
+
+  private
+
+  def stub_json_node!(name)
+    stub_json_component!(Chef::Node, "nodes", name)
+  end
+
+  def stub_json_role!(name)
+    stub_json_component!(Chef::Role, "roles", name)
+  end
+
+  def stub_json_env!(name)
+    stub_json_component!(Chef::Environment, "environments", name)
+  end
+
+  def stub_json_data_bag_item!(bag, name)
+    dir = File.join(@knife.config[:backup_dir], "data_bags", bag)
+    obj = Chef::DataBagItem.new
+    obj.data_bag(bag)
+    obj.raw_data[:id] = name
+    serialize_component(obj, File.join(dir, "#{name}.json"))
+  end
+
+  def stub_json_component!(klass, plural, name)
+    dir = File.join(@knife.config[:backup_dir], plural)
+    obj = klass.new
+    obj.name(name)
+    serialize_component(obj, File.join(dir, "#{name}.json"))
+  end
+
+  def serialize_component(obj, file_path)
+    FileUtils.mkdir_p(File.dirname(file_path))
+    File.open(file_path, "wb") { |f| f.write(obj.to_json) }
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: knife-server
 version: !ruby/object:Gem::Version
-  version: 0.2.2
+  version: 0.3.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-07-05 00:00:00.000000000 Z
+date: 2012-07-07 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: fog
-  requirement: &2164645740 !ruby/object:Gem::Requirement
+  requirement: &2164645600 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -21,10 +21,10 @@ dependencies:
         version: '1.3'
   type: :runtime
   prerelease: false
-  version_requirements: *2164645740
+  version_requirements: *2164645600
 - !ruby/object:Gem::Dependency
   name: net-ssh
-  requirement: &2164645080 !ruby/object:Gem::Requirement
+  requirement: &2164644960 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,10 +32,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *2164645080
+  version_requirements: *2164644960
 - !ruby/object:Gem::Dependency
   name: chef
-  requirement: &2164644240 !ruby/object:Gem::Requirement
+  requirement: &2164644040 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -43,10 +43,10 @@ dependencies:
         version: 0.10.10
   type: :runtime
   prerelease: false
-  version_requirements: *2164644240
+  version_requirements: *2164644040
 - !ruby/object:Gem::Dependency
   name: knife-ec2
-  requirement: &2164643440 !ruby/object:Gem::Requirement
+  requirement: &2164643320 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -54,10 +54,10 @@ dependencies:
         version: 0.5.12
   type: :runtime
   prerelease: false
-  version_requirements: *2164643440
+  version_requirements: *2164643320
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &2164642700 !ruby/object:Gem::Requirement
+  requirement: &2164642520 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -65,10 +65,10 @@ dependencies:
         version: '2.10'
   type: :development
   prerelease: false
-  version_requirements: *2164642700
+  version_requirements: *2164642520
 - !ruby/object:Gem::Dependency
   name: fakefs
-  requirement: &2164642100 !ruby/object:Gem::Requirement
+  requirement: &2164642000 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -76,10 +76,10 @@ dependencies:
         version: 0.4.0
   type: :development
   prerelease: false
-  version_requirements: *2164642100
+  version_requirements: *2164642000
 - !ruby/object:Gem::Dependency
   name: timecop
-  requirement: &2164641600 !ruby/object:Gem::Requirement
+  requirement: &2164641520 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -87,7 +87,7 @@ dependencies:
         version: 0.3.5
   type: :development
   prerelease: false
-  version_requirements: *2164641600
+  version_requirements: *2164641520
 description: Chef Knife plugin to bootstrap Chef Servers
 email:
 - fnichol@nichol.ca
@@ -109,6 +109,7 @@ files:
 - lib/chef/knife/server_bootstrap_base.rb
 - lib/chef/knife/server_bootstrap_ec2.rb
 - lib/chef/knife/server_bootstrap_standalone.rb
+- lib/chef/knife/server_restore.rb
 - lib/knife-server.rb
 - lib/knife/server/credentials.rb
 - lib/knife/server/ec2_security_group.rb
@@ -117,6 +118,7 @@ files:
 - spec/chef/knife/server_backup_spec.rb
 - spec/chef/knife/server_bootstrap_ec2_spec.rb
 - spec/chef/knife/server_bootstrap_standalone_spec.rb
+- spec/chef/knife/server_restore_spec.rb
 - spec/knife/server/credientials_spec.rb
 - spec/knife/server/ec2_security_group_spec.rb
 - spec/knife/server/ssh_spec.rb
@@ -148,6 +150,7 @@ test_files:
 - spec/chef/knife/server_backup_spec.rb
 - spec/chef/knife/server_bootstrap_ec2_spec.rb
 - spec/chef/knife/server_bootstrap_standalone_spec.rb
+- spec/chef/knife/server_restore_spec.rb
 - spec/knife/server/credientials_spec.rb
 - spec/knife/server/ec2_security_group_spec.rb
 - spec/knife/server/ssh_spec.rb