knife-topo 0.0.3

data/LICENSE

@@ -0,0 +1,201 @@
+Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "{}"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2014 ThirdWave Insights, LLC
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

data/README.md

@@ -0,0 +1,308 @@
+knife topo
+==========
+
+This Chef Knife plugin is provided by ThirdWave Insights LLC. 
+It allows you to use knife to create and update topologies consisting of 
+multiple nodes.  
+
+# Installation #
+
+Copy the contents of lib/chef/knife into your plugin directory, e.g.:
+
+	$ mkdir -p ~/.chef/knife/plugins
+	$ cp lib/chef/knife/* ~/.chef/knife/plugins
+
+or install knife-topo as a gem
+
+    $ sudo gem install knife-topo
+
+This plugin has been tested with Chef Version 11.12 on Ubuntu 14.04 LTS.
+
+# Usage #
+
+Define one or more *topologies* in a [*topology file*](#markdown-header-topology-file). 
+
+Import the topology file into your local chef repo using
+[knife topo import](#markdown-header-knife-topo-import). Create and optionally
+bootstrap a topology on the Chef server using 
+[knife topo create](#markdown-header-knife-topo-create), and update it
+using [knife topo update](#markdown-header-knife-topo-update).
+
+# Getting Started #
+
+Try out this plugin using a [test repo](knife-topo/src/master/test-repo) which you can
+download from Github or is included in the installed gem. See the 
+[Instructions](knife-topo/src/master/test-repo/Instructions.txt) for how to use the test repo.
+
+The instructions assume you have
+ [chefDK](http://www.getchef.com/downloads/chef-dk/)
+ or equivalent installed and working with Vagrant and VirtualBox, but
+ chefDK is not a requirement for this plugin. 
+
+# Topology File #
+
+See the [example topology file](knife-topo/src/master/test-repo/topology.json)
+
+The topology file contains a single topology, or an array of topologies.
+Each topology has some overall properties, an array of nodes and 
+an array defining topology cookbook attributes.
+
+## Overall Properties
+
+```json
+    {
+        "name": "test1",
+        "chef_environment": "test",
+        "tags": ["system_sys1", "phase_test" ],
+        "normal": {
+            "owner": {
+              "name": "Christine Draper"
+            }
+        },
+        "nodes" : [
+          ...
+        ],
+        "cookbook_attributes" : [
+        ]
+    },
+```
+The `name` is how you will refer to the topology in the
+`knife topo` subcommands.
+
+The `chef-environment` and `normal` attributes defined
+here will be applied to all nodes in the topology, unless alternative
+values are provided for a specific node.  The `tags` 
+will be added to each node. The normal attributes may either be 
+specified using the dot notation, as shown above, or may be expanded as
+a JSON structure, i.e., the above is equivalent to:
+```json
+    {
+        "normal": {
+            "topo": {
+              "blueprint": "blah"
+            }
+        }
+    }
+    
+## Node List
+Each topology contains a list of `nodes`.
+
+```json
+    "{
+        "name": "test1",
+        ...,
+        "nodes": {
+            "buildserver": {
+                "name": "buildserver01",
+                "ssh_host": "127.0.0.1",
+                "ssh_port": "8241",
+                "chef_environment": "dev",
+                "run_list": ["role[base-ubuntu]", "ypo::db", "recipe[ypo::appserver]"],
+                "normal": {
+                  "topo.node_type": "buildserver",
+                  "another": "Value 1"
+                },
+                "tags": ["build"]
+            },
+            ...
+            }
+        }
+    },
+    ...
+    ]}
+```
+Within `nodes`, the `name` field is the node name that will be used in Chef.
+The fields `chef_environment`, `run_list`, `tags` and the attributes
+within `normal` will also be applied to the node in Chef. All of these
+fields are optional. 
+
+The `ssh_host` and `ssh_port` are optional fields that are used to
+bootstrap a node.
+
+## Topology Cookbook Attributes
+
+
+
+
+# Subcommands #
+
+The main subcommands for `knife topo` are:
+
+* `knife topo import` - Import one or more into your workspace/local repo
+* `knife topo create` - Create and optionally bootstrap a topology of nodes
+* `knife topo update` - Update a topology of nodes
+
+The additional subcommands can also be useful, depending on your
+workflow:
+
+* `knife topo bootstrap` - Bootstraps a topology of nodes
+* `knife topo cookbook create` - Generate the topology cookbooks
+* `knife topo cookbook upload` - Upload the topology cookbooks
+* `knife export` - Export data from a topology (or from nodes that you want in a topology)
+
+The topologies are data bag items in the 'topologies' data bag, so 
+you can also use knife commands such as:
+
+* `knife data bag show topologies` - List the topologies
+* `knife data bag show topologies test1` - Show details of the test1 topology
+* `knife data bag delete topologies test1` - Delete the test1 topology 
+
+### Common Options:
+
+The knife topo subcommands support the following common options.
+
+Option        | Description
+------------  | -----------
+-D, --data-bag DATA_BAG    | The data bag to use for the topologies. Defaults to 'topologies'.
+
+## knife topo bootstrap
+
+	knife topo bootstrap TOPOLOGY
+
+Runs the `knife bootstrap` command for each node in the topology that
+has the `ssh_host` attribute. Specified options will be passed through
+to `knife bootstrap` and applied to each node.
+
+### Options:
+
+The knife topo bootstrap  subcommand supports the following additional options.
+
+Option        | Description
+------------  | -----------
+See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html)  | Options supported by `knife bootstrap` are passed through to the bootstrap command
+
+
+### Examples:
+The following will bootstrap nodes in the test1 topology, using a
+user name of vagrant, password of vagrant, and running using sudo.
+
+	$ knife topo bootstrap sys1_test test1 -x vagrant -P vagrant --sudo
+
+## knife topo cookbook create 
+
+	knife topo cookbook create TOPOLOGY
+
+Generates the topology cookbook attribute files and attributes described in the 
+'cookbook_attributes' property.
+
+### Examples:
+The following will generate the topology cookbook attribute files for
+topology test1.
+
+	$ knife topo cookbook create test1
+	
+## knife topo create 
+
+	knife topo create TOPOLOGY
+
+Creates the specified topology in the chef server as an item in the 
+system environment data bag. Creates the chef environment associated
+with the system environment, if it does not already exist. Uploads any
+topology cookbooks. Creates or 
+updates nodes identified in the topology, using information 
+specified in the system environment and for the specific node. 
+
+### Options:
+
+The knife topo create subcommand supports the following additional options.
+
+Option        | Description
+------------  | -----------
+--bootstrap    | Bootstrap the topology (see [topo bootstrap](#markdown-header-knife-topo-bootstrap))
+See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html)  | Options supported by `knife bootstrap` are passed through to the bootstrap command
+--no-upload   | Do not upload topology cookbooks
+
+### Examples:
+The following will create the 'test1' topology, and bootstrap it.
+
+	$ knife topo create test1 --bootstrap
+
+The following will create the 'test1' topology but will not bootstrap it 
+or upload topology cookbooks.
+
+$ knife topo create test1 --no-upload
+
+## knife topo export 
+
+	knife topo export [ TOPOLOGY [ NODE ... ] 
+
+Exports the specified topology as JSON. If the topology does not already exist, 
+an outline for a new topology will be exported. The exported JSON
+can be used as the basis for a new topology definition.
+
+If nodes are specified, these will be added into the export in addition
+to any nodes that are in the topology. 
+
+If no topology is specified, all existing topologies in that environment 
+will be exported.
+
+### Examples:
+The following will export all topologies to a file called 'sys1_test.json'.
+
+	$ knife topo export sys1_test > sys1_test.json
+	
+The following will create an outline for a new topology called  'christine_test':
+
+	$ knife topo export christine_test > christine_test.json
+
+
+## knife topo import 
+
+	knife topo import [ TOPOLOGY_FILE [ TOPOLOGY ... ]] 
+
+Imports the system environment and topologies from a
+[topology file](#Topology File) into the local repo. If no topology
+file is specified, attempts to read from a file called 'topology.json'
+in the current directory. Generates the topology cookbook attribute 
+files and attributes described in the 'cookbook_attributes' property.
+
+### Examples:
+The following will import all topologies defined in the 'topology.json' file.
+
+	$ knife topo import topology.json
+
+The following will import the 'test1' topology
+ defined in the 'topology.json' file.
+
+	$ knife topo import topology.json test1
+
+## knife topo update
+
+	knife topo update [ TOPOLOGY ] 
+
+Updates the specified topology. Creates or updates nodes 
+identified in the topology, using information specified in the 
+topology for the specific node. 
+
+If no topology is specified, all existing topologies in that environment 
+will be updated.
+
+### Examples:
+The following will update the 'test1' topology.
+
+	$ knife topo update test1
+	
+The following will update all topologies in the 'topologies' data bag.
+
+	$ knife topo update
+	
+
+# License #
+
+Author:: Christine Draper (christine_draper@thirdwaveinsights.com)
+
+Copyright:: Copyright (c) 2014 ThirdWave Insights, LLC
+
+License:: Apache License, Version 2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/knife-topo.gemspec

@@ -0,0 +1,21 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'chef/knife/topo/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "knife-topo"
+  spec.version       = Knife::Topo::VERSION
+  spec.authors       = ["Christine Draper"]
+  spec.email         = ["christine_draper@thirdwaveinsights.com"]
+  spec.summary       = "Knife plugin that to manage topologies of nodes"
+  spec.description   = spec.summary
+  spec.homepage      = "https://github.com/christinedraper/knife-topo"
+  spec.license       = "Apache License (2.0)"
+  
+  spec.files         = Dir.glob("{lib,test-repo}/**/*") +
+  	Dir.glob("test-repo/.chef/*") +
+    ['LICENSE', 'README.md', __FILE__]
+  spec.require_paths = ["lib"]
+
+end

data/lib/chef/knife/topo/version.rb

@@ -0,0 +1,5 @@
+module Knife
+  module Topo
+    VERSION = "0.0.3"
+  end
+end

data/lib/chef/knife/topo_bootstrap.rb

@@ -0,0 +1,114 @@
+#
+# Author:: Christine Draper (<christine_draper@thirdwaveinsights.com>)
+# Copyright:: Copyright (c) 2014 ThirdWave Insights LLC
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+
+require_relative 'topology_helper'
+require 'chef/knife/bootstrap'
+
+class Chef
+  class Knife
+    class TopoBootstrap < Chef::Knife
+
+      deps do
+         Chef::Knife::Bootstrap.load_deps
+      end
+
+      banner "knife topo bootstrap TOPOLOGY (options)"
+
+      option :data_bag,
+      :short => '-D DATA_BAG',
+      :long => "--data-bag DATA_BAG",
+      :description => "The data bag the topologies are stored in"
+
+      # Make the base bootstrap options available on topo bootstrap
+      self.options = (Chef::Knife::Bootstrap.options).merge(self.options)
+
+      def initialize (args)
+        super
+        @bootstrap_args = initialize_cmd_args(args, [ 'bootstrap', '' ])
+
+        # All called commands need to accept union of options
+        Chef::Knife::Bootstrap.options = options
+
+      end
+
+      def run
+        if !@name_args[0]
+          show_usage
+          ui.fatal("You must specify the name of a topology")
+          exit 1
+        end
+
+        @bag_name = topo_bag_name(config[:data_bag])
+        @topo_name = @name_args[0]
+
+        # get the node names for the topology
+        unless topo = load_from_server(@bag_name, @topo_name )
+          ui.fatal("Topology #{@bag_name}/#{@topo_name} does not exist on the server - use 'knife topo create' first")
+          exit(1)
+        end
+
+        # load and bootstrap each node that has a ssh_host
+        nodes = merge_topo_properties(topo['nodes'], topo)
+        @failed = []
+        skipped = 0
+        succeeded = 0
+        if nodes.length > 0
+          nodes.each do |node_data|
+            if node_data['ssh_host']
+              run_bootstrap(node_data)
+              succeeded += 1             
+            else
+              ui.info "Node #{node_data['name']} does not have ssh_host specified - skipping bootstrap"
+              skipped += 1
+            end
+
+          end
+          ui.info "Successfully bootstrapped #{nodes.length - (@failed.length + skipped)} nodes and skipped #{skipped} nodes of #{nodes.length} in topology #{@bag_name}/#{@topo_name}"
+          ui.warn "#{@failed.length} nodes [ #{@failed.join(', ')} ] failed to bootstrap" if @failed.length > 0
+        else
+          ui.info "No nodes found for topology #{@topo_name}"
+        end
+      end
+
+      # Setup the bootstrap args and run the bootstrap command
+      def run_bootstrap(node_data)
+        node_name = node_data['name']
+        args = @bootstrap_args
+        args += ['-N', node_name] if(node_name)
+        args += ['-E', node_data['chef_environment']] if(node_data['chef_environment'])
+        args[1] = node_data['ssh_host']
+        args += [ '--ssh-port', node_data['ssh_port']] if node_data['ssh_port']
+        args += [ '--run-list' , node_data['run_list'].join(',')] if node_data['run_list']
+
+        ui.info "Bootstrapping node #{node_name}"
+        begin
+          run_cmd(Chef::Knife::Bootstrap, args)
+        rescue Exception => e
+          raise if Chef::Config[:verbosity] == 2
+          @failed << node_name
+          ui.warn "bootstrap of node #{node_name} exited with error"
+          humanize_exception(e)
+        end
+      end
+
+      include Chef::Knife::TopologyHelper
+
+    end
+  end
+end