motherbrain 0.0.0.placeholder → 0.13.1

data/OPERATORS_GUIDE.md

@@ -0,0 +1,195 @@
+motherbrain is a tool used to orchestrate clusters of nodes.
+
+It does this through plugins provided by cookbooks that are stored on the Chef Server. These plugins define commands that control various services and other configurations on the servers. Plugins are collections of components which contain groups which are collections of nodes.
+
+### Q: What is a plugin?
+A plugin is a part of a cookbook that provides information to motherbrain to give the ability to orchestrate an environment.
+
+### Q: What is a component?
+A component is a part of a plugin that defines details about a service or part of a service that is running as part of the application that the cookbook deploys.
+
+### Q: What is a group?
+A group is how the component identifies the nodes that are part of the component. This is done via the run list of the nodes.
+
+### Q: What is a command?
+A command allows users to perform an action on a group of nodes. An action can be something like changing the state of a service to started or stopped, migrating a database, or clearing an application's cache.
+
+Commands can be run on a component level to manipulate *just* that component or a command can exist at the plugin level and executing multiple component commands.
+
+## Basic Usage
+
+To find out what commands are available for a given plugin contained inside a particular cookbook, you can get help output:
+
+```
+$ mb pvpnet_core --environment NA_production
+Finding the latest version of the pvpnet_core plugin. This may take a few seconds...
+using pvpnet_core (2.3.189)
+
+Commands:
+  mb pvpnet_core bootstrap MANIFEST           # Bootstrap a manifest of node groups
+  mb pvpnet_core broker [COMMAND]             # command and control the brokers
+  mb pvpnet_core coherence [COMMAND]          # command and control the coherence app
+  mb pvpnet_core help [COMMAND]               # Describe subcommands or one specific subcommand
+  mb pvpnet_core nodes                        # List all nodes grouped by Component and Group
+  mb pvpnet_core platform_content [COMMAND]   # command and control platform content
+  mb pvpnet_core platform_database [COMMAND]  # command and control the database
+  mb pvpnet_core provision MANIFEST           # Create a cluster of nodes and add them to a Chef environment
+  mb pvpnet_core tomcat [COMMAND]             # comand and control the tomcat app
+  mb pvpnet_core upgrade                      # Upgrade an environment to the specified versions
+```
+
+By calling the cookbook name, few commands and a bunch of components (which may each have their own commands) are displayed.
+
+These are basic motherbrain commands not defined by a specific plugin:
+
+* mb pvpnet_core bootstrap MANIFEST - takes a MANIFEST (which is a JSON file) that contains information about what a cluster of nodes should look like. This will apply run lists to nodes as defined in the MANIFEST JSON.
+* mb pvpnet_core nodes - takes an environment (via --environment) and returns the nodes that are running components of pvpnet_core.
+* mb pvpnet_core provision MANIFEST - similar to bootstrap, however before performing a bootstrap, it will request new VMs from a cloud provider and then use those new instances to create a cluster of machines.
+* mb pvpnet_core upgrade - this allows users to upgrade the version of the software running on various components that are controlled by the cookbook. Example: `mb pvpnet_core upgrade --components broker:1.2.4 coherence:4.2.3`
+
+The rest of the entries above are components. To see what motherbrain knows about a specific component, you can ask mb similar to the above:
+
+```
+$ mb pvpnet_core coherence --environment NA_production
+using pvpnet_core (2.3.189)
+
+Commands:
+  mb coherence fluff               # Execute a fluff on the core coherence servers
+  mb coherence fluffcontentcaches  # Execute a fluffcontentcaches on the core coherence servers
+  mb coherence fluffindexes        # Execute a fluffindexes on the core coherence servers
+  mb coherence help [COMMAND]      # Describe subcommands or one specific subcommand
+  mb coherence nodes               # List all nodes grouped by Group
+  mb coherence start               # Execute a start on the core coherence servers
+  mb coherence stop                # Execute a stop on the core coherence servers
+```
+
+Again there is the motherbrain provided `nodes` command which will provide nodes that are a part of this component.
+
+Other than `help`, the rest of commands are generated from the plugin and have a description that describe what each command does and will be unique to each component. For example, here are the tomcat commands:
+
+```
+mb pvpnet_core tomcat --environment NA_production
+using pvpnet_core (2.3.189)
+
+Commands:
+  mb tomcat help [COMMAND]  # Describe subcommands or one specific subcommand
+  mb tomcat nodes           # List all nodes grouped by Group
+  mb tomcat start           # Start the pvpnet_core platform
+  mb tomcat stop            # Stop the pvpnet_core platform
+```
+
+CAVEAT: The mb help output is not displaying the plugin name. This is a quirk of the command line tool framework motherbrain uses under the hood to generate commands.
+
+## motherbrain Flow
+
+### `mb pvpnet_core tomcat nodes --environment NA_production`
+
+This fetches the nodes associated with the component `tomcat` in the environment "NA_production"
+
+* Gather the run list entries specified by the **groups** defined in the component `tomcat`
+* Do a node search on the chef server using a string compiled from all of the run list entries inside the **groups** (the search is also constrained by the environment specified)
+* Print a list of the nodes that are associated with this component as returned by the search
+
+### `mb pvpnet_core tomcat start --environment NA_production`
+
+This runs the start command on the component tomcat
+
+* Load the "start" command in the pvpnet_core plugin
+* Based on how the plugin is implemented, perform actions on various **groups** defined on the `tomcat` component
+* Perform attribute modifications and other steps defined in the start action. Most actions are a simple attribute toggle on the environment, for example, a `tomcat` start might look like this:
+
+```ruby
+node_attribute "pvpnet_core.tomcat.service_state", "start"
+```
+
+This sets the attribute service_state attribute (under pvpnet_core.tomcat) to "start", which will cause the next chef-client run on the box to ensure that the service is running because the recipe looks something like this:
+
+```ruby
+service 'tomcat' do
+  action node[:pvpnet_core][:tomcat][:service_state]
+end
+```
+
+* After the attributes are all set, kick off a chef-client run on the nodes that match a search for nodes matching the component's groups that the command says to run on. Commands specify groups like this:
+
+```ruby
+command 'start' do
+  on("tomcat_servers") do 
+    service("tomcat").run(:start)
+  end
+end
+```
+
+This says, "on the `tomcat` servers, run the action start from the service `tomcat`".  In this case, the group `tomcat_servers` is what motherbrain will be searching for when it finds the nodes that need a `chef-client` run started. Remember that a group is defined by the run list entries in the plugin file.
+
+* When all of the nodes' `chef-client` runs complete, motherbrain will report success or failure based on those runs.
+
+### `mb pvpnet_core provision provision_manifest.json \`
+### `--environment my_new_environment`
+
+This creates a new environment using a VM provider such as Eucalyptus or AWS and bootstraps the boxes into a new Chef environment after they are available using the environment name provided.
+
+* motherbrain reads the provision_manifest.json to figure out what boxes are required for the new environment.  The provision_manifest.json looks like this:
+
+```json
+{
+  "nodes": [
+    {
+      "groups": [ "pvpnet_core::feapp", "pvpnet_core::beapp" ],
+      "type": "m1.large",
+      "count": 2
+    },
+    {
+      "groups": ["pvpnet_core::database"],
+      "type": "m1.large",
+      "count": 1
+    }
+  ]
+}
+```
+
+With the Eucalyptus provisioner, this will create 3 new m1.large nodes, install boostrap Chef onto the boxes with the runlists provided by the groups' definitions in the plugin.
+
+* Next, a `mb bootstrap` is run on the nodes to install chef and run `chef-client` with the proper run list (see the `mb bootstrap` section) 
+
+### `mb bootstrap bootstrap_manifest.json \`
+###`--environment my_existing_environment`
+
+Provided the nodes exist already, a bootstrap is required to get the nodes placed in the correct environment and capable of running `chef-client`.  The bootstrap_manifest.json looks like this:
+
+```json
+{
+  "nodes": [
+    {
+      "groups": [
+        "platform_database::platform",
+        "platform_database::platform_audit",
+        "platform_database::platform_csr",
+        "platform_database::platform_stats",
+        "platform_database::platform_kudos",
+        "platform_content::content"
+      ],
+      "hosts": ["10.11.12.40"]
+    },
+    {
+      "groups": ["coherence::coherence-app"],
+      "hosts": ["10.11.12.1", "10.11.12.2", "10.11.12.3"]
+    },
+    {
+      "groups": [
+        "tomcat::beapp",
+        "tomcat::feapp",
+        "tomcat::game_allocation",
+        "tomcat::matchmaking",
+        "tomcat::csr_services",
+        "tomcat::login_queue",
+        "tomcat::platform_auth"
+      ],
+      "hosts": ["10.11.12.50"]
+    }
+  ]
+}
+```
+
+When the bootstrap command is run, the run lists from the groups in each node section of the json are applied to the hosts in the array in that same section. Chef will be installed if it needs to be installed and then motherbrain will run `chef-client` on all of the nodes.
+

data/PLUGINS.md

@@ -0,0 +1,268 @@
+# Plugin DSL
+
+# `component`
+
+Components are logical groupings of elements that make up an application.
+
+```ruby
+component "webserver" do
+  description "My webserver that serves webs"
+end
+```
+
+## `description`
+
+Provides text to be displayed in the help output generated for the component
+
+```ruby
+component "webserver" do
+  description "serves the myface PHP web application"
+end
+```
+
+## `versioned`
+
+Declares that the component is versioned.
+
+```ruby
+component "webserver" do
+  versioned
+end
+```
+
+This will default to the 'webserver.version' attribute, seen in recipes as `node[:webserver][:version]`. Declaring this will allow you to use the `mb plugin upgrade` command:
+
+```sh
+mb myface upgrade --components webserver:1.2.3
+```
+
+You can also specify a custom version attribute:
+
+```ruby
+component "webserver" do
+  versioned_with "web"
+end
+```
+
+THis would use the attribute 'web', seen in recipes as `node[:web]`.
+
+## `command`
+
+Defines a command to be added to the mb cli generated for the plugin.
+
+```ruby
+  command "start" do
+    description "Start the web server"
+    execute do
+      on("default") do
+        service("apache").run(:start)
+      end
+    end
+  end
+```
+
+### `description`
+
+Provides text to be displayed in the help output generated for the plugin
+
+```ruby
+command "start" do
+  description "Start the web server"
+  # ...
+end
+```
+
+### `execute`
+
+Execute provides a place to define which actions are to be run during a command's run
+
+```ruby
+command "start" do
+  execute do
+    on("default") do
+      service("server").run(:start)
+    end
+  end
+end
+```
+
+#### `on`
+
+`on` specifies a group to perform the actions contained in the block on. These actions will be performed in parallel. If actions need to be performed in sequence, use multiple `on` blocks.
+
+Invocation of actions in `on` blocks follows this syntax: service("service_name").run(:action_name)
+
+```ruby
+command "start" do
+  execute do
+    on("default") do
+      service("server").run(:start)
+    end
+  end
+end
+```
+
+**`service`**
+
+`service` is used to invoke actions in a `command` `on` block. See `on` for more details
+
+```ruby
+on("default") do
+  service("server").run(:start)
+end
+```
+
+**`any`**
+
+To run a command only on portion of the nodes:
+
+```ruby
+on("default", any: 2) do
+  ...
+end
+```
+
+The nodes will be chosen at random.
+
+**`max_concurrent`**
+
+To run a command on all nodes, but limit how many are running at once:
+
+```ruby
+on("default", max_concurrent: 2) do
+  ...
+end
+```
+
+## `group`
+
+Groups allow motherbrain to identify nodes on which `actions` are taken.
+
+```ruby
+component "webserver" do
+  description "My webserver that serves webs"
+
+  group "default" do
+    recipe "myface::webserver"
+  end
+end
+```
+
+Also supported
+
+```ruby
+  group "default" do
+    role "webserver"
+  end
+```
+```ruby
+  group "default" do
+    attribute "activemq.master", true
+  end
+```
+
+### `recipe`
+
+Used in a `group` to identify a server by a recipe entry on its runlist
+
+```ruby
+  group "default" do
+    recipe "myface::webserver"
+  end
+```
+
+### `role`
+
+Used in a `group` to identify a server by a role entry on its runlist
+
+```ruby
+  group "default" do
+    role "webserver"
+  end
+```
+
+### `attribute`
+
+Used in a `group` to identify a server by a node attribute
+
+```ruby
+  group "default" do
+    attribute "activemq.master", true
+  end
+```
+
+
+
+## `service`
+
+Services are defined to represent the running processes on a given node that make up a component.
+
+```ruby
+component "webserver" do
+  # ...
+  service "apache" do
+    # ...
+  end
+end
+```
+
+### `action`
+
+Actions provide a way of interacting with the chef server to change the state of a service. Following the block, `chef-client` will be run on the nodes matched by the `group`.
+
+```ruby
+service "apache" do
+  action :start do
+    node_attribute 'myface.apache.enable', true
+    node_attribute 'myface.apache.start', true
+  end
+
+  action :stop do
+    node_attribute 'myface.apache.enable', false
+    node_attribute 'myface.apache.start', false
+  end
+end
+```
+
+#### `node_attribute`
+
+Used in an `action` to specify the value a node attribute should be set to
+
+```ruby
+action :start do
+  node_attribute 'myface.apache.enable', true
+  node_attribute 'myface.apache.start', true
+end
+```
+
+**`toggle`**
+
+To toggle an attribute for just this Chef run:
+
+```ruby
+action :start do
+  node_attribute 'myface.apache.restart', true, toggle: true
+end
+```
+
+The attribute will be set to true, and then set back to its original value after the Chef run.
+
+# `stack_order`
+
+A stack order specifies the order in which a bootstrap or Chef run occurs.
+
+```ruby
+stack_order do
+  bootstrap("webserver::default")
+end
+```
+
+## `bootstrap`
+
+Used in a `stack_order` block to specify what `component` and `group` should be bootstrapped during a provision
+
+```ruby
+stack_order do
+  bootstrap("webserver::default")
+end
+```