lobot 0.10.3 → 1.0.pre

data/chef/travis-cookbooks/ci_environment/mongodb/templates/default/mongodb.list.erb

@@ -0,0 +1 @@
+deb http://downloads.mongodb.org/distros/<%= node[:platform] %> <%= node[:platform_version].gsub(/\.0*/, '.') %> 10gen

data/chef/travis-cookbooks/ci_environment/mysql/README.md

@@ -0,0 +1,141 @@
+Description
+===========
+
+Installs and configures MySQL client or server.
+
+Changes
+=======
+
+### v1.0.6:
+
+* [COOK-605] install mysql-client package on ubuntu/debian
+
+### v1.0.5:
+
+* [COOK-465] allow optional remote root connections to mysql
+* [COOK-455] improve platform version handling
+* externalize conf_dir attribute for easier cross platform support
+* change datadir attribute to data_dir for consistency
+
+### v1.0.4:
+
+* fix regressions on debian platform
+* [COOK-578] wrap root password in quotes
+* [COOK-562] expose all tunables in my.cnf
+
+
+Requirements
+============
+
+Platform
+--------
+
+* Debian, Ubuntu
+* CentOS, Red Hat, Fedora
+
+Tested on:
+
+* Debian 5.0
+* Ubuntu 10.04
+* CentOS 5.5
+
+Cookbooks
+---------
+
+Requires Opscode's openssl cookbook for secure password generation.
+
+Requires a C compiler and Ruby development package in order to build mysql gem with native extensions. On Debian and Ubuntu systems this is satisfied by installing the "build-essential" and "ruby-dev" packages before running Chef. See USAGE below for information on how to handle this during a Chef run.
+
+Resources and Providers
+=======================
+
+The cookbook contains a LWRP, `mysql_database` which can be used to manage databases through calls to the MySQL API. The mysql gem is installed to make this usable. The provider currently supports three actions:
+
+* `flush_tables_with_read_lock` - sends the sql command "flush tables with read lock", used for setting up mysql master/slave replication.
+* `unflush_tables` - sends the sql command "unflush tables", used for setting up master/slave replication.
+* `create_db` - specify a database to be created.
+* `query` - send an arbitrary query to the database, this should be used with care. Pass the SQL statement to use with the `sql` resource attribute.
+
+For example see the USAGE section below.
+
+Attributes
+==========
+
+* `mysql['server_root_password']` - Set the server's root password with this, default is a randomly generated password with `OpenSSL::Random.random_bytes`.
+* `mysql['server_repl_password']` - Set the replication user 'repl' password with this, default is a randomly generated password with `OpenSSL::Random.random_bytes`.
+* `mysql['server_debian_password']` - Set the debian-sys-maint user password with this, default is a randomly generated password with `OpenSSL::Random.random_bytes`.
+* `mysql['bind_address']` - Listen address for MySQLd, default is node's ipaddress.
+* `mysql['data_dir']` - Location for mysql data directory, default is "/var/lib/mysql"
+* `mysql['conf_dir']` - Location for mysql conf directory, default is "/etc/mysql"
+* `mysql['ec2_path']` - location of mysql data_dir on EC2 nodes, default "/mnt/mysql"
+
+Performance tuning attributes, each corresponds to the same-named parameter in my.cnf; default values listed
+
+* `mysql['tunable']['key_buffer']`          = "250M"
+* `mysql['tunable']['max_connections']`     = "800"
+* `mysql['tunable']['wait_timeout']`        = "180"
+* `mysql['tunable']['net_write_timeout']`   = "30"
+* `mysql['tunable']['net_write_timeout']`   = "30"
+* `mysql['tunable']['back_log']`            = "128"
+* `mysql['tunable']['table_cache']`         = "128"
+* `mysql['tunable']['max_heap_table_size']` = "32M"
+
+Usage
+=====
+
+On client nodes,
+
+    include_recipe "mysql::client"
+
+This will install the MySQL client libraries and development headers on the system. It will also install the Ruby Gem `mysql`, so that the cookbook's LWRP (above) can be used. This is done during the compile-phase of the Chef run. On platforms that are known to have a native package (currently Debian, Ubuntu, Red hat, Centos, Fedora and SUSE), the package will be installed. Other platforms will use the RubyGem.
+
+This creates a resource object for the package and does the installation before other recipes are parsed. You'll need to have the C compiler and such (ie, build-essential on Ubuntu) before running the recipes, but we already do that when installing Chef :-). If you want to be able to access a MySQL database via Ruby within another recipe, you could do so, like so:
+
+    mysql_database "create application_production database" do
+      host "localhost"
+      username "root"
+      password node[:mysql][:server_root_password]
+      database "application_production"
+      action :create_db
+    end
+
+This will connect to the MySQL server running on localhost as "root" and password as `mysql[:server_root_password]` attribute (see below) and create the database specified with the `database` parameter. The provider will attempt to determine whether the database exists first.
+
+On server nodes,
+
+    include_recipe "mysql::server"
+
+On Debian and Ubuntu, this will preseed the mysql-server package with the randomly generated root password from the attributes file. On other platforms, it simply installs the required packages. It will also create an SQL file, /etc/mysql/grants.sql, that will be used to set up grants for the root, repl and debian-sys-maint users.
+
+On EC2 nodes,
+
+    include_recipe "mysql::server_ec2"
+
+When the `ec2_path` doesn't exist we look for a mounted filesystem (eg, EBS) and move the data_dir there.
+
+The client recipe is already included by server and 'default' recipes.
+
+For more infromation on the compile vs execution phase of a Chef run:
+
+* http://wiki.opscode.com/display/chef/Anatomy+of+a+Chef+Run
+
+License and Author
+==================
+
+Author:: Joshua Timberman (<joshua@opscode.com>)
+Author:: AJ Christensen (<aj@opscode.com>)
+Author:: Seth Chisamore (<schisamo@opscode.com>)
+
+Copyright:: 2009-2011 Opscode, Inc
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/chef/travis-cookbooks/ci_environment/mysql/attributes/default.rb

@@ -0,0 +1,3 @@
+default[:mysql] = {
+  :arch => (kernel['machine'] =~ /x86_64/ ? "amd64" : "i386")
+}

data/chef/travis-cookbooks/ci_environment/mysql/attributes/server.rb

@@ -0,0 +1,69 @@
+#
+# Cookbook Name:: mysql
+# Attributes:: server
+#
+# Copyright 2008-2009, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+default['mysql']['bind_address']              = "0.0.0.0"
+default['mysql']['data_dir']                   = "/var/lib/mysql"
+
+case node["platform"]
+when "centos", "redhat", "fedora", "suse"
+  set['mysql']['conf_dir']                    = '/etc'
+  set['mysql']['socket']                      = "/var/lib/mysql/mysql.sock"
+  set['mysql']['pid_file']                    = "/var/run/mysqld/mysqld.pid"
+  set['mysql']['old_passwords']               = 1
+else
+  set['mysql']['conf_dir']                    = '/etc/mysql'
+  set['mysql']['socket']                      = "/var/run/mysqld/mysqld.sock"
+  set['mysql']['pid_file']                    = "/var/run/mysqld/mysqld.pid"
+  set['mysql']['old_passwords']               = 0
+end
+
+# yes, for the CI environment, empty password is a good idea. VM is rolled back after eack run anyway.
+default['mysql']['server_root_password']   = ""
+default['mysql']['server_debian_password'] = ""
+default['mysql']['server_repl_password']   = ""
+
+default['mysql']['ci_user_name']           = "travis"
+default['mysql']['ci_user_password']       = ""
+
+
+
+default['mysql']['allow_remote_root']               = false
+default['mysql']['tunable']['back_log']             = "128"
+default['mysql']['tunable']['key_buffer']           = "256M"
+default['mysql']['tunable']['max_allowed_packet']   = "16M"
+default['mysql']['tunable']['max_connections']      = "800"
+default['mysql']['tunable']['max_heap_table_size']  = "32M"
+default['mysql']['tunable']['myisam_recover']       = "BACKUP"
+default['mysql']['tunable']['net_read_timeout']     = "30"
+default['mysql']['tunable']['net_write_timeout']    = "30"
+default['mysql']['tunable']['table_cache']          = "128"
+default['mysql']['tunable']['table_open_cache']     = "128"
+default['mysql']['tunable']['thread_cache']         = "128"
+default['mysql']['tunable']['thread_cache_size']    = 8
+default['mysql']['tunable']['thread_concurrency']   = 10
+default['mysql']['tunable']['thread_stack']         = "256K"
+default['mysql']['tunable']['wait_timeout']         = "180"
+
+default['mysql']['tunable']['query_cache_limit']    = "1M"
+default['mysql']['tunable']['query_cache_size']     = "16M"
+
+default['mysql']['tunable']['log_slow_queries']     = "/var/log/mysql/slow.log"
+default['mysql']['tunable']['long_query_time']      = 2
+
+default['mysql']['tunable']['innodb_buffer_pool_size']  = "256M"

data/chef/travis-cookbooks/ci_environment/mysql/attributes/server_on_ramfs.rb

@@ -0,0 +1 @@
+include_attribute "mysql::server"

data/chef/travis-cookbooks/ci_environment/mysql/libraries/database.rb

@@ -0,0 +1,19 @@
+begin
+  require 'mysql'
+rescue LoadError
+  Chef::Log.info("Missing gem 'mysql'")
+end
+
+module Opscode
+  module Mysql
+    module Database
+      def db
+        @db ||= ::Mysql.new new_resource.host, new_resource.username, new_resource.password
+      end
+      def close
+        @db.close rescue nil
+        @db = nil
+      end
+    end
+  end
+end

data/chef/travis-cookbooks/ci_environment/mysql/libraries/helpers.rb

@@ -0,0 +1,33 @@
+#
+# Author:: Seth Chisamore (<schisamo@opscode.com>)
+# Copyright:: Copyright (c) 2011 Opscode, Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+module Opscode
+  module Mysql
+    module Helpers
+
+      def debian_before_squeeze?
+        platform?("debian") && (node.platform_version.to_f < 6.0)
+      end
+
+      def ubuntu_before_lucid?
+        platform?("ubuntu") && (node.platform_version.to_f < 10.0)
+      end
+
+    end
+  end
+end

data/chef/travis-cookbooks/ci_environment/mysql/metadata.rb

@@ -0,0 +1,76 @@
+maintainer        "Opscode, Inc."
+maintainer_email  "cookbooks@opscode.com"
+license           "Apache 2.0"
+description       "Installs and configures mysql for client or server"
+long_description  IO.read(File.join(File.dirname(__FILE__), 'README.md'))
+version           "1.1.0"
+recipe            "mysql",         "Includes the client recipe to configure a client"
+recipe            "mysql::client", "Installs packages required for mysql clients using run_action magic"
+recipe            "mysql::server", "Installs packages required for mysql servers w/o manual intervention"
+
+%w{ debian ubuntu centos suse fedora redhat }.each do |os|
+  supports os
+end
+
+depends "openssl"
+
+attribute "mysql/server_root_password",
+  :display_name => "MySQL Server Root Password",
+  :description => "Randomly generated password for the mysqld root user",
+  :default => "randomly generated"
+
+attribute "mysql/bind_address",
+  :display_name => "MySQL Bind Address",
+  :description => "Address that mysqld should listen on",
+  :default => "ipaddress"
+
+attribute "mysql/data_dir",
+  :display_name => "MySQL Data Directory",
+  :description => "Location of mysql databases",
+  :default => "/var/lib/mysql"
+
+attribute "mysql/conf_dir",
+  :display_name => "MySQL Conf Directory",
+  :description => "Location of mysql conf files",
+  :default => "/etc/mysql"
+
+attribute "mysql/tunable",
+  :display_name => "MySQL Tunables",
+  :description => "Hash of MySQL tunable attributes",
+  :type => "hash"
+
+attribute "mysql/tunable/key_buffer",
+  :display_name => "MySQL Tuntable Key Buffer",
+  :default => "250M"
+
+attribute "mysql/tunable/max_connections",
+  :display_name => "MySQL Tunable Max Connections",
+  :default => "800"
+
+attribute "mysql/tunable/wait_timeout",
+  :display_name => "MySQL Tunable Wait Timeout",
+  :default => "180"
+
+attribute "mysql/tunable/net_read_timeout",
+  :display_name => "MySQL Tunable Net Read Timeout",
+  :default => "30"
+
+attribute "mysql/tunable/net_write_timeout",
+  :display_name => "MySQL Tunable Net Write Timeout",
+  :default => "30"
+
+attribute "mysql/tunable/back_log",
+  :display_name => "MySQL Tunable Back Log",
+  :default => "128"
+
+attribute "mysql/tunable/table_cache",
+  :display_name => "MySQL Tunable Table Cache for MySQL < 5.1.3",
+  :default => "128"
+
+attribute "mysql/tunable/table_open_cache",
+  :display_name => "MySQL Tunable Table Cache for MySQL >= 5.1.3",
+  :default => "128"
+
+attribute "mysql/tunable/max_heap_table_size",
+  :display_name => "MySQL Tunable Max Heap Table Size",
+  :default => "32M"

data/chef/travis-cookbooks/ci_environment/mysql/providers/database.rb

@@ -0,0 +1,81 @@
+#
+# Cookbook Name:: mysql
+# Provider:: database
+#
+# Copyright:: 2008-2011, Opscode, Inc <legal@opscode.com>
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+include Opscode::Mysql::Database
+
+action :flush_tables_with_read_lock do
+  if exists?
+    begin
+      Chef::Log.info "mysql_database: flushing tables with read lock"
+      db.query "flush tables with read lock"
+      new_resource.updated_by_last_action(true)
+    ensure
+      db.close
+    end
+  end
+end
+
+action :unflush_tables do
+  if exists?
+    begin
+      Chef::Log.info "mysql_database: unlocking tables"
+      db.query "unlock tables"
+      new_resource.updated_by_last_action(true)
+    ensure
+      db.close
+    end
+  end
+end
+
+action :create_db do
+  unless exists?
+    begin
+      Chef::Log.info "mysql_database: Creating database #{new_resource.database}"
+      db.query("create database #{new_resource.database}")
+      new_resource.updated_by_last_action(true)
+    ensure
+      db.close
+    end
+  end
+end
+
+action :query do
+  if exists?
+    begin
+      Chef::Log.info "mysql_database: Performing Query: #{new_resource.sql}"
+      db.query(new_resource.sql)
+      new_resource.updated_by_last_action(true)
+    ensure
+      db.close
+    end
+  end
+end
+
+def load_current_resource
+  Gem.clear_paths
+  require 'mysql'
+
+  @mysqldb = Chef::Resource::MysqlDatabase.new(new_resource.name)
+  @mysqldb.database(new_resource.database)
+end
+
+private
+def exists?
+  db.list_dbs.include?(new_resource.database)
+end

data/chef/travis-cookbooks/ci_environment/mysql/recipes/client.rb

@@ -0,0 +1,60 @@
+#
+# Cookbook Name:: mysql
+# Recipe:: client
+#
+# Copyright 2008-2011, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+::Chef::Resource::Package.send(:include, Opscode::Mysql::Helpers)
+
+package "mysql-client" do
+  package_name value_for_platform(
+    [ "centos", "redhat", "suse", "fedora"] => { "default" => "mysql" },
+    "default" => "mysql-client"
+  )
+  action :install
+end
+
+package "mysql-devel" do
+  package_name begin
+    if platform?(%w{ centos redhat suse fedora })
+      "mysql-devel"
+    elsif debian_before_squeeze? || ubuntu_before_lucid?
+      "libmysqlclient15-dev"
+    else
+      "libmysqlclient-dev"
+    end
+  end
+  action :install
+end
+
+if platform?(%w{ debian ubuntu redhat centos fedora suse })
+
+  package "mysql-ruby" do
+    package_name value_for_platform(
+      [ "centos", "redhat", "suse", "fedora"] => { "default" => "ruby-mysql" },
+      ["debian", "ubuntu"] => { "default" => 'libmysql-ruby' },
+      "default" => 'libmysql-ruby'
+    )
+    action :install
+  end
+
+else
+
+  gem_package "mysql" do
+    action :install
+    options '--no-ri --no-rdoc'
+  end
+end