machines 0.5.4 → 0.5.6

data/.yardopts

@@ -1,2 +1,8 @@
 --markup-provider=redcarpet
---markup=markdown
+--markup=markdown
+-
+CHANGELOG.md
+EXAMPLES.md
+INSTALL.md
+TODO.md
+LICENSE

data/CHANGELOG.md

@@ -1,22 +1,34 @@
-== 0.5.4
+## 0.5.6
+
+* Gem updates - spec_helper updated to handle features removed from minitest
+* Fixed bug introducted in API cleanup where commands were not included in Core
+
+## 0.5.5
+
+* Cleaned up API - moved all commands into Commands module and turned non-commands into classes
+* Switched to dwm and slim for default template window manager/login manager
+* Added make clean install to `install` command
+* BUG: fixed incorrect link command in timezone package
+
+## 0.5.4
 
 * Added support extracting for bz2 archives
 * `install` can now download an archive and extract to /usr/local/lib and create a link in /usr/local/bin
 
 
-== 0.5.3
+## 0.5.3
 
 * Implement `list` command to list available machines from machines.yml
 * Display syntax when `build` called with no machine
 * Support running multiple tasks - e.g. `machines build machine passenger passenger_nginx nginx webapps`
 * modify git_clone to pull if the repo already exists
 
-== 0.5.2
+## 0.5.2
 
 * Bug fixes
 
 
-== 0.5.1
+## 0.5.1
 
 * Added opensource flag to allow alternatives to be installed (e.g. Chrome/Chromium)
 * Added config.yml option to autostart VBoxClient for WMs that don't use XDG autostart

data/INSTALL.md

@@ -1,3 +1,5 @@
+**If installing to VirtualBox on a Windows host read on.**
+
 Install freeSSHd
 ----------------------------------------
 
@@ -9,6 +11,7 @@ Install freeSSHd
 ** User can use: Shell
 * Copy the public rsa key from your Ubuntu VM and save it in `C:\Program Files (x86)\freeSSHd\vbuser` with no extension
 
+
 Install VM
 ----------------------------------------
 

data/LICENSE

@@ -1,6 +1,6 @@
 (The MIT License)
 
-Copyright (c) 2009, 2010, 2011, 2012 Phil Thompson
+Copyright (c) 2009, 2010, 2011, 2012, 2013 Phil Thompson
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the
@@ -20,4 +20,3 @@ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
 CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
 TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
 SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-

data/README.md

@@ -5,7 +5,7 @@ Machines
 
 Setup Ubuntu development and server **Machines** locally or in the cloud for developing and hosting Ruby, Rails and related environments.
 
-Run commands like:
+Write commands in Ruby like:
 
     sudo install %w(build-essential zlib1g-dev libpcre3-dev)
     sudo write "127.0.1.1\t#{$conf.hostname}", :to => '/etc/hosts'
@@ -13,34 +13,38 @@ Run commands like:
     run download $conf.nginx.url
     run create_from 'nginx/nginx.conf.erb', :to => File.join($conf.nginx.path, 'conf', 'nginx.conf')
 
-Upgrade passenger:
+Example to upgrade passenger:
 
     machines build phil_workstation passenger_nginx
 
+[RubyDoc.info documentation](http://rdoc.info/github/PhilT/machines/)
 
 Status
 -----------------------------------------------------------
 
-September 2012
+**March 2013**
 
-* Released 0.5.2 gem
+* Released 0.5.4 gem
 * Working development and server builds.
-* Cloud deployments to complete.
+* Cloud deployments, cleaner API and some tidy up in development.
+
 
 Features
 -----------------------------------------------------------
 
 * An opinionated Ubuntu configuration script with sensible defaults
 * Easily override the defaults with configuration options and custom ruby
-* Supports several cloud services [Not complete]
 * Default template supports Nginx, Passenger, Ruby, Rails, MySQL, Git, Monit, Logrotate
 * Preconfigured Ruby & Rails light development environment (Openbox or Subtle)
 * Bring up new instances fully configured in less than 15 minutes
+* The best form of documentation for your architecture
+
 
 What it's Not
 -----------------------------------------------------------
 
-* It does not replace the base installation (Yet). A minimal Ubuntu must be installed prior to running the *Machines* install
+* It does not replace the base installation (Yet). A minimal Ubuntu must be installed prior to running the *Machines* install (instructions provided below)
+* It's not a full blown configuration management tool such as Puppet or Chef although it can be used to run packages on the local machine to install new versions
 
 
 Motivation
@@ -52,10 +56,11 @@ Configuration management is a complex topic. I wanted to reduce some of the vari
 Overview
 -----------------------------------------------------------
 
-The top level script is the `Machinesfile`. This contains the packages to include. Packages contain the commands to run. Default packages are provided by Machines. Default packages can be overridden and new ones created.
+The top level script is the `Machinesfile`. This contains a list of packages to run. Packages contain one or more tasks and the tasks contain the commands to run.  Default packages are provided by Machines. Default packages can be overridden and new ones created.
 
 Commands are added to a queue with `sudo` or `run`. [lib/packages](https://github.com/PhilT/machines/tree/master/lib/packages) contains the packages you can add in the `Machinesfile` in Machines. Once the build starts the commands are run and shown with the current progress.
 
+
 Installation and Configuration
 -----------------------------------------------------------
 
@@ -63,26 +68,23 @@ Installation and Configuration
 
     gem install machines
 
-### Generate an example build script
+### Generate an example project
 
     machines new example
 
-Creates the `example` folder and copies in an example template.
-
 ### Configure your deployment
 
 Take a look at the generated project. It contains several folders with templates and
 configuration settings for various programs, your `Machinesfile` and the various `.yml` files.
 
-`machines.yml` is your machine architecture. Here you'll add all the computers in your environment.
+* `machines.yml` is your architecture. Here you'll add all the computers in your setup.
+* `webapps.yml` contains the web applications you develop and maintain.
+* `config.yml` contains settings for various packages. Versions, Cloud setup, paths, etc.
+* `users/` contains user specific preferences, dotfiles, etc
 
-`webapps.yml` lists the web applications you develop and maintain.
+So here is the recommended approach to configuring your environment:
 
-`config.yml` contains settings for various packages. Version numbers, Cloud configuration, paths, etc.
-
-`users/` contains user specific preferences, dotfiles, etc
-
-* Create your architecture in `machines.yml`
+* Create your architecture in `machines.yml` (see notes in file)
 * Add you webapps in `webapps.yml`
 * Edit the build script (`Machinesfile`)
 * Add your websites certificates and amazon private key (if required)
@@ -109,13 +111,11 @@ configuration settings for various programs, your `Machinesfile` and the various
     machines dryrun <machine>
     cat log/output.log
 
-
 ### Check the machine
 
     ssh-keygen -R <host ip> # remove host from known_hosts file (handy when testing)
     $ ssh <IP ADDRESS>      # Make sure you can connect to the machine
 
-
 ### Build the machine
 
     $ machines build <machine>
@@ -131,7 +131,11 @@ Console output:
 
 While running open another terminal to view detailed output:
 
-    tail -f output.log
+    tail -f log/output.log
+
+or debug:
+
+    tail -f log/debug.log
 
 
 Commandline Options
@@ -169,6 +173,19 @@ Some of the settings set and used by Machines are:
 
 Take a look at `template/*.yml` for more.
 
+Commands
+-----------------------------------------------------------
+
+Commands you can use with `run` or `sudo` are in the following modules (links to rdoc):
+
+* [Configuration](http://rdoc.info/github/PhilT/machines/Machines/Commands/Configuration)
+* [Database](http://rdoc.info/github/PhilT/machines/Machines/Commands/Database)
+* [FileOperations](http://rdoc.info/github/PhilT/machines/Machines/Commands/FileOperations)
+* [Installation](http://rdoc.info/github/PhilT/machines/Machines/Commands/Installation)
+* [Services](http://rdoc.info/github/PhilT/machines/Machines/Commands/Services)
+
+When creating your own commands the [Checks](http://rdoc.info/github/PhilT/machines/Machines/Commands/Checks) module should be useful.
+
 
 Setting up a test VM
 -----------------------------------------------------------
@@ -204,11 +221,12 @@ Make sure you've downloaded one of the ISOs from the list in *Prepare the target
 What's happening under the hood
 -----------------------------------------------------------
 
-* An ssh connection is established to send all commands and uploads (similar to Capistrano)
-* Ssh uses the specified user and then sudo is added to commands that require it
-* When sudo is needed for file uploads. The file is uploaded to /tmp then sudo cp'd to the destination
+* An SSH connection is established to send all commands and uploads (similar to Capistrano) to the target machine
+* SSH uses the specified user and then sudo is added to commands that require it
+* When sudo is needed for file uploads. The file is uploaded to `/tmp` then `sudo cp`'d to the destination
 * When `package` is called in the `Machinesfile` that file is loaded either from the projects packages folder
   or from the Machines packages if not found in the project
+* It is executed within the context of the Core class. This class defines basic commands such as `task`, `run`, `sudo` and includes all the Commands modules so they are available to the running package
 
 
 Limitations
@@ -218,8 +236,9 @@ Limitations
 * Servers use www (by default) for nginx/apache, passenger and deployments
 * The system has been designed to allow a certain flexibility in the configuration although some things
   may not yet be totally configurable it should be possible to add or modify the relevant package
-* We are currently focused on Ruby 1.9.2 (Moving to 1.9.3 soon), Rails 3 and Passenger 3
+* We are currently focused on Ruby 1.9.3 (Moving to 2.0 soon), Rails 3 and Passenger 3
 * Some commands may not properly escape quotes when used with sudo (e.g. append and replace). This may be addressed in a future release
+* Tasks not configured to run for a particular setup will not be available to run explicitly from the commandline
 
 
 Development, Patches, Pull Requests
@@ -229,8 +248,8 @@ Development, Patches, Pull Requests
 * Test drive your feature addition or bug fix
 * Commit, do not mess with Rakefile, version, or history
 * Send me a pull request. Please use topic branches
-* Feel free to add/enhance packages and submit pull requests
-* Package tests are a bit of a pain but do catch a lot of potential issues
+* Feel free to add, enhance or update packages and submit pull requests
+* Package tests are a bit of a pain but do catch a lot of potential issues so please add these
 
 
 References
@@ -276,5 +295,5 @@ Thanks to all the people that published the hundreds of articles, blog posts and
 Copyright
 -----------------------------------------------------------
 
-Copyright (c) 2010, 2011, 2012 Phil Thompson. See LICENSE for details.
+Copyright (c) 2010, 2011, 2012, 2013 Phil Thompson. See LICENSE for details.
 

data/Rakefile

@@ -1,7 +1,6 @@
 require 'bundler'
 require 'yard'
 require 'highline/import'
-require 'rake'
 require 'rake/testtask'
 
 task :default => [:coverage, :yard, :install]
@@ -41,13 +40,6 @@ task :install do
   end
 end
 
-desc 'Run machines'
-task :run do
-  $LOAD_PATH << 'lib'
-  require 'machines'
-  Machines::Base.new.start(ARGV[1])
-end
-
 desc 'Git tags and sends the gem to rubygems'
 task :release do
   gemspec_path = Dir['*.gemspec'].first

data/TODO.md

@@ -1,23 +1,36 @@
 TODO next
 ----------------------------------------
 
-Handle unknown tasks and display a proper error message (at the moment we get an obscure undefined method on NilClass)
-Support missing environment (e.g. source repo machine)
-`machines dryrun/build` with no machine name should list the machines available
-DRY up per user config by creating a "common" user config that all users pull default config from
-DRY up further by having default templates in the same way that packages default to built-in ones
+* Move dotfiles into separate repo (managed by user)
+* stubs :required_options should be tested
+* replace subtle window manager with dwm
+* reorg packages into folders by category
+
+EC2 support
+-----------
+
+* Encrypt keys/passwords etc
+
+
+* Handle unknown tasks and display a proper error message (at the moment we get an obscure undefined method on NilClass)
+* Support missing environment (e.g. source repo machine)
+* `machines dryrun/build` with no machine name should list the machines available
+* DRY up per user config by creating a "common" user config that all users pull default config from
+* DRY up further by having default templates in the same way that packages default to built-in ones
   (In other words remove as much as possible from evmachines)
-Complement this by providing a command to view packages and templates
-Move dotfiles to a repo so they can be managed across projects
-Handle apt-get error 110 and retry
-Move packages into a separate gem
-Move template into a separate gem
+* Complement this by providing a command to view packages and templates
+* Move dotfiles to a repo so they can be managed across projects
+* Handle apt-get error 110 and retry
+* Move packages into a separate gem
+* Move template into a separate gem
+* Remove questions API (once passwords are encrypted we just ask for encryption password, similar to `pw`)
 
 
 Cloud
 ----------------------------------------
-Use EC2 IP address for connecting to database servers
-Create elastic IP address for web servers
+
+* Use EC2 IP address for connecting to database servers
+* Create elastic IP address for web servers
 
 * Assign private/public keys
 * Create security groups - check they exist and modify or create as required
@@ -37,58 +50,52 @@ EC2 - Look at assigning and freeing elastic IP addresses
 Package and Task Tasks
 ----------------------------------------
 
-Any methods other than `append` that cannot be repeatedly run?
-Display additional install notes for a particular package (at the end of installation) - e.g. printer setup requires Windows share to be setup
-CODE/DOC: Describe difference between package and task or merge packages with tasks if possible
-BUG: Report error if task is nested
-?? Define packages as a group of tasks?
+* Any methods other than `append` that cannot be repeatedly run?
+* Display additional install notes for a particular package (at the end of installation) - e.g. printer setup requires Windows share to be setup
+* CODE/DOC: Describe difference between package and task or merge packages with tasks if possible
+* BUG: Report error if task is nested
+* Define packages as a group of tasks?
 
 
 Misc Tasks
 ----------------------------------------
 
-Remove percentage from progress once command has completed
-DOC: How webapps environment specific settings override default settings and how to set your own
-DOC: All config files
-DOC: Why we use ~/.profile (with links)
-On initial SSH connection to machine, test we have an Internet connection. Fail if not.
-Add the check that was run to `CHECK_FAILED/CHECK_PASSED`
-Output progress to log/<machine_name>_progress.log
-Rename output log to log/<machine_name>_output.log
-DOC: machines desc <package> - Should display a detailed description of the package
-machines list - Display a list of machines to build (or maybe machines build/dryrun with no machinename)
-?? base package may not be needed on DB installs
-?? Default path for Nginx install is /usr/local - Is it installed correctly for non-default paths?
-?? I have a new webapp - How can I add it to a server that has already been installed?
-BUG: Uploads throw exception if local file is missing - Get upload to check file existence when adding to queue
-BUG: CTRL+C doesn't quite exit cleanly
-Check $conf.db_server is picked up and used to write database.yml on qa/staging/production
-
-Allow $conf.webapps[app].path to be overridden from webapps.yml
-?? Does webapps.yml structure get preserved? (e.g. when modifying keys and resaving)
-Fix guard notifications
-
-Enable YAML to refer to other settings in the same file
-development machines should clone repos
-
-Should be able to run Passenger install easily for new versions
-Can more files be ERB templates? Standardise. Need examples (can't remember what they were)
-webapps has gaps in testing (e.g. ssl)
-escape $$ in passwords
-turn off debug output by default
-DOC: webapps.yml loaded into $conf
-MySQL root pass is not set properly
-Recommended practice for overwritting project with new template. Use Git
-Set cron to sensible times /etc/crontab [DEV]
-Need a way to see optional tasks that are not run as part of full build
-
-Would `set :variable_name, value` be better than `$conf.variable = value`?
-Need a better DSL to handle $conf and also paths File.join is so verbose
+* Remove percentage from progress once command has completed
+* DOC: How webapps environment specific settings override default settings and how to set your own
+* DOC: All config files
+* DOC: Why we use ~/.profile (with links)
+* On initial SSH connection to machine, test we have an Internet connection. Fail if not.
+* Add the check that was run to `CHECK_FAILED/CHECK_PASSED`
+* Output progress to log/<machine_name>_progress.log
+* Rename output log to log/<machine_name>_output.log
+* DOC: machines desc <package> - Should display a detailed description of the package
+* machines list - Display a list of machines to build (or maybe machines build/dryrun with no machinename)
+* ?? base package may not be needed on DB installs
+* ?? Default path for Nginx install is /usr/local - Is it installed correctly for non-default paths?
+* ?? I have a new webapp - How can I add it to a server that has already been installed?
+* BUG: Uploads throw exception if local file is missing - Get upload to check file existence when adding to queue
+* BUG: CTRL+C doesn't quite exit cleanly
+* Check $conf.db_server is picked up and used to write database.yml on qa/staging/production
+
+* Allow $conf.webapps[app].path to be overridden from webapps.yml
+* ?? Does webapps.yml structure get preserved? (e.g. when modifying keys and resaving)
+
+* Can more files be ERB templates? Standardise. Need examples (can't remember what they were)
+* webapps has gaps in testing (e.g. ssl)
+* BUG: escape $$ in passwords
+* turn off debug output by default
+* DOC: webapps.yml loaded into $conf
+* MySQL root pass is not set properly
+* Recommended practice for overwritting project with new template. Use Git
+* Set cron to sensible times /etc/crontab [DEV]
+* Need a way to see optional tasks that are not run as part of full build
+
+* Would `set :variable_name, value` be better than `$conf.variable = value`?
+* Need a better DSL to handle $conf and also paths File.join is so verbose
   For exmaple, instead of:
-    File.join($conf.appsroot, 'subfolder')
+      File.join($conf.appsroot, 'subfolder')
   How about:
-    path :appsroot, 'subfolder'
-rvm ruby@gemset --rvmrc to generate passenger compatible .rvmrc
-setup memcached
-passenger_nginx was installed with rvmsudo. Need to test it still works with just sudo
+      path :appsroot, 'subfolder'
+* rvm ruby@gemset --rvmrc to generate passenger compatible .rvmrc
+* passenger_nginx was installed with rvmsudo. Need to test it still works with just sudo
 

data/bin/machines

@@ -2,5 +2,4 @@
 
 require 'machines'
 
-Machines::Base.new.execute(ARGV)
-
+Machines::Commandline.execute(ARGV)

data/lib/machines.rb

@@ -1,6 +1,7 @@
 # Machines allows simple configuration of development, staging and production computers or images for ec2
 require 'active_support'
 require 'app_conf'
+require 'date'
 require 'erb'
 require 'fileutils'
 require 'highline/import'
@@ -15,5 +16,19 @@ require 'yaml'
 $conf = AppConf.new
 $conf.application_dir = File.dirname(__FILE__)
 
-require 'machines/base'
+files = Dir[File.join($conf.application_dir, 'machines/commands/*.rb')]
+files.sort.each do |file|
+  require file
+end
+
+require 'machines/logger'
+require 'machines/named_buffer'
+require 'machines/app_settings'
+require 'machines/command'
+require 'machines/log_command'
+require 'machines/upload'
+require 'machines/core'
+require 'machines/commandline'
+require 'machines/cloud_machine'
+require 'machines/help'