ironfan 3.1.7 → 3.2.2

data/notes/design_notes-ci_testing.md

@@ -1,169 +0,0 @@
-
-  
-https://github.com/acrmp/chefspec
-
-
-pre-testing -- converge machine 
-  https://github.com/acrmp/chefspec
-
-  http://wiki.opscode.com/display/chef/Knife#Knife-test
-
-benchmarks
-
-  bonnie++ 
-  hdparm -t
-  iozone
-
-
-in-machine
-
-* x ports on x interfaces open
-* daemon is running
-* file exists and has string
-
-* log file is accumulating lines at rate X
-* script x runs successfully
-
-in-chef
-
-* runlist is X
-* chef attribute X should be Y
-
-meta
-
-* chef run was idempotent
-
-
-
-
-
-__________________________________________________________________________
-
-## Notes from around the web
-
-
-* ...
-
-  > I'm thinking that the useful thing to test is NOT did chef install
-  > some package or setup a user, but rather after chef has run can I
-  > interact with the system as I would expect from an external
-  > perspective. For example:
-  > 
-  > * Is the website accessible?
-  > * Are unused ports blocked?
-  > * When I send an email thorough the website does it end up in my inbox?
-  > 
-  > Capybara (http://github.com/jnicklas/capybara) enforces this external
-  > perspective for webapp testing:
-  > 
-  > "Access to session, request and response from the test is not
-  > possible. Maybe we’ll do response headers at some point in the future,
-  > but the others really shouldn’t be touched in an integration test
-  > anyway. "
-  > 
-  > They only let you interact with screen elements that a user could
-  > interact with. It makes sense because the things that users interact
-  > with are what provides the business value
-
-* Andrew Shafer < andrew@cloudscaling.com>
-
-  > Here's my thinking at this point... which could be wrong on every level.
-  > There is really no good way to TDD/BDD configuration management for several
-  > reasons:
-  > The recipes are already relatively declarative
-  > Mocking is useless because it may not reflect 'ground truth'
-  > The cycle times to really test convergence are relatively long
-  > Trying to test if a package is installed or not is testing the framework,
-  > not the recipe IMHO.
-  > I agree with the general sentiment that the functional service is the true
-  > test.
-  > I'm leaning towards 'testing' at that level, ideally with (a superset of?)
-  > what should be used for the production monitoring system.
-  > So the CI builds services, runs all the checks in test, green can go live
-  > and that's that.
-  
-
-* Jeremy Deininger < jeremy@rightscale.com>
-
-  > Thought I'd chime in with my experience testing system configuration code @ RightScale so far.  What we've been building are integration style cucumber tests to run a cookbook through it's paces on all platforms and OSs that we support.  
-  > First we use our API to spin up 'fresh' server clusters in EC2, one for every platform/OS (variation) that the cookbook will be supporting.  The same could be done using other cloud APIs (anyone else doing this with VMware or etc?)  Starting from scratch is important because of chef's idempotent nature.
-  > Then a cucumber test is run against every variation in parallel.  The cucumber test runs a series of recipes on the cluster then uses what we call 'spot checks' to ensure the cluster is configured and functional.  The spot checks are updated when we find a bug, to cover the bug.  An example spot check would be, sshing to every server and checking the mysql.err file for bad strings.
-  > These high level integration tests are long running but have been very useful flushing out bugs.
-  > ...
-  > If you stop by the #rightscale channel on Freenode I'd be happy to embarrass myself by giving you a sneak peak at the features etc..  Would love to bounce ideas around and collaborate if you're interested. jeremydei on Freenode IRC
-
-Ranjib Dey < ranjibd@th...s.com>
-
-  > So far, what we've done for testing is to use rspec for implementing tests.  Here's an example:
-  > 
-  >    it "should respond on port 80" do
-  >      lambda {
-  >        TCPSocket.open(@server, 'http')
-  >      }.should_not raise_error
-  >    end
-  > 
-  > Before running the tests, I have to manually bootstrap a node using knife. If my instance is the only one in its environment, the spec can find it using knife's search feature.  The bootstrap takes a few minutes, and the 20 or so tests take about half a minute to run.
-  > 
-  > While I'm iteratively developing a recipe, my work cycle is to edit source, upload a cookbook, and rerun chef-client (usually by rerunning knife boostrap, because the execution environment is different from invoking chef-client directly).  This feels a bit slower than the cycle I'm used to when coding in Ruby because of the upload and bootstrap steps.
-  > 
-  > I like rspec over other testing tools because of how it generates handy reports, such as this one, which displays an English list of covered test cases:
-  > 
-  >    $ rspec spec/ -f doc
-  > 
-  >    Foo role
-  >      should respond on port 80
-  >      should run memcached
-  >      should accept memcached connections
-  >      should have mysql account
-  >      should allow passwordless sudo to user foo as user bar
-  >      should allow passwordless sudo to root as a member of sysadmin
-  >      should allow key login as user bar
-  >      should mount homedirs on ext4, not NFS
-  >      should rotate production.log
-  >      should have baz as default vhost
-  >      ...
-  > 
-  > That sample report also gives a feel for sort of things we check.  So far, nearly all of our checks are non-intrusive enough to run on a production system.  (The exception is testing of local email delivery configurations.)
-  > 
-  > Areas I'd love to see improvement:
-  > 
-  >    * Shortening the edit-upload-bootstrap-test cycle
-  >    * Automating the bootstrap in the context of testing
-  >    * Adding rspec primitives for Chef-related testing, which might
-  >      include support for multiple platforms
-  > 
-  > As an example of rspec primitives, instead of:
-  > 
-  >    it "should respond on port 80" do
-  >      lambda {
-  >        TCPSocket.open(@server, 'http')
-  >      }.should_not raise_error
-  >    end
-  > 
-  > I'd like to write:
-  > 
-  >    it { should respond_on_port(80) }
-  > 
-  > Rspec supports the the syntactic sugar; it's just a matter of adding some "matcher" plugins.
-  > 
-  > How do other chef users verify that recipes work as expected?
-  > 
-  > I'm not sure how applicable my approach is to opscode/cookbooks because it relies on having a specific server configuration and can only test a cookbook in the context of that single server.  If we automated the boostrap step so it could be embedded into the rspec setup blocks, it would be possible to test a cookbook in several sample contexts, but the time required to setup each server instance might be prohibitive.
-  > 
-
-
-Andrew Crump < acrump@gmail.com>
-
-  > Integration tests that exercise the service you are building definitely give you the most bang for buck.
-  > 
-  > We found the feedback cycle slow as well so I wrote chefspec which builds on RSpec to support unit testing cookbooks:
-  > 
-  > https://github.com/acrmp/chefspec
-  > 
-  > This basically fakes a convergence and allows you to make assertions about the created resources. At first glance Chef's declarative nature makes this less useful, but once you start introducing conditional execution I've found this to be a time saver.
-  > 
-  > If you're looking to do CI (which you should be) converging twice goes some way to verifying that your recipes are idempotent.
-  > 
-  > knife cookbook test is a useful first gate for CI:
-  > 
-  > http://wiki.opscode.com/display/chef/Knife#Knife-test

data/notes/design_notes-cookbook_event_ordering.md

@@ -1,249 +0,0 @@
-# Cookbook event ordering
-
-
-Most cookbooks have some set of the following
-
-* base configuration
-* announce component
-  - before discovery so it can be found
-  - currently done in converge stage -- some aspects might be incompletely defined?
-
-* register apt repository if any
-* create daemon user
-  - before directories so we can set permissions
-  - before package install so uid is stable
-* install, as package, git deploy or install from release
-  - often have to halt legacy services -- config files don't exist
-* create any remaining directories 
-  - after package install so it has final say
-* install plugins
-  - after directories, before config files
-* define service
-  - before config file creation so we can notify
-  - can't start it yet because no config files
-* discover component (own or other, same or other machines)
-* write config files (notify service of changes)
-  - must follow everything so info is current
-* register a minidash dashboard
-* trigger start (or restart) of service
-
-## Proposal:
-
-kill `role`s, in favor of `stack`s.
-
-A runlist is assembled in the following phases:
-
-* `initial` configuration
-* `before_install`
-  - `account`
-* `install`
-  - `plugin` install
-  - `directories`
-* `announcement`
-  - `service` definition
-- `discovery`
-* `commit`
-  - `config_files`: write config files to disk
-* `finalize`
-* `launch`
-
-As you can see, most layers have semantic names (`plugin`, `user`, etc); if you name your recipe correctly, they will be assigned to the correct phase. Otherwise, you can attach them explicitly to a semantic or non-semantic phase.
-
-    elasticsearch_datanode component:
-    
-    elasticsearch::default              # initial
-    elasticsearch::install_from_release #install phase
-    elasticsearch::plugins              # plugin phase
-
-    elasticsearch::server               # service definition; includes announcement
-    elasticsearch::config_files         # config_files; includes discovery
-
-I'm not clear on how much the phases should be strictly broken out into one-recipe-per-phase.
-
-It's also possible to instead define a chef resource that let you defer a block to a given phase from within a callback. It would be similar to a ruby block, but have more explicit timing. This may involve jacking around inside Chef, something we've avoided to now.
-
-__________________________________________________________________________
-
-Run List is 
-
-    [role[systemwide], role[chef_client], role[ssh], role[nfs_client],
-    role[volumes], role[package_set], role[org_base], role[org_users], 
-    
-    role[hadoop],
-    role[hadoop_s3_keys], 
-    role[cassandra_server], role[zookeeper_server],
-    role[flume_master], role[flume_agent], 
-    role[ganglia_master],
-    role[ganglia_agent], role[hadoop_namenode], role[hadoop_datanode],
-    role[hadoop_jobtracker], role[hadoop_secondarynn], role[hadoop_tasktracker],
-    role[hbase_master], role[hbase_regionserver], role[hbase_stargate],
-    role[redis_server], role[mysql_client], role[redis_client],
-    role[cassandra_client], role[elasticsearch_client], role[jruby], role[pig],
-    recipe[ant], recipe[bluepill], recipe[boost], recipe[build-essential],
-    recipe[cron], recipe[git], recipe[hive], recipe[java::sun], recipe[jpackage],
-    recipe[jruby], recipe[nodejs], recipe[ntp], recipe[openssh], recipe[openssl],
-    recipe[rstats], recipe[runit], recipe[thrift], recipe[xfs], recipe[xml],
-    recipe[zabbix], recipe[zlib], recipe[apache2], recipe[nginx],
-    role[el_ridiculoso_cluster], role[el_ridiculoso_gordo], role[minidash],
-    role[org_final], recipe[hadoop_cluster::config_files], role[tuning]]
-
-Run List expands to 
-
-    build-essential, motd, zsh, emacs, ntp, nfs, nfs::client, xfs,
-    volumes::mount, volumes::resize, package_set, 
-    
-    hadoop_cluster,
-    hadoop_cluster::minidash, 
-    
-    cassandra, cassandra::install_from_release,
-    cassandra::autoconf, cassandra::server, cassandra::jna_support,
-    cassandra::config_files, zookeeper::default, zookeeper::server,
-    zookeeper::config_files, flume, flume::master, flume::agent,
-    flume::jruby_plugin, flume::hbase_sink_plugin, ganglia, ganglia::server,
-    ganglia::monitor, 
-    
-    hadoop_cluster::namenode, 
-    hadoop_cluster::datanode,
-    hadoop_cluster::jobtracker, 
-    hadoop_cluster::secondarynn,
-    hadoop_cluster::tasktracker, 
-    
-    zookeeper::client, 
-    hbase::master,
-    hbase::minidash, 
-    
-    minidash::server, 
-    hbase::regionserver, 
-    hbase::stargate,
-    redis, redis::install_from_release, redis::server, 
-    mysql, mysql::client,
-    cassandra::client, elasticsearch::default,
-    
-    elasticsearch::install_from_release, 
-    elasticsearch::plugins,
-    elasticsearch::client, 
-    
-    jruby, jruby::gems, 
-    
-    pig, 
-    pig::install_from_package,
-    pig::piggybank, 
-    pig::integration, 
-    
-    zookeeper, 
-    ant, bluepill, boost, cron,
-    git, hive, 
-    java::sun, jpackage, nodejs, openssh, openssl, rstats, runit,
-    thrift, xml, zabbix, zlib, apache2, nginx, hadoop_cluster::config_files,
-    tuning::default
-    
-
-From an actual run of el_ridiculoso-gordo:
-    
-      2 nfs::client
-      1 java::sun
-      1 aws::default
-      5 build-essential::default
-      3 motd::default
-      2 zsh::default
-      1 emacs::default
-      8 ntp::default
-      1 nfs::default
-      2 nfs::client
-      3 xfs::default
-     46 package_set::default
-      4 java::sun
-      8 tuning::ubuntu
-      6 apt::default
-      1 hadoop_cluster::add_cloudera_repo
-     44 hadoop_cluster::default
-      4 minidash::default
-      2 /srv/chef/file_store/cookbooks/minidash/providers/dashboard.rb
-      1 hadoop_cluster::minidash
-      2 /srv/chef/file_store/cookbooks/minidash/providers/dashboard.rb
-      1 boost::default
-      2 python::package
-      2 python::pip
-      1 python::virtualenv
-      2 install_from::default
-      7 thrift::default
-      9 cassandra::default
-      1 cassandra::install_from_release
-      6 /srv/chef/file_store/cookbooks/install_from/providers/release.rb
-      3 cassandra::install_from_release
-      1 cassandra::bintools
-      3 runit::default
-     11 cassandra::server
-      2 cassandra::jna_support
-      2 cassandra::config_files
-      6 zookeeper::default
-     15 zookeeper::server
-      3 zookeeper::config_files
-     18 flume::default
-      2 flume::master
-      3 flume::agent
-      2 flume::jruby_plugin
-      1 flume::hbase_sink_plugin
-     21 ganglia::server
-     20 ganglia::monitor
-     13 hadoop_cluster::namenode
-     11 hadoop_cluster::datanode
-     11 hadoop_cluster::jobtracker
-     11 hadoop_cluster::secondarynn
-     11 hadoop_cluster::tasktracker
-     14 hbase::default
-     11 hbase::master
-      1 hbase::minidash
-      2 /srv/chef/file_store/cookbooks/minidash/providers/dashboard.rb
-     13 minidash::server
-     11 hbase::regionserver
-     10 hbase::stargate
-      2 redis::default
-      2 redis::install_from_release
-      2 redis::default
-     16 redis::server
-      3 mysql::client
-      1 aws::default
-      7 elasticsearch::default
-      1 elasticsearch::install_from_release
-      6 /srv/chef/file_store/cookbooks/install_from/providers/release.rb
-      2 elasticsearch::plugins
-      3 elasticsearch::client
-      3 elasticsearch::config
-      1 jruby::default
-      9 /srv/chef/file_store/cookbooks/install_from/providers/release.rb
-      7 jruby::default
-     18 jruby::gems
-      2 pig::install_from_package
-      5 pig::piggybank
-      8 pig::integration
-      6 zookeeper::default
-      3 ant::default
-      5 bluepill::default
-      2 cron::default
-      1 git::default
-      1 hive::default
-      2 nodejs::default
-      4 openssh::default
-      7 rstats::default
-      1 xml::default
-     11 zabbix::default
-      1 zlib::default
-     10 apache2::default
-      2 apache2::mod_status
-      2 apache2::mod_alias
-      1 apache2::mod_auth_basic
-      1 apache2::mod_authn_file
-      1 apache2::mod_authz_default
-      1 apache2::mod_authz_groupfile
-      1 apache2::mod_authz_host
-      1 apache2::mod_authz_user
-      2 apache2::mod_autoindex
-      2 apache2::mod_dir
-      1 apache2::mod_env
-      2 apache2::mod_mime
-      2 apache2::mod_negotiation
-      2 apache2::mod_setenvif
-      1 apache2::default
-      8 nginx::default
-      9 hadoop_cluster::config_files

data/notes/design_notes-meta_discovery.md

@@ -1,59 +0,0 @@
-@temujin9 has proposed, and it's a good propose, that there should exist such a thing as an 'integration cookbook'.
-
-The hadoop_cluster cookbook should describe the hadoop_cluster, the ganglia cookbook ganglia, and the zookeeper cookbook zookeeper. Each should provide hooks that are neighborly but not exhibitionist, but should mind its own business.
-
-The job of tying those components together should belong to a separate concern. It should know how and when to copy hbase jars into the pig home dir, or what cluster service_provide'r a redis client should reference.
-
-## Practical implications
-
-* I'm going to revert out the `node[:zookeeper][:cluster_name]` attributes -- services should always announce under their cluster.
-
-* Until we figure out how and when to separate integrations, I'm going to isolate entanglements into their own recipes within cookbooks: so, the ganglia part of hadoop will become `ganglia_integration` or somesuch.
-
-## Example integrations
-
-### Copying jars
-
-Pig needs jars from hbase and zookeeper.  They should announce they have jars; pig should announce its home directory;  the integration should decide how and where to copy the jars.
-
-### Reference a service
-
-Right now in several places we have attributes like `node[:zookeeper][:cluster_name]`, used to specify the cluster that provides_service zookeeper.
-
-* Server recipes should never use `node[:zookeeper][:cluster_name]` -- they should always announce under their native cluster. (I'd kinda like to give `provides_service` some sugar to assume the cluster name, but need to find something backwards-compatible to use)
-
-* Need to take a better survey of usage among clients to determine how to do this.
-
-* cases:
-  - hbase cookbook refs: hadoop, zookeeper, ganglia
-  - flume cookbook refs: zookeeper, ganglia. 
-  - flume agents may reference several different flume provides_service'rs
-  - API using two different elasticsearch clusters
-  
-### Logging, monitoring
-
-* tell flume you have logs to pick up
-* tell ganglia to monitor you
-
-### Service Dashboard
-
-Let everything with a dashboard say so, and then let one resource create a page that links to each.
-
-
-________________________
-
-These are still forming, ideas welcome.
-
-
-
-
-Sometimes we want 
-
-* if there are volumes marked for 'mysql-table_data', use that; otherwise, the 'persistent' datastore, if any; else the 'bulk' datastore, if any; else the 'fallback' datastore (which is guaranteed to exist).
-
-* IP addresses (or hostnames):
-  - `[:private_ip, :public_ip]`
-  - `[:private_ip]`
-  - `:primary_ip`
-  
-* .  

data/notes/ec2-pricing_and_capacity.md

@@ -1,69 +0,0 @@
-## Compute Costs
-
-
-    code         	$/mo	 $/day	 $/hr	CPU/$	Mem/$	  mem 	  cpu 	cores	cpcore	storage	bits	IO      	type       	name
-    t1.micro       	  15	  0.48	  .02	   13	   13	  0.61	  0.25	 0.25	  1   	      0	   32	Low     	Micro      	Micro
-    m1.small       	  58	  1.92	  .08	   13	   21	  1.7 	  1   	 1   	  1   	    160	   32	Moderate	Standard   	Small
-    m1.medium      	 116	  3.84	  .165	   13	   13	  3.75	  2   	 2   	  1   	    410	   32	Moderate	Standard   	Medium
-    c1.medium      	 120	  3.96	  .17	   30	   10	  1.7 	  5   	 2   	  2.5 	    350	   32	Moderate	High-CPU   	Medium
-    m1.large       	 232	  7.68	  .32	   13	   23	  7.5 	  4   	 2   	  2   	    850	   64	High    	Standard   	Large
-    m2.xlarge      	 327	 10.80	  .45	   14	   38	 17.1 	  6.5 	 2   	  3.25	    420	   64	Moderate	High-Memory	Extra Large
-    m1.xlarge      	 465	 15.36	  .64	   13	   23	 15   	  8   	 4   	  2   	   1690	   64	High    	Standard   	Extra Large
-    c1.xlarge      	 479	 15.84	  .66	   30	   11	  7   	 20   	 8   	  2.5 	   1690	   64	High    	High-CPU   	Extra Large
-    m2.2xlarge     	 653	 21.60	  .90	   14	   38	 34.2 	 13   	 4   	  3.25	    850	   64	High    	High-Memory	Double Extra Large
-    cc1.4xlarge    	 944	 31.20	 1.30	   26	   18	 23   	 33.5 	 2   	 16.75	   1690	   64	10GB    	Compute    	Quadruple Extra Large
-    m2.4xlarge     	1307	 43.20	 1.80	   14	   38	 68.4 	 26   	 8   	  3.25	   1690	   64	High    	High-Memory	Quadruple Extra Large
-    cg1.4xlarge    	1525	 50.40	 2.10	   16	   10	 22   	 33.5 	 2   	 16.75	   1690	   64	10GB    	Cluster GPU	Quadruple Extra Large
-    cc2.8xlarge    	1742	 57.60	 2.40	   37	   25	 60.5 	 88   	 2   	 44   	   3370	   64	10GB    	Compute    	Eight Extra Large
-
-    dummy header ln	  15	  0.48	 0.02	12345	12345	  0.61	  0.25	 0.25	  1.00	6712345	32123	Low     	Micro      	Micro                
-
-
-## Storage Costs
-
-                            $/GB..mo		$/GB.mo  	$/Mio	
-    EBS Volume     			$0.10				
-    EBS I/O       			            	         	$0.10
-    EBS Snapshot S3			$0.083				
-
-                         	Std $/GB.mo		Red.Red. $/GB.mo
-    S3 1st tb            	$0.125     	 	$0.093
-    S3 next 49tb         	$0.110     	 	$0.083 
-    S3 next 450tb        	$0.095  		$0.073 
-
-### Storing 1TB data
-
-(Cost of storage, neglecting I/O costs, and assuming the ratio of EBS volume size to snapshot size is as given)
-
-* http://aws.amazon.com/ec2/instance-types/
-* http://aws.amazon.com/ec2/#pricing
-
-### How much does EBS cost?
-
-The costs of EBS will be similar to the pricing structure of data storage on S3.  There are three types of costs associated with EBS.
-
-Storage Cost + Transaction Cost + S3 Snapshot Cost = Total Cost of EBS
-
-NOTE: For current pricing information, be sure to check Amazon EC2 Pricing.
-
-#### Storage Costs
-
-The cost of an EBS Volume is $0.10/GB per month.  You are responsible for paying for the amount of disk space that you reserve, not for the amount of the disk space that you actually use.  If you reserve a 1TB volume, but only use 1GB, you will be paying for 1TB.
-* $0.10/GB per month of provisioned storage
-* $0.10/GB per 1 million I/O requests
- 
-#### Transaction Costs
-
-In addition to the storage cost for EBS Volumes, you will also be charged for I/O transactions. The cost is $0.10 per million I/O transactions, where one transaction is equivalent to one read or write.  This number may be smaller than the actual number of transactions performed by your application because of the Linux cache for all file systems.
-$0.10 per 1 million I/O requests
- 
-#### S3 Snapshot Costs
-
-Snapshot costs are compressed and based on altered blocks from the previous snapshot backup.  Files that have altered blocks on the disk and then been deleted will add cost to the Snapshots for example.  Remember, snapshots are at the data block level.
-$0.15 per GB-month of data stored
-$0.01 per 1,000 PUT requests (when saving a snapshot)
-$0.01 per 10,000 GET requests (when loading a snapshot)
-
-NOTE:  Payment charges stop the moment you delete a volume.  If you delete a volume and the status appears as "deleting" for an extended period of time, you will not be charged for the time needed to complete the deletion.
-
- 

data/notes/homebase-layout.txt

@@ -1,102 +0,0 @@
-Ironfan Homebase Layout
-=======================
-
-Your Chef Homebase contains several directories, and each contains a README.md file describing its purpose and use in greater detail.
-
-This directory structure came out of a *lot* of trial and error, and is working very well where many others didn't. This homebase makes it easy to pull in third-party pantries (cookbook collections) and track upstream changes without being locked in.
-
-## Main Directories
-
-The main assets you'll use are:
-
-* `clusters/`   - Clusters fully describe your machines, from its construction ('an 8-core machine on the Amazon EC2 cloud') to its roles ('install Cassandra, Ganglia for monitoring, and silverware to manage its logs and firewall').
-* `cookbooks/`  - Cookbooks you download or create. Cookbooks install components, for example `cassandra` or `java`.
-* `roles/`      - Roles organize cookbooks and attribute overrides to describe the specific composition of your system. For example, you install Cassandra attaching the `cassandra_server` role to your machine. (.rb or .json files)
-
-These folders hold supporting files. You're less likely to visit here regularly.
-
-* `knife/`        - Chef and cloud configuration, and their myriad attendant credentials.
-* `environments/` - Organization-wide attribute values. (.json or .rb files)
-* `data_bags/`    - Data bags are an occasionally-useful alternative to node metadata for distributing information to your machines. (.json files)
-* `certificates/` - SSL certificates generated by `rake ssl_cert` live here.
-* `tasks/`        - Rake tasks for common administrative tasks.
-* `vendor/`       - cookbooks are checked out to `vendor`; symlinks in the `cookbooks/` directory select which ones will be deployed to chef server. The vendor directory comes with the Ironfan, Opscode and (should you be a lucky customer) Ironfan Enterprise chef pantries.
-* `notes/`        - a submoduled copy of the [ironfan wiki](http://github.com/infochimps-labs/ironfan/wiki`
-
-## Directory setup
-
-The core structure of the homebase is as follows ("├─*" means "git submodule'd"):
-
-    homebase
-    ├── clusters                        - cluster definition files
-    │   └── ( clusters )
-    ├── cookbooks                       - symlinks to cookbooks
-    │   ├── @vendor/ironfan-pantry/cookbooks/...
-    │   ├── @vendor/opscode/cookbooks/...
-    │   └── @vendor/org-pantry/cookbooks/...
-    ├── environments                    - environment definition files
-    │   └── ( environments )
-    ├── data_bags                       - symlinks to data_bags
-    │   ├── @vendor/ironfan-pantry/roles/...
-    │   └── @vendor/org-pantry/roles/...
-    ├── roles                           - symlinks to roles
-    │   ├── @vendor/ironfan-pantry/roles/...
-    │   └── @vendor/org-pantry/roles/...
-    ├── vendor
-    │   ├─* ironfan-pantry              - git submodule of https://github.com/infochimps-labs/ironfan-pantry
-    │   │   ├── cookbooks
-    │   │   ├── data_bags
-    │   │   ├── roles
-    │   │   └── tasks
-    │   ├─* ironfan-enterprise          - git submodule of ironfan-enterprise, if you're a lucky customer
-    │   │   ├── cookbooks
-    │   │   ├── data_bags
-    │   │   ├── roles
-    │   │   └── tasks
-    │   ├── opscode
-    │   │   └─* cookbooks               - git submodule of https://github.com/infochimps-labs/opscode_cookbooks; itself a closely-tracked fork of https://github.com/opscode/cookbooks
-    │   └── org-pantry                  - organization specific roles, cookbooks, etc.
-    │       ├── cookbooks
-    │       ├── data_bags
-    │       ├── roles
-    │       └── tasks
-    ├── knife                           - credentials (see below)
-    ├── certificates
-    ├── config
-    ├─* notes
-    ├── tasks
-    └── vagrants                        - vagrant files (when using ironfan-ci)
-
-The `vendor/opscode/cookbooks` and `vendor/ironfan-pantry` directories are actually [git submodules](http://help.github.com/submodules/). This makes it easy to track upstream changes in each collection while still maintaining your own modifications.
-
-We recommend you place your cookbooks in `vendor/org-pantry`. If you have cookbooks &c from other pantries that have significant changes, you can duplicate it into this `vendor/org-pantry` and simply change the symlink in homebase/cookbooks/.
-
-## Knife dir setup
-
-We recommend you version your credentials directory separately from your homebase. You will want to place it under version control, but you should not place it into a central git repository -- this holds the keys to the kingdom.
-
-We exclude the chef user key (`(user).pem`) and user-specific knife settings (`knife-user-(user).rb`) from the repo as well. Each user has their own revokable client key, and their own cloud credentials set, and those live nowhere but their own computers.
-
-    knife/
-    ├── knife.rb
-    ├── credentials -> (organization)-credentials
-    ├── (organization)-credentials
-    │   ├── knife-org.rb                - org-specific stuff, and cloud assets (elastic IPs, AMI image ids, etc)
-    │   ├── (user).pem                  - your chef client key *GITIGNORED*
-    │   ├── knife-user-(user).rb        - your user-specific knife customizations *GITIGNORED*
-    │   ├── (organization)-validator.pem- chef validator key, used to create client keys
-    │   ├── client_keys
-    │   │   └── (transient client keys) - you can delete these at will; only useful if you're debugging a bad bootstrap
-    │   ├── ec2_keys
-    │   │   └── (transient client keys) - ssh keys for cloud machines (in EC2 parlance, the private half of your keypair)
-    │   ├── certificates
-    │   ├── data_bag_keys
-    │   └── ec2_certs
-    ├── bootstrap                       - knife cluster bootstrap scripts
-    ├── plugins
-    │   └── knife
-    │       └── (knife plugins)         - knife plugins
-    └── .gitignore                      - make sure not to version the secret/user-specific stuff (*-keypairs, *-credentials.rb, knife-user-*.rb)
-
-    
-(You can safely ignore the directories above that aren't annotated; they're useful in certain advanced contexts but not immediately)

data/notes/knife-cluster-commands.md

@@ -1,18 +0,0 @@
-# Ironfan Knife Commands
-
-Available cluster subcommands: (for details, `knife SUB-COMMAND --help`)
-
-    knife cluster list (options)                                  - show available clusters
-    knife cluster bootstrap   CLUSTER-[FACET-[INDEXES]] (options) - bootstrap all servers described by given cluster slice
-    knife cluster kick        CLUSTER-[FACET-[INDEXES]] (options) - start a run of chef-client on each server, tailing the logs and exiting when the run completes.
-    knife cluster kill        CLUSTER-[FACET-[INDEXES]] (options) - kill all servers described by given cluster slice
-    knife cluster launch      CLUSTER-[FACET-[INDEXES]] (options) - Creates chef node and chef apiclient, pre-populates chef node, and instantiates in parallel their cloud machines. With --bootstrap flag, will ssh in to machines as they become ready and launch the bootstrap process
-    knife cluster proxy       CLUSTER-[FACET-[INDEXES]] (options) - Runs the ssh command to open a SOCKS proxy to the given host, and writes a PAC (automatic proxy config) file to /tmp/ironfan_proxy-YOURNAME.pac. Only the first host is used, even if multiple match.
-    knife cluster show        CLUSTER-[FACET-[INDEXES]] (options) - a helpful display of cluster's cloud and chef state
-    knife cluster ssh         CLUSTER-[FACET-[INDEXES]] COMMAND (options) - run an interactive ssh session, or execuse the given command, across a cluster slice
-    knife cluster start       CLUSTER-[FACET-[INDEXES]] (options) - start all servers described by given cluster slice
-    knife cluster stop        CLUSTER-[FACET-[INDEXES]] (options) - stop all servers described by given cluster slice
-    knife cluster sync        CLUSTER-[FACET-[INDEXES]] (options) - Update chef server and cloud machines with current cluster definition
-    knife cluster vagrant CMD CLUSTER-[FACET-[INDEXES]] (options) - runs the given command against a vagrant environment created from your cluster definition. EARLY, use at your own risk
-
-