ironfan 3.1.0.rc1 → 3.1.1

data/notes/tips_and_troubleshooting.md

@@ -0,0 +1,83 @@
+## Tips and Notes
+
+### EC2 Notes Instance attributes: `disable_api_termination` and `delete_on_termination`
+
+To set `delete_on_termination` to 'true' after the fact, run the following (modify the instance and volume to suit):
+
+```
+    ec2-modify-instance-attribute -v i-0704be6c --block-device-mapping /dev/sda1=vol-XX8d2c80::true
+```
+
+If you set `disable_api_termination` to true, in order to terminate the node run
+```
+    ec2-modify-instance-attribute -v i-0704be6c --disable-api-termination false
+```
+
+To view whether an attached volume is deleted when the machine is terminated:
+
+```
+    # show volumes that will be deleted
+    ec2-describe-volumes --filter "attachment.delete-on-termination=true"
+```
+
+You can't (as far as I know) alter the delete-on-termination flag of a running volume. Crazy, huh?
+
+### EC2: See your userdata
+
+curl http://169.254.169.254/latest/user-data
+
+### EBS Volumes for a persistent HDFS
+
+* Make one volume and format for XFS:
+    `$ sudo mkfs.xfs -f /dev/sdh1`
+* options "defaults,nouuid,noatime" give good results. The 'nouuid' part
+  prevents errors when mounting multiple volumes from the same snapshot.
+* poke a file onto the drive :
+  datename=`date +%Y%m%d`
+  sudo bash -c "(echo $datename ; df /data/ebs1 ) > /data/ebs1/xfs-created-at-$datename.txt"
+
+
+If you want to grow the drive:
+* take a snapshot.
+* make a new volume from it
+* mount that, and run `sudo xfs_growfs`. You *should* have the volume mounted, and should stop anything that would be working the volume hard.
+
+### Hadoop: On-the-fly backup of your namenode metadata
+
+bkupdir=/ebs2/hadoop-nn-backup/`date +"%Y%m%d"`
+
+for srcdir in /ebs*/hadoop/hdfs/ /home/hadoop/gibbon/hdfs/  ; do
+  destdir=$bkupdir/$srcdir ; echo $destdir ;
+  sudo mkdir -p $destdir ;
+done
+
+
+### NFS: Halp I am using an NFS-mounted /home and now I can't log in as ubuntu
+
+Say you set up an NFS server 'core-homebase-0' (in the 'core' cluster) to host and serve out `/home` directory; and a machine 'awesome-webserver-0' (in the 'awesome' cluster), that is an NFS client.
+
+In each case, when the machine was born EC2 created a `/home/ubuntu/.ssh/authorized_keys` file listing only the single approved machine keypair -- 'core' for the core cluster, 'awesome' for the awesome cluster.
+
+When chef client runs, however, it mounts the NFS share at /home. This then masks the actual /home directory -- nothing that's on the base directory tree shows up. Which means that after chef runs, the /home/ubuntu/.ssh/authorized_keys file on awesome-webserver-0 is the one for the *'core'* cluster, not the *'awesome'* cluster.
+
+The solution is to use the cookbook ironfan provides -- it moves the 'ubuntu' user's home directory to an alternative path not masked by the NFS.
+
+
+### NFS: Problems starting NFS server on ubuntu maverick
+
+For problems starting NFS server on ubuntu maverick systems, read, understand and then run /tmp/fix_nfs_on_maverick_amis.sh -- See "this thread for more":http://fossplanet.com/f10/[ec2ubuntu]-not-starting-nfs-kernel-daemon-no-support-current-kernel-90948/
+
+
+### Git deploys: My git deploy recipe has gone limp
+
+Suppose you are using the @git@ resource to deploy a recipe (@george@ for sake of example). If @/var/chef/cache/revision_deploys/var/www/george@ exists then *nothing* will get deployed, even if /var/www/george/{release_sha} is empty or screwy.  If git deploy is acting up in any way, nuke that cache from orbit -- it's the only way to be sure.
+
+ $ sudo rm -rf /var/www/george/{release_sha} /var/chef/cache/revision_deploys/var/www/george
+
+### Runit services : 'fail: XXX: unable to change to service directory: file does not exist'
+
+Your service is probably installed but removed from runit's purview; check the `/etc/service` symlink. All of the following should be true:
+
+* directory `/etc/sv/foo`, containing file `run` and dirs `log` and `supervise`
+* `/etc/init.d/foo`  is symlinked to `/usr/bin/sv`
+* `/etc/servics/foo` is symlinked tp `/etc/sv/foo`

metadata

@@ -1,19 +1,19 @@
 --- !ruby/object:Gem::Specification
 name: ironfan
 version: !ruby/object:Gem::Version
-  version: 3.1.0.rc1
-  prerelease: 6
+  version: 3.1.1
+  prerelease: 
 platform: ruby
 authors:
 - Infochimps
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-02-18 00:00:00.000000000 Z
+date: 2012-02-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: chef
-  requirement: &70274003056380 !ruby/object:Gem::Requirement
+  requirement: &70189981680580 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -21,10 +21,10 @@ dependencies:
         version: 0.10.4
   type: :runtime
   prerelease: false
-  version_requirements: *70274003056380
+  version_requirements: *70189981680580
 - !ruby/object:Gem::Dependency
   name: fog
-  requirement: &70274003055860 !ruby/object:Gem::Requirement
+  requirement: &70189981668780 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -32,10 +32,10 @@ dependencies:
         version: 1.1.1
   type: :runtime
   prerelease: false
-  version_requirements: *70274003055860
+  version_requirements: *70189981668780
 - !ruby/object:Gem::Dependency
   name: formatador
-  requirement: &70274003074980 !ruby/object:Gem::Requirement
+  requirement: &70189981667980 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -43,10 +43,10 @@ dependencies:
         version: 0.2.1
   type: :runtime
   prerelease: false
-  version_requirements: *70274003074980
+  version_requirements: *70189981667980
 - !ruby/object:Gem::Dependency
   name: gorillib
-  requirement: &70274003073800 !ruby/object:Gem::Requirement
+  requirement: &70189981667180 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -54,10 +54,10 @@ dependencies:
         version: 0.1.7
   type: :runtime
   prerelease: false
-  version_requirements: *70274003073800
+  version_requirements: *70189981667180
 - !ruby/object:Gem::Dependency
   name: bundler
-  requirement: &70274003073000 !ruby/object:Gem::Requirement
+  requirement: &70189981666180 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -65,10 +65,10 @@ dependencies:
         version: '1'
   type: :development
   prerelease: false
-  version_requirements: *70274003073000
+  version_requirements: *70189981666180
 - !ruby/object:Gem::Dependency
   name: jeweler
-  requirement: &70274003072240 !ruby/object:Gem::Requirement
+  requirement: &70189981665440 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -76,10 +76,10 @@ dependencies:
         version: '1.6'
   type: :development
   prerelease: false
-  version_requirements: *70274003072240
+  version_requirements: *70189981665440
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &70274003071520 !ruby/object:Gem::Requirement
+  requirement: &70189981664920 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -87,10 +87,10 @@ dependencies:
         version: '2.5'
   type: :development
   prerelease: false
-  version_requirements: *70274003071520
+  version_requirements: *70189981664920
 - !ruby/object:Gem::Dependency
   name: yard
-  requirement: &70274003070540 !ruby/object:Gem::Requirement
+  requirement: &70189981664260 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -98,10 +98,10 @@ dependencies:
         version: '0.6'
   type: :development
   prerelease: false
-  version_requirements: *70274003070540
+  version_requirements: *70189981664260
 - !ruby/object:Gem::Dependency
   name: configliere
-  requirement: &70274003069740 !ruby/object:Gem::Requirement
+  requirement: &70189981660620 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -109,7 +109,7 @@ dependencies:
         version: 0.4.8
   type: :development
   prerelease: false
-  version_requirements: *70274003069740
+  version_requirements: *70189981660620
 description: ironfan allows you to orchestrate not just systems but clusters of machines.
   It includes a powerful layer on top of knife and a collection of cloud cookbooks.
 email: coders@infochimps.com
@@ -121,6 +121,7 @@ extra_rdoc_files:
 files:
 - .gitignore
 - .rspec
+- .yardopts
 - CHANGELOG.md
 - Gemfile
 - LICENSE.md
@@ -149,8 +150,12 @@ files:
 - lib/chef/knife/cluster_start.rb
 - lib/chef/knife/cluster_stop.rb
 - lib/chef/knife/cluster_sync.rb
-- lib/chef/knife/generic_command.rb
-- lib/chef/knife/knife_common.rb
+- lib/chef/knife/cluster_vagrant.rb
+- lib/chef/knife/ironfan_knife_common.rb
+- lib/chef/knife/ironfan_script.rb
+- lib/chef/knife/vagrant/ironfan_environment.rb
+- lib/chef/knife/vagrant/ironfan_provisioners.rb
+- lib/chef/knife/vagrant/skeleton_vagrantfile.rb
 - lib/ironfan.rb
 - lib/ironfan/chef_layer.rb
 - lib/ironfan/cloud.rb
@@ -167,6 +172,25 @@ files:
 - lib/ironfan/server.rb
 - lib/ironfan/server_slice.rb
 - lib/ironfan/volume.rb
+- notes/Home.md
+- notes/INSTALL-cloud_setup.md
+- notes/INSTALL.md
+- notes/Knife-Cluster-Commands.md
+- notes/Silverware.md
+- notes/aws_console_screenshot.jpg
+- notes/cookbook-versioning.md
+- notes/declaring_volumes.md
+- notes/design_notes-ci_testing.md
+- notes/design_notes-cookbook_event_ordering.md
+- notes/design_notes-dsl_object.md
+- notes/design_notes-meta_discovery.md
+- notes/ec2-pricing_and_capacity.md
+- notes/ironfan_homebase_layout.md
+- notes/named-cloud-objects.md
+- notes/rake_tasks.md
+- notes/renamed-recipes.txt
+- notes/style_guide.md
+- notes/tips_and_troubleshooting.md
 - spec/ironfan/cluster_spec.rb
 - spec/ironfan/facet_spec.rb
 - spec/ironfan/server_slice_spec.rb
@@ -192,13 +216,13 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 1652744719024657695
+      hash: -2335380690962978772
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
-  - - ! '>'
+  - - ! '>='
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.15