chef 0.9.8.beta.1 → 0.9.8.beta.2

data/distro/common/markdown/knife.mkd

@@ -11,26 +11,28 @@ This manual page documents knife, a command-line utility used to interact with a
 
 Unless otherwise specified, output is in JSON format, and input files are also JSON format.
 
+The Chef class `Chef::Config` that configures the behavior of how knife runs has options that correspond to command-line options. These are noted as `Chef::Config` values.
+
 ## GENERAL OPTIONS
 
   * `-s`, `--server-url` URL:
-    Chef Server URL
+    Chef Server URL, corresponds to `Chef::Config` `chef_server_url`.
   * `-k`, `--key` KEY:
-    API Client Key
+    API Client Key, corresponds to `Chef::Config` `client_key`.
   * `-c`, `--config` CONFIG:
     The configuration file to use
   * `-e`, `--editor` EDITOR:
     Set the editor to use for interactive commands
-  * `-f`, `--format` FORMAT:
+  * `-F`, `--format` FORMAT:
     Which format to use for output
   * `-l`, `--log_level` LEVEL:
-    Set the log level (debug, info, warn, error, fatal)
+    Set the log level (debug, info, warn, error, fatal), corresponds to `Chef::Config` `log_level`.
   * `-L`, `--logfile` LOGLOCATION:
-    Set the log file location, defaults to STDOUT
+    Set the log file location, defaults to STDOUT, corresponds to `Chef::Config` `log_location`.
   * `-n`, `--no-editor`:
     Do not open EDITOR, just accept the data as is
   * `-u`, `--user` USER:
-    API Client Username
+    API Client Username, corresponds to `Chef::Config` `node_name`.
   * `-p`, `--print-after`:
     Show the data after a destructive operation
   * `-v`, `--version`:
@@ -40,6 +42,8 @@ Unless otherwise specified, output is in JSON format, and input files are also J
   * `-h`, `--help`:
     Show this message
 
+Usage information for sub-commands can be displayed with `knife SUB-COMMAND --help`.
+
 ## SUB-COMMANDS
 
 Knife sub-commands are structured as _NOUN verb NOUN (options)_. The sub-commands are meant to be intuitively named. Because the Chef Server API is RESTful, sub-commands generally utilize CRUD operations.
@@ -64,6 +68,10 @@ __configure__ _(options)_
 
 Create a configuration file for knife. This will prompt for values to enter into the file. Default values are listed in square brackets if no other entry is typed. See __CONFIGURATION__ below for available options.
 
+__configure client DIRECTORY__
+
+Read the `knife.rb` config file and generate a config file suitable for use in `/etc/chef/client.rb` and copy the validation certificate into the specified _DIRECTORY_.
+
 __index rebuild__ _(options)_
 
   * `-y`, `--yes`:
@@ -105,7 +113,67 @@ __ssh QUERY COMMAND__ _(options)_
   * `-x`, `--ssh-user USERNAME    `:
     The ssh username
 
-The __ssh__ sub-command opens an ssh session to each of the nodes in the search results of the _QUERY_. This sub-command requires that the net-ssh-multi and highline Ruby libraries are installed. On Debian systems, these are the libnet-ssh-multi-ruby and libhighline-ruby packages. They can also be installed as RubyGems (net-ssh-multi and highline, respectively).
+The _ssh_ sub-command opens an ssh session to each of the nodes in the search results of the _QUERY_. This sub-command requires that the net-ssh-multi and highline Ruby libraries are installed. On Debian systems, these are the libnet-ssh-multi-ruby and libhighline-ruby packages. They can also be installed as RubyGems (net-ssh-multi and highline, respectively).
+
+## BOOTSTRAP SUB-COMMANDS
+
+__bootstrap FQDN__ _(options)_
+
+  * `-i`, `--identity-file IDENTITY_FILE`:
+    The SSH identity file used for authentication
+  * `-N`, `--node-name NAME`:
+    The Chef node name for your new node
+  * `-P`, `--ssh-password PASSWORD`:
+    The ssh password
+  * `-x`, `--ssh-user USERNAME`:
+    The ssh username
+  * `--prerelease`:
+    Install pre-release Chef gems
+  * `-r`, `--run-list RUN_LIST`:
+    Comma separated list of roles/recipes to apply
+  * `-P`, `--ssh-password PASSWORD`:
+    The ssh password
+  * `-x`, `--ssh-user USERNAME`:
+    The ssh username
+  * `--template-file TEMPLATE`:
+    Full path to location of template to use
+  * `--sudo`:
+    Execute the bootstrap via sudo
+  * `-d`, `--distro DISTRO`:
+    Bootstrap a distro using a template
+
+Performs a Chef Bootstrap on the target node. The goal of the bootstrap is to get Chef installed on the target system so it can run Chef Client with a Chef Server. The main assumption is a baseline OS installation exists. This sub-command is used internally by some cloud computing server create commands and the others will be migrated in a future version of Chef.
+
+As of Chef 0.9.8, the bootstrap sub-command supports supplying a template to perform the bootstrap steps. If the distro is not specified (via `-d` or `--distro` option), an Ubuntu 10.04 with RubyGems is assumed. The __DISTRO__ value corresponds to the base filename of the template, in other words `DISTRO`.erb. A template file can be specified with the `--template-file` option in which case the __DISTRO__ is not used. The sub-command looks in the following locations for the template to use:
+
+* `bootstrap` directory in the installed Chef Knife library.
+* `bootstrap` directory in the `$PWD/.chef`.
+* `bootstrap` directory in the users `$HOME/.chef`.
+
+The default bootstrap templates are scripts that get copied to the target node (FQDN). As of Chef 0.9.8, the following distros are supported:
+
+* centos5-gems
+* fedora13-gems
+* ubuntu10.04-gems
+* ubuntu10.04-apt
+
+The gems installations will use RubyGems 1.3.6 and Chef installed as a gem. The apt installation will use the Opscode APT repository. The RubyGems installation requires installing gems with native extensions, so development related packages (ruby-dev, build-essential) are installed. These are not installed with the apt installation, as native extensions are already compiled in the required packages.
+
+In addition to handling the software installation, these bootstrap templates do the following:
+
+  - Write the validation.pem per the local knife configuration.
+  - Write a default config file for Chef (`/etc/chef/client.rb`) using values from the `knife.rb`.
+  - Create a JSON attributes file containing the specified run list and run Chef.
+
+In the case of the RubyGems, the `client.rb` will be written from scratch with a minimal set of values; see __EXAMPLES__. In the case of APT Package installation, `client.rb` will have the `validation_client_name` appended if it is not set to `chef-validator` (default config value), and the `node_name` will be added if `chef_node_name` option is specified.
+
+When this is complete, the bootstrapped node will have:
+
+  - Latest Chef version installed from RubyGems or APT Packages from Opscode. This may be a later version than the local system.
+  - Be validated with the configured Chef Server.
+  - Have run Chef with its default run list if one is specfied.
+
+Additional custom bootstrap templates can be created and stored in `.chef/bootstrap/DISTRO.erb`, replacing __DISTRO__ with the value passed with the `-d` or `--distro` option. See __EXAMPLES__ for more information.
 
 ## CLIENT SUB-COMMANDS
 
@@ -166,11 +234,50 @@ __cookbook bulk delete REGEX__ _(options)_
 
 Delete cookbooks on the Chef Server based on a regular expression. The regular expression (_REGEX_) should be in quotes, not in //'s.
 
+__cookbook create COOKBOOK__ _(options)_
+
+  * `-o`, `--cookbook-path PATH`:
+    The directory where the cookbook will be created
+  * `-r`, `--readme-format FORMAT`:
+    Format of the README file
+  * `-C`, `--copyright COPYRIGHT`:
+    Name of Copyright holder
+  * `-I`, `--license LICENSE`:
+    License for cookbook, apachev2 or none
+  * `-E`, `--email EMAIL`:
+    Email address of cookbook maintainer
+
+This is a helper command that creates a new cookbook directory in the `cookbook_path`. The following directories and files are created for the named cookbook.
+
+* COOKBOOK/attributes
+* COOKBOOK/definitions
+* COOKBOOK/files/default
+* COOKBOOK/libraries
+* COOKBOOK/metadata.rb
+* COOKBOOK/providers
+* COOKBOOK/README.rdoc
+* COOKBOOK/recipes/default.rb
+* COOKBOOK/resources
+* COOKBOOK/templates/default
+
+Supported README formats are 'rdoc' (default), 'md', 'mkd', 'txt'. The README file will be written with the specified extension and a set of helpful starting headers.
+
+Specify `-C` or `--copyright` with the name of the copyright holder as your name or your company/organization name in a quoted string. If this value is not specified an all-caps string `YOUR_COMPANY_NAME` is used which can be easily changed with find/replace.
+
+Specify `-I` or `--license` with the license that the cookbook is distributed under for sharing with other people or posting to the Opscode Cookbooks site. Be aware of the licenses of files you put inside the cookbook and follow any restrictions they describe. When using `none` (default) or `apachev2`, comment header text and metadata file are pre-filled. The `none` license will be treated as non-redistributable.
+
+Specify `-E` or `--email` with the email address of the cookbook's maintainer. If this value is not specified, an all-caps string `YOUR_EMAIL` is used which can easily be changed with find/replace.
+
+The cookbook copyright, license and email settings can be filled in the `knife.rb`, for example with default values:
+
+    cookbook_copyright "YOUR_COMPANY_NAME"
+    cookbook_license "none"
+    cookbook_email "YOUR_EMAIL"
+
 __cookbook delete COOKBOOK [VERSION]__ _(options)_
 
   * `-a`, `--all`:
     Delete all versions
-
   * `-p`, `--purge`:
     Purge files from backing store. This will disable any cookbook that contains any of the same files as the cookbook being purged.
 
@@ -267,6 +374,21 @@ __cookbook site search QUERY__ _(options)_
 
 Searches the Community site with the specified query.
 
+__cookbook site share COOKBOOK CATEGORY__ _(options)_
+
+  * `-k`, `--key KEY`:
+    API Client Key
+  * `-u`, `--user USER`:
+    API Client Username
+  * `-o`, `--cookbook-path PATH:PATH`:
+    A colon-separated path to look for cookbooks in
+
+Uploads the specified cookbook using the given category to the Opscode cookbooks site. Requires a login user and certificate for the Opscode Cookbooks site. See __EXAMPLES__ for usage if the Opscode user and certificate pair are not used for authenticating with the Chef Server. In other words, if the Chef Server is not the Opscode Platform.
+
+__cookbook site unshare COOKBOOK__
+
+Stops sharing the specified cookbook on the Opscode cookbooks site.
+
 __cookbook site show COOKBOOK [VERSION]__ _(options)_
 
 Shows information from the site about a particular cookbook.
@@ -276,7 +398,7 @@ __cookbook site vendor COOKBOOK [VERSION]__ _(options)_
   * `-d`, `--dependencies`:
     Grab dependencies automatically
 
-Downloads a cookbook and untars it in the cookbooks directory. If _-d_ is specified, all the cookbooks it depends on (via metadata _dependencies_) are downloaded and untarred as well.
+Uses `git` version control in conjunction with the cookbook site to download upstream cookbooks. A new vendor branch is created in git, the cookbook downloaded from the site and untarred, then the master branch is merged. This allows the user to track upstream changes to cookbooks while merging in customizations. If _-d_ is specified, all the cookbooks it depends on (via metadata _dependencies_) are downloaded and untarred as well, each using their own vendor branch.
 
 ## DATA BAG SUB-COMMANDS
 
@@ -361,6 +483,7 @@ __node show NODE__ _(options)_
 Show a node.
 
 ## RECIPE SUB-COMMANDS
+
 __recipe list [PATTERN]__
 
 List the recipes available on the server. The results shown can be limited with the optional PATTERN, which is a regular expression. PATTERN should be given in quotes, without slashes.
@@ -406,38 +529,11 @@ Show a specific role.
 
 ## CLOUD COMPUTING SUB-COMMANDS
 
-The next sections describe sub-commands to work with various Cloud Computing APIs to launch server instances with validation and run-time configuration that Chef knows about. These sub-commands require the fog Ruby library. On Debian and Ubuntu systems this is the package `libfog-ruby`. It is also available as a RubyGem, `fog`.
-
-The Rackspace and Terremark sub-commands are early stage of development. Right now when creating instances of these types, knife will assume an Ubuntu image, install Ruby from packages, RubyGems from source and Chef as a RubyGem. As these commands are developed, other installation styles may become available, as options.
-
-__bootstrap FQDN [RUN LIST...]__ _(options)_
-
-  * `-N`, `--node-name NAME`:
-    The Chef node name for your new node
-  * `-P`, `--ssh-password PASSWORD`:
-    The ssh password
-  * `-x`, `--ssh-user USERNAME`:
-    The ssh username
-  * `--prerelease`:
-    Install pre-release Chef gems
-
-Performs a Chef Bootstrap on the target node. This subcommand is used internally by the various cloud computing server create commands. Currently assumes an Ubuntu base OS image and will take the following actions:
+The next sections describe sub-commands to work with various Cloud Computing APIs to launch server instances with validation and run-time configuration that Chef knows about. These sub-commands require the fog Ruby library. On Debian and Ubuntu systems with Opscode's apt repository (apt.opscode.com), this is the package `libfog-ruby`. It is also available as a RubyGem, `fog`.
 
-  - Updates APT cache.
-  - Installs Ruby and packages required to build RubyGems with native extensions.
-  - Installs RubyGems 1.3.6 **from source**.
-  - Installs Chef and Ohai RubyGems.
-  - Writes the validation.pem per the local knife configuration.
-  - Writes a default config file for Chef.
-  - Creates a JSON attributes file containing the node's run list and runs Chef.
+The Rackspace and Terremark server creation sub-commands are at an early stage of development. They do not yet use the `bootstrap` sub-command to install Chef, and make a few assumptions to be aware. Right now when creating instances of these types, knife will assume an Ubuntu image, install Ruby from packages, RubyGems from source and Chef as a RubyGem. As these commands are developed further, they will utilize the `bootstrap` sub-command.
 
-When this is complete, the bootstrapped node will have:
-
-  - Latest Chef version installed from RubyGems.
-  - Be validated with the configured Chef Server.
-  - Have run Chef with its default run list.
-
-Future versions of this sub-command will support performing Chef Bootstrap on other platforms.
+In order to use knife with the various __CLOUD COMPUTING SUB-COMMANDS__, credentials need to be added to the configuration file. See __CONFIGURATION__.
 
 ## EC2 SUB-COMMANDS
 
@@ -454,6 +550,8 @@ __ec2 server create [RUN LIST...]__ _(options)_
 
   * `-Z`, `--availability-zone ZONE`:
     The Availability Zone
+  * `--region`:
+    Your AWS region
   * `-A`, `--aws-access-key-id KEY `:
     Your AWS Access Key ID
   * `-K SECRET`, `--aws-secret-access-key`:
@@ -462,16 +560,24 @@ __ec2 server create [RUN LIST...]__ _(options)_
     The flavor of server (m1.small, m1.medium, etc)
   * `-i`, `--image IMAGE`:
     The AMI for the server
-  * `--prerelease`:
-    Install the pre-release chef gems
   * `-G`, `--groups X,Y,Z     `:
     The security groups for this server
   * `-S`, `--ssh-key KEY      `:
-    The SSH root key
+    The SSH root key, corresponds to an Amazon Keypair.
+  * `-I`, `--identity-file IDENTITY_FILE`:
+    The SSH identity file used for authentication, passed to `bootstrap`.
+  * `-P`, `--ssh-password PASSWORD`:
+    The ssh password, passed to `bootstrap`.
   * `-x`, `--ssh-user USERNAME`:
-    The ssh username
+    The ssh username, passed to `bootstrap`.
+  * `--prerelease`:
+    Install pre-release Chef gems, passed to `bootstrap`.
+  * `--template-file TEMPLATE`:
+    Full path to location of template to use, passed to `bootstrap`.
+  * `-d`, `--distro DISTRO`:
+    Bootstrap a distro using a template, passed to `bootstrap`.
 
-Creates a new Amazon AWS EC2 instance.
+Creates a new Amazon AWS EC2 instance and bootstraps it by calling the `bootstrap` sub-command. The `[RUN LIST...]` items are passed to the bootstrap's `run_list` config parameter/option. See the __BOOTSTRAP SUB-COMMANDS__ section above for more information.
 
 __ec2 server delete SERVER [SERVER]__ _(options)_
 
@@ -518,6 +624,34 @@ __rackspace server list__ _(options)_
 
 Lists running Rackspace Cloud servers.
 
+## SLICEHOST SUB-COMMANDS
+
+As above, Chef can also be used on Slicehost nodes. The following sub-commands allow manipulating Slicehost nodes via the `fog` library.
+
+__slicehost server create [RUN LIST...]__ _(options)_
+  * `-f`, `--flavor FLAVOR`:
+    The flavor of server
+  * `-i`, `--image IMAGE`:
+    The image of the server
+  * `-N`, `--server-name NAME`:
+    The server name
+  * `-K`, `--slicehost-password PASSWORD`:
+    Your slicehost API password
+
+Creates a new slicehost server.
+
+__slicehost server list__ _(options)_
+
+Lists running Slicehost servers.
+
+__slicehost server delete SLICENAME__
+
+Deletes a running Slicehost server.
+
+__slicehost images list__
+
+Lists the available Slicehost server images to boot.
+
 ## TERREMARK SUB-COMMANDS
 
 As above, Chef can also be used on Terremark vCloud nodes. The following sub-commands allow manipulating Terremark vCloud nodes via the `fog` library.
@@ -595,12 +729,83 @@ Specifies the name of the client used to validate new clients. This is requested
 
 Specifies the private key file to use for generating ec2 instance data for validating new clients. This is implied from the `validation_client_name`.
 
+`cookbook_copyright`
+`cookbook_email`
+`cookbook_license`
+
+Used by `knife cookbook create` sub-command to specify the copyright holder, maintainer email and license (respectively) for new cookbooks. The copyright holder is listed as the maintainer in the cookbook's metadata and as the Copyright in the comments of the default recipe. The maintainer email is used in the cookbook metadata. The license determines what preamble to put in the comment of the default recipe, and is listed as the license in the cookbook metadata. Currently supported licenses are "apachev2" and "none". Any other values will result in an empty license in the metadata (needs to be filled in by the author), and no comment preamble in the default recipe.
+
+`knife[:aws_access_key_id]`
+`knife[:aws_secret_access_key]`
+
+Specifies the Amazon AWS EC2 credentials to use when running the ec2 sub-commands.
+
+`knife[:rackspace_api_username]`
+`knife[:rackspace_api_key]`
+
+Specifies the Rackspace Cloud credentials to use when running the rackspace sub-commands.
+
+`knife[:terremark_username]`
+`knife[:terremark_password]`
+`knife[:terremark_service]`
+
+Specifies the Terremark vCloud credentials to use when running the terremark sub-commands.
+
+`knife[:slicehost_password]`
+
+Specifies the Slicehost password to use when running the slicdehost sub-commands.
+
 ## FILES
 
 _~/.chef/knife.rb_
 
 Ruby DSL configuration file for knife. See __CONFIGURATION__.
 
+## CHEF WORKFLOW
+
+When working with Chef and Knife in the local repository, the recommended workflow outline looks like:
+
+* Create repository. A skeleton sample is provided at _http://github.com/opscode/chef-repo/_.
+* Configure knife, see __CONFIGURATION__.
+* Download cookbooks from the Opscode cookbooks site, see __COOKBOOK SITE SUB-COMMANDS__.
+* Or, create new cookbooks, see `cookbook create` sub-command.
+* Commit changes to the version control system. See your tool's documentation.
+* Upload cookbooks to the Chef Server, see __COOKBOOK SUB-COMMANDS__.
+* Launch instances in the Cloud, OR provision new hosts; see __CLOUD COMPUTING SUB-COMMANDS__ and __BOOTSTRAP SUB-COMMANDS__.
+* Watch Chef configure systems!
+
+A note about git: Opscode and many folks in the Chef community use git, but it is not required, except in the case of the `cookbook site vendor` sub-command, as it uses git directly. Version control is strongly recommended though, and git fits with a lot of the workflow paradigms.
+
+## EXAMPLES
+
+Example client config (`/etc/chef/client.rb`) from `knife configure client`. The same configuration is used when using the `knife bootstrap` command with the default `gem` templates that come with Chef.
+
+    log_level        :info
+    log_location     STDOUT
+    chef_server_url  'https://api.opscode.com/organizations/ORGNAME'
+    validation_client_name 'ORGNAME-validator'
+
+Setting up a custom bootstrap is fairly straightforward. Create `.chef/bootstrap` in your Chef Repository directory or in `$HOME/.chef/bootstrap`. Then create the ERB template file.
+
+    mkdir ~/.chef/bootstrap
+    vi ~/.chef/bootstrap/debian5.0-apt.erb
+
+For example, to create a new bootstrap template that should be used when setting up a new Debian node. Edit the template to run the commands, set up the validation certificate and the client configuration file, and finally to run chef-client on completion. The bootstrap template can be called with:
+
+    knife bootstrap mynode.example.com --template-file ~/.chef/bootstrap/debian5.0-apt.erb
+
+Or,
+
+    knife bootstrap mynode.example.com --distro debian5.0-apt
+
+The `--distro` parameter will automatically look in the `~/.chef/bootstrap` directory for a file named `debian5.0-apt.erb`.
+
+Templates provided by the Chef installation are located in `BASEDIR/lib/chef/knife/bootstrap/*.erb`, where _BASEDIR_ is the location where the package or Gem installed the Chef client libraries.
+
+Uploading cookbooks to the Opscode cookbooks site using the user/certificate specifically:
+
+    knife cookbook site share example Other -k ~/.chef/USERNAME.pem -u USERNAME
+
 ## SEE ALSO
 
 Full documentation for Chef is located on the Chef wiki, http://wiki.opscode.com/display/chef/Home/.
@@ -609,6 +814,8 @@ JSON is JavaScript Object Notation and more information can be found at http://j
 
 SOLR is an open source search engine. The Chef Server includes a SOLR installation. More information about SOLR, including the search query syntax, can be found at http://lucene.apache.org/solr/.
 
+Git is a version control system and documented at http://git-scm.com/.
+
 This manual page was generated in nroff from Markdown with ronn. Ryan Tomayko wrote ronn and more information can be found at http://rtomayko.github.com/ronn/ronn.5.html.
 
 ## AUTHOR

data/lib/chef/application/knife.rb

@@ -89,6 +89,10 @@ class Chef::Application::Knife < Chef::Application
     :long => "--yes",
     :description => "Say yes to all prompts for confirmation"
 
+  option :defaults,
+    :long => "--defaults",
+    :description => "Accept default values for all questions"
+
   option :print_after,
     :short => "-p",
     :long => "--print-after",

data/lib/chef/data_bag.rb

@@ -129,11 +129,11 @@ class Chef
     
     def self.list(inflate=false)
       if inflate
-        response = Hash.new
-        Chef::Search::Query.new.search(:data) do |n|
-          response[n.name] = n
+        # Can't search for all data bags like other objects, fall back to N+1 :(
+        list(false).inject({}) do |response, bag_and_uri|
+          response[bag_and_uri.first] = load(bag_and_uri.first)
+          response
         end
-        response
       else
         Chef::REST.new(Chef::Config[:chef_server_url]).get_rest("data")
       end

data/lib/chef/data_bag_item.rb

@@ -211,7 +211,15 @@ class Chef
     
     # As a string
     def to_s
-      "data_bag_item[#{@name}]"
+      "data_bag_item[#{id}]"
+    end
+
+    def pretty_print(pretty_printer)
+      pretty_printer.pp({"data_bag_item('#{data_bag}', '#{id}')" => self.to_hash})
+    end
+
+    def id
+      @raw_data['id']
     end
 
   end