cheftacular 2.6.0 → 2.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c5bcc816f796287851b09a9aae84f8c0c870d0a5
-  data.tar.gz: b88ae2e25f2732f9204f5d1fe425290472cfb0a0
+  metadata.gz: 4afb51eace0ceed81a2347f805bc51c4b11a98a4
+  data.tar.gz: b1bf5947b4cca8298cd296804b93c4790a84524c
 SHA512:
-  metadata.gz: cea832a9a9c55ab8bd3f20a3b8ea76cedc7fcb7227d55437eb510ae8047912bab1f04d46f79492058d1af2c85f415c611d8fd9bba8d1a92d939de61dc6950eaa
-  data.tar.gz: 197bfb84446358b86921378c0777fec8b8542988df3e0b0fe78d361517105db64e61832cb1fc44a3c19e068739a31ecb11d9f467788b77405f4d6240ceb29e52
+  metadata.gz: dd80d668f3015552e00ffef30f6a2460a94f647d6e2578675bb93e610d3156ba6c6bda4707e6baa8b57eb9eee74265055d9e9e1a302025f94274f0b28cd1a7b8
+  data.tar.gz: b78d57f3493f92d659642c3d77931f89c8d6d0ab27b4a85bb0089b31f30f0ebafa1126fde55ddc5c24d79d0009ea4ca33980dedf9aad8219d43a43ca7c92bf2d

data/lib/cheftacular/README.md

@@ -62,7 +62,15 @@
 
 1. [NYI]`cft get_active_ssh_connections` will fetch the active ssh connections from every server and output it into your log directory.
 
-2. `cft backup [activate|deactivate]` this command sets the fetch_backups and restore_backups flags in your config data bag for an environment. These can be used to give application developers a way to trigger / untrigger restores in an environment
+2. `cft backups [activate|deactivate|load|run]` this command sets the fetch_backups and restore_backups flags in your config data bag for an environment. These can be used to give application developers a way to trigger / untrigger restores in an environment
+
+    1. `activate` will turn on automated backup running (turns on the flag for the env in the config bag).
+
+    2. `deactivate` will turn off automated backup running.
+
+    3. `load` will fetch the latest backup from the production primary **if it doesn't already exist on the server** and run the _backup loading command_ to load this backup into the env.
+
+    4. `run` will simply just run the _backup loading command_ to load the latest backup onto the server.
 
 3. `cft check` Checks the commits for all servers for a repository (for an environment) and returns them in a simple chart. Also shows when these commits were deployed to the server.
 
@@ -78,7 +86,15 @@
 
     5. NOTE 2! Cheftacular does not (and will not) support accessing your chef server over ssh with password auth. If you have done this, you should feel bad and immediately switch ssh access to key authentication...
 
-5. `cft client_list` Allows you check the basic information for all the servers setup via chef. Shows the server's short name, its public ip address and roles (run_list) by default.
+5. `cft cheftacular_config [display|sync|overwrite]` this command Allows you to interact with your complete cheftacular configuration, the union of all repository's cheftacular.ymls. 
+
+    1. `display` will show the current overall configuration for cheftacular.
+
+    2. `sync` will sync your local cheftacular yaml keys ONTO the server's keys. This happens automatically whenever a difference is detected between the local keys and the remote keys but can be run manually. Will send a slack notification if slack is configured (the slack notification contains the diffed keys). The sync only occurs if there are CHANGES to the file.
+
+6. `cft clear_caches` this command allows you to clear all of your local caches.
+
+7. `cft client_list` Allows you check the basic information for all the servers setup via chef. Shows the server's short name, its public ip address and roles (run_list) by default.
 
     1. `-v` option will make this command display the server's domain name, whether its password is stored on the chef server and what that password is.
 
@@ -86,13 +102,13 @@
 
     3. This command is aliased to `client-list` with no arguments or cft prefix.
 
-6. `cft console` will create a pry session on the first node found for a codebase.
+8. `cft console` will create a pry session on the first node found for a codebase.
 
-7. `cft db_console` will create a database console session on the first node found for a database stack in the current environment.
+9. `cft db_console` will create a database console session on the first node found for a database stack in the current environment.
 
     1. This command is aliased to psql, typing `cft psql` will drop you into a rails stack database psql session.
 
-8. `cft deploy` will do a simple chef-client run on the servers for a role. Logs of the run itself will be sent to the local log directory in the application (or chef-repo) where the run was conducted.
+10. `cft deploy` will do a simple chef-client run on the servers for a role. Logs of the run itself will be sent to the local log directory in the application (or chef-repo) where the run was conducted.
 
     1.  This command also restarts services on the server and updates the code. Changes behavior slightly with the `-z|-Z` args but only if your cookbooks support switching revisions based on tags / branch names.
 
@@ -100,9 +116,9 @@
 
     3. The `-v|--verbose` option will cause failed deploys to output to the terminal window and to their normal log file. Useful for debugging.
 
-9. `cft disk_report` will fetch useful statistics from every server for every environment and output it into your log directory.
+11. `cft disk_report` will fetch useful statistics from every server for every environment and output it into your log directory.
 
-10. `cft environment boot|destroy` will boot / destroy the current environment
+12. `cft environment boot|destroy` will boot / destroy the current environment
 
     1. `boot` will spin up servers and bring them to a stable state. This includes setting up their subdomains for the target environment.
 
@@ -110,7 +126,7 @@
 
     3. This command will prompt when attempting to destroy servers in staging or production
 
-11. `cft file NODE_NAME LOCATION_ALIAS MODE FILE_NAME` interacts with a file on the remote server
+13. `cft file NODE_NAME LOCATION_ALIAS MODE FILE_NAME` interacts with a file on the remote server
 
     1. `LOCATION_ALIAS` will be parsed as a path if it has backslash characters. Otherwise it will be parsed from your location_aliases hash in your cheftacular.yml
 
@@ -140,25 +156,25 @@
 
         1. `--save-to-file FILE_PATH` can also be used in the `fetch` context to specify where exactly to save the file and what to name it as.
 
-12. `cft fix_known_hosts [HOSTNAME]` this command will delete entries in your known_hosts file for all the servers that are in our system (ip addresses AND dns names)
+14. `cft fix_known_hosts [HOSTNAME]` this command will delete entries in your known_hosts file for all the servers that are in our system (ip addresses AND dns names)
 
     1. Passing in a hostname will make the command only remove entries with that hostname / ip specifically
 
-13. `cft get_haproxy_log` this command will generate a haproxy html file for the load balancer(s) associated with a repository in the log directory. Opening this log file in the browser will show the status of that haproxy at the time of the log. 
+15. `cft get_haproxy_log` this command will generate a haproxy html file for the load balancer(s) associated with a repository in the log directory. Opening this log file in the browser will show the status of that haproxy at the time of the log. 
 
     1. In devops mode, this command will not do anything without the -R repository passed.
 
-14. `cft get_log_from_bag <NODE_NAME-COMMAND_TYPE>` this command grabs the latest command run log from the data bags and saves it to your log directory. There are different types of logs saved per server depending on command.
+16. `cft get_log_from_bag <NODE_NAME-COMMAND_TYPE>` this command grabs the latest command run log from the data bags and saves it to your log directory. There are different types of logs saved per server depending on command.
 
-15. `cft get_pg_pass ['clip']` command will output the current environment's pg_password to your terminal. Optionally you can pass in clip like `cft get_pg_pass clip` to have it also copy the pass to your clipboard.
+17. `cft get_pg_pass ['clip']` command will output the current environment's pg_password to your terminal. Optionally you can pass in clip like `cft get_pg_pass clip` to have it also copy the pass to your clipboard.
 
-16. `cft help COMMAND|MODE` this command returns the documentation for a specific command if COMMAND matches the name of a command. Alternatively, it can be passed `action|arguments|application|current|devops|stateless_action` to fetch the commands for a specific mode.Misspellings of commands will display near hits.
+18. `cft help COMMAND|MODE` this command returns the documentation for a specific command if COMMAND matches the name of a command. Alternatively, it can be passed `action|arguments|application|current|devops|stateless_action` to fetch the commands for a specific mode.Misspellings of commands will display near hits.
 
-17. `cft list_toggleable_roles NODE_NAME` This command will allow you to see all toggleable roles for a node
+19. `cft list_toggleable_roles NODE_NAME` This command will allow you to see all toggleable roles for a node
 
-18. `cft location_aliases` will list all location aliases listed in your cheftacular.yml. These aliases can be used in the `cft file` command.
+20. `cft location_aliases` will list all location aliases listed in your cheftacular.yml. These aliases can be used in the `cft file` command.
 
-19. `cft log` this command will output the last 500 lines of logs from every server set for CODEBASE (can be given additional args to specify) to the log directory
+21. `cft log` this command will output the last 500 lines of logs from every server set for CODEBASE (can be given additional args to specify) to the log directory
 
     1.  `--nginx` will fetch the nginx logs as well as the application logs
 
@@ -170,17 +186,17 @@
 
     4. `--fetch-backup` If doing a pg_data log, this will fetch the latest logs from the pg_data log directory for each database.
 
-20. `cft migrate` this command will grab the first alphabetical node for a repository and run a migration that will hit the database primary server.
+22. `cft migrate` this command will grab the first alphabetical node for a repository and run a migration that will hit the database primary server.
 
-21. `cft pass NODE_NAME` will drop the server's sudo password into your clipboard. Useful for when you need to ssh into the server itself and try advanced linux commands
+23. `cft pass NODE_NAME` will drop the server's sudo password into your clipboard. Useful for when you need to ssh into the server itself and try advanced linux commands
 
-22. `cft reinitialize IP_ADDRESS NODE_NAME` will reconnect a server previously managed by chef to a new chef server. The node name MUST MATCH THE NODE'S ORIGINAL NODE NAME for the roles to be setup correctly.
+24. `cft reinitialize IP_ADDRESS NODE_NAME` will reconnect a server previously managed by chef to a new chef server. The node name MUST MATCH THE NODE'S ORIGINAL NODE NAME for the roles to be setup correctly.
 
-23. `cft role_toggle NODE_NAME ROLE_NAME activate|deactivate` This command will allow you to **toggle** roles on nodes without using `cft upload_nodes`
+25. `cft role_toggle NODE_NAME ROLE_NAME activate|deactivate` This command will allow you to **toggle** roles on nodes without using `cft upload_nodes`
 
     1. This command uses your *role_toggling:deactivated_role_suffix* attribute set in your cheftacular.yml to toggle the role, it checks to see if the toggled name exists then sets the node's run_list to include the toggled role
 
-    2. EX: `cft role_toggle apisc01 worker activate` will find the node apisc01 and attempt to toggle the worker role to on. If the node does NOT have the worker_deactivate role, then it will add it if *role_toggling:strict_roles* is set to **false**
+    2. EX: `cft role_toggle api01 worker activate` will find the node api01 and attempt to toggle the worker role to on. If the node does NOT have the worker_deactivate role, then it will add it if *role_toggling:strict_roles* is set to **false**
 
         1. If *role_toggling:strict_roles* is set to true, then cheftacular would raise an error saying this role is unsettable on the node. On the other hand, if the node already has the worker_deactivaterole, then this command will succeed even if *strict_roles* is set.
 
@@ -188,7 +204,7 @@
 
         1. Please run `cft list_toggleable_roles NODE_NAME` to get a list of your org's toggleable roles for a node.
 
-24. `cft run COMMAND [--all]` will trigger the command on the first server in the role. Can be used to run rake commands or anything else.
+26. `cft run COMMAND [--all]` will trigger the command on the first server in the role. Can be used to run rake commands or anything else.
 
     1. `--all` will make the command run against all servers in a role rather than the first server it comes across. Don't do this if you're modifying the database with the command.
 
@@ -198,16 +214,18 @@
 
     4. IMPORTANT NOTE: You cannot run `cft run rake -T` as is, you have to enclose any command that uses command line dash arguments in quotes like `cft run "rake -T"`
 
-25. `cft scale up|down [NUM_TO_SCALE]` will add (or remove) NUM_TO_SCALE servers from the server array. This command will not let you scale down below 1 server.
+27. `cft scale up|down [NUM_TO_SCALE]` will add (or remove) NUM_TO_SCALE servers from the server array. This command will not let you scale down below 1 server.
 
     1. In the case of server creation, this command takes a great deal of time to execute. It will output what stage it is currently on to the terminal but <b>you must not kill this command while it is executing</b>.A failed build may require the server to be destroyed / examined by a DevOps engineer.
 
-26. `cft tail` will tail the logs (return continuous output) of the first node if finds that has an application matching the repository running on it. Currently only supports rails stacks
+28. `cft tail [PATTERN_TO_MATCH]` will tail the logs (return continuous output) of the first node if finds that has an application matching the repository running on it. Currently only supports rails stacks
 
     1. pass `-n NODE_NAME` to grab the output of a node other than the first.
 
     2. Workers and job servers change the output of this command heavily. Worker and job servers should tail their log to the master log (/var/log/syslog) where <b>all</b> of the major processes on the server output to. While the vast majority of this syslog will be relevant to application developers, some will not (usually firewall blocks and the like).
 
+    3. if the `PATTERN_TO_MATCH` argument exists, the tail will only return entries that have that pattern rather than everything written to the file.
+
 
 ## Commands that can ONLY be run in the devops context
 
@@ -221,7 +239,15 @@
 
     1. `SPECIFIC_REPOSITORY` is a special argument, if left blank the key will be placed in the authorized_keys array in the bag, otherwise it will be placed in the specific_authorized_keys hash under a key named for the repository that is passed. The script will error if SPECIFIC_REPOSITORY does not exist in the cheftacular.yml respositories hash. You can then use this data to give users selective ssh access to certain servers.
 
-5. `cft backup [activate|deactivate]` this command sets the fetch_backups and restore_backups flags in your config data bag for an environment. These can be used to give application developers a way to trigger / untrigger restores in an environment
+5. `cft backups [activate|deactivate|load|run]` this command sets the fetch_backups and restore_backups flags in your config data bag for an environment. These can be used to give application developers a way to trigger / untrigger restores in an environment
+
+    1. `activate` will turn on automated backup running (turns on the flag for the env in the config bag).
+
+    2. `deactivate` will turn off automated backup running.
+
+    3. `load` will fetch the latest backup from the production primary **if it doesn't already exist on the server** and run the _backup loading command_ to load this backup into the env.
+
+    4. `run` will simply just run the _backup loading command_ to load the latest backup onto the server.
 
 6. `cft check_cheftacular_yml_keys` allows you to check to see if your cheftacular yml keys are valid to the current version of cheftacular. It will also set your missing keys to their likely default and let you know to update the cheftacular.yml file.
 
@@ -239,13 +265,21 @@
 
     5. NOTE 2! Cheftacular does not (and will not) support accessing your chef server over ssh with password auth. If you have done this, you should feel bad and immediately switch ssh access to key authentication...
 
-9. `cft clean_cookbooks [force] [remove_cookbooks]` allows you to update the internal chef-repo's cookbooks easily. By default this script will force you to decide what to do with each cookbook individually (shows version numbers and whether to overwrite it to cookbooks or not).
+9. `cft cheftacular_config [display|sync|overwrite]` this command Allows you to interact with your complete cheftacular configuration, the union of all repository's cheftacular.ymls. 
+
+    1. `display` will show the current overall configuration for cheftacular.
+
+    2. `sync` will sync your local cheftacular yaml keys ONTO the server's keys. This happens automatically whenever a difference is detected between the local keys and the remote keys but can be run manually. Will send a slack notification if slack is configured (the slack notification contains the diffed keys). The sync only occurs if there are CHANGES to the file.
+
+10. `cft clean_cookbooks [force] [remove_cookbooks]` allows you to update the internal chef-repo's cookbooks easily. By default this script will force you to decide what to do with each cookbook individually (shows version numbers and whether to overwrite it to cookbooks or not).
 
     1. `force` argument will cause the downloaded cookbooks to *always* overwrite the chef-repo's cookbooks as long as the downloaded cookbook has a higher version number.
 
     2. If you would like to remove all the cookbooks on the chef server, run `knife cookbook bulk delete '.*' -p -c ~/.chef/knife.rb`
 
-10. `cft client_list` Allows you check the basic information for all the servers setup via chef. Shows the server's short name, its public ip address and roles (run_list) by default.
+11. `cft clear_caches` this command allows you to clear all of your local caches.
+
+12. `cft client_list` Allows you check the basic information for all the servers setup via chef. Shows the server's short name, its public ip address and roles (run_list) by default.
 
     1. `-v` option will make this command display the server's domain name, whether its password is stored on the chef server and what that password is.
 
@@ -253,7 +287,7 @@
 
     3. This command is aliased to `client-list` with no arguments or cft prefix.
 
-11. `cft cloud <FIRST_LEVEL_ARG> [<SECOND_LEVEL_ARG>[:<SECOND_LEVEL_ARG_QUERY>]*] ` this command handles talking to various cloud APIs. If no args are passed nothing will happen.
+13. `cft cloud <FIRST_LEVEL_ARG> [<SECOND_LEVEL_ARG>[:<SECOND_LEVEL_ARG_QUERY>]*] ` this command handles talking to various cloud APIs. If no args are passed nothing will happen.
 
     1. `domain` first level argument for interacting with cloud domains
 
@@ -349,7 +383,7 @@
 
         5. `bootstrap` captures the current computer's hostname and checks to see if a key matching this hostname exists on the cloud service. If the key does not exist, the command attempts to read the contents of the ~/.ssh/id_rsa.pub file and create a new key with that data and the hostname of the current computer. Run automatically when creating DigitalOcean servers. It's worth noting that if the computer's key already exists on DigitalOcean under a different name, this specific command will fail with a generic error. Please check your keys.
 
-12. `cft cloud_bootstrap NODE_NAME FLAVOR_NAME [DESCRIPTOR] [--with-dn DOMAIN]` uses a cloud api to create a server and attaches its DOMAIN_NAME to the TLD specified for that environment (IE: example-staging.com for staging)
+14. `cft cloud_bootstrap NODE_NAME FLAVOR_NAME [DESCRIPTOR] [--with-dn DOMAIN]` uses a cloud api to create a server and attaches its DOMAIN_NAME to the TLD specified for that environment (IE: example-staging.com for staging)
 
     1. If no DOMAIN_NAME is supplied it will use the node's NODE_NAME (IE: api01.example-staging.com)
 
@@ -359,11 +393,11 @@
 
     4. DESCRIPTOR is used as an internal tag for the node, if left blank it will become the name of the node. It is recommended to enter a custom repository-dependent tag here to make nodes easier to load-balance like "lb:[CODEBASE_NAME]"
 
-13. `cft compile_audit_log [clean]` compiles the audit logs in each environment's audit data bag a audit-log-CURRENTDAY.md file in the log folder of the application. Bear in mind that the bag can only hold 100K bytes and will need to have that data removed to store more than that.
+15. `cft compile_audit_log [clean]` compiles the audit logs in each environment's audit data bag a audit-log-CURRENTDAY.md file in the log folder of the application. Bear in mind that the bag can only hold 100K bytes and will need to have that data removed to store more than that.
 
-14. `cft compile_readme` compiles all documentation methods and creates a README.md file in the log folder of the application.
+16. `cft compile_readme` compiles all documentation methods and creates a README.md file in the log folder of the application.
 
-15. `cft create_git_key ID_RSA_FILE [OAUTH_TOKEN]` This command will update the default/authentication data bag with new credentials. The [ID_RSA_FILE](https://help.github.com/articles/generating-ssh-keys) needs to exist beforehand.
+17. `cft create_git_key ID_RSA_FILE [OAUTH_TOKEN]` This command will update the default/authentication data bag with new credentials. The [ID_RSA_FILE](https://help.github.com/articles/generating-ssh-keys) needs to exist beforehand.
 
     1. This command will upload both the private and public key to the data bag. The public key should be the one that matches the github user for your deployment github user.
 
@@ -371,9 +405,9 @@
 
     3. NOTE! The ID_RSA_FILE should be in your .chef folder in the root of your home directory!
 
-16. `cft disk_report` will fetch useful statistics from every server for every environment and output it into your log directory.
+18. `cft disk_report` will fetch useful statistics from every server for every environment and output it into your log directory.
 
-17. `cft environment boot|destroy` will boot / destroy the current environment
+19. `cft environment boot|destroy` will boot / destroy the current environment
 
     1. `boot` will spin up servers and bring them to a stable state. This includes setting up their subdomains for the target environment.
 
@@ -381,7 +415,7 @@
 
     3. This command will prompt when attempting to destroy servers in staging or production
 
-18. `cft file NODE_NAME LOCATION_ALIAS MODE FILE_NAME` interacts with a file on the remote server
+20. `cft file NODE_NAME LOCATION_ALIAS MODE FILE_NAME` interacts with a file on the remote server
 
     1. `LOCATION_ALIAS` will be parsed as a path if it has backslash characters. Otherwise it will be parsed from your location_aliases hash in your cheftacular.yml
 
@@ -411,21 +445,21 @@
 
         1. `--save-to-file FILE_PATH` can also be used in the `fetch` context to specify where exactly to save the file and what to name it as.
 
-19. `cft fix_known_hosts [HOSTNAME]` this command will delete entries in your known_hosts file for all the servers that are in our system (ip addresses AND dns names)
+21. `cft fix_known_hosts [HOSTNAME]` this command will delete entries in your known_hosts file for all the servers that are in our system (ip addresses AND dns names)
 
     1. Passing in a hostname will make the command only remove entries with that hostname / ip specifically
 
-20. `cft full_bootstrap ADDRESS ROOT_PASS NODE_NAME` This command performs both ubuntu_bootstrap and chef_bootstrap.
+22. `cft full_bootstrap ADDRESS ROOT_PASS NODE_NAME` This command performs both ubuntu_bootstrap and chef_bootstrap.
 
-21. `cft get_haproxy_log` this command will generate a haproxy html file for the load balancer(s) associated with a repository in the log directory. Opening this log file in the browser will show the status of that haproxy at the time of the log. 
+23. `cft get_haproxy_log` this command will generate a haproxy html file for the load balancer(s) associated with a repository in the log directory. Opening this log file in the browser will show the status of that haproxy at the time of the log. 
 
     1. In devops mode, this command will not do anything without the -R repository passed.
 
-22. `cft get_log_from_bag <NODE_NAME-COMMAND_TYPE>` this command grabs the latest command run log from the data bags and saves it to your log directory. There are different types of logs saved per server depending on command.
+24. `cft get_log_from_bag <NODE_NAME-COMMAND_TYPE>` this command grabs the latest command run log from the data bags and saves it to your log directory. There are different types of logs saved per server depending on command.
 
-23. `cft get_pg_pass ['clip']` command will output the current environment's pg_password to your terminal. Optionally you can pass in clip like `cft get_pg_pass clip` to have it also copy the pass to your clipboard.
+25. `cft get_pg_pass ['clip']` command will output the current environment's pg_password to your terminal. Optionally you can pass in clip like `cft get_pg_pass clip` to have it also copy the pass to your clipboard.
 
-24. `cft get_shorewall_allowed_connections [PATH_TO_LOCAL_FILE] -n NODE_NAME` command will query a single server and return all of its ACCEPT connections from shorewall in it's syslog and return the results in a CSV format. Useful for tracking IP activity.
+26. `cft get_shorewall_allowed_connections [PATH_TO_LOCAL_FILE] -n NODE_NAME` command will query a single server and return all of its ACCEPT connections from shorewall in it's syslog and return the results in a CSV format. Useful for tracking IP activity.
 
     1. You must pass in a node name to query with `-n NODE_NAME`
 
@@ -433,35 +467,41 @@
 
     3. If `PATH_TO_LOCAL_FILE` is not blank, the command will use that file instead of building a file on the remote server
 
-25. `cft help COMMAND|MODE` this command returns the documentation for a specific command if COMMAND matches the name of a command. Alternatively, it can be passed `action|arguments|application|current|devops|stateless_action` to fetch the commands for a specific mode.Misspellings of commands will display near hits.
+27. `cft help COMMAND|MODE` this command returns the documentation for a specific command if COMMAND matches the name of a command. Alternatively, it can be passed `action|arguments|application|current|devops|stateless_action` to fetch the commands for a specific mode.Misspellings of commands will display near hits.
+
+28. `cft initialize_cheftacular_yml [application|TheCheftacularCookbook]` will create a cheftacular.yml file in your config folder (and create the config folder if it does not exist). If you already have a cheftacular.yml file in the config folder, it will create a cheftacular.example.yml file that will contain the new changes / keys in the latest cheftacular version.
 
-26. `cft initialize_cheftacular_yml` will create a cheftacular.yml file in your config folder (and create theconfig folder if it does not exist). If you already have a cheftacular.yml file in the config folder, it will create a cheftacular.example.yml file that will contain the new changes / keys in the latest cheftacular version.
+    1. If `TheCheftacularCookbook` is passed, the generated cheftacular.yml file will include the additional TheCheftacularCookbook keys.
 
-27. `cft initialize_data_bag_contents ENVIRONMENT_NAME` will ensure the data bags always have the correct structure before each run. This command is run every time the gem is started and if called directly, will exit after completion.
+    2. If `application` is passed, the generated cheftacular.yml file will look like one you could use in an application directory.
 
-28. `cft knife_upload` will resync the chef-server with the local chef-repo code. This command is analog for `knife upload /`
+29. `cft initialize_data_bag_contents ENVIRONMENT_NAME` will ensure the data bags always have the correct structure before each run. This command is run every time the gem is started and if called directly, will exit after completion.
 
-29. `cft list_toggleable_roles NODE_NAME` This command will allow you to see all toggleable roles for a node
+30. `cft knife_upload` will resync the chef-server with the local chef-repo code. This command is analog for `knife upload /`
 
-30. `cft location_aliases` will list all location aliases listed in your cheftacular.yml. These aliases can be used in the `cft file` command.
+31. `cft list_toggleable_roles NODE_NAME` This command will allow you to see all toggleable roles for a node
 
-31. `cft pass NODE_NAME` will drop the server's sudo password into your clipboard. Useful for when you need to ssh into the server itself and try advanced linux commands
+32. `cft location_aliases` will list all location aliases listed in your cheftacular.yml. These aliases can be used in the `cft file` command.
 
-32. `cft remove_client NODE_NAME [destroy]` removes a client (and its node data) from the chef-server. It also removes its dns records from the cloud service (if possible). This should not be done lightly as you will have to wipe the server and trigger another chef-client run to get it to register again. Alternatively, you can run `cft reinitialize IP_ADDRESS NODE_NAME as well.
+33. `cft pass NODE_NAME` will drop the server's sudo password into your clipboard. Useful for when you need to ssh into the server itself and try advanced linux commands
+
+34. `cft remove_client NODE_NAME [destroy]` removes a client (and its node data) from the chef-server. It also removes its dns records from the cloud service (if possible). This should not be done lightly as you will have to wipe the server and trigger another chef-client run to get it to register again. Alternatively, you can run `cft reinitialize IP_ADDRESS NODE_NAME as well.
 
     1. `destroy` deletes the server as well as removing it from the chef environment.
 
-33. `cft replication_status` will check the status of the database master and slaves in every environment. Also lists how far behind the slaves are from the master in milliseconds.
+35. `cft replication_status` will check the status of the database master and slaves in every environment. Also lists how far behind the slaves are from the master in milliseconds.
+
+36. `cft reset_bag BAG_NAME` this command allows you to reset a data bag item to an empty state. Run this on full data bags to clear them out. 
 
-34. `cft restart_swap` will restart the swap on every server that doesn't have swap currently on. Useful if you notice servers with no swap activated from `cft disk_report`
+37. `cft restart_swap` will restart the swap on every server that doesn't have swap currently on. Useful if you notice servers with no swap activated from `cft disk_report`
 
     1. There is no risk in running this command. Sometimes swap doesnt reactivate if the server was rebooted and this command fixes that.
 
-35. `cft role_toggle NODE_NAME ROLE_NAME activate|deactivate` This command will allow you to **toggle** roles on nodes without using `cft upload_nodes`
+38. `cft role_toggle NODE_NAME ROLE_NAME activate|deactivate` This command will allow you to **toggle** roles on nodes without using `cft upload_nodes`
 
     1. This command uses your *role_toggling:deactivated_role_suffix* attribute set in your cheftacular.yml to toggle the role, it checks to see if the toggled name exists then sets the node's run_list to include the toggled role
 
-    2. EX: `cft role_toggle apisc01 worker activate` will find the node apisc01 and attempt to toggle the worker role to on. If the node does NOT have the worker_deactivate role, then it will add it if *role_toggling:strict_roles* is set to **false**
+    2. EX: `cft role_toggle api01 worker activate` will find the node api01 and attempt to toggle the worker role to on. If the node does NOT have the worker_deactivate role, then it will add it if *role_toggling:strict_roles* is set to **false**
 
         1. If *role_toggling:strict_roles* is set to true, then cheftacular would raise an error saying this role is unsettable on the node. On the other hand, if the node already has the worker_deactivaterole, then this command will succeed even if *strict_roles* is set.
 
@@ -469,7 +509,7 @@
 
         1. Please run `cft list_toggleable_roles NODE_NAME` to get a list of your org's toggleable roles for a node.
 
-36. `cft rvm [COMMAND] [ADDITIONAL_COMMANDS]*` will run rvm commands on the remote servers. Output from this command for each server will go into your rvm directory under the log directory. Please refer to [the rvm help page](https://rvm.io/rvm) for more information on rvm commands.
+39. `cft rvm [COMMAND] [ADDITIONAL_COMMANDS]*` will run rvm commands on the remote servers. Output from this command for each server will go into your rvm directory under the log directory. Please refer to [the rvm help page](https://rvm.io/rvm) for more information on rvm commands.
 
     1. When no commands are passed, rvm will just run `rvm list` on each server on all servers in the current environment.
 
@@ -485,11 +525,11 @@
 
     7. `upgrade_rvm` will run `rvm get stable --auth-dotfiles` on all servers for the current environment. It will also check and attempt to upgrade pre 1.25 installations of RVM to 1.26+ (which requires a GPG key).
 
-37. `cft server_update [restart]` allows you to force update all nodes' packages for a specific environment. This should be done with caution as this *might* break something.
+40. `cft server_update [restart]` allows you to force update all nodes' packages for a specific environment. This should be done with caution as this *might* break something.
 
     1. `cft apt_update restart` will prompt to ask if you also want to restart all servers in a rolling restart. This should be done with extreme caution and only in a worst-case scenario.
 
-38. `cft service [COMMAND] [SERVICE]` will run service commands on remote servers. This command only runs on the first server it comes across. Specify others with -n NODE_NAME.
+41. `cft service [COMMAND] [SERVICE]` will run service commands on remote servers. This command only runs on the first server it comes across. Specify others with -n NODE_NAME.
 
     1. When no commands are passed, the command will list all the services in the /etc/init directory
 
@@ -497,13 +537,13 @@
 
     3. When `restart|stop|start SERVICE` is passed, the command will attempt to restart|stop|start the service if it has a .conf file on the remote server in the /etc/init directory.
 
-39. `cft slack "MESSAGE" [CHANNEL]` will attempt to post the message to the webhook set in your cheftacular.yml. Slack posts to your default channel by default but if the CHANNEL argument is supplied the message will post there.
+42. `cft slack "MESSAGE" [CHANNEL]` will attempt to post the message to the webhook set in your cheftacular.yml. Slack posts to your default channel by default but if the CHANNEL argument is supplied the message will post there.
 
     1. NOTE: To prevent confusing spam from many possible sources, the username posted to slack will always be *Cheftacular*. This can be overloaded in the StatelessAction method "slack" but this is not recommended.
 
     2. Remember, if you have auditing turned on in your cheftacular.yml, you can track who sends what to slack.
 
-40. `cft test_env [TARGET_ENV] boot|destroy` will create (or destroy) the test nodes for a particular environment (defaults to staging, prod split-envs can be set with `-p`). Please read below for how TARGET_ENV works
+43. `cft test_env [TARGET_ENV] boot|destroy` will create (or destroy) the test nodes for a particular environment (defaults to staging, prod split-envs can be set with `-p`). Please read below for how TARGET_ENV works
 
     1. TARGET_ENV changes functionality depending on the overall (like staging / production) environment
 
@@ -513,9 +553,9 @@
 
         3. The default tld used should change depending on which environment you are booting / destroying. This is set in the environment's config data bag under the tld key
 
-41. `cft ubuntu_bootstrap ADDRESS ROOT_PASS` This command will bring a fresh server to a state where chef-client can be run on it via `cft chef-bootstrap`. It should be noted that it is in this step where a server's randomized deploy_user sudo password is generated.
+44. `cft ubuntu_bootstrap ADDRESS ROOT_PASS` This command will bring a fresh server to a state where chef-client can be run on it via `cft chef-bootstrap`. It should be noted that it is in this step where a server's randomized deploy_user sudo password is generated.
 
-42. `cft update_cloudflare_dns_from_cloud [skip_update_tld]` command will force a full dns update for cloudflare. 
+45. `cft update_cloudflare_dns_from_cloud [skip_update_tld]` command will force a full dns update for cloudflare. 
 
     1. It will ensure all the subdomain entries are correct (based on the contents of the addresses data bag) and update them if they are not. It will also create the local subdomain for the entry as well if it does exist and point it to the correct private address for an environment.
 
@@ -523,7 +563,7 @@
 
     3. The argument `skip_update_tld` will stop the long process of checking and updating all the server domains _before_ cloudflare is updated. Only skip if you believe your domain info on your cloud is accurate.
 
-43. `cft update_split_branches` will perform a series of git commands that will merge all the split branches for your split_branch enabled repositories with what is currently on master and push them.
+46. `cft update_split_branches` will perform a series of git commands that will merge all the split branches for your split_branch enabled repositories with what is currently on master and push them.
 
     1. Repository must be set with `-R REPOSITORY_NAME` for this command to work.
 
@@ -533,9 +573,9 @@
 
     4. This command will return a helpful error statement if you attempt to run the command with changes to your current working directory. You must commit these changes before running this command.
 
-44. `cft update_tld TLD` command will force a full dns update for a tld in the preferred cloud. It will ensure all the subdomain entries are correct (based on the contents of the addresses data bag) and update them if they are not. It will also create the local subdomain for the entry as well if it does exist and point it to the correct private address.
+47. `cft update_tld TLD` command will force a full dns update for a tld in the preferred cloud. It will ensure all the subdomain entries are correct (based on the contents of the addresses data bag) and update them if they are not. It will also create the local subdomain for the entry as well if it does exist and point it to the correct private address.
 
-45. `cft upload_nodes` This command will resync the chef server's nodes with the data in our chef-repo/node_roles. 
+48. `cft upload_nodes` This command will resync the chef server's nodes with the data in our chef-repo/node_roles. 
 
     1. This command changes behavior depending on several factors about both your mode and the state of your environment
 
@@ -547,4 +587,4 @@
 
         1. Due to this, only users running this against their chef-repo need to worry about having a nodes_dir, the way it should be.
 
-46. `cft upload_roles` This command will resync the chef server's roles with the data in the chef-repo/roles.
+49. `cft upload_roles` This command will resync the chef server's roles with the data in the chef-repo/roles.

data/lib/cheftacular/actions/tail.rb

@@ -2,7 +2,7 @@ class Cheftacular
   class ActionDocumentation
     def tail
       @config['documentation']['action'] <<  [
-        "`cft tail` will tail the logs (return continuous output) of the first node if finds " + 
+        "`cft tail [PATTERN_TO_MATCH]` will tail the logs (return continuous output) of the first node if finds " + 
         "that has an application matching the repository running on it. Currently only supports rails stacks",
 
         [
@@ -10,15 +10,18 @@ class Cheftacular
 
           "    2. Workers and job servers change the output of this command heavily. " +
           "Worker and job servers should tail their log to the master log (/var/log/syslog) where <b>all</b> of the major processes on the server output to. " +
-          "While the vast majority of this syslog will be relevant to application developers, some will not (usually firewall blocks and the like)."
+          "While the vast majority of this syslog will be relevant to application developers, some will not (usually firewall blocks and the like).",
+
+          "    3. if the `PATTERN_TO_MATCH` argument exists, the tail will only return entries that have that pattern rather than everything written to the file."
         ]
       ]
     end
   end
 
   class Action
-    #TODO ARG FOR TAILING ANY LOG FILE
-    def tail
+    def tail pattern_to_match=''
+      pattern_to_match = ARGV[1] if pattern_to_match.blank?
+
       nodes = @config['getter'].get_true_node_objects
 
       nodes = @config['parser'].exclude_nodes( nodes, [{ unless: "role[#{ @options['role'] }]" }], true )
@@ -27,28 +30,32 @@ class Cheftacular
         puts("Beginning tail run for #{ n.name } (#{ n.public_ipaddress }) on role #{ @options['role'] }") unless @options['quiet']
 
         if @config['dummy_sshkit'].has_run_list_in_role_map?(n.run_list, @config['cheftacular']['role_maps'])
-          start_tail_role_map( n.public_ipaddress, n.run_list )
+          start_tail_role_map( n.public_ipaddress, n.run_list, pattern_to_match )
         else
-          self.send("start_tail_#{ @config['getter'].get_current_stack }", n.public_ipaddress, n.run_list )
+          self.send("start_tail_#{ @config['getter'].get_current_stack }", n.public_ipaddress, n.run_list, pattern_to_match )
         end
       end
     end
 
     private
 
-    def start_tail_role_map ip_address, run_list
+    def start_tail_role_map ip_address, run_list, pattern_to_match
       log_loc = @config['getter'].get_current_role_map(run_list)['log_location'].split(',').first.gsub('|current_repo_location|', "#{ @config['cheftacular']['base_file_path'] }/#{ @options['repository'] }/current")
 
-      `ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc }" > /dev/tty`
+      `ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc } #{ get_tail_grep_string(pattern_to_match) }" > /dev/tty`
     end
 
-    def start_tail_ruby_on_rails ip_address, run_list
+    def start_tail_ruby_on_rails ip_address, run_list, pattern_to_match
       true_env = @config['dummy_sshkit'].get_true_environment run_list, @config['cheftacular']['run_list_environments'][@options['env']], @options['env']
 
       #special servers should be listed first as most of them will have web role
       log_loc = "#{ @config['cheftacular']['base_file_path'] }/#{ @options['repository'] }/current/log/#{ true_env }.log"
       
-      `ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc }" > /dev/tty`
+      `ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc } #{ get_tail_grep_string(pattern_to_match) }" > /dev/tty`
+    end
+
+    def get_tail_grep_string pattern_to_match=''
+      "| grep -i -E #{ pattern_to_match }" unless pattern_to_match.blank?
     end
   end
 end

data/lib/cheftacular/chef/data_bag.rb

@@ -7,18 +7,18 @@ class Cheftacular
 
     #this will only intialize bags (and their hashes) if they don't exist. Use ridley data bag methods to reload the data etc
     def init_bag bag_env, bag_name, encrypted=true
-      self.instance_eval("@config['ridley'].data_bag.create(name: '#{ bag_env }')") if self.instance_eval("@config['ridley'].data_bag.find('#{ bag_env }').nil?")
+      @config['ridley'].data_bag.create(name: bag_env) if @config['ridley'].data_bag.find(bag_env).nil?
 
-      if self.instance_eval("@config['ridley'].data_bag.find('#{ bag_env }').item.find('#{ bag_name }').nil?")
-        self.instance_eval("@config['ridley'].data_bag.find('#{ bag_env }').item.create(id: '#{ bag_name }')")
+      if @config['ridley'].data_bag.find(bag_env).item.find(bag_name).nil?
+        @config['ridley'].data_bag.find(bag_env).item.create(id: bag_name)
       end
 
       @config[bag_env] ||= {}
 
       if !@config[bag_env].has_key?("#{ bag_name }_bag") || !@config[bag_env].has_key?("#{ bag_name }_bag_hash")
-        self.instance_eval "@config['#{ bag_env }']['#{ bag_name }_bag'] ||= @config['ridley'].data_bag.find('#{ bag_env }').item.find('#{ bag_name }')"
+        @config[bag_env]["#{ bag_name }_bag"] = @config['ridley'].data_bag.find(bag_env).item.find(bag_name)
 
-        self.instance_eval "@config['#{ bag_env }']['#{ bag_name }_bag_hash'] ||= @config['#{ bag_env }']['#{ bag_name }_bag']#{ encrypted ? '.decrypt' : '' }.to_hash"
+        self.instance_eval "@config['#{ bag_env }']['#{ bag_name }_bag_hash'] = @config['#{ bag_env }']['#{ bag_name }_bag']#{ encrypted ? '.decrypt' : '' }.to_hash"
       end
     end
 
@@ -46,7 +46,6 @@ class Cheftacular
       end
     end
 
-    #TODO special save for bag that will compile the data into a different bag for storage (the data will be stored as an audit log and zlib'd)
     def save_audit_bag bag_env="options"
       env = bag_env == 'options' ? @options['env'] : bag_env
 
@@ -57,6 +56,14 @@ class Cheftacular
       save_bag 'authentication', bag_env, @config['default']['authentication_bag'], @config['default']['authentication_bag_hash'], true
     end
 
+    def save_cheftacular_bag bag_env="default"
+      save_bag 'cheftacular', bag_env, @config['default']['cheftacular_bag'], @config['default']['cheftacular_bag_hash']
+    end
+
+    def save_environment_config_bag bag_env='default'
+      save_bag 'environment_config', bag_env, @config['default']['environment_config_bag'], @config['default']['environment_config_bag_hash']
+    end
+
     def save_chef_passwords_bag bag_env="options"
       env = bag_env == 'options' ? @options['env'] : bag_env
 
@@ -87,13 +94,46 @@ class Cheftacular
       save_bag 'node_roles', bag_env, @config[env]['node_roles_bag'], @config[env]['node_roles_bag_hash']
     end
 
+    def reset_addresses_bag bag_env="options"
+      env = bag_env == 'options' ? @options['env'] : bag_env
+
+      reset_bag 'addresses', env
+    end
+
+    def reset_audit_bag bag_env='options'
+      env = bag_env == 'options' ? @options['env'] : bag_env
+
+      reset_bag 'audit', env
+    end
+
+    def reset_cheftacular_bag bag_env="default"
+      reset_bag 'cheftacular', bag_env
+    end
+
+    def reset_environment_config_bag bag_env='default'
+      reset_bag 'environment_config', bag_env
+    end
+
+    def reset_node_roles_bag bag_env="options"
+      env = bag_env == 'options' ? @options['env'] : bag_env
+
+      reset_bag 'node_roles', env
+    end
+
     private
+    def reset_bag bag_name, bag_env, encrypted=false
+      raise "Cannot reset bag #{ bag_name } in #{ bag_env } as it does not exist!" if @config['ridley'].data_bag.find(bag_env).item.find(bag_name).nil?
+
+      @config['ridley'].data_bag.find(bag_env).item.delete(bag_name)
+
+      init_bag bag_env, bag_name, encrypted
+    end
+
     def save_bag bag_name, bag_env, bag, bag_hash, encrypted=false
       return true if @config['helper'].running_on_chef_node?
 
-      new_bag_hash =  bag_hash.deep_dup
-
-      item = bag.reload
+      new_bag_hash = bag_hash.deep_dup
+      item         = bag.reload
 
       load_hash = encrypted ? item.decrypt.to_hash.deep_merge(new_bag_hash) : item.attributes.deep_merge(new_bag_hash)