cheftacular 2.6.0 → 2.7.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/lib/cheftacular/README.md +104 -64
- data/lib/cheftacular/actions/tail.rb +17 -10
- data/lib/cheftacular/chef/data_bag.rb +49 -9
- data/lib/cheftacular/cheftacular.rb +6 -2
- data/lib/cheftacular/file_system.rb +26 -5
- data/lib/cheftacular/getter.rb +7 -0
- data/lib/cheftacular/helper.rb +29 -0
- data/lib/cheftacular/initializer.rb +43 -2
- data/lib/cheftacular/parser.rb +4 -8
- data/lib/cheftacular/queue_master.rb +16 -0
- data/lib/cheftacular/stateless_actions/backups.rb +169 -58
- data/lib/cheftacular/stateless_actions/check_cheftacular_yml_keys.rb +9 -0
- data/lib/cheftacular/stateless_actions/cheftacular_config.rb +51 -0
- data/lib/cheftacular/stateless_actions/clear_caches.rb +24 -0
- data/lib/cheftacular/stateless_actions/client_list.rb +1 -1
- data/lib/cheftacular/stateless_actions/create_git_key.rb +2 -1
- data/lib/cheftacular/stateless_actions/fix_known_hosts.rb +1 -1
- data/lib/cheftacular/stateless_actions/initialize_cheftacular_yml.rb +16 -6
- data/lib/cheftacular/stateless_actions/initialize_data_bag_contents.rb +43 -0
- data/lib/cheftacular/stateless_actions/reset_bag.rb +30 -0
- data/lib/cheftacular/stateless_actions/role_toggle.rb +1 -1
- data/lib/cheftacular/stateless_actions/update_tld.rb +1 -3
- data/lib/cheftacular/version.rb +1 -1
- metadata +6 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4afb51eace0ceed81a2347f805bc51c4b11a98a4
|
4
|
+
data.tar.gz: b1bf5947b4cca8298cd296804b93c4790a84524c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: dd80d668f3015552e00ffef30f6a2460a94f647d6e2578675bb93e610d3156ba6c6bda4707e6baa8b57eb9eee74265055d9e9e1a302025f94274f0b28cd1a7b8
|
7
|
+
data.tar.gz: b78d57f3493f92d659642c3d77931f89c8d6d0ab27b4a85bb0089b31f30f0ebafa1126fde55ddc5c24d79d0009ea4ca33980dedf9aad8219d43a43ca7c92bf2d
|
data/lib/cheftacular/README.md
CHANGED
@@ -62,7 +62,15 @@
|
|
62
62
|
|
63
63
|
1. [NYI]`cft get_active_ssh_connections` will fetch the active ssh connections from every server and output it into your log directory.
|
64
64
|
|
65
|
-
2. `cft
|
65
|
+
2. `cft backups [activate|deactivate|load|run]` this command sets the fetch_backups and restore_backups flags in your config data bag for an environment. These can be used to give application developers a way to trigger / untrigger restores in an environment
|
66
|
+
|
67
|
+
1. `activate` will turn on automated backup running (turns on the flag for the env in the config bag).
|
68
|
+
|
69
|
+
2. `deactivate` will turn off automated backup running.
|
70
|
+
|
71
|
+
3. `load` will fetch the latest backup from the production primary **if it doesn't already exist on the server** and run the _backup loading command_ to load this backup into the env.
|
72
|
+
|
73
|
+
4. `run` will simply just run the _backup loading command_ to load the latest backup onto the server.
|
66
74
|
|
67
75
|
3. `cft check` Checks the commits for all servers for a repository (for an environment) and returns them in a simple chart. Also shows when these commits were deployed to the server.
|
68
76
|
|
@@ -78,7 +86,15 @@
|
|
78
86
|
|
79
87
|
5. NOTE 2! Cheftacular does not (and will not) support accessing your chef server over ssh with password auth. If you have done this, you should feel bad and immediately switch ssh access to key authentication...
|
80
88
|
|
81
|
-
5. `cft
|
89
|
+
5. `cft cheftacular_config [display|sync|overwrite]` this command Allows you to interact with your complete cheftacular configuration, the union of all repository's cheftacular.ymls.
|
90
|
+
|
91
|
+
1. `display` will show the current overall configuration for cheftacular.
|
92
|
+
|
93
|
+
2. `sync` will sync your local cheftacular yaml keys ONTO the server's keys. This happens automatically whenever a difference is detected between the local keys and the remote keys but can be run manually. Will send a slack notification if slack is configured (the slack notification contains the diffed keys). The sync only occurs if there are CHANGES to the file.
|
94
|
+
|
95
|
+
6. `cft clear_caches` this command allows you to clear all of your local caches.
|
96
|
+
|
97
|
+
7. `cft client_list` Allows you check the basic information for all the servers setup via chef. Shows the server's short name, its public ip address and roles (run_list) by default.
|
82
98
|
|
83
99
|
1. `-v` option will make this command display the server's domain name, whether its password is stored on the chef server and what that password is.
|
84
100
|
|
@@ -86,13 +102,13 @@
|
|
86
102
|
|
87
103
|
3. This command is aliased to `client-list` with no arguments or cft prefix.
|
88
104
|
|
89
|
-
|
105
|
+
8. `cft console` will create a pry session on the first node found for a codebase.
|
90
106
|
|
91
|
-
|
107
|
+
9. `cft db_console` will create a database console session on the first node found for a database stack in the current environment.
|
92
108
|
|
93
109
|
1. This command is aliased to psql, typing `cft psql` will drop you into a rails stack database psql session.
|
94
110
|
|
95
|
-
|
111
|
+
10. `cft deploy` will do a simple chef-client run on the servers for a role. Logs of the run itself will be sent to the local log directory in the application (or chef-repo) where the run was conducted.
|
96
112
|
|
97
113
|
1. This command also restarts services on the server and updates the code. Changes behavior slightly with the `-z|-Z` args but only if your cookbooks support switching revisions based on tags / branch names.
|
98
114
|
|
@@ -100,9 +116,9 @@
|
|
100
116
|
|
101
117
|
3. The `-v|--verbose` option will cause failed deploys to output to the terminal window and to their normal log file. Useful for debugging.
|
102
118
|
|
103
|
-
|
119
|
+
11. `cft disk_report` will fetch useful statistics from every server for every environment and output it into your log directory.
|
104
120
|
|
105
|
-
|
121
|
+
12. `cft environment boot|destroy` will boot / destroy the current environment
|
106
122
|
|
107
123
|
1. `boot` will spin up servers and bring them to a stable state. This includes setting up their subdomains for the target environment.
|
108
124
|
|
@@ -110,7 +126,7 @@
|
|
110
126
|
|
111
127
|
3. This command will prompt when attempting to destroy servers in staging or production
|
112
128
|
|
113
|
-
|
129
|
+
13. `cft file NODE_NAME LOCATION_ALIAS MODE FILE_NAME` interacts with a file on the remote server
|
114
130
|
|
115
131
|
1. `LOCATION_ALIAS` will be parsed as a path if it has backslash characters. Otherwise it will be parsed from your location_aliases hash in your cheftacular.yml
|
116
132
|
|
@@ -140,25 +156,25 @@
|
|
140
156
|
|
141
157
|
1. `--save-to-file FILE_PATH` can also be used in the `fetch` context to specify where exactly to save the file and what to name it as.
|
142
158
|
|
143
|
-
|
159
|
+
14. `cft fix_known_hosts [HOSTNAME]` this command will delete entries in your known_hosts file for all the servers that are in our system (ip addresses AND dns names)
|
144
160
|
|
145
161
|
1. Passing in a hostname will make the command only remove entries with that hostname / ip specifically
|
146
162
|
|
147
|
-
|
163
|
+
15. `cft get_haproxy_log` this command will generate a haproxy html file for the load balancer(s) associated with a repository in the log directory. Opening this log file in the browser will show the status of that haproxy at the time of the log.
|
148
164
|
|
149
165
|
1. In devops mode, this command will not do anything without the -R repository passed.
|
150
166
|
|
151
|
-
|
167
|
+
16. `cft get_log_from_bag <NODE_NAME-COMMAND_TYPE>` this command grabs the latest command run log from the data bags and saves it to your log directory. There are different types of logs saved per server depending on command.
|
152
168
|
|
153
|
-
|
169
|
+
17. `cft get_pg_pass ['clip']` command will output the current environment's pg_password to your terminal. Optionally you can pass in clip like `cft get_pg_pass clip` to have it also copy the pass to your clipboard.
|
154
170
|
|
155
|
-
|
171
|
+
18. `cft help COMMAND|MODE` this command returns the documentation for a specific command if COMMAND matches the name of a command. Alternatively, it can be passed `action|arguments|application|current|devops|stateless_action` to fetch the commands for a specific mode.Misspellings of commands will display near hits.
|
156
172
|
|
157
|
-
|
173
|
+
19. `cft list_toggleable_roles NODE_NAME` This command will allow you to see all toggleable roles for a node
|
158
174
|
|
159
|
-
|
175
|
+
20. `cft location_aliases` will list all location aliases listed in your cheftacular.yml. These aliases can be used in the `cft file` command.
|
160
176
|
|
161
|
-
|
177
|
+
21. `cft log` this command will output the last 500 lines of logs from every server set for CODEBASE (can be given additional args to specify) to the log directory
|
162
178
|
|
163
179
|
1. `--nginx` will fetch the nginx logs as well as the application logs
|
164
180
|
|
@@ -170,17 +186,17 @@
|
|
170
186
|
|
171
187
|
4. `--fetch-backup` If doing a pg_data log, this will fetch the latest logs from the pg_data log directory for each database.
|
172
188
|
|
173
|
-
|
189
|
+
22. `cft migrate` this command will grab the first alphabetical node for a repository and run a migration that will hit the database primary server.
|
174
190
|
|
175
|
-
|
191
|
+
23. `cft pass NODE_NAME` will drop the server's sudo password into your clipboard. Useful for when you need to ssh into the server itself and try advanced linux commands
|
176
192
|
|
177
|
-
|
193
|
+
24. `cft reinitialize IP_ADDRESS NODE_NAME` will reconnect a server previously managed by chef to a new chef server. The node name MUST MATCH THE NODE'S ORIGINAL NODE NAME for the roles to be setup correctly.
|
178
194
|
|
179
|
-
|
195
|
+
25. `cft role_toggle NODE_NAME ROLE_NAME activate|deactivate` This command will allow you to **toggle** roles on nodes without using `cft upload_nodes`
|
180
196
|
|
181
197
|
1. This command uses your *role_toggling:deactivated_role_suffix* attribute set in your cheftacular.yml to toggle the role, it checks to see if the toggled name exists then sets the node's run_list to include the toggled role
|
182
198
|
|
183
|
-
2. EX: `cft role_toggle
|
199
|
+
2. EX: `cft role_toggle api01 worker activate` will find the node api01 and attempt to toggle the worker role to on. If the node does NOT have the worker_deactivate role, then it will add it if *role_toggling:strict_roles* is set to **false**
|
184
200
|
|
185
201
|
1. If *role_toggling:strict_roles* is set to true, then cheftacular would raise an error saying this role is unsettable on the node. On the other hand, if the node already has the worker_deactivaterole, then this command will succeed even if *strict_roles* is set.
|
186
202
|
|
@@ -188,7 +204,7 @@
|
|
188
204
|
|
189
205
|
1. Please run `cft list_toggleable_roles NODE_NAME` to get a list of your org's toggleable roles for a node.
|
190
206
|
|
191
|
-
|
207
|
+
26. `cft run COMMAND [--all]` will trigger the command on the first server in the role. Can be used to run rake commands or anything else.
|
192
208
|
|
193
209
|
1. `--all` will make the command run against all servers in a role rather than the first server it comes across. Don't do this if you're modifying the database with the command.
|
194
210
|
|
@@ -198,16 +214,18 @@
|
|
198
214
|
|
199
215
|
4. IMPORTANT NOTE: You cannot run `cft run rake -T` as is, you have to enclose any command that uses command line dash arguments in quotes like `cft run "rake -T"`
|
200
216
|
|
201
|
-
|
217
|
+
27. `cft scale up|down [NUM_TO_SCALE]` will add (or remove) NUM_TO_SCALE servers from the server array. This command will not let you scale down below 1 server.
|
202
218
|
|
203
219
|
1. In the case of server creation, this command takes a great deal of time to execute. It will output what stage it is currently on to the terminal but <b>you must not kill this command while it is executing</b>.A failed build may require the server to be destroyed / examined by a DevOps engineer.
|
204
220
|
|
205
|
-
|
221
|
+
28. `cft tail [PATTERN_TO_MATCH]` will tail the logs (return continuous output) of the first node if finds that has an application matching the repository running on it. Currently only supports rails stacks
|
206
222
|
|
207
223
|
1. pass `-n NODE_NAME` to grab the output of a node other than the first.
|
208
224
|
|
209
225
|
2. Workers and job servers change the output of this command heavily. Worker and job servers should tail their log to the master log (/var/log/syslog) where <b>all</b> of the major processes on the server output to. While the vast majority of this syslog will be relevant to application developers, some will not (usually firewall blocks and the like).
|
210
226
|
|
227
|
+
3. if the `PATTERN_TO_MATCH` argument exists, the tail will only return entries that have that pattern rather than everything written to the file.
|
228
|
+
|
211
229
|
|
212
230
|
## Commands that can ONLY be run in the devops context
|
213
231
|
|
@@ -221,7 +239,15 @@
|
|
221
239
|
|
222
240
|
1. `SPECIFIC_REPOSITORY` is a special argument, if left blank the key will be placed in the authorized_keys array in the bag, otherwise it will be placed in the specific_authorized_keys hash under a key named for the repository that is passed. The script will error if SPECIFIC_REPOSITORY does not exist in the cheftacular.yml respositories hash. You can then use this data to give users selective ssh access to certain servers.
|
223
241
|
|
224
|
-
5. `cft
|
242
|
+
5. `cft backups [activate|deactivate|load|run]` this command sets the fetch_backups and restore_backups flags in your config data bag for an environment. These can be used to give application developers a way to trigger / untrigger restores in an environment
|
243
|
+
|
244
|
+
1. `activate` will turn on automated backup running (turns on the flag for the env in the config bag).
|
245
|
+
|
246
|
+
2. `deactivate` will turn off automated backup running.
|
247
|
+
|
248
|
+
3. `load` will fetch the latest backup from the production primary **if it doesn't already exist on the server** and run the _backup loading command_ to load this backup into the env.
|
249
|
+
|
250
|
+
4. `run` will simply just run the _backup loading command_ to load the latest backup onto the server.
|
225
251
|
|
226
252
|
6. `cft check_cheftacular_yml_keys` allows you to check to see if your cheftacular yml keys are valid to the current version of cheftacular. It will also set your missing keys to their likely default and let you know to update the cheftacular.yml file.
|
227
253
|
|
@@ -239,13 +265,21 @@
|
|
239
265
|
|
240
266
|
5. NOTE 2! Cheftacular does not (and will not) support accessing your chef server over ssh with password auth. If you have done this, you should feel bad and immediately switch ssh access to key authentication...
|
241
267
|
|
242
|
-
9. `cft
|
268
|
+
9. `cft cheftacular_config [display|sync|overwrite]` this command Allows you to interact with your complete cheftacular configuration, the union of all repository's cheftacular.ymls.
|
269
|
+
|
270
|
+
1. `display` will show the current overall configuration for cheftacular.
|
271
|
+
|
272
|
+
2. `sync` will sync your local cheftacular yaml keys ONTO the server's keys. This happens automatically whenever a difference is detected between the local keys and the remote keys but can be run manually. Will send a slack notification if slack is configured (the slack notification contains the diffed keys). The sync only occurs if there are CHANGES to the file.
|
273
|
+
|
274
|
+
10. `cft clean_cookbooks [force] [remove_cookbooks]` allows you to update the internal chef-repo's cookbooks easily. By default this script will force you to decide what to do with each cookbook individually (shows version numbers and whether to overwrite it to cookbooks or not).
|
243
275
|
|
244
276
|
1. `force` argument will cause the downloaded cookbooks to *always* overwrite the chef-repo's cookbooks as long as the downloaded cookbook has a higher version number.
|
245
277
|
|
246
278
|
2. If you would like to remove all the cookbooks on the chef server, run `knife cookbook bulk delete '.*' -p -c ~/.chef/knife.rb`
|
247
279
|
|
248
|
-
|
280
|
+
11. `cft clear_caches` this command allows you to clear all of your local caches.
|
281
|
+
|
282
|
+
12. `cft client_list` Allows you check the basic information for all the servers setup via chef. Shows the server's short name, its public ip address and roles (run_list) by default.
|
249
283
|
|
250
284
|
1. `-v` option will make this command display the server's domain name, whether its password is stored on the chef server and what that password is.
|
251
285
|
|
@@ -253,7 +287,7 @@
|
|
253
287
|
|
254
288
|
3. This command is aliased to `client-list` with no arguments or cft prefix.
|
255
289
|
|
256
|
-
|
290
|
+
13. `cft cloud <FIRST_LEVEL_ARG> [<SECOND_LEVEL_ARG>[:<SECOND_LEVEL_ARG_QUERY>]*] ` this command handles talking to various cloud APIs. If no args are passed nothing will happen.
|
257
291
|
|
258
292
|
1. `domain` first level argument for interacting with cloud domains
|
259
293
|
|
@@ -349,7 +383,7 @@
|
|
349
383
|
|
350
384
|
5. `bootstrap` captures the current computer's hostname and checks to see if a key matching this hostname exists on the cloud service. If the key does not exist, the command attempts to read the contents of the ~/.ssh/id_rsa.pub file and create a new key with that data and the hostname of the current computer. Run automatically when creating DigitalOcean servers. It's worth noting that if the computer's key already exists on DigitalOcean under a different name, this specific command will fail with a generic error. Please check your keys.
|
351
385
|
|
352
|
-
|
386
|
+
14. `cft cloud_bootstrap NODE_NAME FLAVOR_NAME [DESCRIPTOR] [--with-dn DOMAIN]` uses a cloud api to create a server and attaches its DOMAIN_NAME to the TLD specified for that environment (IE: example-staging.com for staging)
|
353
387
|
|
354
388
|
1. If no DOMAIN_NAME is supplied it will use the node's NODE_NAME (IE: api01.example-staging.com)
|
355
389
|
|
@@ -359,11 +393,11 @@
|
|
359
393
|
|
360
394
|
4. DESCRIPTOR is used as an internal tag for the node, if left blank it will become the name of the node. It is recommended to enter a custom repository-dependent tag here to make nodes easier to load-balance like "lb:[CODEBASE_NAME]"
|
361
395
|
|
362
|
-
|
396
|
+
15. `cft compile_audit_log [clean]` compiles the audit logs in each environment's audit data bag a audit-log-CURRENTDAY.md file in the log folder of the application. Bear in mind that the bag can only hold 100K bytes and will need to have that data removed to store more than that.
|
363
397
|
|
364
|
-
|
398
|
+
16. `cft compile_readme` compiles all documentation methods and creates a README.md file in the log folder of the application.
|
365
399
|
|
366
|
-
|
400
|
+
17. `cft create_git_key ID_RSA_FILE [OAUTH_TOKEN]` This command will update the default/authentication data bag with new credentials. The [ID_RSA_FILE](https://help.github.com/articles/generating-ssh-keys) needs to exist beforehand.
|
367
401
|
|
368
402
|
1. This command will upload both the private and public key to the data bag. The public key should be the one that matches the github user for your deployment github user.
|
369
403
|
|
@@ -371,9 +405,9 @@
|
|
371
405
|
|
372
406
|
3. NOTE! The ID_RSA_FILE should be in your .chef folder in the root of your home directory!
|
373
407
|
|
374
|
-
|
408
|
+
18. `cft disk_report` will fetch useful statistics from every server for every environment and output it into your log directory.
|
375
409
|
|
376
|
-
|
410
|
+
19. `cft environment boot|destroy` will boot / destroy the current environment
|
377
411
|
|
378
412
|
1. `boot` will spin up servers and bring them to a stable state. This includes setting up their subdomains for the target environment.
|
379
413
|
|
@@ -381,7 +415,7 @@
|
|
381
415
|
|
382
416
|
3. This command will prompt when attempting to destroy servers in staging or production
|
383
417
|
|
384
|
-
|
418
|
+
20. `cft file NODE_NAME LOCATION_ALIAS MODE FILE_NAME` interacts with a file on the remote server
|
385
419
|
|
386
420
|
1. `LOCATION_ALIAS` will be parsed as a path if it has backslash characters. Otherwise it will be parsed from your location_aliases hash in your cheftacular.yml
|
387
421
|
|
@@ -411,21 +445,21 @@
|
|
411
445
|
|
412
446
|
1. `--save-to-file FILE_PATH` can also be used in the `fetch` context to specify where exactly to save the file and what to name it as.
|
413
447
|
|
414
|
-
|
448
|
+
21. `cft fix_known_hosts [HOSTNAME]` this command will delete entries in your known_hosts file for all the servers that are in our system (ip addresses AND dns names)
|
415
449
|
|
416
450
|
1. Passing in a hostname will make the command only remove entries with that hostname / ip specifically
|
417
451
|
|
418
|
-
|
452
|
+
22. `cft full_bootstrap ADDRESS ROOT_PASS NODE_NAME` This command performs both ubuntu_bootstrap and chef_bootstrap.
|
419
453
|
|
420
|
-
|
454
|
+
23. `cft get_haproxy_log` this command will generate a haproxy html file for the load balancer(s) associated with a repository in the log directory. Opening this log file in the browser will show the status of that haproxy at the time of the log.
|
421
455
|
|
422
456
|
1. In devops mode, this command will not do anything without the -R repository passed.
|
423
457
|
|
424
|
-
|
458
|
+
24. `cft get_log_from_bag <NODE_NAME-COMMAND_TYPE>` this command grabs the latest command run log from the data bags and saves it to your log directory. There are different types of logs saved per server depending on command.
|
425
459
|
|
426
|
-
|
460
|
+
25. `cft get_pg_pass ['clip']` command will output the current environment's pg_password to your terminal. Optionally you can pass in clip like `cft get_pg_pass clip` to have it also copy the pass to your clipboard.
|
427
461
|
|
428
|
-
|
462
|
+
26. `cft get_shorewall_allowed_connections [PATH_TO_LOCAL_FILE] -n NODE_NAME` command will query a single server and return all of its ACCEPT connections from shorewall in it's syslog and return the results in a CSV format. Useful for tracking IP activity.
|
429
463
|
|
430
464
|
1. You must pass in a node name to query with `-n NODE_NAME`
|
431
465
|
|
@@ -433,35 +467,41 @@
|
|
433
467
|
|
434
468
|
3. If `PATH_TO_LOCAL_FILE` is not blank, the command will use that file instead of building a file on the remote server
|
435
469
|
|
436
|
-
|
470
|
+
27. `cft help COMMAND|MODE` this command returns the documentation for a specific command if COMMAND matches the name of a command. Alternatively, it can be passed `action|arguments|application|current|devops|stateless_action` to fetch the commands for a specific mode.Misspellings of commands will display near hits.
|
471
|
+
|
472
|
+
28. `cft initialize_cheftacular_yml [application|TheCheftacularCookbook]` will create a cheftacular.yml file in your config folder (and create the config folder if it does not exist). If you already have a cheftacular.yml file in the config folder, it will create a cheftacular.example.yml file that will contain the new changes / keys in the latest cheftacular version.
|
437
473
|
|
438
|
-
|
474
|
+
1. If `TheCheftacularCookbook` is passed, the generated cheftacular.yml file will include the additional TheCheftacularCookbook keys.
|
439
475
|
|
440
|
-
|
476
|
+
2. If `application` is passed, the generated cheftacular.yml file will look like one you could use in an application directory.
|
441
477
|
|
442
|
-
|
478
|
+
29. `cft initialize_data_bag_contents ENVIRONMENT_NAME` will ensure the data bags always have the correct structure before each run. This command is run every time the gem is started and if called directly, will exit after completion.
|
443
479
|
|
444
|
-
|
480
|
+
30. `cft knife_upload` will resync the chef-server with the local chef-repo code. This command is analog for `knife upload /`
|
445
481
|
|
446
|
-
|
482
|
+
31. `cft list_toggleable_roles NODE_NAME` This command will allow you to see all toggleable roles for a node
|
447
483
|
|
448
|
-
|
484
|
+
32. `cft location_aliases` will list all location aliases listed in your cheftacular.yml. These aliases can be used in the `cft file` command.
|
449
485
|
|
450
|
-
|
486
|
+
33. `cft pass NODE_NAME` will drop the server's sudo password into your clipboard. Useful for when you need to ssh into the server itself and try advanced linux commands
|
487
|
+
|
488
|
+
34. `cft remove_client NODE_NAME [destroy]` removes a client (and its node data) from the chef-server. It also removes its dns records from the cloud service (if possible). This should not be done lightly as you will have to wipe the server and trigger another chef-client run to get it to register again. Alternatively, you can run `cft reinitialize IP_ADDRESS NODE_NAME as well.
|
451
489
|
|
452
490
|
1. `destroy` deletes the server as well as removing it from the chef environment.
|
453
491
|
|
454
|
-
|
492
|
+
35. `cft replication_status` will check the status of the database master and slaves in every environment. Also lists how far behind the slaves are from the master in milliseconds.
|
493
|
+
|
494
|
+
36. `cft reset_bag BAG_NAME` this command allows you to reset a data bag item to an empty state. Run this on full data bags to clear them out.
|
455
495
|
|
456
|
-
|
496
|
+
37. `cft restart_swap` will restart the swap on every server that doesn't have swap currently on. Useful if you notice servers with no swap activated from `cft disk_report`
|
457
497
|
|
458
498
|
1. There is no risk in running this command. Sometimes swap doesnt reactivate if the server was rebooted and this command fixes that.
|
459
499
|
|
460
|
-
|
500
|
+
38. `cft role_toggle NODE_NAME ROLE_NAME activate|deactivate` This command will allow you to **toggle** roles on nodes without using `cft upload_nodes`
|
461
501
|
|
462
502
|
1. This command uses your *role_toggling:deactivated_role_suffix* attribute set in your cheftacular.yml to toggle the role, it checks to see if the toggled name exists then sets the node's run_list to include the toggled role
|
463
503
|
|
464
|
-
2. EX: `cft role_toggle
|
504
|
+
2. EX: `cft role_toggle api01 worker activate` will find the node api01 and attempt to toggle the worker role to on. If the node does NOT have the worker_deactivate role, then it will add it if *role_toggling:strict_roles* is set to **false**
|
465
505
|
|
466
506
|
1. If *role_toggling:strict_roles* is set to true, then cheftacular would raise an error saying this role is unsettable on the node. On the other hand, if the node already has the worker_deactivaterole, then this command will succeed even if *strict_roles* is set.
|
467
507
|
|
@@ -469,7 +509,7 @@
|
|
469
509
|
|
470
510
|
1. Please run `cft list_toggleable_roles NODE_NAME` to get a list of your org's toggleable roles for a node.
|
471
511
|
|
472
|
-
|
512
|
+
39. `cft rvm [COMMAND] [ADDITIONAL_COMMANDS]*` will run rvm commands on the remote servers. Output from this command for each server will go into your rvm directory under the log directory. Please refer to [the rvm help page](https://rvm.io/rvm) for more information on rvm commands.
|
473
513
|
|
474
514
|
1. When no commands are passed, rvm will just run `rvm list` on each server on all servers in the current environment.
|
475
515
|
|
@@ -485,11 +525,11 @@
|
|
485
525
|
|
486
526
|
7. `upgrade_rvm` will run `rvm get stable --auth-dotfiles` on all servers for the current environment. It will also check and attempt to upgrade pre 1.25 installations of RVM to 1.26+ (which requires a GPG key).
|
487
527
|
|
488
|
-
|
528
|
+
40. `cft server_update [restart]` allows you to force update all nodes' packages for a specific environment. This should be done with caution as this *might* break something.
|
489
529
|
|
490
530
|
1. `cft apt_update restart` will prompt to ask if you also want to restart all servers in a rolling restart. This should be done with extreme caution and only in a worst-case scenario.
|
491
531
|
|
492
|
-
|
532
|
+
41. `cft service [COMMAND] [SERVICE]` will run service commands on remote servers. This command only runs on the first server it comes across. Specify others with -n NODE_NAME.
|
493
533
|
|
494
534
|
1. When no commands are passed, the command will list all the services in the /etc/init directory
|
495
535
|
|
@@ -497,13 +537,13 @@
|
|
497
537
|
|
498
538
|
3. When `restart|stop|start SERVICE` is passed, the command will attempt to restart|stop|start the service if it has a .conf file on the remote server in the /etc/init directory.
|
499
539
|
|
500
|
-
|
540
|
+
42. `cft slack "MESSAGE" [CHANNEL]` will attempt to post the message to the webhook set in your cheftacular.yml. Slack posts to your default channel by default but if the CHANNEL argument is supplied the message will post there.
|
501
541
|
|
502
542
|
1. NOTE: To prevent confusing spam from many possible sources, the username posted to slack will always be *Cheftacular*. This can be overloaded in the StatelessAction method "slack" but this is not recommended.
|
503
543
|
|
504
544
|
2. Remember, if you have auditing turned on in your cheftacular.yml, you can track who sends what to slack.
|
505
545
|
|
506
|
-
|
546
|
+
43. `cft test_env [TARGET_ENV] boot|destroy` will create (or destroy) the test nodes for a particular environment (defaults to staging, prod split-envs can be set with `-p`). Please read below for how TARGET_ENV works
|
507
547
|
|
508
548
|
1. TARGET_ENV changes functionality depending on the overall (like staging / production) environment
|
509
549
|
|
@@ -513,9 +553,9 @@
|
|
513
553
|
|
514
554
|
3. The default tld used should change depending on which environment you are booting / destroying. This is set in the environment's config data bag under the tld key
|
515
555
|
|
516
|
-
|
556
|
+
44. `cft ubuntu_bootstrap ADDRESS ROOT_PASS` This command will bring a fresh server to a state where chef-client can be run on it via `cft chef-bootstrap`. It should be noted that it is in this step where a server's randomized deploy_user sudo password is generated.
|
517
557
|
|
518
|
-
|
558
|
+
45. `cft update_cloudflare_dns_from_cloud [skip_update_tld]` command will force a full dns update for cloudflare.
|
519
559
|
|
520
560
|
1. It will ensure all the subdomain entries are correct (based on the contents of the addresses data bag) and update them if they are not. It will also create the local subdomain for the entry as well if it does exist and point it to the correct private address for an environment.
|
521
561
|
|
@@ -523,7 +563,7 @@
|
|
523
563
|
|
524
564
|
3. The argument `skip_update_tld` will stop the long process of checking and updating all the server domains _before_ cloudflare is updated. Only skip if you believe your domain info on your cloud is accurate.
|
525
565
|
|
526
|
-
|
566
|
+
46. `cft update_split_branches` will perform a series of git commands that will merge all the split branches for your split_branch enabled repositories with what is currently on master and push them.
|
527
567
|
|
528
568
|
1. Repository must be set with `-R REPOSITORY_NAME` for this command to work.
|
529
569
|
|
@@ -533,9 +573,9 @@
|
|
533
573
|
|
534
574
|
4. This command will return a helpful error statement if you attempt to run the command with changes to your current working directory. You must commit these changes before running this command.
|
535
575
|
|
536
|
-
|
576
|
+
47. `cft update_tld TLD` command will force a full dns update for a tld in the preferred cloud. It will ensure all the subdomain entries are correct (based on the contents of the addresses data bag) and update them if they are not. It will also create the local subdomain for the entry as well if it does exist and point it to the correct private address.
|
537
577
|
|
538
|
-
|
578
|
+
48. `cft upload_nodes` This command will resync the chef server's nodes with the data in our chef-repo/node_roles.
|
539
579
|
|
540
580
|
1. This command changes behavior depending on several factors about both your mode and the state of your environment
|
541
581
|
|
@@ -547,4 +587,4 @@
|
|
547
587
|
|
548
588
|
1. Due to this, only users running this against their chef-repo need to worry about having a nodes_dir, the way it should be.
|
549
589
|
|
550
|
-
|
590
|
+
49. `cft upload_roles` This command will resync the chef server's roles with the data in the chef-repo/roles.
|
@@ -2,7 +2,7 @@ class Cheftacular
|
|
2
2
|
class ActionDocumentation
|
3
3
|
def tail
|
4
4
|
@config['documentation']['action'] << [
|
5
|
-
"`cft tail` will tail the logs (return continuous output) of the first node if finds " +
|
5
|
+
"`cft tail [PATTERN_TO_MATCH]` will tail the logs (return continuous output) of the first node if finds " +
|
6
6
|
"that has an application matching the repository running on it. Currently only supports rails stacks",
|
7
7
|
|
8
8
|
[
|
@@ -10,15 +10,18 @@ class Cheftacular
|
|
10
10
|
|
11
11
|
" 2. Workers and job servers change the output of this command heavily. " +
|
12
12
|
"Worker and job servers should tail their log to the master log (/var/log/syslog) where <b>all</b> of the major processes on the server output to. " +
|
13
|
-
"While the vast majority of this syslog will be relevant to application developers, some will not (usually firewall blocks and the like)."
|
13
|
+
"While the vast majority of this syslog will be relevant to application developers, some will not (usually firewall blocks and the like).",
|
14
|
+
|
15
|
+
" 3. if the `PATTERN_TO_MATCH` argument exists, the tail will only return entries that have that pattern rather than everything written to the file."
|
14
16
|
]
|
15
17
|
]
|
16
18
|
end
|
17
19
|
end
|
18
20
|
|
19
21
|
class Action
|
20
|
-
|
21
|
-
|
22
|
+
def tail pattern_to_match=''
|
23
|
+
pattern_to_match = ARGV[1] if pattern_to_match.blank?
|
24
|
+
|
22
25
|
nodes = @config['getter'].get_true_node_objects
|
23
26
|
|
24
27
|
nodes = @config['parser'].exclude_nodes( nodes, [{ unless: "role[#{ @options['role'] }]" }], true )
|
@@ -27,28 +30,32 @@ class Cheftacular
|
|
27
30
|
puts("Beginning tail run for #{ n.name } (#{ n.public_ipaddress }) on role #{ @options['role'] }") unless @options['quiet']
|
28
31
|
|
29
32
|
if @config['dummy_sshkit'].has_run_list_in_role_map?(n.run_list, @config['cheftacular']['role_maps'])
|
30
|
-
start_tail_role_map( n.public_ipaddress, n.run_list )
|
33
|
+
start_tail_role_map( n.public_ipaddress, n.run_list, pattern_to_match )
|
31
34
|
else
|
32
|
-
self.send("start_tail_#{ @config['getter'].get_current_stack }", n.public_ipaddress, n.run_list )
|
35
|
+
self.send("start_tail_#{ @config['getter'].get_current_stack }", n.public_ipaddress, n.run_list, pattern_to_match )
|
33
36
|
end
|
34
37
|
end
|
35
38
|
end
|
36
39
|
|
37
40
|
private
|
38
41
|
|
39
|
-
def start_tail_role_map ip_address, run_list
|
42
|
+
def start_tail_role_map ip_address, run_list, pattern_to_match
|
40
43
|
log_loc = @config['getter'].get_current_role_map(run_list)['log_location'].split(',').first.gsub('|current_repo_location|', "#{ @config['cheftacular']['base_file_path'] }/#{ @options['repository'] }/current")
|
41
44
|
|
42
|
-
`ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc }" > /dev/tty`
|
45
|
+
`ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc } #{ get_tail_grep_string(pattern_to_match) }" > /dev/tty`
|
43
46
|
end
|
44
47
|
|
45
|
-
def start_tail_ruby_on_rails ip_address, run_list
|
48
|
+
def start_tail_ruby_on_rails ip_address, run_list, pattern_to_match
|
46
49
|
true_env = @config['dummy_sshkit'].get_true_environment run_list, @config['cheftacular']['run_list_environments'][@options['env']], @options['env']
|
47
50
|
|
48
51
|
#special servers should be listed first as most of them will have web role
|
49
52
|
log_loc = "#{ @config['cheftacular']['base_file_path'] }/#{ @options['repository'] }/current/log/#{ true_env }.log"
|
50
53
|
|
51
|
-
`ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc }" > /dev/tty`
|
54
|
+
`ssh -oStrictHostKeyChecking=no -tt #{ @config['cheftacular']['deploy_user'] }@#{ ip_address } "#{ @config['helper'].sudo(ip_address) } tail -f #{ log_loc } #{ get_tail_grep_string(pattern_to_match) }" > /dev/tty`
|
55
|
+
end
|
56
|
+
|
57
|
+
def get_tail_grep_string pattern_to_match=''
|
58
|
+
"| grep -i -E #{ pattern_to_match }" unless pattern_to_match.blank?
|
52
59
|
end
|
53
60
|
end
|
54
61
|
end
|
@@ -7,18 +7,18 @@ class Cheftacular
|
|
7
7
|
|
8
8
|
#this will only intialize bags (and their hashes) if they don't exist. Use ridley data bag methods to reload the data etc
|
9
9
|
def init_bag bag_env, bag_name, encrypted=true
|
10
|
-
|
10
|
+
@config['ridley'].data_bag.create(name: bag_env) if @config['ridley'].data_bag.find(bag_env).nil?
|
11
11
|
|
12
|
-
if
|
13
|
-
|
12
|
+
if @config['ridley'].data_bag.find(bag_env).item.find(bag_name).nil?
|
13
|
+
@config['ridley'].data_bag.find(bag_env).item.create(id: bag_name)
|
14
14
|
end
|
15
15
|
|
16
16
|
@config[bag_env] ||= {}
|
17
17
|
|
18
18
|
if !@config[bag_env].has_key?("#{ bag_name }_bag") || !@config[bag_env].has_key?("#{ bag_name }_bag_hash")
|
19
|
-
|
19
|
+
@config[bag_env]["#{ bag_name }_bag"] = @config['ridley'].data_bag.find(bag_env).item.find(bag_name)
|
20
20
|
|
21
|
-
self.instance_eval "@config['#{ bag_env }']['#{ bag_name }_bag_hash']
|
21
|
+
self.instance_eval "@config['#{ bag_env }']['#{ bag_name }_bag_hash'] = @config['#{ bag_env }']['#{ bag_name }_bag']#{ encrypted ? '.decrypt' : '' }.to_hash"
|
22
22
|
end
|
23
23
|
end
|
24
24
|
|
@@ -46,7 +46,6 @@ class Cheftacular
|
|
46
46
|
end
|
47
47
|
end
|
48
48
|
|
49
|
-
#TODO special save for bag that will compile the data into a different bag for storage (the data will be stored as an audit log and zlib'd)
|
50
49
|
def save_audit_bag bag_env="options"
|
51
50
|
env = bag_env == 'options' ? @options['env'] : bag_env
|
52
51
|
|
@@ -57,6 +56,14 @@ class Cheftacular
|
|
57
56
|
save_bag 'authentication', bag_env, @config['default']['authentication_bag'], @config['default']['authentication_bag_hash'], true
|
58
57
|
end
|
59
58
|
|
59
|
+
def save_cheftacular_bag bag_env="default"
|
60
|
+
save_bag 'cheftacular', bag_env, @config['default']['cheftacular_bag'], @config['default']['cheftacular_bag_hash']
|
61
|
+
end
|
62
|
+
|
63
|
+
def save_environment_config_bag bag_env='default'
|
64
|
+
save_bag 'environment_config', bag_env, @config['default']['environment_config_bag'], @config['default']['environment_config_bag_hash']
|
65
|
+
end
|
66
|
+
|
60
67
|
def save_chef_passwords_bag bag_env="options"
|
61
68
|
env = bag_env == 'options' ? @options['env'] : bag_env
|
62
69
|
|
@@ -87,13 +94,46 @@ class Cheftacular
|
|
87
94
|
save_bag 'node_roles', bag_env, @config[env]['node_roles_bag'], @config[env]['node_roles_bag_hash']
|
88
95
|
end
|
89
96
|
|
97
|
+
def reset_addresses_bag bag_env="options"
|
98
|
+
env = bag_env == 'options' ? @options['env'] : bag_env
|
99
|
+
|
100
|
+
reset_bag 'addresses', env
|
101
|
+
end
|
102
|
+
|
103
|
+
def reset_audit_bag bag_env='options'
|
104
|
+
env = bag_env == 'options' ? @options['env'] : bag_env
|
105
|
+
|
106
|
+
reset_bag 'audit', env
|
107
|
+
end
|
108
|
+
|
109
|
+
def reset_cheftacular_bag bag_env="default"
|
110
|
+
reset_bag 'cheftacular', bag_env
|
111
|
+
end
|
112
|
+
|
113
|
+
def reset_environment_config_bag bag_env='default'
|
114
|
+
reset_bag 'environment_config', bag_env
|
115
|
+
end
|
116
|
+
|
117
|
+
def reset_node_roles_bag bag_env="options"
|
118
|
+
env = bag_env == 'options' ? @options['env'] : bag_env
|
119
|
+
|
120
|
+
reset_bag 'node_roles', env
|
121
|
+
end
|
122
|
+
|
90
123
|
private
|
124
|
+
def reset_bag bag_name, bag_env, encrypted=false
|
125
|
+
raise "Cannot reset bag #{ bag_name } in #{ bag_env } as it does not exist!" if @config['ridley'].data_bag.find(bag_env).item.find(bag_name).nil?
|
126
|
+
|
127
|
+
@config['ridley'].data_bag.find(bag_env).item.delete(bag_name)
|
128
|
+
|
129
|
+
init_bag bag_env, bag_name, encrypted
|
130
|
+
end
|
131
|
+
|
91
132
|
def save_bag bag_name, bag_env, bag, bag_hash, encrypted=false
|
92
133
|
return true if @config['helper'].running_on_chef_node?
|
93
134
|
|
94
|
-
new_bag_hash =
|
95
|
-
|
96
|
-
item = bag.reload
|
135
|
+
new_bag_hash = bag_hash.deep_dup
|
136
|
+
item = bag.reload
|
97
137
|
|
98
138
|
load_hash = encrypted ? item.decrypt.to_hash.deep_merge(new_bag_hash) : item.attributes.deep_merge(new_bag_hash)
|
99
139
|
|