knife-tidy 0.4.1 → 0.5.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: fdd8bded909178ab1e6055a4415dc0c675ee0441
4
- data.tar.gz: 52b73bfc270082947f232fd333ecdd900bac2ac7
3
+ metadata.gz: e9766f389dc78f0c5c764b04ad056d0c74278751
4
+ data.tar.gz: 6df96008461814fa24005126d3df61a117288cd3
5
5
  SHA512:
6
- metadata.gz: e843f9db36f6baf438c500ff2fd52b8e7e7151f06391597c3809dd52b4bf89124ba7a9cd2579d090c1eb9280b92145ff09a2738a799275cbb4bfb8de75b1865e
7
- data.tar.gz: 7dcb30b9c176ccb3b087cbecdfdbe4ec03854c32643eb7de6b7592a24ae1d58a7b8009a3ce1ab0f38ddcab16114f9eae005ee6505958c4c4987a1f80133d5278
6
+ metadata.gz: 21bf46076b9e8dee58aea293820e13ea93d724ea240cb95c9b44e43942aab098a754aea3e31affbd9f481acacfc3eda39c9f9d1787210039aed5f20691ed00b5
7
+ data.tar.gz: ec2a71913df5a0e69949e3a0a6efe59814df404db9de70d29255a77a340430ca83836cdba78ad8d8927c8a183f2516ba027a54c07be1272681f5e33521e481a9
data/CHANGELOG.md CHANGED
@@ -1,5 +1,21 @@
1
1
  # Change Log
2
2
 
3
+ ## [0.5.0](https://github.com/chef-customers/knife-tidy/tree/0.5.0) (2017-10-06)
4
+ [Full Changelog](https://github.com/chef-customers/knife-tidy/compare/0.4.1...0.5.0)
5
+
6
+ **Closed issues:**
7
+
8
+ - Supermarket.chef.io [\#37](https://github.com/chef-customers/knife-tidy/issues/37)
9
+
10
+ **Merged pull requests:**
11
+
12
+ - enabling deletion of stale nodes [\#44](https://github.com/chef-customers/knife-tidy/pull/44) ([jeremymv2](https://github.com/jeremymv2))
13
+ - better warning message before confirmation [\#43](https://github.com/chef-customers/knife-tidy/pull/43) ([jeremymv2](https://github.com/jeremymv2))
14
+ - Jeremymv2/chef sugar fix default [\#42](https://github.com/chef-customers/knife-tidy/pull/42) ([jeremymv2](https://github.com/jeremymv2))
15
+ - deleting nodes also deletes client [\#41](https://github.com/chef-customers/knife-tidy/pull/41) ([jeremymv2](https://github.com/jeremymv2))
16
+ - Change nodes\_list method to get all nodes [\#40](https://github.com/chef-customers/knife-tidy/pull/40) ([nsdavidson](https://github.com/nsdavidson))
17
+ - bump version to 0.4.1 [\#39](https://github.com/chef-customers/knife-tidy/pull/39) ([jeremymv2](https://github.com/jeremymv2))
18
+
3
19
  ## [0.4.1](https://github.com/chef-customers/knife-tidy/tree/0.4.1) (2017-09-27)
4
20
  [Full Changelog](https://github.com/chef-customers/knife-tidy/compare/0.4.0...0.4.1)
5
21
 
data/README.md CHANGED
@@ -2,9 +2,10 @@
2
2
 
3
3
  # Summary
4
4
 
5
- This Chef Knife plugin has two primary purposes:
6
- * Report on the state of Chef Server objects that can be tidied up (Future: clean up objects)
7
- * A companion tool to [knife-ec-backup](https://github.com/chef/knife-ec-backup) that will clean up data integrity issues in an object backup
5
+ This Chef Knife plugin provides:
6
+ * Reports on the state of Chef Server objects that can be tidied up
7
+ * Removal of stale nodes (and associated clients and ACLs) identified by the above Reports
8
+ * A [knife-ec-backup](https://github.com/chef/knife-ec-backup) companion tool that will clean up data integrity issues in an object backup
8
9
 
9
10
  # Requirements
10
11
 
@@ -56,6 +57,21 @@ org_threshold_numdays_stale_nodes.json | Nodes in that org that have not checked
56
57
  org_cookbook_count.json | Number of cookbook versions for each cookbook that that org.
57
58
  org_unused_cookbooks.json | List of cookbooks and versions that do not appear to be in-use for that org. This is determined by checking the versioned run list of each of the nodes in the org.
58
59
 
60
+ ## $ knife tidy server clean --help
61
+ Remove stale nodes that haven't checked-in to the Chef Server as defined by the `--node-threshold NUM_DAYS` option when the reports were generated.. The associated client and ACLs are also removed.
62
+
63
+ Future: remove unused cookbooks - currently this feature is disabled.
64
+
65
+ ## Options
66
+
67
+ * `--dry-run`
68
+ Do not perform any actual deletion, only report on what would have been deleted.
69
+
70
+ Example:
71
+ ```bash
72
+ knife tidy server clean --orgs brewinc,acmeinc
73
+ ```
74
+
59
75
  ## $ knife tidy backup clean --help
60
76
 
61
77
  ## Options
@@ -80,18 +96,6 @@ knife tidy backup clean --backup-path backups/ --gsub-file substitutions.json
80
96
 
81
97
  ```json
82
98
  {
83
- "chef-sugar":{
84
- "organizations/*/cookbooks/chef-sugar*/metadata.rb":[
85
- {
86
- "pattern":"^require .*/lib/chef/sugar/version",
87
- "replace":"# require File.expand_path('../lib/chef/sugar/version', *__FILE__)"
88
- },
89
- {
90
- "pattern":"version *Chef::Sugar::VERSION",
91
- "replace":"version !COOKBOOK_VERSION!"
92
- }
93
- ]
94
- },
95
99
  "io-read-version-and-readme.md":{
96
100
  "organizations/*/cookbooks/*/metadata.rb":[
97
101
  {
@@ -1,25 +1,9 @@
1
1
  {
2
- "chef-sugar":{
3
- "organizations/*/cookbooks/chef-sugar*/metadata.rb":[
4
- {
5
- "pattern":"^require .*/lib/chef/sugar/version",
6
- "replace":"# require File.expand_path('../lib/chef/sugar/version', *__FILE__)"
7
- },
8
- {
9
- "pattern":"version *Chef::Sugar::VERSION",
10
- "replace":"version !COOKBOOK_VERSION!"
11
- }
12
- ]
13
- },
14
- "io-read-version-and-readme.md":{
2
+ "your-problem-descriptor":{
15
3
  "organizations/*/cookbooks/*/metadata.rb":[
16
4
  {
17
- "pattern":"^version +IO.read.* 'VERSION'.*",
5
+ "pattern":"^version .*GO_PIPELINE_LABEL",
18
6
  "replace":"version !COOKBOOK_VERSION!"
19
- },
20
- {
21
- "pattern":"^long_description +IO.read.* 'README.md'.*",
22
- "replace":"#long_description \"A Long Description..\""
23
7
  }
24
8
  ]
25
9
  }
@@ -45,6 +45,8 @@ class Chef
45
45
  exit 1
46
46
  end
47
47
 
48
+ fix_chef_sugar_metadata
49
+
48
50
  Chef::TidySubstitutions.new(substitutions_file, tidy).run_substitutions if config[:gsub_file]
49
51
 
50
52
  validate_user_emails
@@ -187,6 +189,27 @@ class Chef
187
189
  end
188
190
  end
189
191
 
192
+ def fix_chef_sugar_metadata
193
+ Dir[::File.join(tidy.backup_path, 'organizations/*/cookbooks/chef-sugar*/metadata.rb')].each do |file|
194
+ puts 'INFO: Searching for known chef-sugar problems when uploading.'
195
+ s = Chef::TidySubstitutions.new
196
+ version = s.cookbook_version_from_path(file)
197
+ patterns = [
198
+ {
199
+ search: '^require .*/lib/chef/sugar/version',
200
+ replace: "# require File.expand_path('../lib/chef/sugar/version', *__FILE__)"
201
+ },
202
+ {
203
+ search: '^version *Chef::Sugar::VERSION',
204
+ replace: "version '#{version}'"
205
+ }
206
+ ]
207
+ patterns.each do |p|
208
+ s.sub_in_file(file, Regexp.new(p[:search]), p[:replace])
209
+ end
210
+ end
211
+ end
212
+
190
213
  def fix_self_dependencies(org)
191
214
  for_each_cookbook_path(org) do |cookbook_path|
192
215
  name = tidy.cookbook_name_from_path(cookbook_path)
@@ -15,8 +15,8 @@ class Chef
15
15
 
16
16
  option :concurrency,
17
17
  :long => '--concurrency THREADS',
18
- :default => 10,
19
- :description => 'Maximum number of simultaneous requests to send (default: 10)'
18
+ :default => 1,
19
+ :description => 'Maximum number of simultaneous requests to send (default: 1)'
20
20
 
21
21
  option :only_cookbooks,
22
22
  :long => '--only-cookbooks',
@@ -24,23 +24,17 @@ class Chef
24
24
 
25
25
  option :only_nodes,
26
26
  :long => '--only-nodes',
27
- :description => 'Only delete stale nodes from Chef Server.'
27
+ :description => 'Only delete stale nodes (and associated clients and ACLs) from Chef Server.'
28
28
 
29
29
  option :dry_run,
30
30
  :long => '--dry-run',
31
31
  :description => 'Do not perform any actual deletion, only report on what would have been deleted.'
32
32
 
33
33
  def run
34
- # not enabled
35
- ui.warn "This feature is not enabled"
36
- exit
37
-
38
34
  STDOUT.sync = true
39
35
 
40
36
  ensure_reports_dir
41
- puts "INFO: Reading from #{tidy.reports_dir} directory"
42
37
 
43
- puts "INFO: Using thread concurrency #{config[:concurrency]}"
44
38
  configure_chef
45
39
 
46
40
  if config[:only_cookbooks] && config[:only_nodes]
@@ -48,7 +42,13 @@ class Chef
48
42
  exit 1
49
43
  end
50
44
 
51
- ui.confirm('This operation will delete items on Chef Server, continue') unless config[:unattended]
45
+ deletions = if config[:only_cookbooks]
46
+ "cookbooks"
47
+ elsif config[:only_nodes]
48
+ "nodes (and associated clients and ACLs)"
49
+ else
50
+ "cookbooks and nodes (and associated clients and ACLs)"
51
+ end
52
52
 
53
53
  orgs = if config[:org_list]
54
54
  config[:org_list].split(',')
@@ -56,6 +56,10 @@ class Chef
56
56
  all_orgs
57
57
  end
58
58
 
59
+ ui.warn "This operation will affect the following Orgs on #{server.root_url}\n\n#{orgs}\n\n"
60
+
61
+ ui.confirm("About to delete #{deletions} from the Chef Server identified in the #{tidy.reports_dir} directory! Are you sure you wish to continue") unless config[:unattended]
62
+
59
63
  orgs.each do |org|
60
64
  clean_cookbooks(org) unless config[:only_nodes]
61
65
  clean_nodes(org) unless config[:only_cookbooks]
@@ -65,6 +69,8 @@ class Chef
65
69
  end
66
70
 
67
71
  def clean_cookbooks(org)
72
+ ui.warn "Cleaning cookbooks is a feature not yet enabled."
73
+ return
68
74
  queue = Chef::Util::ThreadedJobQueue.new
69
75
  unused_cookbooks_file = ::File.join(tidy.reports_dir, "#{org}_unused_cookbooks.json")
70
76
  return unless ::File.exist?(unused_cookbooks_file)
@@ -82,19 +88,12 @@ class Chef
82
88
  def delete_cookbook_job(org, cookbook, version)
83
89
  path = "/organizations/#{org}/cookbooks/#{cookbook}/#{version}"
84
90
  if config[:dry_run]
85
- printf("INFO: Would have executed `rest.delete(#{path})`\n")
91
+ printf("DRYRUN: Would have executed `rest.delete(#{path})`\n")
86
92
  return
87
93
  end
94
+ printf("INFO: Deleting #{path}\n")
88
95
  rest.delete(path)
89
- response = '200'
90
96
  rescue Net::HTTPServerException
91
- response = $!.response.code
92
- ensure
93
- return if config[:dry_run]
94
- formatted = response == '200' ?
95
- ui.color(' Deleting %-20s %-10s %10s', :green) :
96
- ui.color(' Deleting %-20s %-10s %10s', :red)
97
- printf("#{formatted}\n", cookbook, version, response)
98
97
  end
99
98
 
100
99
  def clean_nodes(org)
@@ -110,21 +109,19 @@ class Chef
110
109
  end
111
110
 
112
111
  def delete_node_job(org, node)
113
- path = "/organizations/#{org}/nodes/#{node}"
114
- if config[:dry_run]
115
- printf("INFO: Would have executed `rest.delete(#{path})`\n")
116
- return
112
+ paths = ["/organizations/#{org}/nodes/#{node}", "/organizations/#{org}/clients/#{node}"]
113
+ paths.each do |path|
114
+ if config[:dry_run]
115
+ printf("DRYRUN: Would have executed `rest.delete(#{path})`\n")
116
+ next
117
+ else
118
+ begin
119
+ printf("INFO: Deleting #{path}\n")
120
+ rest.delete(path)
121
+ rescue Net::HTTPServerException
122
+ end
123
+ end
117
124
  end
118
- rest.delete(path)
119
- response = '200'
120
- rescue Net::HTTPServerException
121
- response = $!.response.code
122
- ensure
123
- return if config[:dry_run]
124
- formatted = response == '200' ?
125
- ui.color(' Deleting %-20s %10s', :green) :
126
- ui.color(' Deleting %-20s %10s', :red)
127
- printf("#{formatted}\n", node, response)
128
125
  end
129
126
 
130
127
  def ensure_reports_dir
@@ -37,7 +37,7 @@ class Chef
37
37
  cb_list = cookbook_list(org)
38
38
  version_count = cookbook_count(cb_list).sort_by(&:last).reverse.to_h
39
39
  used_cookbooks = {}
40
- nodes = nodes_list(org)[0]
40
+ nodes = nodes_list(org)
41
41
 
42
42
  nodes.select{|node| !node['cookbooks'].nil?}.each do |node|
43
43
  node['cookbooks'].each do |name, version_hash|
@@ -86,7 +86,10 @@ class Chef
86
86
  end
87
87
  end
88
88
 
89
+ # Need the block here to get the search method to invoke multiple searches and
90
+ # aggregate results for result sets over 1k.
89
91
  def nodes_list(org)
92
+ node_results = []
90
93
  Chef::Search::Query.new("#{server.root_url}/organizations/#{org}").search(
91
94
  :node, '*:*',
92
95
  :filter_result => {
@@ -94,7 +97,10 @@ class Chef
94
97
  'cookbooks' => ['cookbooks'],
95
98
  'ohai_time' => ['ohai_time']
96
99
  }
97
- )
100
+ ) do |node|
101
+ node_results << node
102
+ end
103
+ node_results
98
104
  end
99
105
 
100
106
  def cookbook_list(org)
@@ -1,4 +1,4 @@
1
1
  module KnifeTidy
2
- VERSION = '0.4.1'
2
+ VERSION = '0.5.0'
3
3
  MAJOR, MINOR, TINY = VERSION.split('.')
4
4
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: knife-tidy
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.1
4
+ version: 0.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Jeremy Miller
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-09-27 00:00:00.000000000 Z
11
+ date: 2017-10-06 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake