kennel 1.115.0 → 1.116.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 76e2802df7ab2a6583e3529643deb08c382367dea7f90a7253314959d29a5bec
4
- data.tar.gz: 4249c23ca2892cc48ce72dbd9e24fc03f38883d6eaed9b509ef75c78cb31afeb
3
+ metadata.gz: 593a47b9f5fd48c679f9980ce91e3c70e0255e6431412882b502b646ac68a6bb
4
+ data.tar.gz: f4db29cc39cfea1f06d7e46db25f41c164ee7396a9739765750d4172adb8e7b0
5
5
  SHA512:
6
- metadata.gz: 8bd1740361932ba4e6dfe4d7aff4ad7cbcdc96796020428902fc6551b9768c6ccc434a279c214a638aa119e012267fd8dfe00e9107052c570789a1e563075505
7
- data.tar.gz: 266aacb034b8933640c3e95a0319afd5fdc923c857d85a5b039e4e14a2bb47dc1096e8849a1d3060171fed2984d847934465ed845a1e45d27caa465c5ca9fcf4
6
+ metadata.gz: 89183dd1763b2387b67f82b08e99433fdaf237a30dd89795dd97e338a17a7fc5109bac860b2ade4e0839b4f9a058d772e489fee670202b564ee1e883999b648e
7
+ data.tar.gz: 91da9b2df7e862bd7926cca5e7db1e97ffe02ad334e888b3ba51ee31ac41370c10934d2d4e1b5223e4f0f6d9e71ee1ba655f51b10fdfba4b02f677fc8fafd445
data/Readme.md CHANGED
@@ -289,23 +289,27 @@ Remove the code that created the resource. The next update will delete it (see a
289
289
  ```
290
290
 
291
291
  ### Updating existing resources with id
292
-
293
292
  Setting `id` makes kennel take over a manually created datadog resource.
294
293
  When manually creating to import, it is best to remove the `id` and delete the manually created resource.
295
294
 
296
295
  When an `id` is set and the original resource is deleted, kennel will fail to update,
297
296
  removing the `id` will cause kennel to create a new resource in datadog.
298
297
 
299
- ### Organizing many projects
300
- Having many projects (and their sub-resources) can quickly get out of hand.
298
+ ### Organizing projects with many resources
299
+ When project files get too long, this structure can keep things bite-sized.
301
300
 
302
- Use this class structure to keep things organized:
303
301
  ```Ruby
304
302
  # projects/project_a/base.rb
305
303
  module ProjectA
306
304
  class Base < Kennel::Models::Project
307
305
  defaults(
308
306
  kennel_id: -> { "project_a" },
307
+ parts: -> {
308
+ [
309
+ Monitors::FooAlert.new(self),
310
+ ...
311
+ ]
312
+ }
309
313
  ...
310
314
 
311
315
  # projects/project_a/monitors/foo_alert.rb
@@ -315,7 +319,16 @@ module ProjectA
315
319
  ...
316
320
  ```
317
321
 
318
- On the command line, use the projects `kennel_id` (and if none is set then snake_case of the class name) to refer to the project. For example here`PROJECT=project_a` but if it were `Foo::ProjectA` then `foo_project_a`.
322
+ ### Updating a single project or resource
323
+
324
+ - Use `PROJECT=<kennel_id>` for single project:
325
+
326
+ Use the projects `kennel_id` (and if none is set then snake_case of the class name including modules)
327
+ to refer to the project. For example for `class ProjectA` use `PROJECT=project_a` but for `Foo::ProjectA` use `foo_project_a`.
328
+
329
+ - Use `TRACKING_ID=<project-kennel_id>:<resource-kennel_id>` for single resource:
330
+
331
+ Use the project kennel_id and the resources kennel_id, for example `class ProjectA` and `FooAlert` would give `project_a:foo_alert`.
319
332
 
320
333
  ### Skipping validations
321
334
  Some validations might be too strict for your usecase or just wrong, please [open an issue](https://github.com/grosser/kennel/issues) and
@@ -7,12 +7,12 @@ module Kennel
7
7
  def self.progress(name)
8
8
  Kennel.err.print "#{name} ... "
9
9
 
10
- animation = "-\\|/"
11
- count = 0
12
10
  stop = false
13
11
  result = nil
14
12
 
15
13
  spinner = Thread.new do
14
+ animation = "-\\|/"
15
+ count = 0
16
16
  loop do
17
17
  break if stop
18
18
  Kennel.err.print animation[count % animation.size]
@@ -30,7 +30,7 @@ module Kennel
30
30
 
31
31
  result
32
32
  ensure
33
- stop = true
33
+ stop = true # make thread stop without killing it
34
34
  end
35
35
  end
36
36
  end
data/lib/kennel/syncer.rb CHANGED
@@ -4,9 +4,10 @@ module Kennel
4
4
  DELETE_ORDER = ["dashboard", "slo", "monitor", "synthetics/tests"].freeze # dashboards references monitors + slos, slos reference monitors
5
5
  LINE_UP = "\e[1A\033[K" # go up and clear
6
6
 
7
- def initialize(api, expected, project_filter: nil)
7
+ def initialize(api, expected, project_filter: nil, tracking_id_filter: nil)
8
8
  @api = api
9
9
  @project_filter = project_filter
10
+ @tracking_id_filter = tracking_id_filter
10
11
  @expected = Set.new expected # need set to speed up deletion
11
12
  calculate_diff
12
13
  validate_plan
@@ -105,7 +106,7 @@ module Kennel
105
106
 
106
107
  Progress.progress "Diffing" do
107
108
  populate_id_map @expected, actual
108
- filter_actual_by_project! actual
109
+ filter_actual! actual
109
110
  resolve_linked_tracking_ids! @expected # resolve dependencies to avoid diff
110
111
 
111
112
  @expected.each(&:add_tracking_id) # avoid diff with actual
@@ -261,11 +262,12 @@ module Kennel
261
262
  next unless tracking_id = a.fetch(:tracking_id)
262
263
 
263
264
  # ignore when deleted from the codebase
264
- # (when running with project filter we cannot see the other resources in the codebase)
265
+ # (when running with filters we cannot see the other resources in the codebase)
265
266
  api_resource = a.fetch(:klass).api_resource
266
267
  next if
267
268
  !@id_map.get(api_resource, tracking_id) &&
268
- (!project_prefixes || tracking_id.start_with?(*project_prefixes))
269
+ (!project_prefixes || tracking_id.start_with?(*project_prefixes)) &&
270
+ (!@tracking_id_filter || @tracking_id_filter.include?(tracking_id))
269
271
 
270
272
  @id_map.set(api_resource, tracking_id, a.fetch(:id))
271
273
  if a[:klass].api_resource == "synthetics/tests"
@@ -278,12 +280,18 @@ module Kennel
278
280
  list.each { |e| e.resolve_linked_tracking_ids!(@id_map, force: force) }
279
281
  end
280
282
 
281
- def filter_actual_by_project!(actual)
282
- return unless @project_filter
283
- project_prefixes = @project_filter&.map { |p| "#{p}:" }
284
- actual.select! do |a|
285
- tracking_id = a.fetch(:tracking_id)
286
- !tracking_id || tracking_id.start_with?(*project_prefixes)
283
+ def filter_actual!(actual)
284
+ if @tracking_id_filter
285
+ actual.select! do |a|
286
+ tracking_id = a.fetch(:tracking_id)
287
+ !tracking_id || @tracking_id_filter.include?(tracking_id)
288
+ end
289
+ elsif @project_filter
290
+ project_prefixes = @project_filter.map { |p| "#{p}:" }
291
+ actual.select! do |a|
292
+ tracking_id = a.fetch(:tracking_id)
293
+ !tracking_id || tracking_id.start_with?(*project_prefixes)
294
+ end
287
295
  end
288
296
  end
289
297
  end
@@ -1,4 +1,4 @@
1
1
  # frozen_string_literal: true
2
2
  module Kennel
3
- VERSION = "1.115.0"
3
+ VERSION = "1.116.0"
4
4
  end
data/lib/kennel.rb CHANGED
@@ -65,13 +65,22 @@ module Kennel
65
65
 
66
66
  def store(parts)
67
67
  Progress.progress "Storing" do
68
- old = Dir["generated/{#{(project_filter || ["**"]).join(",")}}/*"]
68
+ old = Dir[[
69
+ "generated",
70
+ if project_filter || tracking_id_filter
71
+ [
72
+ "{" + (project_filter || ["*"]).join(",") + "}",
73
+ "{" + (tracking_id_filter || ["*"]).join(",") + "}.json"
74
+ ]
75
+ else
76
+ "**"
77
+ end
78
+ ].join("/")]
69
79
  used = []
70
80
 
71
81
  Utils.parallel(parts, max: 2) do |part|
72
82
  path = "generated/#{part.tracking_id.tr("/", ":").sub(":", "/")}.json"
73
- used << File.dirname(path) # only 1 level of sub folders, so this is safe
74
- used << path
83
+ used.concat [File.dirname(path), path] # only 1 level of sub folders, so this is safe
75
84
  payload = part.as_json.merge(api_resource: part.class.api_resource)
76
85
  write_file_if_necessary(path, JSON.pretty_generate(payload) << "\n")
77
86
  end
@@ -94,7 +103,7 @@ module Kennel
94
103
  end
95
104
 
96
105
  def syncer
97
- @syncer ||= Syncer.new(api, generated, project_filter: project_filter)
106
+ @syncer ||= Syncer.new(api, generated, project_filter: project_filter, tracking_id_filter: tracking_id_filter)
98
107
  end
99
108
 
100
109
  def api
@@ -105,22 +114,12 @@ module Kennel
105
114
  @generated ||= begin
106
115
  Progress.progress "Generating" do
107
116
  load_all
108
- known = []
109
- filter = project_filter
110
-
111
- parts = Utils.parallel(Models::Project.recursive_subclasses) do |project_class|
112
- project = project_class.new
113
- kennel_id = project.kennel_id
114
- if filter
115
- known << kennel_id
116
- next [] unless filter.include?(kennel_id)
117
- end
118
- project.validated_parts
119
- end.flatten(1)
120
-
121
- if filter && parts.empty?
122
- raise "#{filter.join(", ")} does not match any projects, try any of these:\n#{known.uniq.sort.join("\n")}"
123
- end
117
+
118
+ projects = Models::Project.recursive_subclasses.map(&:new)
119
+ filter_resources!(projects, :kennel_id, project_filter, "projects", "PROJECT")
120
+
121
+ parts = Utils.parallel(projects, &:validated_parts).flatten(1)
122
+ filter_resources!(parts, :tracking_id, tracking_id_filter, "resources", "TRACKING_ID")
124
123
 
125
124
  parts.group_by(&:tracking_id).each do |tracking_id, same|
126
125
  next if same.size == 1
@@ -139,7 +138,25 @@ module Kennel
139
138
  end
140
139
 
141
140
  def project_filter
142
- ENV["PROJECT"]&.split(",")
141
+ raise "either use PROJECT= or TRACKING_ID=" if ENV["PROJECT"] && ENV["TRACKING_ID"]
142
+ ENV["PROJECT"]&.split(",") || tracking_id_filter&.map { |id| id.split(":", 2).first }
143
+ end
144
+
145
+ def tracking_id_filter
146
+ (tracking_id = ENV["TRACKING_ID"]) && tracking_id.split(",")
147
+ end
148
+
149
+ def filter_resources!(resources, by, against, name, env)
150
+ return unless against
151
+
152
+ before = resources.dup
153
+ resources.select! { |p| against.include?(p.send(by)) }
154
+ return if resources.size == against.size
155
+
156
+ raise <<~MSG.rstrip
157
+ #{env}=#{against.join(",")} matched #{resources.size} #{name}, try any of these:
158
+ #{before.map(&by).sort.uniq.join("\n")}
159
+ MSG
143
160
  end
144
161
 
145
162
  def load_all
@@ -148,7 +165,7 @@ module Kennel
148
165
  Dir.exist?("parts") && loader.push_dir("parts")
149
166
  loader.setup
150
167
 
151
- # TODO: also do projects and update expected path too
168
+ # TODO: also auto-load projects and update expected path too
152
169
  ["projects"].each do |folder|
153
170
  Dir["#{folder}/**/*.rb"].sort.each { |f| require "./#{f}" }
154
171
  end
data/template/Readme.md CHANGED
@@ -271,23 +271,27 @@ Remove the code that created the resource. The next update will delete it (see a
271
271
  ```
272
272
 
273
273
  ### Updating existing resources with id
274
-
275
274
  Setting `id` makes kennel take over a manually created datadog resource.
276
275
  When manually creating to import, it is best to remove the `id` and delete the manually created resource.
277
276
 
278
277
  When an `id` is set and the original resource is deleted, kennel will fail to update,
279
278
  removing the `id` will cause kennel to create a new resource in datadog.
280
279
 
281
- ### Organizing many projects
282
- Having many projects (and their sub-resources) can quickly get out of hand.
280
+ ### Organizing projects with many resources
281
+ When project files get too long, this structure can keep things bite-sized.
283
282
 
284
- Use this class structure to keep things organized:
285
283
  ```Ruby
286
284
  # projects/project_a/base.rb
287
285
  module ProjectA
288
286
  class Base < Kennel::Models::Project
289
287
  defaults(
290
288
  kennel_id: -> { "project_a" },
289
+ parts: -> {
290
+ [
291
+ Monitors::FooAlert.new(self),
292
+ ...
293
+ ]
294
+ }
291
295
  ...
292
296
 
293
297
  # projects/project_a/monitors/foo_alert.rb
@@ -297,7 +301,16 @@ module ProjectA
297
301
  ...
298
302
  ```
299
303
 
300
- On the command line, use the projects `kennel_id` (and if none is set then snake_case of the class name) to refer to the project. For example here`PROJECT=project_a` but if it were `Foo::ProjectA` then `foo_project_a`.
304
+ ### Updating a single project or resource
305
+
306
+ - Use `PROJECT=<kennel_id>` for single project:
307
+
308
+ Use the projects `kennel_id` (and if none is set then snake_case of the class name including modules)
309
+ to refer to the project. For example for `class ProjectA` use `PROJECT=project_a` but for `Foo::ProjectA` use `foo_project_a`.
310
+
311
+ - Use `TRACKING_ID=<project-kennel_id>:<resource-kennel_id>` for single resource:
312
+
313
+ Use the project kennel_id and the resources kennel_id, for example `class ProjectA` and `FooAlert` would give `project_a:foo_alert`.
301
314
 
302
315
  ### Skipping validations
303
316
  Some validations might be too strict for your usecase or just wrong, please [open an issue](https://github.com/grosser/kennel/issues) and
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: kennel
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.115.0
4
+ version: 1.116.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michael Grosser
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-08-28 00:00:00.000000000 Z
11
+ date: 2022-08-29 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: faraday