gitlab-labkit 5.10.0 → 5.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3cf9de3ab607ca2b1f4e24b04a55efc3081556acaf61c8fd4c451fd786aa632a
4
- data.tar.gz: 6a7d1d5fd0434e9298c41bfa6057f96ae08e37c3ea100f37db6d4879ecea688e
3
+ metadata.gz: adf3a44ad840a0b124ef408e68697340e1c62ca66e0965da67ed3e21aa608a92
4
+ data.tar.gz: d3ccd488458bc4c365c476217e6b1234bb7abec41efee94413d81b69497ff460
5
5
  SHA512:
6
- metadata.gz: 931e421f2c52d3d98602f3657088701bb5a48b2f98bd9289becffad20df9d991c3fc1d2f9b03d5ba6cbc26593ca38176227827afc66b973393302e1f789f89cd
7
- data.tar.gz: b971f78dfcf2529cda540c44e950651876cfd89e5d68af517c10f5013ead19d50c7f603d7ecc582787e790ab6ae1deb46dbc342b010676a48aff8964fb89b109
6
+ metadata.gz: 7bbbd90bd5eb0e65f794845225fd8707817bea9d8a80f47da64063b4e8e9a7c988177dae2ad0f44e0fb7ca22dca6dd2846e0fc2f2dd9db038c4554d2738d4364
7
+ data.tar.gz: c15ff6de4430a52692e33dd685ca88e4e080a12c6026f4c59f6520b91171113db5e9d13e319d1ac783e9dfea2b5648637ab4c0c8224feedc927ef2f70bc704ea
@@ -15,7 +15,8 @@ Gem::Specification.new do |spec|
15
15
  spec.metadata = { "source_code_uri" => "https://gitlab.com/gitlab-org/ruby/gems/labkit-ruby" }
16
16
  spec.license = "MIT"
17
17
 
18
- spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec|tools)/}) }
18
+ # The vendored .proto sources are read by the specs only; the gem ships the generated Ruby.
19
+ spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec|tools)/}) || f.end_with?(".proto") }
19
20
  spec.bindir = "exe"
20
21
  spec.executables = %w[labkit-logging]
21
22
  spec.require_paths = ["lib"]
@@ -724,19 +724,30 @@ A gem schema that lacks or changes a vendored field raises `Labkit::RateLimit::C
724
724
  So upgrade the `gitlab-ratelimit-config-v1` gem in the application first, then the `gitlab-labkit` release that updates the vendored copy.
725
725
 
726
726
  Loading runs in four steps, `parse`, `schema`, `contract` and `rule`, and a failure raises `Labkit::RateLimit::Config::Error` whose `#step` names the step.
727
- Only the parse step exists so far; the schema, contract and rule steps follow.
728
- Until the schema step lands, no mode validates a document: strict mode lets an enum number that no `Action` defines through, such as `action: 7`, and lenient mode leaves an unknown `action` name as `ACTION_UNSPECIFIED`.
729
- The schema step rejects both.
727
+ The parse and schema steps exist; the contract and rule steps follow in [Part 3 of the plan](https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/30263).
730
728
 
731
729
  ```ruby
732
730
  doc = Labkit::RateLimit::Config::Parser.parse(
733
731
  "config/rate_limits/rack_request.yaml",
734
732
  Labkit::RateLimit::Config::Proto::RateLimits,
735
- strict: true # CI rejects unknown fields. Production loads with strict: false once the schema step exists
733
+ strict: true # CI rejects unknown fields. Production loads with strict: false
736
734
  )
735
+ Labkit::RateLimit::Config::Schema.check!(doc)
737
736
  doc.rate_limits["rack_request"].rules.first.name
738
737
  ```
739
738
 
739
+ `Schema.check!` mirrors every protovalidate rule of the `gitlab.ratelimit.config.v1` schema: name format, the period presets, the `limit` and `ban_for_s` ranges, every list cap, and the message rules such as `rule.limit_requires_threshold`.
740
+ It accepts `AvailableLimiters`, `RateLimits` and `PublishRulesRequest`.
741
+ A failure lists every violation with the field path, message and rule id that protovalidate reports, such as `rate_limits["rack_request"].rules[0].period_s: must be in list [1, 15, 30, 60, 180, 300, 600, 900, 1800, 3600, 28800, 86400] (uint32.in)`, so a failure reads the same in Ruby and Go.
742
+ `Schema.violations(doc)` returns them without raising.
743
+ Inside one message the order is protovalidate's: message rules, then the oneof, then fields in declaration order.
744
+ A protobuf map keeps no document order and Go iterates it in an unspecified order, so Ruby reports map entries by sorted key and a cross-SDK comparison matches violations per entry, not by position.
745
+ Map keys in a path are quoted as Go's `strconv.Quote` prints them.
746
+ The check is the strict reading of the schema: a matcher whose operator this binary does not know fails `required`.
747
+ Lenient loading, through the loader that follows in Part 3, drops such a rule with a warning before the schema step, so a newer operator does not reject the file.
748
+ `scripts/vendor-ratelimit-proto.sh` also copies `config.proto` and `constraint.proto` next to the generated code, and `schema_spec.rb` parses them and compares every field rule (caps, bounds, presets, flags), every message rule id and message, and the name pattern with `Schema`, so a `PROTOS_REF` bump that changes any of those fails the suite until `Schema` follows.
749
+ The CEL expressions themselves are mirrored by hand and pinned by the examples.
750
+
740
751
  The parse step converts each YAML scalar by the type of the field it fills, so `eq: off`, `eq: True` and `eq: 9970` stay strings, `limit: "5"` becomes `5`, `action: "1"` becomes `ACTION_LIMIT`, and a YAML null leaves the field unset.
741
752
  A repeated key, also one repeated under its camelCase `json_name`, a merge key (`<<`), an empty file, more than one document, or a non-scalar key fails at step `parse`, in both modes.
742
753
  So does a value of the wrong kind for its field, such as a mapping where a scalar belongs, an integer out of range for its field, two fields of one `oneof`, such as `{ eq: a, re: b }`, a null list item or map value, and a document nested deeper than 64 levels.
@@ -22,8 +22,7 @@ module Labkit
22
22
  # number that no Action defines, such as `action: 7`, passes in both
23
23
  # modes, and in lenient mode protobuf drops an unknown enum name together
24
24
  # with the unknown fields, so a rule with a misspelt action reaches the
25
- # schema step as ACTION_UNSPECIFIED. The schema step rejects or drops
26
- # both.
25
+ # schema step as ACTION_UNSPECIFIED. Schema.check! rejects both.
27
26
  module Parser
28
27
  INT32 = (-(2**31))...(2**31)
29
28
  UINT32 = 0...(2**32)
@@ -0,0 +1,324 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Labkit
4
+ module RateLimit
5
+ module Config
6
+ # Schema is the schema step: the checks that gitlab/ratelimit/config/v1
7
+ # declares with protovalidate, mirrored in Ruby because there is no
8
+ # protovalidate runtime for Ruby. Each check keeps the protovalidate rule
9
+ # id, message and field path. Inside one message the order is
10
+ # protovalidate's: message rules, then the oneof, then fields in
11
+ # declaration order.
12
+ # A protobuf map keeps no document order and Go iterates it in an
13
+ # unspecified order, so Ruby reports map entries by sorted key and the
14
+ # corpus compares per entry.
15
+ #
16
+ # FIELD_RULES and MESSAGE_RULES are copied from the proto by hand, and
17
+ # schema_spec.rb compares both with the vendored config.proto and
18
+ # constraint.proto field by field, so a PROTOS_REF bump that changes a
19
+ # cap, a flag, a rule id or a message fails the suite until this file
20
+ # follows. The CEL expressions have no such check; the examples pin them.
21
+ #
22
+ # The check is the strict reading of the schema. Lenient loading drops a
23
+ # rule whose matcher has no known operator before this step (the loader,
24
+ # Task 3.3), because such a matcher fails `required` here.
25
+ module Schema
26
+ # for_key marks a violation on a map key; its path is the entry's path.
27
+ Violation = Data.define(:path, :rule_id, :message, :for_key) do
28
+ def initialize(path:, rule_id:, message:, for_key: false)
29
+ super
30
+ end
31
+
32
+ def to_s
33
+ path.empty? ? "#{message} (#{rule_id})" : "#{path}: #{message} (#{rule_id})"
34
+ end
35
+ end
36
+
37
+ # CEL matches() is RE2, where ^ and $ anchor the whole text; Ruby's $
38
+ # matches before a newline, so the mirror anchors with \A and \z.
39
+ NAME = /\A[a-z0-9_]{1,64}\z/
40
+ NAME_MESSAGE = "invalid name; must match ^[a-z0-9_]{1,64}$"
41
+ PERIODS = [1, 15, 30, 60, 180, 300, 600, 900, 1800, 3600, 28800, 86400].freeze
42
+ SKIP = :ACTION_SKIP
43
+
44
+ # The field rules of config.proto, keyed "Message.field". The keys of a
45
+ # map field and the items of a repeated field carry their rule under
46
+ # keys_* and items_*.
47
+ FIELD_RULES = {
48
+ "AvailableLimiters.available_limiters" => { min_pairs: 1, keys_is_name: true },
49
+ "LimiterSpec.description" => { max_len: 1024 },
50
+ "LimiterSpec.available_properties" => { unique: true, max_items: 100, items_is_name: true },
51
+ "LimiterSpec.default_rules" => { max_items: 200 },
52
+ "LimiterSpec.protected_rules" => { unique: true, max_items: 200, items_is_name: true },
53
+ "RateLimits.rate_limits" => { keys_is_name: true },
54
+ "RuleList.rules" => { max_items: 200 },
55
+ "Rule.name" => { is_name: true },
56
+ "Rule.description" => { max_len: 1024 },
57
+ "Rule.action" => { required: true, defined_only: true },
58
+ "Rule.characteristics" => { unique: true, max_items: 10, items_is_name: true },
59
+ "Rule.match" => { max_pairs: 20, keys_is_name: true },
60
+ "Rule.limit" => { gte: 1, lte: 1_000_000 },
61
+ "Rule.period_s" => { in: PERIODS },
62
+ "Rule.count_distinct" => { is_name: true, ignore_empty: true },
63
+ "Rule.ban_for_s" => { gte: 1, lte: 86_400 },
64
+ "PublishRulesRequest.rate_limits" => { keys_is_name: true },
65
+ "PublishRulesRequest.remove" => { keys_is_name: true },
66
+ "RuleNames.names" => { min_items: 1, max_items: 200, unique: true, items_is_name: true },
67
+ "Matcher.eq" => { max_len: 1024 },
68
+ "Matcher.ne" => { max_len: 1024 },
69
+ "Matcher.re" => { max_len: 1024 },
70
+ "Matcher.nre" => { max_len: 1024 },
71
+ "StringList.values" => { min_items: 1, max_items: 100, unique: true, items_max_len: 1024 },
72
+ "MatcherList.matchers" => { min_items: 1, max_items: 20 }
73
+ }.freeze
74
+
75
+ # The message-level CEL rules of config.proto, by id.
76
+ MESSAGE_RULES = {
77
+ "limiter_spec.protected_rules_declared" => "every protected rule must be a default rule of this limiter",
78
+ "limiter_spec.unique_default_names" => "default rule names must be unique within a limiter",
79
+ "limiter_spec.no_position_in_defaults" => "position is not allowed on a default rule",
80
+ "rule_list.unique_names" => "rule names must be unique within a limiter",
81
+ "rule.limit_requires_threshold" => "limit and period_s are required unless action is skip",
82
+ "rule.ban_excludes_count_distinct" => "ban_for_s cannot be combined with count_distinct",
83
+ "rule.ban_not_on_skip" => "ban_for_s is not allowed on a skip rule",
84
+ "rule.no_counter_on_skip" => "limit, period_s, characteristics and count_distinct are not allowed on a skip rule",
85
+ "rule.count_distinct_not_characteristic" => "count_distinct cannot also be a characteristic",
86
+ "publish.name_in_both" => "a rule name cannot be in both rate_limits and remove for one limiter",
87
+ "matcher_list.no_nested_all" => "all cannot contain another all"
88
+ }.freeze
89
+
90
+ # The ASCII characters that strconv.Quote prints with a backslash escape.
91
+ ESCAPES = {
92
+ '"' => '\\"', "\\" => "\\\\", "\a" => "\\a", "\b" => "\\b", "\f" => "\\f",
93
+ "\n" => "\\n", "\r" => "\\r", "\t" => "\\t", "\v" => "\\v"
94
+ }.freeze
95
+
96
+ class << self
97
+ # @param message [Proto::AvailableLimiters, Proto::RateLimits, Proto::PublishRulesRequest]
98
+ # @return the message
99
+ # @raise [Error] step :schema, every violation joined with "; "
100
+ def check!(message)
101
+ found = violations(message)
102
+ raise Error.new(:schema, found.join("; ")) unless found.empty?
103
+
104
+ message
105
+ end
106
+
107
+ # @return [Array<Violation>] every violation, in the order described above
108
+ def violations(message)
109
+ checker = Checker.new
110
+ case message
111
+ when Proto::AvailableLimiters then checker.available_limiters(message)
112
+ when Proto::RateLimits then checker.rate_limits(message)
113
+ when Proto::PublishRulesRequest then checker.publish_rules_request(message)
114
+ else
115
+ raise ArgumentError, "no schema for #{message.class}; expected AvailableLimiters, RateLimits or PublishRulesRequest"
116
+ end
117
+ checker.violations
118
+ end
119
+
120
+ # Quotes a map key for a path the way Go's strconv.Quote does, so the
121
+ # path reads the same in every SDK: backslash escapes for the ASCII
122
+ # controls that have one, \xNN for the other ASCII controls and DEL,
123
+ # \uNNNN or \UNNNNNNNN for a non-printable rune, everything else as is.
124
+ def quote(key)
125
+ out = +'"'
126
+ key.each_char { |c| out << (ESCAPES[c] || escape_rune(c)) }
127
+ out << '"'
128
+ end
129
+
130
+ private
131
+
132
+ # Go's unicode.IsPrint: letters, marks, numbers, punctuation, symbols
133
+ # and the ASCII space.
134
+ def escape_rune(char)
135
+ code = char.ord
136
+ return format("\\x%02x", code) if code < 0x20 || code == 0x7f
137
+ return char if char == " " || char.match?(/\A[\p{L}\p{M}\p{N}\p{P}\p{S}]\z/)
138
+
139
+ code > 0xffff ? format("\\U%08x", code) : format("\\u%04x", code)
140
+ end
141
+ end
142
+
143
+ # Walks one message and collects violations. Paths follow protovalidate:
144
+ # `rate_limits["rack_request"].rules[0].match["plan"].eq`.
145
+ class Checker
146
+ attr_reader :violations
147
+
148
+ def initialize
149
+ @violations = []
150
+ end
151
+
152
+ def available_limiters(doc)
153
+ map_field("available_limiters", doc.available_limiters, FIELD_RULES["AvailableLimiters.available_limiters"]) { |path, spec| limiter_spec(path, spec) }
154
+ end
155
+
156
+ def rate_limits(doc)
157
+ map_field("rate_limits", doc.rate_limits, FIELD_RULES["RateLimits.rate_limits"]) { |path, list| rule_list(path, list) }
158
+ end
159
+
160
+ def publish_rules_request(req)
161
+ # The outcome of one name in both fields would depend on the order
162
+ # the store applies them.
163
+ both = req.rate_limits.any? do |key, list|
164
+ req.remove.has_key?(key) && list.rules.any? { |r| req.remove[key].names.include?(r.name) }
165
+ end
166
+ message_rule("", "publish.name_in_both") if both
167
+ map_field("rate_limits", req.rate_limits, FIELD_RULES["PublishRulesRequest.rate_limits"]) { |path, list| rule_list(path, list) }
168
+ map_field("remove", req.remove, FIELD_RULES["PublishRulesRequest.remove"]) do |path, names|
169
+ list("#{path}.names", names.names, FIELD_RULES["RuleNames.names"])
170
+ end
171
+ end
172
+
173
+ private
174
+
175
+ def limiter_spec(path, spec)
176
+ default_names = spec.default_rules.map(&:name)
177
+ message_rule(path, "limiter_spec.protected_rules_declared") unless spec.protected_rules.all? { |n| default_names.include?(n) }
178
+ message_rule(path, "limiter_spec.unique_default_names") unless default_names.uniq.size == default_names.size
179
+ # Default rules keep document order, so a position has no meaning there.
180
+ message_rule(path, "limiter_spec.no_position_in_defaults") if spec.default_rules.any?(&:has_position?)
181
+
182
+ string("#{path}.description", spec.description, FIELD_RULES["LimiterSpec.description"])
183
+ list("#{path}.available_properties", spec.available_properties, FIELD_RULES["LimiterSpec.available_properties"])
184
+ list("#{path}.default_rules", spec.default_rules, FIELD_RULES["LimiterSpec.default_rules"])
185
+ spec.default_rules.each_with_index { |r, i| rule("#{path}.default_rules[#{i}]", r) }
186
+ list("#{path}.protected_rules", spec.protected_rules, FIELD_RULES["LimiterSpec.protected_rules"])
187
+ end
188
+
189
+ def rule_list(path, list)
190
+ names = list.rules.map(&:name)
191
+ message_rule(path, "rule_list.unique_names") unless names.uniq.size == names.size
192
+ list("#{path}.rules", list.rules, FIELD_RULES["RuleList.rules"])
193
+ list.rules.each_with_index { |r, i| rule("#{path}.rules[#{i}]", r) }
194
+ end
195
+
196
+ def rule(path, rule)
197
+ rule_message_rules(path, rule)
198
+ string("#{path}.name", rule.name, FIELD_RULES["Rule.name"])
199
+ string("#{path}.description", rule.description, FIELD_RULES["Rule.description"])
200
+ action("#{path}.action", rule.action, FIELD_RULES["Rule.action"])
201
+ list("#{path}.characteristics", rule.characteristics, FIELD_RULES["Rule.characteristics"])
202
+ map_field("#{path}.match", rule.match, FIELD_RULES["Rule.match"]) { |entry, matcher| matcher(entry, matcher) }
203
+ integer("#{path}.limit", rule.limit, FIELD_RULES["Rule.limit"]) if rule.has_limit?
204
+ integer("#{path}.period_s", rule.period_s, FIELD_RULES["Rule.period_s"]) if rule.has_period_s?
205
+ string("#{path}.count_distinct", rule.count_distinct, FIELD_RULES["Rule.count_distinct"])
206
+ integer("#{path}.ban_for_s", rule.ban_for_s, FIELD_RULES["Rule.ban_for_s"]) if rule.has_ban_for_s?
207
+ end
208
+
209
+ # In the order config.proto declares them.
210
+ def rule_message_rules(path, rule)
211
+ counts_distinct = !rule.count_distinct.empty?
212
+ message_rule(path, "rule.limit_requires_threshold") unless rule.action == SKIP || (rule.has_limit? && rule.has_period_s?)
213
+ message_rule(path, "rule.ban_excludes_count_distinct") if rule.has_ban_for_s? && counts_distinct
214
+ skip_rules(path, rule) if rule.action == SKIP
215
+ message_rule(path, "rule.count_distinct_not_characteristic") if counts_distinct && rule.characteristics.include?(rule.count_distinct)
216
+ end
217
+
218
+ # A skip rule has no counter and no limit to cross.
219
+ def skip_rules(path, rule)
220
+ counter = rule.has_limit? || rule.has_period_s? || !rule.characteristics.empty? || !rule.count_distinct.empty?
221
+ message_rule(path, "rule.ban_not_on_skip") if rule.has_ban_for_s?
222
+ message_rule(path, "rule.no_counter_on_skip") if counter
223
+ end
224
+
225
+ # An unknown enum number decodes as an Integer in Ruby.
226
+ def action(path, value, rules)
227
+ if rules[:required] && value == :ACTION_UNSPECIFIED
228
+ add(path, "required", "value is required")
229
+ elsif rules[:defined_only] && value.is_a?(Integer)
230
+ add(path, "enum.defined_only", "value must be one of the defined enum values")
231
+ end
232
+ end
233
+
234
+ def matcher(path, matcher)
235
+ op = matcher.op
236
+ return add("#{path}.op", "required", "exactly one field is required in oneof") if op.nil?
237
+
238
+ case op
239
+ when :eq, :ne, :re, :nre then string("#{path}.#{op}", matcher[op.to_s], FIELD_RULES["Matcher.#{op}"])
240
+ when :one_of, :none_of then list("#{path}.#{op}.values", matcher[op.to_s].values, FIELD_RULES["StringList.values"])
241
+ when :all then matcher_list("#{path}.all", matcher.all)
242
+ end
243
+ end
244
+
245
+ def matcher_list(path, list)
246
+ # There is no `any`, so an `all` inside an `all` is one flat list.
247
+ message_rule(path, "matcher_list.no_nested_all") if list.matchers.any? { |m| m.op == :all }
248
+ list("#{path}.matchers", list.matchers, FIELD_RULES["MatcherList.matchers"])
249
+ list.matchers.each_with_index { |m, i| matcher("#{path}.matchers[#{i}]", m) }
250
+ end
251
+
252
+ # Rules of RepeatedRules in field order: min_items, max_items, unique,
253
+ # then the items.
254
+ def list(path, values, rules)
255
+ add(path, "repeated.min_items", "must contain at least #{rules[:min_items]} item(s)") if rules[:min_items] && values.size < rules[:min_items]
256
+ add(path, "repeated.max_items", "must contain no more than #{rules[:max_items]} item(s)") if rules[:max_items] && values.size > rules[:max_items]
257
+ add(path, "repeated.unique", "repeated value must contain unique items") if rules[:unique] && values.uniq.size != values.size
258
+ values.each_with_index { |v, i| name("#{path}[#{i}]", v) } if rules[:items_is_name]
259
+ values.each_with_index { |v, i| max_len("#{path}[#{i}]", v, rules[:items_max_len]) } if rules[:items_max_len]
260
+ end
261
+
262
+ # Rules of MapRules in field order: min_pairs, max_pairs, then the
263
+ # keys. Yields each entry's path and value.
264
+ def map_field(path, map, rules)
265
+ add(path, "map.min_pairs", "map must be at least #{rules[:min_pairs]} entries") if rules[:min_pairs] && map.length < rules[:min_pairs]
266
+ add(path, "map.max_pairs", "map must be at most #{rules[:max_pairs]} entries") if rules[:max_pairs] && map.length > rules[:max_pairs]
267
+ map.keys.sort.each do |key|
268
+ value = map[key]
269
+ entry = "#{path}[#{Schema.quote(key)}]"
270
+ add(entry, "string.is_name", NAME_MESSAGE, for_key: true) if rules[:keys_is_name] && !NAME.match?(key)
271
+ yield entry, value
272
+ end
273
+ end
274
+
275
+ def string(path, value, rules)
276
+ return if rules[:ignore_empty] && value.empty?
277
+
278
+ max_len(path, value, rules[:max_len]) if rules[:max_len]
279
+ name(path, value) if rules[:is_name]
280
+ end
281
+
282
+ # protovalidate reports one of three ids for bounds, by which bounds
283
+ # the field declares.
284
+ def integer(path, value, rules)
285
+ if rules[:in]
286
+ add(path, "uint32.in", "must be in list #{rules[:in]}") unless rules[:in].include?(value)
287
+ return
288
+ end
289
+
290
+ gte, lte = rules.values_at(:gte, :lte)
291
+ below = gte && value < gte
292
+ above = lte && value > lte
293
+ return unless below || above
294
+
295
+ if gte && lte
296
+ add(path, "uint32.gte_lte", "must be greater than or equal to #{gte} and less than or equal to #{lte}")
297
+ elsif below
298
+ add(path, "uint32.gte", "must be greater than or equal to #{gte}")
299
+ else
300
+ add(path, "uint32.lte", "must be less than or equal to #{lte}")
301
+ end
302
+ end
303
+
304
+ def message_rule(path, rule_id)
305
+ add(path, rule_id, MESSAGE_RULES.fetch(rule_id))
306
+ end
307
+
308
+ def name(path, value)
309
+ add(path, "string.is_name", NAME_MESSAGE) unless NAME.match?(value)
310
+ end
311
+
312
+ def max_len(path, value, max)
313
+ add(path, "string.max_len", "must be at most #{max} characters") if value.length > max
314
+ end
315
+
316
+ def add(path, rule_id, message, for_key: false)
317
+ @violations << Violation.new(path: path, rule_id: rule_id, message: message, for_key: for_key)
318
+ end
319
+ end
320
+ private_constant :Checker
321
+ end
322
+ end
323
+ end
324
+ end
@@ -17,6 +17,7 @@ module Labkit
17
17
 
18
18
  autoload :Error, "labkit/rate_limit/config/error"
19
19
  autoload :Parser, "labkit/rate_limit/config/parser"
20
+ autoload :Schema, "labkit/rate_limit/config/schema"
20
21
  end
21
22
  end
22
23
  end
@@ -1,5 +1,6 @@
1
1
  #!/usr/bin/env bash
2
- # Copies the generated Ruby code for gitlab.ratelimit.config.v1 from gitlab-org/protos.
2
+ # Copies the generated Ruby code and the proto sources for gitlab.ratelimit.config.v1
3
+ # from gitlab-org/protos. The sources are what schema_spec.rb checks Schema against.
3
4
  # A published gem cannot depend on the protos Bundler git gem, so the file is
4
5
  # vendored and CI checks that it matches PROTOS_REF (job vendored-proto-check).
5
6
  set -euo pipefail
@@ -7,12 +8,20 @@ set -euo pipefail
7
8
  # Pinned protos module tag. Bump it when a newer schema release is needed.
8
9
  PROTOS_REF="${PROTOS_REF:-gen/ruby/gitlab/ratelimit/config/v1/v0.1.0}"
9
10
  SOURCE_PATH="gen/ruby/gitlab/ratelimit/config/v1/lib/gitlab/ratelimit/config/v1/config_pb.rb"
10
- TARGET="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/lib/labkit/rate_limit/config/proto/config_pb.rb"
11
+ PROTO_DIR="proto/ratelimit/gitlab/ratelimit/config/v1"
12
+ TARGET_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/lib/labkit/rate_limit/config/proto"
13
+ TARGET="${TARGET_DIR}/config_pb.rb"
11
14
 
12
- url="https://gitlab.com/gitlab-org/protos/-/raw/${PROTOS_REF}/${SOURCE_PATH}"
13
- tmp="$(mktemp)"
14
- trap 'rm -f "$tmp"' EXIT
15
- curl --fail --silent --show-error --location "$url" -o "$tmp"
15
+ raw="https://gitlab.com/gitlab-org/protos/-/raw/${PROTOS_REF}"
16
+ # Everything lands in a temp dir first, so a failed download or check
17
+ # leaves the vendored files as they were.
18
+ work="$(mktemp -d)"
19
+ trap 'rm -rf "$work"' EXIT
20
+ tmp="${work}/config_pb.rb"
21
+ curl --fail --silent --show-error --location "${raw}/${SOURCE_PATH}" -o "$tmp"
22
+ for proto in config.proto constraint.proto; do
23
+ curl --fail --silent --show-error --location "${raw}/${PROTO_DIR}/${proto}" -o "${work}/${proto}"
24
+ done
16
25
 
17
26
  # The generated file registers gitlab/ratelimit/config/v1/config.proto in the
18
27
  # global descriptor pool. The protos gem registers the same file name, so the
@@ -27,10 +36,12 @@ guard='return unless Labkit::RateLimit::Config::VendoredProto.register(pool, des
27
36
  grep -v '^# frozen_string_literal: true$' "$tmp" |
28
37
  sed -e "s|^require 'google/protobuf'\$|require 'google/protobuf'\nrequire 'labkit/rate_limit/config/vendored_proto'|" \
29
38
  -e "s|^pool.add_serialized_file(descriptor_data)\$|${guard}|"
30
- } >"$TARGET"
31
- if ! grep -q 'VendoredProto.register' "$TARGET" ||
32
- ! grep -qx "require 'labkit/rate_limit/config/vendored_proto'" "$TARGET"; then
39
+ } >"${work}/vendored.rb"
40
+ if ! grep -q 'VendoredProto.register' "${work}/vendored.rb" ||
41
+ ! grep -qx "require 'labkit/rate_limit/config/vendored_proto'" "${work}/vendored.rb"; then
33
42
  echo "vendor: the generated file no longer requires VendoredProto or registers the descriptor as expected; update the sed in this script" >&2
34
43
  exit 1
35
44
  fi
36
- echo "vendored ${SOURCE_PATH}@${PROTOS_REF} -> ${TARGET#"$PWD"/}"
45
+ mv "${work}/vendored.rb" "$TARGET"
46
+ mv "${work}/config.proto" "${work}/constraint.proto" "$TARGET_DIR/"
47
+ echo "vendored ${SOURCE_PATH}, config.proto and constraint.proto @${PROTOS_REF} -> ${TARGET_DIR#"$PWD"/}"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: gitlab-labkit
3
3
  version: !ruby/object:Gem::Version
4
- version: 5.10.0
4
+ version: 5.11.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew Newdigate
@@ -655,6 +655,7 @@ files:
655
655
  - lib/labkit/rate_limit/config/error.rb
656
656
  - lib/labkit/rate_limit/config/parser.rb
657
657
  - lib/labkit/rate_limit/config/proto/config_pb.rb
658
+ - lib/labkit/rate_limit/config/schema.rb
658
659
  - lib/labkit/rate_limit/config/vendored_proto.rb
659
660
  - lib/labkit/rate_limit/configuration.rb
660
661
  - lib/labkit/rate_limit/evaluator.rb