gitlab-labkit 3.0.1 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 5fbd752b05c1341ef1292c7a3a1ae3555dd27230d7afc830e0c713547bdd02ee
4
- data.tar.gz: 6dc8eab0332b9b6851627fe83a95bfb0bacefbe3222d712e28a8e210c19fdb8a
3
+ metadata.gz: c872947164d25ef7f75203c6195fe04bbc8c9002639d59fadf10672d47201862
4
+ data.tar.gz: d0308016fd538669e466d2e499aa283d47e82bb4d53bd530c07bce74162b16b4
5
5
  SHA512:
6
- metadata.gz: 56316fcea7b575a62119a814f3e3b4c85dc8d5e93ed72e2316bd47388bd142dd44ae21f07f2b6f40d070573c04bf6b79c26e14601f90e0a2f6fcf263fa05ce36
7
- data.tar.gz: 65e7e06919637356e25ab5d6f56880df2eb7e340474afeaf01212fe55612439d8bba8707d2d1bc835c864723cd0d15cca135065756ce3f17a9bd959f253775f3
6
+ metadata.gz: e6cb4000c77a3ebe764b61e0d536db95dbe3c79378a1d95744747e7eff2509cb435561eba2963ae83107eea37364f5515e3a64bd9491c799c2b9508c9293c793
7
+ data.tar.gz: 64cc36f40cb6140b09fc54d873a102098a7fe05b2e967960317c782dc53b224c513fe676e6435dae48b6b0fc33e9e95dd8f32f10230900dea809bbef8bdd03e0
@@ -77,8 +77,8 @@ RACK_LIMITER = Labkit::RateLimit::Limiter.new(
77
77
  - `name` must match `/\A[a-z0-9_]+\z/`. It is used as the first segment of
78
78
  every Redis counter key for this limiter, so renaming a `Limiter` abandons
79
79
  any in-flight counters.
80
- - `rules` is an ordered array of `Rule` objects. The first rule whose `match`
81
- hash is satisfied wins (with the exception of `:log` rules — see [Actions](#actions)).
80
+ - `rules` is an ordered array of `Rule` objects. Every rule whose `match` hash
81
+ is satisfied is evaluated and counted (see [Evaluation flow](#evaluation-flow)).
82
82
  - `redis` and `logger` are optional; they fall back to the global
83
83
  `Labkit::RateLimit.config` values.
84
84
 
@@ -99,7 +99,7 @@ result = RACK_LIMITER.check(
99
99
  endpoint: request.path
100
100
  )
101
101
 
102
- if result.exceeded? && result.action == :block
102
+ if result.action == :block
103
103
  response.headers.merge!(result.to_response_headers)
104
104
  render plain: "Too Many Requests", status: 429
105
105
  return
@@ -115,9 +115,11 @@ the same counter.
115
115
  `Limiter#peek(identifier)` returns the same `Result` shape but does not
116
116
  mutate Redis. It is useful when one code path should account for the request
117
117
  (`check`) and another should gate a side-effect on whether the caller is
118
- already over-limit. `peek` skips `:log` rules their state is unobservable
119
- without incrementing. A matched `:skip` rule terminates `peek` the same way
120
- it terminates `check`: matched, `:allow`, no Redis read, no `info`.
118
+ already over-limit. `peek` reads `:log` rules too: `check` can report a
119
+ `:log` rule's evaluation, so excluding them would make `peek` answer a
120
+ different question than `check`. A matched `:skip` rule terminates `peek`
121
+ the same way it terminates `check`: matched, `:allow`, no Redis read, no
122
+ `info`.
121
123
 
122
124
  ## Identifier
123
125
 
@@ -146,7 +148,7 @@ A `Rule` is a `Data.define` value object with the following fields:
146
148
  | `match` | Hash of identifier key/value predicates that must **all** be satisfied for the rule to apply. Empty hash matches anything. See [Matchers](#matchers). |
147
149
  | `limit` | Integer request threshold per `period`. May be a callable resolved on every check. |
148
150
  | `period` | Window length in seconds. May be a callable resolved on every check. |
149
- | `action` | What the result reports when the limit is exceeded. One of `:block`, `:log`, `:skip`. Default `:block`. See [Actions](#actions). |
151
+ | `action` | What the rule does when it matches. One of `:limit`, `:log`, `:skip`. Default `:limit`. See [Actions](#actions). |
150
152
  | `characteristics` | Array of identifier keys whose values are folded into the Redis counter key. Each unique combination gets its own counter. |
151
153
 
152
154
  Making `limit` or `period` callable is the supported pattern for
@@ -180,10 +182,37 @@ Glob and prefix matchers are intentionally out of scope.
180
182
 
181
183
  ### Evaluation flow
182
184
 
183
- `check` walks the rule list in order. The first **terminating** rule wins;
184
- `:log` rules count but do not terminate, so they cannot disable a following
185
- `:block` rule. A pure `:log`-only path still emits one `rule="unmatched"`
186
- metric increment because no terminating rule fired.
185
+ `check` walks the rule list in order and evaluates **every** rule that matches:
186
+ each one increments its own counter, and the request debits `cost` from all of
187
+ them. Matching alone does not stop the walk. Only two things terminate it:
188
+
189
+ - a matched `:skip` rule (bypass, nothing counted), and
190
+ - a `:limit` rule that is **over its limit** — the request is rejected, so the
191
+ remaining rules cannot change the outcome. Rules declared after it are neither
192
+ counted nor evaluated, meaning a blocked request debits every rule up to and
193
+ including the one that blocked, and none after it.
194
+
195
+ A `:limit` rule that is under its limit does not terminate, so it behaves
196
+ exactly like `:log` until the moment it blocks.
197
+
198
+ The returned `Result` collects one `Result::Evaluation` per counted rule
199
+ (`result.evaluations`), and its readers (`rule`, `info`, `exceeded?`,
200
+ `to_response_headers`) report the **most constraining** one: a blocking
201
+ evaluation first, then exceeded ones, then the fewest requests remaining,
202
+ ties broken by declaration order (the ranking is `Result::Evaluation#<=>`).
203
+ That is what keeps `to_response_headers` honest — reporting any other matched
204
+ rule would advertise headroom that a different rule is about to refuse:
205
+
206
+ ```
207
+ per_ip limit 1000 count 20 remaining 980
208
+ per_user limit 100 count 99 remaining 1 <- reported
209
+ ```
210
+
211
+ An exceeded rule has `remaining` 0 and therefore outranks any rule still under
212
+ its limit. For a `:log` rule that means shadow traffic surfaces to the caller
213
+ as `exceeded? == true` (with `action` still `:allow`) — visible, but unable to
214
+ block. A `:log`-only path that matches therefore returns `matched? == true`
215
+ and emits no `rule="unmatched"` metric.
187
216
 
188
217
  ```mermaid
189
218
  flowchart TD
@@ -194,31 +223,53 @@ flowchart TD
194
223
  Skip -->|"yes (no Redis op)"| SkipEmit[Emit calls_total<br/>action=skip]
195
224
  SkipEmit --> SkipReturn([Return matched=true<br/>action=:allow])
196
225
  Skip -->|no| Eval["INCR Redis counter<br/>(see Redis sequence below)"]
197
- Eval --> Build[Build Result<br/>resolve limit/period]
198
- Build --> Emit[Emit calls_total + limit/period gauges]
199
- Emit --> Act{rule.action}
200
- Act -->|":log<br/>(non-terminating)"| Iter
201
- Act -->|:block| Return([Return Result])
202
- Iter -->|no more rules| Unmatched[Emit calls_total<br/>rule=unmatched, action=allow]
226
+ Eval --> Build[Build Evaluation<br/>resolve limit/period]
227
+ Build --> Add[Add evaluation to Result]
228
+ Add --> Emit[Emit calls_total + limit/period gauges]
229
+ Emit --> Act{"result.block?<br/>(:limit rule over limit)"}
230
+ Act -->|yes| Return([Return Result<br/>action=:block])
231
+ Act -->|"no (:log, or :limit under limit)"| Iter
232
+ Iter -->|no more rules| Any{any rule<br/>evaluated?}
233
+ Any -->|yes| ReturnFold([Return Result reporting<br/>most-constraining evaluation])
234
+ Any -->|no| Unmatched[Emit calls_total<br/>rule=unmatched, action=allow]
203
235
  Unmatched --> ReturnUnmatched([Return matched=false<br/>action=:allow])
204
236
  Eval -. StandardError .-> Error[Emit errors_total<br/>log warn]
205
237
  Error --> ReturnErr([Return error=true<br/>action=:allow])
206
238
  ```
207
239
 
240
+ Error handling is whole-check, not per-rule: a Redis failure part-way through
241
+ discards the verdicts of the rules already evaluated, even though their counters
242
+ were incremented. Those requests are counted but produce no verdict — the
243
+ fail-open trade-off is that a request is never blocked on a partial evaluation.
244
+
245
+ Note that `calls_total` is emitted **per matched rule**, so summing it by
246
+ `rate_limiter` counts rule evaluations, not requests.
247
+
208
248
  ### Actions
209
249
 
210
- The rule's `action` controls how the `Result` reports an over-limit hit. The
211
- counter is always incremented when a rule matches, except for `:skip` rules,
212
- which never touch Redis:
213
-
214
- - `:block` — when exceeded, `Result#action` is `:block`. Caller should reject
215
- the request (e.g. with HTTP 429). When under the limit, action is `:allow`.
216
- - `:log` — **non-terminating**. The rule counts the request and records
217
- metrics, but evaluation continues to the next rule. This is the mechanism
218
- for shadow rules during rollout: stack a `:log` rule and a `:block` rule
219
- together and the `:log` rule cannot disable the `:block` rule. Note that a
220
- pure `:log`-only check still emits one `rule="unmatched"` metric entry
221
- because no terminating rule fired.
250
+ The rule's `action` describes what the rule does; the result's `action`
251
+ describes the outcome what the caller should do and is only ever `:allow`
252
+ or `:block`. The counter is always incremented when a rule matches, except for
253
+ `:skip` rules, which never touch Redis:
254
+
255
+ | rule action | what it does | exceeded? | result action | terminating? |
256
+ |-------------|-----------------------------------------------|-----------|---------------|---------------|
257
+ | `:limit` | count against the limit | no | `:allow` | no continue |
258
+ | `:limit` | count against the limit | yes | `:block` | yes stop |
259
+ | `:log` | count against the limit (observability only) | no | `:allow` | no continue |
260
+ | `:log` | count against the limit (observability only) | yes | `:allow` | no — continue |
261
+ | `:skip` | don't count (bypass) | n/a | `:allow` | yes — stop |
262
+
263
+ - `:limit` — when exceeded, `Result#action` is `:block` and evaluation
264
+ terminates. Caller should reject the request (e.g. with HTTP 429). When under
265
+ the limit, evaluation continues to the next rule.
266
+ - `:log` — **never terminating and never blocking**. The rule counts the
267
+ request and records metrics, but evaluation always continues to the next
268
+ rule. This is the mechanism for shadow rules during rollout: stack a `:log`
269
+ rule and a `:limit` rule together and the `:log` rule cannot disable the
270
+ `:limit` rule. When exceeded, a `:log` rule can still be the rule reported in
271
+ the `Result` (`exceeded? == true`, `action == :allow`) if nothing more
272
+ constraining matched.
222
273
  - `:skip` — bypass. A matching rule terminates evaluation with
223
274
  `Result#action` `:allow` **without any Redis operation**: nothing is
224
275
  counted, so `limit`, `period`, `characteristics`, and `count_distinct` are
@@ -277,21 +328,25 @@ no reason to read it again before mutating.
277
328
 
278
329
  `peek` does not use a script: it pipelines `GET` + `TTL` (or `SCARD` + `TTL`)
279
330
  and never issues `EXPIRE`, so it cannot start or extend a window. A missing
280
- key (`GET → nil`, `TTL → -2`) is reported as `count = 0`, and `build_result`
331
+ key (`GET → nil`, `TTL → -2`) is reported as `count = 0`, and `build_evaluation`
281
332
  falls back to the rule's period for `reset_at` since there is no Redis-side
282
333
  window to read.
283
334
 
284
335
  ## Result
285
336
 
286
- `Result` carries the decision back to the caller:
337
+ `Result` carries the decision back to the caller. It collects one
338
+ `Result::Evaluation` per counted rule; `rule`, `exceeded?`, and `info` report
339
+ the most constraining one (see [Evaluation flow](#evaluation-flow)):
287
340
 
288
341
  ```ruby
289
342
  result.matched? # => true if some rule matched
290
- result.exceeded? # => true if the matched rule's counter > limit
291
- result.action # => :block | :log | :allow
292
- result.rule # => the matched Rule, or nil
343
+ result.exceeded? # => true if the reported rule's counter > limit
344
+ result.action # => :block | :allow what the caller should do
345
+ result.block? # => result.action == :block
346
+ result.rule # => the reported Rule, or nil
293
347
  result.error? # => true if Redis failed (see Fail-open)
294
348
  result.info # => Result::Info or nil
349
+ result.evaluations # => every counted Result::Evaluation, in rule order
295
350
  result.to_response_headers
296
351
  # => { "RateLimit-Limit" => "...", "RateLimit-Remaining" => "...", "RateLimit-Reset" => "<unix-ts>" }
297
352
  ```
@@ -313,8 +368,8 @@ safe to merge unconditionally.
313
368
 
314
369
  The evaluator wraps `check` and `peek` in a broad rescue. Any `StandardError`
315
370
  (Redis connection failure, timeout, OOM in user-supplied callables, …) is
316
- logged at WARN with `message: "rate_limit_error"` and returned as a
317
- `Result(matched: false, error: true, action: :allow)`. The
371
+ logged at WARN with `message: "rate_limit_error"` and returned as an error
372
+ `Result` (`matched?` false, `error?` true, `action` `:allow`). The
318
373
  `gitlab_labkit_rate_limiter_errors_total` counter is incremented. The caller
319
374
  should treat the request as allowed.
320
375
 
@@ -325,15 +380,14 @@ should treat the request as allowed.
325
380
 
326
381
  | metric | type | labels | meaning |
327
382
  |-------------------------------------------------|---------|-------------------------------------|----------------------------------------------------------------------|
328
- | `gitlab_labkit_rate_limiter_calls_total` | counter | `rate_limiter`, `rule`, `action` | One increment per terminating decision; also incremented per matched `:log` rule. `action` is one of `"allow"`, `"block"`, `"log"`, `"skip"`. `rule="unmatched", action="allow"` when no rule terminated. |
383
+ | `gitlab_labkit_rate_limiter_calls_total` | counter | `rate_limiter`, `rule`, `action` | One increment per counted rule (plus one per matched `:skip` rule). `action` is the rule-level outcome: `"allow"` (under limit), `"limit"` (blocking `:limit` rule), `"log"` (exceeded `:log` rule), `"skip"`. `rule="unmatched", action="allow"` when no rule matched. |
329
384
  | `gitlab_labkit_rate_limiter_errors_total` | counter | `rate_limiter` | Fail-open events (any `StandardError` in the labkit path). |
330
385
  | `gitlab_labkit_rate_limiter_limit` | gauge | `rate_limiter`, `rule` | Resolved limit at the last check (useful when `limit:` is callable). |
331
386
  | `gitlab_labkit_rate_limiter_period_seconds` | gauge | `rate_limiter`, `rule` | Resolved period at the last check. |
332
387
 
333
- Because `:log` rules do not terminate, a single `check` call can emit
334
- **multiple** `calls_total` increments: one per `:log` rule that matched, plus
335
- one for the terminating decision (or `rule="unmatched"` if no terminating
336
- rule fired).
388
+ Because every matching rule is counted, a single `check` call can emit
389
+ **multiple** `calls_total` increments: one per counted rule (or a single
390
+ `rule="unmatched"` increment if nothing matched).
337
391
 
338
392
  ## Dev/test vs production guards
339
393
 
@@ -74,7 +74,7 @@ module Labkit
74
74
  # timeout, OOM) not only Redis protocol errors.
75
75
  report_error_metrics
76
76
  log_error(e, identifier)
77
- Result.new(matched: false, error: true, action: :allow)
77
+ Result.error
78
78
  end
79
79
 
80
80
  # Read-without-increment counterpart to {#check}. Same matching and Result
@@ -85,30 +85,48 @@ module Labkit
85
85
  rescue StandardError => e
86
86
  report_error_metrics
87
87
  log_error(e, identifier)
88
- Result.new(matched: false, error: true, action: :allow)
88
+ Result.error
89
89
  end
90
90
 
91
91
  private
92
92
 
93
- # :log rules are non-terminating: they emit metrics and continue,
94
- # so a shadow :log rule cannot disable a following :block rule.
93
+ # Every rule that matches is evaluated and counted; matching does not
94
+ # short-circuit the loop. Exactly two cases terminate it early:
95
95
  #
96
- # :skip rules terminate on match without touching Redis: no counter is
97
- # incremented, so the branch sits before the count_distinct check
98
- # (identifier completeness is irrelevant to a rule that builds no key).
99
- # calls_total still increments so the bypass stays observable.
96
+ # - :skip terminates on match without touching Redis. No counter is
97
+ # incremented, so the branch sits before the count_distinct check
98
+ # (identifier completeness is irrelevant to a rule that builds no key).
99
+ # calls_total still increments so the bypass stays observable.
100
+ # - a :limit rule over its limit terminates, because the request is
101
+ # rejected and later rules cannot change that. Rules declared after it
102
+ # are neither counted nor evaluated, so a blocked request debits every
103
+ # rule up to and including the one that blocked, and none after it.
104
+ #
105
+ # Every other matched rule - :log, and :limit while under its limit - is
106
+ # counted and collected into the returned Result, which reports the
107
+ # most-constraining evaluation (ranking lives in Result::Evaluation#<=>).
108
+ # cost is therefore debited from every matching rule, not just the first.
100
109
  #
101
110
  # SET-mode rules (rule.count_distinct set) that match but whose identifier
102
111
  # is missing the count_distinct key fail open + log + bump errors_total, and
103
112
  # the loop continues to the next rule (the rule is treated as not applicable
104
113
  # rather than aborting the whole evaluation).
114
+ #
115
+ # Error handling stays whole-check (see #check): a raise part-way through
116
+ # discards the results of the rules already evaluated, even though their
117
+ # counters were incremented. No blocking verdict is lost that way - a
118
+ # :limit rule over its limit returns before any later rule can raise -
119
+ # but a rule declared after the failing one loses its chance to block.
120
+ # The request is counted and allowed.
105
121
  def check_rules(identifier, cost, rule_context)
122
+ result = Result.new
123
+
106
124
  @rules.each do |rule|
107
125
  next unless rule_matches?(rule, identifier)
108
126
 
109
127
  if rule.action == :skip
110
128
  report_skipped_metrics(rule)
111
- return skip_result(rule)
129
+ return result.skip!(rule)
112
130
  end
113
131
 
114
132
  if rule.count_distinct && missing_count_distinct_value?(rule, identifier)
@@ -117,44 +135,42 @@ module Labkit
117
135
  next
118
136
  end
119
137
 
120
- result = evaluate_rule(rule, identifier, cost, rule_context)
121
- report_matched_metrics(result)
122
- return result unless rule.action == :log
138
+ evaluation = evaluate_rule(rule, identifier, cost, rule_context)
139
+ result.add_evaluation(evaluation)
140
+ report_matched_metrics(evaluation)
141
+ return result if result.block?
123
142
  end
124
143
 
125
- report_unmatched_metrics
126
- Result.new(matched: false, action: :allow)
144
+ report_unmatched_metrics unless result.matched?
145
+ result
127
146
  end
128
147
 
129
- # Mirror of check_rules without metrics: peek skips :log rules (their state
130
- # is unobservable through peek).
148
+ # Mirror of check_rules without metrics or writes. :log rules are read
149
+ # here too: check can report a :log rule's evaluation, so excluding them
150
+ # would make peek answer a different question than check.
131
151
  #
132
152
  # peek does not need the count_distinct identifier key - it reads SCARD on
133
153
  # the rule-keyed compound key, which contains the cardinality across all
134
154
  # members. So missing-key fail-open does not apply here.
135
155
  def peek_rules(identifier, rule_context)
156
+ result = Result.new
157
+
136
158
  @rules.each do |rule|
137
- next if rule.action == :log
138
159
  next unless rule_matches?(rule, identifier)
139
160
 
140
- return skip_result(rule) if rule.action == :skip
161
+ return result.skip!(rule) if rule.action == :skip
141
162
 
142
- return peek_rule(rule, identifier, rule_context)
163
+ result.add_evaluation(peek_rule(rule, identifier, rule_context))
164
+ return result if result.block?
143
165
  end
144
166
 
145
- Result.new(matched: false, action: :allow)
167
+ result
146
168
  end
147
169
 
148
170
  def rule_matches?(rule, identifier)
149
171
  rule.match.all? { |key, matcher| matcher.match?(identifier[key]) }
150
172
  end
151
173
 
152
- # A matched :skip rule allows without evaluating: no counter exists, so
153
- # info is nil (to_response_headers is {} for skip results).
154
- def skip_result(rule)
155
- Result.new(matched: true, action: :allow, rule: rule)
156
- end
157
-
158
174
  def missing_count_distinct_value?(rule, identifier)
159
175
  value = identifier[rule.count_distinct]
160
176
  value.nil? || value.to_s.empty?
@@ -174,7 +190,7 @@ module Labkit
174
190
  incr_with_ttl(redis_key, resolved_period, cost)
175
191
  end
176
192
 
177
- build_result(rule, resolved_limit, resolved_period, count, ttl)
193
+ build_evaluation(rule, resolved_limit, resolved_period, count, ttl)
178
194
  end
179
195
 
180
196
  def peek_rule(rule, identifier, rule_context)
@@ -183,12 +199,10 @@ module Labkit
183
199
  resolved_period = Integer(resolve_value(rule.period, rule_context))
184
200
 
185
201
  count, ttl = rule.count_distinct ? scard_with_ttl(redis_key) : read_with_ttl(redis_key)
186
- build_result(rule, resolved_limit, resolved_period, count, ttl)
202
+ build_evaluation(rule, resolved_limit, resolved_period, count, ttl)
187
203
  end
188
204
 
189
- def build_result(rule, resolved_limit, resolved_period, count, ttl)
190
- exceeded = count > resolved_limit
191
- action = exceeded ? rule.action : :allow
205
+ def build_evaluation(rule, resolved_limit, resolved_period, count, ttl)
192
206
  info = Result::Info.new(
193
207
  resolved_limit: resolved_limit, resolved_period: resolved_period,
194
208
  count: count,
@@ -196,7 +210,7 @@ module Labkit
196
210
  reset_at: Time.now.utc + (ttl >= 0 ? ttl : resolved_period)
197
211
  )
198
212
 
199
- Result.new(matched: true, exceeded: exceeded, action: action, rule: rule, info: info)
213
+ Result::Evaluation.new(rule: rule, exceeded: count > resolved_limit, info: info)
200
214
  end
201
215
 
202
216
  def build_redis_key(rule, identifier)
@@ -264,7 +278,7 @@ module Labkit
264
278
 
265
279
  # Pipelined GET + TTL. No EXPIRE: peek must not extend the window.
266
280
  # A missing key (GET => nil, TTL => -2) is reported as count=0; the
267
- # build_result fallback then derives reset_at from the rule period
281
+ # build_evaluation fallback then derives reset_at from the rule period
268
282
  # since there is no Redis-side window to read.
269
283
  #
270
284
  # Float parsing accepts both INCR-stored ("5") and INCRBYFLOAT-stored
@@ -325,19 +339,19 @@ module Labkit
325
339
  )
326
340
  end
327
341
 
328
- def report_matched_metrics(result)
342
+ def report_matched_metrics(evaluation)
329
343
  Metrics.calls_total.increment(
330
344
  rate_limiter: @name,
331
- rule: result.rule.name,
332
- action: result.action.to_s
345
+ rule: evaluation.rule.name,
346
+ action: (evaluation.exceeded? ? evaluation.rule.action : :allow).to_s
333
347
  )
334
348
  Metrics.limit_gauge.set(
335
- { rate_limiter: @name, rule: result.rule.name },
336
- result.info.resolved_limit
349
+ { rate_limiter: @name, rule: evaluation.rule.name },
350
+ evaluation.info.resolved_limit
337
351
  )
338
352
  Metrics.period_gauge.set(
339
- { rate_limiter: @name, rule: result.rule.name },
340
- result.info.resolved_period
353
+ { rate_limiter: @name, rule: evaluation.rule.name },
354
+ evaluation.info.resolved_period
341
355
  )
342
356
  end
343
357
 
@@ -15,7 +15,7 @@ module Labkit
15
15
  # rules: [Labkit::RateLimit::Rule.new(name: "api_user", limit: 100, period: 60, characteristics: [:user])]
16
16
  # )
17
17
  # result = limiter.check({ user: 42, ip: "1.2.3.4" })
18
- # render_429 if result.exceeded? && result.action == :block
18
+ # render_429 if result.action == :block
19
19
  class Limiter
20
20
  NAME_PATTERN = /\A[a-z0-9_]+\z/
21
21
 
@@ -5,9 +5,10 @@ module Labkit
5
5
  module Metrics
6
6
  module_function
7
7
 
8
- # :log rules are non-terminating: a check that matched only :log rules
9
- # increments calls_total once per matched :log rule AND once with
10
- # rule="unmatched", action="allow", since no terminating decision was made.
8
+ # Emitted once per *matched rule*, not once per check: every rule that
9
+ # matches is evaluated, so summing this by rate_limiter counts rule
10
+ # evaluations rather than requests. rule="unmatched", action="allow" is
11
+ # emitted only when no rule matched at all.
11
12
  def calls_total
12
13
  Labkit::Metrics::Client.counter(
13
14
  :gitlab_labkit_rate_limiter_calls_total,
@@ -3,34 +3,100 @@
3
3
  module Labkit
4
4
  module RateLimit
5
5
  # Result is the return value of Limiter#check.
6
- # matched? - true if a rule's match conditions were satisfied
7
- # exceeded? - true if the matched rule's counter exceeded its limit
8
- # action - the outcome: what the caller should do
9
- # :block = rule matched, exceeded, rule configured to block
10
- # :log = rule matched, exceeded, rule configured to log only
11
- # :allow = rule matched but count within limit, rule configured
12
- # to skip (bypass, nothing counted), no rule matched,
13
- # or error (fail-open)
14
- # The rule's configured action is available via rule.action.
15
- # rule - the matched Rule object (nil when matched? is false)
16
- # error? - true if Redis was unavailable; result fails open (exceeded? is false)
17
- # info - Result::Info with per-window counters; nil when matched? is false,
18
- # error?, or the matched rule is :skip (no counter exists)
19
- Result = Data.define(:matched, :exceeded, :action, :rule, :error, :info) do
20
- def initialize(matched:, action: nil, exceeded: false, rule: nil, error: false, info: nil)
21
- super
6
+ #
7
+ # It accumulates one Result::Evaluation per matched-and-counted rule; the
8
+ # reader methods report the single most-constraining evaluation (see
9
+ # #most_constraining). A matched :skip rule or a fail-open error replaces
10
+ # the evaluations as the source of the reported outcome.
11
+ #
12
+ # Result is a mutable accumulator, not a value object: two Results built
13
+ # from the same evaluations are not #==.
14
+ #
15
+ # matched? - true if at least one rule's match conditions were satisfied
16
+ # exceeded? - true if the reported evaluation's counter exceeded its limit
17
+ # action - the outcome: what the caller should do
18
+ # :block = a :limit rule matched and is over its limit
19
+ # :allow = everything else, including an exceeded :log rule
20
+ # (visible via exceeded?), a matched :skip rule, no rule
21
+ # matched, and error (fail-open).
22
+ # The rule's configured action is available via rule.action.
23
+ # rule - the reported Rule: the :skip rule when one matched, otherwise
24
+ # the most-constraining evaluated Rule (nil when matched? is
25
+ # false). Other rules may also have matched and been counted;
26
+ # see #evaluations.
27
+ # error? - true if Redis was unavailable; result fails open (exceeded? is false)
28
+ # info - Result::Info with per-window counters for the reported rule;
29
+ # nil when matched? is false, error?, or the matched rule is
30
+ # :skip (no counter exists)
31
+ # evaluations - every counted evaluation, in rule declaration order
32
+ class Result
33
+ attr_reader :evaluations
34
+
35
+ def self.error
36
+ new(error: true)
37
+ end
38
+
39
+ def initialize(error: false)
40
+ @evaluations = []
41
+ @skip_rule = nil
42
+ @error = error
43
+ @most_constraining = nil
44
+ end
45
+
46
+ def add_evaluation(evaluation)
47
+ @most_constraining = nil
48
+ @evaluations << evaluation
49
+ self
50
+ end
51
+
52
+ # Records a matched :skip rule: the request is allowed and the skip rule
53
+ # is the one reported, regardless of any evaluations already collected.
54
+ def skip!(rule)
55
+ @skip_rule = rule
56
+ self
22
57
  end
23
58
 
24
59
  def matched?
25
- matched
60
+ skipped? || @evaluations.any?
61
+ end
62
+
63
+ def skipped?
64
+ !!@skip_rule
65
+ end
66
+
67
+ def block?
68
+ !skipped? && @evaluations.any?(&:block?)
26
69
  end
27
70
 
28
71
  def exceeded?
29
- exceeded
72
+ return false if skipped?
73
+
74
+ !most_constraining.nil? && most_constraining.exceeded?
30
75
  end
31
76
 
32
77
  def error?
33
- error
78
+ @error
79
+ end
80
+
81
+ def action
82
+ block? ? :block : :allow
83
+ end
84
+
85
+ def rule
86
+ @skip_rule || most_constraining&.rule
87
+ end
88
+
89
+ def info
90
+ return nil if skipped?
91
+
92
+ most_constraining&.info
93
+ end
94
+
95
+ # The evaluation with the strongest claim on the outcome; ties keep the
96
+ # earliest-declared rule (min returns the first of tied elements).
97
+ # Memoized; add_evaluation invalidates.
98
+ def most_constraining
99
+ @most_constraining ||= @evaluations.min
34
100
  end
35
101
 
36
102
  # Returns RFC-compliant rate limit response headers, or {} when no rule matched or an error occurred.
@@ -48,6 +114,39 @@ module Labkit
48
114
  end
49
115
  end
50
116
 
117
+ # The outcome of counting one matched rule.
118
+ # rule - the evaluated Rule
119
+ # exceeded - whether the post-increment count exceeded the resolved limit
120
+ # info - Result::Info with the per-window counters
121
+ #
122
+ # Evaluations order by constraint: a blocking evaluation (:limit rule over
123
+ # its limit) ranks strictly first - it decides the request no matter what
124
+ # any other rule reports - then exceeded ones, then fewest remaining.
125
+ # remaining floors at 0, so an exceeded :log rule ties with one sitting
126
+ # exactly on its limit; ranking exceeded ahead keeps a breach from being
127
+ # hidden by a rule that merely reached its limit.
128
+ Result::Evaluation = Data.define(:rule, :exceeded, :info) do
129
+ include Comparable
130
+
131
+ def exceeded?
132
+ exceeded
133
+ end
134
+
135
+ def block?
136
+ exceeded? && rule.action == :limit
137
+ end
138
+
139
+ def <=>(other)
140
+ constraint_rank <=> other.constraint_rank
141
+ end
142
+
143
+ protected
144
+
145
+ def constraint_rank
146
+ [block? ? 0 : 1, exceeded? ? 0 : 1, info.remaining]
147
+ end
148
+ end
149
+
51
150
  # Per-window counter data attached to a matched Result.
52
151
  # resolved_limit - the evaluated limit Integer for this rule
53
152
  # resolved_period - the evaluated period Integer (seconds) for this rule
@@ -2,7 +2,7 @@
2
2
 
3
3
  module Labkit
4
4
  module RateLimit
5
- KNOWN_ACTIONS = %i[block log skip].freeze
5
+ KNOWN_ACTIONS = %i[limit log skip].freeze
6
6
  RULE_NAME_PATTERN = /\A[a-z0-9_]+\z/
7
7
  RULE_NAME_MAX_LENGTH = 64
8
8
 
@@ -12,11 +12,12 @@ module Labkit
12
12
  # the rule to apply; empty hash matches any identifier
13
13
  # limit - request threshold; may be a callable (resolved per check)
14
14
  # period - window in seconds; may be a callable (resolved per check)
15
- # action - :block (enforce), :log (count and log only, do not block,
16
- # evaluation continues to subsequent rules), or :skip
17
- # (bypass: permit and terminate evaluation on match without
18
- # counting; performs no Redis operation, so limit, period,
19
- # characteristics, and count_distinct are inert)
15
+ # action - :limit (enforce; the result blocks when the rule is over
16
+ # its limit, which also terminates evaluation), :log (count
17
+ # and log only, never blocks and never terminates), or
18
+ # :skip (bypass: permit and terminate evaluation on match
19
+ # without counting; performs no Redis operation, so limit,
20
+ # period, characteristics, and count_distinct are inert)
20
21
  # characteristics - identifier keys used to build the compound Redis counter key
21
22
  # count_distinct - optional Symbol naming an identifier key. When set, the rule
22
23
  # counts the number of distinct values seen for that key within
@@ -43,7 +44,7 @@ module Labkit
43
44
  sym
44
45
  end
45
46
 
46
- def initialize(name:, limit:, period:, characteristics:, match: {}, action: :block, count_distinct: nil)
47
+ def initialize(name:, limit:, period:, characteristics:, match: {}, action: :limit, count_distinct: nil)
47
48
  raise ArgumentError, "name must be a String or Symbol, got #{name.class}" unless name.is_a?(String) || name.is_a?(Symbol)
48
49
 
49
50
  name_str = name.to_s
@@ -40,7 +40,8 @@ module Labkit
40
40
  #
41
41
  # @param name [String] call site name
42
42
  # @param identifier [Identifier, Hash] caller attributes
43
- # @param rules [Array<Rule>] ordered list of rules (first match wins)
43
+ # @param rules [Array<Rule>] ordered list of rules; every matching rule is
44
+ # counted, and the Result reports the most constraining one
44
45
  # @param redis [Object, nil] Redis client; falls back to config.redis
45
46
  # @param logger [Logger, nil] logger; falls back to config.logger
46
47
  # @param cost [Numeric] amount to add to the counter; see Limiter#check
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: gitlab-labkit
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.0.1
4
+ version: 4.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew Newdigate