gitlab-labkit 5.1.2 → 5.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitlab/CODEOWNERS +1 -1
- data/.gitlab-ci.yml +7 -0
- data/AGENTS.md +136 -0
- data/CODEOWNERS +1 -1
- data/gitlab-labkit.gemspec +1 -0
- data/lib/labkit/fields.rb +15 -2
- data/lib/labkit/rate_limit/README.md +82 -7
- data/lib/labkit/rate_limit/matcher.rb +138 -28
- data/lib/labkit/rate_limit/result.rb +32 -9
- data/lib/labkit/rate_limit/rule.rb +14 -4
- metadata +16 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: fb00f476a5fed0be71839727d4fb5d8230f773831c8c8cca40ff8090c76cb2c5
|
|
4
|
+
data.tar.gz: c382bf0eed9e55b283eb90c3956a00c8dbf73ad5819fb22e8dd6a1be931fd9ff
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5932210330aa7c8c4413de43f8ae710508658846c89f1b7b74e1d8d36af564b61034988a6248bf9795d6343602d4b1579d6a9aff48795cda1163c545377932b9
|
|
7
|
+
data.tar.gz: ddc501d350806e384f122f152f6ceb7066487bc4f41078341f1ae13373631adf52a1f5cd427a69e879482cf3dc1dff75bbfe7cd57dd1237094c595e615acc591
|
data/.gitlab/CODEOWNERS
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
* @reprazent @
|
|
1
|
+
* @reprazent @mkaeppler @ayufan @hmerscher @sankalp_gl @hardikgala @nindurkar @ashs2
|
data/.gitlab-ci.yml
CHANGED
|
@@ -79,6 +79,13 @@ conformance-spec:
|
|
|
79
79
|
variables:
|
|
80
80
|
LABKIT_RUBY_SHA: $CI_COMMIT_SHA
|
|
81
81
|
rules:
|
|
82
|
+
# Renovate's project-token bot cannot create pipelines in labkit-spec
|
|
83
|
+
# (GitLab forbids project bots from joining other projects), so its MRs
|
|
84
|
+
# would hard-fail the bridge. Attempt the trigger but do not block; a
|
|
85
|
+
# human re-running the pipeline gates the MR properly (it runs as them).
|
|
86
|
+
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $GITLAB_USER_LOGIN =~ /^project_\d+_bot/'
|
|
87
|
+
changes: !reference [.conformance-trigger-paths, changes]
|
|
88
|
+
allow_failure: true
|
|
82
89
|
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
|
83
90
|
changes: !reference [.conformance-trigger-paths, changes]
|
|
84
91
|
trigger:
|
data/AGENTS.md
ADDED
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# AGENTS.md
|
|
2
|
+
|
|
3
|
+
Instructions for AI agents and automated tooling working in this repository.
|
|
4
|
+
|
|
5
|
+
## Project context
|
|
6
|
+
|
|
7
|
+
LabKit-Ruby (`gitlab-labkit` on RubyGems) is GitLab's Ruby SDK for cross-project infrastructure concerns: request context and correlation, structured logging, tracing, metrics, SLIs, and rate limiting.
|
|
8
|
+
It is consumed by GitLab Rails and other Ruby services, so every public API is a compatibility commitment.
|
|
9
|
+
|
|
10
|
+
It is one of several LabKit SDKs.
|
|
11
|
+
The others are Go ([labkit](https://gitlab.com/gitlab-org/labkit)), Rust ([labkit-rs](https://gitlab.com/gitlab-org/rust/labkit-rs)), and Python ([labkit-python](https://gitlab.com/gitlab-org/labkit-python)).
|
|
12
|
+
Behaviour that services observe, and anything written to a shared store, must be the same in every SDK.
|
|
13
|
+
The contract lives in [labkit-spec](https://gitlab.com/gitlab-org/quality/tooling/labkit-spec), and its conformance harness is the judge.
|
|
14
|
+
|
|
15
|
+
### Layout
|
|
16
|
+
|
|
17
|
+
One gem, one `Labkit` namespace, one directory per capability under `lib/labkit/`.
|
|
18
|
+
`lib/gitlab-labkit.rb` autoloads each capability unless the table says otherwise, so requiring the gem stays cheap and callers load only what they touch.
|
|
19
|
+
|
|
20
|
+
| Module | Directory | Notes |
|
|
21
|
+
| --- | --- | --- |
|
|
22
|
+
| `Labkit::Context`, `Labkit::Correlation` | `lib/labkit/context.rb`, `lib/labkit/correlation/` | Request context for logs and propagation |
|
|
23
|
+
| `Labkit::Logging` | `lib/labkit/logging/` | Log sanitisation and JSON formatting |
|
|
24
|
+
| `Labkit::Tracing` | `lib/labkit/tracing/` | OpenTelemetry and legacy OpenTracing |
|
|
25
|
+
| `Labkit::Metrics` | `lib/labkit/metrics/` | Prometheus client wrappers |
|
|
26
|
+
| `Labkit::ApplicationSli`, `Labkit::UserExperienceSli` | `lib/labkit/application_sli/`, `lib/labkit/user_experience_sli/` | Apdex and error-rate SLIs |
|
|
27
|
+
| `Labkit::RateLimit` | `lib/labkit/rate_limit/` | Rules-based, Redis-backed rate limiting, see below |
|
|
28
|
+
| `Labkit::Middleware` | `lib/labkit/middleware/` | Rack and Sidekiq middleware for context, tracing, and SLIs |
|
|
29
|
+
| `Labkit::JsonSchema` | `lib/labkit/json_schema/` | `RefResolver` for remote `$ref` in JSON schemas. Not autoloaded; required by `UserExperienceSli` |
|
|
30
|
+
| `Labkit::FIPS` | `lib/labkit/fips.rb` | FIPS mode detection and OpenSSL digest substitution |
|
|
31
|
+
| `Labkit::System` | `lib/labkit/system.rb` | Monotonic clock helper used by metrics, tracing, and logging |
|
|
32
|
+
| `Labkit::Redis` | `lib/labkit/redis/` | Redis helpers shared by capabilities |
|
|
33
|
+
| `Labkit::Fields` | `lib/labkit/fields.rb` | Generated by labkit-spec CI from `schema/fields.yaml`. Change the field there, not here |
|
|
34
|
+
| `Labkit::RSpec` | `lib/labkit/rspec/` | Matchers for consumers' test suites. Not autoloaded; `require "labkit/rspec/matchers"` |
|
|
35
|
+
|
|
36
|
+
`lib/labkit/*_publisher.rb` hook Excon, HTTPClient, and Net::HTTP to publish outbound request notifications.
|
|
37
|
+
|
|
38
|
+
Modules with their own README (`lib/labkit/<module>/README.md`) document their public API there.
|
|
39
|
+
Update that README in the same MR as a behaviour change.
|
|
40
|
+
|
|
41
|
+
## Commands
|
|
42
|
+
|
|
43
|
+
```shell
|
|
44
|
+
bundle install
|
|
45
|
+
bundle exec rake verify # rspec + rubocop, what CI runs
|
|
46
|
+
bundle exec rake fix # rubocop autocorrect
|
|
47
|
+
bundle exec rspec spec/labkit/<module> # one module's specs
|
|
48
|
+
bundle exec rake console # IRB with the gem loaded
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Specs that need Redis use the instance at `LABKIT_TEST_REDIS_URL`, default `redis://localhost:6390/0`.
|
|
52
|
+
If nothing answers there and `LABKIT_TEST_REDIS_URL` is unset, the suite starts one with `docker compose up -d redis` and stops it at exit.
|
|
53
|
+
A Redis already listening on that URL is used as is and left running.
|
|
54
|
+
Autostart is off when `LABKIT_TEST_REDIS_URL` is set, even to the default value, when `CI` is set, or when `LABKIT_TEST_REDIS_NO_AUTOSTART=1` is set.
|
|
55
|
+
In those cases, start Redis yourself.
|
|
56
|
+
|
|
57
|
+
Style is `gitlab-styles` via RuboCop with `NewCops: enable`.
|
|
58
|
+
Do not edit `.rubocop_todo.yml` to silence a new offence; fix the code.
|
|
59
|
+
Tool versions come from `.tool-versions` through mise.
|
|
60
|
+
|
|
61
|
+
## CI
|
|
62
|
+
|
|
63
|
+
- `rspec` runs the suite with a Redis service.
|
|
64
|
+
- `ruby-versions` and `legacy-ruby-versions` repeat it on Ruby 3.4 and 3.3. The versions are set in `.gitlab-ci.yml`, not read from the gemspec.
|
|
65
|
+
- `conformance-spec` triggers a downstream pipeline in labkit-spec that checks out this MR's commit and runs the conformance harness against it.
|
|
66
|
+
It runs when `lib/`, the gemspec, the Gemfile, or `.gitlab-ci.yml` change, and a red downstream run blocks the merge.
|
|
67
|
+
Renovate's bot user cannot start pipelines in labkit-spec, so on its MRs the job has `allow_failure: true` and a human rerun is the gate.
|
|
68
|
+
- Danger runs the shared `gitlab-dangerfiles` checks except changelog and commit message, which semantic-release owns.
|
|
69
|
+
|
|
70
|
+
## Commits, MRs, and releases
|
|
71
|
+
|
|
72
|
+
Conventional Commits: `feat`, `fix`, `perf`, `revert`, `docs`, `refactor`, `test`, `chore`, `ci`, `style`.
|
|
73
|
+
Scope is the module, for example `fix(rate_limit): ...`.
|
|
74
|
+
|
|
75
|
+
Releases are cut from `master` and the `N-N-stable` branches by semantic-release with the `conventionalcommits` preset, so the commit type decides the version bump.
|
|
76
|
+
|
|
77
|
+
| Type | Release |
|
|
78
|
+
| --- | --- |
|
|
79
|
+
| `feat!`, or any type with a `BREAKING CHANGE:` footer | major |
|
|
80
|
+
| `feat` | minor |
|
|
81
|
+
| `fix`, `perf`, `revert` | patch |
|
|
82
|
+
| `docs`, `refactor`, `test`, `chore`, `ci`, `style` | none |
|
|
83
|
+
|
|
84
|
+
There is no hand-edited changelog.
|
|
85
|
+
|
|
86
|
+
Public APIs stay backward compatible.
|
|
87
|
+
Add keyword arguments with defaults; do not add required parameters or rename public methods.
|
|
88
|
+
Keep an MR to one concern, and separate mechanical changes such as renames from behavioural ones.
|
|
89
|
+
|
|
90
|
+
## Rate limiting
|
|
91
|
+
|
|
92
|
+
`Labkit::RateLimit` is documented in `lib/labkit/rate_limit/README.md`.
|
|
93
|
+
It is the one capability that shares state across SDKs: services in any language enforce the same limits against the same Redis.
|
|
94
|
+
That makes the Redis key shape and the Lua scripts as much a contract as the HTTP headers, so changes here follow stricter rules than the rest of the gem.
|
|
95
|
+
|
|
96
|
+
### Where each SDK keeps it
|
|
97
|
+
|
|
98
|
+
| SDK | Repository | Module | Tests | Status |
|
|
99
|
+
| --- | --- | --- | --- | --- |
|
|
100
|
+
| Ruby (this repository) | gitlab-org/ruby/gems/labkit-ruby | `lib/labkit/rate_limit/` | `spec/labkit/rate_limit/` | released |
|
|
101
|
+
| Go | gitlab-org/labkit | `v2/ratelimit/` | `v2/ratelimit/*_test.go`, run with `go -C v2 test ./ratelimit/...` | released |
|
|
102
|
+
| Rust | gitlab-org/rust/labkit-rs | in progress | | not released, skip |
|
|
103
|
+
| Python | gitlab-org/labkit-python | in progress | | not released, skip |
|
|
104
|
+
| Spec | gitlab-org/quality/tooling/labkit-spec | `conformance/README.md` ("v1 decisions"), `conformance/ratelimit_scenarios/*.json`, `conformance/aspect_ratelimit.go` | `conformance/` | judge |
|
|
105
|
+
|
|
106
|
+
### The three layers
|
|
107
|
+
|
|
108
|
+
Every SDK must agree on three layers.
|
|
109
|
+
This section defines the layers; the current values (key grammar, header names, counting rules, metric names, log fields) live in labkit-spec, in the "v1 decisions" list in `conformance/README.md` and the scenarios under `conformance/ratelimit_scenarios/`.
|
|
110
|
+
If this file and labkit-spec disagree, labkit-spec wins.
|
|
111
|
+
|
|
112
|
+
1. **Storage contract.**
|
|
113
|
+
Anything written to or read from Redis: key shape and prefix, how values are encoded, TTL rules, and the Lua scripts.
|
|
114
|
+
Two SDKs sharing one Redis with different storage rules keep two counters for one limit.
|
|
115
|
+
Never change this layer in one SDK.
|
|
116
|
+
It changes in labkit-spec first, then every SDK follows.
|
|
117
|
+
2. **Observable contract.**
|
|
118
|
+
Anything a caller or an operator can see: response status and headers, the counting rule that decides a block, log line messages and field sets, metric families and labels.
|
|
119
|
+
Also changes in labkit-spec first.
|
|
120
|
+
3. **Behaviour within the contract.**
|
|
121
|
+
Input validation and edge cases the contract already implies, where one SDK does something the contract does not allow.
|
|
122
|
+
These may be fixed SDK by SDK, and every SDK should end up the same.
|
|
123
|
+
|
|
124
|
+
A change that mixes layers is governed by the strictest one: storage, then observable, then behaviour.
|
|
125
|
+
When a change does not fit one layer clearly, say so in the MR instead of picking one.
|
|
126
|
+
|
|
127
|
+
### Rules for rate-limit changes
|
|
128
|
+
|
|
129
|
+
- A change to layer 1 or 2 opens a labkit-spec MR first and waits for it to merge.
|
|
130
|
+
- A change to layer 3 should be checked against the Go SDK.
|
|
131
|
+
If Go has the same gap, say so in the MR so it can be ported.
|
|
132
|
+
- When a review finds a divergence between SDKs, a conformance scenario in labkit-spec must follow in the same week, so the harness sees the gap next time.
|
|
133
|
+
The author of the MR where the divergence was found owns this.
|
|
134
|
+
Mention the sync agent on the review thread to have it draft the scenario MR; if it cannot, open a labkit-spec issue and link it from the thread.
|
|
135
|
+
- Never edit Lua scripts, key construction, header names, metric names, or log field names to fix a Ruby-only bug.
|
|
136
|
+
If a fix seems to need that, it is a contract change.
|
data/CODEOWNERS
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
# CODEOWNERS is used to lookup assignees for
|
|
2
2
|
# Renovate Bot dependency change Merge Requests.
|
|
3
3
|
# https://docs.renovatebot.com/configuration-options/#assigneesfromcodeowners
|
|
4
|
-
* @reprazent @
|
|
4
|
+
* @reprazent @ayufan @hmerscher @e_forbes @M_Alvarez @mwoolf @sankalp_gl @hardikgala @nindurkar @ashs2
|
data/gitlab-labkit.gemspec
CHANGED
|
@@ -36,6 +36,7 @@ Gem::Specification.new do |spec|
|
|
|
36
36
|
spec.add_runtime_dependency "pg_query", ">= 6.1.0", "< 7.0"
|
|
37
37
|
spec.add_runtime_dependency "prometheus-client-mmap", ">= 1.2", "< 2.0"
|
|
38
38
|
spec.add_runtime_dependency "redis", "> 3.0.0", "< 6.0.0"
|
|
39
|
+
spec.add_runtime_dependency "thrift", "< 0.25" # Pinned for jaeger-client 1.1.0 compatibility; thrift 0.25 requires transports to implement message_boundaries?
|
|
39
40
|
|
|
40
41
|
# Please maintain alphabetical order for dev dependencies
|
|
41
42
|
spec.add_development_dependency "excon", "~> 0.78.1"
|
data/lib/labkit/fields.rb
CHANGED
|
@@ -138,6 +138,17 @@ module Labkit
|
|
|
138
138
|
# (e.g. newlines) to prevent log injection.
|
|
139
139
|
ADDITIONAL_DETAILS = "additional_details"
|
|
140
140
|
|
|
141
|
+
# Unrounded complexity score of a GraphQL query. Some fields have a
|
|
142
|
+
# fractional cost (e.g. graphql-ruby's __typename costs 0.2), so the total
|
|
143
|
+
# can be a non-integer value.
|
|
144
|
+
GRAPHQL_COMPLEXITY_RAW = "graphql_complexity_raw"
|
|
145
|
+
|
|
146
|
+
# Number of nodes a GraphQL query returned across all its connection fields.
|
|
147
|
+
# The count is taken after authorization removes the nodes the user can't
|
|
148
|
+
# see. Nested connections add up. The value is 0 when the query selects no
|
|
149
|
+
# connection field.
|
|
150
|
+
GRAPHQL_CONNECTION_NODES = "graphql_connection_nodes"
|
|
151
|
+
|
|
141
152
|
# Maps each field constant to its logging field variant version.
|
|
142
153
|
# A version of 0 means the field is not yet assigned to any variant.
|
|
143
154
|
VARIANT_VERSION = {
|
|
@@ -179,6 +190,8 @@ module Labkit
|
|
|
179
190
|
ROOT_CALLER_ID => 0,
|
|
180
191
|
GL_SENT_NOTIFICATION_ID => 0,
|
|
181
192
|
ADDITIONAL_DETAILS => 0,
|
|
193
|
+
GRAPHQL_COMPLEXITY_RAW => 0,
|
|
194
|
+
GRAPHQL_CONNECTION_NODES => 0,
|
|
182
195
|
}.freeze
|
|
183
196
|
|
|
184
197
|
# Get the constant name for a field value
|
|
@@ -207,8 +220,8 @@ module Labkit
|
|
|
207
220
|
Fields::DURATION_S => %w[duration duration_ms elapsed_time actual_duration time_ms total_time gitaly.duration],
|
|
208
221
|
Fields::REMOTE_IP => %w[ip source_ip ip_address],
|
|
209
222
|
Fields::HTTP_HOST => %w[hostname request_host gitlab_host kubernetes.host],
|
|
210
|
-
Fields::GL_PROJECT_ID => %w[extra.project_id meta.project_id meta.search.project_id job_project_id target_project_id],
|
|
211
|
-
Fields::GL_PIPELINE_ID => %w[extra.pipeline_id meta.pipeline_id root_pipeline_id],
|
|
223
|
+
Fields::GL_PROJECT_ID => %w[project_id extra.project_id meta.project_id meta.search.project_id job_project_id target_project_id],
|
|
224
|
+
Fields::GL_PIPELINE_ID => %w[pipeline_id extra.pipeline_id meta.pipeline_id root_pipeline_id],
|
|
212
225
|
Fields::TIMESTAMP => %w[start_time],
|
|
213
226
|
Fields::SEVERITY => %w[level],
|
|
214
227
|
Fields::LOG_MESSAGE => %w[msg custom_message extra.message fields.message graphql.message reason color_message exception.gitaly],
|
|
@@ -171,8 +171,8 @@ A `Rule` is a `Data.define` value object with the following fields:
|
|
|
171
171
|
|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
172
172
|
| `name` | Stable identifier used in Redis keys and metric labels. Must match `/\A[a-z0-9_]+\z/`, max 64 chars. Renaming a rule abandons its in-flight counters. |
|
|
173
173
|
| `match` | Hash of identifier key/value predicates that must **all** be satisfied for the rule to apply. Empty hash matches anything. See [Matchers](#matchers). |
|
|
174
|
-
| `limit` | Integer request threshold per `period
|
|
175
|
-
| `period` | Window length in seconds. May be a callable resolved on every check.
|
|
174
|
+
| `limit` | Integer request threshold per `period`, at least 1. May be a callable resolved on every check. |
|
|
175
|
+
| `period` | Window length in seconds, at least 1; a fraction truncates to whole seconds. May be a callable resolved on every check. |
|
|
176
176
|
| `action` | What the rule does when it matches. One of `:limit`, `:log`, `:skip`. Default `:limit`. See [Actions](#actions). |
|
|
177
177
|
| `characteristics` | Array of identifier keys whose values are folded into the Redis counter key. Each unique combination gets its own counter. |
|
|
178
178
|
| `count_distinct` | Optional identifier key. When set, the rule counts distinct values seen for that key within the window, backed by a Redis SET rather than a counter. Must not overlap `characteristics`. |
|
|
@@ -190,6 +190,13 @@ Labkit::RateLimit::Rule.new(
|
|
|
190
190
|
)
|
|
191
191
|
```
|
|
192
192
|
|
|
193
|
+
`Rule.new` raises `ArgumentError` for a static `limit` under 1 or a static
|
|
194
|
+
`period` under one second, except on `:skip` rules where both are inert.
|
|
195
|
+
Callables are not validated: a `limit` resolving to `0` blocks every matching
|
|
196
|
+
request, which is how to express "deny all" dynamically, and a `period`
|
|
197
|
+
resolving under one second reaches `EXPIRE` as `0` and deletes the counter on
|
|
198
|
+
every write.
|
|
199
|
+
|
|
193
200
|
### Matchers
|
|
194
201
|
|
|
195
202
|
A `match` hash gates whether a rule applies. Each value is normalised through
|
|
@@ -200,17 +207,63 @@ A `match` hash gates whether a rule applies. Each value is normalised through
|
|
|
200
207
|
| plain value | `eq` | `match: { user: nil }`, `match: { method: "POST" }` |
|
|
201
208
|
| `Regexp` | `re` | `match: { endpoint: %r{\A/api/} }` |
|
|
202
209
|
| `{ eq: <value> }` | `eq` | `match: { method: { eq: "POST" } }` (YAML-friendly) |
|
|
210
|
+
| `{ ne: <value> }` | `ne` | `match: { method: { ne: "OPTIONS" } }` |
|
|
203
211
|
| `{ re: <source> }`| `re` | `match: { endpoint: { re: '\A/api/' } }` (YAML-friendly) |
|
|
212
|
+
| `{ nre: <source> }`| `nre` | `match: { endpoint: { nre: '\A/api/' } }` |
|
|
204
213
|
| `{ oneOf: <Array> }` | `oneOf` | `match: { user: { oneOf: ["7", "42"] } }` (YAML-friendly) |
|
|
214
|
+
| `{ noneOf: <Array> }` | `noneOf` | `match: { plan: { noneOf: ["premium"] } }` |
|
|
215
|
+
| `Array` of matchers | `all` | `match: { endpoint: [%r{\A/api/}, { nre: 'internal' }] }` |
|
|
205
216
|
|
|
206
|
-
`re`
|
|
207
|
-
used against non-String values (e.g. matching a 503 status against
|
|
217
|
+
`re` and `nre` coerce the identifier value via `#to_s` before matching, so they
|
|
218
|
+
can be used against non-String values (e.g. matching a 503 status against
|
|
219
|
+
`{ re: '^5' }`).
|
|
208
220
|
|
|
209
221
|
`oneOf` is set membership via `Set#include?` (`eql?`/`hash` equality, no
|
|
210
222
|
coercion): it agrees with `eq` for Strings, Symbols, booleans, nil, and
|
|
211
223
|
same-class numerics, but cross-type numerics differ (`1 == 1.0`, yet
|
|
212
|
-
`{ oneOf: [1] }` does not match `1.0`).
|
|
213
|
-
|
|
224
|
+
`{ oneOf: [1] }` does not match `1.0`). `noneOf` is its complement.
|
|
225
|
+
|
|
226
|
+
An Array means **AND**, so one identifier key can carry several predicates:
|
|
227
|
+
|
|
228
|
+
```yaml
|
|
229
|
+
endpoint:
|
|
230
|
+
- nre: '/oauth/'
|
|
231
|
+
- nre: '\A/api/'
|
|
232
|
+
- nre: '\A/-/(?:health|liveness|readiness|metrics)'
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Its entries must be matcher Hashes or bare `Regexp`s, the two shapes that
|
|
236
|
+
already mean "a matcher" on their own. An Array of plain values is rejected.
|
|
237
|
+
A value meant as one matcher, arriving as a list, therefore fails loudly rather
|
|
238
|
+
than becoming a membership test. Membership is spelled `{ oneOf: [...] }`.
|
|
239
|
+
|
|
240
|
+
An Array holds matchers, not other Arrays, so a match value nests three deep at
|
|
241
|
+
most: an Array, a matcher Hash inside it, and that Hash's `oneOf` list. A value
|
|
242
|
+
nesting deeper than `Matcher::MAX_NESTING_DEPTH` is rejected when the rule is
|
|
243
|
+
built.
|
|
244
|
+
|
|
245
|
+
#### Negation and absent keys
|
|
246
|
+
|
|
247
|
+
`ne`, `nre` and `noneOf` are plain complements of their positive forms, with no
|
|
248
|
+
special case for a missing key. The `Evaluator` looks an identifier key up
|
|
249
|
+
rather than requiring it, so an absent key reaches the matcher as `nil`, and a
|
|
250
|
+
complement is therefore **true** for one: a request with no `endpoint` is not a
|
|
251
|
+
request to an excluded endpoint.
|
|
252
|
+
|
|
253
|
+
That widens a rule rather than narrowing it. A rule that should apply only when
|
|
254
|
+
the key is present says so, with a presence matcher alongside the exclusions:
|
|
255
|
+
|
|
256
|
+
```ruby
|
|
257
|
+
match: { endpoint: [/./, { nre: '\A/api/' }] }
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
Since this follows from inversion, a matcher that does accept `nil` inverts the
|
|
261
|
+
other way: `{ ne: nil }`, `{ noneOf: [nil] }` and `{ nre: '^$' }` are all false
|
|
262
|
+
for an absent key, the last because `nil.to_s` is `""`.
|
|
263
|
+
|
|
264
|
+
There is no cap on regex source length. Rules are written by operators, and the
|
|
265
|
+
only bound on a pattern is `Matcher::MATCH_TIMEOUT_SECONDS`, a 50ms wall-clock
|
|
266
|
+
budget applied per compiled `Regexp`.
|
|
214
267
|
|
|
215
268
|
Glob and prefix matchers are intentionally out of scope.
|
|
216
269
|
|
|
@@ -459,7 +512,8 @@ result.degraded? # => true if a matched count_distinct rule was skipped
|
|
|
459
512
|
result.info # => Result::Info or nil
|
|
460
513
|
result.evaluations # => every counted Result::Evaluation, in rule order
|
|
461
514
|
result.to_response_headers
|
|
462
|
-
# => { "RateLimit-
|
|
515
|
+
# => { "RateLimit-Name" => "<rule name>", "RateLimit-Limit" => "...", "RateLimit-Observed" => "...",
|
|
516
|
+
# "RateLimit-Remaining" => "...", "RateLimit-Reset" => "<unix-ts>" }
|
|
463
517
|
```
|
|
464
518
|
|
|
465
519
|
`Result::Info` holds the per-window counter snapshot:
|
|
@@ -475,6 +529,27 @@ result.to_response_headers
|
|
|
475
529
|
`to_response_headers` returns `{}` for an unmatched or error result, so it is
|
|
476
530
|
safe to merge unconditionally.
|
|
477
531
|
|
|
532
|
+
### Response headers
|
|
533
|
+
|
|
534
|
+
`Result#to_response_headers` reports the most constraining evaluation.
|
|
535
|
+
`Result::Evaluation#to_response_headers` builds the same headers for one evaluation, for a caller that picks its own evaluation across several results.
|
|
536
|
+
|
|
537
|
+
| header | present | value |
|
|
538
|
+
|-----------------------|-------------------|-----------------------------------------------------|
|
|
539
|
+
| `RateLimit-Name` | always | The rule's `name`. |
|
|
540
|
+
| `RateLimit-Limit` | always | `resolved_limit` in the rule's period, at least 0. |
|
|
541
|
+
| `RateLimit-Observed` | always | `count`, as an Integer, at least 0. |
|
|
542
|
+
| `RateLimit-Remaining` | always | `remaining`, as an Integer. |
|
|
543
|
+
| `RateLimit-Reset` | always | `reset_at` as a Unix timestamp. |
|
|
544
|
+
| `Retry-After` | blocking only | Seconds to `reset_at`, rounded up, at least 0. |
|
|
545
|
+
| `RateLimit-ResetTime` | blocking only | `reset_at` as an HTTP date. |
|
|
546
|
+
|
|
547
|
+
`Retry-After` and `RateLimit-ResetTime` belong on the 429, so they appear only when the evaluation blocks.
|
|
548
|
+
For a rule with `ban_for`, `reset_at` is the end of the ban, so these headers report when the ban lifts.
|
|
549
|
+
|
|
550
|
+
`RateLimit-Limit`, `RateLimit-Observed` and `RateLimit-Remaining` all count within the rule's `period`.
|
|
551
|
+
The headers do not state the period, so a client that needs it must know the rule.
|
|
552
|
+
|
|
478
553
|
## Fail-open
|
|
479
554
|
|
|
480
555
|
The evaluator wraps `check` and `peek` in a broad rescue. Any `StandardError`
|
|
@@ -12,27 +12,49 @@ module Labkit
|
|
|
12
12
|
# - any plain value (String, Symbol, Integer, ...) -> :eq matcher
|
|
13
13
|
# - a Regexp instance -> :re matcher (Ruby convenience)
|
|
14
14
|
# - { eq: <value> } -> :eq matcher (canonical, YAML-compatible)
|
|
15
|
+
# - { ne: <value> } -> :ne matcher (complement of :eq)
|
|
15
16
|
# - { re: <String|Regexp> } -> :re matcher (canonical, YAML-compatible)
|
|
17
|
+
# - { nre: <String|Regexp> } -> :nre matcher (complement of :re)
|
|
16
18
|
# - { oneOf: <Array> } -> :oneOf matcher (set membership, YAML-compatible)
|
|
19
|
+
# - { noneOf: <Array> } -> :noneOf matcher (complement of :oneOf)
|
|
20
|
+
# - [ <matcher>, <matcher>, ... ] -> :all matcher (every entry must match)
|
|
17
21
|
#
|
|
18
|
-
#
|
|
19
|
-
#
|
|
20
|
-
#
|
|
22
|
+
# An Array means AND, so several predicates can apply to one identifier key.
|
|
23
|
+
# Its entries must be matcher Hashes or bare Regexps, the two shapes that
|
|
24
|
+
# already mean "a matcher" on their own. An Array of plain values is
|
|
25
|
+
# rejected, so a value that was meant as a single matcher but arrived as a
|
|
26
|
+
# list fails loudly instead of silently becoming a membership test. An Array
|
|
27
|
+
# holds matchers rather than other Arrays, so a match value nests at most
|
|
28
|
+
# MAX_NESTING_DEPTH deep; see too_deeply_nested?.
|
|
21
29
|
#
|
|
22
|
-
# :oneOf membership is Set#include? (eql?/hash equality), not
|
|
23
|
-
# cross-type numerics differ (1 == 1.0, but {oneOf: [1]} does
|
|
30
|
+
# :oneOf and :noneOf membership is Set#include? (eql?/hash equality), not
|
|
31
|
+
# :eq's ==, so cross-type numerics differ (1 == 1.0, but {oneOf: [1]} does
|
|
32
|
+
# not match 1.0).
|
|
24
33
|
#
|
|
25
34
|
# Hash-key naming follows the metrics-catalog selector pattern. Glob,
|
|
26
35
|
# prefix, and other matcher kinds are intentionally out of scope here; see
|
|
27
36
|
# gitlab-com/gl-infra/production-engineering#28853 for that follow-up.
|
|
28
37
|
#
|
|
29
|
-
#
|
|
30
|
-
# the regex, so callers can match non-String identifier values such
|
|
31
|
-
# Integer status codes (e.g. {status: { re: "^5" }} against status: 503).
|
|
38
|
+
# The :re and :nre matchers coerce the identifier value via #to_s before
|
|
39
|
+
# applying the regex, so callers can match non-String identifier values such
|
|
40
|
+
# as Integer status codes (e.g. {status: { re: "^5" }} against status: 503).
|
|
41
|
+
#
|
|
42
|
+
# Each negation matcher is the plain complement of its positive form, with
|
|
43
|
+
# no special case for a missing key. Evaluator passes identifier[key], so an
|
|
44
|
+
# absent key arrives as nil, and the complement is therefore usually true
|
|
45
|
+
# for one: a request with no path is not a request to an excluded path. A
|
|
46
|
+
# rule that should apply only when the key is present says so, by adding a
|
|
47
|
+
# presence matcher alongside the exclusions:
|
|
48
|
+
#
|
|
49
|
+
# path: [/./, { nre: '\A/api/' }]
|
|
32
50
|
class Matcher < Data.define(:type, :value)
|
|
33
|
-
KNOWN_HASH_KEYS = %i[eq re oneOf].freeze
|
|
34
|
-
MAX_REGEX_SOURCE_LENGTH = 200
|
|
51
|
+
KNOWN_HASH_KEYS = %i[eq ne re nre oneOf noneOf].freeze
|
|
35
52
|
ERROR_INSPECT_LIMIT = 80
|
|
53
|
+
# No Hash or Array may sit this many levels below the match value, which
|
|
54
|
+
# is level 0. The deepest valid shape is an Array (0) holding a matcher
|
|
55
|
+
# Hash (1) whose :oneOf list (2) holds plain values; see
|
|
56
|
+
# too_deeply_nested?.
|
|
57
|
+
MAX_NESTING_DEPTH = 3
|
|
36
58
|
|
|
37
59
|
# Wall-clock budget for a single #match? call. Without it a match is
|
|
38
60
|
# bounded only by whatever global the host sets (40s in GitLab Rails,
|
|
@@ -45,16 +67,23 @@ module Labkit
|
|
|
45
67
|
# timeouts were wall-clock stalls, since Ruby counts real time and GC
|
|
46
68
|
# pauses on GitLab's git fleet have a p50 of ~72ms. Under 4x CPU
|
|
47
69
|
# oversubscription 50ms still fires on 0.16% of those matches, 5ms 1.33%.
|
|
70
|
+
#
|
|
71
|
+
# This timeout is the only bound on a pattern. There is no cap on regex
|
|
72
|
+
# source length: rules are written by operators rather than end users, and
|
|
73
|
+
# a cap that applied to {re: "..."} but not to a bare Regexp bounded
|
|
74
|
+
# nothing in practice, since callers pass Regexp constants.
|
|
48
75
|
MATCH_TIMEOUT_SECONDS = 0.05
|
|
49
76
|
|
|
50
77
|
def self.build(input)
|
|
78
|
+
raise ArgumentError, nesting_error(input) if too_deeply_nested?(input)
|
|
79
|
+
|
|
51
80
|
case input
|
|
52
81
|
when Regexp
|
|
53
|
-
new(type: :re, value: with_match_timeout(input))
|
|
82
|
+
new(type: :re, value: with_match_timeout(:re, input))
|
|
54
83
|
when Hash
|
|
55
84
|
from_hash(input)
|
|
56
85
|
when Array
|
|
57
|
-
|
|
86
|
+
compose(input)
|
|
58
87
|
else
|
|
59
88
|
new(type: :eq, value: input)
|
|
60
89
|
end
|
|
@@ -64,6 +93,15 @@ module Labkit
|
|
|
64
93
|
raise ArgumentError, invalid_shape_error(input) if input.size != 1
|
|
65
94
|
|
|
66
95
|
type, source = input.first
|
|
96
|
+
|
|
97
|
+
# Identifier rejects keys the same way. Without this, a key such as an
|
|
98
|
+
# Integer or nil leaves NoMethodError for the caller instead of the
|
|
99
|
+
# ArgumentError every other bad shape raises.
|
|
100
|
+
unless type.respond_to?(:to_sym)
|
|
101
|
+
raise ArgumentError,
|
|
102
|
+
"rate-limit match value type key #{truncate_for_error(type)} must be a String or Symbol"
|
|
103
|
+
end
|
|
104
|
+
|
|
67
105
|
type_sym = type.to_sym
|
|
68
106
|
|
|
69
107
|
unless KNOWN_HASH_KEYS.include?(type_sym)
|
|
@@ -77,31 +115,45 @@ module Labkit
|
|
|
77
115
|
|
|
78
116
|
def self.compile(type_sym, source)
|
|
79
117
|
case type_sym
|
|
80
|
-
when :eq
|
|
81
|
-
new(type:
|
|
82
|
-
when :oneOf
|
|
118
|
+
when :eq, :ne
|
|
119
|
+
new(type: type_sym, value: source)
|
|
120
|
+
when :oneOf, :noneOf
|
|
83
121
|
unless source.is_a?(Array)
|
|
84
122
|
raise ArgumentError,
|
|
85
|
-
"rate-limit match value {
|
|
123
|
+
"rate-limit match value {#{type_sym}: ...} must be an Array, got #{truncate_for_error(source)}"
|
|
86
124
|
end
|
|
87
125
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
126
|
+
# {oneOf: []} matches nothing, so the rule never applies and no harm
|
|
127
|
+
# follows. Its complement matches everything, which is an exclusion
|
|
128
|
+
# that excludes nothing: rejected for the same reason an empty Array
|
|
129
|
+
# of matchers is.
|
|
130
|
+
if type_sym == :noneOf && source.empty?
|
|
91
131
|
raise ArgumentError,
|
|
92
|
-
"rate-limit match value {
|
|
132
|
+
"rate-limit match value {noneOf: []} must not be empty, it matches everything"
|
|
93
133
|
end
|
|
94
134
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
raise ArgumentError,
|
|
99
|
-
"rate-limit match value {re: #{truncate_for_error(source)}} failed to compile: #{e.message}"
|
|
100
|
-
end
|
|
135
|
+
new(type: type_sym, value: Set.new(source).freeze)
|
|
136
|
+
when :re, :nre
|
|
137
|
+
new(type: type_sym, value: with_match_timeout(type_sym, source))
|
|
101
138
|
end
|
|
102
139
|
end
|
|
103
140
|
private_class_method :compile
|
|
104
141
|
|
|
142
|
+
# Builds the :all matcher behind the Array form. Entries go back through
|
|
143
|
+
# .build, which is what keeps a nested regex bounded by the match timeout.
|
|
144
|
+
def self.compose(input)
|
|
145
|
+
raise ArgumentError, "rate-limit match value [] must not be empty" if input.empty?
|
|
146
|
+
|
|
147
|
+
unless input.all? { |entry| entry.is_a?(Hash) || entry.is_a?(Regexp) }
|
|
148
|
+
raise ArgumentError,
|
|
149
|
+
"rate-limit match value Array entries must each be a matcher Hash or Regexp, " \
|
|
150
|
+
"got #{truncate_for_error(input)}"
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
new(type: :all, value: input.map { |entry| build(entry) }.freeze)
|
|
154
|
+
end
|
|
155
|
+
private_class_method :compose
|
|
156
|
+
|
|
105
157
|
# Compiles +source+ (a String pattern or an existing Regexp) into a
|
|
106
158
|
# Regexp bounded by MATCH_TIMEOUT_SECONDS.
|
|
107
159
|
#
|
|
@@ -116,7 +168,7 @@ module Labkit
|
|
|
116
168
|
# Recompiling an existing Regexp preserves its source and options
|
|
117
169
|
# (including the fixed-encoding flags) and therefore +#==+; only object
|
|
118
170
|
# identity changes.
|
|
119
|
-
def self.with_match_timeout(source)
|
|
171
|
+
def self.with_match_timeout(key, source)
|
|
120
172
|
return source if source.is_a?(Regexp) && source.timeout
|
|
121
173
|
|
|
122
174
|
if source.is_a?(Regexp)
|
|
@@ -124,12 +176,19 @@ module Labkit
|
|
|
124
176
|
else
|
|
125
177
|
Regexp.new(source, timeout: MATCH_TIMEOUT_SECONDS)
|
|
126
178
|
end
|
|
179
|
+
rescue RegexpError, TypeError => e
|
|
180
|
+
raise ArgumentError,
|
|
181
|
+
"rate-limit match value {#{key}: #{truncate_for_error(source)}} failed to compile: #{e.message}"
|
|
127
182
|
end
|
|
128
183
|
|
|
129
184
|
private_class_method :with_match_timeout
|
|
130
185
|
|
|
131
186
|
def self.invalid_shape_error(input)
|
|
132
|
-
|
|
187
|
+
# No key list here: this fires on the shape, and from_hash names the
|
|
188
|
+
# accepted keys when the shape is right but the key is not. Keeping the
|
|
189
|
+
# fixed part short is also what lets truncate_for_error bound the whole
|
|
190
|
+
# message.
|
|
191
|
+
"rate-limit match value must be a single-key Hash, or an Array of matchers, " \
|
|
133
192
|
"got #{truncate_for_error(input)}"
|
|
134
193
|
end
|
|
135
194
|
private_class_method :invalid_shape_error
|
|
@@ -140,14 +199,65 @@ module Labkit
|
|
|
140
199
|
end
|
|
141
200
|
private_class_method :truncate_for_error
|
|
142
201
|
|
|
202
|
+
def self.nesting_error(input)
|
|
203
|
+
# Does not inspect the input: it is too deep to render, which is the
|
|
204
|
+
# reason this fires. The class is the only hint available cheaply, and
|
|
205
|
+
# the nesting may be in a key or a value.
|
|
206
|
+
"rate-limit match value nests deeper than #{MAX_NESTING_DEPTH} levels (#{input.class})"
|
|
207
|
+
end
|
|
208
|
+
private_class_method :nesting_error
|
|
209
|
+
|
|
210
|
+
# A valid match value nests three deep at most: an Array of matcher
|
|
211
|
+
# Hashes whose :oneOf value is an Array of plain values. Anything past
|
|
212
|
+
# MAX_NESTING_DEPTH is rejected rather than built.
|
|
213
|
+
#
|
|
214
|
+
# The check is iterative because the alternative is finding out through
|
|
215
|
+
# #inspect, which recurses once per level while rendering an error
|
|
216
|
+
# message. Rules are built at boot and SystemStackError is not a
|
|
217
|
+
# StandardError, so that would take the process down rather than reject
|
|
218
|
+
# one rule, and rescuing it is not an option either: the VM aborts on a
|
|
219
|
+
# second overflow in the same process.
|
|
220
|
+
def self.too_deeply_nested?(input)
|
|
221
|
+
# A flat stack of node, depth pairs, so a visit allocates no Array. The
|
|
222
|
+
# pops must mirror the pushes: depth comes off first, then its node.
|
|
223
|
+
stack = [input, 0]
|
|
224
|
+
|
|
225
|
+
until stack.empty?
|
|
226
|
+
depth = stack.pop
|
|
227
|
+
node = stack.pop
|
|
228
|
+
next unless node.is_a?(Hash) || node.is_a?(Array)
|
|
229
|
+
return true if depth >= MAX_NESTING_DEPTH
|
|
230
|
+
|
|
231
|
+
# Keys as well as values: #inspect renders both, so a deeply nested
|
|
232
|
+
# key overflows the same way a deeply nested value does.
|
|
233
|
+
if node.is_a?(Hash)
|
|
234
|
+
node.each { |key, value| stack.push(key, depth + 1, value, depth + 1) }
|
|
235
|
+
else
|
|
236
|
+
node.each { |child| stack.push(child, depth + 1) }
|
|
237
|
+
end
|
|
238
|
+
end
|
|
239
|
+
|
|
240
|
+
false
|
|
241
|
+
end
|
|
242
|
+
private_class_method :too_deeply_nested?
|
|
243
|
+
|
|
143
244
|
def match?(identifier_value)
|
|
144
245
|
case type
|
|
145
246
|
when :eq
|
|
146
247
|
value == identifier_value
|
|
248
|
+
when :ne
|
|
249
|
+
value != identifier_value
|
|
147
250
|
when :oneOf
|
|
148
251
|
value.include?(identifier_value)
|
|
252
|
+
when :noneOf
|
|
253
|
+
# exclude? is ActiveSupport, and nothing in rate_limit requires it.
|
|
254
|
+
!value.include?(identifier_value) # rubocop:disable Rails/NegateInclude
|
|
149
255
|
when :re
|
|
150
256
|
value.match?(identifier_value.to_s)
|
|
257
|
+
when :nre
|
|
258
|
+
!value.match?(identifier_value.to_s)
|
|
259
|
+
when :all
|
|
260
|
+
value.all? { |matcher| matcher.match?(identifier_value) }
|
|
151
261
|
else
|
|
152
262
|
raise ArgumentError, "unknown matcher type: #{type.inspect}"
|
|
153
263
|
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "time" # Time#httpdate
|
|
4
|
+
|
|
3
5
|
module Labkit
|
|
4
6
|
module RateLimit
|
|
5
7
|
# Result is the return value of Limiter#check.
|
|
@@ -113,18 +115,13 @@ module Labkit
|
|
|
113
115
|
@most_constraining ||= @evaluations.min
|
|
114
116
|
end
|
|
115
117
|
|
|
116
|
-
# Returns
|
|
117
|
-
#
|
|
118
|
-
#
|
|
119
|
-
# the RateLimit header spec requires integer values.
|
|
118
|
+
# Returns the rate limit response headers of the most constraining
|
|
119
|
+
# evaluation, or {} when no rule matched, a :skip rule matched, or an
|
|
120
|
+
# error occurred. See Result::Evaluation#to_response_headers for the keys.
|
|
120
121
|
def to_response_headers
|
|
121
122
|
return {} unless matched? && !error? && info
|
|
122
123
|
|
|
123
|
-
|
|
124
|
-
"RateLimit-Limit" => info.resolved_limit.to_i.to_s,
|
|
125
|
-
"RateLimit-Remaining" => info.remaining.to_i.to_s,
|
|
126
|
-
"RateLimit-Reset" => info.reset_at.to_i.to_s
|
|
127
|
-
}
|
|
124
|
+
most_constraining.to_response_headers
|
|
128
125
|
end
|
|
129
126
|
end
|
|
130
127
|
|
|
@@ -155,6 +152,32 @@ module Labkit
|
|
|
155
152
|
constraint_rank <=> other.constraint_rank
|
|
156
153
|
end
|
|
157
154
|
|
|
155
|
+
# Returns the rate limit response headers for this evaluation. Values are
|
|
156
|
+
# Strings, and counts are coerced to Integer because the RateLimit header
|
|
157
|
+
# spec requires integer values even when count is fractional.
|
|
158
|
+
#
|
|
159
|
+
# RateLimit-Name (the rule name), RateLimit-Limit, RateLimit-Observed,
|
|
160
|
+
# RateLimit-Remaining and RateLimit-Reset (Unix timestamp) are always
|
|
161
|
+
# present. Limit, Observed and Remaining all count within the rule's own
|
|
162
|
+
# period. A blocking evaluation also gets Retry-After (seconds) and
|
|
163
|
+
# RateLimit-ResetTime (HTTP date), which belong on the 429.
|
|
164
|
+
def to_response_headers
|
|
165
|
+
headers = {}
|
|
166
|
+
headers["RateLimit-Name"] = rule.name
|
|
167
|
+
# Floored at 0 like remaining: a callable limit can resolve below zero
|
|
168
|
+
# to deny all, and an older client could credit a counter below zero.
|
|
169
|
+
headers["RateLimit-Limit"] = [info.resolved_limit.to_i, 0].max.to_s
|
|
170
|
+
headers["RateLimit-Observed"] = [info.count.to_i, 0].max.to_s
|
|
171
|
+
headers["RateLimit-Remaining"] = info.remaining.to_i.to_s
|
|
172
|
+
headers["RateLimit-Reset"] = info.reset_at.to_i.to_s
|
|
173
|
+
return headers unless block?
|
|
174
|
+
|
|
175
|
+
# Rounded up so a client that honours it does not retry before the reset.
|
|
176
|
+
headers["Retry-After"] = [(info.reset_at - Time.now.utc).ceil, 0].max.to_s
|
|
177
|
+
headers["RateLimit-ResetTime"] = info.reset_at.httpdate
|
|
178
|
+
headers
|
|
179
|
+
end
|
|
180
|
+
|
|
158
181
|
protected
|
|
159
182
|
|
|
160
183
|
def constraint_rank
|
|
@@ -10,8 +10,8 @@ module Labkit
|
|
|
10
10
|
# name - stable identifier used in Redis keys and log entries
|
|
11
11
|
# match - hash of identifier key/value pairs that must all match for
|
|
12
12
|
# the rule to apply; empty hash matches any identifier
|
|
13
|
-
# limit - request threshold; may be a callable (resolved per check)
|
|
14
|
-
# period - window in seconds; may be a callable (resolved per check)
|
|
13
|
+
# limit - request threshold, at least 1; may be a callable (resolved per check)
|
|
14
|
+
# period - window in seconds, at least 1; may be a callable (resolved per check)
|
|
15
15
|
# action - :limit (enforce; the result blocks when the rule is over
|
|
16
16
|
# its limit, which also terminates evaluation), :log (count
|
|
17
17
|
# and log only, never blocks and never terminates),
|
|
@@ -56,6 +56,16 @@ module Labkit
|
|
|
56
56
|
raise ArgumentError, "ban_for must be a Numeric of at least 1 second or a callable, got #{value.inspect}"
|
|
57
57
|
end
|
|
58
58
|
|
|
59
|
+
# A static limit under 1 blocks every request, and a period under a second
|
|
60
|
+
# reaches EXPIRE as 0, deleting the counter on every write. A :skip rule
|
|
61
|
+
# counts nothing, and a callable may still resolve to 0 to mean "deny all".
|
|
62
|
+
def self.normalize_numeric(field, value, action_sym)
|
|
63
|
+
return value if action_sym == :skip || value.respond_to?(:call)
|
|
64
|
+
return value if value.is_a?(Numeric) && value >= 1
|
|
65
|
+
|
|
66
|
+
raise ArgumentError, "#{field} must be a Numeric of at least 1 or a callable, got #{value.inspect}"
|
|
67
|
+
end
|
|
68
|
+
|
|
59
69
|
def self.normalize_count_distinct(value, characteristics_arr)
|
|
60
70
|
sym =
|
|
61
71
|
case value
|
|
@@ -94,8 +104,8 @@ module Labkit
|
|
|
94
104
|
super(
|
|
95
105
|
name: name_str.freeze,
|
|
96
106
|
match: match.transform_keys(&:to_sym).transform_values { |v| Matcher.build(v) }.freeze,
|
|
97
|
-
limit: limit,
|
|
98
|
-
period: period,
|
|
107
|
+
limit: self.class.normalize_numeric(:limit, limit, action_sym),
|
|
108
|
+
period: self.class.normalize_numeric(:period, period, action_sym),
|
|
99
109
|
action: action_sym,
|
|
100
110
|
characteristics: characteristics_arr,
|
|
101
111
|
count_distinct: count_distinct_sym,
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: gitlab-labkit
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 5.
|
|
4
|
+
version: 5.6.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Andrew Newdigate
|
|
@@ -277,6 +277,20 @@ dependencies:
|
|
|
277
277
|
- - "<"
|
|
278
278
|
- !ruby/object:Gem::Version
|
|
279
279
|
version: 6.0.0
|
|
280
|
+
- !ruby/object:Gem::Dependency
|
|
281
|
+
name: thrift
|
|
282
|
+
requirement: !ruby/object:Gem::Requirement
|
|
283
|
+
requirements:
|
|
284
|
+
- - "<"
|
|
285
|
+
- !ruby/object:Gem::Version
|
|
286
|
+
version: '0.25'
|
|
287
|
+
type: :runtime
|
|
288
|
+
prerelease: false
|
|
289
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
290
|
+
requirements:
|
|
291
|
+
- - "<"
|
|
292
|
+
- !ruby/object:Gem::Version
|
|
293
|
+
version: '0.25'
|
|
280
294
|
- !ruby/object:Gem::Dependency
|
|
281
295
|
name: excon
|
|
282
296
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -568,6 +582,7 @@ files:
|
|
|
568
582
|
- ".rufo"
|
|
569
583
|
- ".tool-versions"
|
|
570
584
|
- ".yamllint.yaml"
|
|
585
|
+
- AGENTS.md
|
|
571
586
|
- CODEOWNERS
|
|
572
587
|
- CONTRIBUTING.md
|
|
573
588
|
- Dangerfile
|