gitlab-labkit 5.8.0 → 5.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bbffb0883d80c949bfb28ffec2943b9cc89ffb88c2c02656afe99ad777e0f152
-  data.tar.gz: 93610eb46b3dbbfebe40c1b97d31d07b69af3f5e5ee0f152be29a66eca8da03e
+  metadata.gz: 23080f09f7295046f43c9519b76d3a71bef133c8178f60fb9d33bc13311c0947
+  data.tar.gz: 35ca7c5427293cad362e700551c1a6a33ba439866f8acd60e065944ab8085b6f
 SHA512:
-  metadata.gz: c01235e9277e28adbde6cb2e14cedebbabe78b46a8f7e771664875da94cd59ea3c31e2033b152ea8e23dce3cd8ce84c0c5fd310c9691d1662006df0e87f66b67
-  data.tar.gz: 87c05f1a0543e7969b35023d227d32cabc87455caa99aeca54a83832a5e40a2968895afb3982c4ecfcadc31d891558ce4abccb249f0758f95903367fd56b82df
+  metadata.gz: e83c05dc9f364acadcf7fa4c5343441476e8c4ab1fdb7b1ed36122499649caef0f6a6607d01e0c511042eb3f940fe4e799e0b422749ed28b8f8857ddc5c5fc6f
+  data.tar.gz: 0d9a4dccbb22b559ba33e2403151b4284b4e8240970384757ab1ec455d478bf8ad9865f7573883b7c11a819f68718c8111781e39856f6b4ad28f99dbf117e0a7

data/AGENTS.md

@@ -122,48 +122,26 @@ Keep an MR to one concern, and separate mechanical changes such as renames from
 
 ## Rate limiting
 
-`Labkit::RateLimit` is documented in `lib/labkit/rate_limit/README.md`.
-It is the one capability that shares state across SDKs: services in any language enforce the same limits against the same Redis.
-That makes the Redis key shape and the Lua scripts as much a contract as the HTTP headers, so changes here follow stricter rules than the rest of the gem.
-
-### Where each SDK keeps it
-
-| SDK | Repository | Module | Tests | Status |
-| --- | --- | --- | --- | --- |
-| Ruby (this repository) | gitlab-org/ruby/gems/labkit-ruby | `lib/labkit/rate_limit/` | `spec/labkit/rate_limit/` | released |
-| Go | gitlab-org/labkit | `v2/ratelimit/` | `v2/ratelimit/*_test.go`, run with `go -C v2 test ./ratelimit/...` | released |
-| Rust | gitlab-org/rust/labkit-rs | in progress | | not released, skip |
-| Python | gitlab-org/labkit-python | in progress | | not released, skip |
-| Spec | gitlab-org/quality/tooling/labkit-spec | `conformance/README.md` ("v1 decisions"), `conformance/ratelimit_scenarios/*.json`, `conformance/aspect_ratelimit.go` | `conformance/` | judge |
+Services in every language enforce the same rate limits against one shared Redis.
+So these are a contract between all SDKs:
 
-### The three layers
+- Redis keys, values, and TTLs
+- Lua scripts
+- the counting rule that decides a block
+- the response status and `RateLimit-*` headers
+- metrics and log fields
 
-Every SDK must agree on three layers.
-This section defines the layers; the current values (key grammar, header names, counting rules, metric names, log fields) live in labkit-spec, in the "v1 decisions" list in `conformance/README.md` and the scenarios under `conformance/ratelimit_scenarios/`.
+[labkit-spec](https://gitlab.com/gitlab-org/quality/tooling/labkit-spec) owns that contract: the "v1 decisions" in `conformance/README.md` and the scenarios in `conformance/ratelimit_scenarios/`.
 If this file and labkit-spec disagree, labkit-spec wins.
+`Labkit::RateLimit` is documented in `lib/labkit/rate_limit/README.md`.
+
+| SDK | Repository | Rate limiter | Tests |
+| --- | --- | --- | --- |
+| Ruby | [labkit-ruby](https://gitlab.com/gitlab-org/ruby/gems/labkit-ruby) | `lib/labkit/rate_limit/` | `bundle exec rspec spec/labkit/rate_limit` |
+| Go | [labkit](https://gitlab.com/gitlab-org/labkit) | `v2/ratelimit/` | `go -C v2 test ./ratelimit/...` |
+| Rust | [labkit-rs](https://gitlab.com/gitlab-org/rust/labkit-rs) | `labkit/src/ratelimit/` | `cargo test -p labkit --features ratelimit-redis --lib ratelimit` |
 
-1. **Storage contract.**
-   Anything written to or read from Redis: key shape and prefix, how values are encoded, TTL rules, and the Lua scripts.
-   Two SDKs sharing one Redis with different storage rules keep two counters for one limit.
-   Never change this layer in one SDK.
-   It changes in labkit-spec first, then every SDK follows.
-2. **Observable contract.**
-   Anything a caller or an operator can see: response status and headers, the counting rule that decides a block, log line messages and field sets, metric families and labels.
-   Also changes in labkit-spec first.
-3. **Behaviour within the contract.**
-   Input validation and edge cases the contract already implies, where one SDK does something the contract does not allow.
-   These may be fixed SDK by SDK, and every SDK should end up the same.
-
-A change that mixes layers is governed by the strictest one: storage, then observable, then behaviour.
-When a change does not fit one layer clearly, say so in the MR instead of picking one.
-
-### Rules for rate-limit changes
-
-- A change to layer 1 or 2 opens a labkit-spec MR first and waits for it to merge.
-- A change to layer 3 should be checked against the Go SDK.
-  If Go has the same gap, say so in the MR so it can be ported.
-- When a review finds a divergence between SDKs, a conformance scenario in labkit-spec must follow in the same week, so the harness sees the gap next time.
-  The author of the MR where the divergence was found owns this.
-  Mention the sync agent on the review thread to have it draft the scenario MR; if it cannot, open a labkit-spec issue and link it from the thread.
-- Never edit Lua scripts, key construction, header names, metric names, or log field names to fix a Ruby-only bug.
-  If a fix seems to need that, it is a contract change.
+- Change the contract in labkit-spec first, then in every SDK. Merge the labkit-spec MR before any SDK MR. Never change the contract in one SDK only.
+- A fix inside the contract can land in one SDK. Check the other SDKs, and say in the MR if they have the same gap.
+- If you are not sure that a change is inside the contract, treat it as a contract change and say so in the MR.
+- When review finds a gap between SDKs, add a labkit-spec scenario for it in the same week, or open a labkit-spec issue and link it from the thread. The author of the MR where the gap was found owns this.

data/CODEOWNERS

@@ -1,4 +1,4 @@
 # CODEOWNERS is used to lookup assignees for
 # Renovate Bot dependency change Merge Requests.
 # https://docs.renovatebot.com/configuration-options/#assigneesfromcodeowners
-* @ayufan @hmerscher @e_forbes @M_Alvarez @mwoolf @sankalp_gl @hardikgala @nindurkar @ashs2
+* @ayufan @hmerscher @e_forbes @M_Alvarez @mwoolf @sankalp_gl @hardikgala @nindurkar @ashs2 @lhollinda

data/lib/labkit/rate_limit/README.md

@@ -177,6 +177,7 @@ A `Rule` is a `Data.define` value object with the following fields:
 | `characteristics` | Array of identifier keys whose values are folded into the Redis counter key. Each unique combination gets its own counter.                                           |
 | `count_distinct`  | Optional identifier key. When set, the rule counts distinct values seen for that key within the window, backed by a Redis SET rather than a counter. Must not overlap `characteristics`. |
 | `ban_for`         | Optional ban duration, whole seconds, minimum 1. Changes the accounting; `action` still decides who is blocked. Rejected on `:skip`. May be a callable resolved on every check.     |
+| `admit_counted_values` | Optional, default `false`. Only valid with `count_distinct`, and rejected on `:skip`. When `true`, the rule counts only the values it admits, so values already counted keep passing once the set is full. Rename the rule when changing it. See [Admitting counted values](#admitting-counted-values). |
 
 Making `limit` or `period` callable is the supported pattern for
 runtime-tunable thresholds (e.g. feature flags or database-backed settings):
@@ -193,9 +194,11 @@ Labkit::RateLimit::Rule.new(
 `Rule.new` raises `ArgumentError` for a static `limit` under 1 or a static
 `period` under one second, except on `:skip` rules where both are inert.
 Callables are not validated: a `limit` resolving to `0` blocks every matching
-request, which is how to express "deny all" dynamically, and a `period`
-resolving under one second reaches `EXPIRE` as `0` and deletes the counter on
-every write.
+request, which is how to express "deny all" dynamically. On an
+`admit_counted_values` rule it only denies new values, since values already
+counted in the window still pass; to block those too, put a counter rule whose
+limit resolves to `0` in front of it. A `period` resolving under one second
+reaches `EXPIRE` as `0` and deletes the counter on every write.
 
 ### Matchers
 
@@ -315,7 +318,7 @@ flowchart TD
     Eval --> Build[Build Evaluation<br/>resolve limit/period]
     Build --> Add[Add evaluation to Result]
     Add --> Emit[Emit rule_evaluations_total<br/>+ limit/period gauges]
-    Emit --> Act{"result.block?<br/>(:limit rule over its limit,<br/>or its ban in force)"}
+    Emit --> Act{"result.block?<br/>(:limit rule over its limit,<br/>its ban in force, or an admitting<br/>rule that refused the value)"}
     Act -->|yes| Return([Return Result<br/>action=:block])
     Act -->|"no (:log, or under limit)"| Iter
     Iter -->|no more rules| Return2([Return Result reporting<br/>most-constraining evaluation,<br/>or matched=false if none])
@@ -349,10 +352,11 @@ visible via `checks_total{error="true"}` and the
 
 The rule's `action` describes what the rule does; the result's `action`
 describes the outcome — what the caller should do — and is only ever `:allow`
-or `:block`. A matching rule increments its counter, with two exceptions:
-`:skip` rules never touch Redis, and a rule carrying `ban_for` whose ban is
-already in force short-circuits before incrementing. The `result` label on
-`rule_evaluations_total` records what each evaluation decided:
+or `:block`. A matching rule increments its counter, with three exceptions:
+`:skip` rules never touch Redis, a rule carrying `ban_for` whose ban is
+already in force short-circuits before incrementing, and an
+`admit_counted_values` rule adds a value only when it admits it. The `result`
+label on `rule_evaluations_total` records what each evaluation decided:
 
 | rule action | what it does                                  | exceeded? | result action | `result` label | terminating?  |
 |-------------|-----------------------------------------------|-----------|---------------|----------------|---------------|
@@ -419,6 +423,60 @@ window, and it stops counting while that ban holds:
   suit counting discrete failures, such as bad credentials, better than they
   suit variable-cost resource limits.
 
+### Admitting counted values
+
+A plain `count_distinct` rule adds every checked value to its set, including
+the values it blocks. Once one value is blocked, the count stays over the
+limit for the rest of the window, so values that were counted earlier are
+blocked too. That suits "block after N distinct values", such as the number
+of users seen from one IP.
+
+Set `admit_counted_values: true` to cap how many distinct values can be
+admitted per window instead:
+
+```ruby
+Labkit::RateLimit::Rule.new(
+  name: "participants_per_namespace",
+  limit: 20,
+  period: 86_400,
+  characteristics: [:namespace],
+  count_distinct: :email_digest,
+  admit_counted_values: true
+)
+```
+
+- A value already in the set is allowed and leaves the count unchanged, even
+  when the set is full.
+- A new value is added only while the count is below the limit. Otherwise the
+  request is blocked and the value is not added, so a blocked value stays
+  blocked when it is retried and the count does not grow past the limit.
+- A blocked request reports the set's count and `RateLimit-Remaining: 0`. The
+  block comes from the admission, not from `count > limit`, the same way a ban
+  decides the block apart from the count.
+- If the limit is lowered during a window, the set can hold more values than
+  the new limit. Values already in it are still allowed, and new values are
+  blocked until the window ends.
+- `peek` reports what `check` would decide when the identifier has the
+  `count_distinct` value, without writing: exceeded only when that value is
+  not counted and the count has reached the limit. Without the value, `check`
+  skips the rule, but `peek` answers for a value that is not counted yet, so
+  it reports exceeded once the count reaches the limit.
+- A `:log` rule does the same accounting, including not adding refused
+  values, and only skips the blocking.
+- `cost:` is ignored, as for a plain `count_distinct` rule, so a `cost: 0`
+  check still admits a new value.
+- Rules count a request up to the one that blocks it. An admitting rule placed
+  before a counter rule that blocks still admits the value, which then takes a
+  slot for the rest of the window. Put admitting rules after the counter rules
+  in the same limiter.
+
+The Redis key and the stored values are the same as for a plain
+`count_distinct` rule, and a plain rule adds every value it checks, including
+the ones it blocks. A rule switched between the two modes keeps those values,
+and they count as admitted until the window ends. Rename the rule when
+changing its mode, which abandons its counters (see [Rule](#rule)), and give a
+rule the same mode in every SDK that writes to it.
+
 ### Redis keys
 
 Each matched check writes a key shaped:
@@ -439,7 +497,8 @@ is not extended on subsequent increments, so the window is a true fixed
 window starting at the first request, not a sliding window.
 
 A check is a single `EVALSHA` of `INCR_SCRIPT` (or `SADD_SCRIPT` for
-`count_distinct` rules, or `BAN_SCRIPT` for rules carrying `ban_for`). Doing the whole
+`count_distinct` rules, `ADMIT_SCRIPT` for `count_distinct` rules with
+`admit_counted_values`, or `BAN_SCRIPT` for rules carrying `ban_for`). Doing the whole
 read-modify-write inside Lua means there is no window between the increment
 and the `EXPIRE` in which a key could be left without a TTL.
 
@@ -488,9 +547,10 @@ key with no expiry, the TTL only needs reading once — after the increment.
 That single read distinguishes both cases the script must handle, so there is
 no reason to read it again before mutating.
 
-`peek` does not use a script: it pipelines `GET` + `TTL` (or `SCARD` + `TTL`)
-and never issues `EXPIRE`, so it cannot start or extend a window. A missing
-key (`GET → nil`, `TTL → -2`) is reported as `count = 0`, and `build_evaluation`
+`peek` does not use a script: it pipelines `GET` + `TTL` (or `SCARD` + `TTL`,
+plus `SISMEMBER` for an `admit_counted_values` rule when the identifier has the
+value) and never issues `EXPIRE`, so it cannot start or extend a window. A
+missing key (`GET → nil`, `TTL → -2`) is reported as `count = 0`, and `build_evaluation`
 falls back to the rule's period for `reset_at` since there is no Redis-side
 window to read.
 
@@ -503,7 +563,7 @@ the most constraining one (see [Evaluation flow](#evaluation-flow)):
 ```ruby
 result.matched?           # => true if some rule matched
 result.skipped?           # => true if a matched :skip rule bypassed the check
-result.exceeded?          # => true if the reported rule's counter > limit
+result.exceeded?          # => true if the reported rule is over its limit
 result.action             # => :block | :allow — what the caller should do
 result.block?             # => result.action == :block
 result.rule               # => the reported Rule, or nil

data/lib/labkit/rate_limit/evaluator.rb

@@ -67,6 +67,40 @@ module Labkit
         return {count, ttl_after}
       LUA
 
+      # Atomic admission for count_distinct rules carrying admit_counted_values.
+      # A member already in the set is admitted without changing it; a new one
+      # is added only while SCARD is below the limit. A refused member is never
+      # added, so it stays refused on retry and the count does not grow past the
+      # limit. Returns post-state {count, TTL, admitted}.
+      #
+      # The same self-healing as SADD_SCRIPT applies to keys left without TTL.
+      # EXPIRE on a missing key does nothing, so a refusal on a missing key (a
+      # limit resolving to 0) writes nothing.
+      ADMIT_SCRIPT = Labkit::Redis::Script.new(<<~LUA)
+        local ttl = ARGV[1]
+        local member = ARGV[2]
+        local limit = tonumber(ARGV[3])
+
+        local count = redis.call('SCARD', KEYS[1])
+        local admitted = 1
+        if redis.call('SISMEMBER', KEYS[1], member) == 0 then
+          if count < limit then
+            redis.call('SADD', KEYS[1], member)
+            count = count + 1
+          else
+            admitted = 0
+          end
+        end
+
+        local ttl_after = redis.call('TTL', KEYS[1])
+        if ttl_after < 0 then
+          redis.call('EXPIRE', KEYS[1], ttl)
+          ttl_after = tonumber(ttl)
+        end
+
+        return {count, ttl_after, admitted}
+      LUA
+
       # Atomic ban check + INCR + conditional ban write, for rules carrying
       # ban_for.
       #
@@ -262,9 +296,10 @@ module Labkit
       # here too: check can report a :log rule's evaluation, so excluding them
       # would make peek answer a different question than check.
       #
-      # peek does not need the count_distinct identifier key - it reads SCARD on
-      # the rule-keyed compound key, which contains the cardinality across all
-      # members. So missing-key fail-open does not apply here.
+      # peek does not require the count_distinct identifier key: it reads SCARD
+      # on the rule-keyed compound key, so missing-key fail-open does not apply.
+      # An admitting rule also looks the value up when it is present (see
+      # peek_admission_rule).
       def peek_rules(identifier, rule_context, cursor)
         result = Result.new
 
@@ -302,6 +337,8 @@ module Labkit
         resolved_limit = Integer(resolve_value(rule.limit, rule_context))
         resolved_period = Integer(resolve_value(rule.period, rule_context))
 
+        return evaluate_admission_rule(rule, redis_key, identifier, resolved_limit, resolved_period) if rule.admit_counted_values
+
         # cost is ignored for count_distinct rules: SADD is binary (a member is
         # either added or not), and the post-add count is SCARD regardless.
         count, ttl =
@@ -314,6 +351,15 @@ module Labkit
         build_evaluation(rule, resolved_limit, resolved_period, count, ttl)
       end
 
+      # The count does not grow past the limit in this mode, so the script's
+      # admission decides the block, the way a ban decides it apart from the count.
+      # cost is ignored, as for a plain count_distinct rule.
+      def evaluate_admission_rule(rule, redis_key, identifier, resolved_limit, resolved_period)
+        count, ttl, admitted = admit_with_ttl(redis_key, identifier[rule.count_distinct], resolved_limit, resolved_period)
+
+        build_evaluation(rule, resolved_limit, resolved_period, count, ttl, exceeded: !admitted)
+      end
+
       def peek_rule(rule, identifier, rule_context)
         return peek_ban_rule(rule, identifier, rule_context) if rule.ban_for
 
@@ -321,10 +367,22 @@ module Labkit
         resolved_limit = Integer(resolve_value(rule.limit, rule_context))
         resolved_period = Integer(resolve_value(rule.period, rule_context))
 
+        return peek_admission_rule(rule, redis_key, identifier, resolved_limit, resolved_period) if rule.admit_counted_values
+
         count, ttl = rule.count_distinct ? scard_with_ttl(redis_key) : read_with_ttl(redis_key)
         build_evaluation(rule, resolved_limit, resolved_period, count, ttl)
       end
 
+      # An admission is decided before the script writes anything, so with the
+      # value peek reports exactly what check would. Without it, check skips the
+      # rule, but peek answers for a value that is not counted yet.
+      def peek_admission_rule(rule, redis_key, identifier, resolved_limit, resolved_period)
+        value = identifier[rule.count_distinct] unless missing_count_distinct_value?(rule, identifier)
+        count, ttl, counted = scard_with_ttl(redis_key, member: value)
+
+        build_evaluation(rule, resolved_limit, resolved_period, count, ttl, exceeded: !counted && count >= resolved_limit)
+      end
+
       def evaluate_ban_rule(rule, identifier, cost, rule_context)
         resolved_limit = Integer(resolve_value(rule.limit, rule_context))
         resolved_period = Integer(resolve_value(rule.period, rule_context))
@@ -382,7 +440,7 @@ module Labkit
         Result::Evaluation.new(rule: rule, exceeded: banned, info: info)
       end
 
-      def build_evaluation(rule, resolved_limit, resolved_period, count, ttl)
+      def build_evaluation(rule, resolved_limit, resolved_period, count, ttl, exceeded: count > resolved_limit)
         info = Result::Info.new(
           resolved_limit: resolved_limit, resolved_period: resolved_period,
           count: count,
@@ -390,12 +448,13 @@ module Labkit
           reset_at: Time.now.utc + (ttl >= 0 ? ttl : resolved_period)
         )
 
-        Result::Evaluation.new(rule: rule, exceeded: count > resolved_limit, info: info)
+        Result::Evaluation.new(rule: rule, exceeded: exceeded, info: info)
       end
 
       # RateLimit-Remaining must not exceed RateLimit-Limit, and a counter can
       # sit below zero if an older client credited it. A limit resolving to a
-      # non-positive value means "deny all", so the ceiling floors at zero too.
+      # non-positive value means "deny all" ("deny new values" on an
+      # admit_counted_values rule), so the ceiling floors at zero too.
       def remaining_of(resolved_limit, count)
         ceiling = [resolved_limit, 0].max
         (ceiling - count).clamp(0, ceiling)
@@ -523,17 +582,28 @@ module Labkit
         end
       end
 
-      # Pipelined SCARD + TTL. SCARD on a missing key returns 0, so no
-      # explicit nil handling is needed (unlike GET in read_with_ttl).
-      # SCARD is integer-valued; coerced to Float for type uniformity with
-      # the INCR path so callers see a consistent count type.
-      def scard_with_ttl(redis_key)
+      # Atomic admission. See ADMIT_SCRIPT.
+      def admit_with_ttl(redis_key, member, limit, period)
+        member_str = encode_char_value(member.to_s)
+        @redis.with do |conn|
+          raw_count, ttl, admitted = ADMIT_SCRIPT.eval(conn, keys: [redis_key], argv: [period, member_str, limit])
+          [Float(raw_count), Integer(ttl), admitted == 1]
+        end
+      end
+
+      # Pipelined SCARD + TTL, plus SISMEMBER when given a member to look up
+      # (admitting rules), returned as the third element. SCARD on a missing key
+      # returns 0, so no explicit nil handling is needed (unlike GET in
+      # read_with_ttl). SCARD is integer-valued; coerced to Float for type
+      # uniformity with the INCR path so callers see a consistent count type.
+      def scard_with_ttl(redis_key, member: nil)
         @redis.with do |conn|
-          scard, ttl = conn.pipelined do |pipe|
+          scard, ttl, counted = conn.pipelined do |pipe|
             pipe.scard(redis_key)
             pipe.ttl(redis_key)
+            pipe.sismember(redis_key, encode_char_value(member.to_s)) unless member.nil?
           end
-          [Float(scard), ttl]
+          [Float(scard), ttl, counted == true]
         end
       end
 

data/lib/labkit/rate_limit/limiter.rb

@@ -35,6 +35,8 @@ module Labkit
       # @param cost [Numeric] amount to add to the counter. Defaults to 1
       #   (count-mode). Pass a non-1 Numeric for cost-mode counters such as
       #   resource-usage limits; passing 0 reads the counter without writing.
+      #   count_distinct rules ignore cost, so a cost of 0 still adds the value
+      #   (or admits it, on an admit_counted_values rule).
       #   A negative cost would credit quota back, so the check fails open and
       #   logs instead of counting.
       # @param rule_context [Hash, nil] optional per-request context passed to
@@ -60,8 +62,10 @@ module Labkit
       #
       # When the underlying Redis key does not exist yet, the result reports
       # count=0, exceeded=false, and remaining=resolved_limit; matched? is
-      # still true because the rule applied. On Redis error the result fails
-      # open identically to {#check}.
+      # still true because the rule applied. The exception is an
+      # admit_counted_values rule whose limit resolves to 0 or less, which
+      # refuses every new value and so reports exceeded. On Redis error the result
+      # fails open identically to {#check}.
       #
       # @param identifier [Identifier, Hash] caller attributes for this request
       # @param rule_context [Hash, nil] see {#check}

data/lib/labkit/rate_limit/result.rb

@@ -15,7 +15,8 @@ module Labkit
     # from the same evaluations are not #==.
     #
     # matched?    - true if at least one rule's match conditions were satisfied
-    # exceeded?   - true if the reported evaluation's counter exceeded its limit
+    # exceeded?   - true if the reported evaluation is over its limit (see
+    #               Result::Evaluation#exceeded)
     # action      - the outcome: what the caller should do
     #               :block = a :limit rule matched and is over its limit
     #               :allow = everything else, including an exceeded :log rule
@@ -127,7 +128,8 @@ module Labkit
 
     # The outcome of counting one matched rule.
     # rule     - the evaluated Rule
-    # exceeded - whether the post-increment count exceeded the resolved limit
+    # exceeded - whether the rule is over its limit: the post-increment count
+    #            exceeded it, a ban holds, or an admitting rule refused the value
     # info     - Result::Info with the per-window counters
     #
     # Evaluations order by constraint: a blocking evaluation (a :limit rule over
@@ -165,7 +167,8 @@ module Labkit
         headers = {}
         headers["RateLimit-Name"] = rule.name
         # Floored at 0 like remaining: a callable limit can resolve below zero
-        # to deny all, and an older client could credit a counter below zero.
+        # to deny all (deny new values on an admit_counted_values rule), and an
+        # older client could credit a counter below zero.
         headers["RateLimit-Limit"] = [info.resolved_limit.to_i, 0].max.to_s
         headers["RateLimit-Observed"] = [info.count.to_i, 0].max.to_s
         headers["RateLimit-Remaining"] = info.remaining.to_i.to_s

data/lib/labkit/rate_limit/rule.rb

@@ -31,11 +31,22 @@ module Labkit
     #                   the (characteristics-bucketed) period, backed by a Redis SET.
     #                   When nil (default), the rule counts the number of calls,
     #                   backed by INCR. The named key must not overlap +characteristics+.
+    # admit_counted_values - optional Boolean, only valid with +count_distinct+ and
+    #                   rejected on :skip.
+    #                   When true, the rule counts only the values it admits: a value
+    #                   already in the window's set is allowed even when the set is
+    #                   full, and a new value is added only while the count is below
+    #                   the limit, so a value this rule refuses is never counted.
+    #                   Rules count a request up to the one that blocks it, so put
+    #                   admitting rules after counter rules. Default false; nil
+    #                   means not set.
     #
     # +name+ must be a lowercase alphanumeric-and-underscore string of at most 64
     # characters. It is used as the middle segment of every Redis counter key for
     # this rule, so changing a rule's name mid-window abandons its in-flight counters.
-    Rule = Data.define(:name, :match, :limit, :period, :action, :characteristics, :count_distinct, :ban_for) do
+    Rule = Data.define(
+      :name, :match, :limit, :period, :action, :characteristics, :count_distinct, :ban_for, :admit_counted_values
+    ) do
       # ban_for changes the accounting; action still decides who gets blocked.
       # A :skip rule counts nothing, so it has no limit to cross and no ban to
       # write.
@@ -58,7 +69,8 @@ module Labkit
 
       # A static limit under 1 blocks every request, and a period under a second
       # reaches EXPIRE as 0, deleting the counter on every write. A :skip rule
-      # counts nothing, and a callable may still resolve to 0 to mean "deny all".
+      # counts nothing, and a callable may still resolve to 0 to mean "deny all"
+      # ("deny new values" on an admit_counted_values rule).
       def self.normalize_numeric(field, value, action_sym)
         return value if action_sym == :skip || value.respond_to?(:call)
         return value if value.is_a?(Numeric) && value >= 1
@@ -66,6 +78,19 @@ module Labkit
         raise ArgumentError, "#{field} must be a Numeric of at least 1 or a callable, got #{value.inspect}"
       end
 
+      # Admission only means something for a SET-backed rule, where a value can
+      # already be counted. A counter has no members to admit, and a :skip rule
+      # counts nothing.
+      def self.normalize_admit_counted_values(value, action_sym, count_distinct_sym)
+        # nil means "not set", like count_distinct and ban_for.
+        value = false if value.nil?
+        raise ArgumentError, "admit_counted_values must be true, false, or nil, got #{value.inspect}" unless [true, false].include?(value)
+        raise ArgumentError, "admit_counted_values is not valid on a :skip rule" if value && action_sym == :skip
+        raise ArgumentError, "admit_counted_values requires count_distinct" if value && count_distinct_sym.nil?
+
+        value
+      end
+
       def self.normalize_count_distinct(value, characteristics_arr)
         sym =
           case value
@@ -81,9 +106,9 @@ module Labkit
         sym
       end
 
-      def initialize(
+      def initialize( # rubocop:disable Metrics/ParameterLists -- one keyword per Rule field; the public API grows by keywords with defaults
         name:, limit:, period:, characteristics:,
-        match: {}, action: :limit, count_distinct: nil, ban_for: nil
+        match: {}, action: :limit, count_distinct: nil, ban_for: nil, admit_counted_values: false
       )
         raise ArgumentError, "name must be a String or Symbol, got #{name.class}" unless name.is_a?(String) || name.is_a?(Symbol)
 
@@ -109,7 +134,8 @@ module Labkit
           action: action_sym,
           characteristics: characteristics_arr,
           count_distinct: count_distinct_sym,
-          ban_for: self.class.normalize_ban_for(ban_for, action_sym, count_distinct_sym)
+          ban_for: self.class.normalize_ban_for(ban_for, action_sym, count_distinct_sym),
+          admit_counted_values: self.class.normalize_admit_counted_values(admit_counted_values, action_sym, count_distinct_sym)
         )
       end
     end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: gitlab-labkit
 version: !ruby/object:Gem::Version
-  version: 5.8.0
+  version: 5.9.0
 platform: ruby
 authors:
 - Andrew Newdigate
@@ -570,7 +570,6 @@ files:
 - ".gitlab-ci-asdf-versions.yml"
 - ".gitlab-ci-other-versions.yml"
 - ".gitlab-ci.yml"
-- ".gitlab/CODEOWNERS"
 - ".gitlab/duo/agent-config.yml"
 - ".gitlab/merge_request_templates/default.md"
 - ".gitleaks.toml"

data/.gitlab/CODEOWNERS

@@ -1 +0,0 @@
-* @mkaeppler @ayufan @hmerscher @sankalp_gl @hardikgala @nindurkar @ashs2