xero-kiwi 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c3c9271654c527275c19a5738effb1f5d334844a14898eaf467fc6671fbc308a
-  data.tar.gz: 0645a88a9ccd58130524e24d9981f3999b74ddf8ee98af9600a73f45cda4d29e
+  metadata.gz: e0e650e95ecce31d4acf75d1123d504bc11cacc9725e838643b6fbae235b04c0
+  data.tar.gz: 950f370a803af1b1f6aacf87bcaadbf5d00f2d44b1cb54061bee0d74b70e5eae
 SHA512:
-  metadata.gz: 74bbf0e97f3207d0aaaa16c85a82fe5c1f8572ef10c3c5d802ed2b051538ad002c9aae5df62da3f9ffd7aa76e64d4446a33cfcb98ac5d9eec3edc2f6a143d1bb
-  data.tar.gz: 32141ab2ffa3f40865a432711e1e6147c99006325e30f633f035790ab1ffefd085021eda40d5f2e22621eda084fb34174691fe6c65c405497ab7452106c9aa69
+  metadata.gz: dbf4f32d568839238ce077d0243fe59ad0ba54f8e332b0870c6f23b53c63425f9a41b7842491d862e6bced6ad8bb14db71b1af5b83e704d1833e53b98a7a5ced
+  data.tar.gz: 32e1d56e608bc844a452ca9a3a20ffff984c5a64ab1a46dc896498c3543c7deb9c581c5c1fc821ea40cff124b0d5e3ead035fc70e8a6857cce97754ef9131b90

data/CHANGELOG.md

@@ -1,5 +1,12 @@
 ## [Unreleased]
 
+## [0.7.0] - 2026-09-27
+
+### Added
+
+- `RateLimitError`, `Throttle::DailyLimitExhausted` and `Throttle::Timeout` now carry the **`tenant_id`** they relate to, and `Throttle::Timeout` also carries **`retry_after`** (it previously carried nothing at all). A caller recording a durable back-off signal — "this tenant is paused until T, because X" — now gets every fact from the exception instead of inferring the tenant from surrounding context, which breaks as soon as one client serves more than one tenant. `docs/retries-and-rate-limits.md` gains a section on the pattern.
+- Documented that `client.rate_limit` is **process-local**. The reported figures live in memory on the `Client` that made the request, so a worker's reading dies with the job. That is fine for "should this loop stop early?", which is what it is for, and is not a way to tell a web request that a tenant is backed off.
+
 ## [0.6.0] - 2026-09-27
 
 ### Fixed

data/docs/retries-and-rate-limits.md

@@ -68,6 +68,59 @@ When neither source has a reading, `day_remaining` is nil and
 reason to halt a sync, and halting would break the common case where the
 first call is what populates the figures.
 
+**`client.rate_limit` is process-local.** The reported figures live in
+memory on the `Client` that made the request, so a Sidekiq worker's reading
+is gone when the job ends and no other process can see it. That's fine for
+"should this loop stop early?", which is what it's for. It is not a way to
+tell a web request that a tenant is currently backed off — for that, see
+below.
+
+## Recording a durable back-off signal
+
+If something outside your sync needs to know a tenant is backed off — a
+dashboard badge, an API response carrying its own `Retry-After`, a guard
+that refuses to start another sync — you need a record that outlives the
+process that observed it.
+
+Kiwi deliberately doesn't keep that record for you, but it gives you
+everything needed to keep one. Both back-off exceptions name the tenant,
+say how long, and say which limit was hit:
+
+```ruby
+begin
+  client.each_invoice_page(tenant_id) { |page| import(page) }
+rescue XeroKiwi::RateLimitError => e
+  # Xero said so. e.problem is "minute" / "day" / "appminute".
+  BackoffRecord.set(e.tenant_id, seconds: e.retry_after || 60, source: :reported)
+  raise
+rescue XeroKiwi::Throttle::DailyLimitExhausted => e
+  # Our own configured day bucket ran out, before a request went out.
+  BackoffRecord.set(e.tenant_id, seconds: e.retry_after, source: :configured)
+  raise
+end
+```
+
+Your readers then hit that record directly and never need a `Client`.
+
+Three things worth knowing when you write this:
+
+- **Only surfaced errors are worth recording.** The retry middleware absorbs
+  transient 429s internally and succeeds, so those never reach your rescue —
+  which is what you want. Recording every observed 429 would flash "paused"
+  at users for something already handled.
+- **`retry_after` can be nil.** Xero usually sends `Retry-After` but isn't
+  guaranteed to, so pick your own fallback. Kiwi doesn't invent one, because
+  the number you choose is the number your users see as the pause duration —
+  that's your decision, not the library's. Watch for a zero-second TTL if
+  you store this in Redis; `SETEX` rejects it.
+- **`tenant_id` is on the exception for a reason.** Don't infer it from
+  surrounding context — that breaks the moment one client serves more than
+  one tenant.
+
+`XeroKiwi::Throttle::Timeout` carries `tenant_id` and `retry_after` too, if
+you want to treat a per-minute wait that exceeded `max_wait` as a back-off
+rather than as an error.
+
 ## What Xero Kiwi does automatically
 
 Xero Kiwi sets up a `faraday-retry` middleware that handles transient failures

data/docs/throttling.md

@@ -132,8 +132,9 @@ is fine. If the wait would exceed `max_wait`, it raises
 probably too many concurrent workers for the configured `per_minute`.
 
 **Per-day:** the limiter raises `XeroKiwi::Throttle::DailyLimitExhausted`
-immediately, with a `retry_after` attribute in seconds. Sleeping for hours is
-never the right move in a Sidekiq worker, so the caller has to decide:
+immediately, with `retry_after` in seconds and the `tenant_id` it relates to.
+Sleeping for hours is never the right move in a Sidekiq worker, so the caller
+has to decide:
 
 ```ruby
 begin
@@ -145,8 +146,17 @@ rescue XeroKiwi::Throttle::DailyLimitExhausted => e
 end
 ```
 
-This mirrors the `XeroKiwi::RateLimitError` shape that the retry layer raises
-after exhausting retries on a 429, so the handling code is familiar.
+Both throttle exceptions carry `tenant_id` and `retry_after`, which mirrors
+the `XeroKiwi::RateLimitError` shape the retry layer raises after exhausting
+retries on a 429 — so one rescue can cover both, and code handling them looks
+the same.
+
+If something outside the sync needs to know a tenant is backed off, see
+[recording a durable back-off
+signal](retries-and-rate-limits.md#recording-a-durable-back-off-signal).
+`#remaining` is not that: it only knows about calls made through this
+limiter, so a tenant another application has rate limited looks healthy to
+your own bucket.
 
 ## Redis key layout
 

data/lib/xero_kiwi/client.rb

@@ -978,7 +978,8 @@ module XeroKiwi
           env.status,
           env.body,
           retry_after: env.response_headers["retry-after"]&.to_f,
-          problem:     env.response_headers["x-rate-limit-problem"]
+          problem:     env.response_headers["x-rate-limit-problem"],
+          tenant_id:   env.request_headers[Throttle::Middleware::TENANT_HEADER]
         )
       end
     end

data/lib/xero_kiwi/errors.rb

@@ -22,12 +22,18 @@ module XeroKiwi
   # should treat this as "the user must re-authorise" and surface accordingly.
   class TokenRefreshError < AuthenticationError; end
 
+  # Raised when Xero returns 429 and the retry middleware has exhausted its
+  # attempts. Carries everything needed to record a durable back-off signal —
+  # which tenant, for how long, and which of Xero's three limits was hit —
+  # without the caller inferring any of it from context. See
+  # docs/retries-and-rate-limits.md for the persistence pattern.
   class RateLimitError < APIError
-    attr_reader :retry_after, :problem
+    attr_reader :retry_after, :problem, :tenant_id
 
-    def initialize(status, body, retry_after: nil, problem: nil)
+    def initialize(status, body, retry_after: nil, problem: nil, tenant_id: nil)
       @retry_after = retry_after
       @problem     = problem
+      @tenant_id   = tenant_id
       super(status, body, "Xero rate limit hit (#{problem || "unknown"}); retry after #{retry_after}s")
     end
   end

data/lib/xero_kiwi/throttle/redis_token_bucket.rb

@@ -133,7 +133,7 @@ module XeroKiwi
           failed, wait_ms = evaluate(key)
           return if failed.zero?
 
-          waited_ms = handle_failure(failed, wait_ms, waited_ms)
+          waited_ms = handle_failure(failed, wait_ms, waited_ms, key)
         end
       rescue Redis::BaseError => e
         log_redis_failure(e)
@@ -158,17 +158,24 @@ module XeroKiwi
 
       private
 
-      def handle_failure(failed, wait_ms, waited_ms)
+      # `key` is threaded down purely so the exceptions can name the tenant
+      # they relate to. A caller recording a durable back-off signal needs it,
+      # and inferring it from surrounding context breaks the moment one client
+      # serves more than one tenant.
+      def handle_failure(failed, wait_ms, waited_ms, key)
         case failed
-        when 1 then wait_for_minute_bucket(wait_ms, waited_ms)
-        when 2 then raise Throttle::DailyLimitExhausted.new(retry_after: wait_ms / 1000.0)
+        when 1 then wait_for_minute_bucket(wait_ms, waited_ms, key)
+        when 2 then raise Throttle::DailyLimitExhausted.new(retry_after: wait_ms / 1000.0, tenant_id: key)
         end
       end
 
-      def wait_for_minute_bucket(wait_ms, waited_ms)
+      def wait_for_minute_bucket(wait_ms, waited_ms, key)
         if (waited_ms + wait_ms) / 1000.0 > @max_wait
-          raise Throttle::Timeout,
-                "waited #{(waited_ms / 1000.0).round(2)}s for rate-limit token, exceeds max_wait=#{@max_wait}s"
+          raise Throttle::Timeout.new(
+            "waited #{(waited_ms / 1000.0).round(2)}s for rate-limit token, exceeds max_wait=#{@max_wait}s",
+            tenant_id:   key,
+            retry_after: wait_ms / 1000.0
+          )
         end
 
         @sleeper.call((wait_ms + POLL_MS) / 1000.0)

data/lib/xero_kiwi/throttle.rb

@@ -13,17 +13,31 @@ module XeroKiwi
     # waited longer than the limiter's configured max_wait. The right fix is
     # usually to slow the caller down or raise headroom; swallowing this
     # quietly tends to hide the problem.
-    class Timeout < Error; end
+    #
+    # Carries `tenant_id` and `retry_after` so a caller recording a durable
+    # back-off signal has both facts without inferring either from context.
+    class Timeout < Error
+      attr_reader :tenant_id, :retry_after
+
+      # `message` stays positional and first so `raise Throttle::Timeout, "…"`
+      # keeps working for anyone who was doing that.
+      def initialize(message = nil, tenant_id: nil, retry_after: nil)
+        @tenant_id   = tenant_id
+        @retry_after = retry_after
+        super(message || "timed out waiting for a rate-limit token")
+      end
+    end
 
     # Raised immediately (no sleep) when the per-day bucket is exhausted. The
     # wait until reset is typically measured in hours, so blocking the caller
     # is the wrong move — re-enqueue the job at `retry_after` instead. Shape
     # mirrors RateLimitError so existing Xero rate-limit handling applies.
     class DailyLimitExhausted < Error
-      attr_reader :retry_after
+      attr_reader :retry_after, :tenant_id
 
-      def initialize(retry_after:)
+      def initialize(retry_after:, tenant_id: nil)
         @retry_after = retry_after
+        @tenant_id   = tenant_id
         super("Xero daily rate limit exhausted; retry in #{retry_after.round}s")
       end
     end

data/lib/xero_kiwi/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module XeroKiwi
-  VERSION = "0.6.0"
+  VERSION = "0.7.0"
 end

data/llms-full.txt

@@ -3847,6 +3847,59 @@ When neither source has a reading, `day_remaining` is nil and
 reason to halt a sync, and halting would break the common case where the
 first call is what populates the figures.
 
+**`client.rate_limit` is process-local.** The reported figures live in
+memory on the `Client` that made the request, so a Sidekiq worker's reading
+is gone when the job ends and no other process can see it. That's fine for
+"should this loop stop early?", which is what it's for. It is not a way to
+tell a web request that a tenant is currently backed off — for that, see
+below.
+
+## Recording a durable back-off signal
+
+If something outside your sync needs to know a tenant is backed off — a
+dashboard badge, an API response carrying its own `Retry-After`, a guard
+that refuses to start another sync — you need a record that outlives the
+process that observed it.
+
+Kiwi deliberately doesn't keep that record for you, but it gives you
+everything needed to keep one. Both back-off exceptions name the tenant,
+say how long, and say which limit was hit:
+
+```ruby
+begin
+  client.each_invoice_page(tenant_id) { |page| import(page) }
+rescue XeroKiwi::RateLimitError => e
+  # Xero said so. e.problem is "minute" / "day" / "appminute".
+  BackoffRecord.set(e.tenant_id, seconds: e.retry_after || 60, source: :reported)
+  raise
+rescue XeroKiwi::Throttle::DailyLimitExhausted => e
+  # Our own configured day bucket ran out, before a request went out.
+  BackoffRecord.set(e.tenant_id, seconds: e.retry_after, source: :configured)
+  raise
+end
+```
+
+Your readers then hit that record directly and never need a `Client`.
+
+Three things worth knowing when you write this:
+
+- **Only surfaced errors are worth recording.** The retry middleware absorbs
+  transient 429s internally and succeeds, so those never reach your rescue —
+  which is what you want. Recording every observed 429 would flash "paused"
+  at users for something already handled.
+- **`retry_after` can be nil.** Xero usually sends `Retry-After` but isn't
+  guaranteed to, so pick your own fallback. Kiwi doesn't invent one, because
+  the number you choose is the number your users see as the pause duration —
+  that's your decision, not the library's. Watch for a zero-second TTL if
+  you store this in Redis; `SETEX` rejects it.
+- **`tenant_id` is on the exception for a reason.** Don't infer it from
+  surrounding context — that breaks the moment one client serves more than
+  one tenant.
+
+`XeroKiwi::Throttle::Timeout` carries `tenant_id` and `retry_after` too, if
+you want to treat a per-minute wait that exceeded `max_wait` as a back-off
+rather than as an error.
+
 ## What Xero Kiwi does automatically
 
 Xero Kiwi sets up a `faraday-retry` middleware that handles transient failures
@@ -4191,8 +4244,9 @@ is fine. If the wait would exceed `max_wait`, it raises
 probably too many concurrent workers for the configured `per_minute`.
 
 **Per-day:** the limiter raises `XeroKiwi::Throttle::DailyLimitExhausted`
-immediately, with a `retry_after` attribute in seconds. Sleeping for hours is
-never the right move in a Sidekiq worker, so the caller has to decide:
+immediately, with `retry_after` in seconds and the `tenant_id` it relates to.
+Sleeping for hours is never the right move in a Sidekiq worker, so the caller
+has to decide:
 
 ```ruby
 begin
@@ -4204,8 +4258,17 @@ rescue XeroKiwi::Throttle::DailyLimitExhausted => e
 end
 ```
 
-This mirrors the `XeroKiwi::RateLimitError` shape that the retry layer raises
-after exhausting retries on a 429, so the handling code is familiar.
+Both throttle exceptions carry `tenant_id` and `retry_after`, which mirrors
+the `XeroKiwi::RateLimitError` shape the retry layer raises after exhausting
+retries on a 429 — so one rescue can cover both, and code handling them looks
+the same.
+
+If something outside the sync needs to know a tenant is backed off, see
+[recording a durable back-off
+signal](retries-and-rate-limits.md#recording-a-durable-back-off-signal).
+`#remaining` is not that: it only knows about calls made through this
+limiter, so a tenant another application has rate limited looks healthy to
+your own bucket.
 
 ## Redis key layout
 

data/llms.txt

@@ -65,6 +65,6 @@ XeroKiwi requires Ruby 3.4.1 or newer. Install with `gem "xero-kiwi"`.
 ## Optional
 
 - [README](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/README.md): high-level overview, installation, the documentation table of contents, and the development and release process
-- [Retries and rate limits](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/retries-and-rate-limits.md): how XeroKiwi handles 429s and transient 5xxs, `client.rate_limit(tenant_id)` for remaining quota, tuning the retry policy, the Faraday middleware ordering, why 500 is deliberately not retried
+- [Retries and rate limits](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/retries-and-rate-limits.md): how XeroKiwi handles 429s and transient 5xxs, `client.rate_limit(tenant_id)` for remaining quota (process-local), recording a durable per-tenant back-off signal from the `tenant_id` and `retry_after` carried by RateLimitError and the throttle errors, tuning the retry policy, the Faraday middleware ordering, why 500 is deliberately not retried
 - [Throttling](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/throttling.md): the Redis-backed per-tenant token bucket for coordinating rate limits across processes, `default_throttle`, `#remaining`, fail-open behaviour, writing a custom limiter
 - [CHANGELOG](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/CHANGELOG.md): version history

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: xero-kiwi
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Douglas Greyling