xero-kiwi 0.5.2 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +25 -0
- data/README.md +36 -0
- data/docs/accounting/credit-note.md +1 -1
- data/docs/accounting/overpayment.md +1 -1
- data/docs/accounting/prepayment.md +1 -1
- data/docs/retries-and-rate-limits.md +53 -0
- data/docs/throttling.md +14 -4
- data/lib/xero_kiwi/accounting/allocation.rb +19 -5
- data/lib/xero_kiwi/client.rb +2 -1
- data/lib/xero_kiwi/errors.rb +8 -2
- data/lib/xero_kiwi/throttle/redis_token_bucket.rb +14 -7
- data/lib/xero_kiwi/throttle.rb +17 -3
- data/lib/xero_kiwi/version.rb +1 -1
- data/llms-full.txt +106 -7
- data/llms.txt +2 -2
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e0e650e95ecce31d4acf75d1123d504bc11cacc9725e838643b6fbae235b04c0
|
|
4
|
+
data.tar.gz: 950f370a803af1b1f6aacf87bcaadbf5d00f2d44b1cb54061bee0d74b70e5eae
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: dbf4f32d568839238ce077d0243fe59ad0ba54f8e332b0870c6f23b53c63425f9a41b7842491d862e6bced6ad8bb14db71b1af5b83e704d1833e53b98a7a5ced
|
|
7
|
+
data.tar.gz: 32e1d56e608bc844a452ca9a3a20ffff984c5a64ab1a46dc896498c3543c7deb9c581c5c1fc821ea40cff124b0d5e3ead035fc70e8a6857cce97754ef9131b90
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,30 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.7.0] - 2026-09-27
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- `RateLimitError`, `Throttle::DailyLimitExhausted` and `Throttle::Timeout` now carry the **`tenant_id`** they relate to, and `Throttle::Timeout` also carries **`retry_after`** (it previously carried nothing at all). A caller recording a durable back-off signal — "this tenant is paused until T, because X" — now gets every fact from the exception instead of inferring the tenant from surrounding context, which breaks as soon as one client serves more than one tenant. `docs/retries-and-rate-limits.md` gains a section on the pattern.
|
|
8
|
+
- Documented that `client.rate_limit` is **process-local**. The reported figures live in memory on the `Client` that made the request, so a worker's reading dies with the job. That is fine for "should this loop stop early?", which is what it is for, and is not a way to tell a web request that a tenant is backed off.
|
|
9
|
+
|
|
10
|
+
## [0.6.0] - 2026-09-27
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **`Accounting::Allocation#amount` always returned nil.** It was mapped to Xero's `"Amount"`, which is the key for the allocation *request* body. Every response — and allocations only ever reach this gem embedded in a CreditNote, Prepayment or Overpayment response — carries `"AppliedAmount"`. The allocated value is now modelled as **`applied_amount`**, and `#amount` is an alias of it so callers who reached for the obvious name get the value they meant. Nothing that depended on a real value is affected, because there was never one to depend on.
|
|
15
|
+
|
|
16
|
+
This mattered more than a missing field usually would: `allocations.map(&:amount)` returned an array of nils rather than raising, so an importer would have written zeroes into every allocated amount with nothing failing anywhere. Reported by a consumer who checked stored payloads before trusting the typed path.
|
|
17
|
+
|
|
18
|
+
Four spec files had fabricated `"Amount"` payloads and so agreed with the bug instead of catching it; all now use the real response shape.
|
|
19
|
+
|
|
20
|
+
### Breaking
|
|
21
|
+
|
|
22
|
+
- `Allocation#to_h` now carries `:applied_amount` instead of `:amount`, and `:amount` is no longer a declared attribute (it is a plain reader, so one value maps to one key). Anyone reading `to_h[:amount]` was reading nil.
|
|
23
|
+
|
|
24
|
+
### Documentation
|
|
25
|
+
|
|
26
|
+
- The README now documents the release process. It covered running specs but said nothing about cutting a release, so the sequencing lived only in one person's head. It leads with the rule and the reason — `rake release` tags whatever is checked out, so a version bump riding along in a pull request gets orphaned the moment that PR is squash-merged, which is what happened to v0.5.2 — and records the thing that is not guessable from the repo: `docs/` ships inside the gem, so a documentation-only fix still needs a patch release to reach anyone.
|
|
27
|
+
|
|
3
28
|
## [0.5.2] - 2026-09-27
|
|
4
29
|
|
|
5
30
|
### Documentation
|
data/README.md
CHANGED
|
@@ -113,6 +113,42 @@ default `redis://127.0.0.1:6379/15`.
|
|
|
113
113
|
docker compose down # when you're done
|
|
114
114
|
```
|
|
115
115
|
|
|
116
|
+
### Releasing
|
|
117
|
+
|
|
118
|
+
**Merge first, then bump.** `rake release` tags whatever is checked out, so a
|
|
119
|
+
version bump that rides along inside a pull request gets orphaned the moment
|
|
120
|
+
that PR is squash-merged — the squash creates a new commit and the tag is left
|
|
121
|
+
pointing at the pre-merge branch commit, unreachable from `main`. Bumping after
|
|
122
|
+
the merge makes that impossible regardless of merge strategy.
|
|
123
|
+
|
|
124
|
+
So: land the work through a PR as normal, then
|
|
125
|
+
|
|
126
|
+
```sh
|
|
127
|
+
git checkout main && git pull
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
1. Bump `VERSION` in [lib/xero_kiwi/version.rb](lib/xero_kiwi/version.rb).
|
|
131
|
+
2. Move the `[Unreleased]` entries in [CHANGELOG.md](CHANGELOG.md) under a
|
|
132
|
+
dated version heading.
|
|
133
|
+
3. `bundle install` to refresh the lockfile's `xero-kiwi (x.y.z)`.
|
|
134
|
+
4. `bundle exec rake` — specs, RuboCop, and the `llms:check` freshness guard.
|
|
135
|
+
5. Commit to `main`. A commit-msg hook requires the `crimesjohnson:` prefix
|
|
136
|
+
there, so: `crimesjohnson: release x.y.z with <the headline>`.
|
|
137
|
+
|
|
138
|
+
```sh
|
|
139
|
+
bundle exec rake release # builds, tags, pushes the tag, publishes
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
The gemspec sets `rubygems_mfa_required`, so this prompts for an OTP and can't
|
|
143
|
+
run unattended. Finally, cut the GitHub release from the changelog section:
|
|
144
|
+
|
|
145
|
+
```sh
|
|
146
|
+
gh release create vx.y.z --title "vx.y.z — <headline>" --notes-file <(…) --verify-tag --latest
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
Note that `docs/` ships inside the gem, so a documentation-only fix still needs
|
|
150
|
+
a patch release to reach anyone — it won't arrive through GitHub alone.
|
|
151
|
+
|
|
116
152
|
## Contributing
|
|
117
153
|
|
|
118
154
|
Bug reports and pull requests are welcome on GitHub at
|
|
@@ -61,7 +61,7 @@ fields Xero returns:
|
|
|
61
61
|
| `reference` | `String` | Additional reference number (ACCRECCREDIT only). |
|
|
62
62
|
| `sent_to_contact` | `Boolean` | Whether the credit note has been sent to the contact. |
|
|
63
63
|
| `remaining_credit` | `Numeric` | The remaining credit balance. |
|
|
64
|
-
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each
|
|
64
|
+
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
|
|
65
65
|
| `branding_theme_id` | `String` | The branding theme ID applied to the credit note. |
|
|
66
66
|
| `has_attachments` | `Boolean` | Whether the credit note has attachments. |
|
|
67
67
|
|
|
@@ -57,7 +57,7 @@ fields Xero returns:
|
|
|
57
57
|
| `currency_code` | `String` | The currency code (e.g. `"NZD"`). |
|
|
58
58
|
| `currency_rate` | `String` | The currency rate (1.0 for base currency). |
|
|
59
59
|
| `remaining_credit` | `String` | The remaining credit balance. |
|
|
60
|
-
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each
|
|
60
|
+
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
|
|
61
61
|
| `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
|
|
62
62
|
| `has_attachments` | `Boolean` | Whether the overpayment has attachments. |
|
|
63
63
|
| `reference` | `String` | Reference for the overpayment. |
|
|
@@ -58,7 +58,7 @@ fields Xero returns:
|
|
|
58
58
|
| `currency_rate` | `String` | The currency rate (1.0 for base currency). |
|
|
59
59
|
| `invoice_number` | `String` | The invoice number (for receive prepayments only). |
|
|
60
60
|
| `remaining_credit` | `String` | The remaining credit balance. |
|
|
61
|
-
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each
|
|
61
|
+
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
|
|
62
62
|
| `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
|
|
63
63
|
| `has_attachments` | `Boolean` | Whether the prepayment has attachments. |
|
|
64
64
|
| `fully_paid_on_date` | `Time` | When the prepayment was fully allocated, parsed as UTC. |
|
|
@@ -68,6 +68,59 @@ When neither source has a reading, `day_remaining` is nil and
|
|
|
68
68
|
reason to halt a sync, and halting would break the common case where the
|
|
69
69
|
first call is what populates the figures.
|
|
70
70
|
|
|
71
|
+
**`client.rate_limit` is process-local.** The reported figures live in
|
|
72
|
+
memory on the `Client` that made the request, so a Sidekiq worker's reading
|
|
73
|
+
is gone when the job ends and no other process can see it. That's fine for
|
|
74
|
+
"should this loop stop early?", which is what it's for. It is not a way to
|
|
75
|
+
tell a web request that a tenant is currently backed off — for that, see
|
|
76
|
+
below.
|
|
77
|
+
|
|
78
|
+
## Recording a durable back-off signal
|
|
79
|
+
|
|
80
|
+
If something outside your sync needs to know a tenant is backed off — a
|
|
81
|
+
dashboard badge, an API response carrying its own `Retry-After`, a guard
|
|
82
|
+
that refuses to start another sync — you need a record that outlives the
|
|
83
|
+
process that observed it.
|
|
84
|
+
|
|
85
|
+
Kiwi deliberately doesn't keep that record for you, but it gives you
|
|
86
|
+
everything needed to keep one. Both back-off exceptions name the tenant,
|
|
87
|
+
say how long, and say which limit was hit:
|
|
88
|
+
|
|
89
|
+
```ruby
|
|
90
|
+
begin
|
|
91
|
+
client.each_invoice_page(tenant_id) { |page| import(page) }
|
|
92
|
+
rescue XeroKiwi::RateLimitError => e
|
|
93
|
+
# Xero said so. e.problem is "minute" / "day" / "appminute".
|
|
94
|
+
BackoffRecord.set(e.tenant_id, seconds: e.retry_after || 60, source: :reported)
|
|
95
|
+
raise
|
|
96
|
+
rescue XeroKiwi::Throttle::DailyLimitExhausted => e
|
|
97
|
+
# Our own configured day bucket ran out, before a request went out.
|
|
98
|
+
BackoffRecord.set(e.tenant_id, seconds: e.retry_after, source: :configured)
|
|
99
|
+
raise
|
|
100
|
+
end
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Your readers then hit that record directly and never need a `Client`.
|
|
104
|
+
|
|
105
|
+
Three things worth knowing when you write this:
|
|
106
|
+
|
|
107
|
+
- **Only surfaced errors are worth recording.** The retry middleware absorbs
|
|
108
|
+
transient 429s internally and succeeds, so those never reach your rescue —
|
|
109
|
+
which is what you want. Recording every observed 429 would flash "paused"
|
|
110
|
+
at users for something already handled.
|
|
111
|
+
- **`retry_after` can be nil.** Xero usually sends `Retry-After` but isn't
|
|
112
|
+
guaranteed to, so pick your own fallback. Kiwi doesn't invent one, because
|
|
113
|
+
the number you choose is the number your users see as the pause duration —
|
|
114
|
+
that's your decision, not the library's. Watch for a zero-second TTL if
|
|
115
|
+
you store this in Redis; `SETEX` rejects it.
|
|
116
|
+
- **`tenant_id` is on the exception for a reason.** Don't infer it from
|
|
117
|
+
surrounding context — that breaks the moment one client serves more than
|
|
118
|
+
one tenant.
|
|
119
|
+
|
|
120
|
+
`XeroKiwi::Throttle::Timeout` carries `tenant_id` and `retry_after` too, if
|
|
121
|
+
you want to treat a per-minute wait that exceeded `max_wait` as a back-off
|
|
122
|
+
rather than as an error.
|
|
123
|
+
|
|
71
124
|
## What Xero Kiwi does automatically
|
|
72
125
|
|
|
73
126
|
Xero Kiwi sets up a `faraday-retry` middleware that handles transient failures
|
data/docs/throttling.md
CHANGED
|
@@ -132,8 +132,9 @@ is fine. If the wait would exceed `max_wait`, it raises
|
|
|
132
132
|
probably too many concurrent workers for the configured `per_minute`.
|
|
133
133
|
|
|
134
134
|
**Per-day:** the limiter raises `XeroKiwi::Throttle::DailyLimitExhausted`
|
|
135
|
-
immediately, with
|
|
136
|
-
never the right move in a Sidekiq worker, so the caller
|
|
135
|
+
immediately, with `retry_after` in seconds and the `tenant_id` it relates to.
|
|
136
|
+
Sleeping for hours is never the right move in a Sidekiq worker, so the caller
|
|
137
|
+
has to decide:
|
|
137
138
|
|
|
138
139
|
```ruby
|
|
139
140
|
begin
|
|
@@ -145,8 +146,17 @@ rescue XeroKiwi::Throttle::DailyLimitExhausted => e
|
|
|
145
146
|
end
|
|
146
147
|
```
|
|
147
148
|
|
|
148
|
-
|
|
149
|
-
|
|
149
|
+
Both throttle exceptions carry `tenant_id` and `retry_after`, which mirrors
|
|
150
|
+
the `XeroKiwi::RateLimitError` shape the retry layer raises after exhausting
|
|
151
|
+
retries on a 429 — so one rescue can cover both, and code handling them looks
|
|
152
|
+
the same.
|
|
153
|
+
|
|
154
|
+
If something outside the sync needs to know a tenant is backed off, see
|
|
155
|
+
[recording a durable back-off
|
|
156
|
+
signal](retries-and-rate-limits.md#recording-a-durable-back-off-signal).
|
|
157
|
+
`#remaining` is not that: it only knows about calls made through this
|
|
158
|
+
limiter, so a tenant another application has rate limited looks healthy to
|
|
159
|
+
your own bucket.
|
|
150
160
|
|
|
151
161
|
## Redis key layout
|
|
152
162
|
|
|
@@ -5,17 +5,31 @@ module XeroKiwi
|
|
|
5
5
|
# Represents an allocation of a credit note, prepayment, or overpayment
|
|
6
6
|
# against an invoice.
|
|
7
7
|
#
|
|
8
|
+
# Xero names the allocated value differently depending on direction: the
|
|
9
|
+
# allocation *request* body takes "Amount", while every *response* — and
|
|
10
|
+
# allocations only ever reach this gem embedded in a CreditNote,
|
|
11
|
+
# Prepayment or Overpayment response — carries "AppliedAmount". This
|
|
12
|
+
# class was originally modelled off the request shape, so `amount`
|
|
13
|
+
# silently returned nil for every allocation the gem could actually
|
|
14
|
+
# produce. `#amount` is now an alias of `#applied_amount` so callers who
|
|
15
|
+
# reached for the obvious name get the value they meant.
|
|
16
|
+
#
|
|
8
17
|
# See: https://developer.xero.com/documentation/api/accounting/overpayments
|
|
9
18
|
class Allocation
|
|
10
19
|
include Resource
|
|
11
20
|
|
|
12
21
|
identity :allocation_id
|
|
13
22
|
|
|
14
|
-
attribute :allocation_id,
|
|
15
|
-
attribute :
|
|
16
|
-
attribute :date,
|
|
17
|
-
attribute :invoice,
|
|
18
|
-
attribute :is_deleted,
|
|
23
|
+
attribute :allocation_id, xero: "AllocationID", type: :guid
|
|
24
|
+
attribute :applied_amount, xero: "AppliedAmount", type: :decimal
|
|
25
|
+
attribute :date, xero: "Date", type: :date
|
|
26
|
+
attribute :invoice, xero: "Invoice", type: :object, of: Invoice, reference: true
|
|
27
|
+
attribute :is_deleted, xero: "IsDeleted", type: :bool
|
|
28
|
+
|
|
29
|
+
# Kept for callers written against the pre-0.6.0 attribute name. It
|
|
30
|
+
# could only ever have returned nil, so nothing that relied on a real
|
|
31
|
+
# value is affected.
|
|
32
|
+
def amount = applied_amount
|
|
19
33
|
end
|
|
20
34
|
end
|
|
21
35
|
end
|
data/lib/xero_kiwi/client.rb
CHANGED
|
@@ -978,7 +978,8 @@ module XeroKiwi
|
|
|
978
978
|
env.status,
|
|
979
979
|
env.body,
|
|
980
980
|
retry_after: env.response_headers["retry-after"]&.to_f,
|
|
981
|
-
problem: env.response_headers["x-rate-limit-problem"]
|
|
981
|
+
problem: env.response_headers["x-rate-limit-problem"],
|
|
982
|
+
tenant_id: env.request_headers[Throttle::Middleware::TENANT_HEADER]
|
|
982
983
|
)
|
|
983
984
|
end
|
|
984
985
|
end
|
data/lib/xero_kiwi/errors.rb
CHANGED
|
@@ -22,12 +22,18 @@ module XeroKiwi
|
|
|
22
22
|
# should treat this as "the user must re-authorise" and surface accordingly.
|
|
23
23
|
class TokenRefreshError < AuthenticationError; end
|
|
24
24
|
|
|
25
|
+
# Raised when Xero returns 429 and the retry middleware has exhausted its
|
|
26
|
+
# attempts. Carries everything needed to record a durable back-off signal —
|
|
27
|
+
# which tenant, for how long, and which of Xero's three limits was hit —
|
|
28
|
+
# without the caller inferring any of it from context. See
|
|
29
|
+
# docs/retries-and-rate-limits.md for the persistence pattern.
|
|
25
30
|
class RateLimitError < APIError
|
|
26
|
-
attr_reader :retry_after, :problem
|
|
31
|
+
attr_reader :retry_after, :problem, :tenant_id
|
|
27
32
|
|
|
28
|
-
def initialize(status, body, retry_after: nil, problem: nil)
|
|
33
|
+
def initialize(status, body, retry_after: nil, problem: nil, tenant_id: nil)
|
|
29
34
|
@retry_after = retry_after
|
|
30
35
|
@problem = problem
|
|
36
|
+
@tenant_id = tenant_id
|
|
31
37
|
super(status, body, "Xero rate limit hit (#{problem || "unknown"}); retry after #{retry_after}s")
|
|
32
38
|
end
|
|
33
39
|
end
|
|
@@ -133,7 +133,7 @@ module XeroKiwi
|
|
|
133
133
|
failed, wait_ms = evaluate(key)
|
|
134
134
|
return if failed.zero?
|
|
135
135
|
|
|
136
|
-
waited_ms = handle_failure(failed, wait_ms, waited_ms)
|
|
136
|
+
waited_ms = handle_failure(failed, wait_ms, waited_ms, key)
|
|
137
137
|
end
|
|
138
138
|
rescue Redis::BaseError => e
|
|
139
139
|
log_redis_failure(e)
|
|
@@ -158,17 +158,24 @@ module XeroKiwi
|
|
|
158
158
|
|
|
159
159
|
private
|
|
160
160
|
|
|
161
|
-
|
|
161
|
+
# `key` is threaded down purely so the exceptions can name the tenant
|
|
162
|
+
# they relate to. A caller recording a durable back-off signal needs it,
|
|
163
|
+
# and inferring it from surrounding context breaks the moment one client
|
|
164
|
+
# serves more than one tenant.
|
|
165
|
+
def handle_failure(failed, wait_ms, waited_ms, key)
|
|
162
166
|
case failed
|
|
163
|
-
when 1 then wait_for_minute_bucket(wait_ms, waited_ms)
|
|
164
|
-
when 2 then raise Throttle::DailyLimitExhausted.new(retry_after: wait_ms / 1000.0)
|
|
167
|
+
when 1 then wait_for_minute_bucket(wait_ms, waited_ms, key)
|
|
168
|
+
when 2 then raise Throttle::DailyLimitExhausted.new(retry_after: wait_ms / 1000.0, tenant_id: key)
|
|
165
169
|
end
|
|
166
170
|
end
|
|
167
171
|
|
|
168
|
-
def wait_for_minute_bucket(wait_ms, waited_ms)
|
|
172
|
+
def wait_for_minute_bucket(wait_ms, waited_ms, key)
|
|
169
173
|
if (waited_ms + wait_ms) / 1000.0 > @max_wait
|
|
170
|
-
raise Throttle::Timeout
|
|
171
|
-
|
|
174
|
+
raise Throttle::Timeout.new(
|
|
175
|
+
"waited #{(waited_ms / 1000.0).round(2)}s for rate-limit token, exceeds max_wait=#{@max_wait}s",
|
|
176
|
+
tenant_id: key,
|
|
177
|
+
retry_after: wait_ms / 1000.0
|
|
178
|
+
)
|
|
172
179
|
end
|
|
173
180
|
|
|
174
181
|
@sleeper.call((wait_ms + POLL_MS) / 1000.0)
|
data/lib/xero_kiwi/throttle.rb
CHANGED
|
@@ -13,17 +13,31 @@ module XeroKiwi
|
|
|
13
13
|
# waited longer than the limiter's configured max_wait. The right fix is
|
|
14
14
|
# usually to slow the caller down or raise headroom; swallowing this
|
|
15
15
|
# quietly tends to hide the problem.
|
|
16
|
-
|
|
16
|
+
#
|
|
17
|
+
# Carries `tenant_id` and `retry_after` so a caller recording a durable
|
|
18
|
+
# back-off signal has both facts without inferring either from context.
|
|
19
|
+
class Timeout < Error
|
|
20
|
+
attr_reader :tenant_id, :retry_after
|
|
21
|
+
|
|
22
|
+
# `message` stays positional and first so `raise Throttle::Timeout, "…"`
|
|
23
|
+
# keeps working for anyone who was doing that.
|
|
24
|
+
def initialize(message = nil, tenant_id: nil, retry_after: nil)
|
|
25
|
+
@tenant_id = tenant_id
|
|
26
|
+
@retry_after = retry_after
|
|
27
|
+
super(message || "timed out waiting for a rate-limit token")
|
|
28
|
+
end
|
|
29
|
+
end
|
|
17
30
|
|
|
18
31
|
# Raised immediately (no sleep) when the per-day bucket is exhausted. The
|
|
19
32
|
# wait until reset is typically measured in hours, so blocking the caller
|
|
20
33
|
# is the wrong move — re-enqueue the job at `retry_after` instead. Shape
|
|
21
34
|
# mirrors RateLimitError so existing Xero rate-limit handling applies.
|
|
22
35
|
class DailyLimitExhausted < Error
|
|
23
|
-
attr_reader :retry_after
|
|
36
|
+
attr_reader :retry_after, :tenant_id
|
|
24
37
|
|
|
25
|
-
def initialize(retry_after:)
|
|
38
|
+
def initialize(retry_after:, tenant_id: nil)
|
|
26
39
|
@retry_after = retry_after
|
|
40
|
+
@tenant_id = tenant_id
|
|
27
41
|
super("Xero daily rate limit exhausted; retry in #{retry_after.round}s")
|
|
28
42
|
end
|
|
29
43
|
end
|
data/lib/xero_kiwi/version.rb
CHANGED
data/llms-full.txt
CHANGED
|
@@ -126,6 +126,42 @@ default `redis://127.0.0.1:6379/15`.
|
|
|
126
126
|
docker compose down # when you're done
|
|
127
127
|
```
|
|
128
128
|
|
|
129
|
+
### Releasing
|
|
130
|
+
|
|
131
|
+
**Merge first, then bump.** `rake release` tags whatever is checked out, so a
|
|
132
|
+
version bump that rides along inside a pull request gets orphaned the moment
|
|
133
|
+
that PR is squash-merged — the squash creates a new commit and the tag is left
|
|
134
|
+
pointing at the pre-merge branch commit, unreachable from `main`. Bumping after
|
|
135
|
+
the merge makes that impossible regardless of merge strategy.
|
|
136
|
+
|
|
137
|
+
So: land the work through a PR as normal, then
|
|
138
|
+
|
|
139
|
+
```sh
|
|
140
|
+
git checkout main && git pull
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
1. Bump `VERSION` in [lib/xero_kiwi/version.rb](lib/xero_kiwi/version.rb).
|
|
144
|
+
2. Move the `[Unreleased]` entries in [CHANGELOG.md](CHANGELOG.md) under a
|
|
145
|
+
dated version heading.
|
|
146
|
+
3. `bundle install` to refresh the lockfile's `xero-kiwi (x.y.z)`.
|
|
147
|
+
4. `bundle exec rake` — specs, RuboCop, and the `llms:check` freshness guard.
|
|
148
|
+
5. Commit to `main`. A commit-msg hook requires the `crimesjohnson:` prefix
|
|
149
|
+
there, so: `crimesjohnson: release x.y.z with <the headline>`.
|
|
150
|
+
|
|
151
|
+
```sh
|
|
152
|
+
bundle exec rake release # builds, tags, pushes the tag, publishes
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
The gemspec sets `rubygems_mfa_required`, so this prompts for an OTP and can't
|
|
156
|
+
run unattended. Finally, cut the GitHub release from the changelog section:
|
|
157
|
+
|
|
158
|
+
```sh
|
|
159
|
+
gh release create vx.y.z --title "vx.y.z — <headline>" --notes-file <(…) --verify-tag --latest
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Note that `docs/` ships inside the gem, so a documentation-only fix still needs
|
|
163
|
+
a patch release to reach anyone — it won't arrive through GitHub alone.
|
|
164
|
+
|
|
129
165
|
## Contributing
|
|
130
166
|
|
|
131
167
|
Bug reports and pull requests are welcome on GitHub at
|
|
@@ -2494,7 +2530,7 @@ fields Xero returns:
|
|
|
2494
2530
|
| `reference` | `String` | Additional reference number (ACCRECCREDIT only). |
|
|
2495
2531
|
| `sent_to_contact` | `Boolean` | Whether the credit note has been sent to the contact. |
|
|
2496
2532
|
| `remaining_credit` | `Numeric` | The remaining credit balance. |
|
|
2497
|
-
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each
|
|
2533
|
+
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
|
|
2498
2534
|
| `branding_theme_id` | `String` | The branding theme ID applied to the credit note. |
|
|
2499
2535
|
| `has_attachments` | `Boolean` | Whether the credit note has attachments. |
|
|
2500
2536
|
|
|
@@ -2838,7 +2874,7 @@ fields Xero returns:
|
|
|
2838
2874
|
| `currency_code` | `String` | The currency code (e.g. `"NZD"`). |
|
|
2839
2875
|
| `currency_rate` | `String` | The currency rate (1.0 for base currency). |
|
|
2840
2876
|
| `remaining_credit` | `String` | The remaining credit balance. |
|
|
2841
|
-
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each
|
|
2877
|
+
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
|
|
2842
2878
|
| `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
|
|
2843
2879
|
| `has_attachments` | `Boolean` | Whether the overpayment has attachments. |
|
|
2844
2880
|
| `reference` | `String` | Reference for the overpayment. |
|
|
@@ -2939,7 +2975,7 @@ fields Xero returns:
|
|
|
2939
2975
|
| `currency_rate` | `String` | The currency rate (1.0 for base currency). |
|
|
2940
2976
|
| `invoice_number` | `String` | The invoice number (for receive prepayments only). |
|
|
2941
2977
|
| `remaining_credit` | `String` | The remaining credit balance. |
|
|
2942
|
-
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each
|
|
2978
|
+
| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
|
|
2943
2979
|
| `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
|
|
2944
2980
|
| `has_attachments` | `Boolean` | Whether the prepayment has attachments. |
|
|
2945
2981
|
| `fully_paid_on_date` | `Time` | When the prepayment was fully allocated, parsed as UTC. |
|
|
@@ -3811,6 +3847,59 @@ When neither source has a reading, `day_remaining` is nil and
|
|
|
3811
3847
|
reason to halt a sync, and halting would break the common case where the
|
|
3812
3848
|
first call is what populates the figures.
|
|
3813
3849
|
|
|
3850
|
+
**`client.rate_limit` is process-local.** The reported figures live in
|
|
3851
|
+
memory on the `Client` that made the request, so a Sidekiq worker's reading
|
|
3852
|
+
is gone when the job ends and no other process can see it. That's fine for
|
|
3853
|
+
"should this loop stop early?", which is what it's for. It is not a way to
|
|
3854
|
+
tell a web request that a tenant is currently backed off — for that, see
|
|
3855
|
+
below.
|
|
3856
|
+
|
|
3857
|
+
## Recording a durable back-off signal
|
|
3858
|
+
|
|
3859
|
+
If something outside your sync needs to know a tenant is backed off — a
|
|
3860
|
+
dashboard badge, an API response carrying its own `Retry-After`, a guard
|
|
3861
|
+
that refuses to start another sync — you need a record that outlives the
|
|
3862
|
+
process that observed it.
|
|
3863
|
+
|
|
3864
|
+
Kiwi deliberately doesn't keep that record for you, but it gives you
|
|
3865
|
+
everything needed to keep one. Both back-off exceptions name the tenant,
|
|
3866
|
+
say how long, and say which limit was hit:
|
|
3867
|
+
|
|
3868
|
+
```ruby
|
|
3869
|
+
begin
|
|
3870
|
+
client.each_invoice_page(tenant_id) { |page| import(page) }
|
|
3871
|
+
rescue XeroKiwi::RateLimitError => e
|
|
3872
|
+
# Xero said so. e.problem is "minute" / "day" / "appminute".
|
|
3873
|
+
BackoffRecord.set(e.tenant_id, seconds: e.retry_after || 60, source: :reported)
|
|
3874
|
+
raise
|
|
3875
|
+
rescue XeroKiwi::Throttle::DailyLimitExhausted => e
|
|
3876
|
+
# Our own configured day bucket ran out, before a request went out.
|
|
3877
|
+
BackoffRecord.set(e.tenant_id, seconds: e.retry_after, source: :configured)
|
|
3878
|
+
raise
|
|
3879
|
+
end
|
|
3880
|
+
```
|
|
3881
|
+
|
|
3882
|
+
Your readers then hit that record directly and never need a `Client`.
|
|
3883
|
+
|
|
3884
|
+
Three things worth knowing when you write this:
|
|
3885
|
+
|
|
3886
|
+
- **Only surfaced errors are worth recording.** The retry middleware absorbs
|
|
3887
|
+
transient 429s internally and succeeds, so those never reach your rescue —
|
|
3888
|
+
which is what you want. Recording every observed 429 would flash "paused"
|
|
3889
|
+
at users for something already handled.
|
|
3890
|
+
- **`retry_after` can be nil.** Xero usually sends `Retry-After` but isn't
|
|
3891
|
+
guaranteed to, so pick your own fallback. Kiwi doesn't invent one, because
|
|
3892
|
+
the number you choose is the number your users see as the pause duration —
|
|
3893
|
+
that's your decision, not the library's. Watch for a zero-second TTL if
|
|
3894
|
+
you store this in Redis; `SETEX` rejects it.
|
|
3895
|
+
- **`tenant_id` is on the exception for a reason.** Don't infer it from
|
|
3896
|
+
surrounding context — that breaks the moment one client serves more than
|
|
3897
|
+
one tenant.
|
|
3898
|
+
|
|
3899
|
+
`XeroKiwi::Throttle::Timeout` carries `tenant_id` and `retry_after` too, if
|
|
3900
|
+
you want to treat a per-minute wait that exceeded `max_wait` as a back-off
|
|
3901
|
+
rather than as an error.
|
|
3902
|
+
|
|
3814
3903
|
## What Xero Kiwi does automatically
|
|
3815
3904
|
|
|
3816
3905
|
Xero Kiwi sets up a `faraday-retry` middleware that handles transient failures
|
|
@@ -4155,8 +4244,9 @@ is fine. If the wait would exceed `max_wait`, it raises
|
|
|
4155
4244
|
probably too many concurrent workers for the configured `per_minute`.
|
|
4156
4245
|
|
|
4157
4246
|
**Per-day:** the limiter raises `XeroKiwi::Throttle::DailyLimitExhausted`
|
|
4158
|
-
immediately, with
|
|
4159
|
-
never the right move in a Sidekiq worker, so the caller
|
|
4247
|
+
immediately, with `retry_after` in seconds and the `tenant_id` it relates to.
|
|
4248
|
+
Sleeping for hours is never the right move in a Sidekiq worker, so the caller
|
|
4249
|
+
has to decide:
|
|
4160
4250
|
|
|
4161
4251
|
```ruby
|
|
4162
4252
|
begin
|
|
@@ -4168,8 +4258,17 @@ rescue XeroKiwi::Throttle::DailyLimitExhausted => e
|
|
|
4168
4258
|
end
|
|
4169
4259
|
```
|
|
4170
4260
|
|
|
4171
|
-
|
|
4172
|
-
|
|
4261
|
+
Both throttle exceptions carry `tenant_id` and `retry_after`, which mirrors
|
|
4262
|
+
the `XeroKiwi::RateLimitError` shape the retry layer raises after exhausting
|
|
4263
|
+
retries on a 429 — so one rescue can cover both, and code handling them looks
|
|
4264
|
+
the same.
|
|
4265
|
+
|
|
4266
|
+
If something outside the sync needs to know a tenant is backed off, see
|
|
4267
|
+
[recording a durable back-off
|
|
4268
|
+
signal](retries-and-rate-limits.md#recording-a-durable-back-off-signal).
|
|
4269
|
+
`#remaining` is not that: it only knows about calls made through this
|
|
4270
|
+
limiter, so a tenant another application has rate limited looks healthy to
|
|
4271
|
+
your own bucket.
|
|
4173
4272
|
|
|
4174
4273
|
## Redis key layout
|
|
4175
4274
|
|
data/llms.txt
CHANGED
|
@@ -64,7 +64,7 @@ XeroKiwi requires Ruby 3.4.1 or newer. Install with `gem "xero-kiwi"`.
|
|
|
64
64
|
|
|
65
65
|
## Optional
|
|
66
66
|
|
|
67
|
-
- [README](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/README.md): high-level overview, installation,
|
|
68
|
-
- [Retries and rate limits](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/retries-and-rate-limits.md): how XeroKiwi handles 429s and transient 5xxs, `client.rate_limit(tenant_id)` for remaining quota, tuning the retry policy, the Faraday middleware ordering, why 500 is deliberately not retried
|
|
67
|
+
- [README](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/README.md): high-level overview, installation, the documentation table of contents, and the development and release process
|
|
68
|
+
- [Retries and rate limits](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/retries-and-rate-limits.md): how XeroKiwi handles 429s and transient 5xxs, `client.rate_limit(tenant_id)` for remaining quota (process-local), recording a durable per-tenant back-off signal from the `tenant_id` and `retry_after` carried by RateLimitError and the throttle errors, tuning the retry policy, the Faraday middleware ordering, why 500 is deliberately not retried
|
|
69
69
|
- [Throttling](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/throttling.md): the Redis-backed per-tenant token bucket for coordinating rate limits across processes, `default_throttle`, `#remaining`, fail-open behaviour, writing a custom limiter
|
|
70
70
|
- [CHANGELOG](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/CHANGELOG.md): version history
|