xero-kiwi 0.7.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e0e650e95ecce31d4acf75d1123d504bc11cacc9725e838643b6fbae235b04c0
-  data.tar.gz: 950f370a803af1b1f6aacf87bcaadbf5d00f2d44b1cb54061bee0d74b70e5eae
+  metadata.gz: deaef4aaa66b0ac56631e391e34e5747c32da981244477ba212ff664cbe41cab
+  data.tar.gz: 6b3bd18bf430e1967d59875308e73749a13e3c2acc2279c9e58d065af94451db
 SHA512:
-  metadata.gz: dbf4f32d568839238ce077d0243fe59ad0ba54f8e332b0870c6f23b53c63425f9a41b7842491d862e6bced6ad8bb14db71b1af5b83e704d1833e53b98a7a5ced
-  data.tar.gz: 32e1d56e608bc844a452ca9a3a20ffff984c5a64ab1a46dc896498c3543c7deb9c581c5c1fc821ea40cff124b0d5e3ead035fc70e8a6857cce97754ef9131b90
+  metadata.gz: f34e40a3b9774b0891a06ded04d66dd5857ab9b036c5dadd301a1480894e1c00d71c414b0631c563309f4ab74318859d0c37cdfb4beee9ad9526a04b4edeb612
+  data.tar.gz: b10602ef4148ab48c07fffaecf1a7b65b4b828ef037384309e693326bd3e823bbda79a439cf59d2f69cdcdc623385787c84f326793dc2779c85d83f1f4a6a6d8

data/CHANGELOG.md

@@ -1,5 +1,44 @@
 ## [Unreleased]
 
+## [0.8.0] - 2026-09-29
+
+### Breaking
+
+- **Empty strings from Xero now read as `nil` on modelled attributes.** Xero sends `""` for a text field with no value, where its XML representation produced `nil`. Left as `""` it reaches a database column and quietly changes what queries match — a consumer's `where.not(xero_logo_url: nil)` started matching themes with no logo, and was one step from putting a blank image into customer statements.
+
+  Applies to `:string`, `:enum` and `:guid` attributes. `:date` has always behaved this way, so strings doing otherwise was an inconsistency rather than a principle. `:bool` and `:decimal` are untouched — there is no evidence Xero sends `""` for a numeric field, and that would want its own decision.
+
+  Only exactly `""` is affected. `" "` is left alone, because trimming it would be editorialising on a value rather than recognising an absent one. **`raw` is untouched** and still holds `""` verbatim, so the original payload is always recoverable.
+
+  Upgrading: code that chains off a string attribute without a guard (`contact.email_address.downcase`) will now raise `NoMethodError` where it silently operated on `""`. That is the intended trade — silent is the failure mode this release exists to remove.
+
+  Where to actually look: **optional text fields** — references, logo URLs, email addresses, invoice numbers — since those are the ones Xero leaves empty. Enums and IDs are effectively unaffected in practice because Xero always populates them, and code reading those tends to fail loudly either way (`hash.fetch("".downcase)` already raised `KeyError`; it now raises `NoMethodError` and loses the field name from the message). One consumer measured 14 unguarded `.downcase` calls against this change and needed no code changes at all.
+
+### Fixed
+
+- `RateLimitCapture` now finds Xero's quota headers whatever case they arrive in. Faraday's own header container is case-insensitive so production was never affected, but a plain Hash is not, and Xero's headers are cased differently depending on who recorded them — a VCR cassette holds `X-Daylimit-Remaining`, not `X-DayLimit-Remaining`. Missing them records nothing, and `day_below?` answers false when nothing is known, so the failure would have been a quota check that passes while measuring nothing. Spotted by a consumer reading their own recorded cassettes against the lookup.
+
+- **`Allocation#applied_amount` was nil for every allocation — 0.6.0 fixed the name and kept the bug.** Xero sends the allocated value under `"Amount"` in JSON and `"AppliedAmount"` in XML. 0.6.0 concluded the opposite, from payloads a legacy XML client had stored, and remapped the attribute to a key this JSON-only client never receives. Before 0.6.0 the mapping was right. Measured on a live tenant: 24 of 24 allocations carry `Amount`, none carry `AppliedAmount`, on both the list and single-resource endpoints.
+
+  Both keys are now modelled and **both `#amount` and `#applied_amount` resolve to whichever one arrived**, so neither can be nil while the other holds a value. This has been got wrong twice in opposite directions, each time producing a nil that an importer coerced to `0.0` and stored with nothing raising; reading both is cheaper than being certain. `to_h` reports the value under both keys, and `raw` still shows which key Xero sent.
+
+### Added
+
+- `Accounting::User` now models **`global_user_id`** (`GlobalUserID`). Xero returns two identifiers that differ for every user: `user_id` is scoped to the organisation, `global_user_id` identifies the person across all of them and is what an OIDC `id_token` subject corresponds to. A consumer keyed membership records on `user_id` while its own user table used `GlobalUserID`, so every membership pointed at an identifier no user row carried — the association came back empty, silently, with the right count and the right roles.
+
+- `Accounting::TrackingOption` now models the four booleans Xero returns alongside `Status`: `is_active`, `is_archived`, `is_deleted` and `has_validation_errors`. Confirmed against a live `GET /TrackingCategories` response — every key the endpoint returns on an option is now modelled. `#active?` still reads `status`, because the two agree on an active option and whether they diverge on an archived one is unconfirmed; `is_archived` and `is_deleted` are the ones to read when you need to tell those apart, since `status` collapses both into `"DELETED"`.
+
+### Documentation
+
+- **Corrected what `Throttle::DailyLimitExhausted#retry_after` actually means.** Its docstring said the wait was "typically measured in hours" and described a reset boundary. The day bucket trickles like the minute bucket — `capacity / window_ms` per millisecond — so at `per_day: 4_900` a token accrues every 17.6 seconds, and that is what `retry_after` returns. There is no reset in the arithmetic. Reported by a consumer who read the comment, believed the wait would be hours, and designed an hourly sweep around it before measuring.
+
+  Re-enqueueing rather than blocking is still correct, but for a different reason than the docstring gave: a sync needing several hundred more calls waits 17.6s for each of them, which is hours in aggregate even though each wait is short.
+
+- **Documented what `per_minute` and `per_day` actually guarantee.** A token bucket's configured value is both its capacity and its refill rate, and a fresh bucket starts full — so the first window can spend the capacity *and* everything refilling during it. Measured: `per_minute: 55` allows **109 calls in the first 60 seconds**, not 55. The `Choosing limits` table recommends exactly that value as headroom under Xero's 60, which it is not at a cold start; steady state does converge on the configured rate. Halve the value if you need a hard ceiling in any single window. Both behaviours now have specs so a future change is deliberate.
+
+- Noted that the two `retry_after` sources are orders of magnitude apart — Xero's reported wait on a daily 429 can be long, while `DailyLimitExhausted` is always seconds — and that anything recording a durable back-off should tag which one it came from.
+- Strengthened the XML-migration guidance in `docs/client.md` to cover **type** divergence, which it previously called "usually fine". XML has no types, so a boolean arrives as the string `"false"` — truthy in Ruby — while JSON sends a real `false`. A reader using plain truthiness rather than a cast has been doing the opposite of what it reads as, and moving to kiwi silently reverses it at cutover. Reported by a consumer who found years of statements going to contact people explicitly marked not to receive them. That is now the third documented axis of divergence, after nesting and key names.
+
 ## [0.7.0] - 2026-09-27
 
 ### Added

data/docs/accounting/credit-note.md

@@ -61,7 +61,7 @@ fields Xero returns:
 | `reference` | `String` | Additional reference number (ACCRECCREDIT only). |
 | `sent_to_contact` | `Boolean` | Whether the credit note has been sent to the contact. |
 | `remaining_credit` | `Numeric` | The remaining credit balance. |
-| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
+| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, the allocated value (as both `amount` and `applied_amount` — see below), `date`, `is_deleted`, and an `invoice` reference. |
 | `branding_theme_id` | `String` | The branding theme ID applied to the credit note. |
 | `has_attachments` | `Boolean` | Whether the credit note has attachments. |
 
@@ -95,3 +95,25 @@ credit_notes = client.credit_notes(tenant_id)
 with_credit = credit_notes.select { |cn| cn.remaining_credit.to_f > 0 }
 with_credit.each { |cn| puts "#{cn.credit_note_number}: #{cn.remaining_credit} remaining" }
 ```
+
+## The allocated value: `amount` and `applied_amount`
+
+Xero sends the allocated value under **different keys in different
+representations**:
+
+| Representation | Key |
+|---|---|
+| JSON — what this client requests | `Amount` |
+| XML — what other Xero clients get | `AppliedAmount` |
+
+Kiwi models both and both readers resolve to whichever one arrived, so
+`allocation.amount` and `allocation.applied_amount` return the same value and
+neither is nil while the other holds one. `to_h` reports it under both keys;
+`raw` shows which key Xero actually sent.
+
+This is worth knowing if you are migrating from a client that spoke XML and
+have stored payloads: your old rows say `AppliedAmount` and Xero's JSON says
+`Amount`, for the same field on the same record. Key names do not reliably
+carry across the two representations — an assumption that has produced two
+separate silent-nil bugs here, each one an importer writing `0.0` into every
+allocated amount with nothing raising.

data/docs/accounting/overpayment.md

@@ -57,7 +57,7 @@ fields Xero returns:
 | `currency_code` | `String` | The currency code (e.g. `"NZD"`). |
 | `currency_rate` | `String` | The currency rate (1.0 for base currency). |
 | `remaining_credit` | `String` | The remaining credit balance. |
-| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
+| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, the allocated value (as both `amount` and `applied_amount` — see below), `date`, `is_deleted`, and an `invoice` reference. |
 | `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
 | `has_attachments` | `Boolean` | Whether the overpayment has attachments. |
 | `reference` | `String` | Reference for the overpayment. |

data/docs/accounting/prepayment.md

@@ -58,7 +58,7 @@ fields Xero returns:
 | `currency_rate` | `String` | The currency rate (1.0 for base currency). |
 | `invoice_number` | `String` | The invoice number (for receive prepayments only). |
 | `remaining_credit` | `String` | The remaining credit balance. |
-| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
+| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, the allocated value (as both `amount` and `applied_amount` — see below), `date`, `is_deleted`, and an `invoice` reference. |
 | `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
 | `has_attachments` | `Boolean` | Whether the prepayment has attachments. |
 | `fully_paid_on_date` | `Time` | When the prepayment was fully allocated, parsed as UTC. |

data/docs/accounting/tracking-category.md

@@ -84,9 +84,20 @@ single `TrackingCategory`, or `nil` if the response is empty.
 | `tracking_option_id` | `String` | The unique Xero identifier for the option. |
 | `name` | `String` | The option's display name (e.g. "Eastside"). |
 | `status` | `String` | `"ACTIVE"` or `"DELETED"`. |
+| `is_active` | `Boolean` | Xero's boolean equivalent of `status`. |
+| `is_archived` | `Boolean` | Whether the option has been archived. |
+| `is_deleted` | `Boolean` | Whether the option has been deleted. |
+| `has_validation_errors` | `Boolean` | Whether Xero flagged validation errors on the option. |
 
 `#active?` is a shorthand for `status == "ACTIVE"`.
 
+Xero sends both `Status` and those booleans, and they overlap. On an active
+option they agree — whether they diverge on an archived or deleted one is
+unconfirmed, which is why `#active?` reads `status` and the booleans are left
+for you to interpret. If you need to distinguish archived from deleted,
+`is_archived` and `is_deleted` are the ones to read; `status` collapses both
+into `"DELETED"`.
+
 ## The Tracking object
 
 The assignment shape, nested on line items and contacts.

data/docs/accounting/user.md

@@ -40,7 +40,8 @@ Xero returns:
 
 | Attribute | Type | What it is |
 |-----------|------|------------|
-| `user_id` | `String` | The unique Xero identifier for the user. |
+| `user_id` | `String` | Xero's identifier for the user **within this organisation**. |
+| `global_user_id` | `String` | Xero's identifier for the person **across all organisations**. Differs from `user_id`, and is what an OIDC `id_token` subject corresponds to. |
 | `email_address` | `String` | The user's email address. |
 | `first_name` | `String` | The user's first name. |
 | `last_name` | `String` | The user's last name. |

data/docs/client.md

@@ -113,6 +113,22 @@ Depend on these; they won't change without a major version.
 The one thing deliberately *not* guaranteed is the presence of `raw` on
 nested objects, per the bullet above.
 
+**This is where `raw` and the typed attributes differ on purpose.** Xero
+sends `""` for a text field with no value; the modelled attribute reads it
+as `nil`, while `raw` keeps the `""` exactly as it arrived:
+
+```ruby
+theme.logo_url        # => nil
+theme.raw["LogoUrl"]  # => ""
+```
+
+That empty string is otherwise easy to carry into a database column, where
+it quietly changes what queries match — a `where.not(logo_url: nil)` starts
+returning rows with no logo. `type: :date` has always made this call;
+strings doing otherwise was an inconsistency. Only exactly `""` is
+affected: `" "` is left alone, because trimming it would be editorialising
+on a value rather than recognising an absent one.
+
 ### Why nested objects don't carry raw
 
 Two reasons, both worth knowing if you're tempted to ask for it.
@@ -148,6 +164,18 @@ contact with one person parses to a Hash and a contact with two parses to
 an Array, from the same endpoint. Code written against that has a
 normalising step somewhere, whether or not its author knew why.
 
+**Key names can differ too, not only nesting.** It is tempting to assume the
+two representations agree on field names and diverge only in structure. They
+don't. An allocation's value is `Amount` in JSON and `AppliedAmount` in XML —
+same field, same record, different name — and nothing in Xero's
+documentation says so. Two separate silent-nil bugs in this gem came from
+assuming otherwise, each one an importer writing `0.0` into every allocated
+amount with nothing raising.
+
+So when you check a field against stored XML-era payloads, you are checking
+its name as well as its shape. If a modelled attribute comes back nil
+against real JSON, suspect the key before suspecting the data.
+
 No client setting reproduces these shapes — they're artefacts of an XML
 parse kiwi doesn't do. But the remedy isn't the same everywhere, and it's
 worth sorting your readers into two piles before planning the work.
@@ -166,10 +194,26 @@ reads are the same in both representations, only the *writer* changes:
 stop unwrapping, store the array. Existing rows stay readable, and you
 skip a migration entirely.
 
-Type coercion is usually fine in that second pile too. XML gives you
-strings — `"false"` rather than `false` — and if the reader hands that to
-something like ActiveModel's boolean cast, both the old string and the new
-real boolean land on the same value.
+**Types change too, and booleans are where it bites.** XML has no types, so
+everything arrives as a string: `"false"`, not `false`. JSON gives you the
+real thing. If your reader passes the value through a caster —
+ActiveModel's boolean cast, say — both land on the same result and you'll
+never notice. If it relies on plain Ruby truthiness, the two are opposites:
+
+```ruby
+"false" ? :yes : :no   # => :yes   ← every non-empty string is truthy
+false   ? :yes : :no   # => :no
+```
+
+A reader doing `select(&:included)` over an XML-derived `"false"` has been
+selecting the records it was meant to exclude, for as long as that code has
+existed. Moving to kiwi *fixes* it — which means the behaviour changes at
+cutover, in a way your users may see. Find those readers before you switch,
+not after: grep for boolean-ish fields consumed without a cast.
+
+This is the third axis on which the two representations differ, after
+nesting and key names. Treat "it's the same field, so it's the same value"
+as the assumption to check rather than the one to rely on.
 
 One piece of luck worth knowing about: the two piles tend to fail
 differently. A `dig` that misses returns nil and writes a blank record

data/docs/retries-and-rate-limits.md

@@ -116,6 +116,21 @@ Three things worth knowing when you write this:
 - **`tenant_id` is on the exception for a reason.** Don't infer it from
   surrounding context — that breaks the moment one client serves more than
   one tenant.
+- **The two `retry_after` values are orders of magnitude apart.** Keep them
+  distinguishable, which is what the `source:` in the example above is for.
+
+| Source | Whose limit | Typical magnitude |
+|---|---|---|
+| `RateLimitError` | Xero's, reported by Xero | seconds to hours, and on a daily 429 it can be genuinely long |
+| `Throttle::DailyLimitExhausted` | yours, from the configured bucket | **seconds** — at `per_day: 4_900`, 17.6s, the time for one token to accrue |
+| `Throttle::Timeout` | yours, per-minute bucket | seconds |
+
+The throttle's day bucket trickles rather than resetting on a boundary, so
+`DailyLimitExhausted#retry_after` is the wait for a *single* token, not the
+wait until your daily allowance renews. It's enough to resume, not enough to
+finish — which is exactly why the job should re-enqueue rather than sleep.
+If your UI shows a countdown, showing 17 seconds when the sync has hundreds
+of calls left to make will read as broken.
 
 `XeroKiwi::Throttle::Timeout` carries `tenant_id` and `retry_after` too, if
 you want to treat a per-minute wait that exceeded `max_wait` as a back-off

data/docs/throttling.md

@@ -117,9 +117,34 @@ Pick values *below* Xero's defaults:
 | 5,000 calls/day per tenant | `per_day: 4,700` – `4,900` |
 
 The exact number depends on how much you care about the occasional 429 vs.
-maximising throughput. If your job batches run for hours, lean conservative —
-the daily limit resets on Xero's clock, not yours, and the first few
-minutes after "daily reset" can be ambiguous.
+maximising throughput.
+
+### What the numbers actually guarantee
+
+This is a token bucket, so your configured value is **two things at once**:
+the bucket's capacity, and its refill rate. A fresh bucket starts full, so
+over the *first* window you can spend the full capacity **and** everything
+that refills during it — close to double.
+
+Measured, from a fresh bucket:
+
+| Setting | Calls in the first 60s | Sustained after that |
+|---|---|---|
+| `per_minute: 55` | 109 | 55/min |
+| `per_minute: 10` | 19 | 10/min |
+| `per_minute: 5` | 9 | 5/min |
+
+So `per_minute: 55` is not a promise that you will never exceed 60 in a
+minute — a cold start can reach 109, and Xero will 429 some of those. The
+reactive retry layer is what catches them; that's why both layers exist and
+neither replaces the other.
+
+Steady state is what the setting really controls, and there it converges on
+exactly the value you configured. If you need a hard ceiling within any
+single 60-second window, halve it: `per_minute: 30` keeps even a cold-start
+burst under Xero's 60.
+
+The same applies to `per_day`, on a 24-hour scale.
 
 ## Per-minute vs per-day failure modes
 
@@ -133,19 +158,32 @@ probably too many concurrent workers for the configured `per_minute`.
 
 **Per-day:** the limiter raises `XeroKiwi::Throttle::DailyLimitExhausted`
 immediately, with `retry_after` in seconds and the `tenant_id` it relates to.
-Sleeping for hours is never the right move in a Sidekiq worker, so the caller
-has to decide:
+
+`retry_after` here is **seconds, not hours**. The day bucket trickles like
+the minute one rather than resetting on a boundary, so at `per_day: 4_900` a
+token accrues every 17.6 seconds and that is what you get back. There is no
+reset in the arithmetic.
+
+Re-enqueueing rather than blocking is still right, but for a different
+reason than the wait length. A sync needing several hundred more calls would
+wait 17.6s for *each* of them — short individually, hours in aggregate. That
+is the thing a worker should not sit through.
 
 ```ruby
 begin
   client.invoices(tenant_id)
 rescue XeroKiwi::Throttle::DailyLimitExhausted => e
-  # Re-enqueue the job for tomorrow. `retry_after` is seconds until the
-  # bucket has at least one token.
+  # `retry_after` is seconds until the bucket has one token — enough to
+  # resume, not enough to finish. Re-enqueue rather than sleep.
   MyJob.perform_in(e.retry_after, org_id)
 end
 ```
 
+Don't conflate this with Xero's own `Retry-After` on a daily 429. That one
+reflects Xero's limit rather than your configured one and can be genuinely
+long. The two are orders of magnitude apart, so anything recording a durable
+back-off should keep them distinguishable — tag the source.
+
 Both throttle exceptions carry `tenant_id` and `retry_after`, which mirrors
 the `XeroKiwi::RateLimitError` shape the retry layer raises after exhausting
 retries on a 429 — so one rescue can cover both, and code handling them looks

data/lib/xero_kiwi/accounting/allocation.rb

@@ -5,14 +5,19 @@ module XeroKiwi
     # Represents an allocation of a credit note, prepayment, or overpayment
     # against an invoice.
     #
-    # Xero names the allocated value differently depending on direction: the
-    # allocation *request* body takes "Amount", while every *response* — and
-    # allocations only ever reach this gem embedded in a CreditNote,
-    # Prepayment or Overpayment response — carries "AppliedAmount". This
-    # class was originally modelled off the request shape, so `amount`
-    # silently returned nil for every allocation the gem could actually
-    # produce. `#amount` is now an alias of `#applied_amount` so callers who
-    # reached for the obvious name get the value they meant.
+    # THE ALLOCATED VALUE ARRIVES UNDER DIFFERENT KEYS IN DIFFERENT XERO
+    # REPRESENTATIONS, and getting it wrong writes zeroes into money:
+    #
+    #   JSON (what this client requests):  "Amount"
+    #   XML  (what other clients get):     "AppliedAmount"
+    #
+    # That is not a request-vs-response split, which is what 0.6.0 assumed
+    # when it renamed `amount` to `applied_amount` — an assumption drawn from
+    # payloads a legacy XML client had stored, and wrong for this JSON-only
+    # client. Both keys are modelled now, and `#amount` and `#applied_amount`
+    # each resolve to whichever one Xero populated, so neither can be nil
+    # when the other holds a value. This has been got wrong twice in opposite
+    # directions; reading both is cheaper than being certain.
     #
     # See: https://developer.xero.com/documentation/api/accounting/overpayments
     class Allocation
@@ -21,15 +26,18 @@ module XeroKiwi
       identity :allocation_id
 
       attribute :allocation_id,  xero: "AllocationID",  type: :guid
+      attribute :amount,         xero: "Amount",        type: :decimal
       attribute :applied_amount, xero: "AppliedAmount", type: :decimal
       attribute :date,           xero: "Date",          type: :date
       attribute :invoice,        xero: "Invoice",       type: :object, of: Invoice, reference: true
       attribute :is_deleted,     xero: "IsDeleted",     type: :bool
 
-      # Kept for callers written against the pre-0.6.0 attribute name. It
-      # could only ever have returned nil, so nothing that relied on a real
-      # value is affected.
-      def amount = applied_amount
+      # Both names are public API and both resolve to the same value. `to_h`
+      # therefore reports it under both keys; `raw` still shows exactly which
+      # one Xero sent.
+      def amount = @amount.nil? ? @applied_amount : @amount
+
+      def applied_amount = @applied_amount.nil? ? @amount : @applied_amount
     end
   end
 end

data/lib/xero_kiwi/accounting/hydrator.rb

@@ -20,25 +20,41 @@ module XeroKiwi
     module Hydrator
       module_function
 
-      def call(raw, spec) # rubocop:disable Metrics/AbcSize,Metrics/CyclomaticComplexity
+      def call(raw, spec)
         return spec[:hydrate].call(raw) if spec[:hydrate]
         return [] if spec[:type] == :collection && raw.nil?
         return nil if raw.nil?
 
+        hydrate_typed(raw, spec)
+      end
+
+      def hydrate_typed(raw, spec)
         case spec[:type]
-        when :string, :enum, :guid, :bool, :decimal
-          raw
-        when :date
-          parse_time(raw)
-        when :object
-          build_object(raw, spec)
-        when :collection
-          raw.map { |item| build_object(item, spec) }
-        else
-          raise ArgumentError, "unknown attribute type: #{spec[:type].inspect}"
+        when :string, :enum, :guid then blank_to_nil(raw)
+        when :bool, :decimal       then raw
+        when :date                 then parse_time(raw)
+        when :object               then build_object(raw, spec)
+        when :collection           then raw.map { |item| build_object(item, spec) }
+        else raise ArgumentError, "unknown attribute type: #{spec[:type].inspect}"
         end
       end
 
+      # Xero sends "" for a text field that has no value, where its XML
+      # representation produced nil. Left alone, that empty string reaches a
+      # database column and quietly changes what queries match — a
+      # `where.not(logo_url: nil)` starts returning rows with no logo.
+      #
+      # Only exactly "" becomes nil. Whitespace is left alone: " " may be
+      # deliberate, and trimming it would be editorialising on a value
+      # rather than recognising an absent one. `raw` is untouched either
+      # way, so the original payload is always recoverable.
+      #
+      # `:date` has always done this (see parse_time); strings behaving
+      # differently was an inconsistency, not a principle.
+      def blank_to_nil(value)
+        value == "" ? nil : value
+      end
+
       # Xero uses two timestamp formats depending on the endpoint:
       #
       #   ISO 8601:  "2019-07-09T23:40:30.1833130" (connections API)

data/lib/xero_kiwi/accounting/tracking_option.rb

@@ -5,6 +5,11 @@ module XeroKiwi
     # One selectable option within a tracking category, as returned nested in
     # the /TrackingCategories response.
     #
+    # Xero sends both `Status` and a set of booleans that overlap it. They
+    # agree on an active option; whether they diverge on an archived or
+    # deleted one is unconfirmed, so `#active?` stays on `Status` and the
+    # booleans are exposed for callers who need the distinction.
+    #
     # See: https://developer.xero.com/documentation/api/accounting/trackingcategories
     class TrackingOption
       include Resource
@@ -12,9 +17,13 @@ module XeroKiwi
       payload_key "Options"
       identity    :tracking_option_id
 
-      attribute :tracking_option_id, xero: "TrackingOptionID", type: :guid
-      attribute :name,               xero: "Name",             query: true
-      attribute :status,             xero: "Status",           type: :enum, query: true
+      attribute :tracking_option_id,    xero: "TrackingOptionID",     type: :guid
+      attribute :name,                  xero: "Name",                 query: true
+      attribute :status,                xero: "Status",               type: :enum, query: true
+      attribute :is_active,             xero: "IsActive",             type: :bool
+      attribute :is_archived,           xero: "IsArchived",           type: :bool
+      attribute :is_deleted,            xero: "IsDeleted",            type: :bool
+      attribute :has_validation_errors, xero: "HasValidationErrors",  type: :bool
 
       def active? = status == "ACTIVE"
     end

data/lib/xero_kiwi/accounting/user.rb

@@ -11,7 +11,13 @@ module XeroKiwi
       payload_key "Users"
       identity    :user_id
 
-      attribute :user_id,           xero: "UserID", type: :guid
+      # Xero returns two distinct identifiers. UserID is scoped to the
+      # organisation; GlobalUserID identifies the person across every
+      # organisation they belong to, and is what an OIDC id_token subject
+      # corresponds to. They differ for every user — key membership records
+      # on the one your own user table uses.
+      attribute :user_id,           xero: "UserID",           type: :guid
+      attribute :global_user_id,    xero: "GlobalUserID",     type: :guid
       attribute :email_address,     xero: "EmailAddress",     query: true
       attribute :first_name,        xero: "FirstName",        query: true
       attribute :last_name,         xero: "LastName",         query: true

data/lib/xero_kiwi/client.rb

@@ -935,17 +935,33 @@ module XeroKiwi
 
         @store.record(
           tenant_id,
-          day:        to_i_or_nil(headers["X-DayLimit-Remaining"]),
-          minute:     to_i_or_nil(headers["X-MinLimit-Remaining"]),
-          app_minute: to_i_or_nil(headers["X-AppMinLimit-Remaining"])
+          day:        header_int(headers, "X-DayLimit-Remaining"),
+          minute:     header_int(headers, "X-MinLimit-Remaining"),
+          app_minute: header_int(headers, "X-AppMinLimit-Remaining")
         )
       end
 
       private
 
-      def to_i_or_nil(value)
+      # Faraday's own header container is case-insensitive, so a direct
+      # lookup is right in production. It is not right everywhere: a plain
+      # Hash is case-sensitive, and Xero's headers come back cased
+      # differently depending on who recorded them — a VCR cassette holds
+      # `X-Daylimit-Remaining`, not `X-DayLimit-Remaining`. Missing them
+      # would record nothing, and `day_below?` answers false when nothing is
+      # known, so the failure is a quota check that silently measures
+      # nothing. Scan on miss rather than rely on the container's manners.
+      def header_int(headers, name)
+        value = headers[name]
+        value = scan_for(headers, name) if value.nil?
+
         value.nil? || value.to_s.empty? ? nil : value.to_i
       end
+
+      def scan_for(headers, name)
+        key = headers.keys.find { |candidate| candidate.to_s.casecmp?(name) }
+        key && headers[key]
+      end
     end
 
     # Faraday middleware that maps non-2xx responses onto our exception

data/lib/xero_kiwi/throttle.rb

@@ -28,10 +28,25 @@ module XeroKiwi
       end
     end
 
-    # Raised immediately (no sleep) when the per-day bucket is exhausted. The
-    # wait until reset is typically measured in hours, so blocking the caller
-    # is the wrong move — re-enqueue the job at `retry_after` instead. Shape
-    # mirrors RateLimitError so existing Xero rate-limit handling applies.
+    # Raised immediately (no sleep) when the per-day bucket is exhausted.
+    #
+    # `retry_after` is the wait for ONE token, and it is seconds, not hours —
+    # the day bucket trickles like the minute bucket rather than resetting on
+    # a boundary. At `per_day: 4_900` a token accrues every 17.6s
+    # (86_400_000ms / 4_900). There is no reset in the arithmetic.
+    #
+    # Re-enqueueing rather than blocking is still the right move, but for a
+    # different reason than the wait length: a sync needing several hundred
+    # more calls would sleep 17.6s for each of them, which is hours in
+    # aggregate even though each individual wait is short.
+    #
+    # Do not conflate this with Xero's own `Retry-After` on a daily 429,
+    # which reflects Xero's limit rather than your configured one and can be
+    # genuinely long. Anything recording a durable back-off should keep the
+    # two distinguishable — see docs/retries-and-rate-limits.md.
+    #
+    # Shape mirrors RateLimitError so existing Xero rate-limit handling
+    # applies.
     class DailyLimitExhausted < Error
       attr_reader :retry_after, :tenant_id
 

data/lib/xero_kiwi/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module XeroKiwi
-  VERSION = "0.7.0"
+  VERSION = "0.8.0"
 end

data/llms-full.txt

@@ -436,6 +436,22 @@ Depend on these; they won't change without a major version.
 The one thing deliberately *not* guaranteed is the presence of `raw` on
 nested objects, per the bullet above.
 
+**This is where `raw` and the typed attributes differ on purpose.** Xero
+sends `""` for a text field with no value; the modelled attribute reads it
+as `nil`, while `raw` keeps the `""` exactly as it arrived:
+
+```ruby
+theme.logo_url        # => nil
+theme.raw["LogoUrl"]  # => ""
+```
+
+That empty string is otherwise easy to carry into a database column, where
+it quietly changes what queries match — a `where.not(logo_url: nil)` starts
+returning rows with no logo. `type: :date` has always made this call;
+strings doing otherwise was an inconsistency. Only exactly `""` is
+affected: `" "` is left alone, because trimming it would be editorialising
+on a value rather than recognising an absent one.
+
 ### Why nested objects don't carry raw
 
 Two reasons, both worth knowing if you're tempted to ask for it.
@@ -471,6 +487,18 @@ contact with one person parses to a Hash and a contact with two parses to
 an Array, from the same endpoint. Code written against that has a
 normalising step somewhere, whether or not its author knew why.
 
+**Key names can differ too, not only nesting.** It is tempting to assume the
+two representations agree on field names and diverge only in structure. They
+don't. An allocation's value is `Amount` in JSON and `AppliedAmount` in XML —
+same field, same record, different name — and nothing in Xero's
+documentation says so. Two separate silent-nil bugs in this gem came from
+assuming otherwise, each one an importer writing `0.0` into every allocated
+amount with nothing raising.
+
+So when you check a field against stored XML-era payloads, you are checking
+its name as well as its shape. If a modelled attribute comes back nil
+against real JSON, suspect the key before suspecting the data.
+
 No client setting reproduces these shapes — they're artefacts of an XML
 parse kiwi doesn't do. But the remedy isn't the same everywhere, and it's
 worth sorting your readers into two piles before planning the work.
@@ -489,10 +517,26 @@ reads are the same in both representations, only the *writer* changes:
 stop unwrapping, store the array. Existing rows stay readable, and you
 skip a migration entirely.
 
-Type coercion is usually fine in that second pile too. XML gives you
-strings — `"false"` rather than `false` — and if the reader hands that to
-something like ActiveModel's boolean cast, both the old string and the new
-real boolean land on the same value.
+**Types change too, and booleans are where it bites.** XML has no types, so
+everything arrives as a string: `"false"`, not `false`. JSON gives you the
+real thing. If your reader passes the value through a caster —
+ActiveModel's boolean cast, say — both land on the same result and you'll
+never notice. If it relies on plain Ruby truthiness, the two are opposites:
+
+```ruby
+"false" ? :yes : :no   # => :yes   ← every non-empty string is truthy
+false   ? :yes : :no   # => :no
+```
+
+A reader doing `select(&:included)` over an XML-derived `"false"` has been
+selecting the records it was meant to exclude, for as long as that code has
+existed. Moving to kiwi *fixes* it — which means the behaviour changes at
+cutover, in a way your users may see. Find those readers before you switch,
+not after: grep for boolean-ish fields consumed without a cast.
+
+This is the third axis on which the two representations differ, after
+nesting and key names. Treat "it's the same field, so it's the same value"
+as the assumption to check rather than the one to rely on.
 
 One piece of luck worth knowing about: the two piles tend to fail
 differently. A `dig` that misses returns nil and writes a blank record
@@ -2394,7 +2438,8 @@ Xero returns:
 
 | Attribute | Type | What it is |
 |-----------|------|------------|
-| `user_id` | `String` | The unique Xero identifier for the user. |
+| `user_id` | `String` | Xero's identifier for the user **within this organisation**. |
+| `global_user_id` | `String` | Xero's identifier for the person **across all organisations**. Differs from `user_id`, and is what an OIDC `id_token` subject corresponds to. |
 | `email_address` | `String` | The user's email address. |
 | `first_name` | `String` | The user's first name. |
 | `last_name` | `String` | The user's last name. |
@@ -2530,7 +2575,7 @@ fields Xero returns:
 | `reference` | `String` | Additional reference number (ACCRECCREDIT only). |
 | `sent_to_contact` | `Boolean` | Whether the credit note has been sent to the contact. |
 | `remaining_credit` | `Numeric` | The remaining credit balance. |
-| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
+| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, the allocated value (as both `amount` and `applied_amount` — see below), `date`, `is_deleted`, and an `invoice` reference. |
 | `branding_theme_id` | `String` | The branding theme ID applied to the credit note. |
 | `has_attachments` | `Boolean` | Whether the credit note has attachments. |
 
@@ -2565,6 +2610,28 @@ with_credit = credit_notes.select { |cn| cn.remaining_credit.to_f > 0 }
 with_credit.each { |cn| puts "#{cn.credit_note_number}: #{cn.remaining_credit} remaining" }
 ```
 
+## The allocated value: `amount` and `applied_amount`
+
+Xero sends the allocated value under **different keys in different
+representations**:
+
+| Representation | Key |
+|---|---|
+| JSON — what this client requests | `Amount` |
+| XML — what other Xero clients get | `AppliedAmount` |
+
+Kiwi models both and both readers resolve to whichever one arrived, so
+`allocation.amount` and `allocation.applied_amount` return the same value and
+neither is nil while the other holds one. `to_h` reports it under both keys;
+`raw` shows which key Xero actually sent.
+
+This is worth knowing if you are migrating from a client that spoke XML and
+have stored payloads: your old rows say `AppliedAmount` and Xero's JSON says
+`Amount`, for the same field on the same record. Key names do not reliably
+carry across the two representations — an assumption that has produced two
+separate silent-nil bugs here, each one an importer writing `0.0` into every
+allocated amount with nothing raising.
+
 
 ================================================================================
 FILE: docs/accounting/invoice.md
@@ -2874,7 +2941,7 @@ fields Xero returns:
 | `currency_code` | `String` | The currency code (e.g. `"NZD"`). |
 | `currency_rate` | `String` | The currency rate (1.0 for base currency). |
 | `remaining_credit` | `String` | The remaining credit balance. |
-| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
+| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, the allocated value (as both `amount` and `applied_amount` — see below), `date`, `is_deleted`, and an `invoice` reference. |
 | `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
 | `has_attachments` | `Boolean` | Whether the overpayment has attachments. |
 | `reference` | `String` | Reference for the overpayment. |
@@ -2975,7 +3042,7 @@ fields Xero returns:
 | `currency_rate` | `String` | The currency rate (1.0 for base currency). |
 | `invoice_number` | `String` | The invoice number (for receive prepayments only). |
 | `remaining_credit` | `String` | The remaining credit balance. |
-| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, `applied_amount`, `date`, `is_deleted`, and an `invoice` reference. |
+| `allocations` | `Array<XeroKiwi::Accounting::Allocation>` | Allocations to invoices. Each has `allocation_id`, the allocated value (as both `amount` and `applied_amount` — see below), `date`, `is_deleted`, and an `invoice` reference. |
 | `payments` | `Array<XeroKiwi::Accounting::Payment>` | Payment records (references). See [Payments](payment.md). |
 | `has_attachments` | `Boolean` | Whether the prepayment has attachments. |
 | `fully_paid_on_date` | `Time` | When the prepayment was fully allocated, parsed as UTC. |
@@ -3216,9 +3283,20 @@ single `TrackingCategory`, or `nil` if the response is empty.
 | `tracking_option_id` | `String` | The unique Xero identifier for the option. |
 | `name` | `String` | The option's display name (e.g. "Eastside"). |
 | `status` | `String` | `"ACTIVE"` or `"DELETED"`. |
+| `is_active` | `Boolean` | Xero's boolean equivalent of `status`. |
+| `is_archived` | `Boolean` | Whether the option has been archived. |
+| `is_deleted` | `Boolean` | Whether the option has been deleted. |
+| `has_validation_errors` | `Boolean` | Whether Xero flagged validation errors on the option. |
 
 `#active?` is a shorthand for `status == "ACTIVE"`.
 
+Xero sends both `Status` and those booleans, and they overlap. On an active
+option they agree — whether they diverge on an archived or deleted one is
+unconfirmed, which is why `#active?` reads `status` and the booleans are left
+for you to interpret. If you need to distinguish archived from deleted,
+`is_archived` and `is_deleted` are the ones to read; `status` collapses both
+into `"DELETED"`.
+
 ## The Tracking object
 
 The assignment shape, nested on line items and contacts.
@@ -3895,6 +3973,21 @@ Three things worth knowing when you write this:
 - **`tenant_id` is on the exception for a reason.** Don't infer it from
   surrounding context — that breaks the moment one client serves more than
   one tenant.
+- **The two `retry_after` values are orders of magnitude apart.** Keep them
+  distinguishable, which is what the `source:` in the example above is for.
+
+| Source | Whose limit | Typical magnitude |
+|---|---|---|
+| `RateLimitError` | Xero's, reported by Xero | seconds to hours, and on a daily 429 it can be genuinely long |
+| `Throttle::DailyLimitExhausted` | yours, from the configured bucket | **seconds** — at `per_day: 4_900`, 17.6s, the time for one token to accrue |
+| `Throttle::Timeout` | yours, per-minute bucket | seconds |
+
+The throttle's day bucket trickles rather than resetting on a boundary, so
+`DailyLimitExhausted#retry_after` is the wait for a *single* token, not the
+wait until your daily allowance renews. It's enough to resume, not enough to
+finish — which is exactly why the job should re-enqueue rather than sleep.
+If your UI shows a countdown, showing 17 seconds when the sync has hundreds
+of calls left to make will read as broken.
 
 `XeroKiwi::Throttle::Timeout` carries `tenant_id` and `retry_after` too, if
 you want to treat a per-minute wait that exceeded `max_wait` as a back-off
@@ -4229,9 +4322,34 @@ Pick values *below* Xero's defaults:
 | 5,000 calls/day per tenant | `per_day: 4,700` – `4,900` |
 
 The exact number depends on how much you care about the occasional 429 vs.
-maximising throughput. If your job batches run for hours, lean conservative —
-the daily limit resets on Xero's clock, not yours, and the first few
-minutes after "daily reset" can be ambiguous.
+maximising throughput.
+
+### What the numbers actually guarantee
+
+This is a token bucket, so your configured value is **two things at once**:
+the bucket's capacity, and its refill rate. A fresh bucket starts full, so
+over the *first* window you can spend the full capacity **and** everything
+that refills during it — close to double.
+
+Measured, from a fresh bucket:
+
+| Setting | Calls in the first 60s | Sustained after that |
+|---|---|---|
+| `per_minute: 55` | 109 | 55/min |
+| `per_minute: 10` | 19 | 10/min |
+| `per_minute: 5` | 9 | 5/min |
+
+So `per_minute: 55` is not a promise that you will never exceed 60 in a
+minute — a cold start can reach 109, and Xero will 429 some of those. The
+reactive retry layer is what catches them; that's why both layers exist and
+neither replaces the other.
+
+Steady state is what the setting really controls, and there it converges on
+exactly the value you configured. If you need a hard ceiling within any
+single 60-second window, halve it: `per_minute: 30` keeps even a cold-start
+burst under Xero's 60.
+
+The same applies to `per_day`, on a 24-hour scale.
 
 ## Per-minute vs per-day failure modes
 
@@ -4245,19 +4363,32 @@ probably too many concurrent workers for the configured `per_minute`.
 
 **Per-day:** the limiter raises `XeroKiwi::Throttle::DailyLimitExhausted`
 immediately, with `retry_after` in seconds and the `tenant_id` it relates to.
-Sleeping for hours is never the right move in a Sidekiq worker, so the caller
-has to decide:
+
+`retry_after` here is **seconds, not hours**. The day bucket trickles like
+the minute one rather than resetting on a boundary, so at `per_day: 4_900` a
+token accrues every 17.6 seconds and that is what you get back. There is no
+reset in the arithmetic.
+
+Re-enqueueing rather than blocking is still right, but for a different
+reason than the wait length. A sync needing several hundred more calls would
+wait 17.6s for *each* of them — short individually, hours in aggregate. That
+is the thing a worker should not sit through.
 
 ```ruby
 begin
   client.invoices(tenant_id)
 rescue XeroKiwi::Throttle::DailyLimitExhausted => e
-  # Re-enqueue the job for tomorrow. `retry_after` is seconds until the
-  # bucket has at least one token.
+  # `retry_after` is seconds until the bucket has one token — enough to
+  # resume, not enough to finish. Re-enqueue rather than sleep.
   MyJob.perform_in(e.retry_after, org_id)
 end
 ```
 
+Don't conflate this with Xero's own `Retry-After` on a daily 429. That one
+reflects Xero's limit rather than your configured one and can be genuinely
+long. The two are orders of magnitude apart, so anything recording a durable
+back-off should keep them distinguishable — tag the source.
+
 Both throttle exceptions carry `tenant_id` and `retry_after`, which mirrors
 the `XeroKiwi::RateLimitError` shape the retry layer raises after exhausting
 retries on a 429 — so one rescue can cover both, and code handling them looks

data/llms.txt

@@ -66,5 +66,5 @@ XeroKiwi requires Ruby 3.4.1 or newer. Install with `gem "xero-kiwi"`.
 
 - [README](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/README.md): high-level overview, installation, the documentation table of contents, and the development and release process
 - [Retries and rate limits](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/retries-and-rate-limits.md): how XeroKiwi handles 429s and transient 5xxs, `client.rate_limit(tenant_id)` for remaining quota (process-local), recording a durable per-tenant back-off signal from the `tenant_id` and `retry_after` carried by RateLimitError and the throttle errors, tuning the retry policy, the Faraday middleware ordering, why 500 is deliberately not retried
-- [Throttling](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/throttling.md): the Redis-backed per-tenant token bucket for coordinating rate limits across processes, `default_throttle`, `#remaining`, fail-open behaviour, writing a custom limiter
+- [Throttling](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/docs/throttling.md): the Redis-backed per-tenant token bucket for coordinating rate limits across processes, `default_throttle`, what per_minute/per_day actually guarantee (a fresh bucket bursts to roughly double in the first window), `#remaining`, fail-open behaviour, writing a custom limiter
 - [CHANGELOG](https://raw.githubusercontent.com/douglasgreyling/xero-kiwi/main/CHANGELOG.md): version history

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: xero-kiwi
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.8.0
 platform: ruby
 authors:
 - Douglas Greyling