xero-kiwi 0.5.0 → 0.5.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b980ba3e8f03590d2ca887887d8509268bf87df52b882eef4730a9ce1512250c
-  data.tar.gz: bdd4952e2b7d1fd3af201f2446d17d5cd7f2bfe86e58ee9002f4115cc5624240
+  metadata.gz: fb7181193042ba51598251bc4bee70c0e84ae0333216d94838e996f813b05770
+  data.tar.gz: 7103cdb42e4cb742cbafa0eae6c807f7958626751dbdd1cd86aa14fbbe4939a8
 SHA512:
-  metadata.gz: e7086fef7948e63b5d9437a164a241f819e5a96beead2eb2701989613e8b95f643d805be209a63b98ab3dcd278c7e81f46165de054f1fd865318abb82a13e0e9
-  data.tar.gz: 94c08345afefa1b130634ab9d643052d14a24ba5200ae60a7b324344706d71465e9cecb6ce30e183876dafdf7bde0fcdbbaea487869ea99073bb388f2a7f8ab2
+  metadata.gz: f5a369e373c45e9d4343666eb59e2adbf16a1620efe57052b89f655be7d09bc63c30d120c7a47dc873e5ed0ac1f6af5fb2b23c4b71847819ac15c9f3c2e0f5a3
+  data.tar.gz: 51ac21d3ace42a9ebefaa03ee2fdf924a6dd9549eb41d053901f0a2c6d115147e35a9f90e77438241937ca2171fdbf6a295792bfbef4528e209b5211f1d598b4

data/CHANGELOG.md

@@ -1,5 +1,19 @@
 ## [Unreleased]
 
+## [0.5.2] - 2026-09-27
+
+### Documentation
+
+- Documented that `Resource#raw` returns nil on nested objects (line items, addresses, contact persons, payment terms) even with `retain_raw: true`, and that the enclosing resource's hash is the access path — `org.raw["Addresses"]` rather than `org.addresses.map(&:raw)`. The behaviour was deliberate and explained in a comment on `from_response`, but the `#raw` docstring never mentioned it and the failure is a silent nil, so a consumer mapped over a nested collection and was one step from storing `[null]`.
+- Added a written contract for `raw` in `docs/client.md`: it is the parsed JSON with only top-level keys stringified, kiwi never normalises it (absent keys stay absent rather than becoming `[]`), unmodelled keys are preserved, and the freeze is shallow — the hash is frozen, the structures inside it are not. Specs pin all four, plus the nested-nil behaviour. The previous "is frozen" example overstated the guarantee and has been replaced.
+- `llms.txt` is now generated alongside `llms-full.txt` from a single manifest in `tasks/llms.rb`, rather than hand-maintained. It had not been touched since the initial commit and was missing four releases of material — no `querying.md`, no `throttling.md`, no `tracking-category.md`, and no mention of `Page`, `RateLimit`, the throttle limiters or the tracking classes. `rake llms:check` now also fails when a doc exists under `docs/` that the manifest does not list, which is the half that was missing: the freshness check could not notice a doc that was never indexed in the first place.
+
+## [0.5.1] - 2026-09-26
+
+### Documentation
+
+- Pinned down the scope of `Resource#raw`, which was ambiguous enough to mislead a real consumer. It holds the resource's own item hash — `contact.raw` has no `"Contacts"` key — and it is the JSON representation specifically. Xero's XML representation nests differently (no arrays, so one child parses to a Hash and several to an Array), so `raw` cannot reproduce an XML-derived shape. `docs/client.md` gains a short migration section covering what that means for anyone replacing a client that sent `Accept: text/xml` and has stored payloads. Specs added for both the envelope scope and the PascalCase/snake_case split. The migration section sorts readers into the two piles that behave differently: ones that dig the XML-only structure and must change, and ones that already normalise (`[value].flatten.compact`) where only the writer changes and stored rows stay readable.
+
 ## [0.5.0] - 2026-09-26
 
 The sync-support release: everything needed to drive a full-tenant sync through kiwi rather than around it.

data/Rakefile

@@ -9,76 +9,35 @@ require "rubocop/rake_task"
 
 RuboCop::RakeTask.new
 
-# Files included in llms-full.txt, in reading order. Keep README first, then
-# the docs in the same order the README's table of contents lists them.
-LLMS_SOURCE_FILES = %w[
-  README.md
-  docs/getting-started.md
-  docs/client.md
-  docs/oauth.md
-  docs/tokens.md
-  docs/connections.md
-  docs/accounting/contact.md
-  docs/accounting/contact-group.md
-  docs/accounting/organisation.md
-  docs/accounting/user.md
-  docs/accounting/credit-note.md
-  docs/accounting/invoice.md
-  docs/accounting/payment.md
-  docs/accounting/overpayment.md
-  docs/accounting/prepayment.md
-  docs/accounting/branding-theme.md
-  docs/accounting/tracking-category.md
-  docs/accounting/address.md
-  docs/accounting/phone.md
-  docs/accounting/external-link.md
-  docs/accounting/payment-terms.md
-  docs/errors.md
-  docs/retries-and-rate-limits.md
-  docs/throttling.md
-  docs/querying.md
-].freeze
-
-LLMS_FULL_PATH = "llms-full.txt"
-
-# Builds the llms-full.txt content from LLMS_SOURCE_FILES. Pure function:
-# returns a String, doesn't touch the filesystem. Both `llms:build` and
-# `llms:check` use this so the two tasks can never disagree about the
-# expected output.
-def build_llms_full
-  out = String.new(encoding: "UTF-8")
-  out << "# Xero Kiwi — full documentation\n\n"
-  out << "This file is the complete documentation for the Xero Kiwi gem (a Ruby wrapper for the Xero Accounting API), assembled into a single document for LLM consumption. It contains the README and every doc in the docs/ folder, in reading order.\n\n"
-  out << "For the curated index version, see llms.txt in the same directory.\n\n"
-  out << "Source: https://github.com/douglasgreyling/xero-kiwi\n\n"
-  LLMS_SOURCE_FILES.each { |path| append_file_block(out, path) }
-  out
-end
-
-def append_file_block(out, path)
-  separator = "=" * 80
-  out << "\n" << separator << "\n"
-  out << "FILE: #{path}\n"
-  out << separator << "\n\n"
-  out << File.read(path, encoding: "UTF-8") << "\n"
-end
+require_relative "tasks/llms"
 
 namespace :llms do
-  desc "Regenerate llms-full.txt from README and docs/"
+  desc "Regenerate llms.txt and llms-full.txt from README and docs/"
   task :build do
-    File.write(LLMS_FULL_PATH, build_llms_full)
-    puts "Wrote #{LLMS_FULL_PATH} (#{File.size(LLMS_FULL_PATH)} bytes, #{LLMS_SOURCE_FILES.size} source files)"
+    Llms.outputs.each do |path, content|
+      File.write(path, content)
+      puts "Wrote #{path} (#{File.size(path)} bytes)"
+    end
   end
 
-  desc "Verify llms-full.txt is up to date with README and docs/"
+  desc "Verify llms.txt and llms-full.txt are up to date, and that every doc is indexed"
   task :check do
-    expected = build_llms_full
-    actual   = File.exist?(LLMS_FULL_PATH) ? File.read(LLMS_FULL_PATH, encoding: "UTF-8") : ""
+    missing = Llms.unindexed_docs
+    unless missing.empty?
+      warn "✗ These docs are not listed in Llms::DOCS, so they are missing from llms.txt and llms-full.txt:"
+      missing.each { |path| warn "    #{path}" }
+      warn "  Add them to tasks/llms.rb and run `bundle exec rake llms:build`."
+      exit 1
+    end
+
+    stale = Llms.outputs.reject do |path, expected|
+      File.exist?(path) && File.read(path, encoding: "UTF-8") == expected
+    end
 
-    if expected == actual
-      puts "✓ #{LLMS_FULL_PATH} is up to date"
+    if stale.empty?
+      puts "✓ llms.txt and llms-full.txt are up to date"
     else
-      warn "✗ #{LLMS_FULL_PATH} is out of date with README/docs."
+      warn "✗ Out of date with README/docs: #{stale.keys.join(", ")}"
       warn "  Run `bundle exec rake llms:build` and commit the result."
       exit 1
     end
@@ -86,7 +45,7 @@ namespace :llms do
 end
 
 # Top-level convenience alias.
-desc "Regenerate llms-full.txt (alias for llms:build)"
+desc "Regenerate the LLM docs bundles (alias for llms:build)"
 task llms: "llms:build"
 
 task default: %i[spec rubocop llms:check]

data/docs/client.md

@@ -59,23 +59,125 @@ contact.raw
 # => {"ContactID" => "…", "ContactPersons" => [...], "SomeNewField" => "…"}
 ```
 
-Use it to reach fields kiwi doesn't model yet, or to store what Xero sent
-verbatim.
-
-Three things to know:
-
+Use it to reach fields kiwi doesn't model yet.
+
+Five things to know:
+
+- **It's the item, not the envelope.** `contact.raw` has no `"Contacts"`
+  key and `organisation.raw` has no `"Organisations"` key — kiwi unwraps
+  the envelope before building a resource, so there's nothing left of it by
+  the time `raw` is populated. If you need the envelope back, you're
+  rebuilding it yourself.
+- **Nested objects return `nil`, even with `retain_raw: true`.** Line
+  items, addresses, contact persons and payment terms do not carry their
+  own payload. Reach them through the enclosing resource's hash:
+
+  ```ruby
+  org.addresses.map(&:raw)   # => [nil]   ← not what you want
+  org.raw["Addresses"]       # => [{"AddressType" => "POBOX", …}]
+  ```
+
+  This is deliberate — see [why nested objects don't carry
+  raw](#why-nested-objects-dont-carry-raw) — but it fails quietly, so it's
+  worth knowing before you map over a collection and store the result.
+- **It's the JSON representation.** Kiwi sends
+  `Accept: application/json`. Xero also serves XML, which nests
+  differently — XML has no arrays, so a single child parses to a Hash and
+  several to an Array, where JSON is always an Array. `raw` cannot
+  reproduce an XML-derived shape. See [migrating from an XML
+  client](#migrating-from-an-xml-based-client) below.
 - **`#raw` is not `#to_h`.** `to_h` is a snake_case projection rebuilt from
   the modelled attributes — different keys, different nesting. If you store
   `to_h` where you meant to store the payload, readers fail by returning
   nil rather than raising.
-- **It's top-level only.** Nested objects (line items, addresses, contact
-  persons) have no `#raw` of their own. They don't need one: the top-level
-  hash already holds every nested payload, so `contact.raw["ContactPersons"]`
-  gets there.
 - **It costs memory.** Every resource holds its source hash alongside the
   hydrated attributes, which roughly doubles the footprint of a large page.
   That's why it's off by default.
 
+### What `raw` guarantees
+
+Depend on these; they won't change without a major version.
+
+- It is the parsed JSON for that resource, with **only the top-level keys
+  stringified**. Everything nested is exactly what `JSON.parse` produced.
+- **Kiwi does not normalise it.** No key renaming, no defaulting, no
+  filling in of absent keys. If Xero omits `Addresses`, `raw` has no
+  `"Addresses"` key — it does not become `[]`. If Xero sends `[]`, you get
+  `[]`. `raw.fetch("Addresses", [])` and `raw["Addresses"] || []` are both
+  safe and mean the same thing here.
+- Keys the gem doesn't model are preserved. That's the point of it.
+- The top-level hash is **frozen; nested structures are not**. `raw` itself
+  can't be mutated, but `raw["Addresses"]` is an ordinary mutable Array.
+  Dup before mutating if that matters to you.
+
+The one thing deliberately *not* guaranteed is the presence of `raw` on
+nested objects, per the bullet above.
+
+### Why nested objects don't carry raw
+
+Two reasons, both worth knowing if you're tempted to ask for it.
+
+It would cost real memory on the workload `retain_raw` exists for. A page
+of 1,000 invoices with five line items each retains about 3.4 MB with
+top-level raw; giving every nested object its own hash adds about 1.6 MB
+on top, roughly half again, and that's on a flag that has already doubled
+your footprint.
+
+More importantly it couldn't be done consistently. Several attributes —
+`payment_terms` on Contact and Organisation, `invoice_addresses` on
+Invoice — hydrate through custom lambdas that construct their objects
+directly, so they'd stay `nil` while their siblings worked. "Nested
+objects never carry raw" is a rule you can hold in your head. "Nested
+objects carry raw except these four" is not.
+
+### Migrating from an XML-based client
+
+If you're replacing a Xero client that sent `Accept: text/xml` — HTTParty
+and similar default to it — any payloads you already have stored are
+XML-shaped, and `raw` will not match them. The differences are structural,
+not cosmetic:
+
+| | XML (`text/xml`) | JSON (`application/json`) |
+|---|---|---|
+| Organisation body | `{"Organisations" => {"Organisation" => {…}}}` | `{"Organisations" => [{…}]}` |
+| One address | `{"Address" => {…}}` | `[{…}]` |
+| Several addresses | `{"Address" => [{…}, {…}]}` | `[{…}, {…}]` |
+
+The single-child collapse is the one that catches people: under XML a
+contact with one person parses to a Hash and a contact with two parses to
+an Array, from the same endpoint. Code written against that has a
+normalising step somewhere, whether or not its author knew why.
+
+No client setting reproduces these shapes — they're artefacts of an XML
+parse kiwi doesn't do. But the remedy isn't the same everywhere, and it's
+worth sorting your readers into two piles before planning the work.
+
+**Readers that dig the XML-only structure have to change.** Something like
+`dig("Organisations", "Organisation", "Addresses", "Address")` returns nil
+against anything kiwi produces, whether you store `raw` or `to_h`.
+Promote the fields those readers need to real columns and use `raw` for
+the backfill — it's the true payload, so it carries everything required to
+populate them.
+
+**Readers that already normalise usually survive untouched.** A reader
+doing `[value].flatten.compact` handles the Hash case, the Array case and
+nil identically, so a JSON array flows straight through. If the keys it
+reads are the same in both representations, only the *writer* changes:
+stop unwrapping, store the array. Existing rows stay readable, and you
+skip a migration entirely.
+
+Type coercion is usually fine in that second pile too. XML gives you
+strings — `"false"` rather than `false` — and if the reader hands that to
+something like ActiveModel's boolean cast, both the old string and the new
+real boolean land on the same value.
+
+One piece of luck worth knowing about: the two piles tend to fail
+differently. A `dig` that misses returns nil and writes a blank record
+quietly. Code that assumed a Hash, such as `Array#to_h` on what is now a
+list, raises `TypeError` on the first record with data in it. The noisy
+failures are the ones you can trust to find themselves — budget your
+review time for the silent ones.
+
 ## What the client gives you
 
 | Method | Returns | Purpose |

data/lib/xero_kiwi/accounting/resource.rb

@@ -135,9 +135,32 @@ module XeroKiwi
         end
       end
 
-      # Xero's response hash for this resource, exactly as it arrived, or nil
-      # unless the client was built with `retain_raw: true`. Use it to reach
-      # fields the gem doesn't model, or to store the payload verbatim.
+      # This resource's own hash from Xero's JSON response, exactly as it
+      # arrived, or nil unless the client was built with `retain_raw: true`.
+      # Use it to reach fields the gem doesn't model.
+      #
+      # Returns nil on NESTED objects even when retain_raw is on — line
+      # items, addresses, contact persons, payment terms. Reach those
+      # through the enclosing resource instead:
+      #
+      #   org.addresses.map(&:raw)  # => [nil]
+      #   org.raw["Addresses"]      # => [{"AddressType" => "POBOX", ...}]
+      #
+      # Scope is the item, NOT the enclosing envelope: `contact.raw` has no
+      # "Contacts" key, and `organisation.raw` no "Organisations" key. Kiwi
+      # unwraps the envelope before a resource is built, so there is nothing
+      # left of it by the time this is populated.
+      #
+      # Kiwi never normalises what it stores here: absent keys stay absent
+      # rather than becoming [] or nil entries, and keys the gem doesn't
+      # model are preserved. The hash itself is frozen; the structures
+      # nested inside it are not.
+      #
+      # It is also the JSON representation specifically. Kiwi sends
+      # `Accept: application/json`; Xero's XML representation nests
+      # differently — no arrays, so one child is a Hash and several are an
+      # Array — and `raw` cannot reproduce that shape. Worth knowing when
+      # migrating off an XML-based Xero client with stored payloads.
       #
       # Note `to_h` is NOT this — it's a snake_case projection rebuilt from
       # the modelled attributes, with different keys and different nesting.

data/lib/xero_kiwi/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module XeroKiwi
-  VERSION = "0.5.0"
+  VERSION = "0.5.2"
 end