familia 2.9.0 → 2.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 42bbbbcb737ab4222505955e5b1af2ab72ed962ba80a02cf324206b4f2379b94
-  data.tar.gz: 331b1d0bb0808618a87d962e1b09af61a31d59b7b8d56b0f49513cb9f3fec63d
+  metadata.gz: ab706939f766966f0471ff1285db5def4d14bfc9b0d428c5b0caf481595d57c2
+  data.tar.gz: 6d08805e52f5acd4a90fc117bb8a6b2e352c036daa9916c0a19079af827ddfec
 SHA512:
-  metadata.gz: 9ff40710ce23c2f3dadbfbf68142800b8f10590c898f30f424819a4f0efea9aa545c52307c487c6fd96bf0c0f95f7504e5614056a30039f75f9db84fe1fcd595
-  data.tar.gz: be6c618b3fab3eb5ac8577c8a4bd7fbbbc53ce300da750baf3f64d70807a0aeb2a3a7a7f28da91d7637ebe86d9f2ccc86b677478ed90ab3ae7878d6d8816cd09
+  metadata.gz: b328108ed4793a4c94ddd1ef1447e759bd369647fc81aca87bafe92eb1f23a71a53ca1d763726da5a631af19e520966075bb65ab40ff29d4a4563ac1a4b1e34a
+  data.tar.gz: 652d0a36301b9321eba9225ec658f3d72b0aa449bc0133b038055900475fe546fca36e2df2133680f69a938409afd6b637d0541be44dd956f51cf2cb50b205b2

data/.github/workflows/claude-code-review.yml

@@ -43,7 +43,7 @@ jobs:
             - Security concerns
             - Test coverage
 
-            Use the repository's CLAUDE.md for guidance on style and conventions. Be constructive and helpful in your feedback.
+            Use the repository's AGENTS.md for guidance on style and conventions. Be constructive and helpful in your feedback.
 
             Use `gh pr comment` with your Bash tool to leave your review as a comment on the PR.
 

data/.gitignore

@@ -30,5 +30,5 @@ public/
 # Exclusions
 !README.md
 !CHANGELOG.md
-!CLAUDE.md
+!AGENTS.md
 !LICENSE.txt

data/AGENTS.md

@@ -0,0 +1,198 @@
+# AGENTS.md
+
+Guidance for AI coding agents working in this repository.
+
+## Development Commands
+
+- **Install**: `bundle install`
+- **Docs**: `bundle exec yard`
+- **Lint**: `bundle exec rubocop`
+- **Test**: `bundle exec try` (auto-discovers `*_try.rb` / `*.try.rb`)
+
+### Testing (Tryouts v3)
+
+Each file has optional setup, testcases, and optional teardown. A testcase is a
+`##` description line, Ruby code, then one or more expectation comments
+(`#=>`, `#==>`, `#=:>`, `#=!>`, ...). The last expression is the result.
+Instance variables (`@var`) persist across sections; locals do not. Write plain
+realistic code; avoid mocks and test DSL.
+
+Run with `--agent` for token-efficient output (`--agent-focus summary|first-failure|critical`).
+See `bundle exec try --help` for the full CLI, framework integration (`--rspec`,
+`--minitest`), and debugging flags.
+
+### Changelog
+
+Add a changelog fragment (RST) with each user-facing change. See @changelog.d/README.md
+
+### Known Issues & Quirks
+
+- **Reserved field names**: `ttl`, `db`, `valkey`, `redis` cannot be field names — use prefixed alternatives.
+- **Empty identifiers**: Cause a stack overflow in key generation — validate before operations.
+- **Lazy initialization**: Connection chains and field collections initialize lazily without synchronization (generally safe under the GIL, not guaranteed).
+
+### Debugging
+
+Ask the user for real-time database command monitoring (commands with timestamps
+and database numbers, live) when debugging multi/exec, pipelining, or
+`logical_database` issues.
+
+## Architecture
+
+**Familia** is a Valkey-compatible ORM providing Ruby object storage with
+expiration, safe dumping, and quantization.
+
+### Core Classes
+
+- **`Familia::Horreum`** (`lib/familia/horreum.rb`) — base class for Valkey-backed objects (ActiveRecord-like). Field definitions, data type relationships, lifecycle.
+- **`Familia::DataType`** (`lib/familia/data_type.rb`) — base for type wrappers (String, JsonStringKey, List, UnsortedSet, SortedSet, HashKey). Each type in `lib/familia/data_type/types/`.
+- **`Familia::Base`** (`lib/familia/base.rb`) — shared module for both, hosts the feature system.
+
+Features (Expiration, SafeDump, Relationships, ...) are modules mixed into
+classes via `Familia::Base`. See `lib/familia/features/`.
+
+### Defining a Model
+
+```ruby
+class User < Familia::Horreum
+  field :email        # scalar field
+  list :sessions      # Valkey/Redis list
+  set :tags           # set
+  zset :metrics       # sorted set
+  hashkey :settings   # hash
+end
+```
+
+Identifier strategies:
+
+```ruby
+identifier_field :email                      # symbol
+identifier ->(user) { "user:#{user.email}" } # proc
+identifier [:type, :email]                    # array
+```
+
+Connection handling lives in `lib/familia/connection.rb` and `lib/familia/settings.rb`;
+select databases with the `logical_database` class method (URI configuration supported).
+
+### Initialization: do not override `initialize` without `super`
+
+Familia's `initialize` sets fields from kwargs, then sets up DataType objects,
+then calls your `init` hook. Overriding `initialize` without `super` breaks
+related-field setup.
+
+Apply defaults in the `init` hook with `||=` (never `=`, which would overwrite
+values Horreum already set from kwargs):
+
+```ruby
+class User < Familia::Horreum
+  field :objid
+  field :email
+
+  def init
+    @objid ||= SecureRandom.uuid
+  end
+end
+
+User.new(email: 'test@example.com').objid # => generated UUID
+```
+
+Only override `initialize` (with `super`) when you must transform arguments
+before Horreum processes them.
+
+## Serialization
+
+Horreum fields are JSON-encoded for storage and JSON-decoded on load, preserving
+Ruby types (Integer, Boolean, String, Float, Hash, Array, nil). `false` and `0`
+are preserved; only `nil` values are omitted from storage.
+
+| Context | Serialize | Ruby `"UK"` stored as | Ruby `123` stored as |
+|---|---|---|---|
+| Horreum `field` | `serialize_value` (JSON) | `"\"UK\""` | `"123"` |
+| `StringKey` | `.to_s` (raw) | `"UK"` | `"123"` |
+| `JsonStringKey` | JSON dump | `"\"UK\""` | `"123"` |
+| List/Set/SortedSet/HashKey values | `serialize_value` (JSON) | `"\"UK\""` | `"123"` |
+
+`StringKey` uses raw `.to_s` (not JSON) to support `INCR`/`DECR`/`APPEND`; a
+Horreum string field stores `"UK"` as `"\"UK\""` while a `StringKey` stores it as
+`"UK"`. Use `instance.debug_fields` to compare Ruby values vs stored JSON.
+
+Database keys are generated as `classname:identifier:fieldname` (aka dbkey).
+DataType instances are frozen after instantiation.
+
+## Write Model: Deferred vs Immediate
+
+**Scalar fields** (`field`) use deferred writes: normal setters
+(`user.name = "Alice"`) only touch memory until `save`/`commit_fields`/`batch_update`.
+Fast writers (`user.name! "Alice"`) do an immediate `HSET`.
+
+**Collection fields** (`list`, `set`, `zset`, `hashkey`) use immediate writes:
+every mutator (`add`, `push`, `remove`, `clear`, `[]=`) hits Redis right away.
+Collections live on separate keys from the object hash.
+
+**Safe pattern — scalars first, then collections:**
+
+```ruby
+# Option A: explicit save, then mutate collections directly
+plan.name = "Premium"
+plan.save  # HMSET for scalar fields
+plan.features.clear
+plan.features.add("sso")
+
+# Option B: convenience wrapper (calls save internally, then yields the block)
+plan.name = "Premium"
+plan.save_with_collections do
+  plan.features.clear
+  plan.features.add("sso")
+end
+```
+
+Mutating collections before `save` is unsafe: if `save` raises, the collections
+are already mutated.
+
+**Atomic pattern — scalars and collections in one MULTI/EXEC:**
+
+```ruby
+plan.atomic_write do
+  plan.name = "Premium"   # deferred: queued as HMSET
+  plan.features.clear     # immediate: queued as DEL in the open MULTI
+  plan.features.add("sso")
+end
+```
+
+`atomic_write` composes the `transaction` infrastructure so every command lands
+in one MULTI/EXEC; collection mutations auto-route into the open transaction via
+`Fiber[:familia_transaction]`. Constraints:
+
+- All related DataTypes must share the parent's `logical_database`, else `Familia::CrossDatabaseError` (fall back to `save_with_collections`). MULTI/EXEC is single-database only.
+- Cannot nest inside another `transaction`/`atomic_write` (`Familia::OperationModeError`).
+- Collection return values inside the block are `Redis::Future` — do not inspect before EXEC.
+
+**Factory — `build` for create-and-populate:**
+
+```ruby
+user = User.build(email: "alice@example.com") do |u|
+  u.name = "Alice"        # deferred scalar
+  u.tags.add("admin")     # folded into the same MULTI
+end
+```
+
+`build` is class-level sugar over `new` + `atomic_write` with create-only
+semantics: raises `RecordExistsError` if the identifier exists, same
+single-database constraint. Without a block it degenerates to `new(...).save`.
+For upsert, use `save`/`save_with_collections`.
+
+## Instances Timeline
+
+Every Horreum subclass has a class-level `instances` sorted set — a timeline of
+last-write timestamps (ZADD score), not a registry.
+
+- **Touch** (`touch_instances!`): `save`/`save_if_not_exists!` (via `persist_to_storage`), `commit_fields`, `batch_update`, `save_fields`, fast writers.
+- **Remove**: instance `destroy!` (`remove_from_instances!`), class `destroy!(id)`, lazy `cleanup_stale_instance_entry` in `find_by_dbkey`.
+- **Ghosts**: a hash key expiring via TTL leaves a stale identifier in `instances`. `find_by_dbkey` prunes on access; raw enumeration (`instances.members`) still sees ghosts.
+- **`in_instances?(id)`** — fast O(log N), may report ghosts or miss non-Familia objects. **`exists?(id)`** — authoritative hash-key check (round-trip). `load`/`find_by_id` read the hash key directly and bypass `instances`.
+
+## Thread Safety
+
+DataType instances are frozen (immutable). Configure module-level settings once
+at startup, before threads spawn. `Familia.start_monitoring!` tracks contention.
+Tests and contention patterns live in `try/thread_safety/`.

data/CHANGELOG.rst

@@ -7,6 +7,199 @@ The format is based on `Keep a Changelog <https://keepachangelog.com/en/1.1.0/>`
 
    <!--scriv-insert-here-->
 
+.. _changelog-2.10.0:
+
+2.10.0 — 2026-06-04
+====================
+
+Added
+-----
+
+- ``Horreum.build``: A factory block that yields a new instance, then commits
+  all scalar and collection changes in a single ``MULTI/EXEC`` upon exit.
+  This avoids sequencing ``save`` before collection writes. Raises
+  ``Familia::RecordExistsError`` if the identifier exists (create-only).
+  Without a block, it behaves as ``new(...).save``. #279
+
+- ``atomic_write`` now supports ``watch_keys:`` (keys to watch) and
+  ``pre_check:`` (a callable run between ``WATCH`` and ``MULTI``) to enable
+  optimistic locking. Retries with exponential backoff on abort. #288
+
+- ``encrypted_field`` now accepts a ``key_material:`` proc. This mixes
+  additional entropy into key derivation (separate from AAD), requiring
+  the correct material at decryption to avoid producing garbage output. PR #280
+
+- Encrypted-field envelopes now store their own ``envelope_version`` and
+  ``aad_fields`` list. Decryption rebuilds AAD from these stored fields
+  rather than the active class declaration, preventing breakage when model
+  definitions change. PR #280
+
+- ``DatabaseLogger.capture_enabled`` (Boolean, default ``true``) controls
+  in-memory buffer capturing. Disabling it bypasses clock checks, message
+  allocations, and buffer appends, offering a zero-overhead production path. Issue #233
+
+- ``Familia::Instrumentation.hooks?(type)`` reports whether hooks are
+  registered for a given event type (e.g. ``:command``, ``:pipeline``). Issue #233
+
+- ``Familia.reset_trace!`` clears the cached trace environment lookup. Issue #233
+
+- ``dirty_write_warnings`` class method configures write-order warnings per
+  class (inheritable). Accepts ``:strict``, ``:warn``, ``:once``, or ``:off``. Issue #277
+
+- ``Familia.dirty_write_warnings`` global setting providing the default mode for
+  classes that do not set their own. Issue #277
+
+- ``Familia.raise_on_unsaved_parent_write`` (default ``true``) controls whether a
+  collection write on a new, unsaved, dirty parent raises or warns. Issue #278
+
+Changed
+-------
+
+- Mutating a collection on a *new, unsaved* parent Horreum now **raises**
+  ``Familia::Problem`` by default. The guard fires *before* the command runs,
+  preventing orphaned data. Save the parent first, or set
+  ``Familia.raise_on_unsaved_parent_write = false`` to restore warnings. Issue #278
+
+- Dirty-write warnings are now **deduplicated per dirty window** (mode ``:once``).
+  Writing to a collection on a parent with unsaved scalar fields warns once per
+  distinct set of unsaved fields instead of on every write. Set
+  ``dirty_write_warnings :warn`` to restore the old behavior. Issue #277
+
+- Dirty-write warnings and strict raises now append the hint:
+  ``(call #save first or wrap in atomic_write)``. Issue #277
+
+- ``trace_enabled?`` now caches the ``FAMILIA_TRACE`` lookup. Use
+  ``Familia.reset_trace!`` to force a re-read of the environment. Issue #233
+
+- ``unique_index`` hashkeys now store identifiers as raw strings rather than
+  JSON-encoded strings. Rebuild existing unique indexes to convert legacy entries,
+  e.g., via ``User.rebuild_email_lookup`` or ``company.rebuild_badge_index``. Issue #276
+
+Fixed
+-----
+
+- ``Horreum.build`` with a block no longer has a TOCTOU race between the
+  ``exists?`` check and the ``atomic_write`` commit. The block path now uses
+  ``atomic_write(watch_keys:, pre_check:)`` so the existence check runs between
+  ``WATCH`` and ``MULTI``. #288
+
+- ``aad_fields`` containing a ``transient_field`` now bind to the field's real
+  value. Previously ``build_aad`` called ``RedactedString#to_s``, which returns
+  ``"[REDACTED]"`` for every value -- so all passphrases produced identical AAD
+  and the binding was defeated. PR #280
+
+- ``each_record`` now works on ``unique_index`` hashkeys. Previously it raised
+  ``Familia::Problem`` because ``unique_index`` created its backing hashkey
+  without the ``class:`` option. Issue #276
+
+- ``each_record`` extracts the stored identifier (the hash *value*) from a
+  HashKey instead of the indexed field (the hash *key*). Issue #276
+
+- The unguarded ``Familia.trace`` sites in ``Horreum#destroy!`` and
+  ``find_by_dbkey`` now carry an inline ``if Familia.debug?`` guard. Issue #233
+
+- Two latent encryption bugs surfaced while repairing the examples (issue #250):
+
+  - ``Familia::Encryption.with_request_cache`` and ``clear_request_cache!``
+    were unreachable. The implementation lived in
+    ``lib/familia/encryption/request_cache.rb``, which was never ``require``\ d.
+    The file is now loaded with the rest of the encryption stack.
+
+  - The XChaCha20-Poly1305 provider derived keys with
+    ``context.force_encoding('BINARY')``, mutating the caller's string. A
+    frozen context raised ``FrozenError``. It now uses ``context.b``.
+
+Security
+--------
+
+- The ``aad_fields`` transient-field fix changes AAD output for any field that
+  lists a ``transient_field``. Values encrypted by an earlier release using a
+  transient field in ``aad_fields`` were bound to ``"[REDACTED]"`` and will no
+  longer decrypt after upgrading. Re-encrypt affected values if any exist.
+  PR #280
+
+Documentation
+-------------
+
+- Repaired every script in ``examples/`` so each runs top-to-bottom and is
+  re-runnable (issue #250). Added ``try/integration/examples/`` with one
+  subprocess-driven tryouts file per example script for automated regression
+  coverage.
+
+- ``Horreum.create!``: added ``@yield``, ``@yieldparam``, and
+  ``@yieldreturn`` YARD tags documenting the post-success block semantics. #286
+
+- ``Horreum#save``: added ``@example`` tags showing idiomatic Ruby patterns
+  for post-save callbacks (``if save`` and ``&&`` short-circuit). #286
+
+- Renamed ``CLAUDE.md`` to ``AGENTS.md`` and pruned it to remove volatile
+  content better served by its source of truth. Kept the non-obvious behavioral
+  contracts like deferred-vs-immediate write model and the serialization table.
+
+AI Assistance
+-------------
+
+- AI implemented ``build`` factory block (#279) and WATCH composition in
+  ``atomic_write`` (#288), including tryouts for both.
+
+- AI refactored encryption envelope handling (#280): unified AAD construction
+  through ``EncryptedData``, added envelope versioning, and fixed the
+  transient-field AAD bypass.
+
+- AI implemented ``DatabaseLogger.capture_enabled`` toggle and middleware
+  consolidation (#233), per-class ``dirty_write_warnings`` (#277), and
+  unsaved-parent guard (#278) with tryouts for each.
+
+- AI diagnosed and fixed ``each_record`` on ``unique_index`` hashkeys (#276)
+  and repaired all example scripts with regression tryouts (#250).
+
+- AI evaluated and rejected ``save_and_then`` (#286) after cross-ORM analysis;
+  added YARD docs and ``create_block_try.rb`` instead.
+
+.. _changelog-2.9.1:
+
+2.9.1 — 2026-05-18
+==================
+
+Added
+-----
+
+- ``SortedSet#update`` (aliased ``merge!``) for bulk member insertion. A sorted
+  set is ``member => score`` -- the same pair shape as ``HashKey``'s
+  ``field => value`` -- so it follows the established ``HashKey#update``/``merge!``
+  convention (a single Hash argument) rather than the variadic splat used by the
+  value-only ``UnsortedSet``/``ListKey``. Pass ``{member => score}`` to issue one
+  ``ZADD`` instead of one round-trip per member. Validates the argument is a Hash
+  and that every score is ``Numeric`` (a missing/``nil`` score raises a clear
+  ``ArgumentError`` instead of a low-level client error -- unlike single-value
+  ``#add``, the bulk path does not default a missing score to ``Familia.now``).
+  Cascades expiration, and is a no-op returning ``0`` for empty input. The
+  single-value ``SortedSet#add`` (and its array-as-single-member contract) is
+  unchanged. PR #269
+
+Changed
+-------
+
+- Bulk-write optimization for multi-value collection mutations. ``UnsortedSet#add``,
+  ``ListKey#push``, and ``ListKey#unshift`` previously issued one Redis command per
+  element (a loop of ``SADD``/``RPUSH``/``LPUSH`` calls), making large populations
+  slow even when pipelined. They now serialize all values and issue a single bulk
+  ``SADD``/``RPUSH``/``LPUSH`` command. Element ordering, ``nil`` compaction, nested
+  array flattening, return values, dirty-write warnings, and expiration cascading
+  are unchanged; empty calls remain no-ops. PR #269
+
+AI Assistance
+-------------
+
+- AI investigated all collection ``DataType`` classes for the same per-element
+  loop anti-pattern, identified the three affected methods, verified
+  behavior-preservation (ordering, edge cases, chainability) at the Redis wire
+  level, and confirmed zero regressions against the existing test suites. The
+  ``SortedSet#update`` API shape was chosen by priority order: existing Familia
+  conventions first (the ``HashKey#update``/``merge!`` precedent for keyed
+  collections), then the upstream redis-rb bulk ``ZADD`` form, then Ruby
+  ``Hash#merge!`` semantics as confirmation.
+
 .. _changelog-2.9.0:
 
 2.9.0 — 2026-05-17

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    familia (2.9.0)
+    familia (2.10.0)
       concurrent-ruby (~> 1.3)
       connection_pool (>= 2.4, < 4.0)
       csv (~> 3.3)
@@ -67,7 +67,7 @@ GEM
       pp (>= 0.6.0)
       rdoc (>= 4.0.0)
       reline (>= 0.4.2)
-    json (2.15.1)
+    json (2.15.2.1)
     json-schema (6.2.0)
       addressable (~> 2.8)
       bigdecimal (>= 3.1, < 5)

data/README.md

@@ -1,4 +1,4 @@
-# Familia - 2.5
+# Familia - v2
 
 **Organize and store Ruby objects in Valkey/Redis using native database types (an ORM of sorts).**
 
@@ -56,7 +56,7 @@ The performance characteristics you rely on in Valkey/Redis remain unchanged. Se
 
 ```bash
 # Add to Gemfile
-gem 'familia', '~> 2.5'
+gem 'familia', '~> 2.10'
 
 # Or install directly
 gem install familia

data/docs/guides/datatype-collections.md

@@ -0,0 +1,159 @@
+# docs/guides/datatype-collections.md
+---
+
+# DataType - Collection classes
+
+UnsortedSet, Sorted Set, List, and Hash data types all include the `Collection` module. This guide covers two performance-sensitive concerns: writing many elements efficiently (a single bulk command instead of one round-trip per element), and iterating large collections efficiently via `each` and `each_record`.
+
+## Bulk writes — single round-trip mutations
+
+Collection mutations are **immediate** — every call hits Valkey/Redis right away, unlike scalar `field` setters which are deferred until `save`. Each call also runs `warn_if_dirty!` and cascades expiration. (See the write-model notes in `AGENTS.md` for the deferred-vs-immediate split.)
+
+Multi-element adds issue **one** command for the whole batch, not one per element. Populating a large collection is therefore a single round-trip even without an explicit pipeline.
+
+The argument shape follows the collection's structure, and is consistent across the codebase:
+
+- **Value-only** collections (`UnsortedSet`, `ListKey`) take a **variadic splat**; arguments are flattened and `nil`-compacted.
+- **Keyed/pair** collections (`HashKey` is `field => value`, `SortedSet` is `member => score`) take a **single Hash** via `update` (aliased `merge!`), raising `ArgumentError` on a non-Hash.
+
+| Type | Bulk method | Call shape | Redis command |
+|---|---|---|---|
+| `UnsortedSet` | `add(*values)` | `tags.add(:a, :b, :c)` | one `SADD` |
+| `ListKey` | `push(*values)` / `unshift(*values)` | `log.push(1, 2, 3)` | one `RPUSH` / `LPUSH` |
+| `HashKey` | `update(hash)` / `merge!` | `cfg.update(a: 1, b: 2)` | one `HMSET` |
+| `SortedSet` | `update(hash)` / `merge!` | `board.update("alice" => 1000, "bob" => 850)` | one `ZADD` |
+
+```ruby
+tags.add(:ruby, :redis, :valkey)              # 1 SADD, returns self
+log.push("a", "b", "c")                        # 1 RPUSH → [a, b, c]
+board.update("alice" => 1000, "bob" => 850)    # 1 ZADD, returns new-member count (2)
+board.merge!("alice" => 1200)                  # 1 ZADD, score updated → returns 0
+```
+
+Behavior notes:
+
+- **Ordering**: `push` preserves argument order; `unshift` prepends each element in turn, so `unshift(a, b, c)` leaves the list head as `c, b, a` (Redis `LPUSH` semantics — unchanged from the prior per-element implementation). Sets are unordered; sorted sets order by score.
+- **Empty input is a no-op**: `add()` / `push()` / `update({})` issue no command. Set/list adds return `self`; `SortedSet#update` returns `0`.
+- **`SortedSet#add(val, score, …)` is unchanged and not bulk** — it takes a single member plus score and the conditional ZADD options (`nx:`, `xx:`, `gt:`, `lt:`, `ch:`). An Array passed as `val` is stored as one JSON-encoded member, not exploded into many. Use `update`/`merge!` for bulk insertion.
+
+The iteration methods `each` and `each_record` efficiently handle large collections by paginating through Valkey/Redis data structures, but they serve different purposes and yield different results. Here's how the two iterate, using `ModelClass.instances` (a `SortedSet` with `reference: true`) as the running example.
+
+## `each` — yields **members** (identifiers, raw strings)
+
+`each` is implemented per type. For the `instances` SortedSet, it pages through the ZSET with either `ZRANGEBYSCORE` (when `since:`/`until:` are given) or `ZSCAN` (unbounded), yielding one deserialized member at a time.
+
+```mermaid
+flowchart TD
+  Caller["ModelClass.instances.each { |id| ... }"] --> EachImpl["SortedSet#each"]
+  EachImpl --> Decide{since/until?}
+  Decide -- yes --> ZRBS["ZRANGEBYSCORE key min max LIMIT 0 batch_size WITHSCORES"]
+  Decide -- no --> ZSCAN["ZSCAN key cursor COUNT batch_size"]
+  ZRBS --> Page["Page of raw members"]
+  ZSCAN --> Page
+  Page --> Yield["yield deserialize_value(member)"]
+  Yield --> More{more pages?}
+  More -- yes --> Decide
+  More -- no --> Done["return self"]
+```
+
+Per-type variations:
+- `ListKey#each` — paginates with `LRANGE start stop` (no SCAN equivalent)
+- `UnsortedSet#each` / `HashKey#each` — `SSCAN` / `HSCAN`, optional `matching:` glob
+- `SortedSet#each` — `ZRANGEBYSCORE` (bounded) or `ZSCAN` (unbounded)
+
+You get **identifiers only**. No record loading. One Redis round-trip per page.
+
+## `each_record` — yields **loaded Horreum records**
+
+`each_record` is defined once in `CollectionBase` and delegates to `each` to collect identifiers, then batches them into `record_class.load_multi` (pipelined `HGETALL`s), filters ghosts, and yields the live records.
+
+```mermaid
+flowchart TD
+  Caller["ModelClass.instances.each_record { |rec| ... }"] --> ER["each_record(batch_size, pipeline, **filters)"]
+  ER --> Validate{"pipeline <= batch_size?"}
+  Validate -- no --> Raise["raise ArgumentError"]
+  Validate -- yes --> CallEach["each(**filters) do |member|"]
+  CallEach --> Extract["id = member.is_a?(Array) ? member.first : member"]
+  Extract --> Buffer["buffer << id"]
+  Buffer --> Full{"buffer.size >= batch_size?"}
+  Full -- no --> CallEach
+  Full -- yes --> Load["record_class.load_multi(ids)  -- pipelined HGETALLs"]
+  Load --> Compact["live = records.compact  -- drop ghosts"]
+  Compact --> Mode{pipeline?}
+  Mode -- nil --> Serial["live.each { |r| block.call(r) }"]
+  Mode -- positive --> Pipe["live.each_slice(pipeline) do |group|<br/>record_class.pipelined { group.each &block }<br/>end"]
+  Serial --> Clear["buffer.clear; resume each"]
+  Pipe --> Clear
+  Clear --> CallEach
+  CallEach -. each exhausted .-> Flush["process_batch(buffer) if any remain"]
+  Flush --> Return["return self"]
+```
+
+### Concrete timeline for `User.instances.each_record(batch_size: 100, pipeline: 25) { |u| u.touch! }`
+
+```
+SortedSet#each (ZSCAN page 1, 100 ids)
+   ├─ buffer fills to 100
+   ├─ load_multi(ids)        → 1 pipeline of 100 HGETALLs
+   ├─ compact ghosts          → e.g. 97 live records
+   ├─ slice(25):
+   │     pipelined { 25 × u.touch! }   ← 1 Redis pipeline
+   │     pipelined { 25 × u.touch! }   ← 1 Redis pipeline
+   │     pipelined { 25 × u.touch! }   ← 1 Redis pipeline
+   │     pipelined { 22 × u.touch! }   ← 1 Redis pipeline
+   └─ buffer.clear
+SortedSet#each (ZSCAN page 2, 100 ids)
+   └─ … repeat …
+SortedSet#each exhausted
+   └─ flush any remaining buffered ids the same way
+```
+
+## Key differences
+
+| Aspect | `each` | `each_record` |
+|---|---|---|
+| Yields | raw identifier (or `[field, value]` for `HashKey`) | loaded Horreum instance |
+| Redis ops per yield | 0 extra (already paged) | amortized `HGETALL` via `load_multi` batch |
+| Requires `reference: true` + `:class` | no | yes (raises `Familia::Problem` otherwise) |
+| Ghost handling | yields the dangling id | `compact` drops them silently |
+| Write pipelining | not built-in | `pipeline:` groups block-body writes into `pipelined` blocks |
+| Filters | type-specific (`since:`, `matching:`, …) | forwarded to underlying `each` |
+
+So `each_record` is a thin orchestration layer: it leans on the type's own `each` for read pagination, then layers (1) batched record hydration and (2) optional write pipelining on top.
+
+## Choosing a `pipeline` mode
+
+`each_record` has two dispatch modes, controlled by `pipeline:`. The parameter answers a single question: **may the dispatch loop wrap your block in a `pipelined { }`?**
+
+| Value | Dispatch | Use when the block… |
+|---|---|---|
+| `nil` (default) | Each record runs in its own connection context, no pipeline wrapper | …reads, OR calls `save` / `commit_fields` / `transaction` / anything with its own internal MULTI |
+| positive integer | Groups of `pipeline` records run inside `record_class.pipelined { ... }` | …only issues fast writers (`record.field!`) that tolerate being queued |
+
+Note: `pipeline: 0` raises `ArgumentError`. Use `pipeline: nil` to disable pipelining.
+
+The read-only case and the serial-write case collapse into the same mode because both require **immediate** execution with real return values. Wrapping `save` in an outer `pipelined` would either return `Redis::Future` objects or raise `ConflictingContextError` when `save`'s internal transaction tries to open.
+
+### The three idiomatic patterns
+
+```ruby
+# 1. Read-only iteration — the default (pipeline: nil) is correct
+User.instances.each_record do |user|
+  puts "#{user.email} #{user.last_login}"
+end
+
+# 2. Serial writes — the default (pipeline: nil) is required for save / commit_fields / transaction
+User.instances.each_record do |user|
+  user.score = recompute(user)
+  user.save
+end
+
+# 3. Pipelined fast writers — opt-in optimization
+User.instances.each_record(pipeline: 50) do |user|
+  user.last_seen_at! Familia.now   # single HSET, safe to queue in pipeline
+end
+```
+
+### Pipelining footgun
+
+If you enable pipelining and your block reads from a related collection (e.g. `user.sessions.size`), that read is queued into the pipeline and returns a `Redis::Future` rather than a value. Omit the `pipeline:` parameter (or explicitly pass `pipeline: nil`) whenever the block needs real return values from Redis.