familia 2.9.1 → 2.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d24c3b38092f1192f8e250553e8ef4d51b05c1bb00c48dfd7f6520d3a48a9a0e
-  data.tar.gz: 1dd0a2aa47682736209f116adf378eb4a0eccafffd7c151b02a8ef4573e52551
+  metadata.gz: ab706939f766966f0471ff1285db5def4d14bfc9b0d428c5b0caf481595d57c2
+  data.tar.gz: 6d08805e52f5acd4a90fc117bb8a6b2e352c036daa9916c0a19079af827ddfec
 SHA512:
-  metadata.gz: 10d69edb30370c7dba83b387f53429878fded6bf736e04fbefec4c228baf375806eeefa6c1d44c45c296e5d79b82345d1bc9cfbb0992ee18fc4c204dac250e10
-  data.tar.gz: 9b52e601ae93d44a6db8b0f995828ae45d55872377a471658a8d63210607c56f70958e372654ac20a13b07344e9b9da0afcdaaa50203f0796326d5d08e93bf6b
+  metadata.gz: b328108ed4793a4c94ddd1ef1447e759bd369647fc81aca87bafe92eb1f23a71a53ca1d763726da5a631af19e520966075bb65ab40ff29d4a4563ac1a4b1e34a
+  data.tar.gz: 652d0a36301b9321eba9225ec658f3d72b0aa449bc0133b038055900475fe546fca36e2df2133680f69a938409afd6b637d0541be44dd956f51cf2cb50b205b2

data/.github/workflows/claude-code-review.yml

@@ -43,7 +43,7 @@ jobs:
             - Security concerns
             - Test coverage
 
-            Use the repository's CLAUDE.md for guidance on style and conventions. Be constructive and helpful in your feedback.
+            Use the repository's AGENTS.md for guidance on style and conventions. Be constructive and helpful in your feedback.
 
             Use `gh pr comment` with your Bash tool to leave your review as a comment on the PR.
 

data/.gitignore

@@ -30,5 +30,5 @@ public/
 # Exclusions
 !README.md
 !CHANGELOG.md
-!CLAUDE.md
+!AGENTS.md
 !LICENSE.txt

data/AGENTS.md

@@ -0,0 +1,198 @@
+# AGENTS.md
+
+Guidance for AI coding agents working in this repository.
+
+## Development Commands
+
+- **Install**: `bundle install`
+- **Docs**: `bundle exec yard`
+- **Lint**: `bundle exec rubocop`
+- **Test**: `bundle exec try` (auto-discovers `*_try.rb` / `*.try.rb`)
+
+### Testing (Tryouts v3)
+
+Each file has optional setup, testcases, and optional teardown. A testcase is a
+`##` description line, Ruby code, then one or more expectation comments
+(`#=>`, `#==>`, `#=:>`, `#=!>`, ...). The last expression is the result.
+Instance variables (`@var`) persist across sections; locals do not. Write plain
+realistic code; avoid mocks and test DSL.
+
+Run with `--agent` for token-efficient output (`--agent-focus summary|first-failure|critical`).
+See `bundle exec try --help` for the full CLI, framework integration (`--rspec`,
+`--minitest`), and debugging flags.
+
+### Changelog
+
+Add a changelog fragment (RST) with each user-facing change. See @changelog.d/README.md
+
+### Known Issues & Quirks
+
+- **Reserved field names**: `ttl`, `db`, `valkey`, `redis` cannot be field names — use prefixed alternatives.
+- **Empty identifiers**: Cause a stack overflow in key generation — validate before operations.
+- **Lazy initialization**: Connection chains and field collections initialize lazily without synchronization (generally safe under the GIL, not guaranteed).
+
+### Debugging
+
+Ask the user for real-time database command monitoring (commands with timestamps
+and database numbers, live) when debugging multi/exec, pipelining, or
+`logical_database` issues.
+
+## Architecture
+
+**Familia** is a Valkey-compatible ORM providing Ruby object storage with
+expiration, safe dumping, and quantization.
+
+### Core Classes
+
+- **`Familia::Horreum`** (`lib/familia/horreum.rb`) — base class for Valkey-backed objects (ActiveRecord-like). Field definitions, data type relationships, lifecycle.
+- **`Familia::DataType`** (`lib/familia/data_type.rb`) — base for type wrappers (String, JsonStringKey, List, UnsortedSet, SortedSet, HashKey). Each type in `lib/familia/data_type/types/`.
+- **`Familia::Base`** (`lib/familia/base.rb`) — shared module for both, hosts the feature system.
+
+Features (Expiration, SafeDump, Relationships, ...) are modules mixed into
+classes via `Familia::Base`. See `lib/familia/features/`.
+
+### Defining a Model
+
+```ruby
+class User < Familia::Horreum
+  field :email        # scalar field
+  list :sessions      # Valkey/Redis list
+  set :tags           # set
+  zset :metrics       # sorted set
+  hashkey :settings   # hash
+end
+```
+
+Identifier strategies:
+
+```ruby
+identifier_field :email                      # symbol
+identifier ->(user) { "user:#{user.email}" } # proc
+identifier [:type, :email]                    # array
+```
+
+Connection handling lives in `lib/familia/connection.rb` and `lib/familia/settings.rb`;
+select databases with the `logical_database` class method (URI configuration supported).
+
+### Initialization: do not override `initialize` without `super`
+
+Familia's `initialize` sets fields from kwargs, then sets up DataType objects,
+then calls your `init` hook. Overriding `initialize` without `super` breaks
+related-field setup.
+
+Apply defaults in the `init` hook with `||=` (never `=`, which would overwrite
+values Horreum already set from kwargs):
+
+```ruby
+class User < Familia::Horreum
+  field :objid
+  field :email
+
+  def init
+    @objid ||= SecureRandom.uuid
+  end
+end
+
+User.new(email: 'test@example.com').objid # => generated UUID
+```
+
+Only override `initialize` (with `super`) when you must transform arguments
+before Horreum processes them.
+
+## Serialization
+
+Horreum fields are JSON-encoded for storage and JSON-decoded on load, preserving
+Ruby types (Integer, Boolean, String, Float, Hash, Array, nil). `false` and `0`
+are preserved; only `nil` values are omitted from storage.
+
+| Context | Serialize | Ruby `"UK"` stored as | Ruby `123` stored as |
+|---|---|---|---|
+| Horreum `field` | `serialize_value` (JSON) | `"\"UK\""` | `"123"` |
+| `StringKey` | `.to_s` (raw) | `"UK"` | `"123"` |
+| `JsonStringKey` | JSON dump | `"\"UK\""` | `"123"` |
+| List/Set/SortedSet/HashKey values | `serialize_value` (JSON) | `"\"UK\""` | `"123"` |
+
+`StringKey` uses raw `.to_s` (not JSON) to support `INCR`/`DECR`/`APPEND`; a
+Horreum string field stores `"UK"` as `"\"UK\""` while a `StringKey` stores it as
+`"UK"`. Use `instance.debug_fields` to compare Ruby values vs stored JSON.
+
+Database keys are generated as `classname:identifier:fieldname` (aka dbkey).
+DataType instances are frozen after instantiation.
+
+## Write Model: Deferred vs Immediate
+
+**Scalar fields** (`field`) use deferred writes: normal setters
+(`user.name = "Alice"`) only touch memory until `save`/`commit_fields`/`batch_update`.
+Fast writers (`user.name! "Alice"`) do an immediate `HSET`.
+
+**Collection fields** (`list`, `set`, `zset`, `hashkey`) use immediate writes:
+every mutator (`add`, `push`, `remove`, `clear`, `[]=`) hits Redis right away.
+Collections live on separate keys from the object hash.
+
+**Safe pattern — scalars first, then collections:**
+
+```ruby
+# Option A: explicit save, then mutate collections directly
+plan.name = "Premium"
+plan.save  # HMSET for scalar fields
+plan.features.clear
+plan.features.add("sso")
+
+# Option B: convenience wrapper (calls save internally, then yields the block)
+plan.name = "Premium"
+plan.save_with_collections do
+  plan.features.clear
+  plan.features.add("sso")
+end
+```
+
+Mutating collections before `save` is unsafe: if `save` raises, the collections
+are already mutated.
+
+**Atomic pattern — scalars and collections in one MULTI/EXEC:**
+
+```ruby
+plan.atomic_write do
+  plan.name = "Premium"   # deferred: queued as HMSET
+  plan.features.clear     # immediate: queued as DEL in the open MULTI
+  plan.features.add("sso")
+end
+```
+
+`atomic_write` composes the `transaction` infrastructure so every command lands
+in one MULTI/EXEC; collection mutations auto-route into the open transaction via
+`Fiber[:familia_transaction]`. Constraints:
+
+- All related DataTypes must share the parent's `logical_database`, else `Familia::CrossDatabaseError` (fall back to `save_with_collections`). MULTI/EXEC is single-database only.
+- Cannot nest inside another `transaction`/`atomic_write` (`Familia::OperationModeError`).
+- Collection return values inside the block are `Redis::Future` — do not inspect before EXEC.
+
+**Factory — `build` for create-and-populate:**
+
+```ruby
+user = User.build(email: "alice@example.com") do |u|
+  u.name = "Alice"        # deferred scalar
+  u.tags.add("admin")     # folded into the same MULTI
+end
+```
+
+`build` is class-level sugar over `new` + `atomic_write` with create-only
+semantics: raises `RecordExistsError` if the identifier exists, same
+single-database constraint. Without a block it degenerates to `new(...).save`.
+For upsert, use `save`/`save_with_collections`.
+
+## Instances Timeline
+
+Every Horreum subclass has a class-level `instances` sorted set — a timeline of
+last-write timestamps (ZADD score), not a registry.
+
+- **Touch** (`touch_instances!`): `save`/`save_if_not_exists!` (via `persist_to_storage`), `commit_fields`, `batch_update`, `save_fields`, fast writers.
+- **Remove**: instance `destroy!` (`remove_from_instances!`), class `destroy!(id)`, lazy `cleanup_stale_instance_entry` in `find_by_dbkey`.
+- **Ghosts**: a hash key expiring via TTL leaves a stale identifier in `instances`. `find_by_dbkey` prunes on access; raw enumeration (`instances.members`) still sees ghosts.
+- **`in_instances?(id)`** — fast O(log N), may report ghosts or miss non-Familia objects. **`exists?(id)`** — authoritative hash-key check (round-trip). `load`/`find_by_id` read the hash key directly and bypass `instances`.
+
+## Thread Safety
+
+DataType instances are frozen (immutable). Configure module-level settings once
+at startup, before threads spawn. `Familia.start_monitoring!` tracks contention.
+Tests and contention patterns live in `try/thread_safety/`.

data/CHANGELOG.rst

@@ -7,6 +7,155 @@ The format is based on `Keep a Changelog <https://keepachangelog.com/en/1.1.0/>`
 
    <!--scriv-insert-here-->
 
+.. _changelog-2.10.0:
+
+2.10.0 — 2026-06-04
+====================
+
+Added
+-----
+
+- ``Horreum.build``: A factory block that yields a new instance, then commits
+  all scalar and collection changes in a single ``MULTI/EXEC`` upon exit.
+  This avoids sequencing ``save`` before collection writes. Raises
+  ``Familia::RecordExistsError`` if the identifier exists (create-only).
+  Without a block, it behaves as ``new(...).save``. #279
+
+- ``atomic_write`` now supports ``watch_keys:`` (keys to watch) and
+  ``pre_check:`` (a callable run between ``WATCH`` and ``MULTI``) to enable
+  optimistic locking. Retries with exponential backoff on abort. #288
+
+- ``encrypted_field`` now accepts a ``key_material:`` proc. This mixes
+  additional entropy into key derivation (separate from AAD), requiring
+  the correct material at decryption to avoid producing garbage output. PR #280
+
+- Encrypted-field envelopes now store their own ``envelope_version`` and
+  ``aad_fields`` list. Decryption rebuilds AAD from these stored fields
+  rather than the active class declaration, preventing breakage when model
+  definitions change. PR #280
+
+- ``DatabaseLogger.capture_enabled`` (Boolean, default ``true``) controls
+  in-memory buffer capturing. Disabling it bypasses clock checks, message
+  allocations, and buffer appends, offering a zero-overhead production path. Issue #233
+
+- ``Familia::Instrumentation.hooks?(type)`` reports whether hooks are
+  registered for a given event type (e.g. ``:command``, ``:pipeline``). Issue #233
+
+- ``Familia.reset_trace!`` clears the cached trace environment lookup. Issue #233
+
+- ``dirty_write_warnings`` class method configures write-order warnings per
+  class (inheritable). Accepts ``:strict``, ``:warn``, ``:once``, or ``:off``. Issue #277
+
+- ``Familia.dirty_write_warnings`` global setting providing the default mode for
+  classes that do not set their own. Issue #277
+
+- ``Familia.raise_on_unsaved_parent_write`` (default ``true``) controls whether a
+  collection write on a new, unsaved, dirty parent raises or warns. Issue #278
+
+Changed
+-------
+
+- Mutating a collection on a *new, unsaved* parent Horreum now **raises**
+  ``Familia::Problem`` by default. The guard fires *before* the command runs,
+  preventing orphaned data. Save the parent first, or set
+  ``Familia.raise_on_unsaved_parent_write = false`` to restore warnings. Issue #278
+
+- Dirty-write warnings are now **deduplicated per dirty window** (mode ``:once``).
+  Writing to a collection on a parent with unsaved scalar fields warns once per
+  distinct set of unsaved fields instead of on every write. Set
+  ``dirty_write_warnings :warn`` to restore the old behavior. Issue #277
+
+- Dirty-write warnings and strict raises now append the hint:
+  ``(call #save first or wrap in atomic_write)``. Issue #277
+
+- ``trace_enabled?`` now caches the ``FAMILIA_TRACE`` lookup. Use
+  ``Familia.reset_trace!`` to force a re-read of the environment. Issue #233
+
+- ``unique_index`` hashkeys now store identifiers as raw strings rather than
+  JSON-encoded strings. Rebuild existing unique indexes to convert legacy entries,
+  e.g., via ``User.rebuild_email_lookup`` or ``company.rebuild_badge_index``. Issue #276
+
+Fixed
+-----
+
+- ``Horreum.build`` with a block no longer has a TOCTOU race between the
+  ``exists?`` check and the ``atomic_write`` commit. The block path now uses
+  ``atomic_write(watch_keys:, pre_check:)`` so the existence check runs between
+  ``WATCH`` and ``MULTI``. #288
+
+- ``aad_fields`` containing a ``transient_field`` now bind to the field's real
+  value. Previously ``build_aad`` called ``RedactedString#to_s``, which returns
+  ``"[REDACTED]"`` for every value -- so all passphrases produced identical AAD
+  and the binding was defeated. PR #280
+
+- ``each_record`` now works on ``unique_index`` hashkeys. Previously it raised
+  ``Familia::Problem`` because ``unique_index`` created its backing hashkey
+  without the ``class:`` option. Issue #276
+
+- ``each_record`` extracts the stored identifier (the hash *value*) from a
+  HashKey instead of the indexed field (the hash *key*). Issue #276
+
+- The unguarded ``Familia.trace`` sites in ``Horreum#destroy!`` and
+  ``find_by_dbkey`` now carry an inline ``if Familia.debug?`` guard. Issue #233
+
+- Two latent encryption bugs surfaced while repairing the examples (issue #250):
+
+  - ``Familia::Encryption.with_request_cache`` and ``clear_request_cache!``
+    were unreachable. The implementation lived in
+    ``lib/familia/encryption/request_cache.rb``, which was never ``require``\ d.
+    The file is now loaded with the rest of the encryption stack.
+
+  - The XChaCha20-Poly1305 provider derived keys with
+    ``context.force_encoding('BINARY')``, mutating the caller's string. A
+    frozen context raised ``FrozenError``. It now uses ``context.b``.
+
+Security
+--------
+
+- The ``aad_fields`` transient-field fix changes AAD output for any field that
+  lists a ``transient_field``. Values encrypted by an earlier release using a
+  transient field in ``aad_fields`` were bound to ``"[REDACTED]"`` and will no
+  longer decrypt after upgrading. Re-encrypt affected values if any exist.
+  PR #280
+
+Documentation
+-------------
+
+- Repaired every script in ``examples/`` so each runs top-to-bottom and is
+  re-runnable (issue #250). Added ``try/integration/examples/`` with one
+  subprocess-driven tryouts file per example script for automated regression
+  coverage.
+
+- ``Horreum.create!``: added ``@yield``, ``@yieldparam``, and
+  ``@yieldreturn`` YARD tags documenting the post-success block semantics. #286
+
+- ``Horreum#save``: added ``@example`` tags showing idiomatic Ruby patterns
+  for post-save callbacks (``if save`` and ``&&`` short-circuit). #286
+
+- Renamed ``CLAUDE.md`` to ``AGENTS.md`` and pruned it to remove volatile
+  content better served by its source of truth. Kept the non-obvious behavioral
+  contracts like deferred-vs-immediate write model and the serialization table.
+
+AI Assistance
+-------------
+
+- AI implemented ``build`` factory block (#279) and WATCH composition in
+  ``atomic_write`` (#288), including tryouts for both.
+
+- AI refactored encryption envelope handling (#280): unified AAD construction
+  through ``EncryptedData``, added envelope versioning, and fixed the
+  transient-field AAD bypass.
+
+- AI implemented ``DatabaseLogger.capture_enabled`` toggle and middleware
+  consolidation (#233), per-class ``dirty_write_warnings`` (#277), and
+  unsaved-parent guard (#278) with tryouts for each.
+
+- AI diagnosed and fixed ``each_record`` on ``unique_index`` hashkeys (#276)
+  and repaired all example scripts with regression tryouts (#250).
+
+- AI evaluated and rejected ``save_and_then`` (#286) after cross-ORM analysis;
+  added YARD docs and ``create_block_try.rb`` instead.
+
 .. _changelog-2.9.1:
 
 2.9.1 — 2026-05-18

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    familia (2.9.1)
+    familia (2.10.0)
       concurrent-ruby (~> 1.3)
       connection_pool (>= 2.4, < 4.0)
       csv (~> 3.3)
@@ -67,7 +67,7 @@ GEM
       pp (>= 0.6.0)
       rdoc (>= 4.0.0)
       reline (>= 0.4.2)
-    json (2.15.1)
+    json (2.15.2.1)
     json-schema (6.2.0)
       addressable (~> 2.8)
       bigdecimal (>= 3.1, < 5)

data/README.md

@@ -1,4 +1,4 @@
-# Familia - 2.5
+# Familia - v2
 
 **Organize and store Ruby objects in Valkey/Redis using native database types (an ORM of sorts).**
 
@@ -56,7 +56,7 @@ The performance characteristics you rely on in Valkey/Redis remain unchanged. Se
 
 ```bash
 # Add to Gemfile
-gem 'familia', '~> 2.5'
+gem 'familia', '~> 2.10'
 
 # Or install directly
 gem install familia

data/docs/guides/datatype-collections.md

@@ -7,7 +7,7 @@ UnsortedSet, Sorted Set, List, and Hash data types all include the `Collection`
 
 ## Bulk writes — single round-trip mutations
 
-Collection mutations are **immediate** — every call hits Valkey/Redis right away, unlike scalar `field` setters which are deferred until `save`. Each call also runs `warn_if_dirty!` and cascades expiration. (See the write-model notes in `CLAUDE.md` for the deferred-vs-immediate split.)
+Collection mutations are **immediate** — every call hits Valkey/Redis right away, unlike scalar `field` setters which are deferred until `save`. Each call also runs `warn_if_dirty!` and cascades expiration. (See the write-model notes in `AGENTS.md` for the deferred-vs-immediate split.)
 
 Multi-element adds issue **one** command for the whole batch, not one per element. Populating a large collection is therefore a single round-trip even without an explicit pipeline.
 

data/docs/guides/getting-started.md

@@ -0,0 +1,87 @@
+# Getting Started with Familia
+
+This guide covers the core mental models you need to work effectively with Familia. If you're coming from ActiveRecord or another ORM, understanding these differences upfront will save debugging time.
+
+## DataTypes: Live Proxies, Not Cached Relations
+
+DataTypes (`list`, `set`, `zset`, `hashkey`) are *live proxies* to Redis keys, not cached relation objects. This is the most important conceptual difference from ActiveRecord.
+
+| Aspect | ActiveRecord Relation | Familia DataType |
+|--------|----------------------|------------------|
+| Object identity | New object per query | Same object every call (memoized) |
+| Data caching | Can memoize loaded records | No cache — every read hits Redis |
+| Mutability | Mutable (chainable) | Frozen at creation |
+
+```ruby
+# ActiveRecord: new relation object each time, can cache results
+User.where(active: true).object_id != User.where(active: true).object_id
+
+# Familia: same frozen wrapper, always hits Redis
+User.instances.object_id == User.instances.object_id  # true
+User.instances.frozen?                                 # true
+User.instances.to_a  # hits Redis now
+User.instances.to_a  # hits Redis again
+```
+
+### Why This Matters
+
+**Testing**: Class-level DataTypes (like `instances`) are frozen for thread safety. Attempting `define_singleton_method` on them raises `FrozenError`. To stub behavior in tests, stub the class method that returns the DataType, not the DataType instance itself:
+
+```ruby
+# Won't work — raises FrozenError
+User.instances.define_singleton_method(:member?) { |_| true }
+
+# Works — stub the class method
+allow(User).to receive(:instances).and_return(mock_sorted_set)
+
+# Or stub a method on the class that uses instances
+allow(ApiConfig).to receive(:delete_for_domain!).and_return(true)
+```
+
+**Performance**: Since every read hits Redis, batch operations when possible:
+
+```ruby
+# Inefficient: N Redis calls
+ids.each { |id| User.instances.member?(id) }
+
+# Better: single pipeline
+User.dbclient.pipelined do
+  ids.each { |id| User.instances.member?(id) }
+end
+```
+
+## Scalar Fields vs Collection Fields
+
+Familia has a two-tier write model. Understanding when data hits Redis is critical.
+
+**Scalar fields** (`field`) use deferred writes:
+- Setters update in-memory only until `save` is called
+- Fast writers (`field_name!`) write immediately
+
+**Collection fields** (`list`, `set`, `zset`, `hashkey`) use immediate writes:
+- Every mutating method executes the Redis command right away
+- Cannot be rolled back if a subsequent operation fails
+
+```ruby
+# Safe pattern: scalars first, then collections
+plan.name = "Premium"
+plan.save
+
+plan.features.clear      # immediate Redis DEL
+plan.features.add("sso") # immediate Redis SADD
+
+# Or use atomic_write for all-or-nothing
+plan.atomic_write do
+  plan.name = "Premium"
+  plan.features.clear
+  plan.features.add("sso")
+end
+```
+
+See [Transaction Safety](../transaction_safety.md) for details.
+
+## Next Steps
+
+- [Field System](field-system.md) — field definitions and types
+- [Feature System](feature-system.md) — modular capabilities
+- [DataType Collections](datatype-collections.md) — working with lists, sets, and hashes

data/docs/guides/index.md

@@ -9,6 +9,10 @@ Welcome to the comprehensive documentation for Familia v2.0. This guide collecti
 
 ## 📚 Guide Structure
 
+### 🚀 Getting Started
+
+0. **[Getting Started](getting-started.md)** - Mental models and key concepts for developers new to Familia
+
 ### 🏗️ Architecture & System Design
 
 1. **[Feature System](feature-system.md)** - Modular architecture with dependencies and autoloader patterns