familia 2.0.0.pre19 → 2.0.0.pre21

data/CLAUDE.md

@@ -53,7 +53,7 @@ Add changelog fragment with each user-facing or documented change (optional but
 ### Known Issues & Quirks
 - **Reserved Keywords**: Cannot use `ttl`, `db`, `valkey`, `redis` as field names - use prefixed alternatives
 - **Empty Identifiers**: Cause stack overflow in key generation - validate before operations
-- **Connection Pool Race Conditions**: Thread safety issues under high concurrency
+- **Lazy Initialization Races**: Connection chains and field collections use lazy initialization without synchronization (generally safe due to Ruby GIL, but not guaranteed)
 
 ### Debugging
 - **Database command logging**: You can request real-time Database command monitoring from the user
@@ -191,3 +191,30 @@ end
 **Memory Efficiency**: Only non-nil values are stored in keystore database to optimize memory usage.
 
 **Thread Safety**: Data types are frozen after instantiation to ensure immutability.
+
+## Thread Safety Considerations
+
+### Current Thread Safety Status (as of 2025-10-21)
+
+Familia has **good thread safety** for standard multi-threaded environments:
+
+### Testing Thread Safety
+
+Thread safety tests are available in `try/thread_safety/`:
+- **100% passing** (56/56 tests)
+- **CyclicBarrier pattern** for maximum contention testing
+- **Test execution**: ~300ms for full suite with 1,000+ concurrent operations
+- **Production monitoring**: 10/10 monitoring tests passing
+
+Run thread safety tests:
+```bash
+bundle exec try --agent try/thread_safety/
+bundle exec try --agent try/unit/thread_safety_monitor_try.rb
+```
+
+### Best Practices for Thread-Safe Usage
+
+1. **Configure Once at Startup**: Module-level configuration should be set before threads spawn
+2. **Use Immutable DataTypes**: Leverage the fact that DataType instances are frozen
+3. **Test Under Concurrency**: Use the patterns in `try/thread_safety/` to verify thread safety
+4. **Enable Production Monitoring**: Use `Familia.start_monitoring!` to track contention in production

data/Gemfile

@@ -9,7 +9,7 @@ group :test do
   gem 'ruby-prof'
   gem 'stackprof'
   gem 'timecop', require: false
-  gem 'tryouts', '~> 3.6.0', require: false
+  gem 'tryouts', '~> 3.7.1', require: false
 end
 
 group :development, :test do

data/Gemfile.lock

@@ -1,8 +1,9 @@
 PATH
   remote: .
   specs:
-    familia (2.0.0.pre19)
+    familia (2.0.0.pre21)
       benchmark (~> 0.4)
+      concurrent-ruby (~> 1.3)
       connection_pool (~> 2.5)
       csv (~> 3.3)
       logger (~> 1.7)
@@ -21,7 +22,7 @@ GEM
     concurrent-ruby (1.3.5)
     connection_pool (2.5.3)
     csv (3.3.5)
-    date (3.4.1)
+    date (3.5.0)
     debug (1.11.0)
       irb (~> 1.10)
       reline (>= 0.3.8)
@@ -55,11 +56,11 @@ GEM
       dry-inflector (~> 1.0)
       dry-logic (~> 1.4)
       zeitwerk (~> 2.6)
-    erb (5.0.2)
+    erb (5.1.3)
     ffi (1.17.2)
     ffi (1.17.2-arm64-darwin)
     io-console (0.8.1)
-    irb (1.15.2)
+    irb (1.15.3)
       pp (>= 0.6.0)
       rdoc (>= 4.0.0)
       reline (>= 0.4.2)
@@ -67,7 +68,7 @@ GEM
     language_server-protocol (3.17.0.5)
     lint_roller (1.1.0)
     logger (1.7.0)
-    minitest (5.25.5)
+    minitest (5.26.0)
     oj (3.16.11)
       bigdecimal (>= 3.0)
       ostruct (>= 0.2)
@@ -78,10 +79,10 @@ GEM
       racc
     pastel (0.8.0)
       tty-color (~> 0.5)
-    pp (0.6.2)
+    pp (0.6.3)
       prettyprint
     prettyprint (0.2.0)
-    prism (1.5.2)
+    prism (1.6.0)
     psych (5.2.6)
       date
       stringio
@@ -91,9 +92,10 @@ GEM
       ffi (~> 1)
     rbs (3.9.5)
       logger
-    rdoc (6.14.2)
+    rdoc (6.15.1)
       erb
       psych (>= 4.0.0)
+      tsort
     redcarpet (3.6.1)
     redis (5.4.1)
       redis-client (>= 0.22.0)
@@ -109,19 +111,19 @@ GEM
     reline (0.6.2)
       io-console (~> 0.5)
     rexml (3.4.1)
-    rspec (3.13.1)
+    rspec (3.13.2)
       rspec-core (~> 3.13.0)
       rspec-expectations (~> 3.13.0)
       rspec-mocks (~> 3.13.0)
-    rspec-core (3.13.5)
+    rspec-core (3.13.6)
       rspec-support (~> 3.13.0)
     rspec-expectations (3.13.5)
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.13.0)
-    rspec-mocks (3.13.5)
+    rspec-mocks (3.13.7)
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.13.0)
-    rspec-support (3.13.4)
+    rspec-support (3.13.6)
     rubocop (1.81.1)
       json (~> 2.3)
       language_server-protocol (~> 3.17.0.2)
@@ -154,15 +156,16 @@ GEM
     stackprof (0.2.27)
     stringio (3.1.7)
     timecop (0.9.10)
-    tryouts (3.6.0)
-      concurrent-ruby (~> 1.0)
+    tryouts (3.7.1)
+      concurrent-ruby (~> 1.0, < 2)
       irb
       minitest (~> 5.0)
       pastel (~> 0.8)
       prism (~> 1.0)
-      rspec (~> 3.0)
+      rspec (>= 3.0, < 5.0)
       tty-cursor (~> 0.7)
       tty-screen (~> 0.8)
+    tsort (0.2.0)
     tty-color (0.6.0)
     tty-cursor (0.7.1)
     tty-screen (0.8.2)
@@ -192,8 +195,8 @@ DEPENDENCIES
   ruby-prof
   stackprof
   timecop
-  tryouts (~> 3.6.0)
+  tryouts (~> 3.7.1)
   yard (~> 0.9)
 
 BUNDLED WITH
-   2.6.2
+   2.7.2

data/bin/try

@@ -0,0 +1,16 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'try' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("tryouts", "try")

data/bin/tryouts

@@ -0,0 +1,16 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'tryouts' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("tryouts", "tryouts")

data/changelog.d/20251105_flexible_external_identifier_format.rst

@@ -0,0 +1,66 @@
+.. Added
+.. -----
+
+.. Changed
+.. -------
+
+- **ExternalIdentifier Format Flexibility**: The `external_identifier` feature now supports customizable format templates via the `format` option. This allows you to control the entire format of generated external IDs, including the prefix, separator, and overall structure.
+
+  **Default format** (unchanged behavior):
+
+  .. code-block:: ruby
+
+     class User < Familia::Horreum
+       feature :external_identifier
+     end
+     user.extid  # => "ext_abc123def456ghi789"
+
+  **Custom format with different prefix**:
+
+  .. code-block:: ruby
+
+     class Customer < Familia::Horreum
+       feature :external_identifier, format: 'cust_%{id}'
+     end
+     customer.extid  # => "cust_abc123def456ghi789"
+
+  **Custom format with different separator**:
+
+  .. code-block:: ruby
+
+     class APIKey < Familia::Horreum
+       feature :external_identifier, format: 'api-%{id}'
+     end
+     key.extid  # => "api-abc123def456ghi789"
+
+  **Custom format without traditional prefix**:
+
+  .. code-block:: ruby
+
+     class Resource < Familia::Horreum
+       feature :external_identifier, format: 'v2/%{id}'
+     end
+     resource.extid  # => "v2/abc123def456ghi789"
+
+  The `format` option accepts a Ruby format string with the `%{id}` placeholder for the generated identifier. The default format is `'ext_%{id}'`. This provides complete flexibility for various ID formatting needs including different prefixes, separators (underscore, hyphen, slash), URL paths, or no prefix at all.
+
+.. Deprecated
+.. ----------
+
+.. Removed
+.. -------
+
+.. Fixed
+.. -----
+
+.. Security
+.. --------
+
+.. Documentation
+.. -------------
+
+.. AI Assistance
+.. -------------
+
+- **Design Review**: Claude Code provided analysis of the current implementation and recommended several idiomatic Ruby approaches for format flexibility, ultimately suggesting the format template pattern using Ruby's native string formatting.
+- **Implementation**: Claude Code implemented the format template feature including code changes, test cases, and documentation updates.

data/changelog.d/20251107_112554_delano_179_participation_asymmetry.rst

@@ -0,0 +1,44 @@
+.. A new scriv changelog fragment.
+..
+.. Uncomment the section that is right (remove the leading dots).
+.. For top level release notes, leave all the headers commented out.
+..
+Added
+-----
+
+- Bidirectional reverse collection methods for ``participates_in`` with ``_instances`` suffix (e.g., ``user.project_team_instances``, ``user.project_team_ids``). Supports union behavior for multiple collections and custom naming via ``as:`` parameter. Closes #179.
+
+.. Changed
+.. -------
+..
+.. - A bullet item for the Changed category.
+..
+.. Deprecated
+.. ----------
+..
+.. - A bullet item for the Deprecated category.
+..
+.. Removed
+.. -------
+..
+.. - A bullet item for the Removed category.
+..
+.. Fixed
+.. -----
+..
+.. - A bullet item for the Fixed category.
+..
+.. Security
+.. --------
+..
+.. - A bullet item for the Security category.
+..
+.. Documentation
+.. -------------
+..
+.. - A bullet item for the Documentation category.
+..
+AI Assistance
+-------------
+
+- Claude Opus 4 assisted with implementation of bidirectional participation relationships using ``_instances`` suffix pattern. Pivoted from initial dry-inflector pluralization approach based on feedback.

data/changelog.d/20251107_213121_delano_fix_thread_safety_races_011CUumCP492Twxm4NLt2FvL.rst

@@ -0,0 +1,20 @@
+.. Fixed mutex race conditions in thread safety implementation
+..
+
+Fixed
+-----
+
+- Fixed critical race condition in mutex initialization for connection chain lazy loading. The mutex itself was being lazily initialized with ``||=``, which is not atomic and could result in multiple threads creating different mutex instances, defeating synchronization. Changed to eager initialization via ``Connection.included`` hook. (`lib/familia/horreum/connection.rb`)
+
+- Fixed critical race condition in mutex initialization for logger lazy loading. Similar to connection chain issue, the logger mutex was lazily initialized with ``||=``. Changed to eager initialization at module definition time. (`lib/familia/logging.rb`)
+
+- Fixed logger assignment atomicity issue where ``Familia.logger=`` set ``DatabaseLogger.logger`` outside the mutex synchronization block, potentially causing ``Familia.logger`` and ``DatabaseLogger.logger`` to be temporarily out of sync during concurrent access. Moved ``DatabaseLogger.logger`` assignment inside the synchronization block. (`lib/familia/logging.rb`)
+
+- Added explicit return statement to ``Familia.logger`` method for robustness against future refactoring. (`lib/familia/logging.rb`)
+
+AI Assistance
+-------------
+
+- Code review analysis to identify critical race conditions in mutex initialization
+- Implementation of proper eager mutex initialization patterns
+- Test file updates to reflect new initialization approach

data/changelog.d/20251107_fix_participates_in_symbol_resolution.rst

@@ -0,0 +1,91 @@
+.. Added
+.. -----
+
+.. Changed
+.. -------
+
+.. Deprecated
+.. ----------
+
+.. Removed
+.. -------
+
+.. Fixed
+.. -----
+
+- **Participation Relationships with Symbol/String Target Classes**: Fixed four bugs that occurred when calling `participates_in` with a Symbol or String target class instead of a Class object.
+
+  **Bug 1 - NoMethodError during relationship definition**:
+
+  The error was: ``private method 'member_by_config_name' called for module Familia``.
+
+  **Background**: The `participates_in` method supports flexible target class specifications:
+
+  .. code-block:: ruby
+
+     class Domain < Familia::Horreum
+       # All three forms should work:
+       participates_in Customer, :domains           # Class object (always worked)
+       participates_in :Customer, :domains          # Symbol (was broken)
+       participates_in 'Customer', :domains         # String (was broken)
+     end
+
+  **Root Cause**: The method had redundant class resolution code that directly called the private `Familia.member_by_config_name` method instead of using the public `Familia.resolve_class` API.
+
+  **Solution**: Removed the redundant resolution code and now uses the already-resolved class from the public API, simplifying the implementation and fixing the visibility issue.
+
+  **Bug 2 - NoMethodError in current_participations**:
+
+  When calling `current_participations` on objects that used Symbol/String target classes, it would fail with ``undefined method 'familia_name' for Symbol``.
+
+  **Root Cause**: The `current_participations` method was calling `.familia_name` on `config.target_class`, which stores the original Symbol/String value passed to `participates_in`.
+
+  **Solution**: Use the resolved `target_class` variable instead of the stored config value. The resolved class is already available from the `Familia.resolve_class` call earlier in the method.
+
+  **Bug 3 - NoMethodError in target_class_config_name**:
+
+  When calling `current_participations`, the internal `target_class_config_name` method would fail with ``undefined method 'config_name' for Symbol``.
+
+  **Root Cause**: The `ParticipationRelationship.target_class_config_name` method was calling `.config_name` directly on the stored `target_class` value, which could be a Symbol or String.
+
+  **Solution**: Resolve the target class before calling `config_name` by using `Familia.resolve_class(target_class)`, which handles all input types (Class, Symbol, String) correctly.
+
+  **Bug 4 - Confusing error when target class not loaded**:
+
+  When the target class hasn't been loaded yet (load order issue), the error was: ``undefined method 'method_defined?' for nil``.
+
+  **Root Cause**: When `Familia.resolve_class` returns `nil` (because the target class isn't registered in `Familia.members` yet), the code would pass `nil` to `TargetMethods::Builder.build`, which then failed with a confusing error message that didn't explain the actual problem.
+
+  **Solution**: Added explicit nil check after `resolve_class` with a detailed ArgumentError that:
+
+  - Clearly states which target class couldn't be resolved
+  - Lists the three most common causes (load order, typo, not inheriting from Horreum)
+  - Shows all currently registered Familia classes for debugging
+  - Provides a clear solution for fixing the load order
+
+  **Impact**: Projects using Symbol or String target classes in `participates_in` declarations will now work correctly throughout the entire lifecycle, including relationship definition, method generation, and participation queries. When there's a load order issue or typo, developers get a clear, actionable error message instead of a confusing nil error. This pattern is common when avoiding circular dependencies or when target classes are defined in different files.
+
+.. Security
+.. --------
+
+.. Documentation
+.. -------------
+
+.. AI Assistance
+.. -------------
+
+- **Root Cause Analysis**: Claude Code analyzed the error stack trace from the implementing project and identified that a private method was being called as a public method from outside the Familia module.
+- **Fix Implementation**: Claude Code identified redundant class resolution code and simplified it to use the already-resolved class from the public API.
+- **Test Coverage**: Claude Code created comprehensive regression tests including:
+
+  - Feature-level tests for Symbol/String target class resolution in participation relationships
+  - Unit tests for the `Familia.resolve_class` public API
+  - Edge case coverage for case-insensitive resolution and modularized classes
+
+- **Second Bug Discovery**: During test execution, Claude Code discovered a related bug in `current_participations` that was also failing with Symbol/String target classes. The test coverage revealed that `.familia_name` was being called on the unresolved config value instead of the resolved class instance.
+
+- **Third Bug Discovery**: Further test execution revealed another Symbol/String bug in `target_class_config_name`, where `.config_name` was being called directly on Symbol/String values. This was fixed by resolving the class first using `Familia.resolve_class`.
+
+- **Test Coverage Refinement**: Claude Code identified and removed unrealistic test cases (all-uppercase, all-lowercase class names) that don't occur in real Ruby code and don't work with the `snake_case` method's design. Updated tests to focus on realistic naming conventions: PascalCase and snake_case, with clear documentation explaining why certain formats aren't supported.
+
+- **Fourth Bug Discovery**: After merging to main, the implementing project revealed a load order issue where `Familia.resolve_class` returned `nil`, causing a confusing "undefined method for nil" error. Claude Code added explicit error handling with a detailed, actionable error message that helps developers quickly identify and fix load order issues, typos, or inheritance problems.

data/changelog.d/20251107_optimized_redis_exists_checks.rst

@@ -0,0 +1,94 @@
+.. Added
+.. -----
+
+- **Pipelined Bulk Loading Methods**: New `load_multi` and `load_multi_by_keys` methods enable efficient bulk object loading using Redis pipelining. These methods reduce network round trips from N×2 commands (EXISTS + HGETALL per object) to a single pipelined batch of HGETALL commands.
+
+  **Standard loading** (N objects, N×2 commands):
+
+  .. code-block:: ruby
+
+     users = ids.map { |id| User.find_by_id(id) }
+     # For 14 objects: 28 Redis commands (14 EXISTS + 14 HGETALL)
+
+  **Pipelined bulk loading** (N objects, 1 round trip):
+
+  .. code-block:: ruby
+
+     users = User.load_multi(ids)
+     # For 14 objects: 1 pipelined batch with 14 HGETALL commands
+     # Up to 2N× performance improvement
+
+  **Load by identifiers**:
+
+  .. code-block:: ruby
+
+     metadata_objects = Metadata.load_multi(['id1', 'id2', 'id3'])
+     # Returns array: [obj1, obj2, obj3]
+
+     # Filter out nils for missing objects
+     existing_only = Metadata.load_multi(ids).compact
+
+  **Load by full dbkeys**:
+
+  .. code-block:: ruby
+
+     keys = ['user:123:object', 'user:456:object']
+     users = User.load_multi_by_keys(keys)
+
+  The methods maintain the same nil-return contract as `find_by_id` for non-existent objects, preserve input order, and properly deserialize all Horreum field types. Ideal for loading collections of related objects, processing query results, or any scenario requiring multiple object lookups.
+
+.. Changed
+.. -------
+
+- **Optional EXISTS Check Optimization**: The `find_by_dbkey` and `find_by_identifier` methods now accept a `check_exists:` parameter (default: `true`) to optionally skip the EXISTS check before HGETALL. This reduces Redis commands from 2 to 1 per object while maintaining backwards compatibility.
+
+- **Parameter Consistency in find_by_identifier**: The `suffix` parameter is now a keyword parameter (was optional positional) for consistency with `check_exists`. This follows Ruby conventions that keyword parameters should not follow optional positional parameters. Maintains backwards compatibility since custom suffixes are rarely used.
+
+  **Safe mode** (default behavior, 2 commands):
+
+  .. code-block:: ruby
+
+     user = User.find_by_id(123)
+     # Commands: EXISTS user:123:object, then HGETALL user:123:object
+
+  **Optimized mode** (1 command):
+
+  .. code-block:: ruby
+
+     user = User.find_by_id(123, check_exists: false)
+     # Command: HGETALL user:123:object only
+     # Returns nil if key doesn't exist (empty hash detected)
+
+  **Use cases for optimized mode**:
+
+  - Performance-critical paths where 50% reduction matters
+  - Bulk operations with known-to-exist keys
+  - High-throughput APIs processing collections
+  - Loading objects from sorted set members (ZRANGEBYSCORE results)
+
+  The optimization is backwards compatible (default unchanged) and maintains the same nil-return behavior for non-existent keys by detecting empty hashes returned from HGETALL.
+
+.. Deprecated
+.. ----------
+
+.. Removed
+.. -------
+
+.. Fixed
+.. -----
+
+- **Position Alignment in load_multi_by_keys**: Fixed bug where empty or nil keys caused result array misalignment. The method now tracks valid positions (like `load_multi`) to ensure the results array maintains the same positions as the input array, with nils for invalid keys.
+
+.. Security
+.. --------
+
+.. Documentation
+.. -------------
+
+.. AI Assistance
+.. -------------
+
+- **Performance Analysis**: Claude Code analyzed the Redis command trace log provided by the user, identifying the EXISTS + HGETALL pattern as the performance bottleneck in bulk object loading scenarios.
+- **Solution Design**: Claude Code designed a multi-faceted optimization approach: (1) optional EXISTS check bypass with backwards compatibility, (2) pipelined bulk loading methods, (3) comprehensive test coverage. The design balanced performance gains with API safety and backwards compatibility.
+- **Implementation**: Claude Code implemented both optimization strategies including parameter additions, new bulk loading methods, comprehensive documentation with performance characteristics, and 28 test cases covering all scenarios including edge cases (nil identifiers, missing objects, order preservation).
+- **Code Review**: Claude Code ensured the implementation follows Familia's existing patterns for field deserialization, maintains nil-return contracts, and properly handles Redis::Future objects in transaction contexts.

data/changelog.d/20251108_frozen_string_literal_pragma.rst

@@ -0,0 +1,44 @@
+.. A new scriv changelog fragment.
+..
+.. Uncomment the section that is right (remove the leading dots).
+.. For top level release notes, leave all the headers commented out.
+..
+.. Added
+.. -----
+..
+.. - A bullet item for the Added category.
+..
+Changed
+-------
+
+- All Ruby files now include consistent headers with ``frozen_string_literal: true`` pragma for improved performance and memory efficiency. Headers follow the format: filename comment, blank comment line, frozen string literal pragma. Executable scripts properly place shebang first.
+
+.. Deprecated
+.. ----------
+..
+.. - A bullet item for the Deprecated category.
+..
+.. Removed
+.. -------
+..
+.. - A bullet item for the Removed category.
+..
+.. Fixed
+.. -----
+..
+.. - A bullet item for the Fixed category.
+..
+.. Security
+.. --------
+..
+.. - A bullet item for the Security category.
+..
+.. Documentation
+.. -------------
+..
+.. - A bullet item for the Documentation category.
+..
+AI Assistance
+-------------
+
+- Claude Sonnet 4.5 automated the addition of consistent file headers with frozen_string_literal pragma across 308 Ruby files, then corrected 35 executable scripts to ensure shebangs remain as the first line.