familia 2.0.0.pre12 → 2.0.0.pre13

data/docs/migrating/v2.0.0-pre13.md

@@ -0,0 +1,329 @@
+# Migrating Guide: v2.0.0-pre13
+
+This version introduces significant improvements to Familia's feature system, making it easier to organize and use features across complex projects through automatic discovery and loading of feature-specific configuration files.
+
+## Feature-Specific Autoloading System
+
+### Overview
+
+The new autoloading system allows features to automatically discover and load extension files from your project directories. When you include a feature in your model, Familia now searches for configuration files using conventional patterns, enabling clean separation between core model definitions and feature-specific configurations.
+
+### Basic Usage
+
+#### Before (Manual Configuration)
+```ruby
+# app/models/user.rb
+class User < Familia::Horreum
+  field :name, :email, :password
+
+  feature :safe_dump
+  # All configuration had to be done in the same file
+  safe_dump_fields :name, :email  # password excluded for security
+end
+```
+
+#### After (Automatic Discovery)
+```ruby
+# app/models/user.rb - Clean model definition
+class User < Familia::Horreum
+  field :name, :email, :password
+  feature :safe_dump  # ← Triggers autoloading
+end
+
+# app/models/user/safe_dump_extensions.rb - Automatically loaded
+class User
+  safe_dump_fields :name, :email  # password excluded for security
+end
+```
+
+### File Naming Conventions
+
+The autoloading system follows these patterns for discovering extension files:
+
+#### Pattern: `{model_name}/{feature_name}_*.rb`
+
+**Example Structures:**
+```
+app/models/
+├── user.rb                              # Main model
+├── user/
+│   ├── safe_dump_extensions.rb          # SafeDump configuration
+│   ├── safe_dump_custom.rb              # Additional SafeDump setup
+│   └── relationships_config.rb          # Relationships configuration
+├── product.rb                           # Another model
+└── product/
+    ├── safe_dump_fields.rb              # Product's SafeDump config
+    └── expiration_settings.rb           # Expiration configuration
+```
+
+#### Supported Patterns
+- `safe_dump_extensions.rb`
+- `safe_dump_*.rb` (any filename starting with the feature name)
+- `expiration_config.rb`
+- `relationships_setup.rb`
+
+### Advanced Configuration Examples
+
+#### Complex SafeDump Setup
+```ruby
+# app/models/customer.rb
+class Customer < Familia::Horreum
+  field :first_name, :last_name, :email, :phone
+  field :credit_card_number, :ssn, :internal_notes
+
+  feature :safe_dump
+end
+
+# app/models/customer/safe_dump_configuration.rb
+class Customer
+  # Define which fields are safe for API responses
+  safe_dump_fields :first_name, :last_name, :email
+
+  # Custom serialization for specific fields
+  def safe_dump_email
+    email&.downcase
+  end
+
+  # Computed fields for API
+  def safe_dump_full_name
+    "#{first_name} #{last_name}".strip
+  end
+end
+```
+
+#### Multi-Feature Organization
+```ruby
+# app/models/session.rb
+class Session < Familia::Horreum
+  field :user_id, :token, :ip_address, :user_agent
+
+  feature :safe_dump
+  feature :expiration
+end
+
+# app/models/session/safe_dump_api.rb
+class Session
+  safe_dump_fields :user_id, :ip_address
+  # token excluded for security
+end
+
+# app/models/session/expiration_policy.rb
+class Session
+  # Sessions expire after 24 hours
+  def self.default_ttl
+    24 * 60 * 60
+  end
+
+  # Cascade expiration to related data
+  cascade_expiration_to :user_sessions
+end
+```
+
+### Consolidated Autoloader Architecture
+
+#### Familia::Autoloader
+
+The new `Familia::Autoloader` class provides a shared utility for consistent file loading patterns across the framework:
+
+```ruby
+# Internal usage example (you typically won't call this directly)
+autoloader = Familia::Autoloader.new(
+  base_class: User,
+  feature_name: :safe_dump,
+  search_patterns: ['user/safe_dump_*.rb']
+)
+
+# Discovers and loads matching files
+autoloader.discover_and_load_extensions
+```
+
+#### Autoloading Strategies
+
+The autoloader supports multiple discovery strategies:
+
+1. **Directory-based**: Search in conventional directories
+2. **Pattern-based**: Use glob patterns for flexible matching
+3. **Explicit paths**: Load specific files when found
+
+### How Autoloading Works
+
+#### Discovery Process
+
+1. **Feature Activation**: When `feature :safe_dump` is called
+2. **Path Resolution**: Autoloader determines search paths based on model location
+3. **Pattern Matching**: Searches for files matching `{model_name}/{feature_name}_*.rb`
+4. **File Loading**: Loads discovered files in alphabetical order
+5. **Extension Application**: Feature-specific methods become available
+
+#### Search Locations
+
+The autoloader searches in these locations (in order):
+
+```ruby
+# If your model is in app/models/user.rb, it searches:
+[
+  'app/models/user/',           # Same directory as model
+  'lib/user/',                  # Lib directory
+  'config/models/user/',        # Config directory
+  './user/'                     # Current directory
+]
+```
+
+### Migration Steps
+
+#### 1. Reorganize Existing Models
+
+Move feature-specific configuration to separate files:
+
+```bash
+# Create directories for feature configurations
+mkdir -p app/models/user
+mkdir -p app/models/product
+mkdir -p app/models/order
+
+# Move configurations
+# From app/models/user.rb, extract safe_dump configuration to:
+# app/models/user/safe_dump_extensions.rb
+```
+
+#### 2. Update Model Files
+
+Clean up your main model files:
+
+```ruby
+# Before: Everything in one file
+class User < Familia::Horreum
+  field :name, :email, :password, :role
+
+  feature :safe_dump
+  safe_dump_fields :name, :email
+
+  feature :expiration
+  def self.default_ttl
+    86400
+  end
+end
+
+# After: Clean separation
+class User < Familia::Horreum
+  field :name, :email, :password, :role
+
+  feature :safe_dump    # Configuration auto-loaded
+  feature :expiration   # Configuration auto-loaded
+end
+```
+
+#### 3. Create Extension Files
+
+Set up your feature-specific configurations:
+
+```ruby
+# app/models/user/safe_dump_extensions.rb
+class User
+  safe_dump_fields :name, :email
+  # password and role excluded for security
+end
+
+# app/models/user/expiration_config.rb
+class User
+  def self.default_ttl
+    86400  # 24 hours
+  end
+end
+```
+
+### Benefits of the New System
+
+#### Code Organization
+- **Separation of Concerns**: Feature logic separated from core model definition
+- **Maintainability**: Easier to find and modify feature-specific code
+- **Readability**: Core models are cleaner and more focused
+
+#### Team Development
+- **Reduced Conflicts**: Multiple developers can work on different features without merge conflicts
+- **Feature Ownership**: Clear boundaries for feature-specific code
+- **Testing**: Easier to test features in isolation
+
+#### Scalability
+- **Large Models**: Complex models remain manageable
+- **Feature Growth**: New features can be added without bloating main model files
+- **Refactoring**: Easier to extract and reorganize feature code
+
+### Debugging Autoloading
+
+#### Enable Debug Output
+
+```ruby
+# In your application initialization
+ENV['FAMILIA_DEBUG'] = '1'
+
+# Or programmatically
+Familia::Features::Autoloadable.debug = true
+```
+
+#### Debug Output Example
+```
+[Familia::Autoloader] Searching for User safe_dump extensions...
+[Familia::Autoloader] Found: app/models/user/safe_dump_extensions.rb
+[Familia::Autoloader] Loading: app/models/user/safe_dump_extensions.rb
+[Familia::Autoloader] SafeDump extension loaded successfully for User
+```
+
+#### Troubleshooting
+
+**Common Issues:**
+
+1. **File Not Found**: Ensure file naming follows the `{feature_name}_*.rb` pattern
+2. **Load Order**: Files are loaded alphabetically; prefix with numbers if order matters
+3. **Class Scope**: Extension files should reopen the same class as your model
+
+**Validation:**
+```ruby
+# Check if autoloading worked
+User.respond_to?(:safe_dump_field_names)  # => true
+User.safe_dump_field_names                # => [:name, :email]
+```
+
+### Backward Compatibility
+
+The autoloading system is fully backward compatible:
+
+- **Existing Models**: Continue to work without changes
+- **Manual Configuration**: Still supported alongside autoloading
+- **Gradual Migration**: You can migrate models one at a time
+
+### Performance Considerations
+
+- **Load Time**: Files are loaded once during feature activation
+- **Memory Usage**: No additional memory overhead after loading
+- **Caching**: Discovered files are cached to avoid repeated filesystem scans
+
+### Testing Your Migration
+
+```ruby
+# Test that autoloading is working
+class TestUser < Familia::Horreum
+  field :name, :email
+  feature :safe_dump
+end
+
+# Verify extension methods are available
+puts TestUser.respond_to?(:safe_dump_field_names)
+puts TestUser.safe_dump_field_names.inspect
+
+# Test instance methods
+user = TestUser.new(name: "John", email: "john@example.com")
+puts user.safe_dump.inspect
+```
+
+## New Capabilities
+
+1. **Automatic Extension Discovery**: No manual require statements needed
+2. **Conventional File Organization**: Standard patterns for consistent project structure
+3. **Feature Isolation**: Clean separation between core models and feature configurations
+4. **Shared Autoloader Infrastructure**: Consistent loading behavior across all features
+5. **Debug Support**: Built-in debugging for troubleshooting autoloading issues
+
+## Breaking Changes
+
+**None** - This release is fully backward compatible. Existing models and feature configurations continue to work without modification.

data/examples/autoloader/mega_customer/safe_dump_fields.rb

@@ -0,0 +1,6 @@
+# examples/autoloader/mega_customer/safe_dump_fields.rb
+
+# Extend the MegaCustomer class to add safe dump fields
+class MegaCustomer
+  safe_dump_fields :custid, :username, :created_at, :updated_at
+end

data/examples/autoloader/mega_customer.rb

@@ -0,0 +1,17 @@
+# examples/autoloader/mega_customer.rb
+
+require_relative '../../lib/familia'
+
+class MegaCustomer < Familia::Horreum
+  field :custid
+  field :username
+  field :email
+  field :fname
+  field :lname
+  field :display_name
+  field :created_at
+  field :updated_at
+
+  feature :safe_dump
+  # feature :deprecated_fields
+end

data/familia.gemspec

@@ -23,6 +23,7 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'connection_pool', '~> 2.5'
   spec.add_dependency 'csv', '~> 3.3'
   spec.add_dependency 'logger', '~> 1.7'
+  spec.add_dependency 'oj', '~> 3.16'
   spec.add_dependency 'redis', '>= 4.8.1', '< 6.0'
   spec.add_dependency 'stringio', '~> 3.1.1'
   spec.add_dependency 'uri-valkey', '~> 1.4'

data/lib/familia/autoloader.rb

@@ -0,0 +1,53 @@
+# frozen_string_literal: true
+
+module Familia
+  # Provides autoloading functionality for Ruby files based on patterns and conventions.
+  #
+  # Used by the Features module at library startup to load feature files, and available
+  # as a utility for other modules requiring file autoloading capabilities.
+  module Autoloader
+    # Autoloads Ruby files matching the given patterns.
+    #
+    # @param patterns [String, Array<String>] file patterns to match (supports Dir.glob patterns)
+    # @param exclude [Array<String>] basenames to exclude from loading
+    # @param log_prefix [String] prefix for debug logging messages
+    def self.autoload_files(patterns, exclude: [], log_prefix: 'Autoloader')
+      patterns = Array(patterns)
+
+      patterns.each do |pattern|
+        Dir.glob(pattern).each do |file_path|
+          basename = File.basename(file_path)
+
+          # Skip excluded files
+          next if exclude.include?(basename)
+
+          Familia.trace :FEATURE, nil, "[#{log_prefix}] Loading #{file_path}", caller(1..1) if Familia.debug?
+          require File.expand_path(file_path)
+        end
+      end
+    end
+
+    # Autoloads feature files when this module is included.
+    #
+    # Discovers and loads all Ruby files in the features/ directory relative to the
+    # including module's location. Typically used by Familia::Features.
+    #
+    # @param base [Module] the module including this autoloader
+    def self.included(base)
+      # Get the directory where the including module is defined
+      # This should be lib/familia for the Features module
+      base_path = File.dirname(caller_locations(1, 1).first.path)
+      features_dir = File.join(base_path, 'features')
+
+      Familia.ld "[DEBUG] Autoloader loading features from #{features_dir}"
+
+      return unless Dir.exist?(features_dir)
+
+      # Use the shared autoload_files method
+      autoload_files(
+        File.join(features_dir, '*.rb'),
+        log_prefix: 'Autoloader'
+      )
+    end
+  end
+end

data/lib/familia/base.rb

@@ -14,6 +14,9 @@ module Familia
   # @see Familia::DataType
   #
   module Base
+
+    using Familia::Refinements::TimeUtils
+
     @features_available = nil
     @feature_definitions = nil
     @dump_method = :to_json
@@ -24,6 +27,8 @@ module Familia
       base.extend(ClassMethods)
     end
 
+    # Familia::Base::ClassMethods
+    #
     module ClassMethods
       attr_reader :features_available, :feature_definitions
       attr_accessor :dump_method, :load_method

data/lib/familia/data_type.rb

@@ -3,6 +3,8 @@
 require_relative 'data_type/commands'
 require_relative 'data_type/serialization'
 
+# Familia
+#
 module Familia
   # DataType - Base class for Database data type wrappers
   #
@@ -14,6 +16,8 @@ module Familia
     include Familia::Base
     extend Familia::Features
 
+    using Familia::Refinements::TimeUtils
+
     @registered_types = {}
     @valid_options = %i[class parent default_expiration default logical_database dbkey dbclient suffix prefix]
     @logical_database = nil

data/lib/familia/encryption/encrypted_data.rb

@@ -9,7 +9,7 @@ module Familia
         return false unless json_string.kind_of?(::String)
 
         begin
-          parsed = JSON.parse(json_string, symbolize_names: true)
+          parsed = Familia::JsonSerializer.parse(json_string, symbolize_names: true)
           return false unless parsed.is_a?(Hash)
 
           # Check for required fields
@@ -17,7 +17,7 @@ module Familia
           result = required_fields.all? { |field| parsed.key?(field) }
           Familia.ld "[valid?] result: #{result}, parsed: #{parsed}, required: #{required_fields}"
           result
-        rescue JSON::ParserError => e
+        rescue Familia::SerializerError => e
           Familia.ld "[valid?] JSON error: #{e.message}"
           false
         end
@@ -31,8 +31,8 @@ module Familia
         end
 
         begin
-          parsed = JSON.parse(json_string, symbolize_names: true)
-        rescue JSON::ParserError => e
+          parsed = Familia::JsonSerializer.parse(json_string, symbolize_names: true)
+        rescue Familia::SerializerError => e
           raise EncryptionError, "Invalid JSON structure: #{e.message}"
         end
 

data/lib/familia/encryption/manager.rb

@@ -19,13 +19,15 @@ module Familia
 
         result = @provider.encrypt(plaintext, key, additional_data)
 
-        Familia::Encryption::EncryptedData.new(
+        encrypted_data = Familia::Encryption::EncryptedData.new(
           algorithm: @provider.algorithm,
           nonce: Base64.strict_encode64(result[:nonce]),
           ciphertext: Base64.strict_encode64(result[:ciphertext]),
           auth_tag: Base64.strict_encode64(result[:auth_tag]),
           key_version: current_key_version
-        ).to_h.to_json
+        ).to_h
+
+        Familia::JsonSerializer.dump(encrypted_data)
       ensure
         Familia::Encryption.secure_wipe(key) if key
       end
@@ -37,7 +39,7 @@ module Familia
         Familia::Encryption.derivation_count.increment
 
         begin
-          data = Familia::Encryption::EncryptedData.new(**JSON.parse(encrypted_json, symbolize_names: true))
+          data = Familia::Encryption::EncryptedData.new(**Familia::JsonSerializer.parse(encrypted_json, symbolize_names: true))
 
           # Validate algorithm support
           provider = Registry.get(data.algorithm)
@@ -51,7 +53,7 @@ module Familia
           provider.decrypt(ciphertext, key, nonce, auth_tag, additional_data)
         rescue EncryptionError
           raise
-        rescue JSON::ParserError => e
+        rescue Familia::SerializerError => e
           raise EncryptionError, "Invalid JSON structure: #{e.message}"
         rescue StandardError => e
           raise EncryptionError, "Decryption failed: #{e.message}"

data/lib/familia/encryption.rb

@@ -1,7 +1,7 @@
 # lib/familia/encryption.rb
 
 require 'base64'
-require 'json'
+require 'oj'
 require 'openssl'
 
 # Provider system components

data/lib/familia/errors.rb

@@ -6,6 +6,9 @@ module Familia
   class NonUniqueKey < Problem; end
 
   class FieldTypeError < Problem; end
+  class AutoloadError < Problem; end
+
+  class SerializerError < Problem; end
 
   class HighRiskFactor < Problem
     attr_reader :value

data/lib/familia/features/autoloadable.rb

@@ -0,0 +1,113 @@
+# frozen_string_literal: true
+
+require_relative '../refinements/snake_case'
+
+module Familia
+  module Features
+    # Enables automatic loading of feature-specific files when a feature is included in a user class.
+    #
+    # When included in a feature module, adds ClassMethods that detect when the feature is
+    # included in user classes, derives the feature name, and autoloads files matching
+    # conventional patterns in the user class's directory structure.
+    module Autoloadable
+      using Familia::Refinements::SnakeCase
+
+      # Sets up a feature module with autoloading capabilities.
+      #
+      # Extends the feature module with ClassMethods to handle post-inclusion autoloading.
+      #
+      # @param feature_module [Module] the feature module being enhanced
+      def self.included(feature_module)
+        feature_module.extend(ClassMethods)
+      end
+
+      # Methods added to feature modules that include Autoloadable.
+      module ClassMethods
+        # Triggered when the feature is included in a user class.
+        #
+        # Sets up for post-inclusion autoloading. The actual autoloading
+        # is deferred until after feature setup completes.
+        #
+        # @param base [Class] the user class including this feature
+        def included(base)
+          # Call parent included method if it exists (defensive programming for mixed-in contexts)
+          super if defined?(super)
+
+          # No autoloading here - it's deferred to post_inclusion_autoload
+          # to ensure the feature is fully set up before extension files are loaded
+        end
+
+        # Called by the feature system after the feature is fully included.
+        #
+        # Uses const_source_location to determine where the base class is defined,
+        # then autoloads feature-specific extension files from that location.
+        #
+        # @param base [Class] the class that included this feature
+        # @param feature_name [Symbol] the name of the feature
+        # @param options [Hash] feature options (unused but kept for compatibility)
+        def post_inclusion_autoload(base, feature_name, options)
+          Familia.trace :FEATURE, nil, "[Autoloadable] post_inclusion_autoload called for #{feature_name} on #{base.name || base}", caller(1..1) if Familia.debug?
+
+          # Get the source location via Ruby's built-in introspection
+          source_location = nil
+
+          # Check for named classes that can be looked up via const_source_location
+          # Class#name always returns String or nil, so type check is redundant
+          if base.name && !base.name.empty?
+            begin
+              location_info = Module.const_source_location(base.name)
+              source_location = location_info&.first
+              Familia.trace :FEATURE, nil, "[Autoloadable] Source location for #{base.name}: #{source_location}", caller(1..1) if Familia.debug?
+            rescue NameError => e
+              # Handle cases where the class name is not a valid constant name
+              # This can happen in test environments with dynamically created classes
+              Familia.trace :FEATURE, nil, "[Autoloadable] Cannot resolve source location for #{base.name}: #{e.message}", caller(1..1) if Familia.debug?
+            end
+          else
+            Familia.trace :FEATURE, nil, "[Autoloadable] Skipping source location detection - base.name=#{base.name.inspect}", caller(1..1) if Familia.debug?
+          end
+
+          # Autoload feature-specific files if we have a valid source location
+          if source_location && !source_location.include?('-e') # Skip eval/irb contexts
+            Familia.trace :FEATURE, nil, "[Autoloadable] Calling autoload_feature_files with #{source_location}", caller(1..1) if Familia.debug?
+            autoload_feature_files(source_location, base, feature_name.to_s.snake_case)
+          else
+            Familia.trace :FEATURE, nil, "[Autoloadable] Skipping autoload - no valid source location", caller(1..1) if Familia.debug?
+          end
+        end
+
+        private
+
+        # Autoloads feature-specific files from conventional directory patterns.
+        #
+        # Searches for files matching patterns like:
+        # - model_name/feature_name_*.rb
+        # - model_name/features/feature_name_*.rb
+        # - features/feature_name_*.rb
+        #
+        # @param location_path [String] path where the user class is defined
+        # @param base [Class] the user class including the feature
+        # @param feature_name [String] snake_case name of the feature
+        def autoload_feature_files(location_path, base, feature_name)
+          base_dir = File.dirname(location_path)
+
+          # Handle anonymous classes gracefully
+          model_name = base.name ? base.name.snake_case : "anonymous_#{base.object_id}"
+
+          # Look for feature-specific files in conventional locations
+          patterns = [
+            File.join(base_dir, model_name, "#{feature_name}_*.rb"),
+            File.join(base_dir, model_name, 'features', "#{feature_name}_*.rb"),
+            File.join(base_dir, 'features', "#{feature_name}_*.rb"),
+          ]
+
+          # Use Autoloader's shared method for consistent file loading
+          Familia::Autoloader.autoload_files(
+            patterns,
+            log_prefix: "Autoloadable(#{feature_name})"
+          )
+        end
+      end
+    end
+  end
+end

data/lib/familia/features/encrypted_fields/concealed_string.rb

@@ -37,6 +37,8 @@
 #   user.to_json                            # Safe - contains [CONCEALED]
 #
 class ConcealedString
+  REDACTED = '[CONCEALED]'.freeze
+
   # Create a concealed string wrapper
   #
   # @param encrypted_data [String] The encrypted JSON data
@@ -264,9 +266,9 @@ class ConcealedString
     { concealed: true }
   end
 
-  # Prevent exposure in JSON serialization
+  # Prevent exposure in JSON serialization - fail closed for security
   def to_json(*)
-    '"[CONCEALED]"'
+    raise Familia::SerializerError, "ConcealedString cannot be serialized to JSON"
   end
 
   # Prevent exposure in Rails serialization (as_json -> to_json)

data/lib/familia/features/expiration.rb

@@ -149,6 +149,8 @@ module Familia
     module Expiration
       @default_expiration = nil
 
+      using Familia::Refinements::TimeUtils
+
       def self.included(base)
         Familia.trace :LOADED, self, base, caller(1..1) if Familia.debug?
         base.extend ClassMethods
@@ -160,6 +162,8 @@ module Familia
         base.instance_variable_set(:@default_expiration, @default_expiration)
       end
 
+      # Familia::Expiration::ClassMethods
+      #
       module ClassMethods
         # Set the default expiration time for instances of this class
         #