familia 2.0.0.pre10 → 2.0.0.pre13

data/docs/migrating/v2.0.0-pre5.md

@@ -0,0 +1,110 @@
+# Migrating Guide: Security Features (v2.0.0-pre5)
+
+This guide covers adopting the security enhancements introduced in v2.0.0-pre5.
+
+## Security Feature Adoption
+
+### 1. Configure Encryption Keys
+
+Before using encrypted fields, configure encryption keys:
+
+```ruby
+Familia.configure do |config|
+  config.encryption_keys = {
+    v1: 'your-32-byte-base64-encoded-key==',
+    v2: 'newer-32-byte-base64-encoded-key=='
+  }
+  config.current_key_version = :v2
+end
+```
+
+**Key Management:**
+- Use secure key storage (environment variables, key management services)
+- Rotate keys regularly by adding new versions
+- Never remove old key versions while data exists
+
+### 2. Identify Sensitive Fields
+
+Mark fields that contain sensitive data:
+
+**For Encryption:**
+```ruby
+class Vault < Familia::Horreum
+  feature :encrypted_fields
+
+  field :name                    # Plaintext
+  encrypted_field :secret_key    # Encrypted at rest
+  encrypted_field :api_token     # Transparent access
+end
+```
+
+**For Transient Fields:**
+```ruby
+class User < Familia::Horreum
+  feature :transient_fields
+
+  field :email                   # Persisted
+  transient_field :password      # Never persisted
+  transient_field :session_token # Runtime only
+end
+```
+
+### 3. Update Serialization Code
+
+Handle `RedactedString` in serialization:
+
+**Before:**
+```ruby
+def to_json
+  { name: name, password: password }.to_json
+end
+```
+
+**After:**
+```ruby
+def to_json
+  # RedactedString automatically excluded from serialization
+  { name: name }.to_json  # password field omitted if transient
+end
+```
+
+**Manual RedactedString Handling:**
+```ruby
+# Access original value when needed
+password.reveal  # Returns actual string value
+password.redacted?  # Returns true if redacted
+```
+
+### 4. Implement Key Rotation Procedures
+
+**Rotation Process:**
+1. Add new key version to configuration
+2. Update `current_key_version`
+3. Re-encrypt existing data gradually
+4. Remove old keys after migration complete
+
+**Example Rotation Script:**
+```ruby
+# Add new key version
+Familia.config.encryption_keys[:v3] = 'new-key'
+Familia.config.current_key_version = :v3
+
+# Re-encrypt existing records
+Vault.all.each do |vault|
+  vault.save  # Automatically uses new key version
+end
+```
+
+## Security Best Practices
+
+- **Environment Variables:** Store keys in environment variables, not code
+- **Key Rotation:** Rotate encryption keys regularly (quarterly/annually)
+- **Field Selection:** Only encrypt fields that truly need protection
+- **Memory Clearing:** Use transient fields for temporary sensitive data
+- **Logging:** Verify RedactedString prevents accidental logging
+
+## Next Steps
+
+After implementing security features:
+1. Review [Architecture Migration](v2.0.0-pre6.md) for persistence improvements
+2. Explore [Relationships Migration](v2.0.0-pre7.md) for the relationship system

data/docs/migrating/v2.0.0-pre6.md

@@ -0,0 +1,154 @@
+# Migrating Guide: Architecture Improvements (v2.0.0-pre6)
+
+This guide covers the architecture enhancements and new persistence methods in v2.0.0-pre6.
+
+## Architecture Improvements
+
+### 1. Enhanced Persistence Operations
+
+**New `save_if_not_exists` Method:**
+```ruby
+user = User.new(email: 'user@example.com')
+
+# Only save if the user doesn't already exist
+if user.save_if_not_exists
+  puts "User created successfully"
+else
+  puts "User already exists"
+end
+```
+
+**Atomic Persistence with Transactions:**
+```ruby
+user.transaction do |conn|
+  conn.set(user.key, user.serialize)
+  conn.sadd("all_users", user.identifier)
+  conn.expire(user.key, user.ttl) if user.ttl
+end
+```
+
+### 2. Modular Class Structure
+
+The Horreum class structure was reorganized for better maintainability:
+
+**Core Modules:**
+- `Familia::Horreum::Core` - Essential functionality
+- `Familia::Horreum::ClassMethods` - Class-level methods
+- `Familia::Horreum::Serialization` - Object serialization
+- `Familia::Horreum::Commands` - Redis command wrappers
+
+**Feature System Improvements:**
+- Dependency management between features
+- Cleaner feature activation
+- Better error handling for missing dependencies
+
+### 3. Enhanced Error Handling
+
+**New Exception Types:**
+```ruby
+begin
+  user.save!
+rescue Familia::PersistenceError => e
+  puts "Failed to save: #{e.message}"
+rescue Familia::ValidationError => e
+  puts "Validation failed: #{e.message}"
+end
+```
+
+**Improved Data Consistency:**
+- Automatic retry for transient Redis connection issues
+- Better handling of concurrent modifications
+- Enhanced validation before persistence operations
+
+## Migration Steps
+
+### 1. Update Error Handling
+
+**Before (v2.0.0-pre5):**
+```ruby
+begin
+  user.save
+rescue => e
+  puts "Something went wrong: #{e}"
+end
+```
+
+**After (v2.0.0-pre6):**
+```ruby
+begin
+  user.save
+rescue Familia::PersistenceError => e
+  puts "Persistence failed: #{e.message}"
+  # Handle specific persistence issues
+rescue Familia::ValidationError => e
+  puts "Invalid data: #{e.message}"
+  # Handle validation failures
+end
+```
+
+### 2. Adopt Conditional Persistence
+
+Replace existence checks with atomic operations:
+
+**Before:**
+```ruby
+user = User.new(email: email)
+unless User.exists?(email)
+  user.save
+end
+```
+
+**After:**
+```ruby
+user = User.new(email: email)
+user.save_if_not_exists
+```
+
+### 3. Leverage Transaction Support
+
+For complex operations, use transactions:
+
+```ruby
+# Before - Multiple separate operations
+user.save
+user.tags.add(tag)
+user.scores.add(score, timestamp)
+
+# After - Atomic transaction
+user.transaction do |conn|
+  conn.set(user.key, user.serialize)
+  conn.sadd(user.tags.key, tag)
+  conn.zadd(user.scores.key, timestamp, score)
+end
+```
+
+## Performance Improvements
+
+### Connection Management
+- Improved connection pooling with better resource utilization
+- Reduced connection overhead through intelligent connection reuse
+- Enhanced concurrent operation support
+
+### Feature System
+- Lazy feature loading reduces memory footprint
+- Optimized method dispatch for feature methods
+- Better dependency resolution
+
+## Breaking Changes
+
+### Method Signatures
+- Some internal methods changed signatures for better consistency
+- Error handling improved with specific exception types
+- Transaction block interface standardized
+
+### Feature Dependencies
+- Features now explicitly declare dependencies
+- Better error messages for missing feature requirements
+- Automatic dependency resolution where possible
+
+## Next Steps
+
+After completing architecture migration:
+1. Explore [Relationships Migration](v2.0.0-pre7.md) for the comprehensive relationship system
+2. Review updated documentation for architectural patterns
+3. Consider adopting new persistence patterns in your application

data/docs/migrating/v2.0.0-pre7.md

@@ -0,0 +1,222 @@
+# Migrating Guide: Relationships System (v2.0.0-pre7)
+
+This guide covers adopting the comprehensive relationships system introduced in v2.0.0-pre7.
+
+## Relationships System Overview
+
+The v2.0.0-pre7 release introduces three powerful relationship types:
+
+- **`tracked_in`** - Multi-presence tracking with score encoding
+- **`indexed_by`** - O(1) hash-based lookups
+- **`member_of`** - Bidirectional membership with collision-free naming
+
+## Migration Steps
+
+### 1. Enable Relationships Feature
+
+Add the relationships feature to your models:
+
+```ruby
+class Customer < Familia::Horreum
+  feature :relationships  # Required for all relationship types
+
+  identifier_field :custid
+  field :custid, :name, :email
+end
+```
+
+### 2. Choose Appropriate Relationship Types
+
+**For Tracking Collections (sorted sets):**
+```ruby
+class Customer < Familia::Horreum
+  feature :relationships
+
+  # Global tracking with score-based sorting
+  tracked_in :all_customers, type: :sorted_set, score: :created_at
+
+  # Scoped tracking
+  tracked_in :active_users, type: :sorted_set, score: :last_seen
+end
+```
+
+**For Fast Lookups (hash indexes):**
+```ruby
+class Customer < Familia::Horreum
+  feature :relationships
+
+  # Global email index for O(1) lookups
+  indexed_by :email, :email_lookup, context: :global
+
+  # Domain-specific indexes
+  indexed_by :account_id, :account_lookup, context: Domain
+end
+```
+
+**For Bidirectional Membership:**
+```ruby
+class Domain < Familia::Horreum
+  feature :relationships
+
+  # Belongs to customer's domains collection
+  member_of Customer, :domains, type: :set
+end
+```
+
+### 3. Implement Permission Systems
+
+For access-controlled relationships:
+
+```ruby
+class Document < Familia::Horreum
+  feature :relationships
+  include Familia::Features::Relationships::PermissionManagement
+
+  # Permission-aware tracking
+  tracked_in Customer, :documents, score: :created_at
+
+  permission_tracking :user_permissions
+
+  def permission_bits
+    # Define permission levels (bit-encoded)
+    @permission_bits || 1  # Default: read-only
+  end
+end
+```
+
+### 4. Update Queries
+
+**Before (manual Redis operations):**
+```ruby
+# Manual sorted set operations
+redis.zadd("all_customers", customer.created_at, customer.custid)
+customers = redis.zrange("all_customers", 0, -1)
+```
+
+**After (relationship methods):**
+```ruby
+# Automatic method generation
+Customer.add_to_all_customers(customer)
+customers = Customer.all_customers.to_a
+```
+
+## Relationship Method Patterns
+
+### `tracked_in` Methods
+```ruby
+# Class methods generated:
+Customer.add_to_all_customers(customer)
+Customer.remove_from_all_customers(customer)
+Customer.all_customers  # Access sorted set directly
+```
+
+### `indexed_by` Methods
+```ruby
+# Class methods generated:
+Customer.add_to_email_lookup(customer)
+Customer.remove_from_email_lookup(customer)
+Customer.email_lookup.get(email)  # O(1) lookup
+```
+
+### `member_of` Methods
+```ruby
+# Instance methods generated:
+domain.add_to_customer_domains(customer)
+domain.remove_from_customer_domains(customer)
+domain.in_customer_domains?(customer)
+```
+
+## Permission System Migration
+
+### Basic Permission Encoding
+```ruby
+# Permission levels (bit-encoded)
+READ    = 1   # 001
+WRITE   = 4   # 100
+DELETE  = 32  # 100000
+ADMIN   = 128 # 10000000
+
+# Combine permissions
+user_permissions = READ | WRITE  # Can read and write
+admin_permissions = ADMIN        # All permissions via hierarchy
+```
+
+### Time-Based Permission Scores
+```ruby
+# Encode timestamp with permission bits
+timestamp = Time.now.to_i
+permissions = READ | WRITE
+score = Familia::Features::Relationships::ScoreEncoding.encode_score(timestamp, permissions)
+
+# Query by permission level
+documents_with_write_access = customer.documents.accessible_items(
+  permissions: WRITE,
+  collection_key: customer.documents.key
+)
+```
+
+## Performance Considerations
+
+### Relationship Type Selection
+
+**Use `indexed_by` for:**
+- Unique field lookups (email, username, ID)
+- Fields that need O(1) access
+- Reference relationships
+
+**Use `tracked_in` for:**
+- Time-ordered collections
+- Scored/ranked relationships
+- Permission-based access control
+
+**Use `member_of` for:**
+- Simple membership tracking
+- Bidirectional relationships
+- Set-based collections
+
+### Optimization Tips
+
+**Batch Operations:**
+```ruby
+# Instead of multiple individual operations
+customers.each { |c| Customer.add_to_all_customers(c) }
+
+# Use batch operations where available
+Customer.all_customers.add_batch(customers.map(&:identifier),
+                                 customers.map(&:created_at))
+```
+
+**Permission Queries:**
+```ruby
+# Efficient permission filtering
+accessible_docs = document.accessible_items(
+  collection_key: customer.documents.key,
+  minimum_permissions: READ
+)
+```
+
+## Breaking Changes
+
+### Method Name Changes
+- Relationship methods follow new naming patterns
+- Previous manual Redis operations need updating
+- Collection access methods standardized
+
+### Permission System
+- New bit-encoded permission system
+- Time-based scoring for temporal access
+- Permission hierarchy implementation required
+
+## Next Steps
+
+1. **Analyze Existing Relationships:** Review current Redis operations for optimization opportunities
+2. **Choose Relationship Types:** Select appropriate types based on usage patterns
+3. **Implement Permissions:** Add access control where needed
+4. **Update Queries:** Replace manual operations with generated relationship methods
+5. **Test Performance:** Benchmark relationship operations under load
+
+## Resources
+
+- [Relationships Methods Guide](../guides/relationships-methods.md) - Complete method reference
+- [Relationships Guide](../guides/Relationships-Guide.md) - Implementation examples
+- [Performance Guide](../guides/Implementation-Guide.md) - Optimization strategies

data/docs/overview.md

@@ -118,17 +118,16 @@ This is ideal for temporary data like authentication tokens or cache entries.
 
 ### Safe Dumping for APIs
 
-Control which fields are exposed when serializing objects:
+Control which fields are exposed when serializing objects using the clean DSL:
 
 ```ruby
 class User < Familia::Horreum
   feature :safe_dump
 
-  @safe_dump_fields = [
-    :id,
-    :email,
-    {full_name: ->(user) { "#{user.first_name} #{user.last_name}" }}
-  ]
+  # Use clean DSL methods instead of @safe_dump_fields
+  safe_dump_field :id
+  safe_dump_field :email
+  safe_dump_field :full_name, ->(user) { "#{user.first_name} #{user.last_name}" }
 
   field :id, :email, :first_name, :last_name, :password_hash
 end
@@ -137,7 +136,7 @@ user.safe_dump
 #=> {id: "123", email: "alice@example.com", full_name: "Alice Smith"}
 ```
 
-Prevents accidental exposure of sensitive data in API responses.
+The new DSL prevents accidental exposure of sensitive data and makes field definitions easier to organize in feature modules.
 
 ### Time-based Quantization
 

data/{examples/redis_command_validation_example.rb → docs/reference/auditing_database_commands.rb}

@@ -50,21 +50,21 @@ class TransferService
   end
 end
 
-puts "🧪 Redis Command Validation Framework Demo"
-puts "=" * 50
+puts '🧪 Redis Command Validation Framework Demo'
+puts '=' * 50
 
 # Clean up any existing test data
-cleanup_keys = Familia.dbclient.keys("account:*")
+cleanup_keys = Familia.dbclient.keys('account:*')
 Familia.dbclient.del(*cleanup_keys) if cleanup_keys.any?
 
 # Example 1: Basic Command Recording
 puts "\n1. Basic Command Recording"
-puts "-" * 30
+puts '-' * 30
 
 CommandRecorder = Familia::Validation::CommandRecorder
 CommandRecorder.start_recording
 
-account = Account.new(account_id: "acc001", balance: "1000", status: "active")
+account = Account.new(account_id: 'acc001', balance: '1000', status: 'active')
 account.save
 
 commands = CommandRecorder.stop_recording
@@ -73,12 +73,12 @@ commands.commands.each { |cmd| puts "  #{cmd}" }
 
 # Example 2: Transaction Detection
 puts "\n2. Transaction Detection"
-puts "-" * 30
+puts '-' * 30
 
 CommandRecorder.start_recording
 
-acc1 = Account.new(account_id: "acc002", balance: "2000")
-acc2 = Account.new(account_id: "acc003", balance: "500")
+acc1 = Account.new(account_id: 'acc002', balance: '2000')
+acc2 = Account.new(account_id: 'acc003', balance: '500')
 acc1.save
 acc2.save
 
@@ -96,7 +96,7 @@ end
 
 # Example 3: Validation with Expectations DSL
 puts "\n3. Command Validation with Expectations"
-puts "-" * 30
+puts '-' * 30
 
 begin
   validator = Familia::Validation::Validator.new
@@ -104,15 +104,15 @@ begin
   # This should pass - we expect the exact Redis commands
   result = validator.validate do |expect|
     expect.transaction do |tx|
-      tx.hset("account:acc004:object", "balance", "1500")
-        .hset("account:acc005:object", "balance", "1000")
-        .hset("account:acc004:object", "last_updated", Familia::Validation::ArgumentMatcher.new(:any_string))
-        .hset("account:acc005:object", "last_updated", Familia::Validation::ArgumentMatcher.new(:any_string))
+      tx.hset('account:acc004:object', 'balance', '1500')
+        .hset('account:acc005:object', 'balance', '1000')
+        .hset('account:acc004:object', 'last_updated', Familia::Validation::ArgumentMatcher.new(:any_string))
+        .hset('account:acc005:object', 'last_updated', Familia::Validation::ArgumentMatcher.new(:any_string))
     end
 
     # Execute the operation
-    acc4 = Account.new(account_id: "acc004", balance: "2000")
-    acc5 = Account.new(account_id: "acc005", balance: "500")
+    acc4 = Account.new(account_id: 'acc004', balance: '2000')
+    acc5 = Account.new(account_id: 'acc005', balance: '500')
     acc4.save
     acc5.save
 
@@ -121,51 +121,49 @@ begin
 
   puts "Validation result: #{result.valid? ? 'PASS ✅' : 'FAIL ❌'}"
   puts "Summary: #{result.summary}"
-
-rescue => e
+rescue StandardError => e
   puts "Validation demo encountered error: #{e.message}"
-  puts "This is expected as the framework needs Redis middleware integration"
+  puts 'This is expected as the framework needs Redis middleware integration'
 end
 
 # Example 4: Performance Analysis
 puts "\n4. Performance Analysis"
-puts "-" * 30
+puts '-' * 30
 
 begin
   commands = Familia::Validation.capture_commands do
     # Create multiple accounts
     accounts = []
     (1..5).each do |i|
-      account = Account.new(account_id: "perf#{i}", balance: "1000")
+      account = Account.new(account_id: "perf#{i}", balance: '1000')
       account.save
       accounts << account
     end
 
     # Perform operations
-    accounts[0].balance = "1100"
+    accounts[0].balance = '1100'
     accounts[0].save
   end
 
   analyzer = Familia::Validation::PerformanceAnalyzer.new(commands)
   analysis = analyzer.analyze
 
-  puts "Performance Analysis:"
+  puts 'Performance Analysis:'
   puts "  Total Commands: #{analysis[:total_commands]}"
   puts "  Command Types: #{analysis[:command_type_breakdown].keys.join(', ')}"
   puts "  Efficiency Score: #{analysis[:efficiency_score]}/100"
-
-rescue => e
+rescue StandardError => e
   puts "Performance analysis encountered error: #{e.message}"
 end
 
 # Example 5: Atomicity Validation
 puts "\n5. Atomicity Validation"
-puts "-" * 30
+puts '-' * 30
 
 begin
   # Test atomic vs non-atomic operations
-  acc6 = Account.new(account_id: "acc006", balance: "3000")
-  acc7 = Account.new(account_id: "acc007", balance: "1000")
+  acc6 = Account.new(account_id: 'acc006', balance: '3000')
+  acc7 = Account.new(account_id: 'acc007', balance: '1000')
   acc6.save
   acc7.save
 
@@ -180,13 +178,12 @@ begin
   result = atomicity_validator.validate
 
   puts "Atomicity validation: #{result.valid? ? 'PASS ✅' : 'FAIL ❌'}"
-
-rescue => e
+rescue StandardError => e
   puts "Atomicity validation encountered error: #{e.message}"
 end
 
 puts "\n6. Framework Architecture Overview"
-puts "-" * 30
+puts '-' * 30
 puts "
 The Redis Command Validation Framework provides:
 
@@ -224,8 +221,8 @@ Key Benefits:
 "
 
 # Cleanup
-cleanup_keys = Familia.dbclient.keys("account:*")
+cleanup_keys = Familia.dbclient.keys('account:*')
 Familia.dbclient.del(*cleanup_keys) if cleanup_keys.any?
 
 puts "\n🎉 Demo complete! The validation framework is ready for use."
-puts "    See try/validation/ for comprehensive test examples."
+puts '    See try/validation/ for comprehensive test examples.'

data/examples/autoloader/mega_customer/safe_dump_fields.rb

@@ -0,0 +1,6 @@
+# examples/autoloader/mega_customer/safe_dump_fields.rb
+
+# Extend the MegaCustomer class to add safe dump fields
+class MegaCustomer
+  safe_dump_fields :custid, :username, :created_at, :updated_at
+end

data/examples/autoloader/mega_customer.rb

@@ -0,0 +1,17 @@
+# examples/autoloader/mega_customer.rb
+
+require_relative '../../lib/familia'
+
+class MegaCustomer < Familia::Horreum
+  field :custid
+  field :username
+  field :email
+  field :fname
+  field :lname
+  field :display_name
+  field :created_at
+  field :updated_at
+
+  feature :safe_dump
+  # feature :deprecated_fields
+end