familia 2.0.0.pre8 → 2.0.0.pre12

data/docs/migrating/v2.0.0-pre12.md

@@ -0,0 +1,306 @@
+# Migrating Guide: v2.0.0-pre12
+
+This version introduces significant security improvements to Familia's identifier system, including verifiable identifiers with HMAC signatures, scoped identifier namespaces, and hardened external identifier derivation to prevent potential security vulnerabilities.
+
+## VerifiableIdentifier Feature
+
+### Overview
+
+The new `Familia::VerifiableIdentifier` module allows applications to create and verify identifiers with embedded HMAC signatures. This enables stateless confirmation that an identifier was generated by your application, preventing forged IDs from malicious sources.
+
+### Basic Usage
+
+```ruby
+class Customer < Familia::Horreum
+  feature :verifiable_identifier
+
+  # Required: Set the HMAC secret (do this once in your app initialization)
+  # Generate with: SecureRandom.hex(64)
+  ENV['VERIFIABLE_ID_HMAC_SECRET'] = 'your_64_character_hex_secret'
+end
+
+# Generate a verifiable identifier
+customer = Customer.new
+verifiable_id = customer.generate_verifiable_id
+# => "cust_1234567890abcdef_a1b2c3d4e5f6789..."
+
+# Verify the identifier later (stateless verification)
+if Customer.verified_identifier?(verifiable_id)
+  # Identifier is valid and was generated by this application
+  original_id = Customer.extract_identifier(verifiable_id)
+  customer = Customer.new(original_id)
+else
+  # Identifier is forged or corrupted
+  raise SecurityError, "Invalid identifier"
+end
+```
+
+### Scoped VerifiableIdentifier
+
+The new `scope` parameter enables cryptographically isolated identifier namespaces for multi-tenant, multi-domain, or multi-environment applications.
+
+#### Before (Global Scope)
+```ruby
+# All identifiers share the same cryptographic space
+admin_id = admin.generate_verifiable_id
+user_id = user.generate_verifiable_id
+
+# Risk: Cross-contamination between different contexts
+```
+
+#### After (Scoped Namespaces)
+```ruby
+# Production environment
+prod_customer_id = customer.generate_verifiable_id(scope: 'production')
+prod_admin_id = admin.generate_verifiable_id(scope: 'production:admin')
+
+# Development environment
+dev_customer_id = customer.generate_verifiable_id(scope: 'development')
+
+# Multi-tenant application
+tenant_a_id = user.generate_verifiable_id(scope: "tenant:#{tenant_a.id}")
+tenant_b_id = user.generate_verifiable_id(scope: "tenant:#{tenant_b.id}")
+
+# Verification requires matching scope
+Customer.verified_identifier?(prod_customer_id, scope: 'production')  # => true
+Customer.verified_identifier?(prod_customer_id, scope: 'development') # => false
+```
+
+**Scope Benefits:**
+- **Multi-tenant isolation**: Tenant A cannot forge identifiers for Tenant B
+- **Environment separation**: Production IDs cannot be used in development
+- **Role-based security**: Admin scopes separate from user scopes
+- **Full backward compatibility**: Existing code without scopes continues to work
+
+### Key Management
+
+#### Secure Secret Generation
+```ruby
+# Generate a cryptographically secure HMAC secret
+require 'securerandom'
+secret = SecureRandom.hex(64)  # 512-bit secret
+puts "VERIFIABLE_ID_HMAC_SECRET=#{secret}"
+```
+
+#### Environment Configuration
+```ruby
+# config/application.rb or equivalent
+# Set this BEFORE any VerifiableIdentifier usage
+ENV['VERIFIABLE_ID_HMAC_SECRET'] = Rails.application.credentials.verifiable_id_secret
+
+# Or configure programmatically
+Familia::VerifiableIdentifier.hmac_secret = your_secret_string
+```
+
+## ObjectIdentifier Feature Improvements
+
+### Method Renaming
+
+Method names have been updated for clarity and consistency:
+
+#### Before
+```ruby
+customer = Customer.new
+objid = customer.generate_objid           # Unclear what this generates
+extid = Customer.generate_extid(objid)    # Less secure class method
+```
+
+#### After
+```ruby
+customer = Customer.new
+objid = customer.generate_object_identifier    # Clear: generates object ID
+extid = customer.derive_external_identifier    # Clear: derives from objid, instance method
+```
+
+**Migration:**
+- Replace `generate_objid` → `generate_object_identifier`
+- Replace `generate_external_identifier` → `derive_external_identifier`
+- Remove usage of `generate_extid` (deprecated for security reasons)
+
+### Provenance Tracking
+
+ObjectIdentifier now tracks which generator was used for each identifier:
+
+```ruby
+class Customer < Familia::Horreum
+  feature :object_identifier
+
+  # Configure generator type
+  object_identifier_generator :uuid_v7  # or :uuid_v4, :hex, custom proc
+end
+
+customer = Customer.new
+objid = customer.generate_object_identifier
+
+# Provenance information available
+puts customer.object_identifier_generator_type  # => :uuid_v7
+puts customer.objid_format                     # => :uuid (normalized format)
+```
+
+**Benefits:**
+- **Security auditing**: Know which generator created each identifier
+- **Format normalization**: Eliminates ambiguity between UUID and hex formats
+- **Migration support**: Track mixed generator usage during transitions
+
+## ExternalIdentifier Security Hardening
+
+### Provenance Validation
+
+ExternalIdentifier now validates that objid values come from the ObjectIdentifier feature before deriving external identifiers.
+
+#### Before (Potential Security Risk)
+```ruby
+# Could derive external IDs from any string, including malicious input
+extid = customer.derive_external_identifier("malicious_input")
+```
+
+#### After (Hardened)
+```ruby
+customer = Customer.new
+customer.generate_object_identifier  # Must generate objid first
+
+# Only works with validated objid from ObjectIdentifier feature
+extid = customer.derive_external_identifier  # Secure: uses validated objid
+```
+
+### Improved Security Model
+
+External identifiers are now derived using the internal objid as a seed for a new random value, rather than directly deriving from objid.
+
+#### Before
+```ruby
+# Direct derivation could leak information about objid
+extid = hash(objid)  # Information leakage risk
+```
+
+#### After
+```ruby
+# objid used as seed for new random value
+extid = secure_hash(objid + additional_entropy)  # No information leakage
+```
+
+### Error Handling Improvements
+
+External identifier now raises clear errors for invalid usage:
+
+```ruby
+class Customer < Familia::Horreum
+  feature :external_identifier  # Missing: object_identifier dependency
+end
+
+customer = Customer.new
+# Raises ExternalIdentifierError instead of returning nil
+customer.derive_external_identifier
+# => Familia::ExternalIdentifierError: Model does not have an objid field
+```
+
+## Migration Steps
+
+### 1. Update Method Names
+
+Replace deprecated method names in your codebase:
+
+```bash
+# Search and replace patterns:
+grep -r "generate_objid" --include="*.rb" .
+# Replace with: generate_object_identifier
+
+grep -r "generate_external_identifier" --include="*.rb" .
+# Replace with: derive_external_identifier
+
+grep -r "generate_extid" --include="*.rb" .
+# Remove usage - use derive_external_identifier instead
+```
+
+### 2. Add HMAC Secret for VerifiableIdentifier
+
+If you plan to use VerifiableIdentifier:
+
+```ruby
+# Generate secret
+require 'securerandom'
+secret = SecureRandom.hex(64)
+
+# Add to your environment configuration
+# .env, Rails credentials, or similar
+VERIFIABLE_ID_HMAC_SECRET=your_generated_secret
+
+# Verify configuration
+puts ENV['VERIFIABLE_ID_HMAC_SECRET']&.length  # Should be 128 characters
+```
+
+### 3. Update ExternalIdentifier Usage
+
+Ensure proper dependency chain:
+
+```ruby
+class YourModel < Familia::Horreum
+  # Required: ObjectIdentifier must come before ExternalIdentifier
+  feature :object_identifier
+  feature :external_identifier
+
+  # Configure generator if needed
+  object_identifier_generator :uuid_v7
+end
+
+# Usage pattern
+model = YourModel.new
+model.generate_object_identifier  # Generate objid first
+extid = model.derive_external_identifier  # Then derive external ID
+```
+
+### 4. Review Security-Sensitive Code
+
+Audit any code that processes identifiers from external sources:
+
+```ruby
+# Before: Potentially unsafe
+def process_identifier(external_id)
+  # Could process forged identifiers
+  model = Model.find_by_external_id(external_id)
+end
+
+# After: With verification
+def process_identifier(verifiable_id)
+  # Verify identifier authenticity first
+  unless Model.verified_identifier?(verifiable_id)
+    raise SecurityError, "Invalid identifier"
+  end
+
+  original_id = Model.extract_identifier(verifiable_id)
+  model = Model.new(original_id)
+end
+```
+
+## Breaking Changes
+
+1. **`generate_extid` removed** - Use instance-level `derive_external_identifier` instead
+2. **ExternalIdentifier validation** - Now raises `ExternalIdentifierError` instead of returning `nil` for models without objid
+3. **Method names changed** - `generate_objid` → `generate_object_identifier`, `generate_external_identifier` → `derive_external_identifier`
+
+## New Security Capabilities
+
+1. **Cryptographic identifier verification** - Prevent forged IDs with HMAC signatures
+2. **Scoped namespaces** - Isolate identifiers by tenant, environment, or role
+3. **Provenance tracking** - Know which generator created each identifier
+4. **Information leakage prevention** - External IDs no longer directly expose internal IDs
+5. **Input validation** - Clear error messages for invalid operations
+
+## Testing Your Migration
+
+```ruby
+# Test ObjectIdentifier changes
+model = YourModel.new
+objid = model.generate_object_identifier
+extid = model.derive_external_identifier
+puts "Generator: #{model.object_identifier_generator_type}"
+
+# Test VerifiableIdentifier (if using)
+vid = model.generate_verifiable_id
+puts "Verifiable: #{YourModel.verified_identifier?(vid)}"
+
+# Test scoped identifiers (if using)
+scoped_vid = model.generate_verifiable_id(scope: 'production')
+puts "Scoped valid: #{YourModel.verified_identifier?(scoped_vid, scope: 'production')}"
+puts "Wrong scope: #{YourModel.verified_identifier?(scoped_vid, scope: 'development')}"
+```

data/docs/migrating/v2.0.0-pre5.md

@@ -0,0 +1,110 @@
+# Migrating Guide: Security Features (v2.0.0-pre5)
+
+This guide covers adopting the security enhancements introduced in v2.0.0-pre5.
+
+## Security Feature Adoption
+
+### 1. Configure Encryption Keys
+
+Before using encrypted fields, configure encryption keys:
+
+```ruby
+Familia.configure do |config|
+  config.encryption_keys = {
+    v1: 'your-32-byte-base64-encoded-key==',
+    v2: 'newer-32-byte-base64-encoded-key=='
+  }
+  config.current_key_version = :v2
+end
+```
+
+**Key Management:**
+- Use secure key storage (environment variables, key management services)
+- Rotate keys regularly by adding new versions
+- Never remove old key versions while data exists
+
+### 2. Identify Sensitive Fields
+
+Mark fields that contain sensitive data:
+
+**For Encryption:**
+```ruby
+class Vault < Familia::Horreum
+  feature :encrypted_fields
+
+  field :name                    # Plaintext
+  encrypted_field :secret_key    # Encrypted at rest
+  encrypted_field :api_token     # Transparent access
+end
+```
+
+**For Transient Fields:**
+```ruby
+class User < Familia::Horreum
+  feature :transient_fields
+
+  field :email                   # Persisted
+  transient_field :password      # Never persisted
+  transient_field :session_token # Runtime only
+end
+```
+
+### 3. Update Serialization Code
+
+Handle `RedactedString` in serialization:
+
+**Before:**
+```ruby
+def to_json
+  { name: name, password: password }.to_json
+end
+```
+
+**After:**
+```ruby
+def to_json
+  # RedactedString automatically excluded from serialization
+  { name: name }.to_json  # password field omitted if transient
+end
+```
+
+**Manual RedactedString Handling:**
+```ruby
+# Access original value when needed
+password.reveal  # Returns actual string value
+password.redacted?  # Returns true if redacted
+```
+
+### 4. Implement Key Rotation Procedures
+
+**Rotation Process:**
+1. Add new key version to configuration
+2. Update `current_key_version`
+3. Re-encrypt existing data gradually
+4. Remove old keys after migration complete
+
+**Example Rotation Script:**
+```ruby
+# Add new key version
+Familia.config.encryption_keys[:v3] = 'new-key'
+Familia.config.current_key_version = :v3
+
+# Re-encrypt existing records
+Vault.all.each do |vault|
+  vault.save  # Automatically uses new key version
+end
+```
+
+## Security Best Practices
+
+- **Environment Variables:** Store keys in environment variables, not code
+- **Key Rotation:** Rotate encryption keys regularly (quarterly/annually)
+- **Field Selection:** Only encrypt fields that truly need protection
+- **Memory Clearing:** Use transient fields for temporary sensitive data
+- **Logging:** Verify RedactedString prevents accidental logging
+
+## Next Steps
+
+After implementing security features:
+1. Review [Architecture Migration](v2.0.0-pre6.md) for persistence improvements
+2. Explore [Relationships Migration](v2.0.0-pre7.md) for the relationship system

data/docs/migrating/v2.0.0-pre6.md

@@ -0,0 +1,154 @@
+# Migrating Guide: Architecture Improvements (v2.0.0-pre6)
+
+This guide covers the architecture enhancements and new persistence methods in v2.0.0-pre6.
+
+## Architecture Improvements
+
+### 1. Enhanced Persistence Operations
+
+**New `save_if_not_exists` Method:**
+```ruby
+user = User.new(email: 'user@example.com')
+
+# Only save if the user doesn't already exist
+if user.save_if_not_exists
+  puts "User created successfully"
+else
+  puts "User already exists"
+end
+```
+
+**Atomic Persistence with Transactions:**
+```ruby
+user.transaction do |conn|
+  conn.set(user.key, user.serialize)
+  conn.sadd("all_users", user.identifier)
+  conn.expire(user.key, user.ttl) if user.ttl
+end
+```
+
+### 2. Modular Class Structure
+
+The Horreum class structure was reorganized for better maintainability:
+
+**Core Modules:**
+- `Familia::Horreum::Core` - Essential functionality
+- `Familia::Horreum::ClassMethods` - Class-level methods
+- `Familia::Horreum::Serialization` - Object serialization
+- `Familia::Horreum::Commands` - Redis command wrappers
+
+**Feature System Improvements:**
+- Dependency management between features
+- Cleaner feature activation
+- Better error handling for missing dependencies
+
+### 3. Enhanced Error Handling
+
+**New Exception Types:**
+```ruby
+begin
+  user.save!
+rescue Familia::PersistenceError => e
+  puts "Failed to save: #{e.message}"
+rescue Familia::ValidationError => e
+  puts "Validation failed: #{e.message}"
+end
+```
+
+**Improved Data Consistency:**
+- Automatic retry for transient Redis connection issues
+- Better handling of concurrent modifications
+- Enhanced validation before persistence operations
+
+## Migration Steps
+
+### 1. Update Error Handling
+
+**Before (v2.0.0-pre5):**
+```ruby
+begin
+  user.save
+rescue => e
+  puts "Something went wrong: #{e}"
+end
+```
+
+**After (v2.0.0-pre6):**
+```ruby
+begin
+  user.save
+rescue Familia::PersistenceError => e
+  puts "Persistence failed: #{e.message}"
+  # Handle specific persistence issues
+rescue Familia::ValidationError => e
+  puts "Invalid data: #{e.message}"
+  # Handle validation failures
+end
+```
+
+### 2. Adopt Conditional Persistence
+
+Replace existence checks with atomic operations:
+
+**Before:**
+```ruby
+user = User.new(email: email)
+unless User.exists?(email)
+  user.save
+end
+```
+
+**After:**
+```ruby
+user = User.new(email: email)
+user.save_if_not_exists
+```
+
+### 3. Leverage Transaction Support
+
+For complex operations, use transactions:
+
+```ruby
+# Before - Multiple separate operations
+user.save
+user.tags.add(tag)
+user.scores.add(score, timestamp)
+
+# After - Atomic transaction
+user.transaction do |conn|
+  conn.set(user.key, user.serialize)
+  conn.sadd(user.tags.key, tag)
+  conn.zadd(user.scores.key, timestamp, score)
+end
+```
+
+## Performance Improvements
+
+### Connection Management
+- Improved connection pooling with better resource utilization
+- Reduced connection overhead through intelligent connection reuse
+- Enhanced concurrent operation support
+
+### Feature System
+- Lazy feature loading reduces memory footprint
+- Optimized method dispatch for feature methods
+- Better dependency resolution
+
+## Breaking Changes
+
+### Method Signatures
+- Some internal methods changed signatures for better consistency
+- Error handling improved with specific exception types
+- Transaction block interface standardized
+
+### Feature Dependencies
+- Features now explicitly declare dependencies
+- Better error messages for missing feature requirements
+- Automatic dependency resolution where possible
+
+## Next Steps
+
+After completing architecture migration:
+1. Explore [Relationships Migration](v2.0.0-pre7.md) for the comprehensive relationship system
+2. Review updated documentation for architectural patterns
+3. Consider adopting new persistence patterns in your application