familia 2.0.0.pre18 → 2.0.0.pre19

data/docs/migrating/v2.0.0-pre19.md

@@ -0,0 +1,197 @@
+# Migrating Guide: v2.0.0-pre19
+
+This version introduces significant improvements to Familia's database operations, making them more atomic, reliable, and consistent with Rails conventions. The changes include enhanced error handling, optimistic locking support, and breaking API changes.
+
+## Breaking Changes
+
+### Management.create → create!
+
+**What Changed:**
+
+The `create` class method has been renamed to `create!` to follow Rails conventions and indicate that it raises exceptions on failure.
+
+**Before:**
+```ruby
+user = User.create(email: "test@example.com")
+```
+
+**After:**
+```ruby
+user = User.create!(email: "test@example.com")
+```
+
+**Why This Matters:**
+
+- Follows Rails naming conventions where `!` methods raise exceptions
+- Makes it clear that `CreationError` will be raised if object already exists
+- Prevents silent failures in object creation
+
+### save_if_not_exists → save_if_not_exists!
+
+**What Changed:**
+
+The `save_if_not_exists` method has been renamed to `save_if_not_exists!` and now includes optimistic locking with automatic retry logic.
+
+**Before:**
+```ruby
+success = user.save_if_not_exists
+```
+
+**After:**
+```ruby
+success = user.save_if_not_exists!
+```
+
+**Behavior Changes:**
+- Now uses Redis WATCH/MULTI/EXEC for optimistic locking
+- Automatically retries up to 3 times on `OptimisticLockError`
+- Raises `RecordExistsError` if object already exists
+- More atomic and thread-safe
+
+## Enhanced Error Handling
+
+### New Error Hierarchy
+
+**What's New:**
+
+A structured error hierarchy provides better error categorization:
+
+```ruby
+Familia::Problem                    # Base class
+├── Familia::PersistenceError       # Redis/database errors
+│   ├── Familia::NonUniqueKey
+│   ├── Familia::OptimisticLockError
+│   ├── Familia::OperationModeError
+│   ├── Familia::NoConnectionAvailable
+│   ├── Familia::NotFound
+│   └── Familia::NotConnected
+└── Familia::HorreumError           # Model-related errors
+    ├── Familia::CreationError
+    ├── Familia::NoIdentifier
+    ├── Familia::FieldTypeError
+    ├── Familia::AutoloadError
+    ├── Familia::SerializerError
+    ├── Familia::UnknownFieldError
+    └── Familia::NotDistinguishableError
+```
+
+**Migration:**
+
+Update exception handling to use specific error classes:
+
+```ruby
+# Before
+rescue Familia::Problem => e
+  # Handle all errors
+
+# After - More granular handling
+rescue Familia::CreationError => e
+  # Handle creation failures specifically
+rescue Familia::OptimisticLockError => e
+  # Handle concurrent modification
+rescue Familia::PersistenceError => e
+  # Handle database-related errors
+```
+
+### New Exception Types
+
+- **`CreationError`**: Raised when object creation fails (replaces generic errors)
+- **`OptimisticLockError`**: Raised when WATCH fails due to concurrent modification
+- **`UnknownFieldError`**: Raised when referencing non-existent fields
+
+## Database Operation Improvements
+
+### Atomic Save Operations
+
+**What Changed:**
+
+The `save` method now uses a single Redis transaction for complete atomicity:
+
+```ruby
+# All operations now happen atomically:
+# 1. Save all fields (HMSET)
+# 2. Set expiration (EXPIRE)
+# 3. Update indexes
+# 4. Add to instances collection
+```
+
+**Benefits:**
+- Eliminates race conditions during save operations
+- Ensures data consistency across related operations
+- Better performance with fewer round trips
+
+### Enhanced Timestamp Precision
+
+**What Changed:**
+
+Created/updated timestamps now use float values instead of integers for higher precision:
+
+**Before:**
+```ruby
+user.created  # => 1697234567 (integer seconds)
+```
+
+**After:**
+```ruby
+user.created  # => 1697234567.123 (float with milliseconds)
+```
+
+**Migration:**
+
+No code changes needed. Existing integer timestamps continue to work.
+
+## Redis Command Enhancements
+
+### New Commands Available
+
+Added support for optimistic locking commands:
+- `watch(key)` - Watch key for changes
+- `unwatch()` - Remove all watches
+- `discard()` - Discard queued commands
+
+### Improved Command Logging
+
+Database command logging now includes:
+- Structured format for better readability
+- Pipelined operation tracking
+- Transaction boundary markers
+- Command timing information
+
+## Terminology Updates
+
+**What Changed:**
+
+Standardized on "pipelined" terminology throughout (previously mixed "pipeline"/"pipelined").
+
+**Files Affected:**
+- Method names now consistently use "pipelined"
+- Documentation updated to match Redis terminology
+- Log messages standardized
+
+**Migration:**
+
+No code changes needed - this was an internal consistency improvement.
+
+## Recommended Actions
+
+1. **Update method calls:**
+   ```ruby
+   # Replace all instances
+   Model.create(...)      → Model.create!(...)
+   obj.save_if_not_exists → obj.save_if_not_exists!
+   ```
+
+2. **Review error handling:**
+   ```ruby
+   # Consider more specific error handling
+   rescue Familia::CreationError
+   rescue Familia::OptimisticLockError
+   ```
+
+3. **Test concurrent operations:**
+   - The new optimistic locking provides better concurrency handling
+   - Verify your application handles `OptimisticLockError` appropriately
+
+4. **Review logging:**
+   - Enhanced database command logging may affect log volume
+   - Adjust log levels if needed for production environments

data/examples/datatype_standalone.rb

@@ -0,0 +1,281 @@
+#!/usr/bin/env ruby
+# examples/datatype_standalone.rb
+
+# Demonstration: Familia::StringKey for Session Storage with Atomic Transactions
+#
+# This example shows how to use Familia's DataType classes independently
+# without inheriting from Familia::Horreum. It implements a Rack-compatible
+# session store using Familia::StringKey for secure, TTL-managed storage.
+#
+# Key Familia Features Demonstrated:
+# - Standalone DataType usage (no parent model required)
+# - Atomic transactions for multi-operation consistency
+# - TTL management for automatic expiration
+# - JSON serialization for complex data structures
+# - Direct Redis access through DataType objects
+
+require 'rack/session/abstract/id'
+require 'securerandom'
+
+require 'base64'
+require 'openssl'
+
+# Load local development version of Familia (not the gem)
+begin
+  require_relative '../lib/familia'
+rescue LoadError
+  # Fall back to installed gem
+  require 'familia'
+end
+
+# SecureSessionStore - a rack-session compatible session store using Familia::StringKey
+#
+# Usage:
+#   ruby examples/datatype_standalone.rb
+#   # Or in your Rack app:
+#   use SecureSessionStore, secret: 'your-secret-key', expire_after: 3600
+#
+# @see https://raw.githubusercontent.com/rack/rack-session/dadcfe60f193e8/lib/rack/session/abstract/id.rb
+# @see https://raw.githubusercontent.com/rack/rack-session/dadcfe60f193e8/lib/rack/session/encryptor.rb
+#
+class SecureSessionStore < Rack::Session::Abstract::PersistedSecure
+  unless defined?(DEFAULT_OPTIONS)
+    DEFAULT_OPTIONS = {
+      key: 'project.session',
+      expire_after: 86_400, # 24 hours default
+      namespace: 'session',
+      sidbits: 256, # Required by Rack::Session::Abstract::Persisted
+      dbclient: nil,
+    }.freeze
+  end
+
+  attr_reader :dbclient
+
+  def initialize(app, options = {})
+    # Require a secret for security
+    raise ArgumentError, 'Secret required for secure sessions' unless options[:secret]
+
+    # Merge options with defaults
+    options = DEFAULT_OPTIONS.merge(options)
+
+    # Configure Familia connection if redis_uri provided
+    @dbclient = options[:dbclient] || Familia.dbclient
+
+    super
+
+    @secret = options[:secret]
+    @expire_after = options[:expire_after]
+    @namespace = options[:namespace] || 'session'
+
+    # Derive different keys for different purposes
+    @hmac_key = derive_key('hmac')
+    @encryption_key = derive_key('encryption')
+  end
+
+  private
+
+  # Create a StringKey instance for a session ID
+  def get_stringkey(sid)
+    return nil if sid.to_s.empty?
+
+    key = Familia.join(@namespace, sid)
+    Familia::StringKey.new(key,
+      ttl: @expire_after,
+      default: nil)
+  end
+
+  def delete_session(_request, sid, _options)
+    # Extract string ID from SessionId object if needed
+    sid_string = sid.respond_to?(:public_id) ? sid.public_id : sid
+
+    get_stringkey(sid_string)&.del
+
+    generate_sid
+  end
+
+  def valid_session_id?(sid)
+    return false if sid.to_s.empty?
+    return false unless sid.match?(/\A[a-f0-9]{64,}\z/)
+
+    # Additional security checks could go here
+    true
+  end
+
+  def valid_hmac?(data, hmac)
+    expected = compute_hmac(data)
+    return false unless hmac.is_a?(String) && expected.is_a?(String) && hmac.bytesize == expected.bytesize
+
+    Rack::Utils.secure_compare(expected, hmac)
+  end
+
+  def derive_key(purpose)
+    OpenSSL::HMAC.hexdigest('SHA256', @secret, "session-#{purpose}")
+  end
+
+  def compute_hmac(data)
+    OpenSSL::HMAC.hexdigest('SHA256', @hmac_key, data)
+  end
+
+  def find_session(_request, sid)
+    # Parent class already extracts sid from cookies
+    # sid may be a SessionId object or nil
+    sid_string = sid.respond_to?(:public_id) ? sid.public_id : sid
+
+    # Only generate new sid if none provided or invalid
+    return [generate_sid, {}] unless sid_string && valid_session_id?(sid_string)
+
+    begin
+      stringkey = get_stringkey(sid_string)
+      stored_data = stringkey.value if stringkey
+
+      # If no data stored, return empty session
+      return [sid, {}] unless stored_data
+
+      # Verify HMAC before deserializing
+      data, hmac = stored_data.split('--', 2)
+
+      # If no HMAC or invalid format, create new session
+      unless hmac && valid_hmac?(data, hmac)
+        # Session tampered with - create new session
+        return [generate_sid, {}]
+      end
+
+      # Decode and parse the session data
+      session_data = Familia::JsonSerializer.parse(Base64.decode64(data))
+
+      [sid, session_data]
+    rescue Familia::PersistenceError => e
+      # Log error in development/debugging
+      Familia.ld "[Session] Error reading session #{sid_string}: #{e.message}"
+
+      # Return new session on any error
+      [generate_sid, {}]
+    end
+  end
+
+  def write_session(_request, sid, session_data, _options)
+    # Extract string ID from SessionId object if needed
+    sid_string = sid.respond_to?(:public_id) ? sid.public_id : sid
+
+    # Serialize and sign the data
+    encoded = Base64.encode64(Familia::JsonSerializer.dump(session_data)).delete("\n")
+    hmac = compute_hmac(encoded)
+    signed_data = "#{encoded}--#{hmac}"
+
+    # Get or create StringKey for this session
+    stringkey = get_stringkey(sid_string)
+
+    # ATOMIC TRANSACTION: Ensures both operations succeed or both fail
+    #
+    # Before DataType transaction support (PR #160), these operations were not atomic:
+    #   stringkey.set(signed_data)
+    #   stringkey.update_expiration(expiration: @expire_after)
+    #
+    # With transaction support, we guarantee atomicity - critical for session storage
+    # where partial writes could lead to sessions without TTL (memory leaks) or
+    # expired sessions with stale data (security issues).
+    #
+    # RECOMMENDED PATTERN: Use DataType methods inside transaction blocks
+    # The transaction block automatically handles the atomic MULTI/EXEC wrapping.
+    # DataType methods handle key generation and provide clean, expressive syntax.
+    stringkey.transaction do
+      stringkey.set(signed_data)
+      stringkey.update_expiration(expiration: @expire_after) if @expire_after&.positive?
+    end
+
+    # ADVANCED: The block yields the Redis connection for low-level access when needed
+    # This is useful for operations that require direct Redis command access or
+    # when working with multiple DataTypes in a single transaction.
+    #
+    # stringkey.transaction do |conn|
+    #   conn.set(stringkey.dbkey, signed_data)
+    #   conn.expire(stringkey.dbkey, @expire_after) if @expire_after&.positive?
+    # end
+
+    # Return the original sid (may be SessionId object)
+    sid
+  rescue Familia::PersistenceError => e
+    # Log error in development/debugging
+    Familia.ld "[Session] Error writing session #{sid_string}: #{e.message}"
+
+    # Return false to indicate failure
+    false
+  end
+
+  # Clean up expired sessions (optional, can be called periodically)
+  def cleanup_expired_sessions
+    # This would typically be handled by Redis TTL automatically
+    # but you could implement manual cleanup if needed
+  end
+end
+
+# Demo application showing session store in action
+class DemoApp
+  def initialize
+    @store = SecureSessionStore.new(
+      proc { |_env| [200, {}, ['Demo App']] },
+      secret: 'demo-secret-key-change-in-production',
+      expire_after: 300, # 5 minutes for demo
+    )
+  end
+
+  def call(env)
+    puts "\n=== Familia::StringKey Session Demo ==="
+
+    # Mock Rack environment
+    env['rack.session'] ||= {}
+    env['HTTP_COOKIE'] ||= ''
+
+    # Simulate session operations
+    session_id = SecureRandom.hex(32)
+    session_data = {
+      'user_id' => '12345',
+      'username' => 'demo_user',
+      'login_time' => Time.now.to_i,
+      'preferences' => { 'theme' => 'dark', 'lang' => 'en' },
+    }
+
+    puts 'Writing session data...'
+    result = @store.send(:write_session, nil, session_id, session_data, {})
+    puts "   Result: #{result ? 'Success' : 'Failed'}"
+
+    puts "\nReading session data..."
+    found_id, found_data = @store.send(:find_session, nil, session_id)
+    puts "   Session ID: #{found_id}"
+    puts "   Data: #{found_data}"
+
+    puts "\nDeleting session..."
+    @store.send(:delete_session, nil, session_id, {})
+
+    puts "\nVerifying deletion..."
+    deleted_id, deleted_data = @store.send(:find_session, nil, session_id)
+    puts "   Data after deletion: #{deleted_data}"
+    puts "   New session ID: #{deleted_id == session_id ? 'Same' : 'Generated'}"
+
+    puts "\n✅ Demo complete!"
+    puts "\nKey Familia Features Used:"
+    puts '• Familia::StringKey for typed Redis storage'
+    puts '• Automatic TTL management'
+    puts '• Direct Redis operations (set, get, del)'
+    puts '• JSON serialization support'
+    puts '• No Horreum inheritance required'
+
+    [200, { 'Content-Type' => 'text/plain' }, ['Familia StringKey Demo - Check console output']]
+  end
+end
+
+# Run demo if executed directly
+if __FILE__ == $0
+  # Ensure Redis is available
+  begin
+    Familia.dbclient.ping
+  rescue Familia::PersistenceError => e
+    puts "❌ Redis connection failed: #{e.message}"
+    puts '   Please ensure Redis is running on localhost:6379'
+    exit 1
+  end
+
+  # Run the demo
+  app = DemoApp.new
+  app.call({})
+end