familia 2.0.0.pre5 → 2.0.0.pre7

data/docs/wiki/Security-Model.md

@@ -2,19 +2,25 @@
 
 ## Cryptographic Design
 
+### Provider-Based Architecture
+
+Familia uses a modular provider system that automatically selects the best available encryption algorithm:
+
 ### Encryption Algorithms
 
-**Primary (with libsodium):**
-- Algorithm: XChaCha20-Poly1305
-- Key Size: 256 bits
-- Nonce Size: 192 bits
-- Authentication Tag: 128 bits
+**XChaCha20-Poly1305 Provider (Priority: 100)**
+- Requires: `rbnacl` gem (libsodium bindings)
+- Key Size: 256 bits (32 bytes)
+- Nonce Size: 192 bits (24 bytes) - extended nonce space
+- Authentication Tag: 128 bits (16 bytes)
+- Key Derivation: BLAKE2b with personalization string
 
-**Fallback (with OpenSSL):**
-- Algorithm: AES-256-GCM
-- Key Size: 256 bits
-- IV Size: 96 bits
-- Authentication Tag: 128 bits
+**AES-256-GCM Provider (Priority: 50)**
+- Requires: OpenSSL (always available)
+- Key Size: 256 bits (32 bytes)
+- Nonce Size: 96 bits (12 bytes) - standard GCM nonce
+- Authentication Tag: 128 bits (16 bytes)
+- Key Derivation: HKDF-SHA256
 
 ### Key Derivation
 
@@ -26,19 +32,40 @@ Field Key = KDF(Master Key, Context)
 Where Context = "ClassName:field_name:record_identifier"
 ```
 
-**KDF Functions:**
-- Libsodium: BLAKE2b with personalization
-- OpenSSL: HKDF-SHA256
+**Provider-Specific KDF:**
+- **XChaCha20-Poly1305**: BLAKE2b with customizable personalization string
+- **AES-256-GCM**: HKDF-SHA256 with salt and info parameters
+
+The personalization string provides cryptographic domain separation:
+```ruby
+Familia.configure do |config|
+  config.encryption_personalization = 'MyApp-2024'  # Default: 'Familia'
+end
+```
 
 ### Ciphertext Format
 
+The encrypted data is stored as JSON with algorithm-specific fields:
+
+**XChaCha20-Poly1305:**
 ```json
 {
-  "library": "libsodium",
   "algorithm": "xchacha20poly1305",
-  "nonce": "base64_encoded_nonce",
-  "ciphertext": "base64_encoded_ciphertext",
-  "key_version": "v1_2504"
+  "nonce": "base64_24_byte_nonce",
+  "ciphertext": "base64_encrypted_data",
+  "auth_tag": "base64_16_byte_tag",
+  "key_version": "v1"
+}
+```
+
+**AES-256-GCM:**
+```json
+{
+  "algorithm": "aes-256-gcm",
+  "nonce": "base64_12_byte_iv",
+  "ciphertext": "base64_encrypted_data",
+  "auth_tag": "base64_16_byte_tag",
+  "key_version": "v1"
 }
 ```
 
@@ -97,15 +124,28 @@ vault.love_letter(passphrase_value: user_passphrase)
 
 ### Memory Safety
 
-**With libsodium:**
-- Automatic zeroing of sensitive memory
-- Constant-time comparisons
-- Protected memory pages when available
+**⚠️ Critical Ruby Memory Limitations:**
+
+Ruby provides **NO** memory safety guarantees for cryptographic secrets. This affects ALL providers:
+
+- **No secure memory wiping**: Ruby cannot guarantee memory zeroing
+- **GC copying**: Garbage collector may copy secrets before cleanup
+- **String operations**: Every `.dup`, `+`, or interpolation creates uncontrolled copies
+- **Memory dumps**: Secrets may persist in swap files or core dumps
+- **Finalizer uncertainty**: `ObjectSpace.define_finalizer` timing is unpredictable
+
+**Provider-Specific Mitigations:**
+
+Both providers attempt best-effort memory clearing:
+- Call `.clear` on sensitive strings after use
+- Set variables to `nil` when done
+- Use finalizers for cleanup (no guarantees)
 
-**Without libsodium:**
-- Warning logged about reduced security
-- Ruby GC may retain plaintext copies
-- Timing attacks theoretically possible
+**Recommendation**: For production systems with high-security requirements, consider:
+- Hardware Security Modules (HSMs)
+- External key management services
+- Languages with manual memory management (C, Rust)
+- Cryptographic appliances with secure enclaves
 
 ### RedactedString
 

data/docs/wiki/Transient-Fields-Guide.md

@@ -0,0 +1,280 @@
+# Transient Fields Guide
+
+## Overview
+
+Transient fields provide secure handling of sensitive runtime data that should never be persisted to Redis/Valkey. Unlike encrypted fields, transient fields exist only in memory and are automatically wrapped in `RedactedString` for security.
+
+## When to Use Transient Fields
+
+Use transient fields for:
+- API keys and tokens that change frequently
+- Temporary passwords or passphrases
+- Session-specific secrets
+- Any sensitive data that should never touch persistent storage
+- Debug or development secrets that need secure handling
+
+## Basic Usage
+
+### Define Transient Fields
+
+```ruby
+class ApiClient < Familia::Horreum
+  feature :transient_fields
+
+  field :endpoint          # Regular persistent field
+  transient_field :token   # Transient field (not persisted)
+  transient_field :secret, as: :api_secret  # Custom accessor name
+end
+```
+
+### Working with Transient Fields
+
+```ruby
+client = ApiClient.new(
+  endpoint: 'https://api.example.com',
+  token: ENV['API_TOKEN'],
+  secret: ENV['API_SECRET']
+)
+
+# Regular field persists
+client.save
+client.endpoint  # => "https://api.example.com"
+
+# Transient fields are RedactedString instances
+puts client.token  # => "[REDACTED]"
+
+# Access the actual value safely
+client.token.expose do |token|
+  response = HTTP.post(client.endpoint,
+    headers: { 'Authorization' => "Bearer #{token}" }
+  )
+  # Token value is only available within this block
+end
+
+# Explicit cleanup when done
+client.token.clear!
+```
+
+## RedactedString Security
+
+### Automatic Wrapping
+
+All transient field values are automatically wrapped in `RedactedString`:
+
+```ruby
+client = ApiClient.new(token: 'secret123')
+client.token.class  # => RedactedString
+```
+
+### Safe Access Pattern
+
+```ruby
+# ✅ Recommended: Use .expose block
+client.token.expose do |token|
+  # Use token directly without creating copies
+  HTTP.auth("Bearer #{token}")  # Safe
+end
+
+# ✅ Direct access (use carefully)
+raw_token = client.token.value
+# Remember to clear original source if needed
+
+# ❌ Avoid: These create uncontrolled copies
+token_copy = client.token.value.dup      # Creates copy in memory
+interpolated = "Bearer #{client.token}"  # Creates copy via to_s
+```
+
+### Memory Management
+
+```ruby
+# Clear individual fields
+client.token.clear!
+
+# Check if cleared
+client.token.cleared?  # => true
+
+# Accessing cleared values raises error
+client.token.value  # => SecurityError: Value already cleared
+```
+
+## Advanced Features
+
+### Custom Accessor Names
+
+```ruby
+class Service < Familia::Horreum
+  transient_field :api_key, as: :secret_key
+end
+
+service = Service.new(api_key: 'secret123')
+service.secret_key.expose { |key| use_api_key(key) }
+```
+
+### Integration with Encrypted Fields
+
+```ruby
+class SecureService < Familia::Horreum
+  feature :transient_fields
+
+  encrypted_field :long_term_secret    # Persisted, encrypted
+  transient_field :session_token       # Runtime only, not persisted
+  field :public_endpoint               # Normal field
+end
+
+service = SecureService.new(
+  long_term_secret: 'stored encrypted in Redis',
+  session_token: 'temporary runtime secret',
+  public_endpoint: 'https://api.example.com'
+)
+
+service.save
+# Only long_term_secret and public_endpoint are saved to Redis
+# session_token exists only in memory
+```
+
+## RedactedString API Reference
+
+### Core Methods
+
+```ruby
+# Create (usually automatic via transient_field)
+secret = RedactedString.new('sensitive_value')
+
+# Safe access
+secret.expose { |value| use_value(value) }
+
+# Direct access (use with caution)
+value = secret.value
+
+# Cleanup
+secret.clear!
+
+# Status
+secret.cleared?  # => true/false
+```
+
+### Security Methods
+
+```ruby
+# Logging/debugging protection
+puts secret.to_s     # => "[REDACTED]"
+puts secret.inspect  # => "[REDACTED]"
+
+# Equality (object identity only)
+secret1 == secret2   # => false (unless same object)
+
+# Hash (constant for all instances)
+secret.hash  # => Same for all RedactedString instances
+```
+
+## Security Considerations
+
+### Ruby Memory Limitations
+
+**⚠️ Important**: Ruby provides no memory safety guarantees:
+
+- **No secure wiping**: `.clear!` is best-effort only
+- **GC copying**: Garbage collector may duplicate secrets
+- **String operations**: Every manipulation creates copies
+- **Memory persistence**: Secrets may remain in memory indefinitely
+
+### Best Practices
+
+```ruby
+# ✅ Wrap immediately after input
+password = RedactedString.new(params[:password])
+params[:password] = nil  # Clear original reference
+
+# ✅ Use .expose for short operations
+token.expose { |t| api_call(t) }
+
+# ✅ Clear explicitly when done
+token.clear!
+
+# ✅ Avoid string operations that create copies
+token.expose { |t| "Bearer #{t}" }  # Creates copy
+# Better: Pass token directly to methods that need it
+
+# ❌ Don't pass RedactedString to logging
+logger.info "Token: #{token}"  # Still logs "[REDACTED]" but safer to avoid
+
+# ❌ Don't store in instance variables outside field system
+@raw_token = token.value  # Creates uncontrolled copy
+```
+
+### Production Recommendations
+
+For highly sensitive applications, consider:
+- External secrets management (HashiCorp Vault, AWS Secrets Manager)
+- Hardware Security Modules (HSMs)
+- Languages with secure memory handling
+- Encrypted swap and memory protection at OS level
+
+## Integration Examples
+
+### Rails Controller
+
+```ruby
+class ApiController < ApplicationController
+  def authenticate
+    service = ApiService.new(
+      endpoint: params[:endpoint],
+      token: params[:token]  # Auto-wrapped in RedactedString
+    )
+
+    result = service.token.expose do |token|
+      # Token only accessible within this block
+      ExternalAPI.authenticate(token)
+    end
+
+    # Clear token when request is done
+    service.token.clear!
+
+    render json: { status: result }
+  end
+end
+```
+
+### Background Job
+
+```ruby
+class ApiSyncJob
+  def perform(user_id, token)
+    user = User.find(user_id)
+
+    # Wrap external token securely
+    secure_token = RedactedString.new(token)
+    token.clear if token.respond_to?(:clear)  # Clear original
+
+    client = ApiClient.new(token: secure_token)
+
+    begin
+      sync_data(client)
+    ensure
+      client.token.clear!  # Always cleanup
+    end
+  end
+
+  private
+
+  def sync_data(client)
+    client.token.expose do |token|
+      # Use token for API calls
+      fetch_and_process_data(token)
+    end
+  end
+end
+```
+
+## Comparison with Encrypted Fields
+
+| Feature | Encrypted Fields | Transient Fields |
+|---------|------------------|------------------|
+| **Persistence** | Saved to Redis/Valkey | Memory only |
+| **Encryption** | AES/XChaCha20 | None (not stored) |
+| **Use Case** | Long-term secrets | Runtime secrets |
+| **Access** | Automatic decrypt | RedactedString wrapper |
+| **Performance** | Crypto overhead | No crypto operations |
+| **Lifecycle** | Survives restarts | Cleared on restart |
+
+Choose encrypted fields for data that must persist across sessions. Choose transient fields for sensitive runtime data that should never be stored.

data/examples/bit_encoding_integration.rb

@@ -0,0 +1,237 @@
+# examples/bit_encoding_integration.rb
+#
+# Production Integration Example: Document Management System with Fine-Grained Permissions
+#
+# This example demonstrates how to use Familia's bit encoding permission system
+# in a real-world document management scenario with sophisticated access control.
+
+require_relative '../lib/familia'
+require_relative '../lib/familia/features/relationships/score_encoding'
+require_relative '../lib/familia/features/relationships/permission_management'
+
+# Document Management System Classes
+class User < Familia::Horreum
+  logical_database 14
+
+  identifier_field :user_id
+  field :user_id
+  field :email
+  field :name
+  field :role  # admin, editor, viewer, guest
+  field :created_at
+
+  sorted_set :documents  # Documents this user can access
+  sorted_set :recent_activity  # Recent document access
+end
+
+class Document < Familia::Horreum
+  include Familia::Features::Relationships::PermissionManagement
+
+  logical_database 14
+
+  # Enable fine-grained permission tracking
+  permission_tracking :user_permissions
+
+  identifier_field :doc_id
+  field :doc_id
+  field :title
+  field :owner_id
+  field :content
+  field :created_at
+  field :updated_at
+  field :document_type  # public, private, confidential
+
+  sorted_set :collaborators  # Users with access to this document
+  list :audit_log  # Track permission changes and access
+
+  # Add document to user's collection with specific permissions
+  def share_with_user(user, *permissions)
+    permissions = [:read] if permissions.empty?
+
+    # Create time-based score with permissions encoded
+    timestamp = updated_at || Time.now
+    score = Familia::Features::Relationships::ScoreEncoding.encode_score(timestamp, permissions)
+
+    # Add to user's document list
+    user.documents.add(score, doc_id)
+
+    # Add user to document's collaborator list
+    collaborators.add(score, user.user_id)
+
+    # Grant permissions via permission management
+    grant(user, *permissions)
+
+    # Log the permission grant
+    log_entry = "#{Time.now.iso8601}: Granted #{permissions.join(', ')} to #{user.email}"
+    audit_log.push(log_entry)
+  end
+
+  # Remove user access
+  def revoke_access(user)
+    user.documents.zrem(doc_id)
+    collaborators.zrem(user.user_id)
+    revoke(user, :read, :write, :edit, :delete, :configure, :transfer, :admin)
+
+    log_entry = "#{Time.now.iso8601}: Revoked all access from #{user.email}"
+    audit_log.push(log_entry)
+  end
+
+  # Get users with specific permission level or higher
+  def users_with_permission(*required_permissions)
+    all_permissions.select do |user_id, user_perms|
+      required_permissions.all? { |perm| user_perms.include?(perm) }
+    end.keys
+  end
+
+  # Advanced: Get document access history for analytics
+  def access_analytics(days_back = 30)
+    start_time = Time.now - (days_back * 24 * 60 * 60)
+    end_time = Time.now
+
+    # Use score range to get recent access
+    range = Familia::Features::Relationships::ScoreEncoding.score_range(
+      start_time,
+      end_time,
+      min_permissions: [:read]
+    )
+
+    # Get collaborators active in time range
+    active_users = collaborators.rangebyscore(*range)
+
+    {
+      active_users: active_users,
+      total_collaborators: collaborators.size,
+      permission_breakdown: all_permissions,
+      audit_entries: audit_log.range(0, 50)
+    }
+  end
+end
+
+# Document Management Service - Business Logic Layer
+class DocumentService
+  # Permission role definitions matching business needs
+  ROLE_PERMISSIONS = {
+    guest: [:read],
+    viewer: [:read],
+    commenter: [:read, :append],
+    editor: [:read, :write, :edit],
+    reviewer: [:read, :write, :edit, :delete],
+    admin: [:read, :write, :edit, :delete, :configure, :transfer, :admin]
+  }.freeze
+
+  def self.create_document(owner, title, content, doc_type = 'private')
+    doc = Document.new(
+      doc_id: "doc_#{Time.now.to_i}_#{rand(1000)}",
+      title: title,
+      content: content,
+      owner_id: owner.user_id,
+      document_type: doc_type,
+      created_at: Time.now,
+      updated_at: Time.now
+    )
+
+    # Owner gets full admin access
+    doc.share_with_user(owner, *ROLE_PERMISSIONS[:admin])
+    doc
+  end
+
+  def self.share_document(document, user, role)
+    permissions = ROLE_PERMISSIONS[role] || ROLE_PERMISSIONS[:viewer]
+    document.share_with_user(user, *permissions)
+  end
+
+  def self.can_user_perform?(user, document, action)
+    case action
+    when :view, :read
+      document.can?(user, :read)
+    when :comment, :append
+      document.can?(user, :read, :append)
+    when :edit, :modify
+      document.can?(user, :read, :write, :edit)
+    when :delete, :remove
+      document.can?(user, :delete)
+    when :share, :configure
+      document.can?(user, :configure)
+    when :transfer_ownership
+      document.can?(user, :admin)
+    else
+      false
+    end
+  end
+
+  def self.bulk_permission_update(documents, users, role)
+    permissions = ROLE_PERMISSIONS[role]
+
+    documents.each do |doc|
+      users.each do |user|
+        doc.revoke_access(user)  # Clear existing
+        doc.share_with_user(user, *permissions) if permissions
+      end
+    end
+  end
+end
+
+# Example Usage and Demonstration
+if __FILE__ == $0
+  puts "🚀 Familia Bit Encoding Integration Example"
+  puts "=" * 50
+
+  # Create users
+  alice = User.new(user_id: 'alice', email: 'alice@company.com', name: 'Alice Smith', role: 'admin')
+  bob = User.new(user_id: 'bob', email: 'bob@company.com', name: 'Bob Jones', role: 'editor')
+  charlie = User.new(user_id: 'charlie', email: 'charlie@company.com', name: 'Charlie Brown', role: 'viewer')
+
+  # Create documents
+  doc1 = DocumentService.create_document(alice, "Q4 Financial Report", "Confidential financial data...", 'confidential')
+  doc2 = DocumentService.create_document(alice, "Team Meeting Notes", "Weekly standup notes...", 'private')
+  doc3 = DocumentService.create_document(bob, "Project Proposal", "New feature proposal...", 'public')
+
+  # Share documents with different permission levels
+  puts "\n📄 Document Sharing:"
+  DocumentService.share_document(doc1, bob, :reviewer)     # Bob can review financial report
+  DocumentService.share_document(doc1, charlie, :viewer)   # Charlie can only view
+
+  DocumentService.share_document(doc2, bob, :editor)       # Bob can edit meeting notes
+  DocumentService.share_document(doc2, charlie, :commenter) # Charlie can comment
+
+  DocumentService.share_document(doc3, alice, :admin)      # Alice gets admin on Bob's doc
+  DocumentService.share_document(doc3, charlie, :editor)   # Charlie can edit proposal
+
+  # Test permission checks
+  puts "\n🔐 Permission Testing:"
+  puts "Can Bob edit financial report? #{DocumentService.can_user_perform?(bob, doc1, :edit)}"
+  puts "Can Bob delete financial report? #{DocumentService.can_user_perform?(bob, doc1, :delete)}"
+  puts "Can Charlie comment on meeting notes? #{DocumentService.can_user_perform?(charlie, doc2, :comment)}"
+  puts "Can Charlie edit project proposal? #{DocumentService.can_user_perform?(charlie, doc3, :edit)}"
+
+  # Advanced analytics
+  puts "\n📊 Document Analytics:"
+  analytics = doc1.access_analytics
+  puts "Financial Report - Active Users: #{analytics[:active_users].size}"
+  puts "Total Collaborators: #{analytics[:total_collaborators]}"
+  puts "Permission Breakdown:"
+  analytics[:permission_breakdown].each do |user_id, perms|
+    puts "  #{user_id}: #{perms.join(', ')}"
+  end
+
+  # Demonstrate bit encoding efficiency
+  puts "\n⚡ Bit Encoding Efficiency:"
+  score = Familia::Features::Relationships::ScoreEncoding.encode_score(Time.now, [:read, :write, :edit, :delete])
+  decoded = Familia::Features::Relationships::ScoreEncoding.decode_score(score)
+  puts "Encoded score: #{score}"
+  puts "Decoded permissions: #{decoded[:permission_list].join(', ')}"
+  puts "Permission bits: #{decoded[:permissions]} (#{decoded[:permissions].to_s(2).rjust(8, '0')})"
+
+  # Cleanup
+  puts "\n🧹 Cleanup:"
+  [alice, bob, charlie].each { |user| user.documents.clear }
+  [doc1, doc2, doc3].each { |doc| doc.clear_all_permissions; doc.collaborators.clear }
+
+  puts "✅ Integration example completed successfully!"
+  puts "\nThis demonstrates:"
+  puts "• Fine-grained permission management with 8-bit encoding"
+  puts "• Role-based access control with business logic"
+  puts "• Time-based analytics and audit trails"
+  puts "• Efficient Redis storage with sorted sets"
+  puts "• Production-ready error handling and validation"
+end