familia 2.0.0.pre18 → 2.0.0.pre21

data/docs/guides/index.md

@@ -11,35 +11,36 @@ Welcome to the comprehensive documentation for Familia v2.0. This guide collecti
 
 ### 🏗️ Architecture & System Design
 
-4. **[Feature System](feature-system.md)** - Modular architecture with dependencies and autoloader patterns
-5. **[Feature System for Developers](feature-system-devs.md)** - Advanced feature development patterns
-3. **[Security Model](security-model.md)** - Cryptographic design and Ruby memory considerations
-6. **[Connection Pooling](config-connection-pooling.md)** - Provider pattern for efficient Redis/Valkey pooling
-7. **[Core Field System](core-field-system.md)** - Field definitions and data type mappings
+1. **[Feature System](feature-system.md)** - Modular architecture with dependencies and autoloader patterns
+2. **[Feature System for Developers](feature-system-devs.md)** - Advanced feature development patterns
+3. **[Field System](field-system.md)** - Field definitions and data type mappings
+4. **[Logging](logging.md)** - Logger configuration and database command logging
 
 
-### 🔐 Special Field Types
+### 🔐 Security & Special Fields
 
-1. **[Encrypted Fields](feature-encrypted-fields.md)** - Persistent encrypted storage with modular providers
-2. **[Transient Fields](feature-transient-fields.md)** - Non-persistent secure data handling with RedactedString
-10. **[Object Identifiers](feature-object-identifiers.md)** - Automatic ID generation with configurable strategies _(new!)_
-11. **[External Identifiers](feature-external-identifiers.md)** - Integration with external systems and legacy data _(new!)_
+5. **[Encrypted Fields](feature-encrypted-fields.md)** - Persistent encrypted storage with modular providers
+6. **[Transient Fields](feature-transient-fields.md)** - Non-persistent secure data handling with RedactedString
+7. **[Encryption Guide](encryption.md)** - Legacy encryption documentation
+8. **[Object Identifiers](feature-object-identifiers.md)** - Automatic ID generation with configurable strategies
+9. **[External Identifiers](feature-external-identifiers.md)** - Integration with external systems and legacy data
 
 
-### 🔗 Object Relationships & Identifiers
+### 🔗 Object Relationships
 
-8. **[Relationships](feature-relationships.md)** - Object relationships and membership system
-9. **[Relationship Methods](feature-relationships-methods.md)** - Detailed method reference for relationships
+10. **[Relationships](feature-relationships.md)** - Object relationships and membership system
+11. **[Relationship Methods](feature-relationships-methods.md)** - Detailed method reference for relationships
 
 ### ⏱️ Time & Analytics Features
 
 12. **[Expiration](feature-expiration.md)** - TTL management and cascading expiration
 13. **[Quantization](feature-quantization.md)** - Time-based data bucketing for analytics
-14. **[Time Utilities](time-utilities.md)** - Time manipulation and formatting utilities
+14. **[Time Literals](time-literals.md)** - Time manipulation and formatting utilities
 
 ### 🛠️ Implementation & Usage
 
-15. **[Implementation Guide](implementation.md)** - Advanced configuration and usage patterns
+15. **[Optimized Loading](optimized-loading.md)** - Reduce Redis commands by 50-96% for bulk object loading _(new!)_
+
 
 ## 🚀 Quick Start Examples
 
@@ -168,6 +169,22 @@ end
 MetricsBucket.record_event("page_view")  # Groups into 10-min buckets
 ```
 
+### Optimized Loading (Performance)
+```ruby
+# Skip EXISTS check (50% reduction)
+user = User.find_by_id(123, check_exists: false)
+
+# Pipelined bulk loading (96% reduction for N objects)
+metadata_ids = customer.metadata.rangebyscore(start_time, end_time)
+# => ["id1", "id2", ..., "id14"]  # 14 metadata objects
+
+# Traditional: 28 Redis commands (14 EXISTS + 14 HGETALL)
+metadata = metadata_ids.map { |id| Metadata.find_by_id(id) }
+
+# Optimized: 1 pipelined batch with 14 HGETALL commands
+metadata = Metadata.load_multi(metadata_ids).compact
+```
+
 
 ## Related Resources
 

data/docs/guides/logging.md

@@ -0,0 +1,187 @@
+# Logging Guide
+
+Familia provides comprehensive logging capabilities through a custom logger with TRACE level support and flexible database command logging middleware.
+
+## Logger Configuration
+
+### Default Logger
+
+Familia uses a custom `FamiliaLogger` that extends Ruby's standard Logger:
+
+```ruby
+# Default logger with TRACE support
+logger = Familia.logger
+logger.trace "Detailed debugging information"
+logger.debug "Standard debug message"
+logger.info "General information"
+```
+
+### Custom Logger Replacement
+
+Replace Familia's logger with any Logger-compatible object:
+
+```ruby
+# Use Rails logger
+Familia.logger = Rails.logger
+
+# Custom file logger with rotation
+Familia.logger = Logger.new('familia.log', 'weekly')
+
+# Syslog integration
+require 'syslog/logger'
+Familia.logger = Syslog::Logger.new('familia')
+```
+
+### Custom Formatters
+
+Control log output formatting:
+
+```ruby
+custom_logger = Logger.new($stdout)
+custom_logger.formatter = proc do |severity, datetime, progname, msg|
+  "[#{datetime.strftime('%Y-%m-%d %H:%M:%S')}] #{severity}: #{msg}\n"
+end
+Familia.logger = custom_logger
+```
+
+## Log Levels
+
+### Standard Levels
+- **INFO**: General informational messages
+- **WARN**: Warning messages
+- **ERROR**: Error messages
+- **DEBUG**: Detailed debugging (controlled by `FAMILIA_DEBUG`)
+- **FATAL**: Fatal errors
+
+### TRACE Level
+- **TRACE**: Extremely detailed debugging (requires both `FAMILIA_DEBUG` and `FAMILIA_TRACE`)
+- Uses fiber-local storage to distinguish from DEBUG level
+- Outputs with 'T' severity letter
+
+## Environment Variables
+
+Control logging behavior via environment variables:
+
+- `FAMILIA_DEBUG`: Enable debug-level logging (`1`, `true`, `yes`)
+- `FAMILIA_TRACE`: Enable trace-level logging (`1`, `true`, `yes`)
+
+Both must be enabled for trace logging to work.
+
+## Configuration Block
+
+Enable debug mode programmatically:
+
+```ruby
+Familia.configure do |config|
+  config.debug = true
+end
+```
+
+## Structured Logging
+
+Familia supports structured logging with key-value context:
+
+```ruby
+# Simple message
+Familia.info "Connection established"
+
+# With structured context
+Familia.info "Pipeline executed", commands: 5, duration: 2340
+# Output: "Pipeline executed commands=5 duration=2340"
+
+Familia.debug "Cache lookup", key: "user:123", hit: true
+Familia.error "Serialization failed", field: :email, error: e.message
+```
+
+## Database
+ Command Logging
+
+### DatabaseLogger Middleware
+
+Familia includes `DatabaseLogger` middleware for Redis command logging:
+
+```ruby
+# Enable command logging (uses redis-rb middleware internally)
+Familia.enable_database_logging = true
+
+# Optional: Configure logger (uses Familia.logger by default)
+DatabaseLogger.logger = Familia.logger
+```
+
+**Note**: Familia automatically registers the middleware with redis-rb when enabled. You work with `Redis.new` connections - the underlying `RedisClient` middleware registration is handled internally.
+
+### Output Formats
+
+**Standard Format:**
+```
+T, 20:43:09.843 [123] 0.001234 567μs > SET key value
+```
+
+**Structured Format:**
+```ruby
+DatabaseLogger.structured_logging = true
+# Output: "Redis command cmd=SET args=[key, value] duration_ms=0.42 db=0"
+```
+
+### Sampling
+
+Reduce log
+ volume in high-traffic scenarios:
+
+```ruby
+# Log 10% of commands
+DatabaseLogger.sample_rate = 0.1
+
+# Log 1% of commands (production-friendly)
+DatabaseLogger.sample_rate = 0.01
+
+# Disable sampling (log everything)
+DatabaseLogger.sample_rate = nil
+```
+
+### Command Capture
+
+Capture commands for testing (unaffected by sampling):
+
+```ruby
+commands = DatabaseLogger.capture_commands do
+  redis.set('key', 'value')
+  redis.get('key')
+end
+
+puts commands.first.command  # => "SET key value"
+puts commands.first.μs       # => 567 (microseconds)
+```
+
+## Default Logger Features
+
+Familia's `FamiliaLogger` provides:
+
+- **TRACE Level**: Distinct from DEBUG with 'T' severity marker
+- **Structured Output**: `SEVERITY, HH:MM:SS.mmm MESSAGE` format
+- **Fiber Support**: Thread/fiber-safe operation
+- **Environment Control**: Automatic debug/trace enabling via env vars
+- **LogFormatter**: Custom formatter with severity letters (T/D/I/W/E/F/U)
+
+## Integration Examples
+
+### Rails Integration
+```ruby
+# config/initializers/familia.rb
+Familia.logger = Rails.logger
+Familia.configure do |config|
+  config.debug = Rails.env.development?
+  config.enable_database_logging = Rails.env.development?
+end
+```
+
+### Production Setup
+```ruby
+# Minimal logging with sampling
+Familia.logger = Logger.new('familia.log')
+Familia.logger.level = Logger::INFO
+DatabaseLogger.sample_rate = 0.01  # 1% sampling
+DatabaseLogger.structured_logging = true
+```
+
+This flexible logging system allows integration with existing logging infrastructure while maintaining Familia's specialized debugging capabilities.