familia 2.0.0.pre14 → 2.0.0.pre16

data/docs/guides/feature-object-identifiers.md

@@ -0,0 +1,435 @@
+# Object Identifiers Guide
+
+> **💡 Quick Reference**
+>
+> Enable automatic object ID generation with configurable strategies:
+> ```ruby
+> class Document < Familia::Horreum
+>   feature :object_identifier, generator: :uuid_v4
+>   field :title, :content
+> end
+> ```
+
+## Overview
+
+The Object Identifier feature provides automatic generation of unique identifiers for Familia objects. Instead of manually creating identifiers, you can configure different generation strategies that suit your application's needs - from globally unique UUIDs to compact hexadecimal strings.
+
+## Why Use Object Identifiers?
+
+**Consistency**: Ensures all objects have properly formatted, unique identifiers without manual management.
+
+**Flexibility**: Different applications need different ID formats - UUIDs for distributed systems, short hex strings for internal tools, or custom formats for specific business requirements.
+
+**Collision Avoidance**: Built-in collision detection and retry logic ensures identifier uniqueness even under high concurrency.
+
+**Integration Ready**: Generated IDs work seamlessly with external APIs, logging systems, and database relationships.
+
+## Quick Start
+
+### Basic UUID Generation
+
+```ruby
+class User < Familia::Horreum
+  feature :object_identifier, generator: :uuid_v4
+
+  field :name, :email, :created_at
+end
+
+# Automatic ID generation on creation
+user = User.create(name: "Alice", email: "alice@example.com")
+puts user.objid  # => "f47ac10b-58cc-4372-a567-0e02b2c3d479"
+```
+
+### Compact Hex Identifiers
+
+```ruby
+class Session < Familia::Horreum
+  feature :object_identifier, generator: :hex, length: 16
+
+  field :user_id, :data, :expires_at
+end
+
+session = Session.create(user_id: "user123")
+puts session.objid  # => "a1b2c3d4e5f67890"
+```
+
+## Generator Types
+
+### UUID v4 Generator
+
+Standard UUID format providing global uniqueness across distributed systems.
+
+```ruby
+class Document < Familia::Horreum
+  feature :object_identifier, generator: :uuid_v4
+  field :title, :content
+end
+
+doc = Document.create(title: "My Document")
+doc.objid  # => "550e8400-e29b-41d4-a716-446655440000"
+```
+
+**Characteristics:**
+- **Format**: 36 characters (8-4-4-4-12 hex pattern)
+- **Uniqueness**: Globally unique across time and space
+- **Performance**: Good for distributed systems
+- **Use Cases**: Public APIs, microservices, external integrations
+
+> **💡 Best Practice**
+>
+> Use UUID v4 for objects that will be exposed externally or across service boundaries.
+
+### Hex Generator
+
+Compact hexadecimal strings ideal for internal use and high-volume scenarios.
+
+```ruby
+class ApiKey < Familia::Horreum
+  feature :object_identifier, generator: :hex, length: 24
+  field :name, :permissions, :created_at
+end
+
+key = ApiKey.create(name: "Production API")
+key.objid  # => "1a2b3c4d5e6f7890abcdef12"
+```
+
+**Configuration Options:**
+- `length`: Number of hex characters (default: 12)
+- `prefix`: Optional prefix for the identifier
+- `charset`: Custom character set (default: hex digits)
+
+```ruby
+class InternalToken < Familia::Horreum
+  feature :object_identifier,
+          generator: :hex,
+          length: 16,
+          prefix: "tk_"
+
+  field :scope, :issued_at
+end
+
+token = InternalToken.create(scope: "read:users")
+token.objid  # => "tk_a1b2c3d4e5f67890"
+```
+
+**Characteristics:**
+- **Format**: Configurable length hexadecimal string
+- **Performance**: Very fast generation
+- **Storage**: Compact representation
+- **Use Cases**: Internal IDs, tokens, session identifiers
+
+> **⚠️ Important**
+>
+> Hex generators provide good uniqueness but aren't globally unique like UUIDs. Use appropriate length for your collision tolerance.
+
+### Custom Generator
+
+Define your own identifier generation logic for business-specific requirements.
+
+```ruby
+class OrderNumber < Familia::Horreum
+  feature :object_identifier, generator: :custom
+
+  field :customer_id, :amount, :created_at
+
+  # Custom generator implementation
+  def self.generate_identifier
+    timestamp = Time.now.strftime('%Y%m%d')
+    sequence = Redis.current.incr("order_sequence:#{timestamp}")
+    "ORD-#{timestamp}-#{sequence.to_s.rjust(6, '0')}"
+  end
+end
+
+order = OrderNumber.create(customer_id: "cust123", amount: 99.99)
+order.objid  # => "ORD-20241215-000001"
+```
+
+**Implementation Requirements:**
+- Must define `self.generate_identifier` class method
+- Should return a string identifier
+- Must handle uniqueness and collision scenarios
+- Consider thread safety for concurrent access
+
+> **🔧 Advanced Pattern**
+>
+> Custom generators can integrate with external services, database sequences, or business rules for sophisticated ID schemes.
+
+## Advanced Configuration
+
+### Collision Detection
+
+Enable automatic collision detection and retry logic:
+
+```ruby
+class Product < Familia::Horreum
+  feature :object_identifier,
+          generator: :hex,
+          collision_check: true,
+          max_retries: 5
+
+  field :name, :price, :sku
+end
+```
+
+**Configuration Options:**
+- `collision_check`: Enable/disable collision detection (default: true)
+- `max_retries`: Maximum retry attempts on collision (default: 3)
+- `retry_delay`: Delay between retries in seconds (default: 0.001)
+
+### Identifier Validation
+
+Add custom validation logic for generated identifiers:
+
+```ruby
+class SecureToken < Familia::Horreum
+  feature :object_identifier, generator: :custom
+
+  def self.generate_identifier
+    loop do
+      candidate = SecureRandom.alphanumeric(32)
+      # Ensure no ambiguous characters
+      next if candidate.match?(/[0O1lI]/)
+      return "st_#{candidate.downcase}"
+    end
+  end
+
+  def self.valid_identifier?(id)
+    id.match?(/^st_[a-z0-9]{32}$/) && !id.match?(/[0O1lI]/)
+  end
+end
+```
+
+## Performance Considerations
+
+### Generation Speed Benchmarks
+
+Different generators have varying performance characteristics:
+
+```ruby
+# Benchmark different generators
+require 'benchmark'
+
+Benchmark.bm(10) do |x|
+  x.report("UUID v4:")    { 10_000.times { SecureRandom.uuid } }
+  x.report("Hex 12:")     { 10_000.times { SecureRandom.hex(6) } }
+  x.report("Hex 24:")     { 10_000.times { SecureRandom.hex(12) } }
+  x.report("Custom:")     { 10_000.times { MyClass.generate_identifier } }
+end
+```
+
+### Memory Usage
+
+- **UUID v4**: 36 bytes per identifier
+- **Hex**: Variable based on length (2 bytes per hex character)
+- **Custom**: Depends on implementation
+
+### Collision Probability
+
+For hex generators, collision probability depends on length and volume:
+
+```ruby
+# Approximate collision probability for hex identifiers
+def collision_probability(length, count)
+  total_space = 16 ** length
+  1 - Math.exp(-(count * (count - 1)) / (2.0 * total_space))
+end
+
+# Examples:
+collision_probability(12, 1_000_000)   # Very low
+collision_probability(8, 100_000)      # Consider longer length
+```
+
+> **📊 Sizing Guidance**
+>
+> - **8 hex chars**: Good for < 10K objects
+> - **12 hex chars**: Good for < 1M objects
+> - **16 hex chars**: Good for < 100M objects
+> - **UUID v4**: Suitable for any scale
+
+## Integration Patterns
+
+### External API Integration
+
+```ruby
+class ExternalReference < Familia::Horreum
+  feature :object_identifier, generator: :uuid_v4
+  field :external_id, :sync_status, :last_sync
+
+  def sync_to_external_api
+    response = ExternalAPI.create_record(
+      id: self.objid,  # Use generated ID
+      data: self.to_h
+    )
+
+    self.external_id = response['id']
+    self.sync_status = 'synced'
+    self.last_sync = Familia.now.to_i
+    save
+  end
+end
+```
+
+### Database Relationships
+
+```ruby
+class Order < Familia::Horreum
+  feature :object_identifier, generator: :custom
+  field :customer_id, :total_amount
+
+  def self.generate_identifier
+    "ORD-#{SecureRandom.hex(8).upcase}"
+  end
+end
+
+class OrderItem < Familia::Horreum
+  feature :object_identifier, generator: :hex
+  field :order_id, :product_id, :quantity
+
+  def order
+    Order.load(order_id)
+  end
+end
+
+# Usage
+order = Order.create(customer_id: "cust123", total_amount: 299.99)
+item = OrderItem.create(
+  order_id: order.objid,  # Reference by generated ID
+  product_id: "prod456",
+  quantity: 2
+)
+```
+
+### Logging and Debugging
+
+Generated identifiers provide excellent debugging context:
+
+```ruby
+class UserSession < Familia::Horreum
+  feature :object_identifier, generator: :hex, length: 16
+  field :user_id, :ip_address, :user_agent
+
+  def log_activity(action)
+    logger.info(
+      "Session #{objid}: User #{user_id} performed #{action}",
+      session_id: objid,
+      user_id: user_id,
+      action: action,
+      timestamp: Familia.now.to_i
+    )
+  end
+end
+```
+
+## Testing Strategies
+
+### Test Identifier Generation
+
+```ruby
+# test/models/user_test.rb
+require 'test_helper'
+
+class UserTest < Minitest::Test
+  def test_uuid_generation
+    user = User.create(name: "Test User")
+
+    # Verify UUID format
+    assert_match(
+      /\A[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}\z/i,
+      user.objid
+    )
+  end
+
+  def test_hex_generation
+    session = Session.create(user_id: "123")
+
+    # Verify hex format and length
+    assert_match(/\A[0-9a-f]{16}\z/i, session.objid)
+    assert_equal 16, session.objid.length
+  end
+
+  def test_custom_identifier_format
+    order = OrderNumber.create(customer_id: "cust123")
+
+    # Verify custom format
+    assert_match(/\AORD-\d{8}-\d{6}\z/, order.objid)
+  end
+end
+```
+
+### Mock Generators for Testing
+
+```ruby
+# test/test_helper.rb
+class TestIdentifierGenerator
+  def self.generate_test_uuid
+    "test-#{Time.now.to_f}-#{rand(1000)}"
+  end
+end
+
+# In tests
+class User < Familia::Horreum
+  feature :object_identifier, generator: :custom
+
+  def self.generate_identifier
+    if Rails.env.test?
+      TestIdentifierGenerator.generate_test_uuid
+    else
+      SecureRandom.uuid
+    end
+  end
+end
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**Identifier Not Generated**
+```ruby
+# Ensure feature is enabled
+class MyModel < Familia::Horreum
+  feature :object_identifier  # This line is required!
+  field :name
+end
+```
+
+**Custom Generator Not Called**
+```ruby
+# Verify method signature
+def self.generate_identifier  # Must be class method
+  # Implementation here
+end
+```
+
+**Collision Detection Failing**
+```ruby
+# Check Valkey/Redis connectivity and permissions
+begin
+  MyModel.create(name: "test")
+rescue Familia::Problem => e
+  puts "Identifier collision: #{e.message}"
+end
+```
+
+### Debug Identifier Generation
+
+```ruby
+# Enable debug logging
+Familia.debug = true
+
+# Check feature configuration
+MyModel.feature_options(:object_identifier)
+#=> {generator: :uuid_v4, collision_check: true, max_retries: 3}
+
+# Verify generation manually
+MyModel.generate_identifier  # Should return new identifier
+```
+
+---
+
+## See Also
+
+- **[Technical Reference](../reference/api-technical.md#object-identifier-feature-v200-pre7)** - Implementation details and advanced patterns
+- **[External Identifiers Guide](feature-external-identifiers.md)** - Integration with external systems
+- **[Feature System Guide](feature-system.md)** - Understanding the feature architecture
+- **[Implementation Guide](implementation.md)** - Advanced configuration patterns

data/docs/guides/{Quantization-Feature-Guide.md → feature-quantization.md}

@@ -1,19 +1,64 @@
 # Quantization Feature Guide
 
-## Overview
+The Quantization feature revolutionizes how you handle time-based data by providing automatic bucketing, consistent timestamps, and analytics-ready data organization. Instead of dealing with precise timestamps that make aggregation difficult, quantization rounds time values to predictable intervals, enabling efficient caching, analytics dashboards, and time-series data management.
 
-The Quantization feature provides time-based data bucketing capabilities for Familia objects. It allows you to round timestamps to specific intervals (quantums) and format them for consistent time-based data organization, analytics, and caching strategies.
+> [!NOTE]
+> **Perfect For:** Analytics dashboards, time-series metrics, cache keys, data retention policies, and any scenario where you need consistent time-based data grouping.
+
+## Understanding Time Quantization
+
+### The Problem with Precise Timestamps
+
+Without quantization, time-based data becomes fragmented and difficult to aggregate:
+
+```ruby
+# Problem: Each request gets a unique timestamp
+user_activity_14_30_23 = "login at 2023-06-15 14:30:23"
+user_activity_14_30_45 = "login at 2023-06-15 14:30:45"
+user_activity_14_31_12 = "login at 2023-06-15 14:31:12"
+
+# Result: Three separate data points instead of aggregated hourly data
+# Makes analytics queries complex and cache keys unpredictable
+```
+
+### The Quantization Solution
+
+Quantization groups timestamps into consistent buckets:
+
+```ruby
+# Solution: All timestamps within an hour become the same bucket
+qstamp(1.hour, time: Time.parse("2023-06-15 14:30:23"))  # => 1687276800 (14:00:00)
+qstamp(1.hour, time: Time.parse("2023-06-15 14:30:45"))  # => 1687276800 (14:00:00)
+qstamp(1.hour, time: Time.parse("2023-06-15 14:31:12"))  # => 1687276800 (14:00:00)
+
+# Result: Single aggregated data point for the entire hour
+# Perfect for analytics and predictable cache keys!
+```
+
+> [!NOTE]
+> **Key Benefits:**
+> - **Consistent Buckets**: All timestamps in a period map to the same value
+> - **Predictable Keys**: Cache keys and identifiers become deterministic
+> - **Efficient Aggregation**: Reduce millions of data points to manageable buckets
+> - **Analytics Ready**: Perfect for dashboards requiring time-series data
 
 ## Core Concepts
 
-### Quantum Intervals
+### Quantum Intervals Explained
 
-A **quantum** is a time interval used to bucket timestamps. Common quantums include:
+A **quantum** represents the time bucket size for grouping timestamps. Think of it as the "resolution" of your time-based data:
 
-- **Minutes**: `1.minute`, `5.minutes`, `15.minutes`
-- **Hours**: `1.hour`, `6.hours`, `12.hours`
-- **Days**: `1.day`, `7.days`
-- **Custom**: Any number of seconds (e.g., `90` for 1.5 minutes)
+**Common Quantum Patterns:**
+- **High-Resolution**: `1.minute`, `5.minutes`, `15.minutes` - For real-time monitoring
+- **Medium-Resolution**: `1.hour`, `6.hours`, `12.hours` - For hourly analytics
+- **Low-Resolution**: `1.day`, `1.week`, `1.month` - For long-term trends
+- **Custom Intervals**: Any number of seconds (e.g., `90` for 1.5 minutes, `300` for 5 minutes)
+
+> [!TIP]
+> **Choosing the Right Quantum:**
+> - **Smaller quantums** = More granular data, more storage
+> - **Larger quantums** = Less granular data, less storage
+> - Consider your analytics needs and storage constraints
 
 ### Quantized Timestamps (qstamp)
 
@@ -33,21 +78,28 @@ qstamp(1.hour, pattern: '%H:%M:%S', time: Time.parse('14:05:12'))  # => "14:00:0
 qstamp(1.hour, pattern: '%H:%M:%S', time: Time.parse('14:55:33'))  # => "14:00:00"
 ```
 
-## Basic Usage
+## Getting Started with Quantization
+
+### Enabling the Feature
 
-### Enabling Quantization
+The quantization feature integrates seamlessly with your existing Familia models:
 
 ```ruby
 class AnalyticsEvent < Familia::Horreum
   feature :quantization
-  default_expiration 300  # 5 minutes (used as default quantum)
+  default_expiration 300  # 5 minutes (also used as default quantum)
 
   identifier_field :event_id
   field :event_id, :event_type, :user_id, :data, :timestamp
 end
 ```
 
-### Simple Quantized Timestamps
+> [!TIP]
+> If you set `default_expiration`, it automatically becomes your default quantum for `qstamp` calls without an explicit interval.
+
+### Your First Quantized Timestamps
+
+The `qstamp` method is your main tool for creating consistent time buckets:
 
 ```ruby
 event = AnalyticsEvent.new
@@ -65,6 +117,9 @@ AnalyticsEvent.qstamp(1.hour)
 # => 1687276800
 ```
 
+> [!IMPORTANT]
+> The `qstamp` method always rounds DOWN to the quantum boundary. A timestamp of 14:30:45 with a 1-hour quantum becomes 14:00:00, not 15:00:00.
+
 ### Formatted Timestamps
 
 ```ruby
@@ -145,7 +200,7 @@ class UserActivity < Familia::Horreum
     bucket.user_count ||= 0
     bucket.event_count ||= 0
 
-    # Use Redis sets/hashes for precise counting
+    # Use Valkey/Redis sets/hashes for precise counting
     bucket_users = bucket.related_set("users")
     bucket_users.add(user_id)
 
@@ -172,7 +227,7 @@ end
 
 # Usage
 UserActivity.record_activity('user_123', 'page_view')
-hourly_buckets = UserActivity.activity_for_hour(Time.now)
+hourly_buckets = UserActivity.activity_for_hour(Familia.now)
 ```
 
 ### Time-Series Data Storage
@@ -217,14 +272,14 @@ end
 
 # Usage - Record CPU usage every minute
 TimeSeriesMetric.record_metric('cpu_usage', 75.5, 1.minute)
-TimeSeriesMetric.record_metric('cpu_usage', 82.1, 1.minute, Time.now + 1.minute)
+TimeSeriesMetric.record_metric('cpu_usage', 82.1, 1.minute, Familia.now + 1.minute)
 
 # Retrieve data for last hour
 series_data = TimeSeriesMetric.get_series(
   'cpu_usage',
   1.minute,
-  Time.now - 1.hour,
-  Time.now
+  Familia.now - 1.hour,
+  Familia.now
 )
 ```
 
@@ -252,7 +307,7 @@ class LogAggregator < Familia::Horreum
     )
 
     aggregator.count += 1
-    aggregator.last_seen = Time.now.to_i
+    aggregator.last_seen = Familia.now.to_i
 
     # Keep sample messages (up to 10)
     sample_key = "sample_#{aggregator.count}"
@@ -265,8 +320,8 @@ class LogAggregator < Familia::Horreum
   end
 
   def self.error_summary(time_range = 1.hour)
-    start_time = Time.now - time_range
-    end_time = Time.now
+    start_time = Familia.now - time_range
+    end_time = Familia.now
 
     # Find all error buckets in time range
     buckets = []
@@ -300,7 +355,7 @@ class ApplicationMetrics
   include Familia::Horreum
   feature :quantization
 
-  # Set up different quantum intervals for different metrics
+  # UnsortedSet up different quantum intervals for different metrics
   QUANTUM_CONFIGS = {
     real_time: 1.minute,    # High frequency metrics
     standard: 5.minutes,    # Regular analytics
@@ -325,7 +380,7 @@ class MetricsCollector
       # Increment counter
       Familia.dbclient.incr(key)
 
-      # Set expiration based on quantum type
+      # UnsortedSet expiration based on quantum type
       ttl = case quantum_type
             when :real_time then 2.hours
             when :standard then 1.day
@@ -352,7 +407,7 @@ class QuantizedDataProcessor
     process_bucket(current_bucket)
 
     # Also process previous bucket in case of delayed data
-    previous_bucket = AnalyticsEvent.qstamp(5.minutes, time: Time.now - 5.minutes)
+    previous_bucket = AnalyticsEvent.qstamp(5.minutes, time: Familia.now - 5.minutes)
     process_bucket(previous_bucket)
   end
 
@@ -463,13 +518,13 @@ class GlobalMetrics < Familia::Horreum
 
   def self.utc_hourly_key(time = nil)
     # Always quantize in UTC for global consistency
-    utc_time = time&.utc || Time.now.utc
+    utc_time = time&.utc || Familia.now
     qstamp(1.hour, pattern: '%Y%m%d%H', time: utc_time)
   end
 
   def self.local_daily_key(timezone, time = nil)
     # Quantize in local timezone for regional reports
-    local_time = time || Time.now
+    local_time = time || Familia.now
     local_time = local_time.in_time_zone(timezone) if local_time.respond_to?(:in_time_zone)
     qstamp(1.day, pattern: '%Y%m%d', time: local_time)
   end
@@ -526,7 +581,7 @@ class CompactTimeSeriesStorage < Familia::Horreum
   identifier_field :series_id
   field :series_id, :metric_name, :quantum
 
-  # Store quantized data in Redis sorted sets for efficiency
+  # Store quantized data in Valkey/Redis sorted sets for efficiency
   def record_value(value, time = nil)
     bucket_timestamp = self.class.qstamp(quantum, time: time)
 
@@ -534,7 +589,7 @@ class CompactTimeSeriesStorage < Familia::Horreum
     data_key = "#{series_id}:data"
     Familia.dbclient.zadd(data_key, bucket_timestamp, value)
 
-    # Set TTL based on quantum (longer quantum = longer retention)
+    # UnsortedSet TTL based on quantum (longer quantum = longer retention)
     ttl = calculate_retention_period
     Familia.dbclient.expire(data_key, ttl)
   end
@@ -696,8 +751,8 @@ end
 class QuantizationMonitor
   def self.analyze_bucket_distribution(metric_name, quantum, time_range = 24.hours)
     buckets = {}
-    current_time = Time.now - time_range
-    end_time = Time.now
+    current_time = Familia.now - time_range
+    end_time = Familia.now
 
     while current_time <= end_time
       bucket = AnalyticsEvent.qstamp(quantum, time: current_time)
@@ -718,4 +773,14 @@ class QuantizationMonitor
 end
 ```
 
+---
+
+## See Also
+
+- **[Technical Reference](../reference/api-technical.md#quantization-feature-v200-pre7)** - Implementation details and advanced patterns
+- **[Overview](../overview.md#time-based-quantization)** - Conceptual introduction to quantization
+- **[Time Utilities Guide](time-utilities.md)** - Time manipulation and formatting utilities
+- **[Feature System Guide](feature-system.md)** - Understanding Familia's feature architecture
+- **[Implementation Guide](implementation.md)** - Production deployment and configuration patterns
+
 The Quantization feature provides powerful time-based data organization capabilities, enabling efficient analytics, caching, and time-series data management in Familia applications.