decision_agent 0.1.2 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5711ffe8ff5b3de7d808292d9cda3a0d8818ff77684b2da9a51a4794546a18b5
-  data.tar.gz: f0aba632f6da2eed6b0a07ca3c1ce73404652fbb4b57c8f812b4b9c679cbc1b4
+  metadata.gz: fac7efbaf0c8afd76053d21e611eca64f5880d5b6434720fd90ae34549f6244d
+  data.tar.gz: ecc22d0a9bb19053fd103b06f2c1be768db432b6d9ed715eac8adddcbd6f68dc
 SHA512:
-  metadata.gz: 99b980e8fec99a261db15b0042891675903b83cf238f14604a7222c4efa30be3153c9ce5eda2857fcd1167cd30b4b1fd33090e4ff87d770959a5ba8b5fd7013b
-  data.tar.gz: be51208704d78c7b76b4dc956384d246e30d64ed8edcfe4d162e5ebd1221365a1e26057628634a56d62f0d04ac45c5482efb7556d614c6f28e6792c515276470
+  metadata.gz: b7a0be6ad95512697b6e49a8145668cde64256051db57a39cf925edbd93886d7772cb93026a9af7a9eaea2e9968756ec628ca94d0ebc5a0d1aef0c409eeae97d
+  data.tar.gz: 06777a3c4155b1adff552f712a05a0d4bd2694c6f8af631e36611528c050cf1ea711268230dba6d1a64522e127c0c312b7221fd5d36bdc56f69009b0a98c17e4

data/README.md

@@ -59,7 +59,11 @@ puts result.explanations  # => ["High value transaction"]
 
 ## Web UI - Visual Rule Builder
 
-Launch the visual rule builder for non-technical users:
+The DecisionAgent Web UI provides a visual interface for building and testing rules.
+
+### Standalone Usage
+
+Launch the visual rule builder:
 
 ```bash
 decision_agent web
@@ -67,45 +71,119 @@ decision_agent web
 
 Open [http://localhost:4567](http://localhost:4567) in your browser.
 
-<img width="1602" alt="DecisionAgent Web UI" src="https://github.com/user-attachments/assets/6ee6859c-f9f2-4f93-8bff-923986ccb1bc" />
+### Mount in Rails
 
-## Documentation
+Add to your `config/routes.rb`:
+
+```ruby
+require 'decision_agent/web/server'
+
+Rails.application.routes.draw do
+  # Mount DecisionAgent Web UI
+  mount DecisionAgent::Web::Server, at: '/decision_agent'
+end
+```
+
+Then visit `http://localhost:3000/decision_agent` in your browser.
+
+**With Authentication:**
+
+```ruby
+authenticate :user, ->(user) { user.admin? } do
+  mount DecisionAgent::Web::Server, at: '/decision_agent'
+end
+```
+
+### Mount in Rack/Sinatra Apps
+
+```ruby
+# config.ru
+require 'decision_agent/web/server'
+
+map '/decision_agent' do
+  run DecisionAgent::Web::Server
+end
+```
+
+<img width="1622" height="820" alt="Screenshot" src="https://github.com/user-attachments/assets/687e9ff6-669a-40f9-be27-085c614392d4" />
+
+See [Web UI Rails Integration Guide](wiki/WEB_UI_RAILS_INTEGRATION.md) for detailed setup instructions.
+
+## Monitoring & Analytics
+
+Real-time monitoring, metrics, and alerting for production environments.
+
+### Quick Start
+
+```ruby
+require 'decision_agent/monitoring/metrics_collector'
+require 'decision_agent/monitoring/dashboard_server'
+
+# Initialize metrics collection
+collector = DecisionAgent::Monitoring::MetricsCollector.new(window_size: 3600)
+
+# Start real-time dashboard
+DecisionAgent::Monitoring::DashboardServer.start!(
+  port: 4568,
+  metrics_collector: collector
+)
+
+# Record decisions
+agent = DecisionAgent::Agent.new(evaluators: [evaluator])
+result = agent.decide(context: { amount: 1500 })
+collector.record_decision(result, context, duration_ms: 25.5)
+```
+
+Open [http://localhost:4568](http://localhost:4568) for the monitoring dashboard.
+
+### Features
+
+- **Real-time Dashboard** - Live metrics with WebSocket updates
+- **Prometheus Export** - Industry-standard metrics format
+- **Intelligent Alerting** - Anomaly detection with customizable rules
+- **Grafana Integration** - Pre-built dashboards and alert rules
+- **Custom KPIs** - Track business-specific metrics
+- **Thread-Safe** - Production-ready performance
+
+### Prometheus & Grafana
 
+```yaml
+# prometheus.yml
+scrape_configs:
+  - job_name: 'decision_agent'
+    static_configs:
+      - targets: ['localhost:4568']
+    metrics_path: '/metrics'
 ```
-📚 DecisionAgent Documentation
-│
-├── 🚀 Getting Started
-│   ├── Installation (above)
-│   ├── Quick Start (above)
-│   └── Examples → examples/README.md
-│
-├── 📖 Core Documentation
-│   ├── Core Concepts → wiki/CORE_CONCEPTS.md
-│   ├── JSON Rule DSL → wiki/JSON_RULE_DSL.md
-│   ├── API Reference → wiki/API_CONTRACT.md
-│   └── Error Handling → wiki/ERROR_HANDLING.md
-│
-├── 🎯 Advanced Features
-│   ├── Versioning System → wiki/VERSIONING.md
-│   ├── Decision Replay → wiki/REPLAY.md
-│   ├── Advanced Usage → wiki/ADVANCED_USAGE.md
-│   └── Custom Components → wiki/ADVANCED_USAGE.md#custom-components
-│
-├── 🔌 Integration Guides
-│   ├── Rails Integration → wiki/INTEGRATION.md#rails
-│   ├── Redmine Plugin → wiki/INTEGRATION.md#redmine
-│   ├── Standalone Service → wiki/INTEGRATION.md#standalone
-│   └── Testing Guide → wiki/TESTING.md
-│
-├── 🎨 Web UI
-│   ├── User Guide → wiki/WEB_UI.md
-│   └── Setup Guide → wiki/WEB_UI_SETUP.md
-│
-└── 📝 Reference
-    ├── Changelog → wiki/CHANGELOG.md
-    └── Full Wiki Index → wiki/README.md
+
+Import the pre-built Grafana dashboard from [grafana/decision_agent_dashboard.json](grafana/decision_agent_dashboard.json).
+
+### Alert Management
+
+```ruby
+alert_manager = DecisionAgent::Monitoring::AlertManager.new(
+  metrics_collector: collector
+)
+
+# Add alert rules
+alert_manager.add_rule(
+  name: 'High Error Rate',
+  condition: AlertManager.high_error_rate(threshold: 0.1),
+  severity: :critical
+)
+
+# Register alert handlers
+alert_manager.add_handler do |alert|
+  SlackNotifier.notify("🚨 #{alert[:message]}")
+end
+
+# Start monitoring
+alert_manager.start_monitoring(interval: 60)
 ```
 
+See [Monitoring & Analytics Guide](wiki/MONITORING_AND_ANALYTICS.md) for complete documentation.
+
+
 ## Key Features
 
 ### Decision Making
@@ -124,11 +202,21 @@ Open [http://localhost:4567](http://localhost:4567) in your browser.
 - **JSON Rule DSL** - Non-technical users can write rules
 - **Visual Rule Builder** - Web UI for rule management
 
+### Monitoring & Observability
+- **Real-time Metrics** - Live dashboard with WebSocket updates (<1 second latency)
+- **Prometheus Export** - Industry-standard metrics format at `/metrics` endpoint
+- **Intelligent Alerting** - Anomaly detection with customizable rules and severity levels
+- **Grafana Integration** - Pre-built dashboards and alert configurations in `grafana/` directory
+- **Custom KPIs** - Track business-specific metrics with thread-safe operations
+- **MonitoredAgent** - Drop-in replacement that auto-records all metrics
+- **AlertManager** - Built-in anomaly detection (error rates, latency spikes, low confidence)
+
 ### Production Ready
 - **Comprehensive Testing** - 90%+ code coverage
 - **Error Handling** - Clear, actionable error messages
 - **Versioning** - Full rule version control and rollback
 - **Performance** - Fast, zero external dependencies
+- **Thread-Safe** - Safe for multi-threaded servers and background jobs
 
 ## Examples
 
@@ -162,6 +250,68 @@ rules = {
 
 See [examples/](examples/) for complete working examples.
 
+## Thread-Safety Guarantees
+
+DecisionAgent is designed to be **thread-safe and FAST** for use in multi-threaded environments:
+
+### Performance
+- **10,000+ decisions/second** throughput
+- **~0.1ms average latency** per decision
+- **Zero performance overhead** from thread-safety
+- **Linear scalability** with thread count
+
+### Safe Concurrent Usage
+- **Agent instances** can be shared across threads safely
+- **Evaluators** are immutable after initialization
+- **Decisions and Evaluations** are deeply frozen
+- **File storage** uses mutex-protected operations
+
+### Best Practices
+```ruby
+# Safe: Reuse agent instance across threads
+agent = DecisionAgent::Agent.new(evaluators: [evaluator])
+
+Thread.new { agent.decide(context: { user_id: 1 }) }
+Thread.new { agent.decide(context: { user_id: 2 }) }
+
+# Safe: Share evaluators across agent instances
+evaluator = DecisionAgent::Evaluators::JsonRuleEvaluator.new(rules_json: rules)
+agent1 = DecisionAgent::Agent.new(evaluators: [evaluator])
+agent2 = DecisionAgent::Agent.new(evaluators: [evaluator])
+```
+
+### What's Frozen
+All data structures are deeply frozen to prevent mutation:
+- Decision objects (decision, confidence, explanations, evaluations)
+- Evaluation objects (decision, weight, reason, metadata)
+- Context data
+- Rule definitions in evaluators
+
+This ensures safe concurrent access without race conditions.
+
+### RFC 8785 Canonical JSON
+DecisionAgent uses **RFC 8785 (JSON Canonicalization Scheme)** for deterministic audit hashing:
+
+- **Industry Standard** - Official IETF specification for canonical JSON
+- **Cryptographically Sound** - Ensures deterministic hashing of decision payloads
+- **Reproducible** - Same decision always produces same audit hash
+- **Interoperable** - Compatible with other systems using RFC 8785
+
+Every decision includes a deterministic SHA-256 hash in the audit payload, enabling:
+- Tamper detection in audit logs
+- Exact replay verification
+- Regulatory compliance documentation
+
+Learn more: [RFC 8785 Specification](https://datatracker.ietf.org/doc/html/rfc8785)
+
+### Performance Benchmark
+Run the included benchmark to verify zero overhead:
+```bash
+ruby examples/thread_safe_performance.rb
+```
+
+See [THREAD_SAFETY.md](wiki/THREAD_SAFETY.md) for detailed implementation guide and [PERFORMANCE_AND_THREAD_SAFETY.md](wiki/PERFORMANCE_AND_THREAD_SAFETY.md) for detailed performance analysis.
+
 ## When to Use DecisionAgent
 
 ✅ **Perfect for:**
@@ -176,6 +326,33 @@ See [examples/](examples/) for complete working examples.
 - Pure AI/ML with no rules
 - Single-step validations
 
+## Documentation
+
+**Getting Started**
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Examples](examples/README.md)
+
+**Core Features**
+- [Versioning System](wiki/VERSIONING.md) - Version control for rules
+- [Web UI](wiki/WEB_UI.md) - Visual rule builder
+- [Web UI Setup](wiki/WEB_UI_SETUP.md) - Setup guide
+- [Web UI Rails Integration](wiki/WEB_UI_RAILS_INTEGRATION.md) - Mount in Rails/Rack apps
+- [Monitoring & Analytics](wiki/MONITORING_AND_ANALYTICS.md) - Real-time monitoring, metrics, and alerting
+- [Monitoring Architecture](wiki/MONITORING_ARCHITECTURE.md) - System architecture and design
+
+**Performance & Thread-Safety**
+- [Performance & Thread-Safety Summary](wiki/PERFORMANCE_AND_THREAD_SAFETY.md) - Benchmarks and production readiness
+- [Thread-Safety Implementation](wiki/THREAD_SAFETY.md) - Technical implementation guide
+
+**Reference**
+- [API Contract](wiki/API_CONTRACT.md) - Full API reference
+- [Changelog](wiki/CHANGELOG.md) - Version history
+
+**More Resources**
+- [Wiki Home](wiki/README.md) - Documentation index
+- [GitHub Issues](https://github.com/samaswin87/decision_agent/issues) - Report bugs or request features
+
 ## Contributing
 
 1. Fork the repository

data/bin/decision_agent

@@ -24,15 +24,13 @@ def print_help
       decision_agent version
 
     For more information, visit:
-    https://github.com/yourusername/decision_agent
+    https://github.com/samaswin87/decision_agent
   HELP
 end
 
 def start_web_ui(port = 4567)
   # Ruby 4.0 compatibility: Puma expects Bundler::ORIGINAL_ENV which was removed
-  if defined?(Bundler) && !Bundler.const_defined?(:ORIGINAL_ENV)
-    Bundler.const_set(:ORIGINAL_ENV, ENV.to_h.dup)
-  end
+  Bundler.const_set(:ORIGINAL_ENV, ENV.to_h.dup) if defined?(Bundler) && !Bundler.const_defined?(:ORIGINAL_ENV)
 
   puts "🎯 Starting DecisionAgent Rule Builder..."
   puts "📍 Server: http://localhost:#{port}"
@@ -60,19 +58,16 @@ def validate_file(filepath)
     puts "   Version: #{data['version'] || data[:version]}"
     puts "   Ruleset: #{data['ruleset'] || data[:ruleset]}"
     puts "   Rules: #{data['rules']&.size || 0}"
-
   rescue JSON::ParserError => e
     puts "❌ JSON Parsing Error:"
     puts "   #{e.message}"
     exit 1
-
   rescue DecisionAgent::InvalidRuleDslError => e
     puts "❌ Validation Failed:"
     puts ""
     puts e.message
     exit 1
-
-  rescue => e
+  rescue StandardError => e
     puts "❌ Unexpected Error:"
     puts "   #{e.message}"
     exit 1

data/lib/decision_agent/ab_testing/ab_test.rb

@@ -0,0 +1,197 @@
+module DecisionAgent
+  module ABTesting
+    # Represents an A/B test configuration for comparing rule versions
+    class ABTest
+      attr_reader :id, :name, :champion_version_id, :challenger_version_id,
+                  :traffic_split, :start_date, :end_date, :status
+
+      # @param name [String] Name of the A/B test
+      # @param champion_version_id [String, Integer] ID of the current/champion version
+      # @param challenger_version_id [String, Integer] ID of the new/challenger version
+      # @param options [Hash] Optional configuration
+      # @option options [Hash] :traffic_split Traffic distribution (default: { champion: 90, challenger: 10 })
+      # @option options [Time] :start_date When the test starts (defaults to now)
+      # @option options [Time] :end_date When the test ends (optional)
+      # @option options [String] :status Test status: running, completed, cancelled, scheduled
+      # @option options [String, Integer] :id Optional ID (for persistence)
+      def initialize(
+        name:,
+        champion_version_id:,
+        challenger_version_id:,
+        **options
+      )
+        @id = options[:id]
+        @name = name
+        @champion_version_id = champion_version_id
+        @challenger_version_id = challenger_version_id
+        @traffic_split = normalize_traffic_split(options[:traffic_split] || { champion: 90, challenger: 10 })
+        @start_date = options[:start_date] || Time.now.utc
+        @end_date = options[:end_date]
+        @status = options[:status] || "scheduled"
+
+        validate!
+      end
+
+      # Assign a variant based on traffic split
+      # Uses consistent hashing to ensure same user gets same variant
+      # @param user_id [String, nil] Optional user identifier for consistent assignment
+      # @return [Symbol] :champion or :challenger
+      def assign_variant(user_id: nil)
+        raise TestNotRunningError, "Test '#{@name}' is not running (status: #{@status})" unless running?
+
+        if user_id
+          # Consistent hashing: same user always gets same variant
+          hash_value = Digest::SHA256.hexdigest("#{@id}:#{user_id}").to_i(16)
+          percentage = hash_value % 100
+        else
+          # Random assignment
+          percentage = rand(100)
+        end
+
+        percentage < @traffic_split[:champion] ? :champion : :challenger
+      end
+
+      # Get the version ID for the assigned variant
+      # @param variant [Symbol] :champion or :challenger
+      # @return [String, Integer] The version ID
+      def version_for_variant(variant)
+        case variant
+        when :champion
+          @champion_version_id
+        when :challenger
+          @challenger_version_id
+        else
+          raise ArgumentError, "Invalid variant: #{variant}. Must be :champion or :challenger"
+        end
+      end
+
+      # Check if test is currently running
+      # @return [Boolean]
+      def running?
+        return false unless @status == "running"
+        return false if @start_date && Time.now.utc < @start_date
+        return false if @end_date && Time.now.utc > @end_date
+
+        true
+      end
+
+      # Check if test is scheduled to start
+      # @return [Boolean]
+      def scheduled?
+        @status == "scheduled" && @start_date && Time.now.utc < @start_date
+      end
+
+      # Check if test is completed
+      # @return [Boolean]
+      def completed?
+        @status == "completed" || (@end_date && Time.now.utc > @end_date)
+      end
+
+      # Start the test
+      def start!
+        raise InvalidStatusTransitionError, "Cannot start test from status: #{@status}" unless can_start?
+
+        @status = "running"
+        @start_date = Time.now.utc if @start_date.nil? || @start_date > Time.now.utc
+      end
+
+      # Complete the test
+      def complete!
+        raise InvalidStatusTransitionError, "Cannot complete test from status: #{@status}" unless can_complete?
+
+        @status = "completed"
+        @end_date = Time.now.utc
+      end
+
+      # Cancel the test
+      def cancel!
+        raise InvalidStatusTransitionError, "Cannot cancel test from status: #{@status}" if @status == "completed"
+
+        @status = "cancelled"
+      end
+
+      # Convert to hash representation
+      # @return [Hash]
+      def to_h
+        {
+          id: @id,
+          name: @name,
+          champion_version_id: @champion_version_id,
+          challenger_version_id: @challenger_version_id,
+          traffic_split: @traffic_split,
+          start_date: @start_date,
+          end_date: @end_date,
+          status: @status
+        }
+      end
+
+      private
+
+      def validate!
+        raise ValidationError, "Test name is required" if @name.nil? || @name.strip.empty?
+        raise ValidationError, "Champion version ID is required" if @champion_version_id.nil?
+        raise ValidationError, "Challenger version ID is required" if @challenger_version_id.nil?
+        raise ValidationError, "Champion and challenger must be different versions" if @champion_version_id == @challenger_version_id
+
+        validate_traffic_split!
+        validate_dates!
+        validate_status!
+      end
+
+      def validate_traffic_split!
+        raise ValidationError, "Traffic split must be a Hash" unless @traffic_split.is_a?(Hash)
+
+        unless @traffic_split.key?(:champion) && @traffic_split.key?(:challenger)
+          raise ValidationError,
+                "Traffic split must have :champion and :challenger keys"
+        end
+
+        total = @traffic_split[:champion] + @traffic_split[:challenger]
+        raise ValidationError, "Traffic split must sum to 100, got #{total}" unless total == 100
+
+        raise ValidationError, "Traffic percentages must be non-negative" if @traffic_split.values.any?(&:negative?)
+      end
+
+      def validate_dates!
+        return unless @start_date && @end_date
+
+        raise ValidationError, "End date must be after start date" if @end_date <= @start_date
+      end
+
+      def validate_status!
+        valid_statuses = %w[scheduled running completed cancelled]
+        return if valid_statuses.include?(@status)
+
+        raise ValidationError, "Invalid status: #{@status}. Must be one of: #{valid_statuses.join(', ')}"
+      end
+
+      def normalize_traffic_split(split)
+        case split
+        when Hash
+          # Handle both string and symbol keys
+          {
+            champion: (split[:champion] || split["champion"] || 50).to_i,
+            challenger: (split[:challenger] || split["challenger"] || 50).to_i
+          }
+        when Array
+          # Handle array format [90, 10]
+          { champion: split[0].to_i, challenger: split[1].to_i }
+        else
+          raise ValidationError, "Traffic split must be a Hash or Array"
+        end
+      end
+
+      def can_start?
+        %w[scheduled].include?(@status)
+      end
+
+      def can_complete?
+        %w[running].include?(@status)
+      end
+    end
+
+    # Custom errors
+    class TestNotRunningError < StandardError; end
+    class InvalidStatusTransitionError < StandardError; end
+  end
+end

data/lib/decision_agent/ab_testing/ab_test_assignment.rb

@@ -0,0 +1,76 @@
+module DecisionAgent
+  module ABTesting
+    # Tracks individual assignments of users/requests to A/B test variants
+    class ABTestAssignment
+      attr_reader :id, :ab_test_id, :user_id, :variant, :version_id,
+                  :timestamp, :decision_result, :confidence, :context
+
+      # @param ab_test_id [String, Integer] The A/B test ID
+      # @param variant [Symbol] :champion or :challenger
+      # @param version_id [String, Integer] The rule version ID that was used
+      # @param options [Hash] Optional configuration
+      # @option options [String] :user_id User identifier (optional)
+      # @option options [Time] :timestamp When the assignment occurred
+      # @option options [String] :decision_result The decision outcome
+      # @option options [Float] :confidence Confidence score of the decision
+      # @option options [Hash] :context Additional context for the decision
+      # @option options [String, Integer] :id Optional ID (for persistence)
+      def initialize(
+        ab_test_id:,
+        variant:,
+        version_id:,
+        **options
+      )
+        @id = options[:id]
+        @ab_test_id = ab_test_id
+        @user_id = options[:user_id]
+        @variant = variant
+        @version_id = version_id
+        @timestamp = options[:timestamp] || Time.now.utc
+        @decision_result = options[:decision_result]
+        @confidence = options[:confidence]
+        @context = options[:context] || {}
+
+        validate!
+      end
+
+      # Update the assignment with decision results
+      # @param decision [String] The decision result
+      # @param confidence [Float] The confidence score
+      def record_decision(decision, confidence)
+        @decision_result = decision
+        @confidence = confidence
+      end
+
+      # Convert to hash representation
+      # @return [Hash]
+      def to_h
+        {
+          id: @id,
+          ab_test_id: @ab_test_id,
+          user_id: @user_id,
+          variant: @variant,
+          version_id: @version_id,
+          timestamp: @timestamp,
+          decision_result: @decision_result,
+          confidence: @confidence,
+          context: @context
+        }
+      end
+
+      private
+
+      def validate!
+        raise ValidationError, "AB test ID is required" if @ab_test_id.nil?
+        raise ValidationError, "Variant is required" if @variant.nil?
+        raise ValidationError, "Version ID is required" if @version_id.nil?
+
+        raise ValidationError, "Variant must be :champion or :challenger, got: #{@variant}" unless %i[champion challenger].include?(@variant)
+
+        return unless @confidence && (@confidence.negative? || @confidence > 1)
+
+        raise ValidationError, "Confidence must be between 0 and 1, got: #{@confidence}"
+      end
+    end
+  end
+end