active_genie 0.0.8 → 0.0.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 834787b505134623a3f6c4995c9dec8a0ce89a1a014600c4d676d8801bf541cd
-  data.tar.gz: 16ae75ffa3926ecd87052d72ca3f5434be80327b712bb2dba54e55c46fa5ff8d
+  metadata.gz: 9d0424a39ba21d821cb2419730387e1b026c35b5e2e5dff9f6d615f3ec54e6a3
+  data.tar.gz: 17b460ccd1a689d0f8709af2b84f3cde65aa0075b76104ee1b4bb8b3b0ffc182
 SHA512:
-  metadata.gz: 1772fbfa59891e7a3673b50a54bd90914007917854e78fb5f0458db681bf89bd1ec118600d23ab98d6255e1b063b5d582d3b73dc738e66aeadc7f4f0bee75af8
-  data.tar.gz: bb6a0ea9ef737f7a2ed3ec558dcea5c67bcc2f90a155828e1e25a5a7a3ebd9d0fe6c5422dd9f3d9ff6fb667b63822e3bc86627cdb3e40a3c72c4ef50cee32c68
+  metadata.gz: ad98b2d5d063d0d1c4009e1a9f92d6d326ed948cbdee71317c94b6a4a0ee57042609c1f405ca7ce00d4beb321bc1e98ad722646349076929bcdc0a28da8b6b8b
+  data.tar.gz: d2cc39b77757619c5235041f12d5182778b4237444f3c2246982ebcf54c0542af0da194783cea57fbe4fbde0985ef635802c648796b4f5c4d7a5d4f42c6519c7

data/README.md

@@ -10,9 +10,7 @@ ActiveGenie is a Ruby gem that provides a polished, production-ready interface f
 
 - 🎯 **Data Extraction**: Extract structured data from unstructured text with type validation
 - 📊 **Smart Scoring**: Multi-reviewer evaluation system with automatic expert selection
-- 💭 **Sentiment Analysis**: Advanced sentiment analysis with customizable rules
-- 🔒 **Safe & Secure**: Built-in validation and sanitization
-- 🛠️ **Configurable**: Supports multiple GenAI providers and models
+- 💭 **Leaderboard**: Consistent rank items based on custom criteria, using multiple tecniques of ranking
 
 ## Installation
 
@@ -32,19 +30,13 @@ echo "ActiveGenie.load_tasks" >> Rakefile
 rails g active_genie:install
 ```
 
-4. [Optional] Configure your credentials in `config/active_genie.yml`:
-```yaml
-GPT-4o-mini:
-  api_key: <%= ENV['OPENAI_API_KEY'] %>
-  provider: "openai"
-
-claude-3-5-sonnet:
-  api_key: <%= ENV['ANTHROPIC_API_KEY'] %>
-  provider: "anthropic"
+4. Configure your credentials in `config/initializers/active_genie.rb`:
+```ruby
+ActiveGenie.configure do |config|
+  config.openai.api_key = ENV['OPENAI_API_KEY']
+end
 ```
 
-> The first key will be used as default in all modules, in this example `GPT-4o-mini`
-
 ## Quick Start
 
 ### Data Extractor
@@ -94,7 +86,7 @@ Text evaluation system that provides detailed scoring and feedback using multipl
 text = "The code implements a binary search algorithm with O(log n) complexity"
 criteria = "Evaluate technical accuracy and clarity"
 
-result = ActiveGenie::Scoring::Basic.call(text, criteria)
+result = ActiveGenie::Scoring.basic(text, criteria)
 # => {
 #      algorithm_expert_score: 95,
 #      algorithm_expert_reasoning: "Accurately describes binary search and its complexity",
@@ -122,7 +114,7 @@ player_a = "Implementation uses dependency injection for better testability"
 player_b = "Code has high test coverage but tightly coupled components"
 criteria = "Evaluate code quality and maintainability"
 
-result = ActiveGenie::Battle::Basic.call(player_a, player_b, criteria)
+result = ActiveGenie::Battle.call(player_a, player_b, criteria)
 # => {
 #      winner_player: "Implementation uses dependency injection for better testability",
 #      reasoning: "Player A's implementation demonstrates better maintainability through dependency injection, 
@@ -140,6 +132,29 @@ Features:
 
 See the [Battle README](lib/active_genie/battle/README.md) for advanced usage, custom reviewers, and detailed interface documentation.
 
+### League
+The League module provides competitive ranking through multi-stage evaluation:
+
+
+```ruby
+require 'active_genie'
+
+players = ['REST API', 'GraphQL API', 'SOAP API', 'gRPC API', 'Websocket API']
+criteria = "Best one to be used into a high changing environment"
+
+result = ActiveGenie::League.call(players, criteria)
+# => {
+#      winner_player: "gRPC API",
+#      reasoning: "gRPC API is the best one to be used into a high changing environment",
+#    }
+```
+
+- **Multi-phase ranking system** combining expert scoring and ELO algorithms
+- **Automatic elimination** of inconsistent performers using statistical analysis
+- **Dynamic ranking adjustments** based on simulated pairwise battles, from bottom to top
+
+See the [League README](lib/active_genie/league/README.md) for implementation details, configuration, and advanced ranking strategies.
+
 ### Summarizer (WIP)
 The summarizer is a tool that can be used to summarize a given text. It uses a set of rules to summarize the text out of the box. Uses the best practices of prompt engineering and engineering to make the summarization as accurate as possible.
 
@@ -184,22 +199,9 @@ sentiment = ActiveGenie::SentimentAnalyzer.call(text)
 puts sentiment # => "positive"
 ```
 
-### Elo ranking (WIP)
-The Elo ranking is a tool that can be used to rank a set of items. It uses a set of rules to rank the items out of the box. Uses the best practices of prompt engineering and engineering to make the ranking as accurate as possible.
-
-```ruby
-require 'active_genie'
+## Configuration
 
-items = ['Square', 'Circle', 'Triangle']
-criterias = 'items that look rounded'
-ranked_items = ActiveGenie::EloRanking.call(items, criterias, rounds: 10)
-puts ranked_items # => [{ name: "Circle", score: 1500 }, { name: "Square", score: 800 }, { name: "Triangle", score: 800 }]
-```
-
-
-## Configuration Options
-
-| Option | Description | Default |
+| Config | Description | Default |
 |--------|-------------|---------|
 | `provider` | LLM provider (openai, anthropic, etc) | `nil` |
 | `model` | Model to use | `nil` |
@@ -207,7 +209,7 @@ puts ranked_items # => [{ name: "Circle", score: 1500 }, { name: "Square", score
 | `timeout` | Request timeout in seconds | `5` |
 | `max_retries` | Maximum retry attempts | `3` |
 
-> **Note:** Each module can append its own set of configuration options, see the individual module documentation for details.
+> **Note:** Each module can append its own set of configuration, see the individual module documentation for details.
 
 ## Contributing
 
@@ -219,4 +221,3 @@ puts ranked_items # => [{ name: "Circle", score: 1500 }, { name: "Square", score
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
-

data/VERSION

@@ -1 +1 @@
-0.0.8
+0.0.10

data/lib/active_genie/battle/README.md

@@ -27,11 +27,11 @@ result = ActiveGenie::Battle::Basic.call(player_a, player_b, criteria)
 ```
 
 ## Interface
-### Basic.call(player_a, player_b, criteria, options: {})
+### Basic.call(player_a, player_b, criteria, config: {})
 - `player_a` [String, Hash] - The content or submission from the first player
 - `player_b` [String, Hash] - The content or submission from the second player
 - `criteria` [String] - The evaluation criteria or rules to assess against
-- `options` [Hash] - Additional configuration options that modify the battle evaluation behavior
+- `config` [Hash] - Additional configuration config that modify the battle evaluation behavior
 
 Returns a Hash containing:
 - `winner_player` [String, Hash] - The winning player's content (either player_a or player_b)

data/lib/active_genie/battle/basic.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require_relative '../clients/router'
+require_relative '../clients/unified_client'
 
 module ActiveGenie::Battle
   # The Basic class provides a foundation for evaluating battles between two players
@@ -14,23 +14,23 @@ module ActiveGenie::Battle
   #   Basic.call("Player A content", "Player B content", "Evaluate keyword usage and pattern matching")
   #
   class Basic
-    def self.call(player_a, player_b, criteria, options: {})
-      new(player_a, player_b, criteria, options:).call
+    def self.call(player_a, player_b, criteria, config: {})
+      new(player_a, player_b, criteria, config:).call
     end
 
     # @param player_a [String] The content or submission from the first player
     # @param player_b [String] The content or submission from the second player
     # @param criteria [String] The evaluation criteria or rules to assess against
-    # @param options [Hash] Additional configuration options that modify the battle evaluation behavior
+    # @param config [Hash] Additional configuration config that modify the battle evaluation behavior
     # @return [Hash] The evaluation result containing the winner and reasoning
     #   @return [String] :winner The @param player_a or player_b
     #   @return [String] :reasoning Detailed explanation of why the winner was chosen
     #   @return [String] :what_could_be_changed_to_avoid_draw A suggestion on how to avoid a draw
-    def initialize(player_a, player_b, criteria, options: {})
+    def initialize(player_a, player_b, criteria, config: {})
       @player_a = player_a
       @player_b = player_b
       @criteria = criteria
-      @options = options
+      @config = config
       @response = nil
     end
 
@@ -42,7 +42,7 @@ module ActiveGenie::Battle
         {  role: 'user', content: "player_b: #{player_content(@player_b)}" },
       ]
 
-      @response = ::ActiveGenie::Clients::Router.function_calling(messages, FUNCTION, options: @options)
+      @response = ::ActiveGenie::Clients::UnifiedClient.function_calling(messages, FUNCTION, config:)
 
       response_formatted
     end
@@ -56,18 +56,12 @@ module ActiveGenie::Battle
     end
 
     def response_formatted
-      if @response['winner'] == 'player_a'
-        @response['winner'] = @player_a
-        @response['loser'] = @player_b
-      elsif @response['winner'] == 'player_b'
-        @response['winner'] = @player_b
-        @response['loser'] = @player_a
-      else
-        @response['winner'] = nil
-        @response['loser'] = nil
-      end
-
-      @response
+      winner = case @response['winner']
+               when 'player_a' then @player_a
+               when 'player_b' then @player_b
+               end
+
+      @response.merge!('winner' => winner, 'loser' => winner ? (winner == @player_a ? @player_b : @player_a) : nil)
     end
 
     PROMPT = <<~PROMPT
@@ -121,5 +115,16 @@ module ActiveGenie::Battle
         }
       }
     }
+
+    def config
+      {
+        all_providers: { model_tier: 'lower_tier' },
+        log: {
+          **(@config.dig(:log) || {}),
+          trace: self.class.name,
+        },
+        **@config
+      }
+    end
   end
 end

data/lib/active_genie/battle.rb

@@ -2,7 +2,7 @@
 require_relative 'battle/basic'
 
 module ActiveGenie
-  # Battle module
+  # See the [Battle README](lib/active_genie/battle/README.md) for more information.
   module Battle
     module_function
 

data/lib/active_genie/clients/openai_client.rb

@@ -0,0 +1,77 @@
+require 'json'
+require 'net/http'
+
+module ActiveGenie::Clients
+  class OpenaiClient
+    def initialize(config)
+      @app_config = config
+    end
+
+    def function_calling(messages, function, config: {})
+      model = config[:model]
+      model = @app_config.tier_to_model(config.dig(:all_providers, :model_tier)) if model.nil? && config.dig(:all_providers, :model_tier)
+      model = @app_config.lower_tier_model if model.nil?
+
+      payload = {
+        messages:,
+        response_format: {
+          type: 'json_schema',
+          json_schema: function
+        },
+        model:,
+      }
+
+      api_key = config[:api_key] || @app_config.api_key
+      headers = DEFAULT_HEADERS.merge(
+        'Authorization': "Bearer #{api_key}"
+      ).compact
+
+      response = request(payload, headers, config:)
+
+      parsed_response = JSON.parse(response.dig('choices', 0, 'message', 'content'))
+      parsed_response.dig('properties') || parsed_response
+    rescue JSON::ParserError
+      nil
+    end
+
+    private
+
+    def request(payload, headers, config:)
+      start_time = Time.now
+      response = Net::HTTP.post(
+        URI("#{@app_config.api_url}/chat/completions"),
+        payload.to_json,
+        headers
+      )
+
+      raise OpenaiError, response.body unless response.is_a?(Net::HTTPSuccess)
+      return nil if response.body.empty?
+
+      parsed_body = JSON.parse(response.body)
+      log_response(start_time, parsed_body, config:)
+
+      parsed_body
+    end
+
+    DEFAULT_HEADERS = {
+      'Content-Type': 'application/json',
+    }
+
+    def log_response(start_time, response, config:)
+      ActiveGenie::Logger.trace(
+        {
+          **config.dig(:log),
+          category: :llm,
+          trace: "#{config.dig(:log, :trace)}/#{self.class.name}",
+          total_tokens: response.dig('usage', 'total_tokens'),
+          model: response.dig('model'),
+          request_duration: Time.now - start_time,
+          openai: response
+        }
+      )
+    end
+
+    # TODO: add some more rich error handling
+    class OpenaiError < StandardError; end
+  end
+end

data/lib/active_genie/clients/unified_client.rb

@@ -0,0 +1,19 @@
+module ActiveGenie::Clients
+  class UnifiedClient
+    class << self
+      def function_calling(messages, function, config: {})
+        provider_name = config[:provider]&.downcase&.strip&.to_sym
+        provider = ActiveGenie.configuration.providers.all[provider_name] || ActiveGenie.configuration.providers.default
+
+        raise InvalidProviderError if provider.nil? || provider.client.nil?
+
+        provider.client.function_calling(messages, function, config:)
+      end
+
+      private
+
+      # TODO: improve error message
+      class InvalidProviderError < StandardError; end
+    end
+  end
+end

data/lib/active_genie/configuration/log_config.rb

@@ -0,0 +1,14 @@
+
+module ActiveGenie::Configuration
+  class LogConfig
+    attr_writer :log_level
+
+    def log_level
+      @log_level ||= :info
+    end
+
+    def to_h(config = {})
+      { log_level:, **config }
+    end
+  end
+end

data/lib/active_genie/configuration/openai_config.rb

@@ -0,0 +1,56 @@
+require_relative '../clients/openai_client'
+
+module ActiveGenie::Configuration
+  class OpenaiConfig
+    attr_writer :api_key, :organization, :api_url, :client,
+      :lower_tier_model, :middle_tier_model, :upper_tier_model
+
+    def api_key
+      @api_key || ENV['OPENAI_API_KEY']
+    end
+
+    def organization
+      @organization || ENV['OPENAI_ORGANIZATION']
+    end
+
+    def lower_tier_model
+      @lower_tier_model || 'gpt-4o-mini'
+    end
+
+    def middle_tier_model
+      @middle_tier_model || 'gpt-4o'
+    end
+
+    def upper_tier_model
+      @upper_tier_model || 'o1-preview'
+    end
+
+    def tier_to_model(tier)
+      {
+        lower_tier: lower_tier_model,
+        middle_tier: middle_tier_model,
+        upper_tier: upper_tier_model
+      }[tier&.to_sym]
+    end
+
+    def api_url
+      @api_url || 'https://api.openai.com/v1'
+    end
+
+    def client
+      @client ||= ::ActiveGenie::Clients::OpenaiClient.new(self)
+    end
+
+    def to_h(config = {})
+      {
+        api_key:,
+        organization:,
+        api_url:,
+        lower_tier_model:,
+        middle_tier_model:,
+        upper_tier_model:,
+        **config
+      }
+    end
+  end
+end

data/lib/active_genie/configuration/providers_config.rb

@@ -0,0 +1,37 @@
+module ActiveGenie::Configuration
+  class ProvidersConfig
+    def initialize
+      @all = {}
+      @default = nil
+    end
+
+    def register(name, provider_class)
+      @all ||= {}
+      @all[name] = provider_class.new
+      define_singleton_method(name) do
+        instance_variable_get("@#{name}") || instance_variable_set("@#{name}", @all[name])
+      end
+
+      self
+    end
+
+    def default
+      @default || @all.values.first
+    end
+
+    def all
+      @all
+    end
+
+    def to_h(config = {})
+      hash_all = {}
+      @all.each do |name, provider|
+        hash_all[name] = provider.to_h(config.dig(name) || {})
+      end
+      hash_all
+    end
+
+    private
+    attr_writer :default
+  end
+end

data/lib/active_genie/configuration.rb

@@ -1,33 +1,28 @@
-require 'yaml'
+require_relative 'configuration/providers_config'
+require_relative 'configuration/openai_config'
+require_relative 'configuration/log_config'
 
 module ActiveGenie
-  class Configuration
-    attr_accessor :path_to_config
+  module Configuration
+    module_function
 
-    def initialize
-      @path_to_config = File.join('config', 'active_genie.yml')
-    end
-
-    def values
-      return @values if @values
-
-      @values = load_values.transform_keys(&:to_sym)
-      @values.each do |key, _value|
-        @values[key][:model] = key
-        @values[key] = @values[key].transform_keys(&:to_sym)
+    def providers
+      @providers ||= begin 
+        p = ProvidersConfig.new
+        p.register(:openai, ActiveGenie::Configuration::OpenaiConfig)
+        p
       end
     end
 
-    private
-
-    def load_values
-      return {} unless File.exist?(@path_to_config)
+    def log
+      @log ||= LogConfig.new
+    end
 
-      yaml_content = ERB.new(File.read(@path_to_config)).result
-      YAML.safe_load(yaml_content, aliases: true) || {}
-    rescue Psych::SyntaxError => e
-      warn "ActiveGenie.warning: Config file '#{@path_to_config}' is not a valid YAML file (#{e.message}), using default configuration"
-      {}
+    def to_h(configs = {})
+      {
+        providers: providers.to_h(configs.dig(:providers) || {}),
+        log: log.to_h(configs.dig(:log) || {})
+      }
     end
   end
 end

data/lib/active_genie/data_extractor/README.md

@@ -95,7 +95,7 @@ result = ActiveGenie::DataExtractor.from_informal(text, schema)
 
 ## Interface
 
-### `.call(text, data_to_extract, options = {})`
+### `.call(text, data_to_extract, config = {})`
 Extracts structured data from text based on a predefined schema.
 
 #### Parameters
@@ -103,9 +103,9 @@ Extracts structured data from text based on a predefined schema.
 | --- | --- | --- | --- | --- |
 | `text` | `String` | The text to analyze and extract data from | Yes | "John Doe is 25 years old" |
 | `data_to_extract` | `Hash` | Schema defining the data structure to extract | Yes | `{ name: { type: 'string' } }` |
-| `options` | `Hash` | Additional extraction configuration | No | `{ model: "gpt-4" }` |
+| `config` | `Hash` | Additional extraction configuration | No | `{ model: "gpt-4" }` |
 
-#### Options
+#### config
 | Name | Type | Description |
 | --- | --- | --- |
 | `model` | `String` | The model to use for the extraction |
@@ -117,7 +117,7 @@ Extracts structured data from text based on a predefined schema.
 - Explanation field for each extracted value
 - Additional analysis fields when using `from_informal`
 
-### `.from_informal(text, data_to_extract, options = {})`
+### `.from_informal(text, data_to_extract, config = {})`
 Extends basic extraction with rhetorical analysis, particularly for litotes.
 
 #### Additional Return Fields

data/lib/active_genie/data_extractor/basic.rb

@@ -1,9 +1,10 @@
-require_relative '../clients/router.rb'
+
+require_relative '../clients/unified_client'
 
 module ActiveGenie::DataExtractor
   class Basic
-    def self.call(text, data_to_extract, options: {})
-      new(text, data_to_extract, options:).call
+    def self.call(text, data_to_extract, config: {})
+      new(text, data_to_extract, config:).call
     end
 
     # Extracts structured data from text based on a predefined schema.
@@ -11,9 +12,7 @@ module ActiveGenie::DataExtractor
     # @param text [String] The input text to analyze and extract data from
     # @param data_to_extract [Hash] Schema defining the data structure to extract.
     #   Each key in the hash represents a field to extract, and its value defines the expected type and constraints.
-    # @param options [Hash] Additional options for the extraction process
-    #   @option options [String] :model The model to use for the extraction
-    #   @option options [String] :api_key The API key to use for the extraction
+    # @param config [Hash] Additional config for the extraction process
     #
     # @return [Hash] The extracted data matching the schema structure. Each field will include
     #   both the extracted value and an explanation of how it was derived.
@@ -27,10 +26,10 @@ module ActiveGenie::DataExtractor
     #   DataExtractor.call(text, schema)
     #   # => { name: "John Doe", name_explanation: "Found directly in text",
     #   #      age: 25, age_explanation: "Explicitly stated as 25 years old" }
-    def initialize(text, data_to_extract, options: {})
+    def initialize(text, data_to_extract, config: {})
       @text = text
       @data_to_extract = data_to_extract
-      @options = options
+      @config = ActiveGenie::Configuration.to_h(config)
     end
 
     def call
@@ -47,7 +46,7 @@ module ActiveGenie::DataExtractor
         }
       }
 
-      ::ActiveGenie::Clients::Router.function_calling(messages, function, options: @options)
+      ::ActiveGenie::Clients::UnifiedClient.function_calling(messages, function, config:)
     end
 
     private
@@ -84,5 +83,16 @@ module ActiveGenie::DataExtractor
       
       with_explaination
     end
+
+    def config
+      {
+        all_providers: { model_tier: 'lower_tier' },
+        log: {
+          **(@config.dig(:log) || {}),
+          trace: self.class.name,
+        },
+        **@config
+      }
+    end
   end
 end

data/lib/active_genie/data_extractor/from_informal.rb

@@ -1,7 +1,7 @@
 module ActiveGenie::DataExtractor
   class FromInformal
-    def self.call(text, data_to_extract, options: {})
-      new(text, data_to_extract, options:).call()
+    def self.call(text, data_to_extract, config: {})
+      new(text, data_to_extract, config:).call()
     end
 
     # Extracts data from informal text while also detecting litotes and their meanings.
@@ -9,7 +9,7 @@ module ActiveGenie::DataExtractor
     #
     # @param text [String] The informal text to analyze
     # @param data_to_extract [Hash] Schema defining the data structure to extract
-    # @param options [Hash] Additional options for the extraction process
+    # @param config [Hash] Additional config for the extraction process
     #
     # @return [Hash] The extracted data including litote analysis. In addition to the
     #   schema-defined fields, includes:
@@ -23,17 +23,17 @@ module ActiveGenie::DataExtractor
     #   # => { mood: "positive", mood_explanation: "Speaker views weather favorably",
     #   #      message_litote: true,
     #   #      litote_rephrased: "The weather is good today" }
-    def initialize(text, data_to_extract, options: {})
+    def initialize(text, data_to_extract, config: {})
       @text = text
       @data_to_extract = data_to_extract
-      @options = options
+      @config = ActiveGenie::Configuration.to_h(config)
     end
 
     def call
-      response = Basic.call(@text, data_to_extract_with_litote, options: @options)
+      response = Basic.call(@text, data_to_extract_with_litote, config:)
 
       if response['message_litote']
-        response = Basic.call(response['litote_rephrased'], @data_to_extract, options: @options)
+        response = Basic.call(response['litote_rephrased'], @data_to_extract, config:)
       end
 
       response
@@ -54,5 +54,16 @@ module ActiveGenie::DataExtractor
         }
       }
     end
+
+    def config
+      {
+        all_providers: { model_tier: 'lower_tier' },
+        **@config,
+        log: {
+          **@config.dig(:log),
+          trace: self.class.name,
+        },
+      }
+    end
   end
 end

data/lib/active_genie/data_extractor.rb

@@ -2,7 +2,7 @@ require_relative 'data_extractor/basic'
 require_relative 'data_extractor/from_informal'
 
 module ActiveGenie
-  # Extract structured data from text using AI-powered analysis, handling informal language and complex expressions.
+  # See the [DataExtractor README](lib/active_genie/data_extractor/README.md) for more information.
   module DataExtractor
     module_function