active_genie 0.0.2 → 0.0.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,88 @@
1
+ require_relative '../clients/router.rb'
2
+
3
+ module ActiveGenie::DataExtractor
4
+ class Basic
5
+ def self.call(text, data_to_extract, options: {})
6
+ new(text, data_to_extract, options:).call
7
+ end
8
+
9
+ # Extracts structured data from text based on a predefined schema.
10
+ #
11
+ # @param text [String] The input text to analyze and extract data from
12
+ # @param data_to_extract [Hash] Schema defining the data structure to extract.
13
+ # Each key in the hash represents a field to extract, and its value defines the expected type and constraints.
14
+ # @param options [Hash] Additional options for the extraction process
15
+ # @option options [String] :model The model to use for the extraction
16
+ # @option options [String] :api_key The API key to use for the extraction
17
+ #
18
+ # @return [Hash] The extracted data matching the schema structure. Each field will include
19
+ # both the extracted value and an explanation of how it was derived.
20
+ #
21
+ # @example Extract a person's details
22
+ # schema = {
23
+ # name: { type: 'string', description: 'Full name of the person' },
24
+ # age: { type: 'integer', description: 'Age in years' }
25
+ # }
26
+ # text = "John Doe is 25 years old"
27
+ # DataExtractor.call(text, schema)
28
+ # # => { name: "John Doe", name_explanation: "Found directly in text",
29
+ # # age: 25, age_explanation: "Explicitly stated as 25 years old" }
30
+ def initialize(text, data_to_extract, options: {})
31
+ @text = text
32
+ @data_to_extract = data_to_extract
33
+ @options = options
34
+ end
35
+
36
+ def call
37
+ messages = [
38
+ { role: 'system', content: PROMPT },
39
+ { role: 'user', content: @text }
40
+ ]
41
+ function = {
42
+ name: 'data_extractor',
43
+ description: 'Extract structured and typed data from user messages.',
44
+ schema: {
45
+ type: "object",
46
+ properties: data_to_extract_with_explaination
47
+ }
48
+ }
49
+
50
+ ::ActiveGenie::Clients::Router.function_calling(messages, function, options: @options)
51
+ end
52
+
53
+ private
54
+
55
+ PROMPT = <<~PROMPT
56
+ Extract structured and typed data from user messages.
57
+ Identify relevant information within user messages and categorize it into predefined data fields with specific data types.
58
+
59
+ # Steps
60
+ 1. **Identify Data Types**: Determine the types of data to collect, such as names, dates, email addresses, phone numbers, etc.
61
+ 2. **Extract Information**: Use pattern recognition and language understanding to identify and extract the relevant pieces of data from the user message.
62
+ 3. **Categorize Data**: Assign the extracted data to the appropriate predefined fields.
63
+ 4. **Structure Data**: Format the extracted and categorized data in a structured format, such as JSON.
64
+
65
+ # Output Format
66
+ The output should be a JSON object containing fields with their corresponding extracted values. If a value is not found, the field should still be included with a null value.
67
+
68
+ # Notes
69
+ - Handle missing or partial information gracefully.
70
+ - Manage multiple occurrences of similar data points by prioritizing the first one unless specified otherwise.
71
+ - Be flexible to handle variations in data format and language clues.
72
+ PROMPT
73
+
74
+ def data_to_extract_with_explaination
75
+ with_explaination = {}
76
+
77
+ @data_to_extract.each do |key, value|
78
+ with_explaination[key] = value
79
+ with_explaination["#{key}_explanation"] = {
80
+ type: 'string',
81
+ description: "The chain of thought that led to the conclusion about: #{key}. Can be blank if the user didn't provide any context",
82
+ }
83
+ end
84
+
85
+ with_explaination
86
+ end
87
+ end
88
+ end
@@ -0,0 +1,58 @@
1
+ module ActiveGenie::DataExtractor
2
+ class FromInformal
3
+ def self.call(text, data_to_extract, options: {})
4
+ new(text, data_to_extract, options:).call()
5
+ end
6
+
7
+ # Extracts data from informal text while also detecting litotes and their meanings.
8
+ # This method extends the basic extraction by analyzing rhetorical devices.
9
+ #
10
+ # @param text [String] The informal text to analyze
11
+ # @param data_to_extract [Hash] Schema defining the data structure to extract
12
+ # @param options [Hash] Additional options for the extraction process
13
+ #
14
+ # @return [Hash] The extracted data including litote analysis. In addition to the
15
+ # schema-defined fields, includes:
16
+ # - message_litote: Whether the text contains a litote
17
+ # - litote_rephrased: The positive rephrasing of any detected litote
18
+ #
19
+ # @example Analyze text with litote
20
+ # text = "The weather isn't bad today"
21
+ # schema = { mood: { type: 'string', description: 'The mood of the message' } }
22
+ # DataExtractor.from_informal(text, schema)
23
+ # # => { mood: "positive", mood_explanation: "Speaker views weather favorably",
24
+ # # message_litote: true,
25
+ # # litote_rephrased: "The weather is good today" }
26
+ def initialize(text, data_to_extract, options: {})
27
+ @text = text
28
+ @data_to_extract = data_to_extract
29
+ @options = options
30
+ end
31
+
32
+ def call
33
+ response = Basic.call(@text, data_to_extract_with_litote, options: @options)
34
+
35
+ if response['message_litote']
36
+ response = Basic.call(response['litote_rephrased'], @data_to_extract, options: @options)
37
+ end
38
+
39
+ response
40
+ end
41
+
42
+ private
43
+
44
+ def data_to_extract_with_litote
45
+ {
46
+ **@data_to_extract,
47
+ message_litote: {
48
+ type: 'boolean',
49
+ description: 'Return true if the message is a litote. A litote is a figure of speech that uses understatement to emphasize a point by stating a negative to further affirm a positive, often incorporating double negatives for effect.'
50
+ },
51
+ litote_rephrased: {
52
+ type: 'string',
53
+ description: 'The true meaning of the litote. Rephrase the message to a positive and active statement.'
54
+ }
55
+ }
56
+ end
57
+ end
58
+ end
@@ -0,0 +1,17 @@
1
+ require_relative 'data_extractor/basic'
2
+ require_relative 'data_extractor/from_informal'
3
+
4
+ module ActiveGenie
5
+ # Extract structured data from text using AI-powered analysis, handling informal language and complex expressions.
6
+ module DataExtractor
7
+ module_function
8
+
9
+ def basic(...)
10
+ Basic.call(...)
11
+ end
12
+
13
+ def from_informal(...)
14
+ FromInformal.call(...)
15
+ end
16
+ end
17
+ end
@@ -0,0 +1,88 @@
1
+ require_relative '../battle/basic'
2
+ require_relative '../utils/math'
3
+
4
+ module ActiveGenie::Leaderboard
5
+ class EloRanking
6
+ def self.call(players, criteria, options: {})
7
+ new(players, criteria, options:).call
8
+ end
9
+
10
+ def initialize(players, criteria, options: {})
11
+ @players = players
12
+ @criteria = criteria
13
+ @options = options
14
+ end
15
+
16
+ def call
17
+ @players.each(&:generate_elo_by_score)
18
+
19
+ while @players.eligible_size > MINIMAL_PLAYERS_TO_BATTLE
20
+ round = create_round(@players.tier_relegation, @players.tier_defense)
21
+
22
+ round.each do |player_a, player_b|
23
+ winner, loser = battle(player_a, player_b) # This can take a while, can be parallelized
24
+ update_elo(winner, loser)
25
+ end
26
+
27
+ @players.tier_relegation.each { |player| player.eliminated = "relegation/#{@players.eligible_size}" }
28
+ end
29
+
30
+ @players
31
+ end
32
+
33
+ private
34
+
35
+ MATCHS_PER_PLAYER = 3
36
+ LOSE_PENALTY = 15
37
+ MINIMAL_PLAYERS_TO_BATTLE = 10
38
+
39
+ # Create a round of matches
40
+ # each round is exactly 1 regation player vs 3 defense players for all regation players
41
+ # each match is unique (player vs player)
42
+ # each defense player is battle exactly 3 times
43
+ def create_round(relegation_players, defense_players)
44
+ matches = []
45
+
46
+ relegation_players.each do |player_a|
47
+ player_enemies = []
48
+ MATCHS_PER_PLAYER.times do
49
+ defender = nil
50
+ while defender.nil? || player_enemies.include?(defender.id)
51
+ defender = defense_players.sample
52
+ end
53
+
54
+ matches << [player_a, defender].shuffle
55
+ player_enemies << defender.id
56
+ end
57
+ end
58
+
59
+ matches
60
+ end
61
+
62
+ def battle(player_a, player_b)
63
+ ActiveGenie::Battle.basic(
64
+ player_a,
65
+ player_b,
66
+ @criteria,
67
+ options: @options
68
+ ).values_at('winner', 'loser')
69
+ end
70
+
71
+ def update_elo(winner, loser)
72
+ return if winner.nil? || loser.nil?
73
+
74
+ new_winner_elo, new_loser_elo = ActiveGenie::Utils::Math.calculate_new_elo(winner.elo, loser.elo)
75
+
76
+ winner.elo = [new_winner_elo, max_defense_elo].min
77
+ loser.elo = [new_loser_elo - LOSE_PENALTY, min_relegation_elo].max
78
+ end
79
+
80
+ def max_defense_elo
81
+ @players.tier_defense.max_by(&:elo).elo
82
+ end
83
+
84
+ def min_relegation_elo
85
+ @players.tier_relegation.min_by(&:elo).elo
86
+ end
87
+ end
88
+ end
@@ -0,0 +1,72 @@
1
+ require_relative './players_collection'
2
+ require_relative './league'
3
+ require_relative './elo_ranking'
4
+ require_relative '../scoring/recommended_reviews'
5
+
6
+ module ActiveGenie::Leaderboard
7
+ class Leaderboard
8
+ def self.call(param_players, criteria, options: {})
9
+ new(param_players, criteria, options:).call
10
+ end
11
+
12
+ def initialize(param_players, criteria, options: {})
13
+ @param_players = param_players
14
+ @criteria = criteria
15
+ @options = options
16
+ end
17
+
18
+ def call
19
+ set_initial_score_players
20
+ eliminate_obvious_bad_players
21
+ run_elo_ranking if players.eligible_size > 10
22
+ run_league
23
+
24
+ players.to_h
25
+ end
26
+
27
+ private
28
+
29
+ SCORE_VARIATION_THRESHOLD = 10
30
+ MATCHS_PER_PLAYER = 3
31
+
32
+ def set_initial_score_players
33
+ players.each do |player|
34
+ player.score = generate_score(player.content) # This can take a while, can be parallelized
35
+ end
36
+ end
37
+
38
+ def generate_score(content)
39
+ ActiveGenie::Scoring::Basic.call(content, @criteria, reviewers, options: @options)['final_score']
40
+ end
41
+
42
+ def eliminate_obvious_bad_players
43
+ while players.coefficient_of_variation >= SCORE_VARIATION_THRESHOLD
44
+ players.eligible.last.eliminated = 'too_low_score'
45
+ end
46
+ end
47
+
48
+ def run_elo_ranking
49
+ EloRanking.call(players, @criteria, options: @options)
50
+ end
51
+
52
+ def run_league
53
+ League.call(players, @criteria, options: @options)
54
+ end
55
+
56
+ def reviewers
57
+ [recommended_reviews['reviewer1'], recommended_reviews['reviewer2'], recommended_reviews['reviewer3']]
58
+ end
59
+
60
+ def recommended_reviews
61
+ @recommended_reviews ||= ActiveGenie::Scoring::RecommendedReviews.call(
62
+ players.sample,
63
+ @criteria,
64
+ options: @options
65
+ )
66
+ end
67
+
68
+ def players
69
+ @players ||= PlayersCollection.new(@param_players)
70
+ end
71
+ end
72
+ end
@@ -0,0 +1,48 @@
1
+ require_relative '../battle/basic'
2
+
3
+ module ActiveGenie::Leaderboard
4
+ class League
5
+ def self.call(players, criteria, options: {})
6
+ new(players, criteria, options:).call
7
+ end
8
+
9
+ def initialize(players, criteria, options: {})
10
+ @players = players
11
+ @criteria = criteria
12
+ @options = options
13
+ end
14
+
15
+ def call
16
+ matches.each do |player_a, player_b|
17
+ winner, loser = battle(player_a, player_b)
18
+
19
+ if winner.nil? || loser.nil?
20
+ player_a.league[:draw] += 1
21
+ player_b.league[:draw] += 1
22
+ else
23
+ winner.league[:win] += 1
24
+ loser.league[:lose] += 1
25
+ end
26
+ end
27
+
28
+ @players
29
+ end
30
+
31
+ private
32
+
33
+ # TODO: reduce the number of matches based on transitivity.
34
+ # For example, if A is better than B, and B is better than C, then A should clearly be better than C
35
+ def matches
36
+ @players.eligible.combination(2).to_a
37
+ end
38
+
39
+ def battle(player_a, player_b)
40
+ ActiveGenie::Battle.basic(
41
+ player_a,
42
+ player_b,
43
+ @criteria,
44
+ options: @options
45
+ ).values_at('winner', 'loser')
46
+ end
47
+ end
48
+ end
@@ -0,0 +1,52 @@
1
+ require 'securerandom'
2
+
3
+ module ActiveGenie::Leaderboard
4
+ class Player
5
+ def initialize(params)
6
+ params = { content: params } if params.is_a?(String)
7
+
8
+ @id = params.dig(:id) || SecureRandom.uuid
9
+ @content = params.dig(:content) || params
10
+ @score = params.dig(:score) || nil
11
+ @elo = params.dig(:elo) || nil
12
+ @league = {
13
+ win: params.dig(:league, :win) || 0,
14
+ lose: params.dig(:league, :lose) || 0,
15
+ draw: params.dig(:league, :draw) || 0
16
+ }
17
+ @eliminated = params.dig(:eliminated) || nil
18
+ end
19
+
20
+ attr_reader :id, :content, :score, :elo, :league, :eliminated
21
+
22
+ def generate_elo_by_score
23
+ return if !@elo.nil? || @score.nil?
24
+
25
+ @elo = BASE_ELO + (@score - 50)
26
+ end
27
+
28
+ def score=(value)
29
+ @score = value
30
+ end
31
+
32
+ def elo=(value)
33
+ @elo = value
34
+ end
35
+
36
+ def eliminated=(value)
37
+ @eliminated = value
38
+ end
39
+
40
+ def league_score
41
+ @league[:win] * 3 + @league[:draw]
42
+ end
43
+
44
+ def to_h
45
+ { id:, content:, score:, elo:, eliminated:, league: }
46
+ end
47
+
48
+ private
49
+
50
+ BASE_ELO = 1000
51
+ end
52
+ end
@@ -0,0 +1,68 @@
1
+ require_relative '../utils/math'
2
+ require_relative './player'
3
+
4
+ module ActiveGenie::Leaderboard
5
+ class PlayersCollection
6
+ def initialize(param_players)
7
+ @players = build(param_players)
8
+ end
9
+ attr_reader :players
10
+
11
+ def coefficient_of_variation
12
+ score_list = eligible.map(&:score)
13
+ mean = score_list.sum.to_f / score_list.size
14
+ return nil if mean == 0 # To avoid division by zero
15
+
16
+ variance = score_list.map { |num| (num - mean) ** 2 }.sum / score_list.size
17
+ standard_deviation = Math.sqrt(variance)
18
+
19
+ (standard_deviation / mean) * 100
20
+ end
21
+
22
+ def tier_relegation
23
+ eligible[(tier_size*-1)..-1]
24
+ end
25
+
26
+ def tier_defense
27
+ eligible[(tier_size*-2)...(tier_size*-1)]
28
+ end
29
+
30
+ def eligible
31
+ sorted.reject(&:eliminated)
32
+ end
33
+
34
+ def eligible_size
35
+ @players.reject(&:eliminated).size
36
+ end
37
+
38
+ def to_h
39
+ sorted.map(&:to_h)
40
+ end
41
+
42
+ def method_missing(...)
43
+ @players.send(...)
44
+ end
45
+
46
+ def sorted
47
+ @players.sort_by { |p| [-p.league_score, -(p.elo || 0), -p.score] }
48
+ end
49
+
50
+ private
51
+
52
+ def build(param_players)
53
+ param_players.map { |player| Player.new(player) }
54
+ end
55
+
56
+ # Returns the number of players to battle in each round
57
+ # based on the eligible size, start fast and go slow until top 10
58
+ # Example:
59
+ # - 50 eligible, tier_size: 15
60
+ # - 35 eligible, tier_size: 11
61
+ # - 24 eligible, tier_size: 10
62
+ # - 14 eligible, tier_size: 4
63
+ # 4 rounds to reach top 10 with 50 players
64
+ def tier_size
65
+ [[(eligible_size / 3).ceil, 10].max, eligible_size - 10].min
66
+ end
67
+ end
68
+ end
@@ -0,0 +1,11 @@
1
+ require_relative 'leaderboard/leaderboard'
2
+
3
+ module ActiveGenie
4
+ module Leaderboard
5
+ module_function
6
+
7
+ def call(...)
8
+ Leaderboard.call(...)
9
+ end
10
+ end
11
+ end
@@ -0,0 +1,80 @@
1
+ # Scoring
2
+ Text evaluation system that provides detailed scoring and feedback using multiple expert reviewers.
3
+
4
+ ## Features
5
+ - Multi-reviewer evaluation - Get scores and feedback from multiple AI-powered expert reviewers
6
+ - Automatic reviewer selection - Smart recommendation of reviewers based on content and criteria
7
+ - Detailed feedback - Comprehensive reasoning for each reviewer's score
8
+ - Customizable weights - Adjust the importance of different reviewers' scores
9
+ - Flexible criteria - Score text against any specified evaluation criteria
10
+
11
+ ## Basic Usage
12
+
13
+ Score text using predefined reviewers:
14
+
15
+ ```ruby
16
+ text = "The code implements a binary search algorithm with O(log n) complexity"
17
+ criteria = "Evaluate technical accuracy and clarity"
18
+ reviewers = ["Algorithm Expert", "Technical Writer"]
19
+
20
+ result = ActiveGenie::Scoring::Basic.call(text, criteria, reviewers)
21
+ # => {
22
+ # algorithm_expert_score: 95,
23
+ # algorithm_expert_reasoning: "Accurately describes binary search and its complexity",
24
+ # technical_writer_score: 90,
25
+ # technical_writer_reasoning: "Clear and concise explanation of the algorithm",
26
+ # final_score: 92.5
27
+ # }
28
+ ```
29
+
30
+ ## Automatic Reviewer Selection
31
+
32
+ When no reviewers are specified, the system automatically recommends appropriate reviewers:
33
+
34
+ ```ruby
35
+ text = "The patient shows signs of improved cardiac function"
36
+ criteria = "Evaluate medical accuracy and clarity"
37
+
38
+ result = ActiveGenie::Scoring::Basic.call(text, criteria)
39
+ # => {
40
+ # cardiologist_score: 88,
41
+ # cardiologist_reasoning: "Accurate assessment of cardiac improvement",
42
+ # medical_writer_score: 85,
43
+ # medical_writer_reasoning: "Clear communication of medical findings",
44
+ # general_practitioner_score: 90,
45
+ # general_practitioner_reasoning: "Well-structured medical observation",
46
+ # final_score: 87.7
47
+ # }
48
+ ```
49
+
50
+ ## Interface
51
+
52
+ ### `Basic.call(text, criteria, reviewers = [], options: {})`
53
+ Main interface for scoring text content.
54
+
55
+ #### Parameters
56
+ - `text` [String] - The text content to be evaluated
57
+ - `criteria` [String] - The evaluation criteria or rubric to assess against
58
+ - `reviewers` [Array<String>] - Optional list of specific reviewers
59
+ - `options` [Hash] - Additional configuration options
60
+ - `:detailed_feedback` [Boolean] - Request more detailed feedback (WIP)
61
+ - `:reviewer_weights` [Hash] - Custom weights for different reviewers (WIP)
62
+
63
+ ### `RecommendedReviews.call(text, criteria, options: {})`
64
+ Recommends appropriate reviewers based on content and criteria.
65
+
66
+ #### Parameters
67
+ - `text` [String] - The text content to analyze
68
+ - `criteria` [String] - The evaluation criteria
69
+ - `options` [Hash] - Additional configuration options
70
+ - `:prefer_technical` [Boolean] - Favor technical expertise (WIP)
71
+ - `:prefer_domain` [Boolean] - Favor domain expertise (WIP)
72
+
73
+ ### Usage Notes
74
+ - Best suited for objective evaluation of text content
75
+ - Provides balanced scoring through multiple reviewers
76
+ - Automatically handles reviewer selection when needed
77
+ - Supports custom weighting of reviewer scores
78
+ - Returns detailed reasoning for each score
79
+
80
+ Performance Impact: Using multiple reviewers or requesting detailed feedback may increase processing time.