active_genie 0.26.6 → 0.28.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 47ad459a3acb6b1e37b8cc9d30e072fb1a699177a628212a2b33406c7d019ace
4
- data.tar.gz: 9ce87f41aa773aae74fa0a822e492263e95d7c4f63aac26eb77a8385351eaa6b
3
+ metadata.gz: a2cb129cd0e5a4f5f0f1daf6ed3cb26a4dab4d10cb56bbfcc419656c2797347a
4
+ data.tar.gz: df1fe343d28aeb72f8fe30dc53bdecd860e8bd44ae009a086c7ad502da77bfd8
5
5
  SHA512:
6
- metadata.gz: 3a15a6742977c8f1a38f1eab5e5d9a5ef30d79cfd9db8ba630775f7629cd25486e00b63a2122a01281a303e396a0d5faefdcb4b399a65b4eaaf9aeb50ddac116
7
- data.tar.gz: 4f24bee6532d2251e9051067582bc4bcd1e9ef685996ff34eff15416976774387731917f66482c62efc9d205c552fcd3aacb8b73f869005585fd66fd86b89d12
6
+ metadata.gz: 1449245c23bcc2c5f3c4266712c67b202e4e2c6724ee9f283bd81d76b2daedc4bc06ee524ed8d959d13f31062b393b5f7cfb7f46721888efb8d89bbd4d8a12b3
7
+ data.tar.gz: 649a20ee71c55d8486d8264b641974c3222baa477133d6924ba039f2ef24f9185d2b60b2f37f7d2ff498b87aaed09b78815feb3bbc52c92613eb810de2c59c2c
data/README.md CHANGED
@@ -1,13 +1,16 @@
1
1
  # ActiveGenie 🧞‍♂️
2
- > The lodash for GenAI, stop reinventing the wheel
2
+ > The Lodash for GenAI: Real Value + Consistent + Model-Agnostic
3
3
 
4
4
  [![Gem Version](https://badge.fury.io/rb/active_genie.svg?icon=si%3Arubygems)](https://badge.fury.io/rb/active_genie)
5
5
  [![Ruby](https://github.com/roriz/active_genie/actions/workflows/benchmark.yml/badge.svg)](https://github.com/roriz/active_genie/actions/workflows/benchmark.yml)
6
6
 
7
- ActiveGenie is a Ruby gem that provides valuable solutions powered by Generative AI (GenAI) models. Just like Lodash or ActiveStorage, ActiveGenie brings a set of Modules to reach real value fast and reliably.
8
- ActiveGenie is backed by a custom benchmarking system that ensures consistent quality and performance across different models and providers in every release.
7
+ ActiveGenie is a developer-first library for GenAI workflows, designed to help you extract, compare, score, and rank with consistency and model-agnostic. Think of it as the Lodash for GenAI: built for real value, consistent results, and freedom from vendor lock-in. It solves the biggest pain in GenAI today: getting predictable, trustworthy answers across use cases, models, and providers.
9
8
 
10
- ## Installation
9
+ Behind the scenes, a custom benchmarking system keeps everything consistent across LLM vendors and versions, release after release.
10
+
11
+ For full documentation, visit [activegenie.ai](https://activegenie.ai).
12
+
13
+ # Installation
11
14
 
12
15
  1. Add to your Gemfile:
13
16
  ```ruby
@@ -32,9 +35,7 @@ ActiveGenie.configure do |config|
32
35
  end
33
36
  ```
34
37
 
35
- ## Quick Start
36
-
37
- ### Data Extractor
38
+ ## Quick start example
38
39
 
39
40
  Extract structured data from text using AI-powered analysis, handling informal language and complex expressions.
40
41
 
@@ -71,197 +72,9 @@ result = ActiveGenie::DataExtractor.call(
71
72
  # }
72
73
  ```
73
74
 
74
- *Recommended model*: `gpt-4o-mini`
75
-
76
- Features:
77
- - Structured data extraction with type validation
78
- - Schema-based extraction with custom constraints
79
- - Informal text analysis (litotes, hedging)
80
- - Detailed explanations for extracted values
81
-
82
- See the [Data Extractor README](lib/active_genie/data_extractor/README.md) for informal text processing, advanced schemas, and detailed interface documentation.
83
-
84
- ### Scoring
85
- Text evaluation system that provides detailed scoring and feedback using multiple expert reviewers. Get balanced scoring through AI-powered expert reviewers that automatically adapt to your content.
86
-
87
- ```ruby
88
- text = "The code implements a binary search algorithm with O(log n) complexity"
89
- criteria = "Evaluate technical accuracy and clarity"
90
-
91
- result = ActiveGenie::Scoring.call(
92
- text,
93
- criteria,
94
- config: { provider: :anthropic, model: 'claude-3-5-haiku-20241022' } # optional
95
- )
96
- # => {
97
- # algorithm_expert_score: 95,
98
- # algorithm_expert_reasoning: "Accurately describes binary search and its complexity",
99
- # technical_writer_score: 90,
100
- # technical_writer_reasoning: "Clear and concise explanation of the algorithm",
101
- # final_score: 92.5
102
- # }
103
- ```
104
-
105
- *Recommended model*: `claude-3-5-haiku-20241022`
106
-
107
- Features:
108
- - Multi-reviewer evaluation with automatic expert selection
109
- - Detailed feedback with scoring reasoning
110
- - Customizable reviewer weights
111
- - Flexible evaluation criteria
112
-
113
- See the [Scoring README](lib/active_genie/scoring/README.md) for advanced usage, custom reviewers, and detailed interface documentation.
114
-
115
- ### Battle
116
- AI-powered battle evaluation system that determines winners between two players based on specified criteria.
117
-
118
- ```ruby
119
- require 'active_genie'
120
-
121
- player_a = "Implementation uses dependency injection for better testability"
122
- player_b = "Code has high test coverage but tightly coupled components"
123
- criteria = "Evaluate code quality and maintainability"
124
-
125
- result = ActiveGenie::Battle.call(
126
- player_a,
127
- player_b,
128
- criteria,
129
- config: { provider: :google, model: 'gemini-2.0-flash-lite' } # optional
130
- )
131
- # => {
132
- # winner_player: "Implementation uses dependency injection for better testability",
133
- # reasoning: "Player 1 implementation demonstrates better maintainability through dependency injection,
134
- # which allows for easier testing and component replacement. While Player 2 has good test coverage,
135
- # the tight coupling makes the code harder to maintain and modify.",
136
- # what_could_be_changed_to_avoid_draw: "Focus on specific architectural patterns and design principles"
137
- # }
138
- ```
139
-
140
- *Recommended model*: `claude-3-5-haiku`
75
+ ## Documentation
141
76
 
142
- Features:
143
- - Multi-reviewer evaluation with automatic expert selection
144
- - Detailed feedback with scoring reasoning
145
- - Customizable reviewer weights
146
- - Flexible evaluation criteria
147
-
148
- See the [Battle README](lib/active_genie/battle/README.md) for advanced usage, custom reviewers, and detailed interface documentation.
149
-
150
- ### Ranking
151
- The Ranking module provides competitive ranking through multi-stage evaluation:
152
-
153
- ```ruby
154
- require 'active_genie'
155
-
156
- players = ['REST API', 'GraphQL API', 'SOAP API', 'gRPC API', 'Websocket API']
157
- criteria = "Best one to be used into a high changing environment"
158
-
159
- result = ActiveGenie::Ranking.call(
160
- players,
161
- criteria,
162
- config: { provider: :google, model: 'gemini-2.0-flash-lite' } # optional
163
- )
164
- # => {
165
- # winner_player: "gRPC API",
166
- # reasoning: "gRPC API is the best one to be used into a high changing environment",
167
- # }
168
- ```
169
-
170
- *Recommended model*: `gemini-2.0-flash-lite`
171
-
172
- - **Multi-phase ranking system** combining expert scoring and ELO algorithms
173
- - **Automatic elimination** of inconsistent performers using statistical analysis
174
- - **Dynamic ranking adjustments** based on simulated pairwise battles, from bottom to top
175
-
176
- See the [Ranking README](lib/active_genie/ranking/README.md) for implementation details, configuration, and advanced ranking strategies.
177
-
178
- ### Text Summarizer (Future)
179
- ### Categorizer (Future)
180
- ### Language detector (Future)
181
- ### Translator (Future)
182
- ### Sentiment analyzer (Future)
183
-
184
- ## Benchmarking 🧪
185
-
186
- ActiveGenie includes a comprehensive benchmarking system to ensure consistent, high-quality outputs across different LLM models and providers.
187
-
188
- ```ruby
189
- # Run all benchmarks
190
- bundle exec rake active_genie:benchmark
191
-
192
- # Run benchmarks for a specific module
193
- bundle exec rake active_genie:benchmark[data_extractor]
194
- ```
195
-
196
- ### Latest Results
197
-
198
- | Model | Overall Precision |
199
- |-------|-------------------|
200
- | claude-3-5-haiku-20241022 | 92.25% |
201
- | gemini-2.0-flash-lite | 84.25% |
202
- | gpt-4o-mini | 62.75% |
203
- | deepseek-chat | 57.25% |
204
-
205
- See the [Benchmark README](benchmark/README.md) for detailed results, methodology, and how to contribute to our test suite.
206
-
207
- ## Basic Configuration
208
-
209
- | Config | Type | Description | Default |
210
- |--------|------|-------------|---------|
211
- | `llm.provider` | Symbol | LLM provider (openai, anthropic, etc) | `nil` |
212
- | `llm.model` | String | Model to use | `nil` |
213
- | `llm.temperature` | Float | Temperature to use | `0` |
214
- | `llm.max_tokens` | Integer | Maximum tokens to use | `4096` |
215
- | `llm.max_retries` | Integer | Maximum retry attempts | `3` |
216
- | `log.output` | Proc | Log output | `->(log) { $stdout.puts log }` |
217
- | `ranking.score_variation_threshold` | Integer | Score variation threshold | `30` |
218
-
219
- > **Note:** Each module can append its own set of configuration, see the individual module documentation for details.
220
-
221
- Read all [configuration](lib/active_genie/config/README.md) for all available options.
222
-
223
- ## How to create a new provider
224
-
225
- ActiveGenie supports adding custom providers to integrate with different LLM services. To create a new provider:
226
-
227
- 1. Create a configuration class for your provider in `lib/active_genie/configuration/providers/`:
228
- 2. Register your client
229
-
230
- ```ruby
231
- class InternalCompanyApi
232
- # @param messages [Array<Hash>] A list of messages representing the conversation history.
233
- # Each hash should have :role ('user', 'assistant', or 'system') and :content (String).
234
- # @param function [Hash] A JSON schema definition describing the desired output format.
235
- # @return [Hash, nil] The parsed JSON object matching the schema, or nil if parsing fails or content is empty.
236
- def function_calling(messages, function)
237
- # ...
238
- end
239
- end
240
-
241
- ActiveGenie.configure do |config|
242
- config.llm.client = InternalCompanyApi
243
- end
244
- # or
245
- ActiveGenie::Battle.call('player_a', 'player_b', 'criteria', { client: InternalCompanyApi })
246
- ```
247
-
248
- ## Observability
249
- Fundamental to managing any production system, observability is crucial for GenAI features. At a minimum, track these key metrics:
250
-
251
- - Usage Rate (e.g., uses_per_minute): Detect anomalies like sudden traffic spikes (potential DDoS) or drops (feature outage or declining usage).
252
- - Failure/Retry Rate (e.g., retry_count, fail_count): Monitor the frequency of errors. Exceeding a defined threshold should trigger downtime or degradation alerts.
253
- - Token Consumption (e.g., tokens_used): Track usage to monitor costs. Set alerts if tokens_used * price_per_token exceeds budget thresholds.
254
-
255
- ```ruby
256
- ActiveGenie.configure do |config|
257
- config.log.add_observer(scope: { code: :llm_usage }) do |log|
258
- puts "LLM Usage: #{log[:model]} - #{log[:total_tokens]} tokens"
259
- end
260
- config.log.add_observer(scope: { code: :retry_attempt }) do |log|
261
- puts "Retry Attempt: #{log[:attempt]} of #{log[:max_retries]}"
262
- end
263
- end
264
- ```
77
+ For full documentation, visit [activegenie.ai](https://activegenie.ai).
265
78
 
266
79
  ## Contributing
267
80
 
data/VERSION CHANGED
@@ -1 +1 @@
1
- 0.26.6
1
+ 0.28.2
@@ -0,0 +1,61 @@
1
+ {
2
+ "name": "fight_evaluation",
3
+ "description": "Evaluate a fight between player_a and player_b using predefined criteria and identify the winner.",
4
+ "parameters": {
5
+ "type": "object",
6
+ "properties": {
7
+ "player_a_introduction": {
8
+ "type": "string",
9
+ "description": "player_a introduces their name and fighting style. Max of 100 words."
10
+ },
11
+ "player_b_introduction": {
12
+ "type": "string",
13
+ "description": "player_b introduces their name and fighting style. Max of 100 words."
14
+ },
15
+ "player_a_turn_1": {
16
+ "type": "string",
17
+ "description": "player_a makes their first turn. Max of 100 words."
18
+ },
19
+ "player_b_turn_1": {
20
+ "type": "string",
21
+ "description": "player_b makes their first turn. Max of 100 words."
22
+ },
23
+ "player_a_turn_2": {
24
+ "type": "string",
25
+ "description": "player_a makes their second turn. Max of 100 words."
26
+ },
27
+ "player_b_turn_2": {
28
+ "type": "string",
29
+ "description": "player_b makes their second turn. Max of 100 words."
30
+ },
31
+ "player_a_turn_3": {
32
+ "type": "string",
33
+ "description": "player_a makes their third turn. Max of 100 words."
34
+ },
35
+ "player_b_turn_3": {
36
+ "type": "string",
37
+ "description": "player_b makes their third turn. Max of 100 words."
38
+ },
39
+ "player_a_turn_4": {
40
+ "type": "string",
41
+ "description": "player_a makes their fourth turn. Max of 100 words."
42
+ },
43
+ "player_b_turn_4": {
44
+ "type": "string",
45
+ "description": "player_b makes their fourth turn. Max of 100 words."
46
+ },
47
+ "impartial_judge_winner_reasoning": {
48
+ "type": "string",
49
+ "description": "The detailed reasoning about why the impartial judge chose the winner. Max of 100 words."
50
+ },
51
+ "impartial_judge_winner": {
52
+ "type": "string",
53
+ "description": "Who is the winner based on the impartial judge reasoning?",
54
+ "enum": ["player_a", "player_b"]
55
+ }
56
+ },
57
+ "required": ["player_a_introduction", "player_b_introduction", "player_a_turn_1", "player_b_turn_1",
58
+ "player_a_turn_2", "player_b_turn_2", "player_a_turn_3", "player_b_turn_3", "player_a_turn_4",
59
+ "player_b_turn_4", "impartial_judge_winner_reasoning", "impartial_judge_winner"]
60
+ }
61
+ }
@@ -0,0 +1,41 @@
1
+ Create a dynamic, back-and-forth verbal match between two fighters, given their names and preferred fighting styles. In this match, fighters do not physically battle, instead, they engage in a spoken confrontation, each describing and boasting about their fighting style, how their techniques would outmaneuver, counter, or overwhelm the other's approach. Ensure that each verbal exchange details:
2
+ - The technique or strategy the speaker would use.
3
+ - How that move is superior to the opponent’s last stated tactic.
4
+ - How the opponent might respond, and what reply the speaker would have.
5
+
6
+ Continue this turn-based verbal sparring until each fighter has made at least two exchanges, and escalate the creativity and competitiveness of their boasts with each round.
7
+
8
+ Explicit steps:
9
+ - Begin by introducing both fighters and naming their styles.
10
+ - Alternate dialogue, with each fighter explaining their tactical move, then the intended counter or rebuttal, going at least two cycles each.
11
+ - Ensure reasoning (the logic or explanation behind each technique and counter) always precedes the conclusion or actual "move" or taunt.
12
+ - Close with a final statement or taunt from each fighter, maintaining the competitive tone.
13
+ - Make the dialogue lively and imaginative, using language consistent with fighters boasting or challenging each other.
14
+
15
+ Output Format:
16
+ - Structure as a script, with each turn labeled by the fighter’s name.
17
+ - Each line should start with the reasoning (the advantage or thinking), then the boast or attack, in the order: reasoning first, conclusion second.
18
+ - At least two cycles per fighter, alternating turns; roughly 6-8 lines, but can be longer for complexity.
19
+ - No code blocks, use markdown for formatting.
20
+
21
+ Example:
22
+
23
+ Example Input:
24
+ Fighter One: Master Crane, Style: Crane Kung Fu
25
+ Fighter Two: Iron Ox, Style: Ox Bull Charge
26
+
27
+ Example Output:
28
+ Master Crane: My Crane Kung Fu relies on lightness and precision, striking where your heavy blows simply can't reach. Your Ox Bull Charge is powerful, but too direct, I'd dance aside and tap your pressure points before you even turn.
29
+ Iron Ox: While speed is impressive, strength dominates in a real fight. My Ox Bull Charge would absorb your nimble attacks, and my sheer mass pins you before you could even flutter away!
30
+ Master Crane: Adaptability ensures survival. As you lumber forward, I use your momentum to redirect you, your own strength becomes your undoing as you stumble into my calculated traps.
31
+ Iron Ox: Anticipating trickery, I’d anchor myself and force your traps to fail. Your frailty means that, once caught, no escape, one bear hug from me, and there’s no dancing away.
32
+ Master Crane: Patience and timing can unravel even the strongest grip. Just as you squeeze, a swift strike to your nerve centers will loosen your hold before you know it.
33
+ Iron Ox: Endurance outlasts finesse; those quick hits may sting, but in the end, the last one standing is the one who never falls, and I never fall!
34
+
35
+ (Real examples can be longer, more diverse in style banter, and use specific technique names or metaphors!)
36
+
37
+ Important Considerations:
38
+ - Always have the reasoning/strategy before the actual attack or boast in each utterance.
39
+ - Maintain the turn order and competitive dialogue style.
40
+
41
+ *Reminder: The main objective is to script an imaginative, turn-based verbal contest between two stylized fighters, with detailed reasoning and taunts about their style advantages, as instructed above.*
@@ -0,0 +1,37 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative '../clients/unified_client'
4
+ require_relative 'generalist'
5
+
6
+ module ActiveGenie
7
+ module Battle
8
+ # The Fight class are battle specialized in a fight between two fighters, like martial arts, heroes, characters.
9
+ # The fight evaluation process simulate a fight using words, techniques, strategies, and reasoning.
10
+ #
11
+ # @example Fight usage with two fighters and criteria
12
+ # Fight.call("Naruto", "Sasuke", "How can win without using jutsu?")
13
+ #
14
+ class Fight < Generalist
15
+ # @return [BattleResponse] The evaluation result containing the winner and reasoning
16
+ def call
17
+ messages = [
18
+ { role: 'system', content: PROMPT },
19
+ { role: 'user', content: "criteria: #{@criteria}" },
20
+ { role: 'user', content: "player_a: #{@player_a}" },
21
+ { role: 'user', content: "player_b: #{@player_b}" }
22
+ ]
23
+
24
+ response = ::ActiveGenie::Clients::UnifiedClient.function_calling(
25
+ messages,
26
+ FUNCTION,
27
+ config: @config
28
+ )
29
+
30
+ response_formatted(response)
31
+ end
32
+
33
+ PROMPT = File.read(File.join(__dir__, 'fight.prompt.md'))
34
+ FUNCTION = JSON.parse(File.read(File.join(__dir__, 'fight.json')), symbolize_names: true)
35
+ end
36
+ end
37
+ end
@@ -15,6 +15,8 @@ module ActiveGenie
15
15
  # Generalist.call("Player A content", "Player B content", "Evaluate keyword usage and pattern matching")
16
16
  #
17
17
  class Generalist
18
+ BattleResponse = Struct.new(:winner, :loser, :reasoning, :raw, keyword_init: true)
19
+
18
20
  def self.call(...)
19
21
  new(...).call
20
22
  end
@@ -23,7 +25,7 @@ module ActiveGenie
23
25
  # @param player_b [String] The content or submission from the second player
24
26
  # @param criteria [String] The evaluation criteria or rules to assess against
25
27
  # @param config [Hash] Additional configuration options that modify the battle evaluation behavior
26
- # @return [Hash] The evaluation result containing the winner and reasoning
28
+ # @return [BattleResponse] The evaluation result containing the winner and reasoning
27
29
  # @return [String] :winner The winner, either player_a or player_b
28
30
  # @return [String] :reasoning Detailed explanation of why the winner was chosen
29
31
  # @return [String] :what_could_be_changed_to_avoid_draw A suggestion on how to avoid a draw
@@ -34,6 +36,7 @@ module ActiveGenie
34
36
  @config = ActiveGenie.configuration.merge(config)
35
37
  end
36
38
 
39
+ # @return [BattleResponse] The evaluation result containing the winner and reasoning
37
40
  def call
38
41
  messages = [
39
42
  { role: 'system', content: PROMPT },
@@ -48,31 +51,35 @@ module ActiveGenie
48
51
  config: @config
49
52
  )
50
53
 
51
- @config.logger.call({
52
- code: :battle,
53
- player_a: @player_a[0..30],
54
- player_b: @player_b[0..30],
55
- criteria: @criteria[0..30],
56
- winner: response['impartial_judge_winner'],
57
- reasoning: response['impartial_judge_winner_reasoning']
58
- })
59
-
60
54
  response_formatted(response)
61
55
  end
62
56
 
63
- PROMPT = File.read(File.join(__dir__, 'generalist.md'))
57
+ PROMPT = File.read(File.join(__dir__, 'generalist.prompt.md'))
64
58
  FUNCTION = JSON.parse(File.read(File.join(__dir__, 'generalist.json')), symbolize_names: true)
65
59
 
66
60
  private
67
61
 
68
62
  def response_formatted(response)
69
- winner = response['impartial_judge_winner']
70
- loser = case winner
71
- when 'player_a' then 'player_b'
72
- when 'player_b' then 'player_a'
73
- end
63
+ winner, loser = case response['impartial_judge_winner']
64
+ when 'player_a' then [@player_a, @player_b]
65
+ when 'player_b' then [@player_b, @player_a]
66
+ end
67
+ reasoning = response['impartial_judge_winner_reasoning']
68
+
69
+ battle_response = BattleResponse.new(winner:, loser:, reasoning:, raw: response)
70
+ log_battle(battle_response)
74
71
 
75
- { 'winner' => winner, 'loser' => loser, 'reasoning' => response['impartial_judge_winner_reasoning'] }
72
+ battle_response
73
+ end
74
+
75
+ def log_battle(battle_response)
76
+ @config.logger.call(
77
+ code: :battle,
78
+ player_a: @player_a[0..30],
79
+ player_b: @player_b[0..30],
80
+ criteria: @criteria[0..30],
81
+ **battle_response.to_h
82
+ )
76
83
  end
77
84
  end
78
85
  end
@@ -1,18 +1,27 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require_relative 'battle/generalist'
4
+ require_relative 'battle/fight'
4
5
 
5
6
  module ActiveGenie
6
7
  # See the [Battle README](lib/active_genie/battle/README.md) for more information.
7
8
  module Battle
8
9
  module_function
9
10
 
11
+ def generalist(...)
12
+ Generalist.call(...)
13
+ end
14
+
10
15
  def call(...)
11
16
  Generalist.call(...)
12
17
  end
13
18
 
14
- def generalist(...)
19
+ def compare(...)
15
20
  Generalist.call(...)
16
21
  end
22
+
23
+ def fight(...)
24
+ Fight.call(...)
25
+ end
17
26
  end
18
27
  end
@@ -119,7 +119,7 @@ module ActiveGenie
119
119
  end
120
120
 
121
121
  def prompt
122
- File.read(File.join(__dir__, 'generalist.md'))
122
+ File.read(File.join(__dir__, 'generalist.prompt.md'))
123
123
  end
124
124
  end
125
125
  end
@@ -64,7 +64,7 @@ module ActiveGenie
64
64
  result
65
65
  end
66
66
 
67
- PROMPT = File.read(File.join(__dir__, 'generalist.md'))
67
+ PROMPT = File.read(File.join(__dir__, 'generalist.prompt.md'))
68
68
 
69
69
  private
70
70
 
metadata CHANGED
@@ -1,19 +1,18 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: active_genie
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.26.6
4
+ version: 0.28.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Radamés Roriz
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2025-06-09 00:00:00.000000000 Z
11
+ date: 2025-07-12 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description: |
14
- The lodash for GenAI, stop reinventing the wheel
15
-
16
- ActiveGenie is a Ruby gem that provides valuable solutions using GenAI. Just like Lodash or ActiveStorage, ActiveGenie brings a set of Modules to reach real value fast and reliably. Backed by a custom benchmarking system that ensures consistent quality and performance across different models and providers in every release.
14
+ ActiveGenie is a Ruby gem that helps developers build reliable, future-proof GenAI features without worrying about changing models, prompts, or providers. Like Lodash for GenAI, it offers simple, reusable modules for tasks like data extraction, scoring, and ranking, so you can focus on your app’s logic, not the shifting AI landscape.
15
+ Behind the scenes, a custom benchmarking system keeps everything consistent across LLM vendors and versions, release after release.
17
16
  email:
18
17
  - radames@roriz.dev
19
18
  executables: []
@@ -25,9 +24,11 @@ files:
25
24
  - VERSION
26
25
  - lib/active_genie.rb
27
26
  - lib/active_genie/battle.rb
28
- - lib/active_genie/battle/README.md
27
+ - lib/active_genie/battle/fight.json
28
+ - lib/active_genie/battle/fight.prompt.md
29
+ - lib/active_genie/battle/fight.rb
29
30
  - lib/active_genie/battle/generalist.json
30
- - lib/active_genie/battle/generalist.md
31
+ - lib/active_genie/battle/generalist.prompt.md
31
32
  - lib/active_genie/battle/generalist.rb
32
33
  - lib/active_genie/clients/providers/anthropic_client.rb
33
34
  - lib/active_genie/clients/providers/base_client.rb
@@ -35,7 +36,6 @@ files:
35
36
  - lib/active_genie/clients/providers/google_client.rb
36
37
  - lib/active_genie/clients/providers/openai_client.rb
37
38
  - lib/active_genie/clients/unified_client.rb
38
- - lib/active_genie/config/README.md
39
39
  - lib/active_genie/config/battle_config.rb
40
40
  - lib/active_genie/config/data_extractor_config.rb
41
41
  - lib/active_genie/config/llm_config.rb
@@ -50,17 +50,15 @@ files:
50
50
  - lib/active_genie/config/scoring_config.rb
51
51
  - lib/active_genie/configuration.rb
52
52
  - lib/active_genie/data_extractor.rb
53
- - lib/active_genie/data_extractor/README.md
54
53
  - lib/active_genie/data_extractor/from_informal.json
55
54
  - lib/active_genie/data_extractor/from_informal.rb
56
55
  - lib/active_genie/data_extractor/generalist.json
57
- - lib/active_genie/data_extractor/generalist.md
56
+ - lib/active_genie/data_extractor/generalist.prompt.md
58
57
  - lib/active_genie/data_extractor/generalist.rb
59
58
  - lib/active_genie/errors/invalid_log_output_error.rb
60
59
  - lib/active_genie/errors/invalid_provider_error.rb
61
60
  - lib/active_genie/logger.rb
62
61
  - lib/active_genie/ranking.rb
63
- - lib/active_genie/ranking/README.md
64
62
  - lib/active_genie/ranking/elo_round.rb
65
63
  - lib/active_genie/ranking/free_for_all.rb
66
64
  - lib/active_genie/ranking/player.rb
@@ -68,9 +66,8 @@ files:
68
66
  - lib/active_genie/ranking/ranking.rb
69
67
  - lib/active_genie/ranking/ranking_scoring.rb
70
68
  - lib/active_genie/scoring.rb
71
- - lib/active_genie/scoring/README.md
72
69
  - lib/active_genie/scoring/generalist.json
73
- - lib/active_genie/scoring/generalist.md
70
+ - lib/active_genie/scoring/generalist.prompt.md
74
71
  - lib/active_genie/scoring/generalist.rb
75
72
  - lib/active_genie/scoring/recommended_reviewers.rb
76
73
  - lib/tasks/benchmark.rake
@@ -103,5 +100,5 @@ requirements: []
103
100
  rubygems_version: 3.4.20
104
101
  signing_key:
105
102
  specification_version: 4
106
- summary: Transform your Ruby application with powerful, production-ready GenAI features
103
+ summary: 'The Lodash for GenAI: Real Value + Consistent + Model-Agnostic'
107
104
  test_files: []
@@ -1,39 +0,0 @@
1
- # Battle
2
- AI-powered battle evaluation system that determines winners between two players based on specified criteria.
3
-
4
- ## Features
5
- - Content comparison - Evaluate submissions from two players against defined criteria
6
- - Objective analysis - AI-powered assessment of how well each player meets requirements
7
- - Detailed reasoning - Comprehensive explanation of why a winner was chosen
8
- - Draw avoidance - Suggestions on how to modify content to avoid draws
9
- - Flexible input - Support for both string content and structured data with content field
10
-
11
- ## Basic Usage
12
- Evaluate a battle between two players with simple text content:
13
-
14
- ```ruby
15
- player_a = "Implementation uses dependency injection for better testability"
16
- player_b = "Code has high test coverage but tightly coupled components"
17
- criteria = "Evaluate code quality and maintainability"
18
-
19
- result = ActiveGenie::Battle.call(player_a, player_b, criteria)
20
- # => {
21
- # winner_player: "Implementation uses dependency injection for better testability",
22
- # reasoning: "Player A's implementation demonstrates better maintainability through dependency injection,
23
- # which allows for easier testing and component replacement. While Player B has good test coverage,
24
- # the tight coupling makes the code harder to maintain and modify.",
25
- # what_could_be_changed_to_avoid_draw: "Focus on specific architectural patterns and design principles"
26
- # }
27
- ```
28
-
29
- ## Interface
30
- ### .call(player_a, player_b, criteria, config: {})
31
- - `player_a` [String, Hash] - The content or submission from the first player
32
- - `player_b` [String, Hash] - The content or submission from the second player
33
- - `criteria` [String] - The evaluation criteria or rules to assess against
34
- - `config` [Hash] - Additional configuration config that modify the battle evaluation behavior
35
-
36
- Returns a Hash containing:
37
- - `winner_player` [String, Hash] - The winning player's content (either player_a or player_b)
38
- - `reasoning` [String] - Detailed explanation of why the winner was chosen
39
- - `what_could_be_changed_to_avoid_draw` [String] - A suggestion on how to avoid a draw
@@ -1,235 +0,0 @@
1
- # ActiveGenie Configuration
2
-
3
- ActiveGenie provides a flexible configuration system that can be customized to suit your needs. This document details all available configuration options.
4
-
5
- ## Initial Configuration
6
-
7
- To configure ActiveGenie, use the `ActiveGenie.configure` block in your application's initialization:
8
-
9
- ```ruby
10
- ActiveGenie.configure do |config|
11
- # Provider configurations
12
- config.providers.openai.api_key = ENV['OPENAI_API_KEY']
13
-
14
- # Log configurations
15
- config.log.file_path = 'log/custom_genie.log'
16
- config.log.fine_tune_file_path = 'log/fine_tune_genie.log'
17
-
18
- # Other configurations can be added here
19
- end
20
- ```
21
-
22
- ## Configuration Sections
23
-
24
- ### 1. Log Configuration (`config.log`)
25
-
26
- The log configuration controls how ActiveGenie handles logging of its operations.
27
-
28
- #### Available Settings:
29
-
30
- | Setting | Type | Default | Description |
31
- |---------|------|---------|-------------|
32
- | `file_path` | String | `'log/active_genie.log'` | Path to the main log file where all logs will be written. |
33
- | `fine_tune_file_path` | String | `'log/active_genie_fine_tune.log'` | Path to the fine-tuning specific log file. |
34
- | `output` | Proc | `->(log) { $stdout.puts log }` | Custom output handler for logs. Must respond to `call`. |
35
- | `additional_context` | Hash | `{}` | Additional context to be added to each log entry. |
36
-
37
- #### Methods:
38
-
39
- - `add_observer(observers: [], scope: {}, &block)`
40
- - Adds log observers that will be notified of log events.
41
- - `observers`: An array of callable objects or a single callable object.
42
- - `scope`: A hash to filter which logs the observer receives.
43
- - `block`: A block that will be called for matching logs.
44
-
45
- - `remove_observer(observers)`
46
- - Removes the specified observers.
47
-
48
- #### Example:
49
-
50
- ```ruby
51
- ActiveGenie.configure do |config|
52
- config.log.file_path = 'log/my_app/genie.log'
53
- config.log.fine_tune_file_path = 'log/my_app/fine_tune.log'
54
-
55
- # Add a custom output handler
56
- config.log.output = ->(log) { MyLogger.info(log) }
57
-
58
- # Add an observer for specific log events
59
- config.log.add_observer(scope: { event: :api_call }) do |log|
60
- StatsD.increment("genie.api_calls")
61
- end
62
- end
63
- ```
64
-
65
- ### 2. LLM Configuration (`config.llm`)
66
-
67
- The LLM configuration is used to define settings for interacting with Large Language Models.
68
-
69
- #### Available Settings:
70
-
71
- | Setting | Type | Default | Description |
72
- |-----------------|---------------------|------------------------------|------------------------------------------------------------------------------------------------------------|
73
- | `model` | String / nil | `nil` | The specific LLM model to use (e.g., 'gpt-4', 'claude-2'). |
74
- | `provider` | Symbol / nil | `nil` | The LLM provider (e.g., `:openai`, `:anthropic`). Set as a string, stored as a symbol. |
75
- | `client` | Object / nil | `nil` | A pre-configured client instance for the LLM provider. If set, other settings might be ignored. |
76
- | `temperature` | Numeric | `0` | Controls randomness. Higher values (e.g., 0.8) = more random, lower (e.g., 0.2) = more deterministic. |
77
- | `max_tokens` | Integer | `4096` | Maximum number of tokens to generate in the LLM response. |
78
- | `max_retries` | Integer / nil | `nil` | Maximum number of times to retry a failed API call to the LLM. |
79
- | `retry_delay` | Numeric / nil | `nil` | Delay (in seconds) between retries for failed API calls. |
80
- | `model_tier` | Enum [lower_tier, middle_tier, higher_tier] | `'lower_tier'` | Specifies the model tier, potentially affecting cost/performance. Will be used if `model` is not set. |
81
- | `read_timeout` | Numeric / nil | `nil` | Timeout (in seconds) for reading data from the LLM API. |
82
- | `open_timeout` | Numeric / nil | `nil` | Timeout (in seconds) for establishing a connection to the LLM API. |
83
-
84
- #### Example:
85
-
86
- ```ruby
87
- ActiveGenie.configure do |config|
88
- config.llm.provider = :openai
89
- config.llm.model = 'gpt-999'
90
- config.llm.temperature = 0.1
91
- config.llm.max_tokens = 8000
92
- config.llm.max_retries = 1
93
- config.llm.retry_delay = 5 # seconds
94
- config.llm.model_tier = 'higher_tier' # If model is not set, will use the model tier to select a model
95
- end
96
- ```
97
-
98
- ### 4. Providers Configuration (`config.providers`)
99
-
100
- The Providers configuration (`config.providers`) manages settings for various Large Language Model (LLM) providers. It allows you to configure multiple providers, set a default, and access individual provider settings. Active Genie automatically loads configurations for supported providers (OpenAI, Anthropic, DeepSeek, Google).
101
-
102
- #### Main Provider Settings:
103
-
104
- | Setting | Type | Default | Description |
105
- |-----------|---------------------|--------------------------------|-----------------------------------------------------------------------------|
106
- | `default` | String / Symbol | First validly configured provider (often `:openai` if its API key is set) | The name of the default LLM provider to use (e.g., `:openai`, `:anthropic`). You can set this to your preferred provider. |
107
-
108
- #### Methods for `config.providers`:
109
-
110
- - `add(provider_classes)`
111
- - Adds one or more custom provider configuration classes. `provider_classes` can be a single class or an array of classes. (Typically not needed for built-in providers).
112
- - `remove(provider_classes)`
113
- - Removes one or more provider configurations based on their classes.
114
-
115
- #### Example for Main Provider Settings:
116
-
117
- ```ruby
118
- ActiveGenie.configure do |config|
119
- # Set the default provider to use if not specifying one explicitly in operations
120
- config.providers.default = :anthropic
121
-
122
- # Configure individual providers (API keys are often set via ENV variables)
123
- config.providers.openai.api_key = ENV['OPENAI_API_KEY']
124
- config.providers.anthropic.api_key = ENV['ANTHROPIC_API_KEY']
125
- # config.providers.deepseek.api_key = ENV['DEEPSEEK_API_KEY']
126
- # config.providers.google.api_key = ENV['GEMINI_API_KEY']
127
- end
128
- ```
129
-
130
- #### Individual Provider Configurations:
131
-
132
- The following subsections detail the configurations for each supported LLM provider. Each provider configuration (e.g., `config.providers.openai`) allows setting an `api_key`, `api_url`, and specific model names for `lower_tier_model`, `middle_tier_model`, and `higher_tier_model`.
133
-
134
- ##### a. OpenAI (`config.providers.openai`)
135
-
136
- Internal Name: `:openai`
137
-
138
- | Setting | Type | Default | Description |
139
- |---------------------|--------|---------------------------------------|-----------------------------------------------------------------------------|
140
- | `api_key` | String | `ENV['OPENAI_API_KEY']` | Your OpenAI API key. |
141
- | `api_url` | String | `'https://api.openai.com/v1'` | Base URL for the OpenAI API. |
142
- | `lower_tier_model` | String | `'gpt-4.1-mini'` | Model for lower-tier usage (cost-effective, faster). |
143
- | `middle_tier_model` | String | `'gpt-4.1'` | Model for middle-tier usage (balanced performance). |
144
- | `higher_tier_model` | String | `'o3-mini'` | Model for higher-tier usage (most capable). |
145
-
146
- ##### b. Anthropic (`config.providers.anthropic`)
147
-
148
- Internal Name: `:anthropic`
149
-
150
- | Setting | Type | Default | Description |
151
- |---------------------|--------|------------------------------------------|-----------------------------------------------------------------------------|
152
- | `api_key` | String | `ENV['ANTHROPIC_API_KEY']` | Your Anthropic API key. |
153
- | `api_url` | String | `'https://api.anthropic.com'` | Base URL for the Anthropic API. |
154
- | `anthropic_version` | String | `'2023-06-01'` | The API version for Anthropic. |
155
- | `lower_tier_model` | String | `'claude-3-5-haiku-20241022'` | Model for lower-tier usage. |
156
- | `middle_tier_model` | String | `'claude-3-7-sonnet-20250219'` | Model for middle-tier usage. |
157
- | `higher_tier_model` | String | `'claude-3-opus-20240229'` | Model for higher-tier usage. |
158
-
159
- ##### c. DeepSeek (`config.providers.deepseek`)
160
-
161
- Internal Name: `:deepseek`
162
-
163
- | Setting | Type | Default | Description |
164
- |---------------------|--------|---------------------------------------|-----------------------------------------------------------------------------|
165
- | `api_key` | String | `ENV['DEEPSEEK_API_KEY']` | Your DeepSeek API key. |
166
- | `api_url` | String | `'https://api.deepseek.com/v1'` | Base URL for the DeepSeek API. |
167
- | `lower_tier_model` | String | `'deepseek-chat'` | Model for lower-tier usage. |
168
- | `middle_tier_model` | String | `'deepseek-chat'` | Model for middle-tier usage. |
169
- | `higher_tier_model` | String | `'deepseek-reasoner'` | Model for higher-tier usage. |
170
-
171
- ##### d. Google (`config.providers.google`)
172
-
173
- Internal Name: `:google`
174
-
175
- | Setting | Type | Default | Description |
176
- |---------------------|--------|---------------------------------------------------------------|-----------------------------------------------------------------------------|
177
- | `api_key` | String | `ENV['GENERATIVE_LANGUAGE_GOOGLE_API_KEY']` or `ENV['GEMINI_API_KEY']` | Your Google API key for Gemini. |
178
- | `api_url` | String | `'https://generativelanguage.googleapis.com'` | Base URL for the Google Generative Language API. |
179
- | `lower_tier_model` | String | `'gemini-2.0-flash-lite'` | Model for lower-tier usage. |
180
- | `middle_tier_model` | String | `'gemini-2.0-flash'` | Model for middle-tier usage. |
181
- | `higher_tier_model` | String | `'gemini-2.5-pro-experimental'` | Model for higher-tier usage. |
182
-
183
- #### Example for Overriding Individual Provider Settings:
184
-
185
- ```ruby
186
- ActiveGenie.configure do |config|
187
- config.providers.openai.api_key = 'sk-yourOpenAiKey...'
188
- config.providers.openai.middle_tier_model = 'gpt-4o' # Override default middle tier for OpenAI
189
-
190
- config.providers.anthropic.api_key = 'sk-ant-yourAnthropicKey...'
191
- config.providers.anthropic.anthropic_version = '2024-02-15' # Override Anthropic API version
192
-
193
- config.providers.google.higher_tier_model = 'gemini-1.5-pro-latest' # Use a specific Google model
194
- end
195
- ```
196
-
197
- ### 5. Ranking Configuration (`config.ranking`)
198
-
199
- The Ranking configuration (`config.ranking`) deals with settings related to how results or items are ranked.
200
-
201
- #### Available Settings:
202
-
203
- | Setting | Type | Default | Description |
204
- |-----------------------------|---------|---------|------------------------------------------------------------------------------------------------------------|
205
- | `score_variation_threshold` | Integer | `30` | A threshold (percentage) used to determine significant variations in scores when ranking or comparing items. |
206
-
207
- #### Example:
208
-
209
- ```ruby
210
- ActiveGenie.configure do |config|
211
- config.ranking.score_variation_threshold = 25 # Set to 25%
212
- end
213
- ```
214
-
215
- ### 6. Data Extractor Configuration (`config.data_extractor`)
216
-
217
- The Data Extractor configuration (`config.data_extractor`) provides settings for how data is extracted and processed, likely from text or other sources using LLMs.
218
-
219
- #### Available Settings:
220
-
221
- | Setting | Type | Default | Description |
222
- |--------------------|---------|---------|---------------------------------------------------------------------------------------------------------|
223
- | `with_explanation` | Boolean | `true` | Whether the data extraction process should also attempt to provide an explanation for the extracted data. |
224
- | `min_accuracy` | Integer | `70` | The minimum accuracy (percentage) required for extracted data to be considered valid or useful. |
225
- | `verbose` | Boolean | `false` | If true, enables more detailed logging or output from the data extraction process. |
226
-
227
- #### Example:
228
-
229
- ```ruby
230
- ActiveGenie.configure do |config|
231
- config.data_extractor.with_explanation = false
232
- config.data_extractor.min_accuracy = 85
233
- config.data_extractor.verbose = true
234
- end
235
- ```
@@ -1,131 +0,0 @@
1
- # Data Extractor
2
- Extract structured data from text using AI-powered analysis, handling informal language and complex expressions.
3
-
4
- ## ✨ Features
5
- - Structured data extraction - Extract typed data from unstructured text using predefined schemas
6
- - Informal text analysis - Identifies and handles informal language patterns, including litotes
7
- - Explanation tracking - Provides reasoning for each extracted data point
8
-
9
- ## Basic Usage
10
-
11
- Extract structured data from text using predefined schemas:
12
-
13
- ```ruby
14
- text = "John Doe is 25 years old"
15
- schema = {
16
- name: { type: 'string', description: 'Full name of the person' },
17
- age: { type: 'integer', description: 'Age in years' }
18
- }
19
- result = ActiveGenie::DataExtractor.call(text, schema)
20
- # => {
21
- # name: "John Doe",
22
- # name_explanation: "Found directly in text",
23
- # age: 25,
24
- # age_explanation: "Explicitly stated as 25 years old"
25
- # }
26
-
27
- product = "Nike Air Max 90 - Size 42 - $199.99"
28
- schema = {
29
- brand: {
30
- type: 'string',
31
- enum: ["Nike", "Adidas", "Puma"]
32
- },
33
- price: {
34
- type: 'number',
35
- minimum: 0
36
- },
37
- currency: {
38
- type: 'string',
39
- enum: ["USD", "EUR"]
40
- },
41
- size: {
42
- type: 'integer',
43
- minimum: 35,
44
- maximum: 46
45
- }
46
- }
47
-
48
- result = ActiveGenie::DataExtractor.call(product, schema)
49
- # => {
50
- # brand: "Nike",
51
- # brand_explanation: "Brand name found at start of text",
52
- # price: 199.99,
53
- # price_explanation: "Price found in USD format at end",
54
- # size: 42,
55
- # size_explanation: "Size explicitly stated in the middle",
56
- # currency: "USD",
57
- # currency_explanation: "Derived from $ symbol"
58
- # }
59
- ```
60
-
61
- ## Informal Text Processing
62
-
63
- The `from_informal` method extends the basic extraction by analyzing rhetorical devices and informal language patterns like:
64
-
65
- - Litotes ("not bad", "isn't terrible")
66
- - Affirmative expressions ("sure", "no problem")
67
- - Negative expressions ("nah", "not really")
68
-
69
- ### Example
70
-
71
- ```ruby
72
- text = "The weather isn't bad today"
73
- schema = {
74
- mood: { type: 'string', description: 'The mood of the message' }
75
- }
76
-
77
- result = ActiveGenie::DataExtractor.from_informal(text, schema)
78
- # => {
79
- # mood: "positive",
80
- # mood_explanation: "Speaker views weather favorably",
81
- # message_litote: true,
82
- # litote_rephrased: "The weather is good today"
83
- # }
84
- ```
85
-
86
- ### Usage Notes
87
- - Best suited for processing conversational user inputs
88
- - Automatically detects and interprets litotes
89
- - Provides rephrased positive statements for litotes
90
- - May require more processing time due to rhetorical analysis
91
- - Accuracy depends on context clarity
92
-
93
- ⚠️ Performance Impact: This method performs additional rhetorical analysis, which can increase processing time.
94
-
95
- ## Interface
96
-
97
- ### `.call(text, data_to_extract, config = {})`
98
- Extracts structured data from text based on a predefined schema.
99
-
100
- #### Parameters
101
- | Name | Type | Description | Required | Example |
102
- | --- | --- | --- | --- | --- |
103
- | `text` | `String` | The text to analyze and extract data from | Yes | "John Doe is 25 years old" |
104
- | `data_to_extract` | `Hash` | Schema defining the data structure to extract | Yes | `{ name: { type: 'string' } }` |
105
- | `config` | `Hash` | Additional extraction configuration | No | `{ model: "gpt-4" }` |
106
-
107
- #### config
108
- | Name | Type | Description |
109
- | --- | --- | --- |
110
- | `model` | `String` | The model to use for the extraction |
111
- | `api_key` | `String` | The API key to use for the extraction |
112
-
113
- #### Returns
114
- `Hash` containing:
115
- - Extracted values matching the schema structure
116
- - Explanation field for each extracted value
117
- - Additional analysis fields when using `from_informal`
118
-
119
- ### `.from_informal(text, data_to_extract, config = {})`
120
- Extends extraction with rhetorical analysis, particularly for litotes.
121
-
122
- #### Additional Return Fields
123
- | Name | Type | Description |
124
- | --- | --- | --- |
125
- | `message_litote` | `Boolean` | Whether the text contains a litote |
126
- | `litote_rephrased` | `String` | Positive rephrasing of any detected litote |
127
-
128
- ⚠️ Performance Considerations
129
- - Both methods may require multiple AI model calls
130
- - Informal processing requires additional rhetorical analysis
131
- - Consider background processing for production use
@@ -1,73 +0,0 @@
1
- # Ranking
2
-
3
- The `ActiveGenie::Ranking` module organizes players based on scores derived from textual evaluations and then ranks them using a multi-stage process. It leverages the scoring system from the `ActiveGenie::Scoring` module to assign initial scores, and then applies advanced ranking methods to produce a competitive ranking.
4
-
5
- ## Overview
6
-
7
- The ranking system performs the following steps:
8
-
9
- 1. **Initial Scoring**: Each player’s textual content is evaluated using `ActiveGenie::Scoring`. This produces a `score` based on multiple expert reviews.
10
-
11
- 2. **Elimination of Poor Performers**: Players whose scores show a high coefficient of variation (indicating inconsistency) are progressively eliminated. This ensures that only competitive candidates continue in the ranking process.
12
-
13
- 3. **ELO Ranking**: If there are more than 10 eligible players, an ELO-based ranking is applied. Battles between players are simulated via `ActiveGenie::Battle`, and scores are updated using an ELO algorithm tailored to the ranking.
14
-
15
- 4. **Free for all Matches**: Finally, the remaining players engage in head-to-head matches where each unique pair competes. Match outcomes (wins, losses, draws) are recorded to finalize the rankings.
16
-
17
- ## Components
18
-
19
- - **ranking**: Orchestrates the entire ranking process. Initializes player scores, eliminates outliers, and coordinates ranking rounds.
20
-
21
- - **EloRanking**: Applies an ELO-based system to rank players through simulated battles. It updates players’ ELO scores based on match outcomes and predefined rules (including penalties for losses).
22
-
23
- - **Free for all**: Conducts complete pairwise matches among eligible players to record win/loss/draw statistics and refine the final standings.
24
-
25
- ## Usage
26
-
27
- Call the ranking using:
28
-
29
- ```ruby
30
- result = ActiveGenie::Ranking.call(players, criteria, config: {})
31
- ```
32
-
33
- - `players`: A collection of player instances, each containing textual content to be scored.
34
- - `criteria`: A string defining the evaluation criteria used by the scoring system.
35
- - `config`: A hash of additional parameters for customization (e.g., model, api_key).
36
-
37
- The method processes the players through scoring, elimination, and ranking phases, then returns a hash containing the player statistics and rankings.
38
-
39
- ## Possible improvements
40
- - Adjust initial criteria to ensure consistency
41
- - Adjust each player's content to ensure consistency
42
- - Support players with images or audio
43
- - Parallelize processing battles and scoring
44
-
45
- ## Ranking Configuration
46
-
47
- | Config | Description | Default |
48
- |--------|-------------|---------|
49
- | `score_variation_threshold` | Threshold for eliminating players with inconsistent scores | `30` |
50
-
51
- ## Ranking Callbacks
52
- Callbacks are optional and can be used to watch any changes in players, battles, or scoring.
53
-
54
- | Callback | Description |
55
- |--------|-------------|
56
- | `watch_players` | Callback to watch any changes in players |
57
- | `watch_battles` | Callback to watch any changes in battles |
58
- | `watch_scoring` | Callback to watch any changes in scoring |
59
-
60
- Example of callback usage:
61
-
62
- ```ruby
63
- result = ActiveGenie::Ranking.call(
64
- players,
65
- criteria,
66
- config: {
67
- watch_players: ->(player) { puts player },
68
- watch_battles: ->(battle) { puts battle },
69
- watch_scoring: ->(scoring) { puts scoring }
70
- }
71
- )
72
- ```
73
-
@@ -1,76 +0,0 @@
1
- # Scoring
2
- Text evaluation system that provides detailed scoring and feedback using multiple expert reviewers.
3
-
4
- ## Features
5
- - Multi-reviewer evaluation - Get scores and feedback from multiple AI-powered expert reviewers
6
- - Automatic reviewer selection - Smart recommendation of reviewers based on content and criteria
7
- - Detailed feedback - Comprehensive reasoning for each reviewer's score
8
- - Customizable weights - Adjust the importance of different reviewers' scores
9
- - Flexible criteria - Score text against any specified evaluation criteria
10
-
11
- ## Basic Usage
12
-
13
- Score text using predefined reviewers:
14
-
15
- ```ruby
16
- text = "The code implements a binary search algorithm with O(log n) complexity"
17
- criteria = "Evaluate technical accuracy and clarity"
18
- reviewers = ["Algorithm Expert", "Technical Writer"]
19
-
20
- result = ActiveGenie::Scoring.call(text, criteria, reviewers)
21
- # => {
22
- # algorithm_expert_score: 95,
23
- # algorithm_expert_reasoning: "Accurately describes binary search and its complexity",
24
- # technical_writer_score: 90,
25
- # technical_writer_reasoning: "Clear and concise explanation of the algorithm",
26
- # final_score: 92.5
27
- # }
28
- ```
29
-
30
- ## Automatic Reviewer Selection
31
-
32
- When no reviewers are specified, the system automatically recommends appropriate reviewers:
33
-
34
- ```ruby
35
- text = "The patient shows signs of improved cardiac function"
36
- criteria = "Evaluate medical accuracy and clarity"
37
-
38
- result = ActiveGenie::Scoring.call(text, criteria)
39
- # => {
40
- # cardiologist_score: 88,
41
- # cardiologist_reasoning: "Accurate assessment of cardiac improvement",
42
- # medical_writer_score: 85,
43
- # medical_writer_reasoning: "Clear communication of medical findings",
44
- # general_practitioner_score: 90,
45
- # general_practitioner_reasoning: "Well-structured medical observation",
46
- # final_score: 87.7
47
- # }
48
- ```
49
-
50
- ## Interface
51
-
52
- ### `.call(text, criteria, reviewers = [], config: {})`
53
- Main interface for scoring text content.
54
-
55
- #### Parameters
56
- - `text` [String] - The text content to be evaluated
57
- - `criteria` [String] - The evaluation criteria or rubric to assess against
58
- - `reviewers` [Array<String>] - Optional list of specific reviewers
59
- - `config` [Hash] - Additional configuration config
60
-
61
- ### `RecommendedReviewers.call(text, criteria, config: {})`
62
- Recommends appropriate reviewers based on content and criteria.
63
-
64
- #### Parameters
65
- - `text` [String] - The text content to analyze
66
- - `criteria` [String] - The evaluation criteria
67
- - `config` [Hash] - Additional configuration config
68
-
69
- ### Usage Notes
70
- - Best suited for objective evaluation of text content
71
- - Provides balanced scoring through multiple reviewers
72
- - Automatically handles reviewer selection when needed
73
- - Supports custom weighting of reviewer scores
74
- - Returns detailed reasoning for each score
75
-
76
- Performance Impact: Using multiple reviewers or requesting detailed feedback may increase processing time.