active_genie 0.26.5 → 0.27.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +10 -193
- data/VERSION +1 -1
- data/lib/active_genie/battle/generalist.rb +9 -9
- data/lib/active_genie/clients/providers/anthropic_client.rb +2 -2
- data/lib/active_genie/clients/providers/base_client.rb +2 -2
- data/lib/active_genie/clients/providers/deepseek_client.rb +2 -2
- data/lib/active_genie/clients/providers/google_client.rb +3 -10
- data/lib/active_genie/clients/providers/openai_client.rb +2 -2
- data/lib/active_genie/config/log_config.rb +12 -15
- data/lib/active_genie/config/providers/anthropic_config.rb +2 -2
- data/lib/active_genie/config/providers/deepseek_config.rb +2 -2
- data/lib/active_genie/config/providers/google_config.rb +2 -2
- data/lib/active_genie/config/providers/openai_config.rb +2 -2
- data/lib/active_genie/config/providers/provider_base.rb +4 -4
- data/lib/active_genie/configuration.rb +16 -4
- data/lib/active_genie/data_extractor/generalist.rb +2 -2
- data/lib/active_genie/logger.rb +40 -22
- data/lib/active_genie/ranking/elo_round.rb +30 -37
- data/lib/active_genie/ranking/free_for_all.rb +29 -28
- data/lib/active_genie/ranking/player.rb +1 -9
- data/lib/active_genie/ranking/ranking.rb +32 -25
- data/lib/active_genie/ranking/ranking_scoring.rb +12 -17
- data/lib/active_genie/scoring/generalist.rb +9 -9
- data/lib/tasks/templates/active_genie.rb +1 -1
- metadata +8 -13
- data/lib/active_genie/data_extractor/README.md +0 -131
- data/lib/active_genie/ranking/README.md +0 -73
- data/lib/active_genie/ranking/concerns/loggable.rb +0 -29
- data/lib/active_genie/scoring/README.md +0 -76
- /data/lib/active_genie/battle/{generalist.md → generalist.prompt.md} +0 -0
- /data/lib/active_genie/data_extractor/{generalist.md → generalist.prompt.md} +0 -0
- /data/lib/active_genie/scoring/{generalist.md → generalist.prompt.md} +0 -0
@@ -1,131 +0,0 @@
|
|
1
|
-
# Data Extractor
|
2
|
-
Extract structured data from text using AI-powered analysis, handling informal language and complex expressions.
|
3
|
-
|
4
|
-
## ✨ Features
|
5
|
-
- Structured data extraction - Extract typed data from unstructured text using predefined schemas
|
6
|
-
- Informal text analysis - Identifies and handles informal language patterns, including litotes
|
7
|
-
- Explanation tracking - Provides reasoning for each extracted data point
|
8
|
-
|
9
|
-
## Basic Usage
|
10
|
-
|
11
|
-
Extract structured data from text using predefined schemas:
|
12
|
-
|
13
|
-
```ruby
|
14
|
-
text = "John Doe is 25 years old"
|
15
|
-
schema = {
|
16
|
-
name: { type: 'string', description: 'Full name of the person' },
|
17
|
-
age: { type: 'integer', description: 'Age in years' }
|
18
|
-
}
|
19
|
-
result = ActiveGenie::DataExtractor.call(text, schema)
|
20
|
-
# => {
|
21
|
-
# name: "John Doe",
|
22
|
-
# name_explanation: "Found directly in text",
|
23
|
-
# age: 25,
|
24
|
-
# age_explanation: "Explicitly stated as 25 years old"
|
25
|
-
# }
|
26
|
-
|
27
|
-
product = "Nike Air Max 90 - Size 42 - $199.99"
|
28
|
-
schema = {
|
29
|
-
brand: {
|
30
|
-
type: 'string',
|
31
|
-
enum: ["Nike", "Adidas", "Puma"]
|
32
|
-
},
|
33
|
-
price: {
|
34
|
-
type: 'number',
|
35
|
-
minimum: 0
|
36
|
-
},
|
37
|
-
currency: {
|
38
|
-
type: 'string',
|
39
|
-
enum: ["USD", "EUR"]
|
40
|
-
},
|
41
|
-
size: {
|
42
|
-
type: 'integer',
|
43
|
-
minimum: 35,
|
44
|
-
maximum: 46
|
45
|
-
}
|
46
|
-
}
|
47
|
-
|
48
|
-
result = ActiveGenie::DataExtractor.call(product, schema)
|
49
|
-
# => {
|
50
|
-
# brand: "Nike",
|
51
|
-
# brand_explanation: "Brand name found at start of text",
|
52
|
-
# price: 199.99,
|
53
|
-
# price_explanation: "Price found in USD format at end",
|
54
|
-
# size: 42,
|
55
|
-
# size_explanation: "Size explicitly stated in the middle",
|
56
|
-
# currency: "USD",
|
57
|
-
# currency_explanation: "Derived from $ symbol"
|
58
|
-
# }
|
59
|
-
```
|
60
|
-
|
61
|
-
## Informal Text Processing
|
62
|
-
|
63
|
-
The `from_informal` method extends the basic extraction by analyzing rhetorical devices and informal language patterns like:
|
64
|
-
|
65
|
-
- Litotes ("not bad", "isn't terrible")
|
66
|
-
- Affirmative expressions ("sure", "no problem")
|
67
|
-
- Negative expressions ("nah", "not really")
|
68
|
-
|
69
|
-
### Example
|
70
|
-
|
71
|
-
```ruby
|
72
|
-
text = "The weather isn't bad today"
|
73
|
-
schema = {
|
74
|
-
mood: { type: 'string', description: 'The mood of the message' }
|
75
|
-
}
|
76
|
-
|
77
|
-
result = ActiveGenie::DataExtractor.from_informal(text, schema)
|
78
|
-
# => {
|
79
|
-
# mood: "positive",
|
80
|
-
# mood_explanation: "Speaker views weather favorably",
|
81
|
-
# message_litote: true,
|
82
|
-
# litote_rephrased: "The weather is good today"
|
83
|
-
# }
|
84
|
-
```
|
85
|
-
|
86
|
-
### Usage Notes
|
87
|
-
- Best suited for processing conversational user inputs
|
88
|
-
- Automatically detects and interprets litotes
|
89
|
-
- Provides rephrased positive statements for litotes
|
90
|
-
- May require more processing time due to rhetorical analysis
|
91
|
-
- Accuracy depends on context clarity
|
92
|
-
|
93
|
-
⚠️ Performance Impact: This method performs additional rhetorical analysis, which can increase processing time.
|
94
|
-
|
95
|
-
## Interface
|
96
|
-
|
97
|
-
### `.call(text, data_to_extract, config = {})`
|
98
|
-
Extracts structured data from text based on a predefined schema.
|
99
|
-
|
100
|
-
#### Parameters
|
101
|
-
| Name | Type | Description | Required | Example |
|
102
|
-
| --- | --- | --- | --- | --- |
|
103
|
-
| `text` | `String` | The text to analyze and extract data from | Yes | "John Doe is 25 years old" |
|
104
|
-
| `data_to_extract` | `Hash` | Schema defining the data structure to extract | Yes | `{ name: { type: 'string' } }` |
|
105
|
-
| `config` | `Hash` | Additional extraction configuration | No | `{ model: "gpt-4" }` |
|
106
|
-
|
107
|
-
#### config
|
108
|
-
| Name | Type | Description |
|
109
|
-
| --- | --- | --- |
|
110
|
-
| `model` | `String` | The model to use for the extraction |
|
111
|
-
| `api_key` | `String` | The API key to use for the extraction |
|
112
|
-
|
113
|
-
#### Returns
|
114
|
-
`Hash` containing:
|
115
|
-
- Extracted values matching the schema structure
|
116
|
-
- Explanation field for each extracted value
|
117
|
-
- Additional analysis fields when using `from_informal`
|
118
|
-
|
119
|
-
### `.from_informal(text, data_to_extract, config = {})`
|
120
|
-
Extends extraction with rhetorical analysis, particularly for litotes.
|
121
|
-
|
122
|
-
#### Additional Return Fields
|
123
|
-
| Name | Type | Description |
|
124
|
-
| --- | --- | --- |
|
125
|
-
| `message_litote` | `Boolean` | Whether the text contains a litote |
|
126
|
-
| `litote_rephrased` | `String` | Positive rephrasing of any detected litote |
|
127
|
-
|
128
|
-
⚠️ Performance Considerations
|
129
|
-
- Both methods may require multiple AI model calls
|
130
|
-
- Informal processing requires additional rhetorical analysis
|
131
|
-
- Consider background processing for production use
|
@@ -1,73 +0,0 @@
|
|
1
|
-
# Ranking
|
2
|
-
|
3
|
-
The `ActiveGenie::Ranking` module organizes players based on scores derived from textual evaluations and then ranks them using a multi-stage process. It leverages the scoring system from the `ActiveGenie::Scoring` module to assign initial scores, and then applies advanced ranking methods to produce a competitive ranking.
|
4
|
-
|
5
|
-
## Overview
|
6
|
-
|
7
|
-
The ranking system performs the following steps:
|
8
|
-
|
9
|
-
1. **Initial Scoring**: Each player’s textual content is evaluated using `ActiveGenie::Scoring`. This produces a `score` based on multiple expert reviews.
|
10
|
-
|
11
|
-
2. **Elimination of Poor Performers**: Players whose scores show a high coefficient of variation (indicating inconsistency) are progressively eliminated. This ensures that only competitive candidates continue in the ranking process.
|
12
|
-
|
13
|
-
3. **ELO Ranking**: If there are more than 10 eligible players, an ELO-based ranking is applied. Battles between players are simulated via `ActiveGenie::Battle`, and scores are updated using an ELO algorithm tailored to the ranking.
|
14
|
-
|
15
|
-
4. **Free for all Matches**: Finally, the remaining players engage in head-to-head matches where each unique pair competes. Match outcomes (wins, losses, draws) are recorded to finalize the rankings.
|
16
|
-
|
17
|
-
## Components
|
18
|
-
|
19
|
-
- **ranking**: Orchestrates the entire ranking process. Initializes player scores, eliminates outliers, and coordinates ranking rounds.
|
20
|
-
|
21
|
-
- **EloRanking**: Applies an ELO-based system to rank players through simulated battles. It updates players’ ELO scores based on match outcomes and predefined rules (including penalties for losses).
|
22
|
-
|
23
|
-
- **Free for all**: Conducts complete pairwise matches among eligible players to record win/loss/draw statistics and refine the final standings.
|
24
|
-
|
25
|
-
## Usage
|
26
|
-
|
27
|
-
Call the ranking using:
|
28
|
-
|
29
|
-
```ruby
|
30
|
-
result = ActiveGenie::Ranking.call(players, criteria, config: {})
|
31
|
-
```
|
32
|
-
|
33
|
-
- `players`: A collection of player instances, each containing textual content to be scored.
|
34
|
-
- `criteria`: A string defining the evaluation criteria used by the scoring system.
|
35
|
-
- `config`: A hash of additional parameters for customization (e.g., model, api_key).
|
36
|
-
|
37
|
-
The method processes the players through scoring, elimination, and ranking phases, then returns a hash containing the player statistics and rankings.
|
38
|
-
|
39
|
-
## Possible improvements
|
40
|
-
- Adjust initial criteria to ensure consistency
|
41
|
-
- Adjust each player's content to ensure consistency
|
42
|
-
- Support players with images or audio
|
43
|
-
- Parallelize processing battles and scoring
|
44
|
-
|
45
|
-
## Ranking Configuration
|
46
|
-
|
47
|
-
| Config | Description | Default |
|
48
|
-
|--------|-------------|---------|
|
49
|
-
| `score_variation_threshold` | Threshold for eliminating players with inconsistent scores | `30` |
|
50
|
-
|
51
|
-
## Ranking Callbacks
|
52
|
-
Callbacks are optional and can be used to watch any changes in players, battles, or scoring.
|
53
|
-
|
54
|
-
| Callback | Description |
|
55
|
-
|--------|-------------|
|
56
|
-
| `watch_players` | Callback to watch any changes in players |
|
57
|
-
| `watch_battles` | Callback to watch any changes in battles |
|
58
|
-
| `watch_scoring` | Callback to watch any changes in scoring |
|
59
|
-
|
60
|
-
Example of callback usage:
|
61
|
-
|
62
|
-
```ruby
|
63
|
-
result = ActiveGenie::Ranking.call(
|
64
|
-
players,
|
65
|
-
criteria,
|
66
|
-
config: {
|
67
|
-
watch_players: ->(player) { puts player },
|
68
|
-
watch_battles: ->(battle) { puts battle },
|
69
|
-
watch_scoring: ->(scoring) { puts scoring }
|
70
|
-
}
|
71
|
-
)
|
72
|
-
```
|
73
|
-
|
@@ -1,29 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
module ActiveGenie
|
4
|
-
module Concerns
|
5
|
-
module Loggable
|
6
|
-
def self.included(base)
|
7
|
-
base.extend(ClassMethods)
|
8
|
-
end
|
9
|
-
|
10
|
-
module ClassMethods
|
11
|
-
def call_with_log_context(context_method)
|
12
|
-
original_method = instance_method(:call)
|
13
|
-
|
14
|
-
define_method(:call) do |*args, **kwargs, &block|
|
15
|
-
context = send(context_method, *args, **kwargs)
|
16
|
-
|
17
|
-
ActiveGenie::Logger.with_context(context) do
|
18
|
-
original_method.bind(self).call(*args, **kwargs, &block)
|
19
|
-
end
|
20
|
-
end
|
21
|
-
end
|
22
|
-
|
23
|
-
def logger(...)
|
24
|
-
ActiveGenie::Logger.call(...)
|
25
|
-
end
|
26
|
-
end
|
27
|
-
end
|
28
|
-
end
|
29
|
-
end
|
@@ -1,76 +0,0 @@
|
|
1
|
-
# Scoring
|
2
|
-
Text evaluation system that provides detailed scoring and feedback using multiple expert reviewers.
|
3
|
-
|
4
|
-
## Features
|
5
|
-
- Multi-reviewer evaluation - Get scores and feedback from multiple AI-powered expert reviewers
|
6
|
-
- Automatic reviewer selection - Smart recommendation of reviewers based on content and criteria
|
7
|
-
- Detailed feedback - Comprehensive reasoning for each reviewer's score
|
8
|
-
- Customizable weights - Adjust the importance of different reviewers' scores
|
9
|
-
- Flexible criteria - Score text against any specified evaluation criteria
|
10
|
-
|
11
|
-
## Basic Usage
|
12
|
-
|
13
|
-
Score text using predefined reviewers:
|
14
|
-
|
15
|
-
```ruby
|
16
|
-
text = "The code implements a binary search algorithm with O(log n) complexity"
|
17
|
-
criteria = "Evaluate technical accuracy and clarity"
|
18
|
-
reviewers = ["Algorithm Expert", "Technical Writer"]
|
19
|
-
|
20
|
-
result = ActiveGenie::Scoring.call(text, criteria, reviewers)
|
21
|
-
# => {
|
22
|
-
# algorithm_expert_score: 95,
|
23
|
-
# algorithm_expert_reasoning: "Accurately describes binary search and its complexity",
|
24
|
-
# technical_writer_score: 90,
|
25
|
-
# technical_writer_reasoning: "Clear and concise explanation of the algorithm",
|
26
|
-
# final_score: 92.5
|
27
|
-
# }
|
28
|
-
```
|
29
|
-
|
30
|
-
## Automatic Reviewer Selection
|
31
|
-
|
32
|
-
When no reviewers are specified, the system automatically recommends appropriate reviewers:
|
33
|
-
|
34
|
-
```ruby
|
35
|
-
text = "The patient shows signs of improved cardiac function"
|
36
|
-
criteria = "Evaluate medical accuracy and clarity"
|
37
|
-
|
38
|
-
result = ActiveGenie::Scoring.call(text, criteria)
|
39
|
-
# => {
|
40
|
-
# cardiologist_score: 88,
|
41
|
-
# cardiologist_reasoning: "Accurate assessment of cardiac improvement",
|
42
|
-
# medical_writer_score: 85,
|
43
|
-
# medical_writer_reasoning: "Clear communication of medical findings",
|
44
|
-
# general_practitioner_score: 90,
|
45
|
-
# general_practitioner_reasoning: "Well-structured medical observation",
|
46
|
-
# final_score: 87.7
|
47
|
-
# }
|
48
|
-
```
|
49
|
-
|
50
|
-
## Interface
|
51
|
-
|
52
|
-
### `.call(text, criteria, reviewers = [], config: {})`
|
53
|
-
Main interface for scoring text content.
|
54
|
-
|
55
|
-
#### Parameters
|
56
|
-
- `text` [String] - The text content to be evaluated
|
57
|
-
- `criteria` [String] - The evaluation criteria or rubric to assess against
|
58
|
-
- `reviewers` [Array<String>] - Optional list of specific reviewers
|
59
|
-
- `config` [Hash] - Additional configuration config
|
60
|
-
|
61
|
-
### `RecommendedReviewers.call(text, criteria, config: {})`
|
62
|
-
Recommends appropriate reviewers based on content and criteria.
|
63
|
-
|
64
|
-
#### Parameters
|
65
|
-
- `text` [String] - The text content to analyze
|
66
|
-
- `criteria` [String] - The evaluation criteria
|
67
|
-
- `config` [Hash] - Additional configuration config
|
68
|
-
|
69
|
-
### Usage Notes
|
70
|
-
- Best suited for objective evaluation of text content
|
71
|
-
- Provides balanced scoring through multiple reviewers
|
72
|
-
- Automatically handles reviewer selection when needed
|
73
|
-
- Supports custom weighting of reviewer scores
|
74
|
-
- Returns detailed reasoning for each score
|
75
|
-
|
76
|
-
Performance Impact: Using multiple reviewers or requesting detailed feedback may increase processing time.
|
File without changes
|
File without changes
|
File without changes
|