active_genie 0.26.5 → 0.27.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +10 -193
  3. data/VERSION +1 -1
  4. data/lib/active_genie/battle/generalist.rb +9 -9
  5. data/lib/active_genie/clients/providers/anthropic_client.rb +2 -2
  6. data/lib/active_genie/clients/providers/base_client.rb +2 -2
  7. data/lib/active_genie/clients/providers/deepseek_client.rb +2 -2
  8. data/lib/active_genie/clients/providers/google_client.rb +3 -10
  9. data/lib/active_genie/clients/providers/openai_client.rb +2 -2
  10. data/lib/active_genie/config/log_config.rb +12 -15
  11. data/lib/active_genie/config/providers/anthropic_config.rb +2 -2
  12. data/lib/active_genie/config/providers/deepseek_config.rb +2 -2
  13. data/lib/active_genie/config/providers/google_config.rb +2 -2
  14. data/lib/active_genie/config/providers/openai_config.rb +2 -2
  15. data/lib/active_genie/config/providers/provider_base.rb +4 -4
  16. data/lib/active_genie/configuration.rb +16 -4
  17. data/lib/active_genie/data_extractor/generalist.rb +2 -2
  18. data/lib/active_genie/logger.rb +40 -22
  19. data/lib/active_genie/ranking/elo_round.rb +30 -37
  20. data/lib/active_genie/ranking/free_for_all.rb +29 -28
  21. data/lib/active_genie/ranking/player.rb +1 -9
  22. data/lib/active_genie/ranking/ranking.rb +32 -25
  23. data/lib/active_genie/ranking/ranking_scoring.rb +12 -17
  24. data/lib/active_genie/scoring/generalist.rb +9 -9
  25. data/lib/tasks/templates/active_genie.rb +1 -1
  26. metadata +8 -13
  27. data/lib/active_genie/data_extractor/README.md +0 -131
  28. data/lib/active_genie/ranking/README.md +0 -73
  29. data/lib/active_genie/ranking/concerns/loggable.rb +0 -29
  30. data/lib/active_genie/scoring/README.md +0 -76
  31. /data/lib/active_genie/battle/{generalist.md → generalist.prompt.md} +0 -0
  32. /data/lib/active_genie/data_extractor/{generalist.md → generalist.prompt.md} +0 -0
  33. /data/lib/active_genie/scoring/{generalist.md → generalist.prompt.md} +0 -0
@@ -1,131 +0,0 @@
1
- # Data Extractor
2
- Extract structured data from text using AI-powered analysis, handling informal language and complex expressions.
3
-
4
- ## ✨ Features
5
- - Structured data extraction - Extract typed data from unstructured text using predefined schemas
6
- - Informal text analysis - Identifies and handles informal language patterns, including litotes
7
- - Explanation tracking - Provides reasoning for each extracted data point
8
-
9
- ## Basic Usage
10
-
11
- Extract structured data from text using predefined schemas:
12
-
13
- ```ruby
14
- text = "John Doe is 25 years old"
15
- schema = {
16
- name: { type: 'string', description: 'Full name of the person' },
17
- age: { type: 'integer', description: 'Age in years' }
18
- }
19
- result = ActiveGenie::DataExtractor.call(text, schema)
20
- # => {
21
- # name: "John Doe",
22
- # name_explanation: "Found directly in text",
23
- # age: 25,
24
- # age_explanation: "Explicitly stated as 25 years old"
25
- # }
26
-
27
- product = "Nike Air Max 90 - Size 42 - $199.99"
28
- schema = {
29
- brand: {
30
- type: 'string',
31
- enum: ["Nike", "Adidas", "Puma"]
32
- },
33
- price: {
34
- type: 'number',
35
- minimum: 0
36
- },
37
- currency: {
38
- type: 'string',
39
- enum: ["USD", "EUR"]
40
- },
41
- size: {
42
- type: 'integer',
43
- minimum: 35,
44
- maximum: 46
45
- }
46
- }
47
-
48
- result = ActiveGenie::DataExtractor.call(product, schema)
49
- # => {
50
- # brand: "Nike",
51
- # brand_explanation: "Brand name found at start of text",
52
- # price: 199.99,
53
- # price_explanation: "Price found in USD format at end",
54
- # size: 42,
55
- # size_explanation: "Size explicitly stated in the middle",
56
- # currency: "USD",
57
- # currency_explanation: "Derived from $ symbol"
58
- # }
59
- ```
60
-
61
- ## Informal Text Processing
62
-
63
- The `from_informal` method extends the basic extraction by analyzing rhetorical devices and informal language patterns like:
64
-
65
- - Litotes ("not bad", "isn't terrible")
66
- - Affirmative expressions ("sure", "no problem")
67
- - Negative expressions ("nah", "not really")
68
-
69
- ### Example
70
-
71
- ```ruby
72
- text = "The weather isn't bad today"
73
- schema = {
74
- mood: { type: 'string', description: 'The mood of the message' }
75
- }
76
-
77
- result = ActiveGenie::DataExtractor.from_informal(text, schema)
78
- # => {
79
- # mood: "positive",
80
- # mood_explanation: "Speaker views weather favorably",
81
- # message_litote: true,
82
- # litote_rephrased: "The weather is good today"
83
- # }
84
- ```
85
-
86
- ### Usage Notes
87
- - Best suited for processing conversational user inputs
88
- - Automatically detects and interprets litotes
89
- - Provides rephrased positive statements for litotes
90
- - May require more processing time due to rhetorical analysis
91
- - Accuracy depends on context clarity
92
-
93
- ⚠️ Performance Impact: This method performs additional rhetorical analysis, which can increase processing time.
94
-
95
- ## Interface
96
-
97
- ### `.call(text, data_to_extract, config = {})`
98
- Extracts structured data from text based on a predefined schema.
99
-
100
- #### Parameters
101
- | Name | Type | Description | Required | Example |
102
- | --- | --- | --- | --- | --- |
103
- | `text` | `String` | The text to analyze and extract data from | Yes | "John Doe is 25 years old" |
104
- | `data_to_extract` | `Hash` | Schema defining the data structure to extract | Yes | `{ name: { type: 'string' } }` |
105
- | `config` | `Hash` | Additional extraction configuration | No | `{ model: "gpt-4" }` |
106
-
107
- #### config
108
- | Name | Type | Description |
109
- | --- | --- | --- |
110
- | `model` | `String` | The model to use for the extraction |
111
- | `api_key` | `String` | The API key to use for the extraction |
112
-
113
- #### Returns
114
- `Hash` containing:
115
- - Extracted values matching the schema structure
116
- - Explanation field for each extracted value
117
- - Additional analysis fields when using `from_informal`
118
-
119
- ### `.from_informal(text, data_to_extract, config = {})`
120
- Extends extraction with rhetorical analysis, particularly for litotes.
121
-
122
- #### Additional Return Fields
123
- | Name | Type | Description |
124
- | --- | --- | --- |
125
- | `message_litote` | `Boolean` | Whether the text contains a litote |
126
- | `litote_rephrased` | `String` | Positive rephrasing of any detected litote |
127
-
128
- ⚠️ Performance Considerations
129
- - Both methods may require multiple AI model calls
130
- - Informal processing requires additional rhetorical analysis
131
- - Consider background processing for production use
@@ -1,73 +0,0 @@
1
- # Ranking
2
-
3
- The `ActiveGenie::Ranking` module organizes players based on scores derived from textual evaluations and then ranks them using a multi-stage process. It leverages the scoring system from the `ActiveGenie::Scoring` module to assign initial scores, and then applies advanced ranking methods to produce a competitive ranking.
4
-
5
- ## Overview
6
-
7
- The ranking system performs the following steps:
8
-
9
- 1. **Initial Scoring**: Each player’s textual content is evaluated using `ActiveGenie::Scoring`. This produces a `score` based on multiple expert reviews.
10
-
11
- 2. **Elimination of Poor Performers**: Players whose scores show a high coefficient of variation (indicating inconsistency) are progressively eliminated. This ensures that only competitive candidates continue in the ranking process.
12
-
13
- 3. **ELO Ranking**: If there are more than 10 eligible players, an ELO-based ranking is applied. Battles between players are simulated via `ActiveGenie::Battle`, and scores are updated using an ELO algorithm tailored to the ranking.
14
-
15
- 4. **Free for all Matches**: Finally, the remaining players engage in head-to-head matches where each unique pair competes. Match outcomes (wins, losses, draws) are recorded to finalize the rankings.
16
-
17
- ## Components
18
-
19
- - **ranking**: Orchestrates the entire ranking process. Initializes player scores, eliminates outliers, and coordinates ranking rounds.
20
-
21
- - **EloRanking**: Applies an ELO-based system to rank players through simulated battles. It updates players’ ELO scores based on match outcomes and predefined rules (including penalties for losses).
22
-
23
- - **Free for all**: Conducts complete pairwise matches among eligible players to record win/loss/draw statistics and refine the final standings.
24
-
25
- ## Usage
26
-
27
- Call the ranking using:
28
-
29
- ```ruby
30
- result = ActiveGenie::Ranking.call(players, criteria, config: {})
31
- ```
32
-
33
- - `players`: A collection of player instances, each containing textual content to be scored.
34
- - `criteria`: A string defining the evaluation criteria used by the scoring system.
35
- - `config`: A hash of additional parameters for customization (e.g., model, api_key).
36
-
37
- The method processes the players through scoring, elimination, and ranking phases, then returns a hash containing the player statistics and rankings.
38
-
39
- ## Possible improvements
40
- - Adjust initial criteria to ensure consistency
41
- - Adjust each player's content to ensure consistency
42
- - Support players with images or audio
43
- - Parallelize processing battles and scoring
44
-
45
- ## Ranking Configuration
46
-
47
- | Config | Description | Default |
48
- |--------|-------------|---------|
49
- | `score_variation_threshold` | Threshold for eliminating players with inconsistent scores | `30` |
50
-
51
- ## Ranking Callbacks
52
- Callbacks are optional and can be used to watch any changes in players, battles, or scoring.
53
-
54
- | Callback | Description |
55
- |--------|-------------|
56
- | `watch_players` | Callback to watch any changes in players |
57
- | `watch_battles` | Callback to watch any changes in battles |
58
- | `watch_scoring` | Callback to watch any changes in scoring |
59
-
60
- Example of callback usage:
61
-
62
- ```ruby
63
- result = ActiveGenie::Ranking.call(
64
- players,
65
- criteria,
66
- config: {
67
- watch_players: ->(player) { puts player },
68
- watch_battles: ->(battle) { puts battle },
69
- watch_scoring: ->(scoring) { puts scoring }
70
- }
71
- )
72
- ```
73
-
@@ -1,29 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module ActiveGenie
4
- module Concerns
5
- module Loggable
6
- def self.included(base)
7
- base.extend(ClassMethods)
8
- end
9
-
10
- module ClassMethods
11
- def call_with_log_context(context_method)
12
- original_method = instance_method(:call)
13
-
14
- define_method(:call) do |*args, **kwargs, &block|
15
- context = send(context_method, *args, **kwargs)
16
-
17
- ActiveGenie::Logger.with_context(context) do
18
- original_method.bind(self).call(*args, **kwargs, &block)
19
- end
20
- end
21
- end
22
-
23
- def logger(...)
24
- ActiveGenie::Logger.call(...)
25
- end
26
- end
27
- end
28
- end
29
- end
@@ -1,76 +0,0 @@
1
- # Scoring
2
- Text evaluation system that provides detailed scoring and feedback using multiple expert reviewers.
3
-
4
- ## Features
5
- - Multi-reviewer evaluation - Get scores and feedback from multiple AI-powered expert reviewers
6
- - Automatic reviewer selection - Smart recommendation of reviewers based on content and criteria
7
- - Detailed feedback - Comprehensive reasoning for each reviewer's score
8
- - Customizable weights - Adjust the importance of different reviewers' scores
9
- - Flexible criteria - Score text against any specified evaluation criteria
10
-
11
- ## Basic Usage
12
-
13
- Score text using predefined reviewers:
14
-
15
- ```ruby
16
- text = "The code implements a binary search algorithm with O(log n) complexity"
17
- criteria = "Evaluate technical accuracy and clarity"
18
- reviewers = ["Algorithm Expert", "Technical Writer"]
19
-
20
- result = ActiveGenie::Scoring.call(text, criteria, reviewers)
21
- # => {
22
- # algorithm_expert_score: 95,
23
- # algorithm_expert_reasoning: "Accurately describes binary search and its complexity",
24
- # technical_writer_score: 90,
25
- # technical_writer_reasoning: "Clear and concise explanation of the algorithm",
26
- # final_score: 92.5
27
- # }
28
- ```
29
-
30
- ## Automatic Reviewer Selection
31
-
32
- When no reviewers are specified, the system automatically recommends appropriate reviewers:
33
-
34
- ```ruby
35
- text = "The patient shows signs of improved cardiac function"
36
- criteria = "Evaluate medical accuracy and clarity"
37
-
38
- result = ActiveGenie::Scoring.call(text, criteria)
39
- # => {
40
- # cardiologist_score: 88,
41
- # cardiologist_reasoning: "Accurate assessment of cardiac improvement",
42
- # medical_writer_score: 85,
43
- # medical_writer_reasoning: "Clear communication of medical findings",
44
- # general_practitioner_score: 90,
45
- # general_practitioner_reasoning: "Well-structured medical observation",
46
- # final_score: 87.7
47
- # }
48
- ```
49
-
50
- ## Interface
51
-
52
- ### `.call(text, criteria, reviewers = [], config: {})`
53
- Main interface for scoring text content.
54
-
55
- #### Parameters
56
- - `text` [String] - The text content to be evaluated
57
- - `criteria` [String] - The evaluation criteria or rubric to assess against
58
- - `reviewers` [Array<String>] - Optional list of specific reviewers
59
- - `config` [Hash] - Additional configuration config
60
-
61
- ### `RecommendedReviewers.call(text, criteria, config: {})`
62
- Recommends appropriate reviewers based on content and criteria.
63
-
64
- #### Parameters
65
- - `text` [String] - The text content to analyze
66
- - `criteria` [String] - The evaluation criteria
67
- - `config` [Hash] - Additional configuration config
68
-
69
- ### Usage Notes
70
- - Best suited for objective evaluation of text content
71
- - Provides balanced scoring through multiple reviewers
72
- - Automatically handles reviewer selection when needed
73
- - Supports custom weighting of reviewer scores
74
- - Returns detailed reasoning for each score
75
-
76
- Performance Impact: Using multiple reviewers or requesting detailed feedback may increase processing time.