active_genie 0.0.8 → 0.0.10

Sign up to get free protection for your applications and to get access to all the features.
Files changed (39) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +34 -33
  3. data/VERSION +1 -1
  4. data/lib/active_genie/battle/README.md +2 -2
  5. data/lib/active_genie/battle/basic.rb +24 -19
  6. data/lib/active_genie/battle.rb +1 -1
  7. data/lib/active_genie/clients/openai_client.rb +77 -0
  8. data/lib/active_genie/clients/unified_client.rb +19 -0
  9. data/lib/active_genie/configuration/log_config.rb +14 -0
  10. data/lib/active_genie/configuration/openai_config.rb +56 -0
  11. data/lib/active_genie/configuration/providers_config.rb +37 -0
  12. data/lib/active_genie/configuration.rb +18 -23
  13. data/lib/active_genie/data_extractor/README.md +4 -4
  14. data/lib/active_genie/data_extractor/basic.rb +19 -9
  15. data/lib/active_genie/data_extractor/from_informal.rb +18 -7
  16. data/lib/active_genie/data_extractor.rb +1 -1
  17. data/lib/active_genie/league/README.md +43 -0
  18. data/lib/active_genie/{leaderboard → league}/elo_ranking.rb +41 -8
  19. data/lib/active_genie/league/free_for_all.rb +62 -0
  20. data/lib/active_genie/league/league.rb +120 -0
  21. data/lib/active_genie/{leaderboard → league}/player.rb +17 -10
  22. data/lib/active_genie/league.rb +12 -0
  23. data/lib/active_genie/logger.rb +45 -0
  24. data/lib/active_genie/scoring/README.md +4 -8
  25. data/lib/active_genie/scoring/basic.rb +19 -10
  26. data/lib/active_genie/scoring/recommended_reviews.rb +7 -9
  27. data/lib/active_genie/scoring.rb +1 -1
  28. data/lib/active_genie.rb +9 -17
  29. data/lib/tasks/install.rake +3 -3
  30. data/lib/tasks/templates/active_genie.rb +17 -0
  31. metadata +85 -80
  32. data/lib/active_genie/clients/openai.rb +0 -61
  33. data/lib/active_genie/clients/router.rb +0 -41
  34. data/lib/active_genie/leaderboard/leaderboard.rb +0 -72
  35. data/lib/active_genie/leaderboard/league.rb +0 -48
  36. data/lib/active_genie/leaderboard.rb +0 -11
  37. data/lib/active_genie/utils/math.rb +0 -15
  38. data/lib/tasks/templates/active_genie.yml +0 -7
  39. /data/lib/active_genie/{leaderboard → league}/players_collection.rb +0 -0
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: active_genie
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.8
4
+ version: 0.0.10
5
5
  platform: ruby
6
6
  authors:
7
7
  - Radamés Roriz
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2025-02-06 00:00:00.000000000 Z
11
+ date: 2025-02-10 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description: "# ActiveGenie \U0001F9DE‍♂️\n> Transform your Ruby application with
14
14
  powerful, production-ready GenAI features\n\n[![Gem Version](https://badge.fury.io/rb/active_genie.svg?icon=si%3Arubygems)](https://badge.fury.io/rb/active_genie)\n[![Ruby](https://github.com/roriz/active_genie/actions/workflows/ruby.yml/badge.svg)](https://github.com/roriz/active_genie/actions/workflows/ruby.yml)\n\nActiveGenie
@@ -17,47 +17,44 @@ description: "# ActiveGenie \U0001F9DE‍♂️\n> Transform your Ruby applicati
17
17
  Rails, ActiveGenie makes it effortless to integrate GenAI capabilities into your
18
18
  Ruby applications.\n\n## Features\n\n- \U0001F3AF **Data Extraction**: Extract structured
19
19
  data from unstructured text with type validation\n- \U0001F4CA **Smart Scoring**:
20
- Multi-reviewer evaluation system with automatic expert selection\n- \U0001F4AD **Sentiment
21
- Analysis**: Advanced sentiment analysis with customizable rules\n- \U0001F512 **Safe
22
- & Secure**: Built-in validation and sanitization\n- \U0001F6E0️ **Configurable**:
23
- Supports multiple GenAI providers and models\n\n## Installation\n\n1. Add to your
24
- Gemfile:\n```ruby\ngem 'active_genie'\n```\n\n2. Install the gem:\n```shell\nbundle
25
- install\n```\n\n3. Generate the configuration:\n```shell\necho \"ActiveGenie.load_tasks\"
26
- >> Rakefile\nrails g active_genie:install\n```\n\n4. [Optional] Configure your credentials
27
- in `config/active_genie.yml`:\n```yaml\nGPT-4o-mini:\n api_key: <%= ENV['OPENAI_API_KEY']
28
- %>\n provider: \"openai\"\n\nclaude-3-5-sonnet:\n api_key: <%= ENV['ANTHROPIC_API_KEY']
29
- %>\n provider: \"anthropic\"\n```\n\n> The first key will be used as default in
30
- all modules, in this example `GPT-4o-mini`\n\n## Quick Start\n\n### Data Extractor\nExtract
31
- structured data from text using AI-powered analysis, handling informal language
32
- and complex expressions.\n\n```ruby\ntext = \"Nike Air Max 90 - Size 42 - $199.99\"\nschema
33
- = {\n brand: { \n type: 'string',\n enum: [\"Nike\", \"Adidas\", \"Puma\"]\n
34
- \ },\n price: { \n type: 'number',\n minimum: 0\n },\n size: {\n type:
35
- 'integer',\n minimum: 35,\n maximum: 46\n }\n}\n\nresult = ActiveGenie::DataExtractor.call(text,
36
- schema)\n# => { \n# brand: \"Nike\", \n# brand_explanation: \"Brand name
37
- found at start of text\",\n# price: 199.99,\n# price_explanation: \"Price
38
- found in USD format at end\",\n# size: 42,\n# size_explanation: \"Size
39
- explicitly stated in the middle\"\n# }\n```\n\nFeatures:\n- Structured data extraction
40
- with type validation\n- Schema-based extraction with custom constraints\n- Informal
41
- text analysis (litotes, hedging)\n- Detailed explanations for extracted values\n\nSee
42
- the [Data Extractor README](lib/active_genie/data_extractor/README.md) for informal
43
- text processing, advanced schemas, and detailed interface documentation.\n\n###
44
- Scoring\nText evaluation system that provides detailed scoring and feedback using
45
- multiple expert reviewers. Get balanced scoring through AI-powered expert reviewers
46
- that automatically adapt to your content.\n\n```ruby\ntext = \"The code implements
47
- a binary search algorithm with O(log n) complexity\"\ncriteria = \"Evaluate technical
48
- accuracy and clarity\"\n\nresult = ActiveGenie::Scoring::Basic.call(text, criteria)\n#
49
- => {\n# algorithm_expert_score: 95,\n# algorithm_expert_reasoning: \"Accurately
50
- describes binary search and its complexity\",\n# technical_writer_score: 90,\n#
51
- \ technical_writer_reasoning: \"Clear and concise explanation of the algorithm\",\n#
52
- \ final_score: 92.5\n# }\n```\n\nFeatures:\n- Multi-reviewer evaluation with
53
- automatic expert selection\n- Detailed feedback with scoring reasoning\n- Customizable
54
- reviewer weights\n- Flexible evaluation criteria\n\nSee the [Scoring README](lib/active_genie/scoring/README.md)
20
+ Multi-reviewer evaluation system with automatic expert selection\n- \U0001F4AD **Leaderboard**:
21
+ Consistent rank items based on custom criteria, using multiple tecniques of ranking\n\n##
22
+ Installation\n\n1. Add to your Gemfile:\n```ruby\ngem 'active_genie'\n```\n\n2.
23
+ Install the gem:\n```shell\nbundle install\n```\n\n3. Generate the configuration:\n```shell\necho
24
+ \"ActiveGenie.load_tasks\" >> Rakefile\nrails g active_genie:install\n```\n\n4.
25
+ Configure your credentials in `config/initializers/active_genie.rb`:\n```ruby\nActiveGenie.configure
26
+ do |config|\n config.openai.api_key = ENV['OPENAI_API_KEY']\nend\n```\n\n## Quick
27
+ Start\n\n### Data Extractor\nExtract structured data from text using AI-powered
28
+ analysis, handling informal language and complex expressions.\n\n```ruby\ntext =
29
+ \"Nike Air Max 90 - Size 42 - $199.99\"\nschema = {\n brand: { \n type: 'string',\n
30
+ \ enum: [\"Nike\", \"Adidas\", \"Puma\"]\n },\n price: { \n type: 'number',\n
31
+ \ minimum: 0\n },\n size: {\n type: 'integer',\n minimum: 35,\n maximum:
32
+ 46\n }\n}\n\nresult = ActiveGenie::DataExtractor.call(text, schema)\n# => { \n#
33
+ \ brand: \"Nike\", \n# brand_explanation: \"Brand name found at start of
34
+ text\",\n# price: 199.99,\n# price_explanation: \"Price found in USD format
35
+ at end\",\n# size: 42,\n# size_explanation: \"Size explicitly stated in
36
+ the middle\"\n# }\n```\n\nFeatures:\n- Structured data extraction with type validation\n-
37
+ Schema-based extraction with custom constraints\n- Informal text analysis (litotes,
38
+ hedging)\n- Detailed explanations for extracted values\n\nSee the [Data Extractor
39
+ README](lib/active_genie/data_extractor/README.md) for informal text processing,
40
+ advanced schemas, and detailed interface documentation.\n\n### Scoring\nText evaluation
41
+ system that provides detailed scoring and feedback using multiple expert reviewers.
42
+ Get balanced scoring through AI-powered expert reviewers that automatically adapt
43
+ to your content.\n\n```ruby\ntext = \"The code implements a binary search algorithm
44
+ with O(log n) complexity\"\ncriteria = \"Evaluate technical accuracy and clarity\"\n\nresult
45
+ = ActiveGenie::Scoring.basic(text, criteria)\n# => {\n# algorithm_expert_score:
46
+ 95,\n# algorithm_expert_reasoning: \"Accurately describes binary search and
47
+ its complexity\",\n# technical_writer_score: 90,\n# technical_writer_reasoning:
48
+ \"Clear and concise explanation of the algorithm\",\n# final_score: 92.5\n#
49
+ \ }\n```\n\nFeatures:\n- Multi-reviewer evaluation with automatic expert selection\n-
50
+ Detailed feedback with scoring reasoning\n- Customizable reviewer weights\n- Flexible
51
+ evaluation criteria\n\nSee the [Scoring README](lib/active_genie/scoring/README.md)
55
52
  for advanced usage, custom reviewers, and detailed interface documentation.\n\n###
56
53
  Battle\nAI-powered battle evaluation system that determines winners between two
57
54
  players based on specified criteria.\n\n```ruby\nrequire 'active_genie'\n\nplayer_a
58
55
  = \"Implementation uses dependency injection for better testability\"\nplayer_b
59
56
  = \"Code has high test coverage but tightly coupled components\"\ncriteria = \"Evaluate
60
- code quality and maintainability\"\n\nresult = ActiveGenie::Battle::Basic.call(player_a,
57
+ code quality and maintainability\"\n\nresult = ActiveGenie::Battle.call(player_a,
61
58
  player_b, criteria)\n# => {\n# winner_player: \"Implementation uses dependency
62
59
  injection for better testability\",\n# reasoning: \"Player A's implementation
63
60
  demonstrates better maintainability through dependency injection, \n# which
@@ -68,46 +65,50 @@ description: "# ActiveGenie \U0001F9DE‍♂️\n> Transform your Ruby applicati
68
65
  evaluation with automatic expert selection\n- Detailed feedback with scoring reasoning\n-
69
66
  Customizable reviewer weights\n- Flexible evaluation criteria\n\nSee the [Battle
70
67
  README](lib/active_genie/battle/README.md) for advanced usage, custom reviewers,
71
- and detailed interface documentation.\n\n### Summarizer (WIP)\nThe summarizer is
72
- a tool that can be used to summarize a given text. It uses a set of rules to summarize
73
- the text out of the box. Uses the best practices of prompt engineering and engineering
74
- to make the summarization as accurate as possible.\n\n```ruby\nrequire 'active_genie'\n\ntext
75
- = \"Example text to be summarized. The fox jumps over the dog\"\nsummarized_text
76
- = ActiveGenie::Summarizer.call(text)\nputs summarized_text # => \"The fox jumps
77
- over the dog\"\n```\n\n### Language detector (WIP)\nThe language detector is a tool
78
- that can be used to detect the language of a given text. It uses a set of rules
79
- to detect the language of the text out of the box. Uses the best practices of prompt
80
- engineering and engineering to make the language detection as accurate as possible.\n\n```ruby\nrequire
81
- 'active_genie'\n\ntext = \"Example text to be detected\"\nlanguage = ActiveGenie::LanguageDetector.call(text)\nputs
82
- language # => \"en\"\n```\n\n### Translator (WIP)\nThe translator is a tool that
83
- can be used to translate a given text. It uses a set of rules to translate the text
84
- out of the box. Uses the best practices of prompt engineering and engineering to
85
- make the translation as accurate as possible.\n\n```ruby\nrequire 'active_genie'\n\ntext
86
- = \"Example text to be translated\"\ntranslated_text = ActiveGenie::Translator.call(text,
87
- from: 'en', to: 'pt')\nputs translated_text # => \"Exemplo de texto a ser traduzido\"\n```\n\n###
68
+ and detailed interface documentation.\n\n### League\nThe League module provides
69
+ competitive ranking through multi-stage evaluation:\n\n\n```ruby\nrequire 'active_genie'\n\nplayers
70
+ = ['REST API', 'GraphQL API', 'SOAP API', 'gRPC API', 'Websocket API']\ncriteria
71
+ = \"Best one to be used into a high changing environment\"\n\nresult = ActiveGenie::League.call(players,
72
+ criteria)\n# => {\n# winner_player: \"gRPC API\",\n# reasoning: \"gRPC
73
+ API is the best one to be used into a high changing environment\",\n# }\n```\n\n-
74
+ **Multi-phase ranking system** combining expert scoring and ELO algorithms\n- **Automatic
75
+ elimination** of inconsistent performers using statistical analysis\n- **Dynamic
76
+ ranking adjustments** based on simulated pairwise battles, from bottom to top\n\nSee
77
+ the [League README](lib/active_genie/league/README.md) for implementation details,
78
+ configuration, and advanced ranking strategies.\n\n### Summarizer (WIP)\nThe summarizer
79
+ is a tool that can be used to summarize a given text. It uses a set of rules to
80
+ summarize the text out of the box. Uses the best practices of prompt engineering
81
+ and engineering to make the summarization as accurate as possible.\n\n```ruby\nrequire
82
+ 'active_genie'\n\ntext = \"Example text to be summarized. The fox jumps over the
83
+ dog\"\nsummarized_text = ActiveGenie::Summarizer.call(text)\nputs summarized_text
84
+ # => \"The fox jumps over the dog\"\n```\n\n### Language detector (WIP)\nThe language
85
+ detector is a tool that can be used to detect the language of a given text. It uses
86
+ a set of rules to detect the language of the text out of the box. Uses the best
87
+ practices of prompt engineering and engineering to make the language detection as
88
+ accurate as possible.\n\n```ruby\nrequire 'active_genie'\n\ntext = \"Example text
89
+ to be detected\"\nlanguage = ActiveGenie::LanguageDetector.call(text)\nputs language
90
+ # => \"en\"\n```\n\n### Translator (WIP)\nThe translator is a tool that can be used
91
+ to translate a given text. It uses a set of rules to translate the text out of the
92
+ box. Uses the best practices of prompt engineering and engineering to make the translation
93
+ as accurate as possible.\n\n```ruby\nrequire 'active_genie'\n\ntext = \"Example
94
+ text to be translated\"\ntranslated_text = ActiveGenie::Translator.call(text, from:
95
+ 'en', to: 'pt')\nputs translated_text # => \"Exemplo de texto a ser traduzido\"\n```\n\n###
88
96
  Sentiment analyzer (WIP)\nThe sentiment analyzer is a tool that can be used to analyze
89
97
  the sentiment of a given text. It uses a set of rules to analyze the sentiment of
90
98
  the text out of the box. Uses the best practices of prompt engineering and engineering
91
99
  to make the sentiment analysis as accurate as possible.\n\n```ruby\nrequire 'active_genie'\n\ntext
92
100
  = \"Example text to be analyzed\"\nsentiment = ActiveGenie::SentimentAnalyzer.call(text)\nputs
93
- sentiment # => \"positive\"\n```\n\n### Elo ranking (WIP)\nThe Elo ranking is a
94
- tool that can be used to rank a set of items. It uses a set of rules to rank the
95
- items out of the box. Uses the best practices of prompt engineering and engineering
96
- to make the ranking as accurate as possible.\n\n```ruby\nrequire 'active_genie'\n\nitems
97
- = ['Square', 'Circle', 'Triangle']\ncriterias = 'items that look rounded'\nranked_items
98
- = ActiveGenie::EloRanking.call(items, criterias, rounds: 10)\nputs ranked_items
99
- # => [{ name: \"Circle\", score: 1500 }, { name: \"Square\", score: 800 }, { name:
100
- \"Triangle\", score: 800 }]\n```\n\n\n## Configuration Options\n\n| Option | Description
101
- | Default |\n|--------|-------------|---------|\n| `provider` | LLM provider (openai,
101
+ sentiment # => \"positive\"\n```\n\n## Configuration\n\n| Config | Description |
102
+ Default |\n|--------|-------------|---------|\n| `provider` | LLM provider (openai,
102
103
  anthropic, etc) | `nil` |\n| `model` | Model to use | `nil` |\n| `api_key` | Provider
103
104
  API key | `nil` |\n| `timeout` | Request timeout in seconds | `5` |\n| `max_retries`
104
105
  | Maximum retry attempts | `3` |\n\n> **Note:** Each module can append its own set
105
- of configuration options, see the individual module documentation for details.\n\n##
106
- Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout
107
- -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add amazing
108
- feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5.
109
- Open a Pull Request\n## License\n\nThis project is licensed under the MIT License
110
- - see the [LICENSE](LICENSE) file for details.\n\n"
106
+ of configuration, see the individual module documentation for details.\n\n## Contributing\n\n1.
107
+ Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3.
108
+ Commit your changes (`git commit -m 'Add amazing feature'`)\n4. Push to the branch
109
+ (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n## License\n\nThis
110
+ project is licensed under the MIT License - see the [LICENSE](LICENSE) file for
111
+ details.\n"
111
112
  email:
112
113
  - radames@roriz.dev
113
114
  executables: []
@@ -121,26 +122,30 @@ files:
121
122
  - lib/active_genie/battle.rb
122
123
  - lib/active_genie/battle/README.md
123
124
  - lib/active_genie/battle/basic.rb
124
- - lib/active_genie/clients/openai.rb
125
- - lib/active_genie/clients/router.rb
125
+ - lib/active_genie/clients/openai_client.rb
126
+ - lib/active_genie/clients/unified_client.rb
126
127
  - lib/active_genie/configuration.rb
128
+ - lib/active_genie/configuration/log_config.rb
129
+ - lib/active_genie/configuration/openai_config.rb
130
+ - lib/active_genie/configuration/providers_config.rb
127
131
  - lib/active_genie/data_extractor.rb
128
132
  - lib/active_genie/data_extractor/README.md
129
133
  - lib/active_genie/data_extractor/basic.rb
130
134
  - lib/active_genie/data_extractor/from_informal.rb
131
- - lib/active_genie/leaderboard.rb
132
- - lib/active_genie/leaderboard/elo_ranking.rb
133
- - lib/active_genie/leaderboard/leaderboard.rb
134
- - lib/active_genie/leaderboard/league.rb
135
- - lib/active_genie/leaderboard/player.rb
136
- - lib/active_genie/leaderboard/players_collection.rb
135
+ - lib/active_genie/league.rb
136
+ - lib/active_genie/league/README.md
137
+ - lib/active_genie/league/elo_ranking.rb
138
+ - lib/active_genie/league/free_for_all.rb
139
+ - lib/active_genie/league/league.rb
140
+ - lib/active_genie/league/player.rb
141
+ - lib/active_genie/league/players_collection.rb
142
+ - lib/active_genie/logger.rb
137
143
  - lib/active_genie/scoring.rb
138
144
  - lib/active_genie/scoring/README.md
139
145
  - lib/active_genie/scoring/basic.rb
140
146
  - lib/active_genie/scoring/recommended_reviews.rb
141
- - lib/active_genie/utils/math.rb
142
147
  - lib/tasks/install.rake
143
- - lib/tasks/templates/active_genie.yml
148
+ - lib/tasks/templates/active_genie.rb
144
149
  homepage: https://github.com/Roriz/active_genie
145
150
  licenses:
146
151
  - Apache-2.0
@@ -1,61 +0,0 @@
1
- require 'json'
2
- require 'net/http'
3
-
4
- module ActiveGenie::Clients
5
- class Openai
6
- class << self
7
- def function_calling(messages, function, options: {})
8
- app_config = ActiveGenie.config_by_model(options[:model])
9
-
10
- model = options[:model] || app_config[:model]
11
-
12
- raise "Model can't be blank" if model.nil?
13
-
14
- payload = {
15
- messages:,
16
- response_format: {
17
- type: 'json_schema',
18
- json_schema: function
19
- },
20
- model:,
21
- }
22
-
23
- api_key = options[:api_key] || app_config[:api_key]
24
-
25
- headers = DEFAULT_HEADERS.merge(
26
- 'Authorization': "Bearer #{api_key}"
27
- ).compact
28
-
29
- response = request(payload, headers)
30
-
31
- parsed_response = JSON.parse(response.dig('choices', 0, 'message', 'content'))
32
-
33
- parsed_response.dig('properties') || parsed_response
34
- rescue JSON::ParserError
35
- nil
36
- end
37
-
38
- def request(payload, headers)
39
- response = Net::HTTP.post(
40
- URI(API_URL),
41
- payload.to_json,
42
- headers
43
- )
44
-
45
- raise OpenaiError, response.body unless response.is_a?(Net::HTTPSuccess)
46
- return nil if response.body.empty?
47
-
48
- JSON.parse(response.body)
49
- end
50
-
51
- end
52
-
53
- API_URL = 'https://api.openai.com/v1/chat/completions'.freeze
54
- DEFAULT_HEADERS = {
55
- 'Content-Type': 'application/json',
56
- }
57
-
58
- # TODO: add some more rich error handling
59
- class OpenaiError < StandardError; end
60
- end
61
- end
@@ -1,41 +0,0 @@
1
- require_relative './openai'
2
-
3
- module ActiveGenie::Clients
4
- class Router
5
- class << self
6
- def function_calling(messages, function, options: {})
7
- app_config = ActiveGenie.config_by_model(options[:model])
8
-
9
- provider = options[:provider] || app_config[:provider]
10
- client = PROVIDER_TO_CLIENT[provider&.downcase&.strip&.to_sym]
11
- raise "Provider \"#{provider}\" not supported" unless client
12
-
13
- response = client.function_calling(messages, function, options:)
14
-
15
- clear_invalid_values(response)
16
- end
17
-
18
- private
19
-
20
- PROVIDER_TO_CLIENT = {
21
- openai: Openai,
22
- }
23
-
24
- INVALID_VALUES = [
25
- 'not sure',
26
- 'not clear',
27
- 'not specified',
28
- 'none',
29
- 'null',
30
- 'undefined',
31
- ].freeze
32
-
33
- def clear_invalid_values(data)
34
- data.reduce({}) do |acc, (field, value)|
35
- acc[field] = value unless INVALID_VALUES.include?(value)
36
- acc
37
- end
38
- end
39
- end
40
- end
41
- end
@@ -1,72 +0,0 @@
1
- require_relative './players_collection'
2
- require_relative './league'
3
- require_relative './elo_ranking'
4
- require_relative '../scoring/recommended_reviews'
5
-
6
- module ActiveGenie::Leaderboard
7
- class Leaderboard
8
- def self.call(param_players, criteria, options: {})
9
- new(param_players, criteria, options:).call
10
- end
11
-
12
- def initialize(param_players, criteria, options: {})
13
- @param_players = param_players
14
- @criteria = criteria
15
- @options = options
16
- end
17
-
18
- def call
19
- set_initial_score_players
20
- eliminate_obvious_bad_players
21
- run_elo_ranking if players.eligible_size > 10
22
- run_league
23
-
24
- players.to_h
25
- end
26
-
27
- private
28
-
29
- SCORE_VARIATION_THRESHOLD = 10
30
- MATCHS_PER_PLAYER = 3
31
-
32
- def set_initial_score_players
33
- players.each do |player|
34
- player.score = generate_score(player.content) # This can take a while, can be parallelized
35
- end
36
- end
37
-
38
- def generate_score(content)
39
- ActiveGenie::Scoring::Basic.call(content, @criteria, reviewers, options: @options)['final_score']
40
- end
41
-
42
- def eliminate_obvious_bad_players
43
- while players.coefficient_of_variation >= SCORE_VARIATION_THRESHOLD
44
- players.eligible.last.eliminated = 'too_low_score'
45
- end
46
- end
47
-
48
- def run_elo_ranking
49
- EloRanking.call(players, @criteria, options: @options)
50
- end
51
-
52
- def run_league
53
- League.call(players, @criteria, options: @options)
54
- end
55
-
56
- def reviewers
57
- [recommended_reviews['reviewer1'], recommended_reviews['reviewer2'], recommended_reviews['reviewer3']]
58
- end
59
-
60
- def recommended_reviews
61
- @recommended_reviews ||= ActiveGenie::Scoring::RecommendedReviews.call(
62
- players.sample,
63
- @criteria,
64
- options: @options
65
- )
66
- end
67
-
68
- def players
69
- @players ||= PlayersCollection.new(@param_players)
70
- end
71
- end
72
- end
@@ -1,48 +0,0 @@
1
- require_relative '../battle/basic'
2
-
3
- module ActiveGenie::Leaderboard
4
- class League
5
- def self.call(players, criteria, options: {})
6
- new(players, criteria, options:).call
7
- end
8
-
9
- def initialize(players, criteria, options: {})
10
- @players = players
11
- @criteria = criteria
12
- @options = options
13
- end
14
-
15
- def call
16
- matches.each do |player_a, player_b|
17
- winner, loser = battle(player_a, player_b)
18
-
19
- if winner.nil? || loser.nil?
20
- player_a.league[:draw] += 1
21
- player_b.league[:draw] += 1
22
- else
23
- winner.league[:win] += 1
24
- loser.league[:lose] += 1
25
- end
26
- end
27
-
28
- @players
29
- end
30
-
31
- private
32
-
33
- # TODO: reduce the number of matches based on transitivity.
34
- # For example, if A is better than B, and B is better than C, then A should clearly be better than C
35
- def matches
36
- @players.eligible.combination(2).to_a
37
- end
38
-
39
- def battle(player_a, player_b)
40
- ActiveGenie::Battle.basic(
41
- player_a,
42
- player_b,
43
- @criteria,
44
- options: @options
45
- ).values_at('winner', 'loser')
46
- end
47
- end
48
- end
@@ -1,11 +0,0 @@
1
- require_relative 'leaderboard/leaderboard'
2
-
3
- module ActiveGenie
4
- module Leaderboard
5
- module_function
6
-
7
- def call(...)
8
- Leaderboard.call(...)
9
- end
10
- end
11
- end
@@ -1,15 +0,0 @@
1
- module ActiveGenie::Utils
2
- module Math
3
- module_function
4
-
5
- def self.calculate_new_elo(winner, loser, k: 32)
6
- expected_score_a = 1 / (1 + 10**((loser - winner) / 400))
7
- expected_score_b = 1 - expected_score_a
8
-
9
- new_elo_winner = winner + k * (1 - expected_score_a)
10
- new_elo_loser = loser + k * (1 - expected_score_b)
11
-
12
- [new_elo_winner, new_elo_loser]
13
- end
14
- end
15
- end
@@ -1,7 +0,0 @@
1
- # gpt-4o-mini:
2
- # api_key: <%= ENV['OPENAI_API_KEY'] %>
3
- # provider: 'openai'
4
- #
5
- # claude-3-5-sonnet:
6
- # api_key: <%= ENV['ANTHROPIC_API_KEY'] %>
7
- # provider: 'anthropic'