active_genie 0.0.3 → 0.0.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e7f834cd4da9f695c2a3f69b1d7d5516831afaa2d6830f89527b46e7e19bc6f9
-  data.tar.gz: 8e14a0011c2551b975105d2120506461fb5f0e9e33206a6960a9de544148016b
+  metadata.gz: 9d0424a39ba21d821cb2419730387e1b026c35b5e2e5dff9f6d615f3ec54e6a3
+  data.tar.gz: 17b460ccd1a689d0f8709af2b84f3cde65aa0075b76104ee1b4bb8b3b0ffc182
 SHA512:
-  metadata.gz: ed3662a287028dacf2f60bda458d0de38ad16d4666421e3b836780d9f2dbb469be935e0b6c3d1fff851f82282c4c4564bebc725354f2f213d0cf21e30ce5ce93
-  data.tar.gz: 50b0d666a061361322ec2bfb38ef0e49110d6b31be3c3b55e3873f6ca5389003d739f209c7a9e1b695e3901de703effe02344863fed80495c104e886b1d1fa3f
+  metadata.gz: ad98b2d5d063d0d1c4009e1a9f92d6d326ed948cbdee71317c94b6a4a0ee57042609c1f405ca7ce00d4beb321bc1e98ad722646349076929bcdc0a28da8b6b8b
+  data.tar.gz: d2cc39b77757619c5235041f12d5182778b4237444f3c2246982ebcf54c0542af0da194783cea57fbe4fbde0985ef635802c648796b4f5c4d7a5d4f42c6519c7

data/README.md

@@ -1,86 +1,169 @@
-# ActiveGenie
-Ruby gem for out of the box AI features. LLMs are just a raw tools that need to be polished and refined to be useful. This gem is a collection of tools that can be used to make LLMs more useful and easier to use.
-Think this gem is like ActiveStorage but for LLMs.
+# ActiveGenie 🧞‍♂️
+> Transform your Ruby application with powerful, production-ready GenAI features
 
 [![Gem Version](https://badge.fury.io/rb/active_genie.svg?icon=si%3Arubygems)](https://badge.fury.io/rb/active_genie)
+[![Ruby](https://github.com/roriz/active_genie/actions/workflows/ruby.yml/badge.svg)](https://github.com/roriz/active_genie/actions/workflows/ruby.yml)
+
+ActiveGenie is a Ruby gem that provides a polished, production-ready interface for working with Generative AI (GenAI) models. Just like ActiveStorage simplifies file handling in Rails, ActiveGenie makes it effortless to integrate GenAI capabilities into your Ruby applications.
+
+## Features
+
+- 🎯 **Data Extraction**: Extract structured data from unstructured text with type validation
+- 📊 **Smart Scoring**: Multi-reviewer evaluation system with automatic expert selection
+- 💭 **Leaderboard**: Consistent rank items based on custom criteria, using multiple tecniques of ranking
 
 ## Installation
-Add this line to your application's Gemfile:
 
-1. Add the gem to your Gemfile
+1. Add to your Gemfile:
 ```ruby
 gem 'active_genie'
 ```
-2. install the gem
+
+2. Install the gem:
 ```shell
 bundle install
 ```
-3. call the initializer to create default configurations
+
+3. Generate the configuration:
 ```shell
+echo "ActiveGenie.load_tasks" >> Rakefile
 rails g active_genie:install
 ```
-4. Add the right credentials to the `config/active_genie.yml` file
-```yaml
-GPT-4o-mini:
-  api_key: <%= ENV['OPENAI_API_KEY'] %>
-  provider: "openai"
+
+4. Configure your credentials in `config/initializers/active_genie.rb`:
+```ruby
+ActiveGenie.configure do |config|
+  config.openai.api_key = ENV['OPENAI_API_KEY']
+end
 ```
 
-## Quick Example
+## Quick Start
+
+### Data Extractor
+Extract structured data from text using AI-powered analysis, handling informal language and complex expressions.
+
 ```ruby
-require 'active_genie'
+text = "Nike Air Max 90 - Size 42 - $199.99"
+schema = {
+  brand: { 
+    type: 'string',
+    enum: ["Nike", "Adidas", "Puma"]
+  },
+  price: { 
+    type: 'number',
+    minimum: 0
+  },
+  size: {
+    type: 'integer',
+    minimum: 35,
+    maximum: 46
+  }
+}
 
-puts ActiveGenie::DataExtractor.call(
-  "Hello, my name is Radamés Roriz",
-  { full_name: { type: 'string' } },
-  options: { model: 'gpt-4o-mini', api_key: 'your-api-key', provider: 'openai' } # Optional if has config/active_genie.yml
-) # => { full_name: "Radamés Roriz" }
+result = ActiveGenie::DataExtractor.call(text, schema)
+# => { 
+#      brand: "Nike", 
+#      brand_explanation: "Brand name found at start of text",
+#      price: 199.99,
+#      price_explanation: "Price found in USD format at end",
+#      size: 42,
+#      size_explanation: "Size explicitly stated in the middle"
+#    }
 ```
 
-### Data Extractor
-Extract structured data from text using LLM-powered analysis, handling informal language and complex expressions.
+Features:
+- Structured data extraction with type validation
+- Schema-based extraction with custom constraints
+- Informal text analysis (litotes, hedging)
+- Detailed explanations for extracted values
+
+See the [Data Extractor README](lib/active_genie/data_extractor/README.md) for informal text processing, advanced schemas, and detailed interface documentation.
+
+### Scoring
+Text evaluation system that provides detailed scoring and feedback using multiple expert reviewers. Get balanced scoring through AI-powered expert reviewers that automatically adapt to your content.
+
+```ruby
+text = "The code implements a binary search algorithm with O(log n) complexity"
+criteria = "Evaluate technical accuracy and clarity"
+
+result = ActiveGenie::Scoring.basic(text, criteria)
+# => {
+#      algorithm_expert_score: 95,
+#      algorithm_expert_reasoning: "Accurately describes binary search and its complexity",
+#      technical_writer_score: 90,
+#      technical_writer_reasoning: "Clear and concise explanation of the algorithm",
+#      final_score: 92.5
+#    }
+```
+
+Features:
+- Multi-reviewer evaluation with automatic expert selection
+- Detailed feedback with scoring reasoning
+- Customizable reviewer weights
+- Flexible evaluation criteria
+
+See the [Scoring README](lib/active_genie/scoring/README.md) for advanced usage, custom reviewers, and detailed interface documentation.
+
+### Battle
+AI-powered battle evaluation system that determines winners between two players based on specified criteria.
 
 ```ruby
 require 'active_genie'
 
-text = "iPhone 14 Pro Max"
-schema = {
-  brand: { type: 'string' },
-  model: { type: 'string' }
-}
-result = ActiveGenie::DataExtractor.call(
-  text,
-  schema,
-  options: { model: 'GPT-4o-mini', api_key: 'your-api-key', provider: 'openai' } # Optional if 
-)
-# => { brand: "iPhone", model: "14 Pro Max" }
+player_a = "Implementation uses dependency injection for better testability"
+player_b = "Code has high test coverage but tightly coupled components"
+criteria = "Evaluate code quality and maintainability"
+
+result = ActiveGenie::Battle.call(player_a, player_b, criteria)
+# => {
+#      winner_player: "Implementation uses dependency injection for better testability",
+#      reasoning: "Player A's implementation demonstrates better maintainability through dependency injection, 
+#                 which allows for easier testing and component replacement. While Player B has good test coverage, 
+#                 the tight coupling makes the code harder to maintain and modify.",
+#      what_could_be_changed_to_avoid_draw: "Focus on specific architectural patterns and design principles"
+#    }
 ```
 
-- More examples in the [Data Extractor README](lib/data_extractor/README.md)
-- Extract from ambiguous [from_informal](lib/data_extractor/README.md#extract-from-informal-text)
-- Interface details in the [Interface](lib/data_extractor/README.md#interface)
+Features:
+- Multi-reviewer evaluation with automatic expert selection
+- Detailed feedback with scoring reasoning
+- Customizable reviewer weights
+- Flexible evaluation criteria
+
+See the [Battle README](lib/active_genie/battle/README.md) for advanced usage, custom reviewers, and detailed interface documentation.
+
+### League
+The League module provides competitive ranking through multi-stage evaluation:
 
-### Summarizer (WIP)
-The summarizer is a tool that can be used to summarize a given text. It uses a set of rules to summarize the text out of the box. Uses the best practices of prompt engineering and engineering to make the summarization as accurate as possible.
 
 ```ruby
 require 'active_genie'
 
-text = "Example text to be summarized. The fox jumps over the dog"
-summarized_text = ActiveGenie::Summarizer.call(text)
-puts summarized_text # => "The fox jumps over the dog"
+players = ['REST API', 'GraphQL API', 'SOAP API', 'gRPC API', 'Websocket API']
+criteria = "Best one to be used into a high changing environment"
+
+result = ActiveGenie::League.call(players, criteria)
+# => {
+#      winner_player: "gRPC API",
+#      reasoning: "gRPC API is the best one to be used into a high changing environment",
+#    }
 ```
 
-### Scorer (WIP)
-The scorer is a tool that can be used to score a given text. It uses a set of rules to score the text out of the box. Uses the best practices of prompt engineering and engineering to make the scoring as accurate as possible.
+- **Multi-phase ranking system** combining expert scoring and ELO algorithms
+- **Automatic elimination** of inconsistent performers using statistical analysis
+- **Dynamic ranking adjustments** based on simulated pairwise battles, from bottom to top
+
+See the [League README](lib/active_genie/league/README.md) for implementation details, configuration, and advanced ranking strategies.
+
+### Summarizer (WIP)
+The summarizer is a tool that can be used to summarize a given text. It uses a set of rules to summarize the text out of the box. Uses the best practices of prompt engineering and engineering to make the summarization as accurate as possible.
 
 ```ruby
 require 'active_genie'
 
-text = "Example text to be scored. The fox jumps over the dog"
-criterias = 'Grammar, Relevance'
-score = ActiveGenie::Scorer.call(text, criterias)
-puts score # => { 'Grammar' => 0.8, 'Relevance' => 1.0, total: 0.9 }
+text = "Example text to be summarized. The fox jumps over the dog"
+summarized_text = ActiveGenie::Summarizer.call(text)
+puts summarized_text # => "The fox jumps over the dog"
 ```
 
 ### Language detector (WIP)
@@ -116,21 +199,25 @@ sentiment = ActiveGenie::SentimentAnalyzer.call(text)
 puts sentiment # => "positive"
 ```
 
-### Elo ranking (WIP)
-The Elo ranking is a tool that can be used to rank a set of items. It uses a set of rules to rank the items out of the box. Uses the best practices of prompt engineering and engineering to make the ranking as accurate as possible.
+## Configuration
 
-```ruby
-require 'active_genie'
+| Config | Description | Default |
+|--------|-------------|---------|
+| `provider` | LLM provider (openai, anthropic, etc) | `nil` |
+| `model` | Model to use | `nil` |
+| `api_key` | Provider API key | `nil` |
+| `timeout` | Request timeout in seconds | `5` |
+| `max_retries` | Maximum retry attempts | `3` |
 
-items = ['Square', 'Circle', 'Triangle']
-criterias = 'items that look rounded'
-ranked_items = ActiveGenie::EloRanking.call(items, criterias, rounds: 10)
-puts ranked_items # => [{ name: "Circle", score: 1500 }, { name: "Square", score: 800 }, { name: "Triangle", score: 800 }]
-```
+> **Note:** Each module can append its own set of configuration, see the individual module documentation for details.
 
-## Good practices
-- LLMs can take a long time to respond, so avoid putting them in the main thread
-- Do not use the LLMs to extract sensitive or personal data
+## Contributing
 
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
 ## License
-See the [LICENSE](LICENSE)
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

data/VERSION

@@ -1 +1 @@
-0.0.3
+0.0.10

data/lib/active_genie/battle/README.md

@@ -0,0 +1,39 @@
+# Battle
+AI-powered battle evaluation system that determines winners between two players based on specified criteria.
+
+## Features
+- Content comparison - Evaluate submissions from two players against defined criteria
+- Objective analysis - AI-powered assessment of how well each player meets requirements
+- Detailed reasoning - Comprehensive explanation of why a winner was chosen
+- Draw avoidance - Suggestions on how to modify content to avoid draws
+- Flexible input - Support for both string content and structured data with content field
+
+## Basic Usage
+Evaluate a battle between two players with simple text content:
+
+```ruby
+player_a = "Implementation uses dependency injection for better testability"
+player_b = "Code has high test coverage but tightly coupled components"
+criteria = "Evaluate code quality and maintainability"
+
+result = ActiveGenie::Battle::Basic.call(player_a, player_b, criteria)
+# => {
+#      winner_player: "Implementation uses dependency injection for better testability",
+#      reasoning: "Player A's implementation demonstrates better maintainability through dependency injection, 
+#                 which allows for easier testing and component replacement. While Player B has good test coverage, 
+#                 the tight coupling makes the code harder to maintain and modify.",
+#      what_could_be_changed_to_avoid_draw: "Focus on specific architectural patterns and design principles"
+#    }
+```
+
+## Interface
+### Basic.call(player_a, player_b, criteria, config: {})
+- `player_a` [String, Hash] - The content or submission from the first player
+- `player_b` [String, Hash] - The content or submission from the second player
+- `criteria` [String] - The evaluation criteria or rules to assess against
+- `config` [Hash] - Additional configuration config that modify the battle evaluation behavior
+
+Returns a Hash containing:
+- `winner_player` [String, Hash] - The winning player's content (either player_a or player_b)
+- `reasoning` [String] - Detailed explanation of why the winner was chosen
+- `what_could_be_changed_to_avoid_draw` [String] - A suggestion on how to avoid a draw

data/lib/active_genie/battle/basic.rb

@@ -0,0 +1,130 @@
+# frozen_string_literal: true
+
+require_relative '../clients/unified_client'
+
+module ActiveGenie::Battle
+  # The Basic class provides a foundation for evaluating battles between two players
+  # using AI-powered evaluation. It determines a winner based on specified criteria,
+  # analyzing how well each player meets the requirements.
+  #
+  # The battle evaluation process compares two players' content against given criteria
+  # and returns detailed feedback including the winner and reasoning for the decision.
+  #
+  # @example Basic usage with two players and criteria
+  #   Basic.call("Player A content", "Player B content", "Evaluate keyword usage and pattern matching")
+  #
+  class Basic
+    def self.call(player_a, player_b, criteria, config: {})
+      new(player_a, player_b, criteria, config:).call
+    end
+
+    # @param player_a [String] The content or submission from the first player
+    # @param player_b [String] The content or submission from the second player
+    # @param criteria [String] The evaluation criteria or rules to assess against
+    # @param config [Hash] Additional configuration config that modify the battle evaluation behavior
+    # @return [Hash] The evaluation result containing the winner and reasoning
+    #   @return [String] :winner The @param player_a or player_b
+    #   @return [String] :reasoning Detailed explanation of why the winner was chosen
+    #   @return [String] :what_could_be_changed_to_avoid_draw A suggestion on how to avoid a draw
+    def initialize(player_a, player_b, criteria, config: {})
+      @player_a = player_a
+      @player_b = player_b
+      @criteria = criteria
+      @config = config
+      @response = nil
+    end
+
+    def call
+      messages = [
+        {  role: 'system', content: PROMPT },
+        {  role: 'user', content: "criteria: #{@criteria}" },
+        {  role: 'user', content: "player_a: #{player_content(@player_a)}" },
+        {  role: 'user', content: "player_b: #{player_content(@player_b)}" },
+      ]
+
+      @response = ::ActiveGenie::Clients::UnifiedClient.function_calling(messages, FUNCTION, config:)
+
+      response_formatted
+    end
+
+    private
+
+    def player_content(player)
+      return player.dig('content') if player.is_a?(Hash)
+
+      player
+    end
+
+    def response_formatted
+      winner = case @response['winner']
+               when 'player_a' then @player_a
+               when 'player_b' then @player_b
+               end
+
+      @response.merge!('winner' => winner, 'loser' => winner ? (winner == @player_a ? @player_b : @player_a) : nil)
+    end
+
+    PROMPT = <<~PROMPT
+    Evaluate a battle between player_a and player_b using predefined criteria and identify the winner.
+
+    Consider rules, keywords, and patterns as the criteria for evaluation. Analyze the content from both players objectively, focusing on who meets the criteria most effectively. Explain your decision clearly, with specific reasoning on how the chosen player fulfilled the criteria better than the other. Avoid selecting a draw unless absolutely necessary.
+
+    # Steps
+    1. **Review Predefined Criteria**: Understand the specific rules, keywords, and patterns that serve as the basis for evaluation.
+    2. **Analyze Content**: Examine the contributions of both player_a and player_b. Look for how each player meets or fails to meet the criteria.
+    3. **Comparison**: Compare both players against each criterion to determine who aligns better with the standards set.
+    4. **Decision-Making**: Based on the analysis, determine the player who meets the most or all criteria effectively.
+    5. **Provide Justification**: Offer a clear and concise reason for your choice detailing how the winner outperformed the other.
+
+    # Examples
+    - **Example 1**:
+      - Input: Player A uses keyword X, follows rule Y, Player B uses keyword Z, breaks rule Y.
+      - Output: winner: player_a
+        - Justification: Player A successfully used keyword X and followed rule Y, whereas Player B broke rule Y.
+
+    - **Example 2**:
+      - Input: Player A matches pattern P, Player B matches pattern P, uses keyword Q.
+      - Output: winner: player_b
+        - Justification: Both matched pattern P, but Player B also used keyword Q, meeting more criteria.
+
+    # Notes
+    - Avoid drawing if a clear winner can be discerned.
+    - Critically assess each player's adherence to the criteria.
+    - Clearly communicate the reasoning behind your decision.
+    PROMPT
+
+    FUNCTION =  {
+      name: 'battle_evaluation',
+      description: 'Evaluate a battle between player_a and player_b using predefined criteria and identify the winner.',
+      schema: {
+        type: "object",
+        properties: {
+          winner: {
+            type: 'string',
+            description: 'The player who won the battle based on the criteria.',
+            enum: ['player_a', 'player_b', 'draw']
+          },
+          reasoning_of_winner: {
+            type: 'string',
+            description: 'The detailed reasoning about why the winner won based on the criteria.',
+          },
+          what_could_be_changed_to_avoid_draw: {
+            type: 'string',
+            description: 'Suggestions on how to avoid a draw based on the criteria. Be as objective and short as possible. Can be empty.',
+          }
+        }
+      }
+    }
+
+    def config
+      {
+        all_providers: { model_tier: 'lower_tier' },
+        log: {
+          **(@config.dig(:log) || {}),
+          trace: self.class.name,
+        },
+        **@config
+      }
+    end
+  end
+end

data/lib/active_genie/battle.rb

@@ -0,0 +1,13 @@
+
+require_relative 'battle/basic'
+
+module ActiveGenie
+  # See the [Battle README](lib/active_genie/battle/README.md) for more information.
+  module Battle
+    module_function
+
+    def basic(...)
+      Basic.call(...)
+    end
+  end
+end

data/lib/active_genie/clients/openai_client.rb

@@ -0,0 +1,77 @@
+require 'json'
+require 'net/http'
+
+module ActiveGenie::Clients
+  class OpenaiClient
+    def initialize(config)
+      @app_config = config
+    end
+
+    def function_calling(messages, function, config: {})
+      model = config[:model]
+      model = @app_config.tier_to_model(config.dig(:all_providers, :model_tier)) if model.nil? && config.dig(:all_providers, :model_tier)
+      model = @app_config.lower_tier_model if model.nil?
+
+      payload = {
+        messages:,
+        response_format: {
+          type: 'json_schema',
+          json_schema: function
+        },
+        model:,
+      }
+
+      api_key = config[:api_key] || @app_config.api_key
+      headers = DEFAULT_HEADERS.merge(
+        'Authorization': "Bearer #{api_key}"
+      ).compact
+
+      response = request(payload, headers, config:)
+
+      parsed_response = JSON.parse(response.dig('choices', 0, 'message', 'content'))
+      parsed_response.dig('properties') || parsed_response
+    rescue JSON::ParserError
+      nil
+    end
+
+    private
+
+    def request(payload, headers, config:)
+      start_time = Time.now
+      response = Net::HTTP.post(
+        URI("#{@app_config.api_url}/chat/completions"),
+        payload.to_json,
+        headers
+      )
+
+      raise OpenaiError, response.body unless response.is_a?(Net::HTTPSuccess)
+      return nil if response.body.empty?
+
+      parsed_body = JSON.parse(response.body)
+      log_response(start_time, parsed_body, config:)
+
+      parsed_body
+    end
+
+    DEFAULT_HEADERS = {
+      'Content-Type': 'application/json',
+    }
+
+    def log_response(start_time, response, config:)
+      ActiveGenie::Logger.trace(
+        {
+          **config.dig(:log),
+          category: :llm,
+          trace: "#{config.dig(:log, :trace)}/#{self.class.name}",
+          total_tokens: response.dig('usage', 'total_tokens'),
+          model: response.dig('model'),
+          request_duration: Time.now - start_time,
+          openai: response
+        }
+      )
+    end
+
+    # TODO: add some more rich error handling
+    class OpenaiError < StandardError; end
+  end
+end

data/lib/active_genie/clients/unified_client.rb

@@ -0,0 +1,19 @@
+module ActiveGenie::Clients
+  class UnifiedClient
+    class << self
+      def function_calling(messages, function, config: {})
+        provider_name = config[:provider]&.downcase&.strip&.to_sym
+        provider = ActiveGenie.configuration.providers.all[provider_name] || ActiveGenie.configuration.providers.default
+
+        raise InvalidProviderError if provider.nil? || provider.client.nil?
+
+        provider.client.function_calling(messages, function, config:)
+      end
+
+      private
+
+      # TODO: improve error message
+      class InvalidProviderError < StandardError; end
+    end
+  end
+end

data/lib/active_genie/configuration/log_config.rb

@@ -0,0 +1,14 @@
+
+module ActiveGenie::Configuration
+  class LogConfig
+    attr_writer :log_level
+
+    def log_level
+      @log_level ||= :info
+    end
+
+    def to_h(config = {})
+      { log_level:, **config }
+    end
+  end
+end

data/lib/active_genie/configuration/openai_config.rb

@@ -0,0 +1,56 @@
+require_relative '../clients/openai_client'
+
+module ActiveGenie::Configuration
+  class OpenaiConfig
+    attr_writer :api_key, :organization, :api_url, :client,
+      :lower_tier_model, :middle_tier_model, :upper_tier_model
+
+    def api_key
+      @api_key || ENV['OPENAI_API_KEY']
+    end
+
+    def organization
+      @organization || ENV['OPENAI_ORGANIZATION']
+    end
+
+    def lower_tier_model
+      @lower_tier_model || 'gpt-4o-mini'
+    end
+
+    def middle_tier_model
+      @middle_tier_model || 'gpt-4o'
+    end
+
+    def upper_tier_model
+      @upper_tier_model || 'o1-preview'
+    end
+
+    def tier_to_model(tier)
+      {
+        lower_tier: lower_tier_model,
+        middle_tier: middle_tier_model,
+        upper_tier: upper_tier_model
+      }[tier&.to_sym]
+    end
+
+    def api_url
+      @api_url || 'https://api.openai.com/v1'
+    end
+
+    def client
+      @client ||= ::ActiveGenie::Clients::OpenaiClient.new(self)
+    end
+
+    def to_h(config = {})
+      {
+        api_key:,
+        organization:,
+        api_url:,
+        lower_tier_model:,
+        middle_tier_model:,
+        upper_tier_model:,
+        **config
+      }
+    end
+  end
+end

data/lib/active_genie/configuration/providers_config.rb

@@ -0,0 +1,37 @@
+module ActiveGenie::Configuration
+  class ProvidersConfig
+    def initialize
+      @all = {}
+      @default = nil
+    end
+
+    def register(name, provider_class)
+      @all ||= {}
+      @all[name] = provider_class.new
+      define_singleton_method(name) do
+        instance_variable_get("@#{name}") || instance_variable_set("@#{name}", @all[name])
+      end
+
+      self
+    end
+
+    def default
+      @default || @all.values.first
+    end
+
+    def all
+      @all
+    end
+
+    def to_h(config = {})
+      hash_all = {}
+      @all.each do |name, provider|
+        hash_all[name] = provider.to_h(config.dig(name) || {})
+      end
+      hash_all
+    end
+
+    private
+    attr_writer :default
+  end
+end