better_translate 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1110968d8e49f85d5231775f9c05fcbca610021a33723999c75246b687f56b77
-  data.tar.gz: 255e51ff5e5105ebf314b703d9aa841c66eaa6b0ffb5ca1f631ea2cbe81ab6db
+  metadata.gz: 6941ef1c334c2ddbbb7141c8257f8906957fd0b60eadeee867d9a11b83788ae6
+  data.tar.gz: 8678aa23790a06d905592efc1571a91408e58a9f1359e70ee90174d5d2ee8975
 SHA512:
-  metadata.gz: b971940521bce1ad359990e13fe7d17a339f9d5e62fe5ea9b565fe1ffbd51fe2c2dd24789cd02b7e838a20a3ae88de919ed529adf325b3c11d7377c6bced68e5
-  data.tar.gz: 1d8d785607ba4439df35ceb70b0c302ba1944bfdd74e197d2686cbe7781cfab55bfc93ed7a2affa9ef50b5fc1a459034b1d3cdf6f60420cb2df1b0252d5e3ac5
+  metadata.gz: 548433e5223f37320a42c7ba35644b46470dc89baf42581612753d0542d039e2729cad8dd8bf4fbde1cd962ac970f397fdd1a3d98374f83fbc0380805771daae
+  data.tar.gz: 704eaeb04b49d7f165631443567e89afdff27a30c42893d0d04e424fe340fb113a9a4553ded29c1b4f8e9af36d75eccea0e138d0056c72442f75570340b9fa97

data/CHANGELOG.md

@@ -0,0 +1,87 @@
+# Changelog
+
+All notable changes to BetterTranslate will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.4.0] - 2025-03-11
+
+### Added
+- New Translation Similarity Analyzer:
+  - Identifies similar translations across language files
+  - Generates detailed JSON reports and human-readable summaries
+  - Uses Levenshtein distance for similarity calculation
+  - Configurable similarity threshold
+- New Rails generator: `rails generate better_translate:analyze`
+  - Analyzes all YAML files in the locales directory
+  - Provides immediate feedback in the console
+  - Generates comprehensive similarity reports
+
+## [0.3.1] - 2025-03-11
+
+### Added
+- Comprehensive RSpec test suite covering:
+  - Core translation functionality
+  - LRU cache implementation
+  - Provider selection and initialization
+  - Error handling
+  - Configuration management
+- Improved documentation with badges and testing information
+
+### Changed
+- Made `translate` method public in `Service` class for better testability
+- Reorganized README.md with better structure and modern layout
+
+## [0.3.0] - 2025-03-11
+
+### Added
+- New translation helper methods:
+  - `translate_text_to_languages`: Translate single text to multiple languages
+  - `translate_texts_to_languages`: Translate multiple texts to multiple languages
+- LRU caching for improved performance
+
+### Changed
+- Enhanced error handling in translation providers
+- Improved method documentation
+
+## [0.2.0] - 2025-03-11
+
+### Added
+- Two-step filtering process:
+  - Global exclusions using `global_exclusions`
+  - Language-specific exclusions using `exclusions_per_language`
+
+### Changed
+- Improved filtering logic to handle language-specific exclusions independently
+- Enhanced documentation for exclusion configuration
+
+### Fixed
+- Issue with language-specific exclusions affecting global filtering
+
+## [0.1.1] - 2025-03-10
+
+### Added
+- New Rails generator: `rails generate better_translate:translate`
+  - Triggers translation process
+  - Displays progress messages
+  - Integrates with existing configuration
+
+## [0.1.0] - 2025-03-10
+
+### Added
+- Initial release with core features:
+  - YAML file translation from source to multiple target languages
+  - Multiple provider support (ChatGPT and Google Gemini)
+  - Progress tracking with ruby-progressbar
+  - Centralized configuration via initializer
+  - Two translation modes: override and incremental
+  - Key exclusion system
+  - Rails integration
+
+### Configuration
+- API key management for providers
+- Source and target language settings
+- Output folder configuration
+- Translation mode selection
+- Exclusion patterns support

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,132 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official email address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[INSERT CONTACT METHOD].
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

data/README.md

@@ -1,30 +1,44 @@
-# BetterTranslate
-
-BetterTranslate is a Ruby gem that enables you to translate YAML files from a source language into one or more target languages using translation providers such as ChatGPT (OpenAI) and Google Gemini. The gem supports two translation modes:
-
-- **Override**: Completely rewrites the translated file.
-- **Incremental**: Updates the translated file only with new or modified keys while keeping the existing ones.
-
-Configuration is centralized via an initializer (for example, in a Rails app), where you can set API keys, the source language, target languages, key exclusions, the output folder, and the translation mode. Additionally, BetterTranslate integrates progress tracking using the [ruby-progressbar](https://github.com/jfelchner/ruby-progressbar) gem.
-
-## Features
-
-- **Multi-language YAML Translation**: Translates YAML files from a source language into one or more target languages.
-- **Multiple Providers**: Supports ChatGPT (OpenAI) and Google Gemini (with potential for extension to other providers in the future).
-- **Translation Modes**: 
-  - **Override**: Rewrites the file from scratch.
-  - **Incremental**: Updates only missing or modified keys.
-- **Centralized Configuration**: Configured via an initializer with settings for API keys, source language, target languages, exclusions (using dot notation), and the output folder.
-- **Two-Step Exclusion Filtering**: 
-  - **Global Exclusions**: Removes keys defined in `global_exclusions` from the entire YAML structure.
-  - **Language-Specific Exclusions**: Applies additional filtering using the `exclusions_per_language` map for each target language.
-- **Progress Bar**: Displays translation progress using ruby-progressbar.
-- **Rails Generators**: 
-  - `rails generate better_translate:install` to generate the initializer.
-  - `rails generate better_translate:translate` to trigger the translation process directly from your Rails app.
-- **Translation Helpers**: 
-  - `translate_text_to_languages` for translating a single text into multiple target languages.
-  - `translate_texts_to_languages` for translating an array of texts into multiple target languages.
+# BetterTranslate 🌍
+
+[![Gem Version](https://badge.fury.io/rb/better_translate.svg)](https://badge.fury.io/rb/better_translate)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-rubocop-brightgreen.svg)](https://github.com/rubocop/rubocop)
+
+> 🚀 A powerful Ruby gem for seamless YAML file translations using AI providers like ChatGPT and Google Gemini
+
+BetterTranslate simplifies the translation of YAML files in your Ruby/Rails applications by leveraging advanced AI models. With support for both ChatGPT (OpenAI) and Google Gemini, it offers:
+
+✨ **Key Features**
+- 🔄 Smart translation modes (Override/Incremental)
+- 🎯 Precise key exclusion control
+- 📊 Real-time progress tracking
+- 🧪 Comprehensive test coverage
+- ⚡️ LRU caching for performance
+- 🔍 Translation similarity analysis
+
+## Why BetterTranslate? 🤔
+
+- 🌐 **AI-Powered Translation**: Leverage ChatGPT and Google Gemini for high-quality translations
+- 🔄 **Smart Translation Modes**:
+  - `Override`: Full file rewrite
+  - `Incremental`: Update only new/modified keys
+- 🎯 **Precise Control**:
+  - Global key exclusions
+  - Language-specific exclusions
+  - Dot notation support
+- 🛠 **Developer-Friendly**:
+  - Rails generators included
+  - Comprehensive test suite
+  - LRU caching for performance
+  - Progress tracking
+- 🔍 **Translation Analysis**:
+  - Similarity detection
+  - Detailed reports
+  - Optimization suggestions
+- 🔧 **Flexible Helpers**:
+  - Single text translation
+  - Bulk text translation
+  - Multiple target languages support
 
 ## Installation
 
@@ -137,6 +151,25 @@ The gem includes generators to simplify tasks:
 
   The `better_translate:translate` generator will trigger the translation process (via `BetterTranslate.magic`) and display progress in the terminal.
 
+- **Analyze Translations:**
+
+  ```bash
+  rails generate better_translate:analyze
+  ```
+
+  The `better_translate:analyze` generator will:
+  - Scan all YAML files in your locales directory
+  - Find similar translations using Levenshtein distance
+  - Generate two reports:
+    - `translation_similarities.json`: Detailed JSON report
+    - `translation_similarities_summary.txt`: Human-readable summary
+  
+  This helps you:
+  - Identify potentially redundant translations
+  - Maintain consistency across your translations
+  - Optimize your translation files
+  - Reduce translation costs
+
 ### Translation Helpers
 
 BetterTranslate provides helper methods to simplify translation tasks.
@@ -194,11 +227,38 @@ puts translated_texts
 # }
 ```
 
-## Contact & Feature Requests
+## Testing 🧪
+
+BetterTranslate comes with a comprehensive test suite using RSpec. To run the tests:
+
+```bash
+bundle exec rspec
+```
+
+The test suite covers:
+- Core translation functionality
+- Cache implementation (LRU)
+- Provider selection and initialization
+- Error handling
+- Configuration management
+
+## Contributing 🤝
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+## Contact & Support 📬
+
+- **Email**: alessio.bussolari@pandev.it
+- **Issues**: [GitHub Issues](https://github.com/alessiobussolari/better_translate/issues)
+- **Discussions**: [GitHub Discussions](https://github.com/alessiobussolari/better_translate/discussions)
 
-For suggestions, bug reports, or to request new features, please reach out via email at: **alessio.bussolari@pandev.it**.
+## License 📄
 
-## Conclusions
+This project is licensed under the MIT License - see the [LICENSE.txt](LICENSE.txt) file for details.
 
 BetterTranslate aims to simplify the translation of YAML files in Ruby projects by providing a flexible, configurable, and extensible system. Whether you need a complete file rewrite or an incremental update, BetterTranslate streamlines the translation process using advanced providers like ChatGPT and Google Gemini. Contributions, feedback, and feature requests are highly encouraged to help improve the gem further.
 

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]

data/lib/better_translate/helper.rb

@@ -68,8 +68,4 @@ module BetterTranslate
       end
     end
   end
-end
-
-__END__
-
-translated = BetterTranslate::Helper.translate_text_to_languages("Hello world!",[{ short_name: "it", name: "Italian" }, { short_name: "fr", name: "French" }],"en",:chatgpt)
+end

data/lib/better_translate/providers/base_provider.rb

@@ -3,15 +3,57 @@ module BetterTranslate
     class BaseProvider
       def initialize(api_key)
         @api_key = api_key
+        @last_request_time = Time.now - 1
       end
 
-      # Metodo da implementare nelle classi derivate.
-      # @param text [String] testo da tradurre.
-      # @param target_lang_code [String] codice della lingua di destinazione (es. "en").
-      # @param target_lang_name [String] nome della lingua di destinazione (es. "English").
+      private
+
+      def rate_limit
+        time_since_last_request = Time.now - @last_request_time
+        sleep(0.5 - time_since_last_request) if time_since_last_request < 0.5
+        @last_request_time = Time.now
+      end
+
+      def validate_input(text, target_lang_code)
+        raise ArgumentError, "Text cannot be empty" if text.nil? || text.strip.empty?
+        raise ArgumentError, "Invalid target language code" unless target_lang_code.match?(/^[a-z]{2}(-[A-Z]{2})?$/)
+      end
+
+      # Method to be implemented in derived classes.
+      # @param text [String] text to translate.
+      # @param target_lang_code [String] target language code (e.g. "en").
+      # @param target_lang_name [String] target language name (e.g. "English").
       # @return [String] testo tradotto.
+      MAX_RETRIES = 3
+      RETRY_DELAY = 2 # seconds
+
       def translate(text, target_lang_code, target_lang_name)
-        raise NotImplementedError, "Il provider #{self.class} deve implementare il metodo translate"
+        retries = 0
+        begin
+          perform_translation(text, target_lang_code, target_lang_name)
+        rescue StandardError => e
+          retries += 1
+          if retries <= MAX_RETRIES
+            message = "Translation attempt #{retries} failed. Retrying in #{RETRY_DELAY} seconds..."
+            BetterTranslate::Utils.logger(message: message)
+            sleep(RETRY_DELAY)
+            retry
+          else
+            message = "Translation failed after #{MAX_RETRIES} attempts: #{e.message}"
+            BetterTranslate::Utils.logger(message: message)
+            raise
+          end
+        end
+      end
+
+      def perform_translation(text, target_lang_code, target_lang_name)
+        validate_input(text, target_lang_code)
+        rate_limit
+        translate_text(text, target_lang_code, target_lang_name)
+      end
+
+      def translate_text(text, target_lang_code, target_lang_name)
+        raise NotImplementedError, "The provider #{self.class} must implement the translate_text method"
       end
     end
   end

data/lib/better_translate/providers/chatgpt_provider.rb

@@ -1,19 +1,19 @@
 module BetterTranslate
   module Providers
     class ChatgptProvider < BaseProvider
-      # Utilizza l'API di OpenAI per tradurre il testo.
-      def translate(text, target_lang_code, target_lang_name)
+      # Uses the OpenAI API to translate the text.
+      def translate_text(text, target_lang_code, target_lang_name)
         uri = URI("https://api.openai.com/v1/chat/completions")
         headers = {
           "Content-Type"  => "application/json",
           "Authorization" => "Bearer #{@api_key}"
         }
 
-        # Costruiamo il prompt per tradurre il testo.
+        # Build the prompt to translate the text.
         body = {
           model: "gpt-3.5-turbo",
           messages: [
-            { role: "system", content: "Sei un traduttore professionale. Traduci esattamente il seguente testo da #{BetterTranslate.configuration.source_language} a #{target_lang_name} senza aggiungere commenti, spiegazioni o alternative. Fornisci solamente la traduzione diretta:" },
+            { role: "system", content: "You are a professional translator. Translate the following text exactly from #{BetterTranslate.configuration.source_language} to #{target_lang_name} without adding comments, explanations, or alternatives. Provide only the direct translation:" },
             { role: "user", content: "#{text}" }
           ],
           temperature: 0.3
@@ -30,7 +30,7 @@ module BetterTranslate
           translated_text = json.dig("choices", 0, "message", "content")
           translated_text ? translated_text.strip : text
         else
-          raise "Errore durante la traduzione con ChatGPT: #{response.body}"
+          raise "Error during translation with ChatGPT: #{response.body}"
         end
       end
     end

data/lib/better_translate/providers/gemini_provider.rb

@@ -3,7 +3,7 @@ module BetterTranslate
     class GeminiProvider < BaseProvider
       # Esempio di implementazione per Google Gemini.
       # Nota: L'endpoint e i parametri sono ipotetici e vanno sostituiti con quelli reali secondo la documentazione ufficiale.
-      def translate(text, target_lang_code, target_lang_name)
+      def translate_text(text, target_lang_code, target_lang_name)
         uri = URI("https://gemini.googleapis.com/v1/translate")
         headers = {
           "Content-Type"  => "application/json",

data/lib/better_translate/service.rb

@@ -1,16 +1,60 @@
 module BetterTranslate
   class Service
+    MAX_CACHE_SIZE = 1000
+
     def initialize
       @provider_name = BetterTranslate.configuration.provider
+      @translation_cache = {}
+      @cache_order = []
     end
 
-    # Metodo per tradurre un testo utilizzando il provider selezionato.
+    # Method to translate a text using the selected provider.
     def translate(text, target_lang_code, target_lang_name)
-      provider_instance.translate(text, target_lang_code, target_lang_name)
+      cache_key = "#{text}:#{target_lang_code}"
+      
+      # Prova a recuperare dalla cache
+      cached = cache_get(cache_key)
+      return cached if cached
+      
+      # Traduci e salva in cache
+      start_time = Time.now
+      result = provider_instance.translate_text(text, target_lang_code, target_lang_name)
+      duration = Time.now - start_time
+      
+      BetterTranslate::Utils.track_metric("translation_request_duration", {
+        provider: @provider_name,
+        text_length: text.length,
+        duration: duration
+      })
+      
+      cache_set(cache_key, result)
     end
 
     private
 
+    def cache_get(key)
+      if @translation_cache.key?(key)
+        # Aggiorna l'ordine LRU
+        @cache_order.delete(key)
+        @cache_order.push(key)
+        @translation_cache[key]
+      end
+    end
+
+    def cache_set(key, value)
+      if @translation_cache.size >= MAX_CACHE_SIZE
+        # Rimuovi l'elemento meno recentemente usato
+        oldest_key = @cache_order.shift
+        @translation_cache.delete(oldest_key)
+      end
+      
+      @translation_cache[key] = value
+      @cache_order.push(key)
+      value
+    end
+
+
+
     def provider_instance
       @provider_instance ||= case @provider_name
                              when :chatgpt