better_translate 0.4.2 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 04ce5f4b9e73378af8eec2409688e26dfef6fb42268b8817800c8c3433877eaf
-  data.tar.gz: 0b34308c1ef74dd3ad74b8cbc0b546cf831f2e43c3f3e2628cb636511ab33289
+  metadata.gz: 6a19ffdafe02b71cad6e1d93eba868299d8551adc3e7952450c709c3f71d1292
+  data.tar.gz: 7c5d7dde1f0b120124d13f75c31637b2a304845955a2974fc7f0c2f58eec8e76
 SHA512:
-  metadata.gz: e63e571fa9f0a881c25d35d4cb31f5cfaa0fbed2e84d3847e50f9b6f811685b5a13666408b48f0e61666232916c8cbf803dda11a6e51bcca9813f1e97b696435
-  data.tar.gz: 94f9664d32402395d9a575c2e1ae8f04b29781421fd7a4b266ae7bf0c7a2c21d345ab73762397d50028d01482255b0a10319c4799c25056256c39e0d43269df0
+  metadata.gz: a0c9c62cc2a35a0663d79af39c8e4a87cd86106b906b185c805f4ff9cba2bdfa82c1eaf8c366b5e91e70fe0c5ea87d5da852e03c32ae2a3959ae32ef2318d62a
+  data.tar.gz: 6fb2cac27269257f878d3055027213dc2bef5d75332132f559493064a4c83f1360903080571a3fd1e71c7c6feec73cf41ff619d8ecc8da180246779b5e893e59

data/.env.example

@@ -0,0 +1,14 @@
+# API Keys for Translation Providers
+# Copy this file to .env and replace with your actual API keys
+
+# OpenAI API Key (for ChatGPT provider)
+# Get your key from: https://platform.openai.com/api-keys
+OPENAI_API_KEY=your_openai_api_key_here
+
+# Google Gemini API Key
+# Get your key from: https://aistudio.google.com/app/apikey
+GEMINI_API_KEY=your_gemini_api_key_here
+
+# Anthropic API Key (for Claude provider)
+# Get your key from: https://console.anthropic.com/settings/keys
+ANTHROPIC_API_KEY=your_anthropic_api_key_here

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.rubocop.yml

@@ -0,0 +1,8 @@
+AllCops:
+  TargetRubyVersion: 3.0
+
+Style/StringLiterals:
+  EnforcedStyle: double_quotes
+
+Style/StringLiteralsInInterpolation:
+  EnforcedStyle: double_quotes

data/.yardopts

@@ -0,0 +1,10 @@
+--markup markdown
+--readme README.md
+--private
+--protected
+--output-dir doc
+lib/**/*.rb
+-
+CHANGELOG.md
+LICENSE.txt
+docs/FEATURES.md

data/CHANGELOG.md

@@ -5,109 +5,138 @@ All notable changes to BetterTranslate will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [0.4.2] - 2025-03-11
+## [1.0.0] - 2025-10-22
 
-### Added
-- Comprehensive YARD-style documentation across the entire codebase:
-  - Added detailed class and method documentation for all core components
-  - Added parameter and return type documentation
-  - Added usage examples for key classes and methods
-  - Improved inline comments for complex logic
-
-### Changed
-- Improved code readability and maintainability through better documentation
-- Enhanced developer experience with clearer API documentation
-
-## [0.4.1] - 2025-03-11
-
-### Fixed
-- Migliorati i test RSpec per garantire maggiore affidabilità:
-  - Corretti gli stub per i provider di traduzione
-  - Migliorata la gestione delle richieste HTTP nei test
-  - Ottimizzati i file YAML temporanei per i test di similarità
-- Risolti problemi di compatibilità con WebMock
-
-### Changed
-- Sostituito l'approccio di stubbing specifico con pattern più flessibili
-- Migliorata la struttura dei test per il SimilarityAnalyzer
-
-## [0.4.0] - 2025-03-11
-
-### Added
-- New Translation Similarity Analyzer:
-  - Identifies similar translations across language files
-  - Generates detailed JSON reports and human-readable summaries
-  - Uses Levenshtein distance for similarity calculation
-  - Configurable similarity threshold
-- New Rails generator: `rails generate better_translate:analyze`
-  - Analyzes all YAML files in the locales directory
-  - Provides immediate feedback in the console
-  - Generates comprehensive similarity reports
-
-## [0.3.1] - 2025-03-11
-
-### Added
-- Comprehensive RSpec test suite covering:
-  - Core translation functionality
-  - LRU cache implementation
-  - Provider selection and initialization
-  - Error handling
-  - Configuration management
-- Improved documentation with badges and testing information
-
-### Changed
-- Made `translate` method public in `Service` class for better testability
-- Reorganized README.md with better structure and modern layout
-
-## [0.3.0] - 2025-03-11
+### Complete Rewrite 🎉
 
-### Added
-- New translation helper methods:
-  - `translate_text_to_languages`: Translate single text to multiple languages
-  - `translate_texts_to_languages`: Translate multiple texts to multiple languages
-- LRU caching for improved performance
-
-### Changed
-- Enhanced error handling in translation providers
-- Improved method documentation
+This version represents a complete architectural rewrite of BetterTranslate with improved design, better testing, and enhanced features.
 
-## [0.2.0] - 2025-03-11
+**Note:** This release is not backward compatible with versions 0.x.x
 
 ### Added
-- Two-step filtering process:
-  - Global exclusions using `global_exclusions`
-  - Language-specific exclusions using `exclusions_per_language`
-
-### Changed
-- Improved filtering logic to handle language-specific exclusions independently
-- Enhanced documentation for exclusion configuration
 
-### Fixed
-- Issue with language-specific exclusions affecting global filtering
-
-## [0.1.1] - 2025-03-10
-
-### Added
-- New Rails generator: `rails generate better_translate:translate`
-  - Triggers translation process
+#### Core Infrastructure
+- **Configuration System**: Type-safe configuration with comprehensive validation
+  - Support for multiple providers (ChatGPT, Gemini, Anthropic)
+  - Configurable timeouts, retries, and concurrency
+  - Translation modes: override and incremental
+  - Optional translation context for domain-specific terminology
+
+- **LRU Cache**: Intelligent caching system for improved performance
+  - Configurable capacity (default: 1000 items)
+  - Optional TTL (Time To Live) support
+  - Thread-safe with Mutex protection
+  - Cache key format: `"#{text}:#{target_lang_code}"`
+
+- **Rate Limiter**: Thread-safe request throttling
+  - Configurable delay between requests (default: 0.5s)
+  - Prevents API overload and rate limit errors
+  - Mutex-based synchronization
+
+- **Validator**: Comprehensive input validation
+  - Language code validation (2-letter ISO codes)
+  - Text validation for translation
+  - File path validation
+  - API key validation
+
+- **Error Handling**: Custom exception hierarchy
+  - `ConfigurationError`: Configuration issues
+  - `ValidationError`: Input validation failures
+  - `TranslationError`: Translation failures
+  - `ProviderError`: Provider-specific errors
+  - `ApiError`: API call failures
+  - `RateLimitError`: Rate limit exceeded
+  - `FileError`: File operation failures
+  - `YamlError`: YAML parsing errors
+  - `ProviderNotFoundError`: Unknown provider
+
+#### Provider Architecture
+- **BaseHttpProvider**: Abstract base class for HTTP-based providers
+  - Faraday-based HTTP client (required for all providers)
+  - Retry logic with exponential backoff (3 attempts, 2s base delay, 60s max)
+  - Built-in rate limiting (0.5s between requests)
+  - Configurable timeouts (default: 30s)
+
+- **ChatGPT Provider**: OpenAI GPT integration
+  - Model: GPT-5-nano
+  - Temperature: 1.0
+
+- **Gemini Provider**: Google Gemini integration
+  - Model: gemini-2.0-flash-exp
+
+- **Anthropic Provider**: Claude integration (planned)
+  - Model: claude-3-5-sonnet-20241022
+
+#### Translation Features
+- **Translation Strategies**: Automatic strategy selection based on content size
+  - Deep Translation (< 50 strings): Individual translation with detailed progress
+  - Batch Translation (≥ 50 strings): Processes in batches of 10 for performance
+
+- **Exclusion System**: Two-tier exclusion mechanism
+  - Global exclusions: Apply to all target languages (e.g., brand names)
+  - Language-specific exclusions: Exclude keys only for specific languages
+
+- **Translation Modes**:
+  - Override mode: Replaces entire target YAML files
+  - Incremental mode: Merges with existing files, only translates missing keys
+
+- **Translation Context**: Domain-specific context for improved accuracy
+  - Medical terminology
+  - Legal terminology
+  - Financial terminology
+  - E-commerce
+  - Technical documentation
+
+#### Rails Integration
+- **Install Generator**: `rails generate better_translate:install`
+  - Creates initializer with example configuration
+  - Configures all supported providers
+
+- **Translate Generator**: `rails generate better_translate:translate`
+  - Runs translation process
   - Displays progress messages
   - Integrates with existing configuration
 
-## [0.1.0] - 2025-03-10
+- **Analyze Generator**: `rails generate better_translate:analyze`
+  - Analyzes translation similarities using Levenshtein distance
+  - Generates detailed JSON reports
+  - Provides human-readable summaries
+  - Configurable similarity threshold
 
-### Added
-- Initial release with core features:
-  - YAML file translation from source to multiple target languages
-  - Multiple provider support (ChatGPT and Google Gemini)
-  - Progress tracking with ruby-progressbar
-  - Centralized configuration via initializer
-  - Two translation modes: override and incremental
-  - Key exclusion system
-  - Rails integration
-
-### Configuration
-- API key management for providers
-- Source and target language settings
-- Output folder configuration
-- Translation mode selection
-- Exclusion patterns support
+#### Utilities
+- **HashFlattener**: Converts nested YAML to flat structure and vice versa
+  - Flatten with dot-notation keys
+  - Unflatten back to nested structure
+  - Preserves data types and structure
+
+### Development
+- **YARD Documentation**: Comprehensive documentation for all public APIs
+  - `@param` with types
+  - `@return` with types
+  - `@raise` for exceptions
+  - `@example` blocks
+
+- **RSpec Test Suite**: Full test coverage for core components
+  - Configuration tests
+  - Cache tests
+  - Rate limiter tests
+  - Validator tests
+  - Error handling tests
+  - Hash flattener tests
+
+- **RuboCop**: Code style compliance
+  - Ruby 3.0+ target
+  - Frozen string literals required
+  - Double quotes for strings
+
+### Security
+- Environment variable-based API key management
+- No hardcoded credentials
+- Input validation for all user-provided data
+- VCR cassettes with automatic API key anonymization
+
+### Performance
+- LRU caching reduces API costs
+- Batch processing for large files (≥50 strings)
+- Configurable concurrent requests (default: 3)
+- Rate limiting prevents API overload

data/CLAUDE.md

@@ -0,0 +1,385 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Development Commands
+
+### Testing
+```bash
+# Run all tests (unit + integration)
+bundle exec rake spec
+# or
+bundle exec rspec
+
+# Run only unit tests (fast, no API calls)
+bundle exec rspec spec/better_translate/
+
+# Run only integration tests (with real API calls via VCR)
+bundle exec rspec spec/integration/ --tag integration
+
+# Run specific test file
+bundle exec rspec spec/better_translate_spec.rb
+
+# Run with specific example (line number)
+bundle exec rspec spec/better_translate_spec.rb:42
+```
+
+### VCR Cassettes & API Testing
+
+**Setup API Keys**:
+1. Copy `.env.example` to `.env`:
+   ```bash
+   cp .env.example .env
+   ```
+2. Edit `.env` and add your real API keys:
+   ```env
+   OPENAI_API_KEY=sk-...
+   GEMINI_API_KEY=...
+   ANTHROPIC_API_KEY=sk-ant-...
+   ```
+3. **IMPORTANT**: Never commit `.env` file (already in `.gitignore`)
+
+**VCR Cassette Modes**:
+- `:once` (default): Use existing cassettes, record new interactions
+- `:new_episodes`: Record new interactions, keep existing ones
+- `:all`: Re-record all cassettes (use when API changes)
+
+**Re-recording Cassettes**:
+```bash
+# Delete existing cassettes and re-record with real API calls
+rm -rf spec/vcr_cassettes/
+bundle exec rspec spec/integration/ --tag integration
+
+# Re-record specific provider
+rm -rf spec/vcr_cassettes/chatgpt/
+bundle exec rspec spec/integration/chatgpt_integration_spec.rb
+```
+
+**Cassette Location**: `spec/vcr_cassettes/`
+- Cassettes are automatically anonymized (API keys replaced with placeholders)
+- Cassettes should be committed to git for CI/CD pipelines
+- Tests run without API keys when cassettes exist
+
+### Code Quality
+```bash
+# Run RuboCop linter
+bundle exec rake rubocop
+# or
+bundle exec rubocop
+
+# Auto-fix RuboCop violations
+bundle exec rubocop -a
+
+# Run type checking with Steep
+bundle exec rake steep
+# or
+bundle exec steep check
+
+# Run default rake task (runs spec, rubocop, and steep)
+bundle exec rake
+```
+
+### Documentation
+```bash
+# Generate YARD documentation
+bundle exec yard doc
+
+# Start YARD server (view docs at http://localhost:8808)
+bundle exec yard server
+
+# Check documentation coverage
+bundle exec yard stats
+```
+
+### Security
+```bash
+# Check for security vulnerabilities in dependencies
+bundle exec bundler-audit check --update
+```
+
+### Type Checking (RBS/Steep)
+```bash
+# Run type checking
+bundle exec steep check
+
+# Type check specific files
+bundle exec steep check lib/better_translate/cache.rb
+
+# Show statistics
+bundle exec steep stats
+
+# Validate RBS syntax only
+bundle exec rbs validate
+```
+
+**RBS Files**: Type signatures are in `sig/` directory
+- All public APIs have RBS signatures
+- Steep is integrated in CI/CD pipeline
+- Default rake task includes type checking
+
+**Status**: 51 type errors remaining (down from 112 initial)
+- Most errors are related to empty collection annotations
+- All critical paths are type-checked
+- Continuous improvement in progress
+
+### Gem Management
+```bash
+# Install dependencies
+bundle install
+
+# Install gem locally for testing
+bundle exec rake install
+
+# Interactive console with gem loaded
+bin/console
+```
+
+## Architecture Overview
+
+### Provider-Based System
+The gem uses a provider architecture to support multiple AI translation services:
+
+- **BaseHttpProvider**: Abstract base class for all HTTP-based providers
+  - Uses Faraday for all HTTP connections (REQUIRED - do not use Net::HTTP or other libraries)
+  - Implements retry logic with exponential backoff (3 attempts, 2s base delay, 60s max)
+  - Handles rate limiting (0.5s between requests, thread-safe with Mutex)
+  - Configurable timeouts (default: 30s)
+
+- **Providers**:
+  - ChatGPT (OpenAI): GPT-5-nano model, temperature=1.0
+  - Google Gemini: gemini-2.0-flash-exp model
+  - Anthropic Claude: Planned support
+
+### Translation Strategies
+The gem automatically selects the optimal strategy based on content size:
+
+- **Deep Translation** (< 50 strings): Individual translation with detailed progress
+- **Batch Translation** (≥ 50 strings): Processes in batches of 10 for performance
+
+### Configuration System
+Type-safe `Configuration` class with mandatory validation:
+- Required: provider, API keys, source language, target languages, file paths
+- Optional: translation mode (override/incremental), context, caching, rate limiting
+- Validation enforced via `config.validate!` before translation
+
+### Caching System
+LRU cache implementation:
+- Default capacity: 1000 items (configurable)
+- Cache key format: `"#{text}:#{target_lang_code}"`
+- Optional TTL support
+- Thread-safe with Mutex protection
+- Toggleable via `cache_enabled` config
+
+### Exclusion System
+Two-tier exclusion mechanism:
+- **Global exclusions**: Apply to all target languages (e.g., brand names)
+- **Language-specific exclusions**: Exclude keys only for specific languages (e.g., legal text that was manually translated)
+
+### Translation Modes
+- **Override**: Replaces entire target YAML files
+- **Incremental**: Merges with existing files, only translates missing keys
+
+## Rails Integration
+
+The gem provides three generators for Rails applications:
+
+```bash
+# Generate initializer with example configuration
+rails generate better_translate:install
+
+# Run translation process
+rails generate better_translate:translate
+
+# Analyze translation similarities (Levenshtein distance)
+rails generate better_translate:analyze
+```
+
+Configuration is typically done in `config/initializers/better_translate.rb`.
+
+## Development Requirements
+
+### YARD Documentation (MANDATORY)
+ALL public methods, classes, and modules must have comprehensive YARD documentation:
+
+- Use `@param` for parameters with types (e.g., `@param text [String]`)
+- Use `@return` for return values with types
+- Use `@raise` for exceptions
+- Provide `@example` blocks for public APIs
+- Mark private methods with `@api private`
+
+Example:
+```ruby
+# Translates text to a target language
+#
+# @param text [String] The text to translate
+# @param target_lang_code [String] Language code (e.g., "it", "fr")
+# @return [String] The translated text
+# @raise [ValidationError] If input is invalid
+# @raise [TranslationError] If translation fails
+#
+# @example
+#   translate("Hello", "it") #=> "Ciao"
+def translate(text, target_lang_code)
+  # ...
+end
+```
+
+### HTTP Client (MANDATORY)
+- Use Faraday for ALL HTTP connections
+- Do NOT use Net::HTTP, HTTParty, or other HTTP libraries
+- Implement retry logic and error handling as shown in BaseHttpProvider
+
+### Code Style
+- RuboCop compliance required before commits
+- String literals: Use double quotes (enforced by RuboCop)
+- Target Ruby version: 3.0+
+- Frozen string literals: Required at top of all files
+
+### Security
+- NEVER hardcode API keys in code
+- Use environment variables: `ENV['OPENAI_API_KEY']`, `ENV['GEMINI_API_KEY']`
+- VCR cassettes must anonymize API keys automatically
+- Input validation required for all user-provided data (language codes, file paths, text)
+
+## Error Handling
+
+Custom exception hierarchy (all inherit from `BetterTranslate::Error`):
+- `ConfigurationError`: Configuration issues
+- `ValidationError`: Input validation failures
+- `TranslationError`: Translation failures
+- `ProviderError`: Provider-specific errors
+- `ApiError`: API call failures
+- `RateLimitError`: Rate limit exceeded
+- `FileError`: File operation failures
+- `YamlError`: YAML parsing errors
+- `ProviderNotFoundError`: Unknown provider
+
+All errors include detailed messages and context hash for debugging.
+
+## Testing Practices
+
+### Test-Driven Development (TDD) - MANDATORY
+**ALWAYS write tests BEFORE implementing any new feature or bug fix.**
+
+This is a strict requirement for all development in this project. Follow the Red-Green-Refactor cycle:
+
+#### TDD Workflow (REQUIRED)
+1. **RED**: Write failing tests first
+   - Write RSpec tests that describe the desired behavior
+   - Run tests and verify they fail: `bundle exec rspec`
+   - Failing tests prove that the test is valid and catches the missing functionality
+
+2. **GREEN**: Implement minimum code to pass tests
+   - Write the simplest implementation that makes tests pass
+   - Run tests again and verify they pass: `bundle exec rspec`
+   - DO NOT add extra features beyond what tests require
+
+3. **REFACTOR**: Clean up code while keeping tests green
+   - Improve code quality, remove duplication
+   - Run tests after each refactoring to ensure nothing breaks
+   - Update documentation (YARD) as needed
+
+#### Example TDD Workflow
+```bash
+# 1. RED - Write failing test
+# Edit spec/providers/new_provider_spec.rb with test cases
+bundle exec rspec spec/providers/new_provider_spec.rb
+# => Should see failures (RED)
+
+# 2. GREEN - Implement feature
+# Edit lib/better_translate/providers/new_provider.rb
+bundle exec rspec spec/providers/new_provider_spec.rb
+# => Should see passing tests (GREEN)
+
+# 3. REFACTOR - Improve code
+# Refactor implementation while keeping tests green
+bundle exec rspec spec/providers/new_provider_spec.rb
+# => Should still see passing tests (GREEN)
+```
+
+#### Why TDD is Mandatory
+- Ensures all code is testable by design
+- Prevents regression bugs
+- Provides living documentation of expected behavior
+- Catches edge cases early
+- Makes refactoring safer
+
+#### Exceptions
+The ONLY acceptable exception to writing tests first is for critical production hotfixes where immediate deployment is required. In such cases:
+- Document the technical debt in code comments
+- Create a GitHub issue to add tests
+- Add tests within 24 hours of the hotfix
+
+### RSpec Setup
+
+**Test Organization**:
+- `spec/better_translate/`: Unit tests with WebMock stubs (fast, no API calls)
+- `spec/integration/`: Integration tests with VCR cassettes (real API interactions)
+
+**Unit Tests** (WebMock):
+- Fast execution, no API keys required
+- Test code structure, request formatting, and error handling
+- Use `stub_request` to mock HTTP responses
+- Example: `spec/better_translate/providers/chatgpt_provider_spec.rb`
+
+**Integration Tests** (VCR):
+- Test real API interactions
+- Require API keys in `.env` file for first run (to record cassettes)
+- Subsequent runs use recorded cassettes (no API keys needed)
+- Tag with `:integration` and `:vcr`
+- Example: `spec/integration/chatgpt_integration_spec.rb`
+
+**Running Tests**:
+```bash
+# Unit tests only (fast, recommended for TDD)
+bundle exec rspec spec/better_translate/
+
+# Integration tests only (slower, validates API compatibility)
+bundle exec rspec spec/integration/ --tag integration
+
+# All tests
+bundle exec rspec
+```
+
+### VCR Configuration Details
+
+VCR is configured in `spec_helper.rb` with:
+- **Cassette library**: `spec/vcr_cassettes/`
+- **Record mode**: `:once` (use existing, record new)
+- **API key filtering**: Automatically replaces keys with `<OPENAI_API_KEY>`, etc.
+- **Match on**: HTTP method, URI, and request body
+
+**When to Re-record Cassettes**:
+1. API response format changes
+2. Adding new test scenarios
+3. Provider updates model or endpoint
+4. Testing error conditions
+
+**Cassette Workflow**:
+1. First run: Needs real API keys, records responses
+2. Subsequent runs: Uses cassettes, no API calls
+3. CI/CD: Uses committed cassettes, no secrets needed
+
+## Translation Context Feature
+
+The `translation_context` configuration allows providing domain-specific context to improve translation accuracy:
+
+```ruby
+config.translation_context = "Medical terminology for healthcare applications"
+```
+
+This context is included in the AI system prompt, helping with specialized terminology in fields like:
+- Medical/Healthcare
+- Legal
+- Financial
+- E-commerce
+- Technical documentation
+
+## Performance Considerations
+
+- Enable caching for repeated translations to reduce API costs
+- Use incremental mode to preserve manual corrections
+- Monitor API usage through provider dashboards
+- Batch processing automatically used for large files (≥50 strings)
+- Rate limiting prevents API overload (configurable, default 0.5s between requests)
+- Concurrent requests configurable via `max_concurrent_requests` (default: 3)