mistral_translator 0.1.0 → 0.3.0

data/docs/advanced-usage/concurrent-async.md

@@ -0,0 +1,270 @@
+# Concurrent & Asynchronous Processing
+
+MistralTranslator is thread-safe and optimized for concurrent usage with protected metrics, connection pooling, and built-in rate limiting.
+
+## Thread Safety
+
+**Protected components:**
+- Metrics tracking with Mutex
+- Logger cache with concurrent access control
+- Rate limiter with thread synchronization
+- HTTP connection pooling (Net::HTTP::Persistent)
+
+## Ruby Threads
+
+### Basic Usage
+
+```ruby
+languages = ['fr', 'es', 'de', 'it']
+threads = languages.map do |lang|
+  Thread.new do
+    MistralTranslator.translate("Hello", from: 'en', to: lang)
+  rescue MistralTranslator::Error => e
+    { error: e.message }
+  end
+end
+
+results = threads.map(&:value)
+```
+
+### Thread Pool Pattern
+
+```ruby
+# Safe: Limited threads
+texts.each_slice(5) do |batch|
+  threads = batch.map { |t| Thread.new { translate(t) } }
+  threads.each(&:join)
+end
+
+# Dangerous: Unlimited threads can exhaust memory
+texts.map { |t| Thread.new { translate(t) } }  # Avoid this
+```
+
+## Concurrent Ruby (Recommended)
+
+### Installation
+
+```ruby
+# Gemfile
+gem 'concurrent-ruby', '~> 1.2'
+```
+
+### Fixed Thread Pool
+
+```ruby
+require 'concurrent'
+
+pool = Concurrent::FixedThreadPool.new(5)
+
+futures = languages.map do |lang|
+  Concurrent::Future.execute(executor: pool) do
+    MistralTranslator.translate(text, from: 'en', to: lang)
+  end
+end
+
+results = futures.map(&:value)
+
+pool.shutdown
+pool.wait_for_termination
+```
+
+### Promise Chains
+
+```ruby
+promise = Concurrent::Promise.execute do
+  MistralTranslator.translate_auto(text, to: 'en')
+end.then do |english|
+  target_langs.map do |lang|
+    MistralTranslator.translate(english, from: 'en', to: lang)
+  end
+end.rescue do |error|
+  Rails.logger.error "Pipeline failed: #{error}"
+  []
+end
+
+results = promise.value
+```
+
+## Background Jobs
+
+### SolidQueue (Rails 8+, Recommended)
+
+SolidQueue is the default ActiveJob backend in Rails 8, database-backed and Redis-free.
+
+```ruby
+# config/database.yml - SolidQueue uses your existing database
+production:
+  primary:
+    <<: *default
+  queue:  # Separate DB for jobs (optional)
+    <<: *default
+    database: app_queue
+    migrations_paths: db/queue_migrate
+
+# app/jobs/translation_job.rb
+class TranslationJob < ApplicationJob
+  queue_as :translations
+
+  retry_on MistralTranslator::RateLimitError, wait: :polynomially_longer
+  retry_on MistralTranslator::ApiError, wait: 5.seconds, attempts: 3
+
+  def perform(text, from_locale, to_locale)
+    MistralTranslator.translate(text, from: from_locale, to: to_locale)
+  end
+end
+
+# Usage
+languages.each { |lang| TranslationJob.perform_later(text, 'en', lang) }
+```
+
+**Configuration:**
+```ruby
+# config/recurring.yml - Schedule periodic translations
+production:
+  sync_translations:
+    class: TranslationSyncJob
+    schedule: every day at 3am
+    queue: translations
+```
+
+### Sidekiq / Other Backends
+
+Compatible with any ActiveJob backend:
+
+```ruby
+class TranslationJob < ApplicationJob
+  queue_as :translations
+  retry_on MistralTranslator::RateLimitError, wait: :exponentially_longer
+
+  def perform(text, from, to)
+    MistralTranslator.translate(text, from: from, to: to)
+  end
+end
+```
+
+## Performance Best Practices
+
+### Configuration for Concurrent Use
+
+```ruby
+MistralTranslator.configure do |config|
+  config.enable_metrics = true
+  config.retry_delays = [1, 2, 4, 8]  # Shorter for concurrent use
+
+  config.on_rate_limit = ->(from, to, wait, attempt, ts) {
+    Rails.logger.warn "Rate limit: #{wait}s wait"
+  }
+end
+```
+
+### Recommendations
+
+**Do:**
+- Use fixed thread pools (5-10 threads)
+- Enable metrics for monitoring
+- Use background jobs for non-critical tasks
+- Batch similar requests
+
+**Avoid:**
+- Unlimited thread creation
+- Blocking user requests with translations
+- Sharing API keys across environments
+
+## Rate Limiting
+
+The built-in rate limiter (50 requests/60s) is thread-safe but per-process.
+
+**Multiple processes:** Each has its own limiter. With 4 Puma workers = 200 requests/min total.
+
+### Custom Rate Limiter
+
+```ruby
+class RedisRateLimiter
+  def wait_and_record!
+    key = "mistral:#{Time.now.to_i / 60}"
+    count = REDIS.incr(key)
+    REDIS.expire(key, 60) if count == 1
+
+    sleep(60 - Time.now.to_i % 60) if count > 50
+  end
+end
+
+client = MistralTranslator::Client.new(
+  rate_limiter: RedisRateLimiter.new
+)
+```
+
+## Monitoring
+
+```ruby
+MistralTranslator.configure do |config|
+  config.enable_metrics = true
+
+  config.on_translation_complete = ->(from, to, orig, trans, duration) {
+    Rails.logger.info "[#{Thread.current.object_id}] #{from}→#{to}: #{duration.round(2)}s"
+  }
+end
+
+# Thread-safe metrics
+metrics = MistralTranslator.metrics
+puts "Total: #{metrics[:total_translations]}"
+puts "Avg time: #{metrics[:average_translation_time]}s"
+puts "Error rate: #{metrics[:error_rate]}%"
+```
+
+## Example: High-Performance Service
+
+```ruby
+class TranslationService
+  def initialize(max_threads: 5)
+    @pool = Concurrent::FixedThreadPool.new(max_threads)
+  end
+
+  def translate_all(texts, from:, to_languages:)
+    futures = texts.flat_map do |text|
+      to_languages.map do |lang|
+        Concurrent::Future.execute(executor: @pool) do
+          MistralTranslator.translate(text, from: from, to: lang)
+        end
+      end
+    end
+    futures.map(&:value)
+  ensure
+    @pool.shutdown
+    @pool.wait_for_termination(30)
+  end
+end
+```
+
+## Troubleshooting
+
+**"Too many open files"**
+```bash
+ulimit -n 4096
+```
+
+**Rate limits despite low volume**
+Check if multiple processes share the API key. Consider Redis-based rate limiting.
+
+**Memory growth**
+```ruby
+# Join threads periodically
+threads = []
+texts.each do |text|
+  threads << Thread.new { translate(text) }
+
+  if threads.size >= 10
+    threads.each(&:join)
+    threads.clear
+  end
+end
+```
+
+## Summary
+
+- Thread-safe with protected metrics and configuration
+- Connection pooling automatic with net-http-persistent
+- Use fixed-size thread pools (5-10 recommended)
+- Built-in rate limiting works across threads
+- Background jobs recommended for non-critical tasks
+- Separate API keys per environment

data/docs/advanced-usage/error-handling.md

@@ -0,0 +1,106 @@
+> **Navigation :** [🏠 Home](README.md) • [📖 API Reference](api-reference/methods.md) • [⚡ Advanced Usage](advanced-usage/translations.md) • [🛤️ Rails Integration](rails-integration/setup.md)
+
+---
+
+# Gestion des Erreurs
+
+Patterns et stratégies pour gérer les erreurs de traduction de façon robuste.
+
+## 🚨 Types d'Erreurs
+
+### Erreurs de Configuration
+
+- **`ConfigurationError`** : Clé API manquante
+- **`AuthenticationError`** : Clé API invalide
+
+### Erreurs d'API
+
+- **`RateLimitError`** : Quota dépassé
+- **`ApiError`** : Erreurs serveur (500, 502, 503)
+- **`InvalidResponseError`** : Réponse malformée
+
+### Erreurs de Contenu
+
+- **`EmptyTranslationError`** : Traduction vide
+- **`UnsupportedLanguageError`** : Langue non reconnue
+
+## ⚡ Stratégies de Retry
+
+### Configuration
+
+```ruby
+MistralTranslator.configure do |config|
+  config.retry_delays = [1, 3, 6, 12]  # Délais exponentiels
+end
+```
+
+### Retry par Type d'Erreur
+
+- **Rate Limit** → Délai long (30s, 60s)
+- **Erreur Serveur** → Délai court (2s, 4s)
+- **Auth/Config** → Pas de retry
+- **Response Invalide** → 1-2 tentatives max
+
+## 🔄 Circuit Breaker
+
+### Principe
+
+```
+FERMÉ → OUVERT → DEMI-OUVERT → FERMÉ
+```
+
+- **FERMÉ** : Fonctionnement normal
+- **OUVERT** : Échecs répétés → court-circuite
+- **DEMI-OUVERT** : Test après timeout
+
+### Paramètres
+
+- Seuil : 5 erreurs consécutives
+- Timeout : 5 minutes
+- Reset : 3 succès consécutifs
+
+## 🛡️ Fallback Strategies
+
+### Cascade Recommandée
+
+1. API Mistral
+2. Cache de traductions
+3. Service alternatif
+4. Texte original
+
+### Validation Qualité
+
+- Ratio longueur : 0.3x - 3x de l'original
+- Pas identique à l'original
+- Encodage correct
+
+## 📊 Monitoring
+
+### Métriques Clés
+
+- Taux d'erreur global (< 5%)
+- Types d'erreurs par fréquence
+- Efficacité des fallbacks
+
+### Alertes
+
+```yaml
+Critique:
+  - Auth échoue
+  - Erreurs > 20% sur 5min
+
+Important:
+  - Rate limit > 10/h
+  - Circuit breaker ouvert
+```
+
+## 🎯 Configuration par Environnement
+
+**Development** : Logs verbeux, retry rapides  
+**Test** : Mocks d'erreurs, validation timeouts  
+**Production** : Circuit breaker, fallbacks gracieux
+
+---
+
+**Advanced Usage Navigation:**
+[← Translations](advanced-usage/translations.md) | [Batch Processing](advanced-usage/batch-processing.md) | [Error Handling](advanced-usage/error-handling.md) | [Monitoring](advanced-usage/monitoring.md) | [Summarization](advanced-usage/summarization.md) →

data/docs/advanced-usage/monitoring.md

@@ -0,0 +1,133 @@
+> **Navigation :** [🏠 Home](README.md) • [📖 API Reference](api-reference/methods.md) • [⚡ Advanced Usage](advanced-usage/translations.md) • [🛤️ Rails Integration](rails-integration/setup.md)
+
+---
+
+# Monitoring et Métriques
+
+Surveillez et optimisez vos traductions avec des métriques pertinentes et des alertes intelligentes.
+
+## 📊 Métriques Essentielles
+
+### Configuration de base
+
+```ruby
+MistralTranslator.configure do |config|
+  config.enable_metrics = true
+
+  # Callbacks pour vos systèmes de monitoring
+  config.on_translation_complete = ->(from, to, orig_len, trans_len, duration) {
+    # Intégrer avec votre système (StatsD, DataDog, etc.)
+  }
+end
+```
+
+### Métriques clés à suivre
+
+**Performance :**
+
+- Temps de réponse moyen/médian/P95
+- Throughput (traductions/minute)
+- Taille des textes traités
+
+**Qualité :**
+
+- Taux de succès vs erreurs
+- Types d'erreurs (auth, rate limit, timeout)
+- Score de confiance moyen
+
+**Usage :**
+
+- Paires de langues populaires
+- Volume par heure/jour/mois
+- Coût estimé
+
+## 🚨 Alertes Recommandées
+
+### Alertes Critiques
+
+- **Taux d'erreur > 10%** sur 5 minutes
+- **Temps de réponse > 30s** de façon répétée
+- **Erreurs d'authentification** (problème de clé API)
+
+### Alertes d'Information
+
+- **Rate limit atteint** (ajuster le throttling)
+- **Pic d'usage inhabituel** (analyser la cause)
+- **Nouvelle paire de langues** utilisée
+
+## 📈 Dashboard Suggéré
+
+### Vue d'ensemble
+
+```
+┌─────────────────┬─────────────────┬─────────────────┐
+│ Traductions/h   │ Temps moyen     │ Taux de succès  │
+│      250        │     1.2s        │     99.2%       │
+├─────────────────┼─────────────────┼─────────────────┤
+│ Top langues     │ Erreurs/h       │ Coût du jour    │
+│ fr→en (45%)     │       2         │     $12.34      │
+│ en→es (23%)     │                 │                 │
+└─────────────────┴─────────────────┴─────────────────┘
+```
+
+### Graphiques utiles
+
+- **Timeline** : Volume de traductions dans le temps
+- **Heatmap** : Paires de langues par popularité
+- **Latency** : Distribution des temps de réponse
+- **Errors** : Types d'erreurs par période
+
+## 🔍 Logging Efficace
+
+### Structure de logs recommandée
+
+```
+[TIMESTAMP] [LEVEL] [MistralTranslator] [OPERATION] from=fr to=en chars=150 duration=1.2s status=success
+[TIMESTAMP] [ERROR] [MistralTranslator] [TRANSLATE] from=fr to=en error=rate_limit attempt=2
+```
+
+### Niveaux de log par environnement
+
+- **Development** : DEBUG (tout)
+- **Staging** : INFO (succès + erreurs)
+- **Production** : WARN (erreurs + rate limits)
+
+## ⚡ Optimisation Basée sur les Métriques
+
+### Patterns à identifier
+
+**Rate limiting :**
+
+- Si beaucoup de rate limits → ajuster les délais
+- Répartir les requêtes dans le temps
+
+**Performance :**
+
+- Textes longs = temps longs → découper si possible
+- Certaines paires de langues plus lentes
+
+**Usage :**
+
+- Cache les traductions populaires
+- Pre-traduire le contenu critique
+
+### Seuils d'alerte suggérés
+
+```yaml
+performance:
+  response_time_p95: 10s
+  error_rate_5min: 5%
+
+capacity:
+  requests_per_minute: 50
+  daily_cost: $100
+
+quality:
+  confidence_score_avg: 0.7
+  empty_translations: 1%
+```
+
+---
+
+**Advanced Usage Navigation:**
+[← Translations](advanced-usage/translations.md) | [Batch Processing](advanced-usage/batch-processing.md) | [Error Handling](advanced-usage/error-handling.md) | [Monitoring](advanced-usage/monitoring.md) | [Summarization](advanced-usage/summarization.md) →

data/docs/advanced-usage/summarization.md

@@ -0,0 +1,86 @@
+> **Navigation :** [🏠 Home](README.md) • [📖 API Reference](api-reference/methods.md) • [⚡ Advanced Usage](advanced-usage/translations.md) • [🛤️ Rails Integration](rails-integration/setup.md)
+
+---
+
+# Résumés Intelligents
+
+Résumés automatiques, multi-niveaux et multilingues.
+
+## 📝 Résumé Simple
+
+```ruby
+summarizer = MistralTranslator::Summarizer.new
+
+summary = summarizer.summarize(
+  long_text,
+  language: "fr",
+  max_words: 100
+)
+```
+
+## 🌍 Résumé + Traduction
+
+```ruby
+# Résume ET traduit en une opération
+french_summary = summarizer.summarize_and_translate(
+  english_article,
+  from: "en",
+  to: "fr",
+  max_words: 150
+)
+```
+
+## 📊 Multi-niveaux
+
+```ruby
+summaries = summarizer.summarize_tiered(
+  article,
+  language: "fr",
+  short: 50,    # Tweet
+  medium: 150,  # Paragraphe
+  long: 400     # Article court
+)
+
+puts summaries[:short]
+puts summaries[:medium]
+puts summaries[:long]
+```
+
+## 🗺️ Multi-langues
+
+```ruby
+results = summarizer.summarize_to_multiple(
+  document,
+  languages: ["fr", "en", "es"],
+  max_words: 200
+)
+# => { "fr" => "résumé...", "en" => "summary...", "es" => "resumen..." }
+```
+
+## 🎯 Styles de Résumé
+
+```ruby
+# Différents styles selon l'usage
+summarizer.summarize(content, context: "Executive summary, key metrics")
+summarizer.summarize(content, context: "Social media, engaging tone")
+summarizer.summarize(content, context: "Technical documentation")
+```
+
+## 📈 Longueurs Recommandées
+
+**Selon taille du contenu :**
+
+- 0-200 mots → Résumé 30-50 mots
+- 200-800 mots → Résumé 50-100 mots
+- 800+ mots → Résumé 100-200 mots
+
+**Selon usage :**
+
+- Tweet → 50 mots max
+- Meta description → 150 mots max
+- Résumé exécutif → 200-400 mots
+
+---
+
+**Advanced Usage Navigation:**
+[← Translations](advanced-usage/translations.md) | [Batch Processing](advanced-usage/batch-processing.md) | [Error Handling](advanced-usage/error-handling.md) | [Monitoring](advanced-usage/monitoring.md) | [Summarization](advanced-usage/summarization.md) →