mistral_translator 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +33 -1
- data/README.md +6 -1
- data/docs/api-reference/configuration.md +25 -2
- data/docs/getting-started.md +2 -0
- data/docs/installation.md +0 -2
- data/docs/rails-integration/setup.md +0 -1
- data/lib/mistral_translator/client_helpers.rb +2 -2
- data/lib/mistral_translator/configuration.rb +24 -4
- data/lib/mistral_translator/deprecated_options.rb +45 -0
- data/lib/mistral_translator/helpers.rb +0 -5
- data/lib/mistral_translator/helpers_extensions.rb +0 -58
- data/lib/mistral_translator/prompt_builder.rb +0 -94
- data/lib/mistral_translator/prompt_helpers.rb +0 -23
- data/lib/mistral_translator/prompt_metadata_helpers.rb +0 -5
- data/lib/mistral_translator/response_parser.rb +53 -110
- data/lib/mistral_translator/retries.rb +50 -0
- data/lib/mistral_translator/summarizer.rb +13 -50
- data/lib/mistral_translator/translator.rb +9 -23
- data/lib/mistral_translator/translator_helpers.rb +0 -89
- data/lib/mistral_translator/version.rb +1 -1
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 22b02e4c378ffdd992bbb525c57a03fd15c4c82e2ee916c43a3753b09398c8e0
|
|
4
|
+
data.tar.gz: 1bdf19928ac211fbbc7e7fe064f600d53639c3b0aa57063b14b74fe3b95384c0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 67bb96d4142aa345f2feb2f1cf35ef30e131d529ad22338e9bd31c0c237d165a446b74e8ed75aa7715d903e486657250fb823e4add5994edade18d763fa50e2f
|
|
7
|
+
data.tar.gz: 39fcfd92f88aa10bb1ca930a8c9539a9d3e12ce7a8b4d1564d2cd0336ac5f349d9903334abec4a7f0d8fe8876a90d82738d8e6bc2a024bc91df5303c8f096588
|
data/CHANGELOG.md
CHANGED
|
@@ -1,4 +1,36 @@
|
|
|
1
|
-
## [
|
|
1
|
+
## [0.5.0] - 2026-10-29
|
|
2
|
+
|
|
3
|
+
### Added
|
|
4
|
+
|
|
5
|
+
- `config.read_timeout`: how long to wait for the API's answer, in seconds. Defaults to 60, the value the gem used until now; anything but a positive number raises `ConfigurationError` when it is set.
|
|
6
|
+
- `config.open_timeout`: how long to wait for the connection to open (TCP, then TLS), in seconds. Defaults to 60, validated the same way.
|
|
7
|
+
|
|
8
|
+
### Changed
|
|
9
|
+
|
|
10
|
+
No change to translation results:
|
|
11
|
+
|
|
12
|
+
- Translations and summaries share one retry policy, with the same delays and counts as before. Their retry log lines now share one format, such as `translation retry 1/3 for fr->en in 2s: …`.
|
|
13
|
+
- The `Summarizer` no longer prints to stdout when `MISTRAL_TRANSLATOR_TEST_OUTPUT` is set.
|
|
14
|
+
- A response is parsed once instead of twice.
|
|
15
|
+
|
|
16
|
+
### Deprecated
|
|
17
|
+
|
|
18
|
+
These options still work, print a warning, and will be removed in 1.0:
|
|
19
|
+
|
|
20
|
+
- `config.ssl_timeout`, the former name of `open_timeout`, which it still sets. It is now validated like `open_timeout`, so `nil` (no limit) is rejected.
|
|
21
|
+
- `config.default_max_tokens` and `config.default_temperature`, which the gem never read.
|
|
22
|
+
|
|
23
|
+
### Removed
|
|
24
|
+
|
|
25
|
+
Undocumented code that the gem never called:
|
|
26
|
+
|
|
27
|
+
- the `HelpersExtensions` copies of `translate_with_progress`, `translate_multi_style` and `find_locale_suggestions`, which `Helpers` redefines;
|
|
28
|
+
- `TranslatorHelpers::RetryHandler`, `TranslatorHelpers::LoggingHelper` and `InputValidator#validate_batch_inputs!`;
|
|
29
|
+
- `PromptBuilder.bulk_translation_prompt`, `PromptBuilder.tiered_summary_prompt`, `PromptHelpers::FormatInstructions`, `ResponseParser.parse_bulk_translation_response` and `ResponseParser::JsonParserState`.
|
|
30
|
+
|
|
31
|
+
### Security
|
|
32
|
+
|
|
33
|
+
- When the scan for a JSON object inside text reached its limit, the error kept the `JSON::ParserError` as its `cause`, and that error quotes the response.
|
|
2
34
|
|
|
3
35
|
## [0.4.0] - 2026-10-28
|
|
4
36
|
|
data/README.md
CHANGED
|
@@ -37,7 +37,7 @@ MistralTranslator.translate("Bonjour le monde", from: "fr", to: "en")
|
|
|
37
37
|
|
|
38
38
|
## Supported Languages
|
|
39
39
|
|
|
40
|
-
All 182 ISO 639-1 languages named by CLDR (`fr`, `en`, `es`, `de`, `ja`, `br`, `wo`, `mg`…). `MistralTranslator.supported_locales` lists them, and English or French names are accepted in place of codes (`"German"`, `"allemand"`).
|
|
40
|
+
All 182 ISO 639-1 languages named by CLDR (`fr`, `en`, `es`, `de`, `ja`, `br`, `wo`, `mg`…). `MistralTranslator.supported_locales` lists them, and English or French names are accepted in place of codes (`"German"`, `"allemand"`). The list and the names come from the CLDR data of macOS 27.2, through `bin/generate_languages` (2026-10-28).
|
|
41
41
|
|
|
42
42
|
Translation quality depends on the Mistral model: excellent for widely used languages, weaker for rare ones. Check the output before relying on a rare language.
|
|
43
43
|
|
|
@@ -166,6 +166,9 @@ MistralTranslator.configure do |config|
|
|
|
166
166
|
config.enable_metrics = true # Enable performance metrics
|
|
167
167
|
config.proxy = :env # HTTP proxy: nil (default), "http://host:port", or :env
|
|
168
168
|
# to read HTTPS_PROXY / HTTP_PROXY and NO_PROXY
|
|
169
|
+
config.open_timeout = 60 # Seconds to open the connection (default 60)
|
|
170
|
+
config.read_timeout = 60 # Seconds to wait for the API's answer (default 60);
|
|
171
|
+
# raise it for long texts or summaries
|
|
169
172
|
|
|
170
173
|
# Callbacks for monitoring
|
|
171
174
|
config.on_translation_complete = ->(from, to, orig_len, trans_len, duration) {
|
|
@@ -178,6 +181,8 @@ MistralTranslator.configure do |config|
|
|
|
178
181
|
end
|
|
179
182
|
```
|
|
180
183
|
|
|
184
|
+
Configure the gem before the first call. The module-level methods (`MistralTranslator.translate`, `summarize`…) share one client, which reads `api_key`, `model`, `api_url`, `retry_delays` and the connection settings (`read_timeout`, `proxy`, SSL) on the first call. To change them afterwards, call `MistralTranslator.reset_configuration!` and configure again, or create a `MistralTranslator::Translator.new` (or `Summarizer.new`), which builds its own client from the current configuration.
|
|
185
|
+
|
|
181
186
|
## Error Handling
|
|
182
187
|
|
|
183
188
|
```ruby
|
|
@@ -13,14 +13,37 @@ MistralTranslator.configure do |config|
|
|
|
13
13
|
config.api_key = ENV['MISTRAL_API_KEY'] # Obligatoire
|
|
14
14
|
config.model = "mistral-small-latest" # Défaut
|
|
15
15
|
config.api_url = "https://api.mistral.ai" # URL API
|
|
16
|
-
config.default_max_tokens = 4000 # Limite tokens
|
|
17
|
-
config.default_temperature = 0.3 # Créativité (0-1)
|
|
18
16
|
config.retry_delays = [2, 4, 8, 16, 32] # Délais retry (secondes)
|
|
19
17
|
config.enable_metrics = false # Suivi des stats
|
|
20
18
|
config.proxy = nil # Proxy HTTP (désactivé par défaut)
|
|
19
|
+
config.open_timeout = 60 # Ouverture de la connexion (secondes)
|
|
20
|
+
config.read_timeout = 60 # Attente maximale de la réponse (secondes)
|
|
21
21
|
end
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
+
Configurez la gem avant le premier appel. Les méthodes du module (`MistralTranslator.translate`, `summarize`…) partagent un client, qui lit `api_key`, `model`, `api_url`, `retry_delays` et les réglages de connexion (`read_timeout`, `proxy`, SSL) lors du premier appel. Pour les changer ensuite, appelez `MistralTranslator.reset_configuration!` puis configurez à nouveau, ou créez un `MistralTranslator::Translator.new` (ou `Summarizer.new`), qui construit son propre client avec la configuration du moment.
|
|
25
|
+
|
|
26
|
+
## Délais
|
|
27
|
+
|
|
28
|
+
Deux délais, en secondes, 60 par défaut :
|
|
29
|
+
|
|
30
|
+
- `open_timeout` : ouverture de la connexion à l'API (TCP puis TLS) ;
|
|
31
|
+
- `read_timeout` : attente de la réponse. Un long texte ou un résumé détaillé peut demander plus au modèle ; augmentez-le dans ce cas.
|
|
32
|
+
|
|
33
|
+
```ruby
|
|
34
|
+
config.read_timeout = 180
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
- Seul un nombre strictement positif est accepté (entier ou décimal) ; toute autre valeur lève `ConfigurationError` dès l'affectation.
|
|
38
|
+
- Délai de réponse dépassé : la requête lève `ApiError` (« Request timeout »).
|
|
39
|
+
|
|
40
|
+
## Options obsolètes
|
|
41
|
+
|
|
42
|
+
Elles fonctionnent encore, avec un avertissement, et seront retirées en 1.0 :
|
|
43
|
+
|
|
44
|
+
- `ssl_timeout` : ancien nom d'`open_timeout`, qu'il règle toujours ;
|
|
45
|
+
- `default_max_tokens` et `default_temperature` : jamais lues par la gem, elles n'ont aucun effet. Pour régler ces paramètres, passez `max_tokens:` et `temperature:` à `Client#chat` ou `Client#complete`.
|
|
46
|
+
|
|
24
47
|
## Proxy
|
|
25
48
|
|
|
26
49
|
Désactivé par défaut. Trois valeurs possibles :
|
data/docs/getting-started.md
CHANGED
|
@@ -52,6 +52,8 @@ MistralTranslator::LocaleHelper.locale_supported?("ja")
|
|
|
52
52
|
# => true
|
|
53
53
|
```
|
|
54
54
|
|
|
55
|
+
La liste et les noms viennent des données CLDR de macOS 27.2, via `bin/generate_languages` (2026-10-28).
|
|
56
|
+
|
|
55
57
|
La qualité de traduction dépend du modèle Mistral : excellente pour les langues les plus parlées, plus faible pour les langues rares. Vérifiez le résultat avant de vous appuyer sur une langue rare.
|
|
56
58
|
|
|
57
59
|
## 🚨 Gestion d'Erreurs Simple
|
data/docs/installation.md
CHANGED
|
@@ -79,8 +79,6 @@ MistralTranslator.configure do |config|
|
|
|
79
79
|
config.model = "mistral-small-latest" # Modèle par défaut
|
|
80
80
|
config.retry_delays = [2, 4, 8, 16] # Délais retry
|
|
81
81
|
config.enable_metrics = true # Suivi des stats
|
|
82
|
-
config.default_max_tokens = 4000 # Limite tokens
|
|
83
|
-
config.default_temperature = 0.3 # Créativité
|
|
84
82
|
end
|
|
85
83
|
```
|
|
86
84
|
|
|
@@ -272,7 +272,6 @@ MistralTranslator.configure do |config|
|
|
|
272
272
|
|
|
273
273
|
# Optimisation Rails
|
|
274
274
|
config.enable_metrics = true
|
|
275
|
-
config.default_max_tokens = 4000
|
|
276
275
|
|
|
277
276
|
# Cache intelligent des traductions
|
|
278
277
|
config.on_translation_complete = ->(from, to, orig_len, trans_len, duration) {
|
|
@@ -30,9 +30,9 @@ module MistralTranslator
|
|
|
30
30
|
def http_pool
|
|
31
31
|
@http_pool ||= begin
|
|
32
32
|
pool = Net::HTTP::Persistent.new(name: "mistral_translator")
|
|
33
|
-
pool.read_timeout =
|
|
33
|
+
pool.read_timeout = MistralTranslator.configuration.read_timeout
|
|
34
34
|
pool.idle_timeout = 30
|
|
35
|
-
pool.open_timeout = MistralTranslator.configuration.
|
|
35
|
+
pool.open_timeout = MistralTranslator.configuration.open_timeout
|
|
36
36
|
pool.max_requests = 100
|
|
37
37
|
|
|
38
38
|
# SSL/TLS configuration
|
|
@@ -1,14 +1,17 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "proxy"
|
|
4
|
+
require_relative "deprecated_options"
|
|
4
5
|
|
|
5
6
|
module MistralTranslator
|
|
6
7
|
class Configuration
|
|
7
|
-
|
|
8
|
+
include DeprecatedOptions
|
|
9
|
+
|
|
10
|
+
attr_accessor :api_key, :model, :retry_delays,
|
|
8
11
|
:on_translation_start, :on_translation_complete, :on_translation_error,
|
|
9
12
|
:on_rate_limit, :on_batch_complete, :enable_metrics,
|
|
10
|
-
:ssl_verify_mode, :ssl_ca_file, :ssl_ca_path
|
|
11
|
-
attr_reader :api_url, :proxy
|
|
13
|
+
:ssl_verify_mode, :ssl_ca_file, :ssl_ca_path
|
|
14
|
+
attr_reader :api_url, :proxy, :open_timeout, :read_timeout
|
|
12
15
|
|
|
13
16
|
def initialize
|
|
14
17
|
@api_key = nil
|
|
@@ -30,7 +33,10 @@ module MistralTranslator
|
|
|
30
33
|
@ssl_verify_mode = :peer # :peer, :none, or OpenSSL constant
|
|
31
34
|
@ssl_ca_file = nil # Path to CA certificate file
|
|
32
35
|
@ssl_ca_path = nil # Path to CA certificates directory
|
|
33
|
-
|
|
36
|
+
|
|
37
|
+
# Délais en secondes : ouverture de la connexion (TCP puis TLS), puis attente de la réponse
|
|
38
|
+
@open_timeout = 60
|
|
39
|
+
@read_timeout = 60
|
|
34
40
|
|
|
35
41
|
# Proxy : nil (aucun), URI ou String, ou :env (HTTPS_PROXY, HTTP_PROXY, NO_PROXY)
|
|
36
42
|
@proxy = nil
|
|
@@ -66,6 +72,14 @@ module MistralTranslator
|
|
|
66
72
|
@proxy = value
|
|
67
73
|
end
|
|
68
74
|
|
|
75
|
+
def open_timeout=(seconds)
|
|
76
|
+
@open_timeout = positive_seconds!(:open_timeout, seconds)
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def read_timeout=(seconds)
|
|
80
|
+
@read_timeout = positive_seconds!(:read_timeout, seconds)
|
|
81
|
+
end
|
|
82
|
+
|
|
69
83
|
# Méthodes pour les callbacks
|
|
70
84
|
def trigger_translation_start(from_locale, to_locale, text_length)
|
|
71
85
|
@on_translation_start&.call(from_locale, to_locale, text_length, Time.now)
|
|
@@ -176,6 +190,12 @@ module MistralTranslator
|
|
|
176
190
|
|
|
177
191
|
private
|
|
178
192
|
|
|
193
|
+
def positive_seconds!(name, seconds)
|
|
194
|
+
return seconds if seconds.is_a?(Numeric) && seconds.real? && seconds.positive?
|
|
195
|
+
|
|
196
|
+
raise ConfigurationError, "#{name} must be a positive number of seconds, got: #{seconds.inspect}"
|
|
197
|
+
end
|
|
198
|
+
|
|
179
199
|
def validate_api_url(url)
|
|
180
200
|
uri = parse_uri(url)
|
|
181
201
|
validate_uri_scheme(uri, url)
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module MistralTranslator
|
|
4
|
+
# Options de configuration obsolètes, retirées en 1.0 : ssl_timeout a toujours réglé
|
|
5
|
+
# l'ouverture de la connexion, default_max_tokens et default_temperature n'ont jamais été lus.
|
|
6
|
+
module DeprecatedOptions
|
|
7
|
+
def ssl_timeout
|
|
8
|
+
deprecated(:ssl_timeout, "use open_timeout")
|
|
9
|
+
open_timeout
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def ssl_timeout=(seconds)
|
|
13
|
+
deprecated(:ssl_timeout, "use open_timeout")
|
|
14
|
+
self.open_timeout = seconds
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def default_max_tokens
|
|
18
|
+
deprecated(:default_max_tokens, "it has no effect")
|
|
19
|
+
@default_max_tokens
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def default_max_tokens=(value)
|
|
23
|
+
deprecated(:default_max_tokens, "it has no effect")
|
|
24
|
+
@default_max_tokens = value
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def default_temperature
|
|
28
|
+
deprecated(:default_temperature, "it has no effect")
|
|
29
|
+
@default_temperature
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def default_temperature=(value)
|
|
33
|
+
deprecated(:default_temperature, "it has no effect")
|
|
34
|
+
@default_temperature = value
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
private
|
|
38
|
+
|
|
39
|
+
# L'avertissement désigne la ligne de l'appelant, pas celle de la gem
|
|
40
|
+
def deprecated(name, advice)
|
|
41
|
+
Kernel.warn("[MistralTranslator] config.#{name} is deprecated and will be removed in 1.0: #{advice}",
|
|
42
|
+
uplevel: 2)
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
@@ -252,11 +252,6 @@ module MistralTranslator
|
|
|
252
252
|
|
|
253
253
|
LevenshteinHelpers.levenshtein_distance(input, name) <= (name.length >= 8 ? 2 : 1)
|
|
254
254
|
end
|
|
255
|
-
|
|
256
|
-
# Exposer la distance de Levenshtein en privé via délégation pour les tests
|
|
257
|
-
def levenshtein_distance(source_string, target_string)
|
|
258
|
-
LevenshteinHelpers.levenshtein_distance(source_string, target_string)
|
|
259
|
-
end
|
|
260
255
|
end
|
|
261
256
|
|
|
262
257
|
# Module pour inclure dans les classes Rails si souhaité
|
|
@@ -25,43 +25,6 @@ module MistralTranslator
|
|
|
25
25
|
opts = { context: options[:context], glossary: options[:glossary] }
|
|
26
26
|
MistralTranslator.translate(text, from: from, to: to, preserve_html: true, **opts)
|
|
27
27
|
end
|
|
28
|
-
|
|
29
|
-
def translate_with_progress(items, from:, to:, **options)
|
|
30
|
-
results = {}
|
|
31
|
-
total = items.size
|
|
32
|
-
processed = 0
|
|
33
|
-
|
|
34
|
-
items.each do |key, text|
|
|
35
|
-
results[key] = MistralTranslator.translate(text, from: from, to: to, **options)
|
|
36
|
-
processed += 1
|
|
37
|
-
yield(processed, total, key) if block_given?
|
|
38
|
-
end
|
|
39
|
-
|
|
40
|
-
results
|
|
41
|
-
end
|
|
42
|
-
|
|
43
|
-
def translate_multi_style(text, from:, to:, **options)
|
|
44
|
-
results = {}
|
|
45
|
-
|
|
46
|
-
styles = options[:styles] || %i[formal casual]
|
|
47
|
-
styles.each do |style|
|
|
48
|
-
style_context = options[:context] ? "#{options[:context]} (Style: #{style})" : "Style: #{style}"
|
|
49
|
-
|
|
50
|
-
begin
|
|
51
|
-
results[style] = MistralTranslator.translate(
|
|
52
|
-
text,
|
|
53
|
-
from: from,
|
|
54
|
-
to: to,
|
|
55
|
-
context: style_context,
|
|
56
|
-
glossary: options[:glossary]
|
|
57
|
-
)
|
|
58
|
-
rescue StandardError => e
|
|
59
|
-
results[style] = { error: e.message }
|
|
60
|
-
end
|
|
61
|
-
end
|
|
62
|
-
|
|
63
|
-
results
|
|
64
|
-
end
|
|
65
28
|
end
|
|
66
29
|
|
|
67
30
|
# Extensions pour les helpers d'analyse
|
|
@@ -103,27 +66,6 @@ module MistralTranslator
|
|
|
103
66
|
compression_ratio: (optimal_words.to_f / word_count * 100).round(1)
|
|
104
67
|
}
|
|
105
68
|
end
|
|
106
|
-
|
|
107
|
-
def find_locale_suggestions(invalid_locale)
|
|
108
|
-
return [] unless invalid_locale.is_a?(String)
|
|
109
|
-
|
|
110
|
-
supported = LocaleHelper.supported_locales.map(&:to_s)
|
|
111
|
-
|
|
112
|
-
# Recherche par similarité basique
|
|
113
|
-
suggestions = supported.select do |locale|
|
|
114
|
-
locale.start_with?(invalid_locale.downcase) ||
|
|
115
|
-
invalid_locale.downcase.start_with?(locale)
|
|
116
|
-
end
|
|
117
|
-
|
|
118
|
-
# Si pas de suggestions par préfixe, chercher par distance
|
|
119
|
-
if suggestions.empty?
|
|
120
|
-
suggestions = supported.select do |locale|
|
|
121
|
-
levenshtein_distance(invalid_locale.downcase, locale) <= 2
|
|
122
|
-
end
|
|
123
|
-
end
|
|
124
|
-
|
|
125
|
-
suggestions.first(3) # Limiter à 3 suggestions
|
|
126
|
-
end
|
|
127
69
|
end
|
|
128
70
|
|
|
129
71
|
# Extensions pour les helpers de coût
|
|
@@ -9,7 +9,6 @@ module MistralTranslator
|
|
|
9
9
|
module PromptBuilder
|
|
10
10
|
extend PromptHelpers::ContextBuilder
|
|
11
11
|
extend PromptHelpers::HtmlInstructions
|
|
12
|
-
extend PromptHelpers::FormatInstructions
|
|
13
12
|
extend PromptMetadataHelpers
|
|
14
13
|
|
|
15
14
|
class << self
|
|
@@ -49,55 +48,6 @@ module MistralTranslator
|
|
|
49
48
|
PROMPT
|
|
50
49
|
end
|
|
51
50
|
|
|
52
|
-
# rubocop:disable Metrics/MethodLength
|
|
53
|
-
def bulk_translation_prompt(texts, source_language, target_language, **options)
|
|
54
|
-
source_name = LocaleHelper.prompt_language_name(source_language)
|
|
55
|
-
target_name = LocaleHelper.prompt_language_name(target_language)
|
|
56
|
-
|
|
57
|
-
context_section = build_context_section(options[:context], options[:glossary])
|
|
58
|
-
html_instruction = options[:preserve_html] ? build_html_preservation_instruction : ""
|
|
59
|
-
|
|
60
|
-
# Extraire les valeurs pour les métadonnées
|
|
61
|
-
context = options[:context]
|
|
62
|
-
glossary = options[:glossary]
|
|
63
|
-
preserve_html = options[:preserve_html]
|
|
64
|
-
|
|
65
|
-
Prompt.new(instructions: <<~PROMPT, text: numbered_texts(texts))
|
|
66
|
-
Tu es un traducteur professionnel. Traduis les textes fournis, numérotés.
|
|
67
|
-
Langue source : #{source_name}. Langue cible : #{target_name}.
|
|
68
|
-
#{context_section}
|
|
69
|
-
RÈGLES :
|
|
70
|
-
- Traduis fidèlement chaque texte sans ajouter d'informations
|
|
71
|
-
- Conserve le style, ton et format originaux#{html_instruction}
|
|
72
|
-
- Maintiens la cohérence terminologique entre tous les textes
|
|
73
|
-
- #{Prompt::DATA_RULE}
|
|
74
|
-
- Réponds UNIQUEMENT en JSON valide
|
|
75
|
-
|
|
76
|
-
FORMAT OBLIGATOIRE :
|
|
77
|
-
{
|
|
78
|
-
"translations": [
|
|
79
|
-
{
|
|
80
|
-
"index": 1,
|
|
81
|
-
"source": "texte original 1",
|
|
82
|
-
"target": "texte traduit 1"
|
|
83
|
-
},
|
|
84
|
-
{
|
|
85
|
-
"index": 2,
|
|
86
|
-
"source": "texte original 2",
|
|
87
|
-
"target": "texte traduit 2"
|
|
88
|
-
}
|
|
89
|
-
],
|
|
90
|
-
"metadata": {
|
|
91
|
-
"source_language": "#{source_language}",
|
|
92
|
-
"target_language": "#{target_language}",
|
|
93
|
-
"count": #{texts.length},
|
|
94
|
-
"operation": "bulk_translation"#{build_metadata_additions(context, glossary, preserve_html)}
|
|
95
|
-
}
|
|
96
|
-
}
|
|
97
|
-
PROMPT
|
|
98
|
-
end
|
|
99
|
-
# rubocop:enable Metrics/MethodLength
|
|
100
|
-
|
|
101
51
|
def summary_prompt(text, max_words, target_language = "fr", context: nil, style: nil)
|
|
102
52
|
target_name = LocaleHelper.prompt_language_name(target_language)
|
|
103
53
|
|
|
@@ -168,50 +118,6 @@ module MistralTranslator
|
|
|
168
118
|
PROMPT
|
|
169
119
|
end
|
|
170
120
|
|
|
171
|
-
def tiered_summary_prompt(text, target_language, **options)
|
|
172
|
-
target_name = LocaleHelper.prompt_language_name(target_language)
|
|
173
|
-
|
|
174
|
-
context_section = build_summary_context_section(options[:context], options[:style])
|
|
175
|
-
style_instruction = build_style_instruction(options[:style])
|
|
176
|
-
|
|
177
|
-
# Extraire les valeurs pour les métadonnées
|
|
178
|
-
context = options[:context]
|
|
179
|
-
style = options[:style]
|
|
180
|
-
|
|
181
|
-
Prompt.new(instructions: <<~PROMPT, text: text)
|
|
182
|
-
Tu es un rédacteur professionnel. Crée trois résumés du texte fourni en #{target_name}.
|
|
183
|
-
#{context_section}
|
|
184
|
-
RÈGLES :
|
|
185
|
-
- Résume fidèlement sans ajouter d'informations
|
|
186
|
-
- Respecte strictement les longueurs demandées
|
|
187
|
-
- Court: #{options[:short]} mots, Moyen: #{options[:medium]} mots, Long: #{options[:long]} mots#{style_instruction}
|
|
188
|
-
- #{Prompt::DATA_RULE}
|
|
189
|
-
- Réponds UNIQUEMENT en JSON valide
|
|
190
|
-
|
|
191
|
-
FORMAT OBLIGATOIRE :
|
|
192
|
-
{
|
|
193
|
-
"content": {
|
|
194
|
-
"source": "texte original",
|
|
195
|
-
"summaries": {
|
|
196
|
-
"short": "résumé court (#{options[:short]} mots)",
|
|
197
|
-
"medium": "résumé moyen (#{options[:medium]} mots)",
|
|
198
|
-
"long": "résumé long (#{options[:long]} mots)"
|
|
199
|
-
}
|
|
200
|
-
},
|
|
201
|
-
"metadata": {
|
|
202
|
-
"source_language": "original",
|
|
203
|
-
"target_language": "#{target_language}",
|
|
204
|
-
"summaries": {
|
|
205
|
-
"short": #{options[:short]},
|
|
206
|
-
"medium": #{options[:medium]},
|
|
207
|
-
"long": #{options[:long]}
|
|
208
|
-
},
|
|
209
|
-
"operation": "tiered_summarization"#{build_summary_metadata_additions(context, style)}
|
|
210
|
-
}
|
|
211
|
-
}
|
|
212
|
-
PROMPT
|
|
213
|
-
end
|
|
214
|
-
|
|
215
121
|
def language_detection_prompt(text, confidence_score: false)
|
|
216
122
|
confidence_instruction = if confidence_score
|
|
217
123
|
', "confidence": score_de_confiance_entre_0_et_1'
|
|
@@ -69,28 +69,5 @@ module MistralTranslator
|
|
|
69
69
|
HTML_INSTRUCTION
|
|
70
70
|
end
|
|
71
71
|
end
|
|
72
|
-
|
|
73
|
-
# Helper pour les instructions de formatage
|
|
74
|
-
module FormatInstructions
|
|
75
|
-
def build_json_format_instruction
|
|
76
|
-
<<~JSON_INSTRUCTION
|
|
77
|
-
|
|
78
|
-
FORMAT DE RÉPONSE : Réponds UNIQUEMENT avec un objet JSON valide.
|
|
79
|
-
Pas de texte avant ou après le JSON.
|
|
80
|
-
JSON_INSTRUCTION
|
|
81
|
-
end
|
|
82
|
-
|
|
83
|
-
def numbered_texts(texts)
|
|
84
|
-
texts.map.with_index { |text, i| "#{i + 1}. #{text}" }.join("\n")
|
|
85
|
-
end
|
|
86
|
-
|
|
87
|
-
def build_batch_format_instruction
|
|
88
|
-
<<~BATCH_INSTRUCTION
|
|
89
|
-
|
|
90
|
-
FORMAT DE RÉPONSE : Réponds avec un tableau JSON contenant les traductions dans l'ordre.
|
|
91
|
-
Chaque élément doit être la traduction correspondante.
|
|
92
|
-
BATCH_INSTRUCTION
|
|
93
|
-
end
|
|
94
|
-
end
|
|
95
72
|
end
|
|
96
73
|
end
|
|
@@ -32,10 +32,5 @@ module MistralTranslator
|
|
|
32
32
|
|
|
33
33
|
additions.any? ? ",\n #{additions.join(",\n ")}" : ""
|
|
34
34
|
end
|
|
35
|
-
|
|
36
|
-
def log_prompt_generation(prompt_type, source_locale, target_locale)
|
|
37
|
-
message = "Generated #{prompt_type} prompt for #{source_locale} -> #{target_locale}"
|
|
38
|
-
Logger.debug_if_verbose(message, sensitive: false)
|
|
39
|
-
end
|
|
40
35
|
end
|
|
41
36
|
end
|
|
@@ -16,6 +16,11 @@ module MistralTranslator
|
|
|
16
16
|
%w[content original], %w[original]
|
|
17
17
|
].freeze
|
|
18
18
|
|
|
19
|
+
# Garde-fous contre les réponses démesurées
|
|
20
|
+
MAX_CONTENT_SIZE = 1_000_000
|
|
21
|
+
MAX_SCAN_LENGTH = 100_000
|
|
22
|
+
BRACE_DEPTH = { "{" => 1, "}" => -1 }.freeze
|
|
23
|
+
|
|
19
24
|
class << self
|
|
20
25
|
def parse_translation_response(raw_content)
|
|
21
26
|
return nil if raw_content.nil? || raw_content.empty?
|
|
@@ -56,25 +61,6 @@ module MistralTranslator
|
|
|
56
61
|
end
|
|
57
62
|
end
|
|
58
63
|
|
|
59
|
-
def parse_bulk_translation_response(raw_content)
|
|
60
|
-
return [] if raw_content.nil? || raw_content.empty?
|
|
61
|
-
|
|
62
|
-
with_response_errors("bulk", raw_content) do
|
|
63
|
-
translations = parse_response_json(raw_content, "bulk")["translations"]
|
|
64
|
-
unless translations.is_a?(Array)
|
|
65
|
-
raise InvalidResponseError.new("No translations array in response", raw_content)
|
|
66
|
-
end
|
|
67
|
-
|
|
68
|
-
translations.map do |translation|
|
|
69
|
-
{
|
|
70
|
-
index: translation["index"],
|
|
71
|
-
original: translation["source"],
|
|
72
|
-
translated: translation["target"]
|
|
73
|
-
}
|
|
74
|
-
end
|
|
75
|
-
end
|
|
76
|
-
end
|
|
77
|
-
|
|
78
64
|
def parse_json_content(json_content)
|
|
79
65
|
JSON.parse(json_content)
|
|
80
66
|
rescue JSON::ParserError
|
|
@@ -111,118 +97,75 @@ module MistralTranslator
|
|
|
111
97
|
raise InvalidResponseError.new("Error processing #{label} response (#{e.class})", raw_content), cause: nil
|
|
112
98
|
end
|
|
113
99
|
|
|
100
|
+
# Le mode JSON de l'API renvoie un objet seul, lu tel quel. Sinon, on lit le premier objet
|
|
101
|
+
# trouvé dans le texte, en réparant les continuations de ligne.
|
|
114
102
|
def parse_response_json(raw_content, label)
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
data = parse_json_content(json_content)
|
|
103
|
+
ensure_content_size!(raw_content)
|
|
104
|
+
data = parse_whole_or_embedded(raw_content, label)
|
|
119
105
|
raise InvalidResponseError.new("Unexpected JSON in #{label} response", raw_content) unless data.is_a?(Hash)
|
|
120
106
|
|
|
121
107
|
data
|
|
122
|
-
rescue JSON::ParserError
|
|
123
|
-
details = "raw_len=#{raw_content.length} json_len=#{json_content&.length}"
|
|
124
|
-
Logger.debug_if_verbose("JSON parse failed: #{details}", sensitive: false)
|
|
125
|
-
raise InvalidResponseError.new("Invalid JSON in #{label} response (#{details})", raw_content), cause: nil
|
|
126
108
|
end
|
|
127
109
|
|
|
128
|
-
def
|
|
129
|
-
return
|
|
130
|
-
|
|
131
|
-
# Limiter la taille pour éviter les attaques DoS
|
|
132
|
-
max_content_size = 1_000_000 # 1MB max
|
|
133
|
-
if content.length > max_content_size
|
|
134
|
-
raise InvalidResponseError, "Response content too large (#{content.length} bytes, max: #{max_content_size})"
|
|
135
|
-
end
|
|
110
|
+
def ensure_content_size!(content)
|
|
111
|
+
return if content.length <= MAX_CONTENT_SIZE
|
|
136
112
|
|
|
137
|
-
|
|
138
|
-
begin
|
|
139
|
-
JSON.parse(content)
|
|
140
|
-
content
|
|
141
|
-
rescue JSON::ParserError
|
|
142
|
-
# Si ça échoue, chercher le JSON dans la réponse (peut être entouré de texte)
|
|
143
|
-
# Utiliser une approche plus robuste pour les JSON complexes
|
|
144
|
-
find_json_in_text(content)
|
|
145
|
-
end
|
|
113
|
+
raise InvalidResponseError, "Response content too large (#{content.length} bytes, max: #{MAX_CONTENT_SIZE})"
|
|
146
114
|
end
|
|
147
115
|
|
|
148
|
-
def
|
|
149
|
-
|
|
150
|
-
|
|
116
|
+
def parse_whole_or_embedded(raw_content, label)
|
|
117
|
+
JSON.parse(raw_content)
|
|
118
|
+
rescue JSON::ParserError
|
|
119
|
+
json_content = find_json_in_text(raw_content)
|
|
120
|
+
raise InvalidResponseError.new("No JSON in #{label} response", raw_content), cause: nil unless json_content
|
|
151
121
|
|
|
152
|
-
|
|
122
|
+
parse_embedded_json(json_content, raw_content, label)
|
|
153
123
|
end
|
|
154
124
|
|
|
155
|
-
def
|
|
156
|
-
|
|
125
|
+
def parse_embedded_json(json_content, raw_content, label)
|
|
126
|
+
parse_json_content(json_content)
|
|
127
|
+
rescue JSON::ParserError
|
|
128
|
+
details = "raw_len=#{raw_content.length} json_len=#{json_content.length}"
|
|
129
|
+
Logger.debug_if_verbose("JSON parse failed: #{details}", sensitive: false)
|
|
130
|
+
raise InvalidResponseError.new("Invalid JSON in #{label} response (#{details})", raw_content), cause: nil
|
|
157
131
|
end
|
|
158
132
|
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
133
|
+
# Le premier objet du texte, de sa première accolade à celle qui la referme.
|
|
134
|
+
def find_json_in_text(text)
|
|
135
|
+
start = text.index("{")
|
|
136
|
+
finish = closing_brace_index(text, start) if start
|
|
137
|
+
text[start..finish] if finish
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
# Les accolades des chaînes ne comptent pas ; le parcours est plafonné.
|
|
141
|
+
def closing_brace_index(text, start)
|
|
142
|
+
depth = 0
|
|
143
|
+
in_string = false
|
|
144
|
+
escaped = false
|
|
145
|
+
|
|
146
|
+
(start...text.length).each do |index|
|
|
147
|
+
ensure_scan_length!(index - start)
|
|
148
|
+
char = text[index]
|
|
149
|
+
|
|
150
|
+
if escaped
|
|
151
|
+
escaped = false
|
|
152
|
+
elsif char == "\\"
|
|
153
|
+
escaped = true
|
|
154
|
+
elsif char == '"'
|
|
155
|
+
in_string = !in_string
|
|
156
|
+
elsif !in_string
|
|
157
|
+
depth += BRACE_DEPTH.fetch(char, 0)
|
|
158
|
+
return index if depth.zero?
|
|
168
159
|
end
|
|
169
|
-
|
|
170
|
-
char = text[i]
|
|
171
|
-
parser_state.process_character(char)
|
|
172
|
-
|
|
173
|
-
return text[start_pos..i] if parser_state.found_complete_json?
|
|
174
160
|
end
|
|
175
161
|
|
|
176
162
|
nil
|
|
177
163
|
end
|
|
178
164
|
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
attr_reader :iterations
|
|
182
|
-
|
|
183
|
-
def initialize
|
|
184
|
-
@brace_count = 0
|
|
185
|
-
@in_string = false
|
|
186
|
-
@escape_next = false
|
|
187
|
-
@iterations = 0
|
|
188
|
-
end
|
|
189
|
-
|
|
190
|
-
def increment_iterations
|
|
191
|
-
@iterations += 1
|
|
192
|
-
end
|
|
193
|
-
|
|
194
|
-
def process_character(char)
|
|
195
|
-
return handle_escape_character if @escape_next
|
|
196
|
-
return handle_backslash_character if char == "\\"
|
|
197
|
-
return handle_quote_character(char) if char == '"' && !@escape_next
|
|
198
|
-
return if @in_string
|
|
199
|
-
|
|
200
|
-
handle_brace_character(char)
|
|
201
|
-
end
|
|
165
|
+
def ensure_scan_length!(scanned)
|
|
166
|
+
return if scanned < MAX_SCAN_LENGTH
|
|
202
167
|
|
|
203
|
-
|
|
204
|
-
@escape_next = false
|
|
205
|
-
end
|
|
206
|
-
|
|
207
|
-
def handle_backslash_character
|
|
208
|
-
@escape_next = true
|
|
209
|
-
end
|
|
210
|
-
|
|
211
|
-
def handle_quote_character(_char)
|
|
212
|
-
@in_string = !@in_string
|
|
213
|
-
end
|
|
214
|
-
|
|
215
|
-
def handle_brace_character(char)
|
|
216
|
-
if char == "{"
|
|
217
|
-
@brace_count += 1
|
|
218
|
-
elsif char == "}"
|
|
219
|
-
@brace_count -= 1
|
|
220
|
-
end
|
|
221
|
-
end
|
|
222
|
-
|
|
223
|
-
def found_complete_json?
|
|
224
|
-
@brace_count.zero?
|
|
225
|
-
end
|
|
168
|
+
raise InvalidResponseError, "JSON parsing exceeded maximum iterations", cause: nil
|
|
226
169
|
end
|
|
227
170
|
|
|
228
171
|
# Le texte traduit ou résumé : toujours une String non vide, jamais un objet ou un tableau.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "logger"
|
|
4
|
+
|
|
5
|
+
module MistralTranslator
|
|
6
|
+
# Relances communes aux traductions et aux résumés.
|
|
7
|
+
# Le client relance déjà les 429 avec backoff : ici, quelques tentatives de plus suffisent.
|
|
8
|
+
module Retries
|
|
9
|
+
RETRY_COUNT = 3
|
|
10
|
+
RETRY_DELAY = 2
|
|
11
|
+
|
|
12
|
+
private
|
|
13
|
+
|
|
14
|
+
# Exécute le bloc, qui reçoit le numéro de tentative (0 pour la première).
|
|
15
|
+
# Une réponse inexploitable est relancée avec un backoff exponentiel, un quota dépassé
|
|
16
|
+
# après un délai fixe ; au-delà de RETRY_COUNT relances, l'erreur remonte.
|
|
17
|
+
# operation et subject nomment la requête dans les journaux, jamais son contenu.
|
|
18
|
+
def with_retries(operation:, subject:)
|
|
19
|
+
attempt = 0
|
|
20
|
+
begin
|
|
21
|
+
yield attempt
|
|
22
|
+
rescue EmptyTranslationError, InvalidResponseError => e
|
|
23
|
+
raise if attempt >= RETRY_COUNT
|
|
24
|
+
|
|
25
|
+
wait_time = RETRY_DELAY * (2**attempt)
|
|
26
|
+
attempt += 1
|
|
27
|
+
log_retry(operation, subject, e, attempt, wait_time)
|
|
28
|
+
sleep(wait_time)
|
|
29
|
+
retry
|
|
30
|
+
rescue RateLimitError
|
|
31
|
+
raise if attempt >= RETRY_COUNT
|
|
32
|
+
|
|
33
|
+
attempt += 1
|
|
34
|
+
log_rate_limit_hit(operation, subject)
|
|
35
|
+
sleep(RETRY_DELAY)
|
|
36
|
+
retry
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def log_retry(operation, subject, error, attempt, wait_time)
|
|
41
|
+
message = "#{operation} retry #{attempt}/#{RETRY_COUNT} for #{subject} in #{wait_time}s: #{error.message}"
|
|
42
|
+
Logger.warn_once(message, key: "#{operation}_retry_#{subject}_#{error.class.name}", sensitive: false, ttl: 120)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def log_rate_limit_hit(operation, subject)
|
|
46
|
+
Logger.warn_once("Rate limit hit for #{operation} #{subject}, retrying",
|
|
47
|
+
key: "#{operation}_rate_limit_#{subject}", sensitive: false, ttl: 300)
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
@@ -1,13 +1,16 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "logger"
|
|
4
|
+
require_relative "retries"
|
|
4
5
|
require_relative "security"
|
|
5
6
|
|
|
6
7
|
module MistralTranslator
|
|
7
8
|
class Summarizer
|
|
9
|
+
include Retries
|
|
10
|
+
|
|
8
11
|
DEFAULT_MAX_WORDS = 250
|
|
9
|
-
DEFAULT_RETRY_COUNT =
|
|
10
|
-
DEFAULT_RETRY_DELAY =
|
|
12
|
+
DEFAULT_RETRY_COUNT = Retries::RETRY_COUNT
|
|
13
|
+
DEFAULT_RETRY_DELAY = Retries::RETRY_DELAY
|
|
11
14
|
|
|
12
15
|
def initialize(client: nil)
|
|
13
16
|
@client = client || Client.new
|
|
@@ -84,28 +87,16 @@ module MistralTranslator
|
|
|
84
87
|
|
|
85
88
|
private
|
|
86
89
|
|
|
87
|
-
#
|
|
88
|
-
def request_summary(prompt, target_locale
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
result = ResponseParser.parse_summary_response(@client.complete(prompt, json: true))
|
|
92
|
-
raise EmptyTranslationError, "Empty summary received" if result.nil? || result[:summary].to_s.empty?
|
|
90
|
+
# Commun à tous les résumés ; une réponse inexploitable finit par lever une erreur.
|
|
91
|
+
def request_summary(prompt, target_locale)
|
|
92
|
+
with_retries(operation: "summary", subject: target_locale) do |attempt|
|
|
93
|
+
log_debug("Summary attempt #{attempt + 1} for #{target_locale}")
|
|
93
94
|
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
raise if attempt >= DEFAULT_RETRY_COUNT
|
|
95
|
+
result = ResponseParser.parse_summary_response(@client.complete(prompt, json: true))
|
|
96
|
+
raise EmptyTranslationError, "Empty summary received" if result.nil? || result[:summary].to_s.empty?
|
|
97
97
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
sleep(wait_time)
|
|
101
|
-
request_summary(prompt, target_locale, attempt + 1)
|
|
102
|
-
rescue RateLimitError
|
|
103
|
-
# Le client a déjà relancé avec backoff : quelques tentatives de plus suffisent.
|
|
104
|
-
raise if attempt >= DEFAULT_RETRY_COUNT
|
|
105
|
-
|
|
106
|
-
log_rate_limit_hit("summary", target_locale)
|
|
107
|
-
sleep(DEFAULT_RETRY_DELAY)
|
|
108
|
-
request_summary(prompt, target_locale, attempt + 1)
|
|
98
|
+
result[:summary]
|
|
99
|
+
end
|
|
109
100
|
end
|
|
110
101
|
|
|
111
102
|
def clean_document_content(content)
|
|
@@ -157,36 +148,8 @@ module MistralTranslator
|
|
|
157
148
|
raise ArgumentError, "Long length must be greater than medium" unless long > medium
|
|
158
149
|
end
|
|
159
150
|
|
|
160
|
-
def log_retry(error, attempt, wait_time, locale)
|
|
161
|
-
message = "Summary retry #{attempt}/#{DEFAULT_RETRY_COUNT} for #{locale} in #{wait_time}s: #{error.message}"
|
|
162
|
-
# Log une seule fois par locale et type d'erreur
|
|
163
|
-
Logger.warn_once(message, key: "summary_retry_#{locale}_#{error.class.name}", sensitive: false, ttl: 120)
|
|
164
|
-
end
|
|
165
|
-
|
|
166
|
-
def log_rate_limit_hit(operation, locale)
|
|
167
|
-
message = "Rate limit hit for #{operation} in #{locale}, retrying..."
|
|
168
|
-
# Log une seule fois par opération et locale
|
|
169
|
-
Logger.warn_once(message, key: "summary_rate_limit_#{operation}_#{locale}", sensitive: false, ttl: 300)
|
|
170
|
-
end
|
|
171
|
-
|
|
172
151
|
def log_debug(message)
|
|
173
|
-
# Log de debug seulement si mode verbose activé
|
|
174
152
|
Logger.debug_if_verbose(message, sensitive: false)
|
|
175
|
-
|
|
176
|
-
# Pour les tests, permettre un output dans stdout si nécessaire
|
|
177
|
-
return unless ENV["MISTRAL_TRANSLATOR_TEST_OUTPUT"] == "true"
|
|
178
|
-
|
|
179
|
-
puts "[MistralTranslator] #{message}"
|
|
180
|
-
end
|
|
181
|
-
|
|
182
|
-
def log_error(message)
|
|
183
|
-
# Utiliser le logger centralisé
|
|
184
|
-
Logger.warn(message, sensitive: false)
|
|
185
|
-
|
|
186
|
-
# Pour les tests, optionnellement afficher en stdout
|
|
187
|
-
return unless ENV["MISTRAL_TRANSLATOR_TEST_OUTPUT"] == "true"
|
|
188
|
-
|
|
189
|
-
puts "[MistralTranslator] ERROR: #{message}"
|
|
190
153
|
end
|
|
191
154
|
end
|
|
192
155
|
end
|
|
@@ -1,20 +1,20 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "logger"
|
|
4
|
+
require_relative "retries"
|
|
4
5
|
require_relative "translator_helpers"
|
|
5
6
|
require_relative "security"
|
|
6
7
|
|
|
7
8
|
module MistralTranslator
|
|
8
9
|
class Translator
|
|
10
|
+
include Retries
|
|
9
11
|
include TranslatorHelpers::InputValidator
|
|
10
|
-
include TranslatorHelpers::RetryHandler
|
|
11
|
-
include TranslatorHelpers::LoggingHelper
|
|
12
12
|
include TranslatorHelpers::AnalysisHelper
|
|
13
13
|
include TranslatorHelpers::RequestHelper
|
|
14
14
|
include TranslatorHelpers::MultiTargetHelper
|
|
15
15
|
|
|
16
|
-
DEFAULT_RETRY_COUNT =
|
|
17
|
-
DEFAULT_RETRY_DELAY =
|
|
16
|
+
DEFAULT_RETRY_COUNT = Retries::RETRY_COUNT
|
|
17
|
+
DEFAULT_RETRY_DELAY = Retries::RETRY_DELAY
|
|
18
18
|
|
|
19
19
|
def initialize(client: nil)
|
|
20
20
|
@client = client || Client.new
|
|
@@ -176,29 +176,15 @@ module MistralTranslator
|
|
|
176
176
|
nil
|
|
177
177
|
end
|
|
178
178
|
|
|
179
|
-
def translate_with_retry(text, source_locale, target_locale,
|
|
180
|
-
prompt =
|
|
181
|
-
request_context = build_request_context(source_locale, target_locale, attempt)
|
|
179
|
+
def translate_with_retry(text, source_locale, target_locale, **)
|
|
180
|
+
prompt = build_translation_prompt(text, source_locale, target_locale, **)
|
|
182
181
|
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
handle_retryable_error!(e, attempt) do
|
|
187
|
-
translate_with_retry(text, source_locale, target_locale, attempt + 1, **options)
|
|
182
|
+
with_retries(operation: "translation", subject: "#{source_locale}->#{target_locale}") do |attempt|
|
|
183
|
+
raw_response = perform_client_request(prompt, build_request_context(source_locale, target_locale, attempt))
|
|
184
|
+
extract_translated_text!(raw_response)
|
|
188
185
|
end
|
|
189
|
-
rescue RateLimitError
|
|
190
|
-
# Le client a déjà relancé avec backoff : quelques tentatives de plus suffisent.
|
|
191
|
-
raise if attempt >= DEFAULT_RETRY_COUNT
|
|
192
|
-
|
|
193
|
-
log_rate_limit_hit(source_locale, target_locale)
|
|
194
|
-
sleep(DEFAULT_RETRY_DELAY)
|
|
195
|
-
translate_with_retry(text, source_locale, target_locale, attempt + 1, **options)
|
|
196
186
|
end
|
|
197
187
|
|
|
198
|
-
# Ces méthodes sont héritées via RequestHelper
|
|
199
|
-
|
|
200
|
-
# Log via LoggingHelper
|
|
201
|
-
|
|
202
188
|
def build_translation_prompt(text, source_locale, target_locale, **options)
|
|
203
189
|
PromptBuilder.translation_prompt(text, source_locale, target_locale,
|
|
204
190
|
context: options[:context],
|
|
@@ -20,75 +20,6 @@ module MistralTranslator
|
|
|
20
20
|
validate_inputs!(text, from, to)
|
|
21
21
|
raise ArgumentError, "Target languages cannot be empty" if Array(to).empty?
|
|
22
22
|
end
|
|
23
|
-
|
|
24
|
-
def validate_batch_inputs!(texts, from, to)
|
|
25
|
-
raise ArgumentError, "Texts array cannot be nil or empty" if texts.nil? || texts.empty?
|
|
26
|
-
|
|
27
|
-
ensure_present!(from, "Source language")
|
|
28
|
-
ensure_present!(to, "Target language")
|
|
29
|
-
|
|
30
|
-
texts.each_with_index do |t, i|
|
|
31
|
-
ensure_present!(t, "Text at index #{i}")
|
|
32
|
-
end
|
|
33
|
-
end
|
|
34
|
-
|
|
35
|
-
private
|
|
36
|
-
|
|
37
|
-
def ensure_present!(value, field_name)
|
|
38
|
-
raise ArgumentError, "#{field_name} cannot be nil or empty" if value.nil? || value.to_s.strip.empty?
|
|
39
|
-
end
|
|
40
|
-
end
|
|
41
|
-
|
|
42
|
-
# Helper pour la gestion des retry
|
|
43
|
-
module RetryHandler
|
|
44
|
-
DEFAULT_RETRY_DELAY = 2
|
|
45
|
-
|
|
46
|
-
def handle_retry_error(error, source_locale, target_locale, attempt)
|
|
47
|
-
case error
|
|
48
|
-
when RateLimitError
|
|
49
|
-
handle_rate_limit_retry?(source_locale, target_locale, attempt)
|
|
50
|
-
when ApiError
|
|
51
|
-
handle_api_error_retry?(error, source_locale, target_locale, attempt)
|
|
52
|
-
else
|
|
53
|
-
raise error
|
|
54
|
-
end
|
|
55
|
-
end
|
|
56
|
-
|
|
57
|
-
# Renvoie true si un retry a été effectué
|
|
58
|
-
def handle_rate_limit_retry?(source_locale, target_locale, _attempt)
|
|
59
|
-
log_rate_limit_hit(source_locale, target_locale)
|
|
60
|
-
sleep(DEFAULT_RETRY_DELAY)
|
|
61
|
-
# NOTE: retry logic should be handled by the calling method
|
|
62
|
-
true
|
|
63
|
-
end
|
|
64
|
-
|
|
65
|
-
# Renvoie true si un retry a été effectué
|
|
66
|
-
def handle_api_error_retry?(error, _source_locale, _target_locale, attempt)
|
|
67
|
-
raise error unless attempt < 3 && error.message.include?("timeout")
|
|
68
|
-
|
|
69
|
-
Logger.warn("API timeout, retrying... (attempt #{attempt + 1})")
|
|
70
|
-
sleep(DEFAULT_RETRY_DELAY)
|
|
71
|
-
# NOTE: retry logic should be handled by the calling method
|
|
72
|
-
true
|
|
73
|
-
end
|
|
74
|
-
end
|
|
75
|
-
|
|
76
|
-
# Helper pour le logging
|
|
77
|
-
module LoggingHelper
|
|
78
|
-
def log_rate_limit_hit(source_locale, target_locale)
|
|
79
|
-
message = "Rate limit hit for #{source_locale}->#{target_locale}"
|
|
80
|
-
Logger.warn_once(message, key: "rate_limit_#{source_locale}_#{target_locale}", sensitive: false, ttl: 300)
|
|
81
|
-
end
|
|
82
|
-
|
|
83
|
-
def log_translation_attempt(source_locale, target_locale, attempt)
|
|
84
|
-
Logger.debug("Translation attempt #{attempt} for #{source_locale}->#{target_locale}")
|
|
85
|
-
end
|
|
86
|
-
|
|
87
|
-
def log_retry(error, attempt, wait_time)
|
|
88
|
-
default_retry_count = MistralTranslator::Translator::DEFAULT_RETRY_COUNT
|
|
89
|
-
message = "Translation retry #{attempt}/#{default_retry_count} in #{wait_time}s: #{error.message}"
|
|
90
|
-
Logger.warn_once(message, key: "translate_retry_#{error.class.name}_#{attempt}", sensitive: false, ttl: 120)
|
|
91
|
-
end
|
|
92
23
|
end
|
|
93
24
|
|
|
94
25
|
# Helper pour l'analyse et la détection de langue
|
|
@@ -140,17 +71,6 @@ module MistralTranslator
|
|
|
140
71
|
|
|
141
72
|
# Helper pour requêtes client et retries
|
|
142
73
|
module RequestHelper
|
|
143
|
-
def build_prompt_for_retry(text, source_locale, target_locale, **options)
|
|
144
|
-
build_translation_prompt(
|
|
145
|
-
text,
|
|
146
|
-
source_locale,
|
|
147
|
-
target_locale,
|
|
148
|
-
context: options[:context],
|
|
149
|
-
glossary: options[:glossary],
|
|
150
|
-
preserve_html: options.fetch(:preserve_html, false)
|
|
151
|
-
)
|
|
152
|
-
end
|
|
153
|
-
|
|
154
74
|
def build_request_context(source_locale, target_locale, attempt)
|
|
155
75
|
{
|
|
156
76
|
from_locale: source_locale,
|
|
@@ -170,15 +90,6 @@ module MistralTranslator
|
|
|
170
90
|
|
|
171
91
|
raise EmptyTranslationError
|
|
172
92
|
end
|
|
173
|
-
|
|
174
|
-
def handle_retryable_error!(error, attempt)
|
|
175
|
-
raise error unless attempt < MistralTranslator::Translator::DEFAULT_RETRY_COUNT
|
|
176
|
-
|
|
177
|
-
wait_time = MistralTranslator::Translator::DEFAULT_RETRY_DELAY * (2**attempt)
|
|
178
|
-
log_retry(error, attempt + 1, wait_time)
|
|
179
|
-
sleep(wait_time)
|
|
180
|
-
yield
|
|
181
|
-
end
|
|
182
93
|
end
|
|
183
94
|
|
|
184
95
|
# Helper pour traductions multi-cibles
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: mistral_translator
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.5.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Pierre Cherbero
|
|
@@ -160,6 +160,7 @@ files:
|
|
|
160
160
|
- lib/mistral_translator/client_helpers.rb
|
|
161
161
|
- lib/mistral_translator/configuration.rb
|
|
162
162
|
- lib/mistral_translator/data/languages.json
|
|
163
|
+
- lib/mistral_translator/deprecated_options.rb
|
|
163
164
|
- lib/mistral_translator/errors.rb
|
|
164
165
|
- lib/mistral_translator/helpers.rb
|
|
165
166
|
- lib/mistral_translator/helpers_extensions.rb
|
|
@@ -172,6 +173,7 @@ files:
|
|
|
172
173
|
- lib/mistral_translator/prompt_metadata_helpers.rb
|
|
173
174
|
- lib/mistral_translator/proxy.rb
|
|
174
175
|
- lib/mistral_translator/response_parser.rb
|
|
176
|
+
- lib/mistral_translator/retries.rb
|
|
175
177
|
- lib/mistral_translator/security.rb
|
|
176
178
|
- lib/mistral_translator/summarizer.rb
|
|
177
179
|
- lib/mistral_translator/translator.rb
|