gai18n 0.1.0 → 0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CODE_OF_CONDUCT.md +133 -0
- data/LICENSE.txt +21 -0
- data/lib/gai18n/cli.rb +79 -0
- data/lib/gai18n/config_template.rb +59 -0
- data/lib/gai18n/configuration.rb +45 -0
- data/lib/gai18n/content.rb +48 -0
- data/lib/gai18n/errors.rb +5 -0
- data/lib/gai18n/git_comparison.rb +38 -0
- data/lib/gai18n/locale.rb +60 -0
- data/lib/gai18n/locale_file.rb +118 -0
- data/lib/gai18n/openai/assistant.rb +72 -0
- data/lib/gai18n/openai/message.rb +40 -0
- data/lib/gai18n/openai/run.rb +116 -0
- data/lib/gai18n/openai/thread.rb +23 -0
- data/lib/gai18n/setup.rb +38 -0
- data/lib/gai18n/translation.rb +15 -0
- data/lib/gai18n/translator.rb +10 -0
- data/lib/gai18n/version.rb +3 -0
- data/lib/gai18n.rb +22 -0
- data/lib/openai/overrides.rb +16 -0
- metadata +21 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 872e8312f105964dda9b785fbecb875fc66c64e87c51c527b3b44bb6369642f9
|
4
|
+
data.tar.gz: d7647cdf55655c08aafe5210fca2679bcc5125001ace9dfab573b57f00e5fb98
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 71981288cb3de0ab007efa0de5026182e1f1e836aab007e38d1089764c22ef4eeeeb05a0e06e6bdeaef55b4a03d59d633646b0ab518f83cff472894e9fc885c4
|
7
|
+
data.tar.gz: 49fb2d2d9f3f49470086ffce89427833f28bb6a47395be05b13a616379d0cbbd4c109b3f8133b3c089f62b5bc999266a7e244eaa0e8a32377ffbe5da1a80075f
|
data/CODE_OF_CONDUCT.md
ADDED
@@ -0,0 +1,133 @@
|
|
1
|
+
|
2
|
+
# Contributor Covenant Code of Conduct
|
3
|
+
|
4
|
+
## Our Pledge
|
5
|
+
|
6
|
+
We as members, contributors, and leaders pledge to make participation in our
|
7
|
+
community a harassment-free experience for everyone, regardless of age, body
|
8
|
+
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
9
|
+
identity and expression, level of experience, education, socio-economic status,
|
10
|
+
nationality, personal appearance, race, caste, color, religion, or sexual
|
11
|
+
identity and orientation.
|
12
|
+
|
13
|
+
We pledge to act and interact in ways that contribute to an open, welcoming,
|
14
|
+
diverse, inclusive, and healthy community.
|
15
|
+
|
16
|
+
## Our Standards
|
17
|
+
|
18
|
+
Examples of behavior that contributes to a positive environment for our
|
19
|
+
community include:
|
20
|
+
|
21
|
+
* Demonstrating empathy and kindness toward other people
|
22
|
+
* Being respectful of differing opinions, viewpoints, and experiences
|
23
|
+
* Giving and gracefully accepting constructive feedback
|
24
|
+
* Accepting responsibility and apologizing to those affected by our mistakes,
|
25
|
+
and learning from the experience
|
26
|
+
* Focusing on what is best not just for us as individuals, but for the overall
|
27
|
+
community
|
28
|
+
|
29
|
+
Examples of unacceptable behavior include:
|
30
|
+
|
31
|
+
* The use of sexualized language or imagery, and sexual attention or advances of
|
32
|
+
any kind
|
33
|
+
* Trolling, insulting or derogatory comments, and personal or political attacks
|
34
|
+
* Public or private harassment
|
35
|
+
* Publishing others' private information, such as a physical or email address,
|
36
|
+
without their explicit permission
|
37
|
+
* Other conduct which could reasonably be considered inappropriate in a
|
38
|
+
professional setting
|
39
|
+
|
40
|
+
## Enforcement Responsibilities
|
41
|
+
|
42
|
+
Community leaders are responsible for clarifying and enforcing our standards of
|
43
|
+
acceptable behavior and will take appropriate and fair corrective action in
|
44
|
+
response to any behavior that they deem inappropriate, threatening, offensive,
|
45
|
+
or harmful.
|
46
|
+
|
47
|
+
Community leaders have the right and responsibility to remove, edit, or reject
|
48
|
+
comments, commits, code, wiki edits, issues, and other contributions that are
|
49
|
+
not aligned to this Code of Conduct, and will communicate reasons for moderation
|
50
|
+
decisions when appropriate.
|
51
|
+
|
52
|
+
## Scope
|
53
|
+
|
54
|
+
This Code of Conduct applies within all community spaces, and also applies when
|
55
|
+
an individual is officially representing the community in public spaces.
|
56
|
+
Examples of representing our community include using an official email address,
|
57
|
+
posting via an official social media account, or acting as an appointed
|
58
|
+
representative at an online or offline event.
|
59
|
+
|
60
|
+
## Enforcement
|
61
|
+
|
62
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
63
|
+
reported to the community leaders responsible for enforcement at
|
64
|
+
[INSERT CONTACT METHOD].
|
65
|
+
All complaints will be reviewed and investigated promptly and fairly.
|
66
|
+
|
67
|
+
All community leaders are obligated to respect the privacy and security of the
|
68
|
+
reporter of any incident.
|
69
|
+
|
70
|
+
## Enforcement Guidelines
|
71
|
+
|
72
|
+
Community leaders will follow these Community Impact Guidelines in determining
|
73
|
+
the consequences for any action they deem in violation of this Code of Conduct:
|
74
|
+
|
75
|
+
### 1. Correction
|
76
|
+
|
77
|
+
**Community Impact**: Use of inappropriate language or other behavior deemed
|
78
|
+
unprofessional or unwelcome in the community.
|
79
|
+
|
80
|
+
**Consequence**: A private, written warning from community leaders, providing
|
81
|
+
clarity around the nature of the violation and an explanation of why the
|
82
|
+
behavior was inappropriate. A public apology may be requested.
|
83
|
+
|
84
|
+
### 2. Warning
|
85
|
+
|
86
|
+
**Community Impact**: A violation through a single incident or series of
|
87
|
+
actions.
|
88
|
+
|
89
|
+
**Consequence**: A warning with consequences for continued behavior. No
|
90
|
+
interaction with the people involved, including unsolicited interaction with
|
91
|
+
those enforcing the Code of Conduct, for a specified period of time. This
|
92
|
+
includes avoiding interactions in community spaces as well as external channels
|
93
|
+
like social media. Violating these terms may lead to a temporary or permanent
|
94
|
+
ban.
|
95
|
+
|
96
|
+
### 3. Temporary Ban
|
97
|
+
|
98
|
+
**Community Impact**: A serious violation of community standards, including
|
99
|
+
sustained inappropriate behavior.
|
100
|
+
|
101
|
+
**Consequence**: A temporary ban from any sort of interaction or public
|
102
|
+
communication with the community for a specified period of time. No public or
|
103
|
+
private interaction with the people involved, including unsolicited interaction
|
104
|
+
with those enforcing the Code of Conduct, is allowed during this period.
|
105
|
+
Violating these terms may lead to a permanent ban.
|
106
|
+
|
107
|
+
### 4. Permanent Ban
|
108
|
+
|
109
|
+
**Community Impact**: Demonstrating a pattern of violation of community
|
110
|
+
standards, including sustained inappropriate behavior, harassment of an
|
111
|
+
individual, or aggression toward or disparagement of classes of individuals.
|
112
|
+
|
113
|
+
**Consequence**: A permanent ban from any sort of public interaction within the
|
114
|
+
community.
|
115
|
+
|
116
|
+
## Attribution
|
117
|
+
|
118
|
+
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
119
|
+
version 2.1, available at
|
120
|
+
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
|
121
|
+
|
122
|
+
Community Impact Guidelines were inspired by
|
123
|
+
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
|
124
|
+
|
125
|
+
For answers to common questions about this code of conduct, see the FAQ at
|
126
|
+
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
|
127
|
+
[https://www.contributor-covenant.org/translations][translations].
|
128
|
+
|
129
|
+
[homepage]: https://www.contributor-covenant.org
|
130
|
+
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
|
131
|
+
[Mozilla CoC]: https://github.com/mozilla/diversity
|
132
|
+
[FAQ]: https://www.contributor-covenant.org/faq
|
133
|
+
[translations]: https://www.contributor-covenant.org/translations
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
MIT License
|
2
|
+
|
3
|
+
Copyright (c) 2024 Philip Q Nguyen
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
13
|
+
copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
21
|
+
SOFTWARE.
|
data/lib/gai18n/cli.rb
ADDED
@@ -0,0 +1,79 @@
|
|
1
|
+
require 'optparse'
|
2
|
+
|
3
|
+
module GAI18n
|
4
|
+
class CLI
|
5
|
+
class Parser
|
6
|
+
Options = Struct.new(:secret)
|
7
|
+
|
8
|
+
def self.parse(options)
|
9
|
+
args = Options.new
|
10
|
+
|
11
|
+
opt_parser = OptionParser.new do |opts|
|
12
|
+
opts.banner = "Usage: gai18n [setup|translate|assistant:create] [options]"
|
13
|
+
|
14
|
+
if options[0] == 'setup'
|
15
|
+
opts.on("-sSECRET", "--secret=SECRET", "OpenAI Secret Key") do |secret|
|
16
|
+
args.secret = secret
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
opts.on("-h", "--help", "Prints this help") do
|
21
|
+
puts opts
|
22
|
+
exit
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
opt_parser.parse!(options)
|
27
|
+
return args
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
def run(argv)
|
32
|
+
if argv[0] == 'setup'
|
33
|
+
setup openai_api_key: Parser.parse(argv).secret
|
34
|
+
elsif argv[0] == 'translate'
|
35
|
+
translate
|
36
|
+
elsif argv[0] == 'assistant:create'
|
37
|
+
assistant_create
|
38
|
+
else
|
39
|
+
Parser.parse(argv + ['-h'])
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
private
|
44
|
+
|
45
|
+
def setup(openai_api_key: nil)
|
46
|
+
GAI18n::Setup.new.run(openai_api_key: openai_api_key)
|
47
|
+
end
|
48
|
+
|
49
|
+
def translate
|
50
|
+
require_config_file
|
51
|
+
GAI18n::Translator.new.translate
|
52
|
+
end
|
53
|
+
|
54
|
+
def assistant_create
|
55
|
+
require_config_file
|
56
|
+
puts 'Creating OpenAI assistant...'
|
57
|
+
assistant = GAI18n::OpenAI::Assistant.create
|
58
|
+
puts 'Created OpenAI assistant.'
|
59
|
+
puts 'Please add the following to your gai18n.rb config file.'
|
60
|
+
puts 'GAI18n.configure do |config|'
|
61
|
+
puts " config.openai_assistant_id = '#{assistant.id}'"
|
62
|
+
puts 'end'
|
63
|
+
end
|
64
|
+
|
65
|
+
def require_config_file
|
66
|
+
if File.exist?('./config/gai18n.rb' )
|
67
|
+
require './config/gai18n'
|
68
|
+
elsif File.exist?('./gai18n.rb' )
|
69
|
+
require './gai18n'
|
70
|
+
else
|
71
|
+
message = [
|
72
|
+
'GAI18n configuration file not found.',
|
73
|
+
'Ensure there is one at ./config/gai18n.rb or ./gai18n.rb'
|
74
|
+
].join(' ')
|
75
|
+
raise GAI18n::LoadError, message
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
79
|
+
end
|
@@ -0,0 +1,59 @@
|
|
1
|
+
GAI18n.configure do |config|
|
2
|
+
config.source_locale = {
|
3
|
+
english: {
|
4
|
+
# `files`: this specifies where to match one or more source yml files.
|
5
|
+
# Change this to match your source language file(s) location.
|
6
|
+
files: 'config/**/en.yml',
|
7
|
+
|
8
|
+
# `file_identifier`: this is used to specify the identifier for the
|
9
|
+
# source language file. It could be a part of the file name or the entire
|
10
|
+
# file path. This identifier is used by the gem to locate and process the
|
11
|
+
# source language file. For example, if your source language file is
|
12
|
+
# located at 'config/dashboards/en.yml' or 'config/en/dashboard.yml' you
|
13
|
+
# should set `file_identifier` to 'en'.
|
14
|
+
file_identifier: 'en',
|
15
|
+
|
16
|
+
# `root_key`: this is the root key of the yaml content in your source
|
17
|
+
#file(s). For example, if your root key in the source language file is
|
18
|
+
# `en`, you should set `root_key` to 'en'.
|
19
|
+
root_key: 'en'
|
20
|
+
}
|
21
|
+
}
|
22
|
+
|
23
|
+
config.target_locales = {
|
24
|
+
japanese: {
|
25
|
+
# `file_identifier`: this is used to specify the identifier for the
|
26
|
+
# target language file. It could be a part of the file name or the entire
|
27
|
+
# file path. This identifier is used by the gem to locate and process the
|
28
|
+
# target language file. For example, if your target language file is
|
29
|
+
# located at 'config/dashboards/jp.yml' or 'config/jp/dashboard.yml' you
|
30
|
+
# should set `file_identifier` to 'jp'.
|
31
|
+
file_identifier: 'jp',
|
32
|
+
|
33
|
+
# `root_key`: this is the root key of the yaml content in your source
|
34
|
+
# file(s). For example, if your root key in the target language file is
|
35
|
+
# `jp`, you should set `root_key` to 'jp'.
|
36
|
+
root_key: 'jp'
|
37
|
+
},
|
38
|
+
french: {
|
39
|
+
file_identifier: 'fr',
|
40
|
+
root_key: 'fr'
|
41
|
+
}
|
42
|
+
}
|
43
|
+
|
44
|
+
# `openai_secret_key`: this is the secret key from OpenAI. You can get this
|
45
|
+
# from your OpenAI account.
|
46
|
+
config.openai_secret_key = 'replace_with_the_openai_secret_key'
|
47
|
+
|
48
|
+
# `base_git_branch`: we need the base branch to compare changes between the
|
49
|
+
# current source file to the source file in the base branch. This is used to
|
50
|
+
# determine the changes that need to be translated.
|
51
|
+
config.base_git_branch = 'main'
|
52
|
+
|
53
|
+
# `openai_assistant_id`: this is the assistant id from OpenAI. Please create
|
54
|
+
# this assistant using the `bundle exec gai18n assistant:create` command.
|
55
|
+
# Do not use an existing assisntant. This is because the assistant created
|
56
|
+
# from the command is configured with specific instructions and tools that
|
57
|
+
# are required for translation.
|
58
|
+
config.openai_assistant_id = 'replace_with_the_assistant_id'
|
59
|
+
end
|
@@ -0,0 +1,45 @@
|
|
1
|
+
module GAI18n
|
2
|
+
class Configuration
|
3
|
+
attr_accessor :source_locale, :openai_secret_key, :locales,
|
4
|
+
:openai_assistant_id, :target_locales
|
5
|
+
|
6
|
+
attr_writer :base_git_branch, :model, :keys_per_paginated_requests
|
7
|
+
|
8
|
+
def openai_client
|
9
|
+
args = {
|
10
|
+
access_token: openai_secret_key
|
11
|
+
}
|
12
|
+
@openai_client ||= ::OpenAI::Client.new args
|
13
|
+
end
|
14
|
+
|
15
|
+
def project_root
|
16
|
+
return Rails.root if defined? Rails
|
17
|
+
return Bundler.root if defined? Bundler
|
18
|
+
Dir.pwd
|
19
|
+
end
|
20
|
+
|
21
|
+
def base_git_branch
|
22
|
+
@base_git_branch ||= 'main'
|
23
|
+
end
|
24
|
+
|
25
|
+
def model
|
26
|
+
@model ||= 'gpt-3.5-turbo-1106'
|
27
|
+
end
|
28
|
+
|
29
|
+
def keys_per_paginated_requests
|
30
|
+
@keys_per_paginated_requests ||= 20
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
def self.configuration
|
35
|
+
@configuration ||= Configuration.new
|
36
|
+
end
|
37
|
+
|
38
|
+
def self.config
|
39
|
+
configuration
|
40
|
+
end
|
41
|
+
|
42
|
+
def self.configure
|
43
|
+
yield configuration if block_given?
|
44
|
+
end
|
45
|
+
end
|
@@ -0,0 +1,48 @@
|
|
1
|
+
module GAI18n
|
2
|
+
class Content
|
3
|
+
attr_reader :current
|
4
|
+
|
5
|
+
def initialize(raw)
|
6
|
+
@current = raw
|
7
|
+
end
|
8
|
+
|
9
|
+
def deep_merge!(second)
|
10
|
+
merger = proc do |key, v1, v2|
|
11
|
+
if Hash === v1 && Hash === v2
|
12
|
+
v1.merge!(v2, &merger)
|
13
|
+
else
|
14
|
+
v2
|
15
|
+
end
|
16
|
+
end
|
17
|
+
current.merge!(second, &merger)
|
18
|
+
self
|
19
|
+
end
|
20
|
+
|
21
|
+
def to_h
|
22
|
+
current
|
23
|
+
end
|
24
|
+
|
25
|
+
def keys
|
26
|
+
flatten_keys current
|
27
|
+
end
|
28
|
+
|
29
|
+
def value_for(key)
|
30
|
+
keys = key.split('.')
|
31
|
+
keys.inject(current) do |hash, key|
|
32
|
+
hash[key]
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
private
|
37
|
+
|
38
|
+
def flatten_keys(hash, prefix = nil)
|
39
|
+
hash.flat_map do |key, value|
|
40
|
+
if value.is_a? Hash
|
41
|
+
flatten_keys(value, prefix ? "#{prefix}.#{key}" : key)
|
42
|
+
else
|
43
|
+
prefix ? "#{prefix}.#{key}" : key
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -0,0 +1,38 @@
|
|
1
|
+
module GAI18n
|
2
|
+
class GitComparison
|
3
|
+
attr_reader :base_git_branch, :project_root, :source_path, :source_root_key
|
4
|
+
|
5
|
+
def initialize(locale_file)
|
6
|
+
@base_git_branch = GAI18n.config.base_git_branch
|
7
|
+
@project_root = GAI18n.config.project_root
|
8
|
+
@source_path = locale_file.source_path
|
9
|
+
@source_root_key = locale_file.source_root_key
|
10
|
+
end
|
11
|
+
|
12
|
+
def changes
|
13
|
+
git = Git.open(project_root, :log => Logger.new(STDOUT))
|
14
|
+
diff = git.diff(base_git_branch).path(source_path)
|
15
|
+
base_content = diff.first&.blob(:src)&.contents
|
16
|
+
return [] if base_content.nil?
|
17
|
+
base_yaml = YAML.load(base_content)[source_root_key.to_s]
|
18
|
+
current_yaml = YAML.load_file(source_path)[source_root_key.to_s]
|
19
|
+
keys_with_changed_values(base_yaml, current_yaml)
|
20
|
+
end
|
21
|
+
|
22
|
+
private
|
23
|
+
|
24
|
+
def keys_with_changed_values(base_yaml, current_yaml, prefix = '')
|
25
|
+
base_yaml.keys.inject([]) do |keys, key|
|
26
|
+
full_key = prefix.empty? ? key : "#{prefix}.#{key}"
|
27
|
+
if base_yaml[key] != current_yaml[key]
|
28
|
+
if base_yaml[key].is_a?(Hash) && current_yaml[key].is_a?(Hash)
|
29
|
+
keys.concat(keys_with_changed_values(base_yaml[key], current_yaml[key], full_key))
|
30
|
+
else
|
31
|
+
keys << full_key
|
32
|
+
end
|
33
|
+
end
|
34
|
+
keys
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -0,0 +1,60 @@
|
|
1
|
+
module GAI18n
|
2
|
+
class Locale
|
3
|
+
class << self
|
4
|
+
def source_locale
|
5
|
+
GAI18n.config.source_locale
|
6
|
+
end
|
7
|
+
|
8
|
+
def source_language
|
9
|
+
source_locale.keys.first
|
10
|
+
end
|
11
|
+
|
12
|
+
def source_file_identifier
|
13
|
+
source_locale[source_language][:file_identifier]
|
14
|
+
end
|
15
|
+
|
16
|
+
def source_paths
|
17
|
+
Dir.glob(source_locale[source_language][:files])
|
18
|
+
end
|
19
|
+
|
20
|
+
def source_root_key
|
21
|
+
source_locale[source_language][:root_key] || source_file_identifier
|
22
|
+
end
|
23
|
+
|
24
|
+
def target_locale_files
|
25
|
+
target_locales = GAI18n.config.target_locales
|
26
|
+
target_locales.flat_map do |(lang, options)|
|
27
|
+
source_paths.map do |source_path|
|
28
|
+
file_identifier = options[:file_identifier]
|
29
|
+
root_key = options[:root_key] || file_identifier
|
30
|
+
path = source_path.gsub source_file_identifier, file_identifier
|
31
|
+
Locale.new lang: lang,
|
32
|
+
root_key: root_key,
|
33
|
+
path: path,
|
34
|
+
source_lang: source_language,
|
35
|
+
source_path: source_path,
|
36
|
+
source_root_key: source_root_key
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
41
|
+
|
42
|
+
attr_reader :lang, :root_key, :path, :file_identifier, :source_lang,
|
43
|
+
:source_path, :source_root_key, :source_file_identifier,
|
44
|
+
:locale_file_class
|
45
|
+
|
46
|
+
def initialize(lang:, root_key:, path:, source_lang:, source_path:,
|
47
|
+
source_root_key:)
|
48
|
+
@lang = lang
|
49
|
+
@root_key = root_key
|
50
|
+
@path = path
|
51
|
+
@source_lang = source_lang
|
52
|
+
@source_path = source_path
|
53
|
+
@source_root_key = source_root_key
|
54
|
+
end
|
55
|
+
|
56
|
+
def translate
|
57
|
+
LocaleFile.new(self).translate
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
@@ -0,0 +1,118 @@
|
|
1
|
+
module GAI18n
|
2
|
+
class LocaleFile
|
3
|
+
extend Forwardable
|
4
|
+
|
5
|
+
attr_reader :locale, :assistant_id, :skip_keys
|
6
|
+
|
7
|
+
def_delegators :@locale, :lang, :root_key, :path, :source_lang,
|
8
|
+
:source_path, :source_root_key
|
9
|
+
|
10
|
+
def initialize(locale)
|
11
|
+
@locale = locale
|
12
|
+
@assistant_id = GAI18n.config.openai_assistant_id
|
13
|
+
@skip_keys = []
|
14
|
+
end
|
15
|
+
|
16
|
+
def translate(skip_keys = [])
|
17
|
+
@skip_keys = skip_keys
|
18
|
+
write_file and return if translatable_key_values.empty?
|
19
|
+
thread = GAI18n::OpenAI::Thread.create
|
20
|
+
OpenAI::Message.create thread_id: thread.id, content: message_content
|
21
|
+
run = OpenAI::Run.create assistant_id: assistant_id, thread_id: thread.id
|
22
|
+
run = run.wait_until_done
|
23
|
+
if run.requires_action?
|
24
|
+
run.accept_translations.each {|translation| insert translation}
|
25
|
+
write_file
|
26
|
+
locale_file = self.class.new locale
|
27
|
+
locale_file.translate(skip_keys + translatable_keys)
|
28
|
+
else
|
29
|
+
msg = ['Either the Run took too long or received incorrect',
|
30
|
+
"Run status from OpenAI: #{run.status}"].join(' ')
|
31
|
+
raise IncorrectResponseError, msg
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
private
|
36
|
+
|
37
|
+
def insert(translated_hash)
|
38
|
+
content.deep_merge!(translated_hash)
|
39
|
+
end
|
40
|
+
|
41
|
+
def write_file
|
42
|
+
ordered_content = source_content.deep_merge!(content.to_h)
|
43
|
+
filtered_content = undeleted_content ordered_content.to_h, content.to_h
|
44
|
+
content_yaml = {root_key.to_s => filtered_content}.to_yaml
|
45
|
+
file_path = if path.start_with? '/'
|
46
|
+
path
|
47
|
+
else
|
48
|
+
GAI18n.config.project_root.join(path)
|
49
|
+
end
|
50
|
+
File.open(file_path, 'w') { |f| f.write content_yaml }
|
51
|
+
end
|
52
|
+
|
53
|
+
def translatable_key_values
|
54
|
+
@translatable_key_values ||= translatable_keys.map do |key|
|
55
|
+
[key, source_content.value_for(key)]
|
56
|
+
end.to_h
|
57
|
+
end
|
58
|
+
|
59
|
+
def translatable_keys
|
60
|
+
@_translatable_keys ||= begin
|
61
|
+
keys = source_content.keys - content.keys + keys_with_changed_values - skip_keys
|
62
|
+
keys.first GAI18n.config.keys_per_paginated_requests
|
63
|
+
end
|
64
|
+
end
|
65
|
+
|
66
|
+
def keys_with_changed_values
|
67
|
+
GitComparison.new(self).changes
|
68
|
+
end
|
69
|
+
|
70
|
+
def deep_merge(first, second)
|
71
|
+
merger = proc do |key, v1, v2|
|
72
|
+
if Hash === v1 && Hash === v2
|
73
|
+
v1.merge(v2, &merger)
|
74
|
+
else
|
75
|
+
v2
|
76
|
+
end
|
77
|
+
end
|
78
|
+
first.merge(second, &merger)
|
79
|
+
end
|
80
|
+
|
81
|
+
def source_content
|
82
|
+
@source_content ||= begin
|
83
|
+
raw = YAML.load_file(source_path)[source_root_key.to_s]
|
84
|
+
Content.new raw
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
88
|
+
def content
|
89
|
+
@content ||= begin
|
90
|
+
raw_content = if File.exist?(path)
|
91
|
+
raw_content = YAML.load_file(path)
|
92
|
+
raw_content.respond_to?(:[]) ? raw_content[root_key.to_s] : {}
|
93
|
+
else
|
94
|
+
{}
|
95
|
+
end
|
96
|
+
Content.new undeleted_content(raw_content, source_content.to_h)
|
97
|
+
end
|
98
|
+
end
|
99
|
+
|
100
|
+
def undeleted_content(first_content, second_content)
|
101
|
+
first_content.each_with_object({}) do |(key, value), new_hash|
|
102
|
+
if second_content.key?(key)
|
103
|
+
new_hash[key] = value.is_a?(Hash) ? undeleted_content(value, second_content[key]) : value
|
104
|
+
end
|
105
|
+
end
|
106
|
+
end
|
107
|
+
|
108
|
+
def message_content
|
109
|
+
<<~HEREDOC
|
110
|
+
Below are keys and values. Please translate the values from #{source_lang} to #{lang}, then submit the translations.
|
111
|
+
|
112
|
+
```
|
113
|
+
#{translatable_key_values.map {|key, val| "#{key}: #{val}"}.join("\n")}
|
114
|
+
```
|
115
|
+
HEREDOC
|
116
|
+
end
|
117
|
+
end
|
118
|
+
end
|
@@ -0,0 +1,72 @@
|
|
1
|
+
module GAI18n
|
2
|
+
module OpenAI
|
3
|
+
class Assistant
|
4
|
+
class << self
|
5
|
+
def create
|
6
|
+
description = ['This assisistant was created by GAI18n',
|
7
|
+
'to help with internationalization.'].join ' '
|
8
|
+
instructions = "You are a software localization engineer. You have been tasked with translating the English source to the target language. You will be given a key and a source string. You will be asked to translate the source string to the target language. Once translated, please call the translation.submit function with the key, translated string, and the target language. If you are asked to translate into multiple target languages, please call the translation.submit function for each target language."
|
9
|
+
openai_client = GAI18n.config.openai_client
|
10
|
+
model = GAI18n.config.model
|
11
|
+
parameters = {
|
12
|
+
model: model,
|
13
|
+
name: "GAI18n-#{Time.now.to_i}",
|
14
|
+
description: description,
|
15
|
+
instructions: instructions,
|
16
|
+
tools: [
|
17
|
+
{
|
18
|
+
type: "function",
|
19
|
+
function: {
|
20
|
+
name: "translation.submit",
|
21
|
+
description: "Submit a translation",
|
22
|
+
parameters: {
|
23
|
+
type: "object",
|
24
|
+
properties: {
|
25
|
+
key: {
|
26
|
+
type: "string",
|
27
|
+
description: "The given key that's associated to the string needing translation."
|
28
|
+
},
|
29
|
+
translation: {
|
30
|
+
type: "string",
|
31
|
+
description: "The translated string."
|
32
|
+
},
|
33
|
+
language: {
|
34
|
+
type: "string",
|
35
|
+
description: "The language of the translation."
|
36
|
+
}
|
37
|
+
},
|
38
|
+
required: ["key", "translation", "language"]
|
39
|
+
}
|
40
|
+
}
|
41
|
+
}
|
42
|
+
]
|
43
|
+
}
|
44
|
+
response = openai_client.assistants.create parameters: parameters
|
45
|
+
new response
|
46
|
+
end
|
47
|
+
|
48
|
+
def find(id)
|
49
|
+
openai_client = GAI18n.config.openai_client
|
50
|
+
response = openai_client.assistants.retrieve id: id
|
51
|
+
new response
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
attr_reader :id, :object, :created_at, :name, :description,
|
56
|
+
:model, :instructions, :tools, :file_ids, :metadata
|
57
|
+
|
58
|
+
def initialize(response)
|
59
|
+
@id = response['id']
|
60
|
+
@object = response['object']
|
61
|
+
@created_at = response['created_at']
|
62
|
+
@name = response['name']
|
63
|
+
@description = response['description']
|
64
|
+
@model = response['model']
|
65
|
+
@instructions = response['instructions']
|
66
|
+
@tools = response['tools']
|
67
|
+
@file_ids = response['file_ids']
|
68
|
+
@metadata = response['metadata']
|
69
|
+
end
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
@@ -0,0 +1,40 @@
|
|
1
|
+
module GAI18n
|
2
|
+
module OpenAI
|
3
|
+
class Message
|
4
|
+
class << self
|
5
|
+
def create(thread_id:, content:, uploads: [])
|
6
|
+
openai_client = GAI18n.config.openai_client
|
7
|
+
parameters = {
|
8
|
+
role: 'user',
|
9
|
+
content: content
|
10
|
+
}
|
11
|
+
response = openai_client.messages.create thread_id: thread_id,
|
12
|
+
parameters: parameters
|
13
|
+
new response
|
14
|
+
end
|
15
|
+
|
16
|
+
def all(thread_id:)
|
17
|
+
openai_client = GAI18n.config.openai_client
|
18
|
+
response = openai_client.messages.list(thread_id: thread_id)
|
19
|
+
response['data'].map { |message| new message }
|
20
|
+
end
|
21
|
+
end
|
22
|
+
|
23
|
+
attr_reader :id, :object, :created_at, :role, :thread_id, :content,
|
24
|
+
:file_ids, :assistant_id, :run_id, :metadata
|
25
|
+
|
26
|
+
def initialize(response)
|
27
|
+
@id = response['id']
|
28
|
+
@object = response['object']
|
29
|
+
@created_at = response['created_at']
|
30
|
+
@role = response['role']
|
31
|
+
@thread_id = response['thread_id']
|
32
|
+
@content = response['content']
|
33
|
+
@file_ids = response['file_ids']
|
34
|
+
@assisistant_id = response['assistant_id']
|
35
|
+
@run_id = response['run_id']
|
36
|
+
@metadata = response['metadata']
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
@@ -0,0 +1,116 @@
|
|
1
|
+
module GAI18n
|
2
|
+
module OpenAI
|
3
|
+
class Run
|
4
|
+
class << self
|
5
|
+
def create(assistant_id:, thread_id:)
|
6
|
+
openai_client = GAI18n.config.openai_client
|
7
|
+
parameters = {assistant_id: assistant_id}
|
8
|
+
response = openai_client.runs.create thread_id: thread_id,
|
9
|
+
parameters: parameters
|
10
|
+
new response
|
11
|
+
end
|
12
|
+
|
13
|
+
def find(thread_id:, id:)
|
14
|
+
openai_client = GAI18n.config.openai_client
|
15
|
+
response = openai_client.runs.retrieve thread_id: thread_id, id: id
|
16
|
+
new response
|
17
|
+
end
|
18
|
+
end
|
19
|
+
|
20
|
+
attr_accessor :id, :object, :created_at, :assistant_id, :thread_id,
|
21
|
+
:status, :started_at, :expires_at, :cancelled_at, :failed_at,
|
22
|
+
:completed_at, :last_error, :model, :instructions, :tools,
|
23
|
+
:file_ids, :metadata, :tool_calls
|
24
|
+
|
25
|
+
def initialize(response)
|
26
|
+
set_attributes_from(response)
|
27
|
+
end
|
28
|
+
|
29
|
+
def completed?
|
30
|
+
status == 'completed'
|
31
|
+
end
|
32
|
+
|
33
|
+
def requires_action?
|
34
|
+
status == 'requires_action'
|
35
|
+
end
|
36
|
+
|
37
|
+
def failed?
|
38
|
+
status == 'failed'
|
39
|
+
end
|
40
|
+
|
41
|
+
def cancelled?
|
42
|
+
status == 'cancelled'
|
43
|
+
end
|
44
|
+
|
45
|
+
def expired?
|
46
|
+
status == 'expired'
|
47
|
+
end
|
48
|
+
|
49
|
+
def accept_translations
|
50
|
+
submit_tool_outputs
|
51
|
+
submit_translations
|
52
|
+
end
|
53
|
+
|
54
|
+
def reload
|
55
|
+
openai_client = GAI18n.config.openai_client
|
56
|
+
response = openai_client.runs.retrieve thread_id: thread_id, id: id
|
57
|
+
set_attributes_from response
|
58
|
+
end
|
59
|
+
|
60
|
+
def wait_until_done(count = 0)
|
61
|
+
return self if count > 10
|
62
|
+
reload
|
63
|
+
halt_statuses = %w[requires_action completed failed cancelled expired]
|
64
|
+
return self if halt_statuses.include? status
|
65
|
+
sleep 5
|
66
|
+
wait_until_done count + 1
|
67
|
+
end
|
68
|
+
|
69
|
+
private
|
70
|
+
|
71
|
+
def set_attributes_from(response)
|
72
|
+
@id = response['id']
|
73
|
+
@object = response['object']
|
74
|
+
@created_at = response['created_at']
|
75
|
+
@assistant_id = response['assistant_id']
|
76
|
+
@thread_id = response['thread_id']
|
77
|
+
@status = response['status']
|
78
|
+
@started_at = response['started_at']
|
79
|
+
@expires_at = response['expires_at']
|
80
|
+
@cancelled_at = response['cancelled_at']
|
81
|
+
@failed_at = response['failed_at']
|
82
|
+
@completed_at = response['completed_at']
|
83
|
+
@last_error = response['last_error']
|
84
|
+
@model = response['model']
|
85
|
+
@instructions = response['instructions']
|
86
|
+
@tools = response['tools']
|
87
|
+
@file_ids = response['file_ids']
|
88
|
+
@metadata = response['metadata']
|
89
|
+
required_action = response.fetch('required_action', {}) || {}
|
90
|
+
submit_tool_outputs = required_action.fetch('submit_tool_outputs', {})
|
91
|
+
@tool_calls = submit_tool_outputs.fetch('tool_calls', [])
|
92
|
+
end
|
93
|
+
|
94
|
+
def submit_tool_outputs
|
95
|
+
parameters = tool_calls.inject({tool_outputs: []}) do |hash, tool_call|
|
96
|
+
hash[:tool_outputs] << {
|
97
|
+
tool_call_id: tool_call['id'],
|
98
|
+
output: "Accepted"
|
99
|
+
}
|
100
|
+
hash
|
101
|
+
end
|
102
|
+
openai_client = GAI18n.config.openai_client
|
103
|
+
openai_client.runs.submit_tool_outputs run_id: id,
|
104
|
+
thread_id: thread_id,
|
105
|
+
parameters: parameters
|
106
|
+
end
|
107
|
+
|
108
|
+
def submit_translations
|
109
|
+
tool_calls.map do |tool_call|
|
110
|
+
args = JSON.parse(tool_call["function"]["arguments"]).transform_keys(&:to_sym)
|
111
|
+
Translation.new.submit **args
|
112
|
+
end
|
113
|
+
end
|
114
|
+
end
|
115
|
+
end
|
116
|
+
end
|
@@ -0,0 +1,23 @@
|
|
1
|
+
module GAI18n
|
2
|
+
module OpenAI
|
3
|
+
class Thread
|
4
|
+
class << self
|
5
|
+
def create
|
6
|
+
openai_client = GAI18n.config.openai_client
|
7
|
+
response = openai_client.threads.create
|
8
|
+
new response
|
9
|
+
end
|
10
|
+
|
11
|
+
end
|
12
|
+
|
13
|
+
attr_reader :id, :object, :created_at, :metadata
|
14
|
+
|
15
|
+
def initialize(response)
|
16
|
+
@id = response['id']
|
17
|
+
@object = response['object']
|
18
|
+
@created_at = response['created_at']
|
19
|
+
@metadata = response['metadata']
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
data/lib/gai18n/setup.rb
ADDED
@@ -0,0 +1,38 @@
|
|
1
|
+
module GAI18n
|
2
|
+
class Setup
|
3
|
+
def run(openai_api_key: nil)
|
4
|
+
copy_template
|
5
|
+
puts "GAI18n configuration file created at #{destination}"
|
6
|
+
if openai_api_key
|
7
|
+
add_api_key_to_template openai_api_key
|
8
|
+
puts 'Added OpenAI API key to configuration file.'
|
9
|
+
puts 'Please run `bundle exec gai18n assistant:create` to create an OpenAI assistant, and then add the assistant id to the configuration file.'
|
10
|
+
else
|
11
|
+
puts '1. Please add your OpenAI API key to the configuration file.'
|
12
|
+
puts '2. Then run `bundle exec gai18n assistant:create` to create an OpenAI assistant, and then add the assistant id to the configuration file.'
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
private
|
17
|
+
|
18
|
+
def destination
|
19
|
+
if File.directory?('./config')
|
20
|
+
"#{GAI18n.config.project_root}/config/gai18n.rb"
|
21
|
+
else
|
22
|
+
"#{GAI18n.config.project_root}/gai18n.rb"
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
def copy_template
|
27
|
+
template_file = File.expand_path '../config_template.rb', __FILE__
|
28
|
+
FileUtils.cp template_file, destination
|
29
|
+
end
|
30
|
+
|
31
|
+
def add_api_key_to_template(openai_api_key)
|
32
|
+
content = File.read(destination).gsub('replace_with_the_openai_secret_key', openai_api_key)
|
33
|
+
File.open(destination, 'w') do |file|
|
34
|
+
file.write content
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
module GAI18n
|
2
|
+
class Translation
|
3
|
+
def submit(key:, translation:, language:)
|
4
|
+
arr = key.split('.') + [translation]
|
5
|
+
result = arr.reverse.inject({}) do |hash, k|
|
6
|
+
if k == translation
|
7
|
+
k
|
8
|
+
else
|
9
|
+
{ k => hash }
|
10
|
+
end
|
11
|
+
end
|
12
|
+
result
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
data/lib/gai18n.rb
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
require 'openai'
|
2
|
+
require 'json'
|
3
|
+
require 'yaml'
|
4
|
+
require 'git'
|
5
|
+
require 'openai/overrides'
|
6
|
+
require 'gai18n/errors'
|
7
|
+
require 'gai18n/openai/assistant'
|
8
|
+
require 'gai18n/openai/message'
|
9
|
+
require 'gai18n/openai/thread'
|
10
|
+
require 'gai18n/openai/run'
|
11
|
+
require 'gai18n/version'
|
12
|
+
require 'gai18n/configuration'
|
13
|
+
require 'gai18n/git_comparison'
|
14
|
+
require 'gai18n/setup'
|
15
|
+
require 'gai18n/translator'
|
16
|
+
require 'gai18n/translation'
|
17
|
+
require 'gai18n/locale_file'
|
18
|
+
require 'gai18n/locale'
|
19
|
+
require 'gai18n/content'
|
20
|
+
|
21
|
+
module GAI18n
|
22
|
+
end
|
@@ -0,0 +1,16 @@
|
|
1
|
+
|
2
|
+
if ENV['ENABLE_OPENAI_LOGS'] == 'true'
|
3
|
+
module OpenAI
|
4
|
+
module HTTP
|
5
|
+
def conn(multipart: false)
|
6
|
+
Faraday.new do |f|
|
7
|
+
f.options[:timeout] = @request_timeout
|
8
|
+
f.request(:multipart) if multipart
|
9
|
+
f.response :raise_error
|
10
|
+
f.response :json
|
11
|
+
f.response :logger, ::Logger.new(STDOUT), bodies: true
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: gai18n
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Philip Q Nguyen
|
@@ -116,9 +116,29 @@ executables:
|
|
116
116
|
extensions: []
|
117
117
|
extra_rdoc_files: []
|
118
118
|
files:
|
119
|
+
- CODE_OF_CONDUCT.md
|
120
|
+
- LICENSE.txt
|
119
121
|
- README.md
|
120
122
|
- Rakefile
|
121
123
|
- bin/gai18n
|
124
|
+
- lib/gai18n.rb
|
125
|
+
- lib/gai18n/cli.rb
|
126
|
+
- lib/gai18n/config_template.rb
|
127
|
+
- lib/gai18n/configuration.rb
|
128
|
+
- lib/gai18n/content.rb
|
129
|
+
- lib/gai18n/errors.rb
|
130
|
+
- lib/gai18n/git_comparison.rb
|
131
|
+
- lib/gai18n/locale.rb
|
132
|
+
- lib/gai18n/locale_file.rb
|
133
|
+
- lib/gai18n/openai/assistant.rb
|
134
|
+
- lib/gai18n/openai/message.rb
|
135
|
+
- lib/gai18n/openai/run.rb
|
136
|
+
- lib/gai18n/openai/thread.rb
|
137
|
+
- lib/gai18n/setup.rb
|
138
|
+
- lib/gai18n/translation.rb
|
139
|
+
- lib/gai18n/translator.rb
|
140
|
+
- lib/gai18n/version.rb
|
141
|
+
- lib/openai/overrides.rb
|
122
142
|
homepage: https://github.com/philipqnguyen/gai18n
|
123
143
|
licenses: []
|
124
144
|
metadata:
|