msgai-cli 1.0.1 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -15
- package/README.md +129 -38
- package/dist/src/cli/index.js +11 -2
- package/dist/src/cli/runTranslate.js +19 -4
- package/dist/src/po.js +3 -3
- package/dist/src/translate.js +156 -23
- package/package.json +9 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,8 +1,30 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
All notable changes to this project will be documented in this file.
|
|
3
|
+
All notable changes to this project will be documented in this file. See [commit-and-tag-version](https://github.com/absolute-version/commit-and-tag-version) for commit guidelines.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
## [1.1.0](https://github.com/AlexMost/msgai/compare/v1.0.2...v1.1.0) (2026-03-01)
|
|
6
|
+
|
|
7
|
+
### Features
|
|
8
|
+
|
|
9
|
+
- **cli:** add --model option for translation ([61a2ed8](https://github.com/AlexMost/msgai/commit/61a2ed8502a0f90bbb09a1715bbb58d0ddabe01e))
|
|
10
|
+
- use structured outputs for translations ([c57e75b](https://github.com/AlexMost/msgai/commit/c57e75bda4229923dfca1daf1b8146a50f1d9b7f))
|
|
11
|
+
|
|
12
|
+
## 1.0.2 (2026-02-27)
|
|
13
|
+
|
|
14
|
+
### Bug Fixes
|
|
15
|
+
|
|
16
|
+
- **po:** preserve PO file order in `getEntriesToTranslate` ([55ba8d8](https://github.com/AlexMost/msgai/commit/55ba8d8c4c43d26adf4d1e23fa5dac2bebbf2052))
|
|
17
|
+
|
|
18
|
+
## 1.0.1 (2026-02-27)
|
|
19
|
+
|
|
20
|
+
### Documentation
|
|
21
|
+
|
|
22
|
+
- run formatter after each change and verify formatting in agent workflow ([1a1d909](https://github.com/AlexMost/msgai/commit/1a1d9097f9a88f6b260f5564fc6c800b52ea95a3))
|
|
23
|
+
|
|
24
|
+
### Chores
|
|
25
|
+
|
|
26
|
+
- clean `dist` folder before each build ([0e65234](https://github.com/AlexMost/msgai/commit/0e652342d076867d1795b69ca1cb07d7988b92d2))
|
|
27
|
+
- fix formatting ([0cb5742](https://github.com/AlexMost/msgai/commit/0cb5742452f37f03a292f70ff6acc433f6f3666f))
|
|
6
28
|
|
|
7
29
|
## 1.0.0 (2026-02-27)
|
|
8
30
|
|
|
@@ -17,16 +39,3 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
|
17
39
|
- **ci:** use github.event.workflow_run.conclusion in release-please ([0892b17](https://github.com/AlexMost/msgai/commit/0892b17aad1ec95a4e96a8208cb31d197264830f))
|
|
18
40
|
- **ci:** use RELEASE_PLEASE_TOKEN so release-please can create PRs ([204c6cd](https://github.com/AlexMost/msgai/commit/204c6cdf88d480decae841f91c9fae168c4c0ce9))
|
|
19
41
|
- use console.warn instead of console.error for pipeline compatibility ([8b30ebf](https://github.com/AlexMost/msgai/commit/8b30ebffba7c7447db3f51d931398938c39b6b85))
|
|
20
|
-
|
|
21
|
-
## [Unreleased]
|
|
22
|
-
|
|
23
|
-
## [1.0.0] - 2025-02-27
|
|
24
|
-
|
|
25
|
-
### Added
|
|
26
|
-
|
|
27
|
-
- CLI to translate untranslated strings in gettext (`.po`) files using AI (OpenAI LLM)
|
|
28
|
-
- `msgai <file.po>`: translate empty `msgstr` entries and write back to the file
|
|
29
|
-
- `--dry-run`: list untranslated `msgid` values without API calls or file changes
|
|
30
|
-
- `--source-lang LANG`: specify source language (ISO 639-1); optional, model can infer
|
|
31
|
-
- `--include-fuzzy`: include fuzzy entries for re-translation and clear fuzzy flag
|
|
32
|
-
- `--api-key KEY` and `OPENAI_API_KEY` environment variable for API authentication
|
package/README.md
CHANGED
|
@@ -1,71 +1,162 @@
|
|
|
1
1
|
# msgai
|
|
2
2
|
|
|
3
|
-
`msgai` is
|
|
3
|
+
`msgai` is an AI-powered CLI for translating gettext `.po` files. It finds untranslated entries, sends them to an LLM, and writes the translated strings back into the same file.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
## 🤖 Project Purpose
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
`msgai` is built for teams that already use gettext and want a simple way to translate missing strings without building a separate localization workflow.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Main features:
|
|
10
10
|
|
|
11
|
-
-
|
|
12
|
-
-
|
|
13
|
-
- `
|
|
11
|
+
- `📝` Works directly with gettext `.po` files
|
|
12
|
+
- `🤖` Translates only untranslated entries using AI
|
|
13
|
+
- `🧠` Uses OpenAI `gpt-4o` by default for translation
|
|
14
|
+
- `🏷️` Respects gettext context (`msgctxt`) when translating entries
|
|
15
|
+
- `🔁` Supports singular and plural translations
|
|
16
|
+
- `⚠️` Skips fuzzy entries by default
|
|
17
|
+
- `🧭` Can infer source language or use `--source-lang`
|
|
18
|
+
- `💻` Runs as a small CLI that updates files in place
|
|
14
19
|
|
|
15
|
-
|
|
20
|
+
## ⚙️ How It Works
|
|
16
21
|
|
|
17
|
-
|
|
18
|
-
|
|
22
|
+
1. Read the `.po` file and parse its entries.
|
|
23
|
+
2. Find entries with empty or missing translations.
|
|
24
|
+
3. Send those strings to OpenAI `gpt-4o` for translation while preserving gettext context such as `msgctxt`.
|
|
25
|
+
4. Write the translated values back into the same `.po` file.
|
|
19
26
|
|
|
20
|
-
|
|
27
|
+
The translation API uses OpenAI `json_schema` structured outputs. Only models that support `json_schema` structured outputs are valid for `msgai`.
|
|
21
28
|
|
|
22
|
-
|
|
29
|
+
<details>
|
|
30
|
+
<summary>Supported model families</summary>
|
|
23
31
|
|
|
24
|
-
|
|
32
|
+
- `gpt-4o`
|
|
33
|
+
- `gpt-4o-mini`
|
|
34
|
+
- `gpt-4.1`
|
|
35
|
+
- `gpt-4.1-mini`
|
|
36
|
+
- `gpt-4.1-nano`
|
|
37
|
+
- `gpt-5`
|
|
38
|
+
- `gpt-5-mini`
|
|
39
|
+
- `gpt-5-nano`
|
|
40
|
+
- `gpt-5-pro`
|
|
41
|
+
- `gpt-5.1`
|
|
42
|
+
- `gpt-5.2`
|
|
43
|
+
- `gpt-5-codex`
|
|
44
|
+
- `gpt-5.1-codex`
|
|
45
|
+
- `gpt-5.1-codex-mini`
|
|
46
|
+
- `gpt-5.1-codex-max`
|
|
47
|
+
- `gpt-5.2-codex`
|
|
25
48
|
|
|
26
|
-
|
|
49
|
+
Dated snapshots are accepted where the model family supports them.
|
|
27
50
|
|
|
28
|
-
|
|
29
|
-
- **CLI option**: pass `--api-key KEY` (e.g. `msgai messages.po --api-key sk-...`).
|
|
51
|
+
</details>
|
|
30
52
|
|
|
31
|
-
If
|
|
53
|
+
By default, entries marked as `fuzzy` are skipped. If you use `--include-fuzzy`, `msgai` will translate those entries too and remove the fuzzy flag after applying the result.
|
|
32
54
|
|
|
33
|
-
|
|
55
|
+
## 📦 Install
|
|
34
56
|
|
|
35
|
-
|
|
57
|
+
Install the CLI globally:
|
|
36
58
|
|
|
37
|
-
|
|
59
|
+
```bash
|
|
60
|
+
npm install -g msgai-cli
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Set your OpenAI API key before running translations:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
export OPENAI_API_KEY=your_api_key_here
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
You can also pass the key directly:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
msgai messages.po --api-key sk-...
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
`OPENAI_API_KEY` can be loaded from your environment or from a `.env` file in the current directory.
|
|
76
|
+
|
|
77
|
+
## 💻 CLI Usage
|
|
78
|
+
|
|
79
|
+
Usage:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
msgai <file.po> [--dry-run] [--api-key KEY] [--source-lang LANG] [--model MODEL] [--include-fuzzy]
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Options:
|
|
86
|
+
|
|
87
|
+
- `--dry-run`: list untranslated `msgid` values only, with no API calls and no file changes
|
|
88
|
+
- `--include-fuzzy`: include fuzzy entries for translation and clear their fuzzy flag after translation
|
|
89
|
+
- `--source-lang LANG`: set the source language of `msgid` strings as an ISO 639-1 code such as `en` or `uk`
|
|
90
|
+
- `--model MODEL`: set the OpenAI model used for translation; default is `gpt-4o`. Only models with `json_schema` structured outputs are supported.
|
|
91
|
+
- `--api-key KEY`: pass the OpenAI API key directly instead of using `OPENAI_API_KEY`
|
|
92
|
+
- `--help`: print command usage
|
|
38
93
|
|
|
39
|
-
-
|
|
40
|
-
- npm 10+
|
|
94
|
+
If no API key is provided for a non-dry run, the CLI exits with code `1` and prints an error message.
|
|
41
95
|
|
|
42
|
-
|
|
96
|
+
On API failures such as rate limits, quota issues, or server errors, the CLI exits with code `1` and shows a status-specific message. For API error details, see [OpenAI API error codes](https://developers.openai.com/api/docs/guides/error-codes#api-errors).
|
|
97
|
+
|
|
98
|
+
## 🧪 Development
|
|
99
|
+
|
|
100
|
+
Requirements:
|
|
101
|
+
|
|
102
|
+
- Node.js `20+`
|
|
103
|
+
- npm `10+`
|
|
104
|
+
|
|
105
|
+
Install dependencies:
|
|
43
106
|
|
|
44
107
|
```bash
|
|
45
108
|
npm install
|
|
46
109
|
```
|
|
47
110
|
|
|
48
|
-
|
|
111
|
+
Useful scripts:
|
|
112
|
+
|
|
113
|
+
- `npm run build`: compile TypeScript to `dist/`
|
|
114
|
+
- `npm test`: build the project and run Jest tests
|
|
115
|
+
- `npm run test:integration`: run integration tests
|
|
116
|
+
- `npm run test:watch`: run tests in watch mode
|
|
117
|
+
- `npm run lint`: run ESLint
|
|
118
|
+
- `npm run lint:format`: check formatting with Prettier
|
|
119
|
+
- `npm run format`: format the repository with Prettier
|
|
120
|
+
- `npm run release:dry-run`: preview the `commit-and-tag-version` release without writing files
|
|
121
|
+
- `npm run release`: run release checks, update `CHANGELOG.md`, bump the npm version, create a release commit, and create a local tag
|
|
122
|
+
|
|
123
|
+
This repo follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) for commit messages.
|
|
124
|
+
|
|
125
|
+
### Release Flow
|
|
126
|
+
|
|
127
|
+
Maintainer releases are local-first and use `commit-and-tag-version`. The release command does not publish to npm or push tags for you.
|
|
128
|
+
|
|
129
|
+
Preview the next release:
|
|
49
130
|
|
|
50
|
-
|
|
131
|
+
```bash
|
|
132
|
+
npm run release:dry-run
|
|
133
|
+
```
|
|
51
134
|
|
|
52
|
-
|
|
135
|
+
Create the release locally:
|
|
53
136
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
- `npm run format`: format code with Prettier.
|
|
58
|
-
- `npm run lint:format`: check formatting with Prettier.
|
|
137
|
+
```bash
|
|
138
|
+
npm run release
|
|
139
|
+
```
|
|
59
140
|
|
|
60
|
-
|
|
141
|
+
This command:
|
|
61
142
|
|
|
62
|
-
|
|
143
|
+
- runs `build`, unit tests, integration tests, lint, and formatting checks through the `prerelease` lifecycle hook
|
|
144
|
+
- lets `commit-and-tag-version` infer `major`, `minor`, or `patch` from Conventional Commits since the latest `v*` tag
|
|
145
|
+
- updates `CHANGELOG.md`
|
|
146
|
+
- creates `chore(release): X.Y.Z`
|
|
147
|
+
- creates a local annotated tag `vX.Y.Z`
|
|
63
148
|
|
|
64
|
-
|
|
149
|
+
For reliable version bumps and changelog entries, keep commits in Conventional Commit format.
|
|
65
150
|
|
|
66
|
-
|
|
151
|
+
If you need to override the inferred bump manually:
|
|
67
152
|
|
|
68
|
-
|
|
69
|
-
|
|
153
|
+
```bash
|
|
154
|
+
npm run release -- --release-as minor
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
After the local release is created:
|
|
70
158
|
|
|
71
|
-
|
|
159
|
+
```bash
|
|
160
|
+
git push --follow-tags
|
|
161
|
+
npm publish
|
|
162
|
+
```
|
package/dist/src/cli/index.js
CHANGED
|
@@ -11,7 +11,7 @@ function parseArgs(argv) {
|
|
|
11
11
|
try {
|
|
12
12
|
const parsedArgs = (0, yargs_1.default)(argv)
|
|
13
13
|
.scriptName('msgai')
|
|
14
|
-
.usage('Usage: msgai <file.po> [--dry-run] [--api-key KEY] [--source-lang LANG] [--include-fuzzy]')
|
|
14
|
+
.usage('Usage: msgai <file.po> [--dry-run] [--api-key KEY] [--source-lang LANG] [--model MODEL] [--include-fuzzy]')
|
|
15
15
|
.option('dry-run', {
|
|
16
16
|
type: 'boolean',
|
|
17
17
|
default: false,
|
|
@@ -28,6 +28,10 @@ function parseArgs(argv) {
|
|
|
28
28
|
.option('source-lang', {
|
|
29
29
|
type: 'string',
|
|
30
30
|
description: 'Source language of msgid strings (ISO 639-1 code, e.g. en, uk). If omitted, the model will detect it.',
|
|
31
|
+
})
|
|
32
|
+
.option('model', {
|
|
33
|
+
type: 'string',
|
|
34
|
+
description: 'OpenAI model to use for translation. Default: gpt-4o',
|
|
31
35
|
})
|
|
32
36
|
.option('help', {
|
|
33
37
|
alias: 'h',
|
|
@@ -49,12 +53,15 @@ function parseArgs(argv) {
|
|
|
49
53
|
const sourceLang = sourceLangRaw != null && String(sourceLangRaw).trim() !== ''
|
|
50
54
|
? String(sourceLangRaw).trim().toLowerCase()
|
|
51
55
|
: undefined;
|
|
56
|
+
const modelRaw = parsedArgs.model;
|
|
57
|
+
const model = modelRaw != null && String(modelRaw).trim() !== '' ? String(modelRaw).trim() : undefined;
|
|
52
58
|
if (positionalArgs.length > 1) {
|
|
53
59
|
return {
|
|
54
60
|
dryRun: Boolean(parsedArgs['dry-run']),
|
|
55
61
|
help: Boolean(parsedArgs.help),
|
|
56
62
|
apiKey: parsedArgs['api-key'],
|
|
57
63
|
sourceLang,
|
|
64
|
+
model,
|
|
58
65
|
includeFuzzy: Boolean(parsedArgs['include-fuzzy']),
|
|
59
66
|
error: `Unexpected argument: ${positionalArgs[1]}`,
|
|
60
67
|
};
|
|
@@ -65,6 +72,7 @@ function parseArgs(argv) {
|
|
|
65
72
|
help: Boolean(parsedArgs.help),
|
|
66
73
|
apiKey: parsedArgs['api-key'],
|
|
67
74
|
sourceLang,
|
|
75
|
+
model,
|
|
68
76
|
includeFuzzy: Boolean(parsedArgs['include-fuzzy']),
|
|
69
77
|
};
|
|
70
78
|
}
|
|
@@ -73,7 +81,7 @@ function parseArgs(argv) {
|
|
|
73
81
|
return { dryRun: false, help: false, error: message };
|
|
74
82
|
}
|
|
75
83
|
}
|
|
76
|
-
const USAGE = 'Usage: msgai <file.po> [--dry-run] [--api-key KEY] [--source-lang LANG] [--include-fuzzy]';
|
|
84
|
+
const USAGE = 'Usage: msgai <file.po> [--dry-run] [--api-key KEY] [--source-lang LANG] [--model MODEL] [--include-fuzzy]';
|
|
77
85
|
function main(argv) {
|
|
78
86
|
const args = parseArgs(argv);
|
|
79
87
|
if (args.error) {
|
|
@@ -90,6 +98,7 @@ function main(argv) {
|
|
|
90
98
|
dryRun: args.dryRun,
|
|
91
99
|
apiKey: args.apiKey,
|
|
92
100
|
sourceLang: args.sourceLang,
|
|
101
|
+
model: args.model,
|
|
93
102
|
includeFuzzy: args.includeFuzzy,
|
|
94
103
|
});
|
|
95
104
|
if (result instanceof Promise) {
|
|
@@ -41,7 +41,13 @@ function getApiErrorMessage(err) {
|
|
|
41
41
|
return null;
|
|
42
42
|
}
|
|
43
43
|
}
|
|
44
|
-
|
|
44
|
+
function getInvalidModelMessage(model) {
|
|
45
|
+
return [
|
|
46
|
+
`Invalid --model "${model}". msgai only supports OpenAI models with json_schema structured outputs.`,
|
|
47
|
+
`Supported model families: ${translate_1.SUPPORTED_STRUCTURED_OUTPUT_MODELS.join(', ')}.`,
|
|
48
|
+
].join(' ');
|
|
49
|
+
}
|
|
50
|
+
async function runTranslate(poFilePath, apiKey, sourceLang, model, includeFuzzy) {
|
|
45
51
|
try {
|
|
46
52
|
const poContent = node_fs_1.default.readFileSync(poFilePath, 'utf8');
|
|
47
53
|
const parsedPo = (0, po_1.parsePoContent)(poContent);
|
|
@@ -62,7 +68,7 @@ async function runTranslate(poFilePath, apiKey, sourceLang, includeFuzzy) {
|
|
|
62
68
|
// locale not in plural-forms; rely on formula only
|
|
63
69
|
}
|
|
64
70
|
}
|
|
65
|
-
const options = { apiKey, sourceLanguage: sourceLang, formula, pluralSamples };
|
|
71
|
+
const options = { apiKey, sourceLanguage: sourceLang, formula, pluralSamples, model };
|
|
66
72
|
for (let i = 0; i < entries.length; i += TRANSLATE_BATCH_SIZE) {
|
|
67
73
|
const batch = entries.slice(i, i + TRANSLATE_BATCH_SIZE);
|
|
68
74
|
const batchNum = Math.floor(i / TRANSLATE_BATCH_SIZE) + 1;
|
|
@@ -96,7 +102,7 @@ async function runTranslate(poFilePath, apiKey, sourceLang, includeFuzzy) {
|
|
|
96
102
|
return 1;
|
|
97
103
|
}
|
|
98
104
|
}
|
|
99
|
-
const USAGE = 'Usage: msgai <file.po> [--dry-run] [--api-key KEY] [--source-lang LANG] [--include-fuzzy]';
|
|
105
|
+
const USAGE = 'Usage: msgai <file.po> [--dry-run] [--api-key KEY] [--source-lang LANG] [--model MODEL] [--include-fuzzy]';
|
|
100
106
|
function runTranslateCommand(args) {
|
|
101
107
|
if (!args.poFilePath) {
|
|
102
108
|
console.warn(USAGE);
|
|
@@ -112,6 +118,15 @@ function runTranslateCommand(args) {
|
|
|
112
118
|
return 1;
|
|
113
119
|
}
|
|
114
120
|
}
|
|
121
|
+
if (args.model != null) {
|
|
122
|
+
try {
|
|
123
|
+
(0, translate_1.validateModel)(args.model);
|
|
124
|
+
}
|
|
125
|
+
catch {
|
|
126
|
+
console.warn(getInvalidModelMessage(args.model));
|
|
127
|
+
return 1;
|
|
128
|
+
}
|
|
129
|
+
}
|
|
115
130
|
if (!args.dryRun) {
|
|
116
131
|
let resultApiKey;
|
|
117
132
|
try {
|
|
@@ -122,7 +137,7 @@ function runTranslateCommand(args) {
|
|
|
122
137
|
console.warn(message.replace('pass apiKey in options', 'pass --api-key'));
|
|
123
138
|
return 1;
|
|
124
139
|
}
|
|
125
|
-
return runTranslate(args.poFilePath, resultApiKey, args.sourceLang, args.includeFuzzy);
|
|
140
|
+
return runTranslate(args.poFilePath, resultApiKey, args.sourceLang, args.model, args.includeFuzzy);
|
|
126
141
|
}
|
|
127
142
|
try {
|
|
128
143
|
const poContent = node_fs_1.default.readFileSync(args.poFilePath, 'utf8');
|
package/dist/src/po.js
CHANGED
|
@@ -52,7 +52,7 @@ function parsePoContent(poContent) {
|
|
|
52
52
|
return gettext_parser_1.po.parse(Buffer.from(poContent, 'utf8'));
|
|
53
53
|
}
|
|
54
54
|
/**
|
|
55
|
-
* Returns untranslated entries and their keys in
|
|
55
|
+
* Returns untranslated entries and their keys in the order they appear in the original PO file (parser order).
|
|
56
56
|
* Skips the header (msgid "").
|
|
57
57
|
* By default skips fuzzy entries; set includeFuzzy to include them (with empty msgstr for the request).
|
|
58
58
|
*/
|
|
@@ -61,12 +61,12 @@ function getEntriesToTranslate(parsedPo, options) {
|
|
|
61
61
|
const entries = [];
|
|
62
62
|
const keys = [];
|
|
63
63
|
const translations = parsedPo.translations;
|
|
64
|
-
const contextNames = Object.keys(translations)
|
|
64
|
+
const contextNames = Object.keys(translations);
|
|
65
65
|
for (const context of contextNames) {
|
|
66
66
|
const contextEntries = translations[context];
|
|
67
67
|
if (contextEntries == null)
|
|
68
68
|
continue;
|
|
69
|
-
const msgids = Object.keys(contextEntries)
|
|
69
|
+
const msgids = Object.keys(contextEntries);
|
|
70
70
|
for (const msgid of msgids) {
|
|
71
71
|
if (msgid === '')
|
|
72
72
|
continue;
|
package/dist/src/translate.js
CHANGED
|
@@ -3,7 +3,9 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
|
3
3
|
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
4
4
|
};
|
|
5
5
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
6
|
+
exports.SUPPORTED_STRUCTURED_OUTPUT_MODELS = void 0;
|
|
6
7
|
exports.resolveApiKey = resolveApiKey;
|
|
8
|
+
exports.validateModel = validateModel;
|
|
7
9
|
exports.translatePayload = translatePayload;
|
|
8
10
|
exports.translateItems = translateItems;
|
|
9
11
|
exports.translateStrings = translateStrings;
|
|
@@ -19,6 +21,42 @@ function resolveApiKey(apiKey) {
|
|
|
19
21
|
throw new Error('OpenAI API key not set. Set OPENAI_API_KEY in the environment or pass apiKey in options.');
|
|
20
22
|
}
|
|
21
23
|
const DEFAULT_MODEL = 'gpt-4o';
|
|
24
|
+
const SUPPORTED_STRUCTURED_OUTPUT_MODEL_PATTERNS = [
|
|
25
|
+
/^gpt-4o(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
26
|
+
/^gpt-4o-mini(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
27
|
+
/^gpt-4\.1(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
28
|
+
/^gpt-4\.1-mini(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
29
|
+
/^gpt-4\.1-nano(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
30
|
+
/^gpt-5(?:-\d{4}-\d{2}-\d{2}|-chat-latest)?$/,
|
|
31
|
+
/^gpt-5-mini(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
32
|
+
/^gpt-5-nano(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
33
|
+
/^gpt-5-pro(?:-\d{4}-\d{2}-\d{2})?$/,
|
|
34
|
+
/^gpt-5\.1(?:-\d{4}-\d{2}-\d{2}|-chat-latest)?$/,
|
|
35
|
+
/^gpt-5\.2(?:-\d{4}-\d{2}-\d{2}|-chat-latest)?$/,
|
|
36
|
+
/^gpt-5-codex$/,
|
|
37
|
+
/^gpt-5\.1-codex$/,
|
|
38
|
+
/^gpt-5\.1-codex-mini$/,
|
|
39
|
+
/^gpt-5\.1-codex-max$/,
|
|
40
|
+
/^gpt-5\.2-codex$/,
|
|
41
|
+
];
|
|
42
|
+
exports.SUPPORTED_STRUCTURED_OUTPUT_MODELS = [
|
|
43
|
+
'gpt-4o',
|
|
44
|
+
'gpt-4o-mini',
|
|
45
|
+
'gpt-4.1',
|
|
46
|
+
'gpt-4.1-mini',
|
|
47
|
+
'gpt-4.1-nano',
|
|
48
|
+
'gpt-5',
|
|
49
|
+
'gpt-5-mini',
|
|
50
|
+
'gpt-5-nano',
|
|
51
|
+
'gpt-5-pro',
|
|
52
|
+
'gpt-5.1',
|
|
53
|
+
'gpt-5.2',
|
|
54
|
+
'gpt-5-codex',
|
|
55
|
+
'gpt-5.1-codex',
|
|
56
|
+
'gpt-5.1-codex-mini',
|
|
57
|
+
'gpt-5.1-codex-max',
|
|
58
|
+
'gpt-5.2-codex',
|
|
59
|
+
];
|
|
22
60
|
/** Error codes: https://developers.openai.com/api/docs/guides/error-codes#api-errors */
|
|
23
61
|
const MAX_RETRIES = 3;
|
|
24
62
|
const RETRY_DELAYS_MS = [1000, 2000, 4000];
|
|
@@ -34,20 +72,94 @@ function isApiError(err) {
|
|
|
34
72
|
function isRetryableStatus(status) {
|
|
35
73
|
return status === 429 || status === 500 || status === 503;
|
|
36
74
|
}
|
|
75
|
+
function isSupportedStructuredOutputModel(model) {
|
|
76
|
+
return SUPPORTED_STRUCTURED_OUTPUT_MODEL_PATTERNS.some((pattern) => pattern.test(model));
|
|
77
|
+
}
|
|
78
|
+
function validateStructuredOutputModel(model) {
|
|
79
|
+
if (isSupportedStructuredOutputModel(model))
|
|
80
|
+
return;
|
|
81
|
+
throw new Error(`Model "${model}" is not supported. This package requires an OpenAI Chat Completions model with json_schema structured outputs. Supported model families: ${exports.SUPPORTED_STRUCTURED_OUTPUT_MODELS.join(', ')}.`);
|
|
82
|
+
}
|
|
83
|
+
function validateModel(model) {
|
|
84
|
+
validateStructuredOutputModel(model);
|
|
85
|
+
}
|
|
86
|
+
const TRANSLATION_RESPONSE_SCHEMA = {
|
|
87
|
+
name: 'translation_payload',
|
|
88
|
+
strict: true,
|
|
89
|
+
schema: {
|
|
90
|
+
type: 'object',
|
|
91
|
+
additionalProperties: false,
|
|
92
|
+
required: ['translations'],
|
|
93
|
+
properties: {
|
|
94
|
+
translations: {
|
|
95
|
+
type: 'array',
|
|
96
|
+
items: {
|
|
97
|
+
anyOf: [
|
|
98
|
+
{
|
|
99
|
+
type: 'object',
|
|
100
|
+
additionalProperties: false,
|
|
101
|
+
required: ['msgid', 'msgstr'],
|
|
102
|
+
properties: {
|
|
103
|
+
msgid: { type: 'string' },
|
|
104
|
+
msgstr: { type: 'string' },
|
|
105
|
+
},
|
|
106
|
+
},
|
|
107
|
+
{
|
|
108
|
+
type: 'object',
|
|
109
|
+
additionalProperties: false,
|
|
110
|
+
required: ['msgid', 'msgctxt', 'msgstr'],
|
|
111
|
+
properties: {
|
|
112
|
+
msgid: { type: 'string' },
|
|
113
|
+
msgctxt: { type: 'string' },
|
|
114
|
+
msgstr: { type: 'string' },
|
|
115
|
+
},
|
|
116
|
+
},
|
|
117
|
+
{
|
|
118
|
+
type: 'object',
|
|
119
|
+
additionalProperties: false,
|
|
120
|
+
required: ['msgid_plural', 'msgstr'],
|
|
121
|
+
properties: {
|
|
122
|
+
msgid_plural: { type: 'string' },
|
|
123
|
+
msgstr: {
|
|
124
|
+
type: 'array',
|
|
125
|
+
items: { type: 'string' },
|
|
126
|
+
},
|
|
127
|
+
},
|
|
128
|
+
},
|
|
129
|
+
{
|
|
130
|
+
type: 'object',
|
|
131
|
+
additionalProperties: false,
|
|
132
|
+
required: ['msgid_plural', 'msgctxt', 'msgstr'],
|
|
133
|
+
properties: {
|
|
134
|
+
msgid_plural: { type: 'string' },
|
|
135
|
+
msgctxt: { type: 'string' },
|
|
136
|
+
msgstr: {
|
|
137
|
+
type: 'array',
|
|
138
|
+
items: { type: 'string' },
|
|
139
|
+
},
|
|
140
|
+
},
|
|
141
|
+
},
|
|
142
|
+
],
|
|
143
|
+
},
|
|
144
|
+
},
|
|
145
|
+
},
|
|
146
|
+
},
|
|
147
|
+
};
|
|
37
148
|
function buildSystemMessage() {
|
|
38
149
|
return `You are a deterministic translation engine for gettext PO entries.
|
|
39
150
|
|
|
151
|
+
Return exactly one JSON object that matches the provided response schema.
|
|
152
|
+
|
|
40
153
|
Your task:
|
|
41
154
|
For each input entry, produce a translation in the corresponding "msgstr" field.
|
|
42
155
|
Each output entry MUST correspond exactly to the matching input entry.
|
|
43
|
-
Do not change, remove, or reorder any "msgid" or "
|
|
156
|
+
Do not change, remove, or reorder any "msgid", "msgid_plural", or "msgctxt" values.
|
|
44
157
|
Only fill the "msgstr" field.
|
|
45
|
-
If an entry has "msgctxt", leave it unchanged and return it in the output (so same msgids for different contexts are not mixed).
|
|
46
158
|
|
|
47
159
|
Critical rules:
|
|
48
160
|
|
|
49
|
-
-
|
|
50
|
-
- NEVER translate, modify, reorder, or
|
|
161
|
+
- Copy ALL placeholders and non-linguistic tokens exactly, byte-for-byte.
|
|
162
|
+
- NEVER translate, modify, reorder, remove, escape, or unescape placeholders.
|
|
51
163
|
|
|
52
164
|
Placeholders include (but are not limited to):
|
|
53
165
|
- printf-style specifiers: %s, %d, %f, %1$s, %(name)s
|
|
@@ -87,22 +199,9 @@ Output:
|
|
|
87
199
|
|
|
88
200
|
You MUST respond with nothing but a single JSON object. No markdown, no code fences (no \`\`\`json or \`\`\`), no explanatory text before or after. The response must be parseable by JSON.parse() directly.
|
|
89
201
|
|
|
90
|
-
Return EXACTLY this structure (and nothing else):
|
|
91
|
-
|
|
92
|
-
{
|
|
93
|
-
"formula": "...",
|
|
94
|
-
"target_language": "...",
|
|
95
|
-
"source_language": "...",
|
|
96
|
-
"translations": [
|
|
97
|
-
{ "msgid": "...", "msgstr": "..." },
|
|
98
|
-
{ "msgid_plural": "...", "msgstr": ["...", "..."] }
|
|
99
|
-
]
|
|
100
|
-
}
|
|
101
|
-
|
|
102
202
|
Additional constraints:
|
|
103
203
|
|
|
104
204
|
- Preserve the exact input order of entries.
|
|
105
|
-
- Do not modify "formula", "target_language", or "source_language".
|
|
106
205
|
- Do not add, remove, or rename fields.`;
|
|
107
206
|
}
|
|
108
207
|
/** Strip markdown code fences if the model wrapped JSON in ```json ... ``` */
|
|
@@ -112,7 +211,7 @@ function stripJsonFences(raw) {
|
|
|
112
211
|
const match = trimmed.match(jsonBlock);
|
|
113
212
|
return match ? match[1].trim() : trimmed;
|
|
114
213
|
}
|
|
115
|
-
function parsePayloadResponse(content) {
|
|
214
|
+
function parsePayloadResponse(request, content) {
|
|
116
215
|
if (content == null || content.trim() === '') {
|
|
117
216
|
throw new Error('Empty response from OpenAI');
|
|
118
217
|
}
|
|
@@ -135,6 +234,9 @@ function parsePayloadResponse(content) {
|
|
|
135
234
|
console.warn('OpenAI model returned (raw):', raw);
|
|
136
235
|
throw new Error(`OpenAI response "translations" must be an array: ${raw.slice(0, 200)}`);
|
|
137
236
|
}
|
|
237
|
+
if (payload.translations.length !== request.translations.length) {
|
|
238
|
+
throw new Error('OpenAI response "translations" must have the same number of entries as input');
|
|
239
|
+
}
|
|
138
240
|
for (let i = 0; i < payload.translations.length; i++) {
|
|
139
241
|
const t = payload.translations[i];
|
|
140
242
|
if (t == null || typeof t !== 'object' || !('msgstr' in t)) {
|
|
@@ -143,12 +245,37 @@ function parsePayloadResponse(content) {
|
|
|
143
245
|
}
|
|
144
246
|
const entry = t;
|
|
145
247
|
const msgstr = entry.msgstr;
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
if (
|
|
248
|
+
const requestEntry = request.translations[i];
|
|
249
|
+
const requestContext = requestEntry.msgctxt;
|
|
250
|
+
if (entry.msgctxt !== requestContext) {
|
|
251
|
+
throw new Error(`OpenAI response translations[${i}].msgctxt must match the input exactly`);
|
|
252
|
+
}
|
|
253
|
+
if ('msgid' in requestEntry) {
|
|
254
|
+
if (entry.msgid !== requestEntry.msgid) {
|
|
255
|
+
throw new Error(`OpenAI response translations[${i}].msgid must match the input exactly`);
|
|
256
|
+
}
|
|
257
|
+
if ('msgid_plural' in entry) {
|
|
258
|
+
throw new Error(`OpenAI response translations[${i}] must not include msgid_plural`);
|
|
259
|
+
}
|
|
260
|
+
if (typeof msgstr === 'string')
|
|
261
|
+
continue;
|
|
262
|
+
console.warn('OpenAI model returned (raw):', raw);
|
|
263
|
+
throw new Error(`OpenAI response translations[${i}].msgstr must be a string`);
|
|
264
|
+
}
|
|
265
|
+
if (entry.msgid_plural !== requestEntry.msgid_plural) {
|
|
266
|
+
throw new Error(`OpenAI response translations[${i}].msgid_plural must match the input exactly`);
|
|
267
|
+
}
|
|
268
|
+
if ('msgid' in entry) {
|
|
269
|
+
throw new Error(`OpenAI response translations[${i}] must not include msgid`);
|
|
270
|
+
}
|
|
271
|
+
if (Array.isArray(msgstr) && msgstr.every((s) => typeof s === 'string')) {
|
|
272
|
+
if (request.plural_samples != null && msgstr.length !== request.plural_samples.length) {
|
|
273
|
+
throw new Error(`OpenAI response translations[${i}].msgstr must have length ${request.plural_samples.length}`);
|
|
274
|
+
}
|
|
149
275
|
continue;
|
|
276
|
+
}
|
|
150
277
|
console.warn('OpenAI model returned (raw):', raw);
|
|
151
|
-
throw new Error(`OpenAI response translations[${i}].msgstr must be
|
|
278
|
+
throw new Error(`OpenAI response translations[${i}].msgstr must be an array of strings`);
|
|
152
279
|
}
|
|
153
280
|
return payload;
|
|
154
281
|
}
|
|
@@ -161,17 +288,23 @@ async function translatePayload(payload, options) {
|
|
|
161
288
|
apiKey: options.apiKey,
|
|
162
289
|
});
|
|
163
290
|
const model = options?.model ?? DEFAULT_MODEL;
|
|
291
|
+
validateStructuredOutputModel(model);
|
|
164
292
|
for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
|
|
165
293
|
try {
|
|
166
294
|
const response = await client.chat.completions.create({
|
|
167
295
|
model,
|
|
296
|
+
temperature: 0,
|
|
297
|
+
response_format: {
|
|
298
|
+
type: 'json_schema',
|
|
299
|
+
json_schema: TRANSLATION_RESPONSE_SCHEMA,
|
|
300
|
+
},
|
|
168
301
|
messages: [
|
|
169
302
|
{ role: 'system', content: buildSystemMessage() },
|
|
170
303
|
{ role: 'user', content: JSON.stringify(payload) },
|
|
171
304
|
],
|
|
172
305
|
});
|
|
173
306
|
const content = response.choices[0]?.message?.content ?? null;
|
|
174
|
-
return parsePayloadResponse(content);
|
|
307
|
+
return parsePayloadResponse(payload, content);
|
|
175
308
|
}
|
|
176
309
|
catch (err) {
|
|
177
310
|
const shouldRetry = attempt < MAX_RETRIES && isApiError(err) && isRetryableStatus(err.status);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "msgai-cli",
|
|
3
|
-
"version": "1.0
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "CLI that automatically translates all untranslated strings in gettext (.po) files using AI (LLM)",
|
|
5
5
|
"main": "dist/src/cli/index.js",
|
|
6
6
|
"bin": {
|
|
@@ -15,6 +15,9 @@
|
|
|
15
15
|
"format": "prettier --write .",
|
|
16
16
|
"lint": "eslint .",
|
|
17
17
|
"lint:format": "prettier --check .",
|
|
18
|
+
"prerelease": "npm run build && npm test && npm run test:integration && npm run lint && npm run lint:format",
|
|
19
|
+
"release": "commit-and-tag-version",
|
|
20
|
+
"release:dry-run": "commit-and-tag-version --dry-run",
|
|
18
21
|
"script:test-openai": "ts-node scripts/test-openai.ts",
|
|
19
22
|
"prepublishOnly": "npm run build && npm test && npm run test:integration && npm run lint && npm run lint:format"
|
|
20
23
|
},
|
|
@@ -46,6 +49,10 @@
|
|
|
46
49
|
"url": "https://github.com/AlexMost/msgai/issues"
|
|
47
50
|
},
|
|
48
51
|
"homepage": "https://github.com/AlexMost/msgai#readme",
|
|
52
|
+
"commit-and-tag-version": {
|
|
53
|
+
"tagPrefix": "v",
|
|
54
|
+
"releaseCommitMessageFormat": "chore(release): {{currentTag}}"
|
|
55
|
+
},
|
|
49
56
|
"devDependencies": {
|
|
50
57
|
"@babel/core": "^7.29.0",
|
|
51
58
|
"@babel/preset-env": "^7.29.0",
|
|
@@ -54,6 +61,7 @@
|
|
|
54
61
|
"@types/jest": "^30.0.0",
|
|
55
62
|
"@types/node": "^25.3.0",
|
|
56
63
|
"babel-jest": "^30.2.0",
|
|
64
|
+
"commit-and-tag-version": "^12.6.1",
|
|
57
65
|
"eslint": "^10.0.2",
|
|
58
66
|
"jest": "^30.2.0",
|
|
59
67
|
"prettier": "^3.8.1",
|