@esgettext/tools 1.0.0 → 1.2.0

package/LICENSE

@@ -1,4 +1,4 @@
-This software is Copyright (C) 2020 by Guido Flohr.
+This software is Copyright (C) 2020-2024 by Guido Flohr.
 
 This is free software, licensed under:
 

package/README.md

@@ -6,15 +6,37 @@ Supporting tools for https://github.com/gflohr/esgettext.
 
 - [Status](#status)
 - [Prerequisites](#prerequisites)
-	- [Un*x/Linux](#unxlinux)
+	- [Un\*x/Linux](#unxlinux)
 	- [Mac OS X](#mac-os-x)
 		- [MacPorts](#macports)
 		- [HomeBrew](#homebrew)
 	- [MS-DOS (Microsoft Windows)](#ms-dos-microsoft-windows)
 - [Installation](#installation)
 - [The Tools](#the-tools)
-	- [`esgettext-gettextize`](#esgettext-gettextize)
-	- [`esgettext-xgettext`](#esgettext-xgettext)
+	- [`esgettext xgettext`](#esgettext-xgettext)
+	- [Other Commands](#other-commands)
+- [Configuration](#configuration)
+	- [Configuration Files](#configuration-files)
+	- [Validation](#validation)
+	- [Configuration Schema](#configuration-schema)
+		- [`package`](#package)
+			- [`package.textdomain`](#packagetextdomain)
+			- [`package.msgid-bugs-address`](#packagemsgid-bugs-address)
+			- [`package.name`](#packagename)
+			- [`package.copyright-holder`](#packagecopyright-holder)
+			- [`package.version`](#packageversion)
+		- [`po`:](#po)
+			- [`po.directory`](#podirectory)
+			- [`po.locales`](#polocales)
+		- [`install`](#install)
+			- [`install.directory`](#installdirectory)
+		- [`programs`](#programs)
+			- [`programs.msgmerge`](#programsmsgmerge)
+				- [`programs.msgmerge.path`](#programsmsgmergepath)
+				- [`programs.msgmerge.options`](#programsmsgmergeoptions)
+			- [`programs.msgfmt`](#programsmsgfmt)
+				- [`programs.msgfmt.path`](#programsmsgfmtpath)
+				- [`programs.msgfmt.options`](#programsmsgfmtoptions)
 - [Copyright](#copyright)
 
 ## Status
@@ -75,51 +97,204 @@ $ sudo npm install --global @esgettext/tools
 
 For [Yarn](https://yarnpkg.com/), you would do this:
 
-```
+```shell
 $ sudo yarn global add @esgettext/tools
 ```
 
-But you should rather install [npx](https://www.npmjs.com/package/npx), because
-then you can (es)gettextize your project like this:
+Normally, it is better to install them just locally:
 
-```
+```shell
 $ npm install --save-dev @esgettext/tools
-$ npx @esgettext/gettextize
 ```
 
 or for yarn:
 
-```
+```shell
 $ yarn add --dev @esgettext/tools
-$ npx @esgettext/gettextize
 ```
 
-This will modify your `package.json` with a couple of scripts, all named
-`po:.*`. You can then finetune your setup by just editing `package.json`.
-
 ## The Tools
 
-### `esgettext-gettextize`
+The tools are really just one tool `esgettext` that supports several
+commands.
 
-The script `esgettext-gettextize` sets up your project for being
-internationalized with esgettext.
+In the following, it is assumed that you have installed the tools locally.
+If you have installed them globally, just omit the leading `npx`.
 
-See the detailed
-[esgettext-gettextize usage information](./README-esgettext-gettextize.md)
-for more details.
+Try this for an overview:
 
-### `esgettext-xgettext`
+```shell
+$ npx esgettext --help
+```
+
+### `esgettext xgettext`
 
-The script `esgetext-xgettext` extracts translatable strings from your source
+The command `npx esgetext xgettext` extracts translatable strings from your source
 files into `.po` resp. `.pot` files.
 
+You can use `npx esgettext extract` as an alias for `npx esgettext xgettext`.
+
 See the detailed
 [esgettext-xgettext usage information](./README-esgettext-xgettext.md)
 for more details.
 
+### Other Commands
+
+All other commands should be understandable by their help output.  Try
+`npx esgettext --help` for an overview over all commands, and
+`npx esgettext COMMAND --help` for help for a specific command.
+
+## Configuration
+
+Instead of passing options on the command line, you can add a lot of defaults
+in a configuration file or just in `package.json`.
+
+### Configuration Files
+
+Configuration files are checked in this order:
+
+* esgettext.config.mjs
+* esgettext.config.cjs
+* esgettext.config.js
+* esgettext.config.json
+* package.json
+
+The JavaScript versions should have one default export with the configuration.
+The JSON version just defines the configuration. Alteratively, you can add
+a field "esgettext" to your `package.json`.
+
+You should always configure `po.locales` with a list of locale identifiers that
+your package supports.  Otherwise, using `esgettext` does not make sense.
+
+You should also set `package.textdomain` unless you are happy with the
+default which is just your package name read from `package.json`.
+
+All other configuration values have sane defaults.
+
+### Validation
+
+The required format is always the same.  Note that the configuration gets
+validated against a schema.  All tools will fail with a validation error if
+you pass an invalid configuration.
+
+### Configuration Schema
+
+All fields in the configuration are optional because you are always able to
+pass them on the command line.
+
+Options passed on the command line have higher precedence than options given
+in a configuration file.  Options that you define inside a section `esgettext`
+in `package.json` have higher precedence than options that default to
+general fields inside `package.json`.
+
+A complete example for a configuration can look like this:
+
+```javascript
+package: {
+	textdomain: 'com.example.my-package',
+	'msgid-bugs-address': 'you@example.com',
+	name: 'my-package',
+	'copyright-holder': 'Yours Truly <you@example.com>',
+},
+po: {
+	directory: 'po',
+	locales: ['de', 'fr-CA', 'fr-FR', 'it'],
+}
+```
+
+#### `package`
+
+General information about your package.
+
+##### `package.textdomain`
+
+The textdomain of your package, usually something like
+`com.example.YOUR-PACKAGE`.  You should configure this.
+
+##### `package.msgid-bugs-address`
+
+An email address or URL where to send bug reports or questions about message
+ids.  This is added to the respective field in all generated po files.
+
+If you omit this field, it will be read from the fields `bugs.email` or
+`bugs.url` (in that order) in `package.json`.
+
+##### `package.name`
+
+The name of your package.
+
+If you omit this field, it will be read from the field `name` in
+`package.json`.
+
+##### `package.copyright-holder`
+
+The copyright holder of your package.  This is added as a comment to all
+generated po files.
+
+If you omit this field, it will be read from the field `people.author` in
+`package.json`.
+
+
+##### `package.version`
+
+The version of your package.
+
+If you omit this field, it will be read from the field `version` in
+`package.json`.
+
+#### `po`:
+
+The location of your translation and translation workflow files and feature
+thereof.
+
+##### `po.directory`
+
+The directory where all translation related files reside, usually 'po'.
+
+##### `po.locales`
+
+An array of locale identifiers.  This should always be configured.
+
+#### `install`
+
+Installation options.
+
+##### `install.directory`
+
+Where to install compiled translation files, usually something like
+`src/locale`, `assets/locale`, or `dist/assets/locale`.
+
+#### `programs`
+
+Information for helper programs.
+
+##### `programs.msgmerge`
+
+###### `programs.msgmerge.path`
+
+The path to the `msgmerge` program of your system.
+
+###### `programs.msgmerge.options`
+
+An array of options to pass to the `msgmerge` program.  Only boolean options
+without arguments are supported.  And you have to omit the leading hyphens,
+for example `['verbose']` or `['v']`, and not `['--verbose']` or `['-v']`.
+
+##### `programs.msgfmt`
+
+###### `programs.msgfmt.path`
+
+The path to the `msgfmt` program of your system.
+
+###### `programs.msgfmt.options`
+
+An array of options to pass to the `msgmerge` program.  Only boolean options
+without arguments are supported.  And you have to omit the leading hyphens,
+for example `['verbose']` or `['v']`, and not `['--verbose']` or `['-v']`.
+
 ## Copyright
 
-Copyright (C) 2020 Guido Flohr <guido.flohr@cantanea.com>, all
+Copyright (C) 2020-2024 Guido Flohr <guido.flohr@cantanea.com>, all
 rights reserved.
 
 This software is available under the terms and conditions of the

package/dist/command.js

@@ -0,0 +1,2 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });

package/dist/commands/convert.js

@@ -0,0 +1,286 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || function (mod) {
+    if (mod && mod.__esModule) return mod;
+    var result = {};
+    if (mod != null) for (var k in mod) if (k !== "default" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);
+    __setModuleDefault(result, mod);
+    return result;
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.Convert = void 0;
+const fs = __importStar(require("fs"));
+const runtime_1 = require("@esgettext/runtime");
+const package_1 = require("../package");
+const gtx = runtime_1.Textdomain.getInstance('com.cantanea.esgettext-tools');
+class Convert {
+    // eslint-disable-next-line @typescript-eslint/no-unused-vars
+    constructor(_) {
+        this.options = undefined;
+    }
+    synopsis() {
+        return `[${gtx._('INPUTFILE')}] [${gtx._('OUTPUTFILE')}]`;
+    }
+    description() {
+        return gtx._('Convert translation catalogs.');
+    }
+    aliases() {
+        return [];
+    }
+    args() {
+        return {
+            input: {
+                alias: 'i',
+                type: 'string',
+                describe: gtx._('Input file (stdin when omitted)'),
+                group: gtx._('File locations:'),
+            },
+            output: {
+                alias: 'o',
+                type: 'string',
+                describe: gtx._('Output file (stdout when omitted)'),
+                group: gtx._('File locations:'),
+            },
+            'input-format': {
+                alias: 'I',
+                type: 'string',
+                describe: gtx._('Input file format (defaul: derived from filename)'),
+                choices: ['mo.json', 'mo', 'gmo', 'json'],
+                coerce: arg => arg.toLowerCase(),
+                group: gtx._('File formats:'),
+            },
+            'output-format': {
+                alias: 'O',
+                type: 'string',
+                describe: gtx._('Output file format (default: derived from format)'),
+                choices: ['mo.json', 'json'],
+                coerce: arg => arg.toLowerCase(),
+                group: gtx._('File formats:'),
+            },
+            verbose: {
+                alias: 'V',
+                type: 'boolean',
+                describe: gtx._('Enable verbose output'),
+            },
+        };
+    }
+    additional(argv) {
+        argv.positional(gtx._('INPUTFILE'), {
+            type: 'string',
+            describe: gtx._('Input file (or option -i or standard input)'),
+        });
+        argv.positional(gtx._('OUTPUTFILE'), {
+            type: 'string',
+            describe: gtx._('Output file (or option -o or standard output)'),
+        });
+        argv.conflicts('input', 'INPUTFILE');
+        argv.conflicts('output', 'OUTPUTFILE');
+    }
+    init(argv) {
+        const options = argv;
+        this.options = options;
+    }
+    run(argv) {
+        this.init(argv);
+        const options = this.options;
+        return new Promise(resolve => {
+            const input = options.input || options[gtx._('INPUTFILE')];
+            const output = options.output || options[gtx._('OUTPUTFILE')];
+            const inputFormat = this.getFileFormat(input);
+            if (!this.checkInputFormat(inputFormat, input)) {
+                return resolve(1);
+            }
+            if (options.verbose) {
+                console.error(gtx._x('Detected input format {inputFormat}.', { inputFormat }));
+            }
+            const outputFormat = this.getFileFormat(output);
+            if (!this.checkOutputFormat(outputFormat, output)) {
+                return resolve(1);
+            }
+            if (options.verbose) {
+                console.error(gtx._x('Detected output format {outputFormat}.', { outputFormat }));
+            }
+            try {
+                const inBuffer = this.readInput(input);
+                let catalog;
+                switch (options.inputFormat) {
+                    case 'mo.json':
+                        catalog = (0, runtime_1.parseMoJsonCatalog)(inBuffer);
+                        break;
+                    case 'mo':
+                    case 'gmo':
+                    default:
+                        catalog = (0, runtime_1.parseMoCatalog)(inBuffer);
+                        break;
+                }
+                const converted = this.convert(catalog, input, outputFormat);
+                if (!converted) {
+                    return resolve(1);
+                }
+                this.output(output, converted);
+            }
+            catch (e) {
+                console.error(gtx._x('{programName}: Error: {e}'));
+                return resolve(1);
+            }
+            return resolve(0);
+        });
+    }
+    getFileFormat(input) {
+        if (this.options.inputFormat) {
+            return this.options.inputFormat;
+        }
+        else if (typeof input !== 'undefined') {
+            const match = input.match(/\.(mo\.json|g?mo|json)$/i);
+            if (match) {
+                return match[1].toLowerCase();
+            }
+        }
+        return;
+    }
+    checkInputFormat(format, filename) {
+        if (!format) {
+            if (typeof filename === 'undefined') {
+                console.error(gtx._x("{programName}: Error: The option '--inputFormat' is mandatory, when reading from standard input", { programName: package_1.Package.getName() }));
+            }
+            else {
+                console.error(gtx._x("{programName}: Error: Please specify the input format with '--inputFormat'! Cannot guess it from input filename '{filename}'!", { programName: package_1.Package.getName(), filename }));
+            }
+            return false;
+        }
+        else if (format !== 'mo.json' &&
+            format !== 'mo' &&
+            format !== 'gmo' &&
+            format !== 'json') {
+            console.error(gtx._("Only 'mo.json', 'mo', 'gmo', and 'json' are allowed as input formats!"));
+            return false;
+        }
+        return true;
+    }
+    checkOutputFormat(format, filename) {
+        if (!format) {
+            if (typeof filename === 'undefined') {
+                console.error(gtx._x("{programName}: Error: The option '--outputFormat' is mandatory, when writing to standard input", { programName: package_1.Package.getName() }));
+            }
+            else {
+                console.error(gtx._x("{programName}: Error: Please specify the output format with '--outputFormat'! Cannot guess it from input filename '{filename}'!", { programName: package_1.Package.getName(), filename }));
+            }
+            return false;
+        }
+        else if (format !== 'mo.json' && format !== 'json') {
+            console.error(gtx._("Only 'mo.json' and 'json' are allowed as output formats!"));
+            return false;
+        }
+        return true;
+    }
+    readInput(filename) {
+        try {
+            if (filename === '-' && !fs.existsSync(filename)) {
+                filename = undefined;
+            }
+            if (typeof filename !== 'undefined') {
+                return fs.readFileSync(filename);
+            }
+            // Must read from standard input.
+            const stdinFd = 0;
+            const chunks = [];
+            let bytesRead;
+            const buffer = Buffer.alloc(8192);
+            while ((bytesRead = fs.readSync(stdinFd, buffer, 0, buffer.length, null)) > 0) {
+                chunks.push(Buffer.from(buffer.subarray(0, bytesRead)));
+            }
+            return Buffer.concat(chunks);
+        }
+        catch (error) {
+            if (typeof filename === 'undefined') {
+                filename = gtx._('[standard input]');
+            }
+            throw Error(gtx._x('{filename}: read failed: {error}', { filename, error }));
+        }
+    }
+    convert(catalog, filename, outputFormat) {
+        if (typeof filename === 'undefined') {
+            filename = gtx._('[standard input]');
+        }
+        if (outputFormat === 'json') {
+            const entries = {};
+            let errors = 0;
+            for (const [msgid, msgstr] of Object.entries(catalog.entries)) {
+                if (msgstr.length > 1) {
+                    ++errors;
+                    console.error(gtx._x("{filename}: Error: msgid '{msgid}': plural forms are not supported by the 'json' catalog format!", { filename, msgid: this.shortenAndEscapeString(msgid, 40) }));
+                }
+                else {
+                    entries[msgid] = msgstr[0];
+                }
+            }
+            if (errors) {
+                return undefined;
+            }
+            return JSON.stringify(entries);
+        }
+        else {
+            return JSON.stringify(catalog);
+        }
+    }
+    shortenAndEscapeString(input, maxLength) {
+        const controlCharMap = {
+            '\0': '\\0', // Null character
+            '\b': '\\b', // Backspace
+            '\t': '\\t', // Horizontal Tab
+            '\n': '\\n', // New Line
+            '\v': '\\v', // Vertical Tab
+            '\f': '\\f', // Form Feed
+            '\r': '\\r', // Carriage Return
+            '"': '\\"', // Double quote
+            "'": "\\'", // Single quote
+            '\\': '\\\\', // Backslash
+        };
+        const escapeControlChar = (char) => {
+            if (controlCharMap[char]) {
+                return controlCharMap[char];
+            }
+            const code = char.charCodeAt(0);
+            if (code < 32) {
+                return `\\x${code.toString(16).padStart(2, '0')}`;
+            }
+            return char; // This line is technically redundant with the current regex.
+        };
+        // Escape all control characters
+        const escapedInput = input.replace(/[\x00-\x1F"'\\]/g, escapeControlChar);
+        let truncatedString = escapedInput;
+        if (escapedInput.length > maxLength) {
+            truncatedString = escapedInput.slice(0, maxLength - 3) + '...';
+        }
+        return truncatedString;
+    }
+    output(filename, data) {
+        if (typeof filename === 'undefined') {
+            console.log(data);
+        }
+        else {
+            try {
+                fs.writeFileSync(filename, data, { encoding: 'utf-8' });
+            }
+            catch (error) {
+                throw Error(gtx._x('{filename}: write failed: {error}', { filename, error }));
+            }
+        }
+    }
+}
+exports.Convert = Convert;