@shipi18n/cli 2.9.0 → 2.11.0

package/README.md

@@ -96,7 +96,8 @@ directories, Apple String Catalogs (`shipi18n check Localizable.xcstrings`), And
 
 **What it catches:** missing and orphaned keys · dropped or invented placeholders (`{{name}}`,
 `{count}`, `%s`, `%1$s`, `%@`, `%lld`, `$t(...)`, `%{name}`, HTML tags) · collapsed vue-i18n pipe
-plurals · empty values · untranslated copy · stale `.xcstrings` states.
+plurals · empty values · untranslated copy · stale `.xcstrings` states · Android unescaped apostrophes
+and unbalanced quotes (the `values-fr/strings.xml` AAPT build breaker).
 
 | Flag | Default | Meaning |
 | --- | --- | --- |
@@ -104,12 +105,36 @@ plurals · empty values · untranslated copy · stale `.xcstrings` states.
 | `-r, --reporter <name>` | `human` | `human` \| `json` \| `sarif` \| `junit` |
 | `-o, --output <file>` | stdout | Write the report to a file |
 | `--ignore-keys <globs>` | — | Silence keys: `'*.copyright,home:mcp.badge'` |
+| `--severity <spec>` | — | Per-rule level override: `'untranslated=off,placeholder-added=error'` |
+| `--baseline <file>` | — | Fail only on findings NOT already in the baseline |
+| `--write-baseline` | — | Snapshot current findings into `--baseline` (default `.shipi18n/baseline.json`) and exit |
+| `--detect-secrets` | — | Flag secrets/PII (API keys, private keys, emails, cards) in locale strings |
 | `--fail-on <level>` | `error` | `error` \| `warning` \| `none` |
 | `--min-coverage <pct>` | — | Fail any language below this coverage |
 
 Exit codes: `0` pass, `1` findings at the fail level, `2` usage error. Errors may fail CI; warnings
 never do by default — a warning that blocks PRs gets the tool uninstalled.
 
+### Adopting on a messy catalog — baseline & severity
+
+A linter that fails on 2,000 pre-existing findings gets uninstalled by lunch. Baseline first, then
+fail only on what's **new** — the Stylelint/RuboCop pattern:
+
+```bash
+# 1. Snapshot today's findings (commit the file).
+npx @shipi18n/cli check ./locales --baseline .shipi18n/baseline.json --write-baseline
+
+# 2. CI from now on fails only on NEW findings; the backlog is accepted.
+npx @shipi18n/cli check ./locales --baseline .shipi18n/baseline.json
+```
+
+The baseline keys each finding on `(language, namespace, key path, rule)` — not the message text, so
+rewording a message never invalidates it. A missing baseline file is a cold start (warn + report all),
+not an error. Burn the backlog down by re-running `--write-baseline` whenever it shrinks.
+
+`--severity` tunes or silences a rule everywhere: `error`, `warning`, `info` (reported, never fails),
+or `off` (dropped entirely). Example: `--severity 'untranslated=off,empty-value=warning'`.
+
 ### GitHub Actions with PR annotations
 
 ```yaml
@@ -123,6 +148,91 @@ never do by default — a warning that blocks PRs gets the tool uninstalled.
     sarif_file: i18n.sarif
 ```
 
+### No Node? Run the check in any CI with Docker
+
+GitHub runners already have Node — the [Action](https://github.com/Shipi18n/shipi18n-github-action) is
+the easiest path there. Everywhere else (GitLab, Bitbucket, Jenkins, CircleCI, or local), the published
+image runs the linter with **no Node toolchain** — built for the PHP / Python / Ruby shops the
+`.po` / XLIFF / Android formats unlocked:
+
+```bash
+docker run --rm -v "$PWD:/work" ghcr.io/shipi18n/cli:latest check ./locales -s en
+```
+
+The check needs no API key. All CLI flags work — `--baseline .shipi18n/baseline.json`, `--reporter sarif`,
+etc. (`--write-baseline` writes back into the mounted repo). Pass `-e ANTHROPIC_API_KEY=...` only if you
+add `--semantic`.
+
+**GitLab CI** (`.gitlab-ci.yml`):
+
+```yaml
+i18n-check:
+  image: ghcr.io/shipi18n/cli:latest
+  script: [shipi18n check ./locales -s en]
+```
+
+**Bitbucket Pipelines** (`bitbucket-pipelines.yml`):
+
+```yaml
+pipelines:
+  default:
+    - step:
+        image: ghcr.io/shipi18n/cli:latest
+        script: [shipi18n check ./locales -s en]
+```
+
+### pre-commit
+
+Catch a dropped placeholder before it's even committed. Add to `.pre-commit-config.yaml` — pre-commit
+runs the Docker image, so **no Node is required**:
+
+```yaml
+repos:
+  - repo: https://github.com/Shipi18n/shipi18n
+    rev: v2.11.0
+    hooks:
+      - id: shipi18n-check
+        # args: ['check', './i18n', '-s', 'en']   # if not ./locales
+```
+
+Node users who prefer to skip Docker can use a repo-local hook instead:
+
+```yaml
+repos:
+  - repo: local
+    hooks:
+      - id: shipi18n-check
+        name: shipi18n check
+        language: node
+        additional_dependencies: ['@shipi18n/cli@2.11.0']
+        entry: shipi18n check ./locales -s en
+        pass_filenames: false
+        files: '\.(json|ya?ml|po|xlf|xliff|xml|arb|xcstrings)$'
+```
+
+## WordPress — `.po` ↔ JED `.json` sync (`wp-sync`)
+
+Since WP 5.0, JavaScript strings are translated from a JED-format JSON that `wp i18n make-json`
+**generates** from your `.po`. Edit the `.po`, forget to re-run make-json, and the PHP side shows the
+new translation while the JS side silently ships the stale one — and nothing in the WP toolchain
+flags it. `wp-sync` does:
+
+```bash
+# Auto-discovers the sibling *.json JED files next to the .po
+npx @shipi18n/cli wp-sync languages/plugin-es_ES.po
+
+# Or point at specific JED files / a directory
+npx @shipi18n/cli wp-sync languages/plugin-es_ES.po languages/build/*.json
+```
+
+It reports **`jed-drift`** (a JS translation that no longer matches the `.po` — re-run make-json) and
+**`jed-orphan`** (a JS string the `.po` dropped). A `.po`-only string is *not* flagged — the JED is a
+legitimate JS-only subset. Context and plural forms are compared independently.
+
+Shares the check flags: `--reporter json|sarif|junit` (SARIF gives PR annotations), `--baseline` /
+`--write-baseline` (fail only on new drift), `--severity` (e.g. `--severity 'jed-orphan=off'`),
+`--fail-on`. Exit `0` in sync, `1` drift/orphan, `2` no JED files found.
+
 ## Semantic QA — `--semantic` (the judge)
 
 The structural check cannot see a translation that is *fluent but wrong*. `--semantic` adds an
@@ -133,6 +243,25 @@ npx @shipi18n/cli check ./locales -s en --semantic          # advisory: warnings
 npx @shipi18n/cli check ./locales -s en --semantic --glossary glossary.json
 ```
 
+### Privacy — secret & PII pre-flight
+
+`--semantic` sends your source/translation pairs to a third-party model. A **pre-flight runs
+automatically** and *withholds* any pair containing an API key, private key, JWT, credit-card number,
+email, or phone — it is never transmitted (fail-closed), and a `secret-preflight` finding is recorded
+instead. Because the judge only ever sends what the pre-flight scans, nothing with a detected secret
+leaves your machine.
+
+You can also run the scan on its own, no LLM involved, to catch secrets that shouldn't be sitting in
+locale strings at all:
+
+```bash
+npx @shipi18n/cli check ./locales -s en --detect-secrets
+```
+
+High-confidence secrets (keys, cards via Luhn) are errors; emails/phones are warnings. Detection is
+precision-biased — known key prefixes, example/test emails ignored, phones need a `+` country code —
+and matches are always masked, never echoed. Tune per rule with `--severity secret-detected=off`.
+
 **Honest limitations, up front:** the judge is probabilistic. Every key is judged across 3 passes
 and flagged only on a majority vote, unparseable passes are discarded, and semantic findings are
 **warnings by default** — they never fail CI unless you opt in with `--semantic-fail`. It augments

package/bin/shipi18n.js

@@ -7,6 +7,7 @@ import { fileURLToPath } from 'node:url'
 import { translateCommand } from '../src/commands/translate.js'
 import { checkCommand } from '../src/commands/check.js'
 import { lockCommand } from '../src/commands/lock.js'
+import { wpSyncCommand } from '../src/commands/wp-sync.js'
 
 const __dirname = dirname(fileURLToPath(import.meta.url))
 const pkg = JSON.parse(readFileSync(join(__dirname, '../package.json'), 'utf8'))
@@ -37,5 +38,6 @@ ${chalk.gray('https://github.com/Shipi18n/shipi18n')}
 translateCommand(program)
 checkCommand(program)
 lockCommand(program)
+wpSyncCommand(program)
 program.parse(process.argv)
 if (!process.argv.slice(2).length) program.outputHelp()

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@shipi18n/cli",
-  "version": "2.9.0",
+  "version": "2.11.0",
   "description": "Catch broken translations before you ship: dropped placeholders, missing keys, collapsed plurals and — with your own LLM key — mistranslations the structure checks cannot see. CI-ready (SARIF, JUnit, exit codes). Translates too.",
   "type": "module",
   "bin": {
@@ -49,7 +49,7 @@
     "chalk": "^5.3.0",
     "commander": "^12.0.0",
     "ora": "^8.0.1",
-    "@shipi18n/core": "^2.9.0"
+    "@shipi18n/core": "^2.11.0"
   },
   "peerDependencies": {
     "@anthropic-ai/sdk": ">=0.30.0",

package/src/commands/check.js

@@ -24,6 +24,10 @@ import {
   aggregateLanguage,
   SEP,
   verdict,
+  applyPolicy,
+  buildBaseline,
+  parseSeverity,
+  scanResultSecrets,
 } from '@shipi18n/core'
 import { REPORTERS } from '../reporters.js'
 import { locksFor, DEFAULT_LOCKS_PATH } from './lock.js'
@@ -56,8 +60,12 @@ export function checkCommand(program) {
     .option('-o, --output <file>', 'Write the report to a file instead of stdout')
     .option('--json', 'Shorthand for --reporter json')
     .option('--ignore-keys <patterns>', "Comma-separated '*' globs of keys to silence (path or ns:path)")
+    .option('--severity <spec>', "Per-rule severity overrides, e.g. 'untranslated=off,placeholder-added=error' (error|warning|info|off)")
+    .option('--baseline <file>', 'Baseline file: findings already recorded in it do not fail the build (only NEW ones do)')
+    .option('--write-baseline', 'Snapshot current findings into --baseline (default .shipi18n/baseline.json) and exit')
     .option('--fail-on <level>', 'Exit non-zero on: error | warning | none', 'error')
     .option('--min-coverage <pct>', 'Fail any language below this coverage percentage', parseFloat)
+    .option('--detect-secrets', 'Flag secrets/PII (API keys, private keys, emails, cards) sitting in locale strings')
     .option('--glossary <file>', 'Glossary JSON: DNT terms + locked per-language translations (deterministic)')
     .option('--semantic', 'Add the LLM-as-judge pass (BYO key; advisory warnings by default)')
     .option('--semantic-fail', 'Escalate semantic findings to errors (opt-in)')
@@ -99,6 +107,11 @@ export function checkCommand(program) {
         return
       }
 
+      if (opts.detectSecrets) {
+        const flagged = scanResultSecrets(result)
+        if (flagged) console.error(chalk.gray(`secrets: ${flagged} locale string(s) flagged for possible secrets/PII`))
+      }
+
       if (opts.semantic) {
         const cachePath = resolve(opts.semanticCache)
         let cache = {}
@@ -130,7 +143,8 @@ export function checkCommand(program) {
           console.error(
             chalk.gray(
               `semantic: judged ${judge.judged} (${judge.cached} cached), flagged ${judge.flagged}, ` +
-                `${judge.calls} model call(s), ${judge.parseFailures} discarded pass(es)`
+                `${judge.calls} model call(s), ${judge.parseFailures} discarded pass(es)` +
+                (judge.redacted ? `, ${judge.redacted} withheld (secrets/PII)` : '')
             )
           )
           // "judged 0" on a badly broken tree is correct but reads as a broken
@@ -150,6 +164,62 @@ export function checkCommand(program) {
         }
       }
 
+      const DEFAULT_BASELINE = '.shipi18n/baseline.json'
+
+      // --write-baseline: snapshot every current finding and exit 0. Run after
+      // the (optional) semantic pass so a baseline can capture judge findings too.
+      if (opts.writeBaseline) {
+        const file = resolve(opts.baseline || DEFAULT_BASELINE)
+        const bl = buildBaseline(result)
+        try {
+          mkdirSync(dirname(file), { recursive: true })
+          writeFileSync(file, JSON.stringify(bl, null, 2) + '\n')
+        } catch (err) {
+          console.error(chalk.red(`Error: cannot write baseline ${file}: ${err.message}`))
+          process.exitCode = 2
+          return
+        }
+        console.error(chalk.gray(`baseline: recorded ${bl.count} finding(s) → ${opts.baseline || DEFAULT_BASELINE}`))
+        return
+      }
+
+      let severityMap
+      if (opts.severity) {
+        try {
+          severityMap = parseSeverity(opts.severity)
+        } catch (err) {
+          console.error(chalk.red(`Error: ${err.message}`))
+          process.exitCode = 2
+          return
+        }
+      }
+
+      let baseline
+      if (opts.baseline) {
+        const file = resolve(opts.baseline)
+        if (existsSync(file)) {
+          try {
+            baseline = JSON.parse(readFileSync(file, 'utf8'))
+          } catch (err) {
+            console.error(chalk.red(`Error: cannot read baseline ${opts.baseline}: ${err.message}`))
+            process.exitCode = 2
+            return
+          }
+        } else {
+          // A missing baseline is a cold start, not an error: nothing is suppressed
+          // and the run reports every finding. Hint how to create one.
+          console.error(chalk.yellow(`note: baseline ${opts.baseline} not found — reporting all findings. Create it with --write-baseline.`))
+        }
+      }
+
+      if (severityMap || baseline) {
+        const suppressed = applyPolicy(result, { severity: severityMap, baseline })
+        const parts = []
+        if (suppressed.suppressedByBaseline) parts.push(`${suppressed.suppressedByBaseline} baselined`)
+        if (suppressed.suppressedBySeverity) parts.push(`${suppressed.suppressedBySeverity} silenced (severity=off)`)
+        if (parts.length) console.error(chalk.gray(`policy: ${parts.join(', ')}`))
+      }
+
       const verdictResult = verdict(result, { failOn: opts.failOn, minCoverage: opts.minCoverage })
 
       const name = opts.json ? 'json' : opts.reporter

package/src/commands/wp-sync.js

@@ -0,0 +1,207 @@
+/**
+ * `shipi18n wp-sync` — the WordPress .po ↔ JED .json drift check.
+ *
+ * WP JS strings are translated from a JED JSON that `wp i18n make-json` generates
+ * from the .po. Edit the .po, forget to re-run make-json, and the JS side quietly
+ * ships the stale translation. This command catches that, per JED file, and reuses
+ * the same reporters/verdict/baseline machinery as `check` (so SARIF PR annotations
+ * and --baseline "fail only on new" work here too).
+ */
+import { readFileSync, writeFileSync, existsSync, mkdirSync, readdirSync, statSync } from 'node:fs'
+import { resolve, dirname, join, basename, relative } from 'node:path'
+import chalk from 'chalk'
+import {
+  checkJedSync,
+  isJed,
+  statsFrom,
+  aggregateLanguage,
+  verdict,
+  applyPolicy,
+  buildBaseline,
+  parseSeverity,
+} from '@shipi18n/core'
+import { REPORTERS } from '../reporters.js'
+
+const DEFAULT_BASELINE = '.shipi18n/baseline.json'
+const rel = (p) => relative(process.cwd(), p) || p
+
+/** JED .json files to check: explicit args (dirs expanded) or sibling auto-discovery. */
+function resolveJedFiles(poFile, args) {
+  const collect = (dir) => {
+    let out = []
+    try {
+      out = readdirSync(dir)
+        .filter((n) => n.endsWith('.json'))
+        .map((n) => join(dir, n))
+        .filter((f) => {
+          try {
+            return isJed(JSON.parse(readFileSync(f, 'utf8')))
+          } catch {
+            return false
+          }
+        })
+    } catch {
+      /* unreadable dir → nothing */
+    }
+    return out
+  }
+  if (!args || args.length === 0) return collect(dirname(resolve(poFile)))
+  const files = []
+  for (const a of args) {
+    const p = resolve(a)
+    if (existsSync(p) && statSync(p).isDirectory()) files.push(...collect(p))
+    else files.push(p)
+  }
+  return files
+}
+
+/** Wrap one JED file's findings as a "language" in the standard result shape. */
+function fileToLanguage(jedFile, findings) {
+  const ns = rel(jedFile)
+  const stats = statsFrom(findings, findings.length || 1, findings.length || 1)
+  return aggregateLanguage(basename(jedFile), [{ ns, findings, stats }])
+}
+
+/** Compact, wp-sync-flavoured human output (the default reporter). */
+function humanWpReport(result, v) {
+  const lines = ['', `🔎 shipi18n wp-sync — .po '${result.source}' vs ${result.languages.length} JED file(s)`, '']
+  for (const l of result.languages) {
+    const all = l.namespaces.flatMap((n) => n.findings)
+    const n = (t) => all.filter((f) => f.type === t).length
+    const mark = l.stats.errors ? chalk.red('✗') : all.length ? chalk.yellow('⚠') : chalk.green('✓')
+    const bad = n('invalid-file')
+    lines.push(
+      `${mark} ${chalk.bold(l.lang)}  ${n('jed-drift')} drift, ${n('jed-orphan')} orphan${bad ? `, ${bad} unreadable` : ''}`
+    )
+    for (const f of all.slice(0, 50)) {
+      const color = f.severity === 'error' ? chalk.red : f.severity === 'warning' ? chalk.yellow : chalk.gray
+      const detail =
+        f.type === 'jed-drift' ? chalk.gray(`  (.po: "${f.source}" → JSON: "${f.translation}")`) : ''
+      lines.push(`    ${color(f.severity)}  ${chalk.cyan(f.path)}  ${f.type} — ${f.message}${detail}`)
+    }
+    if (all.length > 50) lines.push(chalk.gray(`    … and ${all.length - 50} more`))
+  }
+  lines.push('')
+  lines.push(v.ok ? chalk.green('✓ .po and JED in sync') : chalk.red(`✗ out of sync: ${v.failures.join('; ')}`))
+  return lines.join('\n')
+}
+
+export function wpSyncCommand(program) {
+  program
+    .command('wp-sync <po> [jed...]')
+    .description('WordPress: check that JED .json files are in sync with their source .po (re-run make-json?)')
+    .option('-r, --reporter <name>', 'Output format: human | json | sarif | junit', 'human')
+    .option('-o, --output <file>', 'Write the report to a file instead of stdout')
+    .option('--json', 'Shorthand for --reporter json')
+    .option('--severity <spec>', "Per-rule severity overrides, e.g. 'jed-orphan=off' (error|warning|info|off)")
+    .option('--baseline <file>', 'Baseline file: findings already recorded in it do not fail the build')
+    .option('--write-baseline', 'Snapshot current findings into --baseline (default .shipi18n/baseline.json) and exit')
+    .option('--fail-on <level>', 'Exit non-zero on: error | warning | none', 'error')
+    .action((po, jed, opts) => {
+      const poFile = resolve(po)
+      let poText
+      try {
+        poText = readFileSync(poFile, 'utf8')
+      } catch (err) {
+        console.error(chalk.red(`Error: cannot read .po ${po}: ${err.message}`))
+        process.exitCode = 2
+        return
+      }
+
+      const jedFiles = resolveJedFiles(poFile, jed)
+      if (jedFiles.length === 0) {
+        console.error(
+          chalk.yellow(
+            `No JED .json files found next to ${po}. Pass them explicitly, or run \`wp i18n make-json\` first.`
+          )
+        )
+        process.exitCode = 2
+        return
+      }
+
+      const languages = []
+      for (const jf of jedFiles) {
+        let json
+        try {
+          json = JSON.parse(readFileSync(jf, 'utf8'))
+        } catch (err) {
+          languages.push(
+            fileToLanguage(jf, [{ type: 'invalid-file', severity: 'error', path: rel(jf), message: `invalid JSON: ${err.message}` }])
+          )
+          continue
+        }
+        const { findings } = checkJedSync(poText, json)
+        languages.push(fileToLanguage(jf, findings))
+      }
+
+      const result = { layout: 'wp-jed', source: basename(poFile), languages }
+      result.totals = languages.reduce(
+        (a, l) => ({ errors: a.errors + l.stats.errors, warnings: a.warnings + l.stats.warnings }),
+        { errors: 0, warnings: 0 }
+      )
+
+      if (opts.writeBaseline) {
+        const file = resolve(opts.baseline || DEFAULT_BASELINE)
+        try {
+          mkdirSync(dirname(file), { recursive: true })
+          writeFileSync(file, JSON.stringify(buildBaseline(result), null, 2) + '\n')
+        } catch (err) {
+          console.error(chalk.red(`Error: cannot write baseline ${file}: ${err.message}`))
+          process.exitCode = 2
+          return
+        }
+        console.error(chalk.gray(`baseline: recorded findings → ${opts.baseline || DEFAULT_BASELINE}`))
+        return
+      }
+
+      let severityMap
+      if (opts.severity) {
+        try {
+          severityMap = parseSeverity(opts.severity)
+        } catch (err) {
+          console.error(chalk.red(`Error: ${err.message}`))
+          process.exitCode = 2
+          return
+        }
+      }
+      let baseline
+      if (opts.baseline) {
+        const file = resolve(opts.baseline)
+        if (existsSync(file)) {
+          try {
+            baseline = JSON.parse(readFileSync(file, 'utf8'))
+          } catch (err) {
+            console.error(chalk.red(`Error: cannot read baseline ${opts.baseline}: ${err.message}`))
+            process.exitCode = 2
+            return
+          }
+        } else {
+          console.error(chalk.yellow(`note: baseline ${opts.baseline} not found — reporting all findings.`))
+        }
+      }
+      if (severityMap || baseline) applyPolicy(result, { severity: severityMap, baseline })
+
+      const verdictResult = verdict(result, { failOn: opts.failOn })
+      const name = opts.json ? 'json' : opts.reporter
+      let report
+      if (name === 'human') {
+        report = humanWpReport(result, verdictResult)
+      } else {
+        const reporter = REPORTERS[name]
+        if (!reporter) {
+          console.error(chalk.red(`Error: unknown reporter '${name}' (human | json | sarif | junit)`))
+          process.exitCode = 2
+          return
+        }
+        report = reporter(result, verdictResult, {})
+      }
+
+      if (opts.output) {
+        writeFileSync(opts.output, report.endsWith('\n') ? report : report + '\n')
+        if (name !== 'human') console.error(chalk.gray(`report written to ${opts.output}`))
+      } else {
+        console.log(report)
+      }
+      if (!verdictResult.ok) process.exitCode = 1
+    })
+}