altalandi 0.5.0 → 0.6.0

package/README.md

@@ -114,6 +114,85 @@ altalandi --report --file draft.txt    # also write a Markdown report to ./altal
 | `--report-dir <dir>` | | write the report to `<dir>` instead (implies `--report`) |
 | `--fast` | off | `preserveMeaning:false`, `scoreNativeness:false`, `onlyImprove:false`, `polishPass:false`, `scoring:"none"` — normally one model call (structured input can still add a structure-preserving retry) |
 
+## Folders, i18n catalogs and websites
+
+One command for a whole folder, one web page, or a site. The input is never
+written to.
+
+```bash
+altalandi ./docs                        # every .md .txt .html .json under ./docs → ./docs-is
+altalandi ./locales --out ./locales-is  # i18n catalogs: values only, placeholders kept
+altalandi ./docs --dry-run              # every passage and the estimate; sends nothing
+altalandi https://example.com/pricing --out pricing-is.md   # one page → Markdown
+altalandi https://example.com/docs/ --site --max-pages 50    # a site → ./example.com-is/
+```
+
+Each passage (a paragraph, list item, table cell, heading or string) is its own
+call. Everything outside the passages is written back byte for byte:
+
+| format | refined | never touched |
+|--------|---------|---------------|
+| Markdown | paragraphs, list items, table cells, heading text, front-matter `title`/`description`-style values | code blocks, inline code, URLs, link targets, HTML tags, front-matter keys, heading markup |
+| HTML | text in elements (inline tags travel as numbered markers and come back exactly) | tags, attributes, entities, `script`, `style`, `pre`, `code`, `translate="no"` |
+| JSON | string values of i18n catalogs | keys, `@`/`$` metadata, `{name}` `{{name}}` `%s` `%(name)s`, ICU plural/select structure, HTML in strings |
+| Text | paragraphs (re-wrapped to their width) and lines | |
+
+Passages with nothing to refine (code, numbers, URLs or placeholders alone) are
+skipped, and a passage that appears twice is sent once. `node_modules`, `.git`,
+`dist`, `build`, `altalandi-reports`, hidden files and folders and symlinks are
+skipped. JSON config (`package.json`, `tsconfig.json`, …) and JSON with numbers
+or booleans is copied unchanged; other file types are not copied.
+
+Pages are fetched one at a time with a User-Agent naming altalandi, only where
+robots.txt allows it for `altalandi` and for `*` (`Crawl-delay` is honoured).
+The readable part (`<main>` or `<article>`, else `<body>` without navigation,
+header, footer, sidebars, forms, scripts and hidden elements) becomes Markdown.
+`--site` reads `/sitemap.xml`, or follows links when there is none, staying on
+the same origin and under the URL's folder.
+
+**Cost and safety.** Before anything is sent, the CLI prints an estimate, priced
+from the public tariff:
+
+```text
+5 files, 24 passages, about 851 characters (23 calls: repeated passages are sent once).
+1 more file has nothing to refine and is copied unchanged.
+Billing: about 920 units (at least 40 a call), about 74 kr if paid from credit. The free allowance is 2,000 units a day.
+```
+
+Every call costs at least the per-call minimum, so many short strings cost more
+than their word count. In a terminal it asks before sending; without one (CI,
+agents) it sends nothing unless you pass `--yes`.
+
+| flag | default | meaning |
+|------|---------|---------|
+| `--out <path>` | `<folder>-is` | output folder, refused if it is the input, inside it or around it; for one page, the `.md` file |
+| `--dry-run` | off | list every passage and the estimate; send nothing, write nothing |
+| `--yes`, `-y` | off | send without asking |
+| `--max-chars <n>` | | files are sent in order until the next would go over n characters; the rest are neither sent nor written |
+| `--concurrency <n>` | `2` | calls at once, 1–4 |
+| `--site` / `--max-pages <n>` | off / `20` | read a site from the URL, up to n pages |
+| `--no-store` | off | keep none of the passages (free use is otherwise kept, scrubbed of personal data) |
+| `--json` | off | a machine-readable summary of every file and passage on stdout |
+
+A `429` waits as long as `Retry-After` asks (up to 5 minutes) and retries. A
+`402` stops the run cleanly: finished files are written, the rest are listed and
+not written; run the same command again after topping up.
+
+**Results.** Only verified changes are written. A passage keeps its original
+text when its meaning check did not pass (`meaningCheck` other than `passed`, or
+`meaningPreserved` not `true`), or when the refined text lost or changed code, a
+URL, a placeholder or a tag. One report next to the output
+(`altalandi-reports/altalandi-<time>-<input>.md`) lists every file and every
+changed passage with its meaning verdict, and the kept passages under their own
+heading with the unused suggestion and the reason.
+
+| exit | meaning |
+|------|---------|
+| `0` | done |
+| `2` | done, but some passages kept their original text (see the report) |
+| `4` | stopped: out of credit or allowance |
+| `1` | error, or not everything was done (a failed call, `--max-chars`, no `--yes`) |
+
 ## Before and after
 
 `--diff` prints what the website shows inline, in the terminal:

package/altalandi.mjs

@@ -5,10 +5,14 @@
 //   npx altalandi login                 # once: connects this computer to your account
 //   altalandi "Make sure to reach out to us today."
 //   echo "some text" | altalandi --domain marketing
+//   altalandi ./docs                    # a whole folder → ./docs-is (batch.mjs)
+//   altalandi https://example.com --site
 //
 // stdout = the refined text (so you can pipe it); stderr = nativeness score,
 // meaning verdict, API warnings, and the diff.
 
+import { existsSync, statSync } from "node:fs";
+
 import { refine } from "./client.mjs";
 import { beforeAfterText, costLines, outcomeOf, reportEnabled, reviewMarkdown, writeReport } from "./report.mjs";
 import { readCredentials, resolveKey, runLogin, runLogout, runWhoami } from "./auth.mjs";
@@ -72,7 +76,10 @@ applied · 1 the check or the update failed.`);
 }
 
 if (!handled) {
-const opts = { domain: "general", register: "neutral", json: false, fast: false, diff: false, review: false, report: undefined };
+const opts = {
+  domain: "general", register: "neutral", json: false, fast: false, diff: false, review: false, report: undefined,
+  site: false, dryRun: false, yes: false,
+};
 const positional = [];
 for (let i = 0; i < args.length; i++) {
   const a = args[i];
@@ -90,6 +97,12 @@ for (let i = 0; i < args.length; i++) {
   else if (a === "--no-store") opts.noStore = true;
   else if (a === "--report") opts.report = true;
   else if (a === "--report-dir") { opts.report = true; opts.reportDir = args[++i]; }
+  else if (a === "--site") opts.site = true;
+  else if (a === "--max-pages") opts.maxPages = Number(args[++i]);
+  else if (a === "--dry-run") opts.dryRun = true;
+  else if (a === "--yes" || a === "-y") opts.yes = true;
+  else if (a === "--max-chars") opts.maxChars = Number(args[++i]);
+  else if (a === "--concurrency") opts.concurrency = Number(args[++i]);
   else positional.push(a);
 }
 
@@ -105,6 +118,13 @@ Usage:
   echo "text" | altalandi
   altalandi --domain marketing --register warm "text"
 
+Folders and websites:
+  altalandi ./docs                      every .md .txt .html .json under ./docs → ./docs-is
+  altalandi ./locales --out ./locales-is
+  altalandi https://example.com/about --out about-is.md    one page, as Markdown
+  altalandi https://example.com --site                      a site: /sitemap.xml, else its links
+  altalandi ./docs --dry-run            list what would be sent and the estimate; send nothing
+
 Commands:
   altalandi login      open altalandi.is, sign in or sign up, approve; stores the key
                        and installs the Claude Code skill (--no-skill, --no-browser)
@@ -122,7 +142,8 @@ Options:
   --diff       show the text before and after, and every change with its reason (stderr)
   --review     print one Markdown block on stdout to show the user as it is: verdict,
                before, after, every change with its reason, the cost and the balance
-  --out        write the refined text to this UTF-8 file (use with --review)
+  --out        write the refined text to this UTF-8 file (use with --review;
+               for a folder or a URL, see below)
   --no-store   don't keep this text: free use is otherwise kept, scrubbed of personal
                data, to improve the service (paid use is kept only with consent)
   --report     also write a Markdown report (input, output, changes, reasons, verdict)
@@ -131,7 +152,23 @@ Options:
   --base-url   API base  (default $ALTALANDI_BASE_URL or https://altalandi.is/api)
 
 stdout = refined text; stderr = nativeness / meaning / warnings / diff.
-Without --fast, false/null meaning verification exits 3 after printing the response.`);
+Without --fast, false/null meaning verification exits 3 after printing the response.
+
+Folder and website options:
+  --out          output folder (default <folder>-is; never the input or inside it),
+                 or the .md file for one page
+  --dry-run      show every passage and the estimate; send nothing, write nothing
+  --yes, -y      send without asking (needed when there is no terminal to ask in)
+  --max-chars    send at most this many characters; files past the budget are left out
+  --concurrency  calls at once, 1-4 (default 2)
+  --site         with a URL: its pages on the same site, under the URL's folder
+  --max-pages    pages to read with --site (default 20)
+  --json         print a machine-readable summary at the end
+Code, URLs, link targets, placeholders ({name}, %s, ICU), HTML tags and JSON keys
+stay byte for byte. Only changes whose meaning check passed are written; other
+passages keep the original text and are listed in the report
+(./altalandi-reports/). Exit: 0 done · 2 some passages kept their original ·
+4 out of credit · 1 error.`);
 }
 
 function readStdin() {
@@ -152,6 +189,19 @@ if (opts.key) {
   console.error("⚠ --key is deprecated because command arguments can leak through shell history/process listings; load ALTALANDI_API_KEY instead.");
 }
 
+// A folder or a URL as the one argument: many documents in one run (batch.mjs).
+const target = positional.length === 1 && !opts.file ? positional[0] : "";
+const many = /^https?:\/\//i.test(target) || (Boolean(target) && existsSync(target) && statSync(target).isDirectory());
+if (many) {
+  const { runBatch } = await import("./batch.mjs");
+  const { currentVersion } = await import("./update.mjs");
+  process.exitCode = await runBatch({ ...opts, target, key, baseUrl }, { version: await currentVersion() });
+} else {
+if (opts.site || opts.dryRun || opts.maxChars !== undefined || opts.maxPages !== undefined || opts.concurrency !== undefined) {
+  console.error("--site, --dry-run, --max-chars, --max-pages and --concurrency need a folder or a URL:  altalandi ./docs --dry-run");
+  process.exit(2);
+}
+
 let text = positional.join(" ").trim();
 if (opts.file) {
   const { readFile } = await import("node:fs/promises");
@@ -271,4 +321,5 @@ try {
   if (e.status === 401) console.error("→ The key is invalid or revoked. Run:  npx altalandi login");
   process.exit(e.status === 402 ? 4 : 1);
 }
+} // single text
 }