@opencraw/captcha-tesseract 0.0.2

package/README.md

@@ -0,0 +1,130 @@
+# @opencraw/captcha-tesseract
+
+A captcha reader for [OpenCraw](../../README.md): it reads the code in an image captcha with Tesseract and fills
+the form's answer field. The engine then posts the form, checks the page and retries.
+
+It is meant for simple image captchas: a short code drawn in a clean font, like the public Vahan registrations
+report's. Widget captchas (reCAPTCHA, hCaptcha, Turnstile) are behavioural checks with no text to read, and need a
+widget solver instead.
+
+```sh
+npm install @opencraw/captcha-tesseract
+```
+
+Tesseract runs in WebAssembly, and the English model ships in the package, so nothing is downloaded at run time.
+
+## Use
+
+Register the reader as a captcha solver:
+
+```js
+// plugins.mjs (opencraw run --plugins plugins.mjs), or createCrawler({ captchaSolvers })
+import { tesseractReader } from '@opencraw/captcha-tesseract'
+
+export const captchaSolvers = [
+  tesseractReader({
+    charset:       'A-Za-z0-9',   // the characters the site uses: a list with ranges, or /[A-HJ-NP-Z2-9]/
+    length:        6,             // a read of any other length is refreshed, never submitted
+    caseSensitive: true,
+    audit:         (read) => console.log(read.outcome, read.text, read.confidence),
+  }),
+]
+```
+
+Then name it in a form captcha step, placed after the steps that fill the form:
+
+```json
+{ "type": "captcha", "solver": "tesseract",
+  "image":   "#captchaImage",
+  "refresh": "#captchaImg",
+  "field":   "#externalCaptcha",
+  "submit":  [ { "type": "click", "selector": "#applyTrigger" } ],
+  "verify":  { "selector": "#makerDynamicReportHeader", "failure": "#captchaMsg" },
+  "attempts": 5 }
+```
+
+## What one solve does
+
+1. **Wait for the image to settle.** Changing a filter can make a page draw a new captcha in the background, so the
+   reader waits until the image's `src` stops changing.
+2. **Screenshot the image element.** It never downloads the `src` again: many sites draw a new code on every
+   request, so a second download would show a different code from the one on screen.
+3. **Clean the image.**
+   - Scale it up 3× smoothly, before thresholding, so curves stay curves and an `8` stays an `8`.
+   - Threshold it to black ink on white. Otsu's method picks the cut, and light-on-dark text is inverted.
+   - Pad it with a white margin.
+   - Optionally, a median filter removes speckle.
+4. **Read it with Tesseract,** as one line of text, limited to the `charset`.
+5. **Fix the case of look-alike letters** (with `caseSensitive`):
+   - `c o s u v w x z` are told from their capitals by height, against the capitals and digits on the same line;
+   - `p` and `y` by the tail under the line.
+
+   Tesseract reads one line with no size reference, so it cannot tell `o` from `O` on its own.
+6. **Check the read.** Only characters from the charset are kept. The read must be exactly `length` characters,
+   and every character must be read with at least `minConfidence`. The least sure character decides, because
+   Tesseract's whole-word confidence is too noisy for six loose characters.
+7. **Refresh or fill.**
+   - A read that fails the check makes the reader click `refresh` and read again, up to `refreshes` times. It
+     never submits it. This is the only check possible before the form is posted, and it is free: refreshing
+     posts no form and risks no lockout. A last character cut off by the image's edge is the usual failure.
+   - A read that passes goes into the `field`, and the reader returns `solved`.
+
+The engine then runs `submit`, reads `verify.selector` (accepted) or `verify.failure` (refused), and gives the
+verdict back to the reader, which passes it to `audit`.
+
+## Options
+
+| Option | Default | What it is |
+|---|---|---|
+| `name` | `tesseract` | The name recipes use. Register several readers under different names, one per site. |
+| `charset` | `A-Za-z0-9` | The characters the captcha uses: a list with `a-z` ranges, or a one-character RegExp class. |
+| `length` | any | The code's length, or `[min, max]`. |
+| `caseSensitive` | `false` | Whether `a` and `A` differ. When `false`, each letter allows its other case too. |
+| `minConfidence` | 50 | The least confidence (0–100) every character must have. |
+| `refreshes` | 5 | Reads (each after a refresh) before an attempt gives up. |
+| `preprocess` | | `{ scale: 3, threshold: 'otsu', invert: 'auto', median: false, padding: 10 }` |
+| `pageSegMode` | 7 | Tesseract page segmentation: `7` one line, `8` one word, `13` a raw line. `7` measured best. |
+| `lang`, `langPath` | the bundled English model | Another Tesseract language, and where its `.traineddata.gz` is. |
+| `timeoutMs` | 60000 | Time one attempt may take, refreshes included, when the recipe sets none. |
+| `audit` | | Called for every read. A throw is logged and ignored. |
+
+`reader.read(png)` reads a PNG without a page, for measuring a site's captchas offline.
+`reader.close()` stops the Tesseract worker; the crawler calls it when it closes.
+
+## The audit
+
+Each read reaches `audit` once, with:
+- `text` (filtered) and `raw` (as Tesseract read it);
+- `confidence` (the least sure character) and `symbols` (each character with its confidence and box);
+- `image` and `cleaned` (PNG buffers of the captured image and of what Tesseract saw);
+- `url`, `attempt`, `read` (the read within the attempt), `durationMs`;
+- `outcome`, plus `reason` when it failed:
+  - `refreshed`: failed the check, never submitted;
+  - `rejected`: submitted, and the site refused it;
+  - `solved`: submitted, and the site accepted it.
+
+Every `solved` record pairs an image with its confirmed text. Use them to narrow `charset` to what the site really
+draws, to set `minConfidence`, or later to fine-tune a Tesseract model for the site. To collect them without a
+working reader, run the built-in `manual` solver headed: a person types the codes.
+
+## Accuracy
+
+These figures were measured when the package was written.
+
+**The e2e corpus:** 6-character mixed-case codes in a bold sans-serif, 140×40, drawn like the Vahan report's.
+Over 80 codes:
+- 78% read right;
+- 10% refused and refreshed;
+- 13% read wrong and submitted, so the site refuses them and the engine tries again.
+
+Of the reads that were submitted, 86% were right. With 5 attempts, a captcha is left unsolved in fewer than one
+run in 10,000.
+
+**Real Vahan images:** of 5, 4 were read right, including one whose last character is clipped. The fifth was
+read with a doubled letter (7 characters), so it was refused and refreshed, never submitted.
+
+**The misses** are the letters mixed case makes ambiguous: `k`/`K`, `j`/`J`, `8`/`B`, `q`/`g`. A site with an
+unambiguous charset (no look-alikes, one case) reads better.
+
+Solving a site's captcha can break its terms of service. Read OpenCraw's [captcha guide](../../docs/recipes/captcha.md)
+first.

package/dist/index.d.ts

@@ -0,0 +1 @@
+export * from "./src/index.js";

package/dist/index.esm.js

@@ -0,0 +1,501 @@
+import { PNG } from 'pngjs';
+import { createRequire } from 'node:module';
+import { join, dirname } from 'node:path';
+import { createWorker, OEM } from 'tesseract.js';
+
+/** The printable ASCII characters a RegExp charset is tested against. */
+const PRINTABLE = Array.from({
+  length: 0x7E - 0x21 + 1
+}, (_, index) => String.fromCodePoint(0x21 + index));
+/**
+ * The characters a captcha may use, as one string without repeats: what
+ * Tesseract is limited to (`tessedit_char_whitelist`) and what a read is
+ * filtered through.
+ *
+ * A string lists characters, with `a-z` style ranges (`'A-Za-z0-9'`,
+ * `'ABCDEFGHJKLMNPQRSTUVWXYZ23456789'`); a `-` first or last is itself. A
+ * RegExp is a one-character class (`/[A-HJ-NP-Z2-9]/`), tested against every
+ * printable ASCII character. Unless `caseSensitive`, each letter brings its
+ * other case: a case-blind site draws either.
+ *
+ * @param charset - The list, or the class.
+ * @param caseSensitive - Whether `a` and `A` are different characters.
+ * @returns The characters.
+ * @throws Error when the charset allows nothing.
+ */
+function expandCharset(charset, caseSensitive) {
+  const listed = typeof charset === 'string' ? expandRanges(charset) : PRINTABLE.filter(character => new RegExp(charset.source, charset.flags.replaceAll('g', '')).test(character));
+  const characters = new Set(caseSensitive ? listed : listed.flatMap(character => [character, character.toLowerCase(), character.toUpperCase()]));
+  if (characters.size === 0) throw new Error(`the charset ${String(charset)} allows no character`);
+  return [...characters].join('');
+}
+/**
+ * A read kept to the charset: characters outside it (a stray `|` for an `l`, a
+ * space) are dropped.
+ *
+ * @param text - What Tesseract read.
+ * @param characters - From {@link expandCharset}.
+ * @returns The filtered read.
+ */
+function keepCharset(text, characters) {
+  return [...text].filter(character => characters.includes(character)).join('');
+}
+function expandRanges(list) {
+  const characters = [];
+  const chars = [...list];
+  for (let index = 0; index < chars.length; index += 1) {
+    const from = chars[index];
+    const to = chars[index + 2];
+    if (to !== undefined && chars[index + 1] === '-') {
+      const start = from.codePointAt(0) ?? 0;
+      const end = to.codePointAt(0) ?? 0;
+      if (end < start) throw new Error(`the charset range ${from}-${to} runs backwards`);
+      for (let code = start; code <= end; code += 1) characters.push(String.fromCodePoint(code));
+      index += 2;
+    } else {
+      characters.push(from);
+    }
+  }
+  return characters;
+}
+
+const WHITE = 255;
+const BLACK = 0;
+/**
+ * Cleans a captcha image for OCR: grayscale, black ink on white (a threshold
+ * picked by Otsu's method unless given, turned the right way round), an
+ * optional median filter, scaled up with hard edges, and a white margin.
+ *
+ * Like photocopying a smudged note at high contrast before reading it.
+ *
+ * @param png - The image, as PNG bytes (a screenshot).
+ * @param options - The cleanup settings.
+ * @returns The cleaned image, as PNG bytes.
+ */
+function cleanImage(png, options = {}) {
+  const image = PNG.sync.read(png);
+  const scale = Math.max(1, Math.round(options.scale ?? 3));
+  const padding = Math.max(0, Math.round(options.padding ?? 10));
+  // Scaled up smoothly before thresholding: anti-aliased edges become smooth curves, not staircases (an 8 stays an 8).
+  const width = image.width * scale;
+  const height = image.height * scale;
+  const luminance = upscale(luminanceOf(image.data, image.width * image.height), image.width, image.height, scale);
+  const threshold = options.threshold === undefined || options.threshold === 'otsu' ? otsuThreshold(luminance) : options.threshold;
+  let ink = luminance.map(value => value <= threshold ? 1 : 0);
+  const inkShare = ink.reduce((sum, value) => sum + value, 0) / ink.length;
+  if (options.invert === true || (options.invert ?? 'auto') === 'auto' && inkShare > 0.5) ink = ink.map(value => 1 - value);
+  if (options.median === true) ink = medianOf(ink, width, height);
+  return PNG.sync.write(render(ink, width, height, padding));
+}
+/**
+ * The threshold that best splits the luminances in two groups (Otsu): the
+ * value maximising the variance between ink and ground.
+ *
+ * @param luminance - One 0–255 value per pixel.
+ * @returns The threshold: values at or under it are ink.
+ */
+function otsuThreshold(luminance) {
+  const histogram = Array.from({
+    length: 256
+  }, () => 0);
+  for (const value of luminance) histogram[value] += 1;
+  const total = luminance.length;
+  const sum = histogram.reduce((accumulator, count, value) => accumulator + value * count, 0);
+  let below = 0;
+  let belowSum = 0;
+  let best = -1;
+  let threshold = 127;
+  for (let value = 0; value < 256; value += 1) {
+    below += histogram[value];
+    if (below === 0) continue;
+    const above = total - below;
+    if (above === 0) break;
+    belowSum += value * histogram[value];
+    const between = below * above * (belowSum / below - (sum - belowSum) / above) ** 2;
+    if (between > best) {
+      best = between;
+      threshold = value;
+    }
+  }
+  return threshold;
+}
+function luminanceOf(rgba, pixels) {
+  const luminance = new Uint8Array(pixels);
+  for (let pixel = 0; pixel < pixels; pixel += 1) {
+    const offset = pixel * 4;
+    const alpha = rgba[offset + 3] / 255;
+    // A transparent pixel is the white page behind it.
+    const blend = channel => channel * alpha + WHITE * (1 - alpha);
+    const red = blend(rgba[offset]);
+    const green = blend(rgba[offset + 1]);
+    const blue = blend(rgba[offset + 2]);
+    luminance[pixel] = Math.round(0.299 * red + 0.587 * green + 0.114 * blue);
+  }
+  return luminance;
+}
+function medianOf(ink, width, height) {
+  const filtered = new Uint8Array(ink.length);
+  for (let y = 0; y < height; y += 1) {
+    for (let x = 0; x < width; x += 1) filtered[y * width + x] = majorityAround(ink, width, height, x, y);
+  }
+  return filtered;
+}
+/** 1 when most of the pixel's 3×3 neighbourhood (inside the image) is ink. */
+function majorityAround(ink, width, height, x, y) {
+  let votes = 0;
+  let seen = 0;
+  for (let ny = Math.max(0, y - 1); ny <= Math.min(height - 1, y + 1); ny += 1) {
+    for (let nx = Math.max(0, x - 1); nx <= Math.min(width - 1, x + 1); nx += 1) {
+      votes += ink[ny * width + nx];
+      seen += 1;
+    }
+  }
+  return votes * 2 > seen ? 1 : 0;
+}
+function render(ink, width, height, padding) {
+  const outWidth = width + padding * 2;
+  const out = new PNG({
+    width: outWidth,
+    height: height + padding * 2
+  });
+  out.data.fill(WHITE);
+  for (let y = 0; y < height; y += 1) {
+    for (let x = 0; x < width; x += 1) {
+      const start = ((y + padding) * outWidth + x + padding) * 4;
+      if (ink[y * width + x] === 1) out.data.fill(BLACK, start, start + 3);
+    }
+  }
+  return out;
+}
+/** Bilinear upscaling of a luminance map. */
+function upscale(luminance, width, height, scale) {
+  if (scale === 1) return luminance;
+  const outWidth = width * scale;
+  const scaled = new Uint8Array(outWidth * height * scale);
+  const at = (x, y) => luminance[Math.min(height - 1, y) * width + Math.min(width - 1, x)];
+  for (let y = 0; y < height * scale; y += 1) {
+    const sourceY = Math.max(0, (y + 0.5) / scale - 0.5);
+    const top = Math.floor(sourceY);
+    const down = sourceY - top;
+    for (let x = 0; x < outWidth; x += 1) {
+      const sourceX = Math.max(0, (x + 0.5) / scale - 0.5);
+      const left = Math.floor(sourceX);
+      const right = sourceX - left;
+      const upper = at(left, top) * (1 - right) + at(left + 1, top) * right;
+      const lower = at(left, top + 1) * (1 - right) + at(left + 1, top + 1) * right;
+      scaled[y * outWidth + x] = Math.round(upper * (1 - down) + lower * down);
+    }
+  }
+  return scaled;
+}
+
+/** Letters whose small and capital forms have the same shape: only their height tells them apart. */
+const SAME_SHAPE = new Set('cosuvwxz');
+/** Letters whose small form has a tail under the line, and whose capital does not. */
+const TAILED = new Set('py');
+/** Small letters that reach under the line: not used to find it. */
+const DESCENDERS = new Set('gjpqy');
+/** Characters as tall as a capital: digits and capitals. */
+const CAPITAL_HEIGHT = /^[\dA-Z]$/;
+/**
+ * Puts right the case of letters that Tesseract, reading one line with no
+ * reference, cannot tell by shape: `c`/`C`, `o`/`O`, `s`/`S`, `v`/`V`, `w`/`W`,
+ * `x`/`X`, `z`/`Z` by their height against the line's capitals and digits,
+ * `p`/`P` and `y`/`Y` by a tail under the line. (`j`/`J` are left: many
+ * fonts drop the capital under the line too.) Only letters whose two cases
+ * the charset both allows are changed; a line with no capital or digit to
+ * measure against is left as read.
+ *
+ * Like telling `o` from `O` in a handwritten code by comparing it with the `8`
+ * next to it.
+ *
+ * @param symbols - The characters read, with their boxes.
+ * @param characters - The charset.
+ * @returns The characters, with their case put right.
+ */
+function fixLetterCase(symbols, characters) {
+  const boxed = symbols.filter(symbol => symbol.box !== undefined);
+  const capitals = boxed.filter(symbol => CAPITAL_HEIGHT.test(symbol.text) && !SAME_SHAPE.has(symbol.text.toLowerCase()) && !TAILED.has(symbol.text.toLowerCase()));
+  if (capitals.length === 0) return [...symbols];
+  const capital = median(capitals.map(symbol => heightOf(symbol)));
+  // Either case of a tailed letter may be misread, so neither helps find the line.
+  const baseline = median(boxed.filter(symbol => !DESCENDERS.has(symbol.text.toLowerCase())).map(symbol => symbol.box?.y1 ?? 0));
+  return symbols.map(symbol => {
+    const lower = symbol.text.toLowerCase();
+    const upper = symbol.text.toUpperCase();
+    if (lower === upper || symbol.box === undefined || !characters.includes(lower) || !characters.includes(upper)) return symbol;
+    if (SAME_SHAPE.has(lower)) {
+      const ratio = heightOf(symbol) / capital;
+      if (ratio >= 0.86) return {
+        ...symbol,
+        text: upper
+      };
+      if (ratio <= 0.8) return {
+        ...symbol,
+        text: lower
+      };
+    }
+    if (TAILED.has(lower)) return {
+      ...symbol,
+      text: symbol.box.y1 > baseline + capital * 0.12 ? lower : upper
+    };
+    return symbol;
+  });
+}
+function heightOf(symbol) {
+  return symbol.box === undefined ? 0 : symbol.box.y1 - symbol.box.y0;
+}
+function median(values) {
+  // A copy, sorted in place: the package targets ES2022, which has no toSorted.
+  const sorted = Float64Array.from(values).sort();
+  return sorted[Math.floor(sorted.length / 2)] ?? 0;
+}
+
+/**
+ * Judges a read before anything is submitted: kept to the charset, it must
+ * have the code's length, and every character must be sure enough. This is
+ * the only check possible before posting the form, and it is free: a read
+ * that fails it is refreshed, never sent. A clipped last character is the
+ * usual failure: Tesseract reads it short, or unsure.
+ *
+ * @param raw - Tesseract's text.
+ * @param symbols - Its characters with their confidence.
+ * @param rules - The charset, the length, the least confidence.
+ * @returns The filtered text, its least confidence, and the problem, if any.
+ */
+function judgeRead(raw, symbols, rules) {
+  const text = keepCharset(raw, rules.characters);
+  const kept = symbols.filter(symbol => keepCharset(symbol.text, rules.characters) !== '');
+  const confidence = kept.length === 0 ? 0 : Math.round(Math.min(...kept.map(symbol => symbol.confidence)));
+  const [shortest, longest] = typeof rules.length === 'number' ? [rules.length, rules.length] : rules.length ?? [1, Infinity];
+  if (text.length < shortest || text.length > longest) return {
+    text,
+    confidence,
+    problem: `read ${text.length} characters ("${text}"), expected ${shortest === longest ? shortest : `${shortest} to ${longest}`}`
+  };
+  if (confidence < rules.minConfidence) return {
+    text,
+    confidence,
+    problem: `a character read at ${confidence}% confidence, under ${rules.minConfidence}%`
+  };
+  return {
+    text,
+    confidence
+  };
+}
+
+/** The bundled English model: the small LSTM one, which is what a line of loose characters needs. */
+function bundledLangPath() {
+  const require = createRequire(import.meta.url);
+  return join(dirname(require.resolve('@tesseract.js-data/eng/package.json')), '4.0.0_best_int');
+}
+/**
+ * One Tesseract worker (a thread running Tesseract in WebAssembly), started
+ * on the first read and reused: reads queue on it. The language data comes
+ * from disk, never from a CDN, and nothing is cached in the working
+ * directory.
+ */
+class TesseractEngine {
+  options;
+  worker;
+  constructor(options) {
+    this.options = options;
+  }
+  start() {
+    this.worker ??= (async () => {
+      const worker = await createWorker(this.options.lang, OEM.LSTM_ONLY, {
+        langPath: this.options.langPath ?? bundledLangPath(),
+        gzip: true,
+        cacheMethod: 'none'
+      });
+      await worker.setParameters({
+        tessedit_char_whitelist: this.options.whitelist,
+        tessedit_pageseg_mode: String(this.options.pageSegMode)
+      });
+      return worker;
+    })();
+    return this.worker;
+  }
+  /**
+   * Reads one image.
+   *
+   * @param png - The cleaned image.
+   * @returns The text and each character with its confidence.
+   */
+  async read(png) {
+    const worker = await this.start();
+    const {
+      data
+    } = await worker.recognize(png, {}, {
+      text: true,
+      blocks: true
+    });
+    return {
+      raw: data.text.trim(),
+      symbols: symbolsOf(data.blocks ?? [])
+    };
+  }
+  /** Stops the worker thread, if it started. */
+  async close() {
+    const worker = this.worker;
+    this.worker = undefined;
+    if (worker === undefined) return;
+    const started = await worker;
+    await started.terminate();
+  }
+}
+function symbolsOf(blocks) {
+  return blocks.flatMap(block => block.paragraphs.flatMap(paragraph => paragraph.lines.flatMap(line => line.words.flatMap(word => word.symbols.map(symbol => ({
+    text: symbol.text,
+    confidence: symbol.confidence,
+    box: symbol.bbox
+  }))))));
+}
+
+const DEFAULT_REFRESHES = 5;
+const DEFAULT_MIN_CONFIDENCE = 50;
+const DEFAULT_TIMEOUT_MS = 60_000;
+/** An image counts as settled once its `src` stayed the same this long. */
+const SETTLE_MS = 300;
+const SETTLE_LIMIT_MS = 5000;
+/**
+ * A captcha reader for OpenCraw's form captchas: it reads the code in the
+ * challenge's image with Tesseract, refreshes the image while the read is not
+ * good enough (wrong length, an unsure character, and no form is posted for
+ * that), fills the challenge's answer field, and returns `solved`. The engine
+ * posts the form, checks the page and tells the verdict, which goes to
+ * `audit` with the read.
+ *
+ * @param options - The charset, the length, the checks, the cleanup, the audit.
+ * @returns The reader, to register as a captcha solver.
+ */
+function tesseractReader(options = {}) {
+  const caseSensitive = options.caseSensitive ?? false;
+  const characters = expandCharset(options.charset ?? 'A-Za-z0-9', caseSensitive);
+  const rules = {
+    characters,
+    length: options.length,
+    minConfidence: options.minConfidence ?? DEFAULT_MIN_CONFIDENCE
+  };
+  const engine = new TesseractEngine({
+    lang: options.lang ?? 'eng',
+    langPath: options.langPath,
+    whitelist: characters,
+    pageSegMode: options.pageSegMode ?? 7
+  });
+  const refreshes = options.refreshes ?? DEFAULT_REFRESHES;
+  /** The read an attempt submitted, waiting for the page's verdict. */
+  const submitted = new WeakMap();
+  const audit = async (read, context) => {
+    try {
+      await options.audit?.(read);
+    } catch (error) {
+      context?.log('warn', `audit failed: ${error instanceof Error ? error.message : String(error)}`);
+    }
+  };
+  const read = async png => {
+    const started = Date.now();
+    const cleaned = cleanImage(png, options.preprocess);
+    const {
+      raw,
+      symbols: read
+    } = await engine.read(cleaned);
+    // Tesseract sees one line with no size reference: the case of same-shape letters is put right by their size.
+    const symbols = caseSensitive ? fixLetterCase(read, characters) : read;
+    const judged = judgeRead(caseSensitive ? symbols.map(symbol => symbol.text).join('') : raw, symbols, rules);
+    return {
+      ...judged,
+      raw,
+      symbols,
+      image: png,
+      cleaned,
+      durationMs: Date.now() - started
+    };
+  };
+  const solve = async (challenge, context) => {
+    const {
+      page,
+      signal
+    } = context;
+    if (challenge.selector === undefined) return {
+      status: 'failed',
+      reason: `a ${challenge.kind} challenge has no image to read`
+    };
+    if (challenge.field === undefined) return {
+      status: 'failed',
+      reason: 'the captcha step names no field for the answer'
+    };
+    let problem = 'no read';
+    for (let attempt = 1; attempt <= refreshes; attempt += 1) {
+      if (signal.aborted) break;
+      await settle(page, challenge.selector);
+      const result = await read(await page.locator(challenge.selector).first().screenshot());
+      const record = {
+        ...result,
+        url: challenge.url,
+        attempt: context.attempt,
+        read: attempt,
+        outcome: 'refreshed'
+      };
+      if (result.problem === undefined) {
+        await page.locator(challenge.field).first().fill(result.text);
+        submitted.set(challenge, record);
+        context.log('debug', `read "${result.text}" (${result.confidence}%)`);
+        return {
+          status: 'solved'
+        };
+      }
+      problem = result.problem;
+      await audit({
+        ...record,
+        reason: problem
+      }, context);
+      if (attempt === refreshes || challenge.refresh === undefined) break;
+      context.log('debug', `refreshing the image: ${problem}`);
+      await page.locator(challenge.refresh).first().click();
+    }
+    return {
+      status: 'failed',
+      reason: `no good read after ${refreshes} image${refreshes === 1 ? '' : 's'}: ${problem}`
+    };
+  };
+  return {
+    name: options.name ?? 'tesseract',
+    timeoutMs: options.timeoutMs ?? DEFAULT_TIMEOUT_MS,
+    solve,
+    read,
+    async verdict(challenge, verdict) {
+      const record = submitted.get(challenge);
+      if (record === undefined) return;
+      submitted.delete(challenge);
+      await audit(verdict.status === 'solved' ? {
+        ...record,
+        outcome: 'solved'
+      } : {
+        ...record,
+        outcome: 'rejected',
+        reason: verdict.reason
+      });
+    },
+    close: () => engine.close()
+  };
+}
+/**
+ * Waits for the image to settle: its `src` unchanged for a moment and the
+ * element visible. Changing a filter can make a page draw a new captcha in
+ * the background; reading the old one would submit a stale code.
+ */
+async function settle(page, selector) {
+  const image = page.locator(selector).first();
+  const deadline = Date.now() + SETTLE_LIMIT_MS;
+  let source = await image.getAttribute('src');
+  for (;;) {
+    await page.waitForTimeout(SETTLE_MS);
+    const now = await image.getAttribute('src');
+    if (Date.now() >= deadline || now === source && (await image.isVisible())) return;
+    source = now;
+  }
+}
+
+export { cleanImage, expandCharset, judgeRead, keepCharset, otsuThreshold, tesseractReader };
+//# sourceMappingURL=index.esm.js.map

package/dist/src/charset/charset.algorithm.d.ts

@@ -0,0 +1,27 @@
+/**
+ * The characters a captcha may use, as one string without repeats: what
+ * Tesseract is limited to (`tessedit_char_whitelist`) and what a read is
+ * filtered through.
+ *
+ * A string lists characters, with `a-z` style ranges (`'A-Za-z0-9'`,
+ * `'ABCDEFGHJKLMNPQRSTUVWXYZ23456789'`); a `-` first or last is itself. A
+ * RegExp is a one-character class (`/[A-HJ-NP-Z2-9]/`), tested against every
+ * printable ASCII character. Unless `caseSensitive`, each letter brings its
+ * other case: a case-blind site draws either.
+ *
+ * @param charset - The list, or the class.
+ * @param caseSensitive - Whether `a` and `A` are different characters.
+ * @returns The characters.
+ * @throws Error when the charset allows nothing.
+ */
+export declare function expandCharset(charset: string | RegExp, caseSensitive: boolean): string;
+/**
+ * A read kept to the charset: characters outside it (a stray `|` for an `l`, a
+ * space) are dropped.
+ *
+ * @param text - What Tesseract read.
+ * @param characters - From {@link expandCharset}.
+ * @returns The filtered read.
+ */
+export declare function keepCharset(text: string, characters: string): string;
+//# sourceMappingURL=charset.algorithm.d.ts.map

package/dist/src/charset/index.d.ts

@@ -0,0 +1,2 @@
+export { expandCharset, keepCharset } from './charset.algorithm.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/src/image-cleanup/clean-image.algorithm.d.ts

@@ -0,0 +1,34 @@
+/** How a captcha image is cleaned before Tesseract reads it. */
+export interface CleanupOptions {
+    /** How many times larger: Tesseract reads glyphs of 20 px and more best. Default 3. */
+    scale?: number;
+    /** The luminance (0–255) under which a pixel is ink; `'otsu'` (the default) finds it from the image. */
+    threshold?: number | 'otsu';
+    /** Light text on a dark ground is turned dark on light: `'auto'` (the default) when ink covers most of the image. */
+    invert?: boolean | 'auto';
+    /** A 3×3 median filter against salt-and-pepper noise, run after thresholding. Default off. */
+    median?: boolean;
+    /** White margin around the text, in output pixels: Tesseract misses glyphs touching the edge. Default 10. */
+    padding?: number;
+}
+/**
+ * Cleans a captcha image for OCR: grayscale, black ink on white (a threshold
+ * picked by Otsu's method unless given, turned the right way round), an
+ * optional median filter, scaled up with hard edges, and a white margin.
+ *
+ * Like photocopying a smudged note at high contrast before reading it.
+ *
+ * @param png - The image, as PNG bytes (a screenshot).
+ * @param options - The cleanup settings.
+ * @returns The cleaned image, as PNG bytes.
+ */
+export declare function cleanImage(png: Buffer, options?: CleanupOptions): Buffer;
+/**
+ * The threshold that best splits the luminances in two groups (Otsu): the
+ * value maximising the variance between ink and ground.
+ *
+ * @param luminance - One 0–255 value per pixel.
+ * @returns The threshold: values at or under it are ink.
+ */
+export declare function otsuThreshold(luminance: Uint8Array): number;
+//# sourceMappingURL=clean-image.algorithm.d.ts.map

package/dist/src/image-cleanup/index.d.ts

@@ -0,0 +1,3 @@
+export { cleanImage, otsuThreshold } from './clean-image.algorithm.js';
+export type { CleanupOptions } from './clean-image.algorithm.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/src/index.d.ts

@@ -0,0 +1,6 @@
+export { tesseractReader, judgeRead } from './reader/index.js';
+export type { ChallengeLike, ImageRead, ReadSymbol, SolveContextLike, TesseractRead, TesseractReader, TesseractReaderOptions } from './reader/index.js';
+export { expandCharset, keepCharset } from './charset/index.js';
+export { cleanImage, otsuThreshold } from './image-cleanup/index.js';
+export type { CleanupOptions } from './image-cleanup/index.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/src/reader/index.d.ts

@@ -0,0 +1,4 @@
+export { tesseractReader } from './tesseract-reader.use-case.js';
+export { judgeRead } from './read-check.policy.js';
+export type { ChallengeLike, ImageRead, ReadSymbol, SolveContextLike, TesseractRead, TesseractReader, TesseractReaderOptions } from './reader.contract.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/src/reader/letter-case.algorithm.d.ts

@@ -0,0 +1,19 @@
+import type { ReadSymbol } from './reader.contract.js';
+/**
+ * Puts right the case of letters that Tesseract, reading one line with no
+ * reference, cannot tell by shape: `c`/`C`, `o`/`O`, `s`/`S`, `v`/`V`, `w`/`W`,
+ * `x`/`X`, `z`/`Z` by their height against the line's capitals and digits,
+ * `p`/`P` and `y`/`Y` by a tail under the line. (`j`/`J` are left: many
+ * fonts drop the capital under the line too.) Only letters whose two cases
+ * the charset both allows are changed; a line with no capital or digit to
+ * measure against is left as read.
+ *
+ * Like telling `o` from `O` in a handwritten code by comparing it with the `8`
+ * next to it.
+ *
+ * @param symbols - The characters read, with their boxes.
+ * @param characters - The charset.
+ * @returns The characters, with their case put right.
+ */
+export declare function fixLetterCase(symbols: readonly ReadSymbol[], characters: string): ReadSymbol[];
+//# sourceMappingURL=letter-case.algorithm.d.ts.map

package/dist/src/reader/read-check.policy.d.ts

@@ -0,0 +1,25 @@
+import type { ReadSymbol } from './reader.contract.js';
+export interface ReadRules {
+    characters: string;
+    length?: number | [number, number];
+    minConfidence: number;
+}
+export interface JudgedRead {
+    text: string;
+    confidence: number;
+    problem?: string;
+}
+/**
+ * Judges a read before anything is submitted: kept to the charset, it must
+ * have the code's length, and every character must be sure enough. This is
+ * the only check possible before posting the form, and it is free: a read
+ * that fails it is refreshed, never sent. A clipped last character is the
+ * usual failure: Tesseract reads it short, or unsure.
+ *
+ * @param raw - Tesseract's text.
+ * @param symbols - Its characters with their confidence.
+ * @param rules - The charset, the length, the least confidence.
+ * @returns The filtered text, its least confidence, and the problem, if any.
+ */
+export declare function judgeRead(raw: string, symbols: readonly ReadSymbol[], rules: ReadRules): JudgedRead;
+//# sourceMappingURL=read-check.policy.d.ts.map

package/dist/src/reader/reader.contract.d.ts

@@ -0,0 +1,114 @@
+import type { CleanupOptions } from '../image-cleanup/index.js';
+/** An element on the page, as Playwright's `Locator` offers it. */
+export interface ElementHandleLike {
+    first: () => ElementHandleLike;
+    screenshot: () => Promise<Buffer>;
+    click: () => Promise<void>;
+    fill: (value: string) => Promise<void>;
+    getAttribute: (name: string) => Promise<string | null>;
+    isVisible: () => Promise<boolean>;
+}
+export interface PageLike {
+    locator: (selector: string) => ElementHandleLike;
+    waitForTimeout: (ms: number) => Promise<void>;
+}
+/** The challenge as the engine hands it over: the image's selector, and the form's field and refresh control. */
+export interface ChallengeLike {
+    kind: string;
+    url: string;
+    selector?: string;
+    field?: string;
+    refresh?: string;
+}
+export interface SolveContextLike {
+    page: PageLike;
+    attempt: number;
+    signal: AbortSignal;
+    log: (level: 'debug' | 'info' | 'warn' | 'error', message: string, meta?: Record<string, unknown>) => void;
+}
+export type SolveOutcome = {
+    status: 'solved';
+} | {
+    status: 'failed';
+    reason: string;
+};
+export type SolveVerdict = {
+    status: 'solved';
+} | {
+    status: 'rejected';
+    reason: string;
+};
+/** One character Tesseract read, how sure it was (0–100), and where (pixels of the cleaned image). */
+export interface ReadSymbol {
+    text: string;
+    confidence: number;
+    box?: {
+        x0: number;
+        y0: number;
+        x1: number;
+        y1: number;
+    };
+}
+/** What one reading of an image gave. */
+export interface ImageRead {
+    /** What was read, kept to the charset. */
+    text: string;
+    /** Tesseract's text before filtering. */
+    raw: string;
+    /** The least sure character's confidence (0–100): whole-word confidence is too noisy on six loose characters. */
+    confidence: number;
+    symbols: ReadSymbol[];
+    /** Why the read is not good enough to submit (a wrong length, a low confidence), when it is not. */
+    problem?: string;
+    /** The image as captured, and as Tesseract saw it after cleanup (PNG). */
+    image: Buffer;
+    cleaned: Buffer;
+    durationMs: number;
+}
+/** One read as the audit sees it, with how it ended. */
+export interface TesseractRead extends ImageRead {
+    url: string;
+    /** The engine's attempt, from 1. */
+    attempt: number;
+    /** The read within the attempt, from 1: each refresh adds one. */
+    read: number;
+    /** `refreshed`: not good enough, never submitted. `rejected` / `solved`: submitted, and what the page said. */
+    outcome: 'refreshed' | 'rejected' | 'solved';
+    reason?: string;
+}
+export interface TesseractReaderOptions {
+    /** The name recipes use. Default `tesseract`. */
+    name?: string;
+    /** The characters the captcha uses: a list with ranges (`'A-Za-z0-9'`, the default) or a one-character RegExp class. */
+    charset?: string | RegExp;
+    /** The code's length, or its range. A read of another length is refreshed, never submitted. */
+    length?: number | [number, number];
+    /** Whether `a` and `A` differ. Default `false`: each letter allows its other case too. */
+    caseSensitive?: boolean;
+    /** The least confidence (0–100) every character must have. Default 50. */
+    minConfidence?: number;
+    /** Reads (after refreshing the image) before an attempt gives up. Default 5. */
+    refreshes?: number;
+    /** Image cleanup before OCR. */
+    preprocess?: CleanupOptions;
+    /** Tesseract's page segmentation mode: 7 one line (default), 8 one word, 13 a raw line. */
+    pageSegMode?: 7 | 8 | 13;
+    /** Tesseract language (default `eng`) and where its `.traineddata.gz` is (default: the bundled English model). */
+    lang?: string;
+    langPath?: string;
+    /** Called for every read; a throw is logged and ignored. */
+    audit?: (read: TesseractRead) => Promise<void> | void;
+    /** How long one attempt may take, refreshes included, when the recipe does not say. Default 60000. */
+    timeoutMs?: number;
+}
+/** The reader: an OpenCraw captcha solver, and `read` for reading an image directly. */
+export interface TesseractReader {
+    readonly name: string;
+    readonly timeoutMs: number;
+    solve(challenge: ChallengeLike, context: SolveContextLike): Promise<SolveOutcome>;
+    verdict(challenge: ChallengeLike, verdict: SolveVerdict): Promise<void>;
+    close(): Promise<void>;
+    /** Reads a captcha image (PNG bytes) and judges the read, without a page. */
+    read(png: Buffer): Promise<ImageRead>;
+}
+//# sourceMappingURL=reader.contract.d.ts.map

package/dist/src/reader/tesseract-engine.client.d.ts

@@ -0,0 +1,32 @@
+import type { ReadSymbol } from './reader.contract.js';
+export interface EngineOptions {
+    lang: string;
+    langPath?: string;
+    whitelist: string;
+    pageSegMode: 7 | 8 | 13;
+}
+/**
+ * One Tesseract worker (a thread running Tesseract in WebAssembly), started
+ * on the first read and reused: reads queue on it. The language data comes
+ * from disk, never from a CDN, and nothing is cached in the working
+ * directory.
+ */
+export declare class TesseractEngine {
+    private readonly options;
+    private worker;
+    constructor(options: EngineOptions);
+    private start;
+    /**
+     * Reads one image.
+     *
+     * @param png - The cleaned image.
+     * @returns The text and each character with its confidence.
+     */
+    read(png: Buffer): Promise<{
+        raw: string;
+        symbols: ReadSymbol[];
+    }>;
+    /** Stops the worker thread, if it started. */
+    close(): Promise<void>;
+}
+//# sourceMappingURL=tesseract-engine.client.d.ts.map

package/dist/src/reader/tesseract-reader.use-case.d.ts

@@ -0,0 +1,14 @@
+import type { TesseractReader, TesseractReaderOptions } from './reader.contract.js';
+/**
+ * A captcha reader for OpenCraw's form captchas: it reads the code in the
+ * challenge's image with Tesseract, refreshes the image while the read is not
+ * good enough (wrong length, an unsure character, and no form is posted for
+ * that), fills the challenge's answer field, and returns `solved`. The engine
+ * posts the form, checks the page and tells the verdict, which goes to
+ * `audit` with the read.
+ *
+ * @param options - The charset, the length, the checks, the cleanup, the audit.
+ * @returns The reader, to register as a captcha solver.
+ */
+export declare function tesseractReader(options?: TesseractReaderOptions): TesseractReader;
+//# sourceMappingURL=tesseract-reader.use-case.d.ts.map

package/package.json

@@ -0,0 +1,74 @@
+{
+  "name": "@opencraw/captcha-tesseract",
+  "version": "0.0.2",
+  "type": "module",
+  "main": "./dist/index.esm.js",
+  "module": "./dist/index.esm.js",
+  "types": "./dist/src/index.d.ts",
+  "exports": {
+    "./package.json": "./package.json",
+    ".": {
+      "types": "./dist/src/index.d.ts",
+      "import": "./dist/index.esm.js",
+      "default": "./dist/index.esm.js"
+    }
+  },
+  "files": [
+    "dist",
+    "!**/*.tsbuildinfo",
+    "!**/*.d.ts.map",
+    "!**/*.js.map"
+  ],
+  "dependencies": {
+    "@tesseract.js-data/eng": "^1.0.0",
+    "pngjs": "^7.0.0",
+    "tesseract.js": "^7.0.0"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "description": "An image-captcha reader for OpenCraw's form captchas: Tesseract (WebAssembly, bundled English model, nothing fetched at run time) reads the code, a read of the wrong length or with an unsure character is refreshed instead of submitted, and every read can be audited with the page's verdict.",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/russoedu/open.craw.git",
+    "directory": "packages/captcha-tesseract"
+  },
+  "keywords": [
+    "captcha",
+    "ocr",
+    "tesseract",
+    "crawler",
+    "opencraw",
+    "image-captcha"
+  ],
+  "engines": {
+    "node": ">=20"
+  },
+  "sideEffects": false,
+  "nx": {
+    "targets": {
+      "e2e": {
+        "executor": "nx:run-commands",
+        "dependsOn": [
+          "build",
+          {
+            "projects": [
+              "@opencraw/core"
+            ],
+            "target": "build"
+          }
+        ],
+        "inputs": [
+          "default",
+          "^production",
+          "{workspaceRoot}/packages/core/e2e/**"
+        ],
+        "options": {
+          "command": "tsc -p tsconfig.e2e.json && jest --config jest.e2e.config.cts",
+          "cwd": "packages/captcha-tesseract"
+        }
+      }
+    }
+  }
+}