@scanmate/diff 0.0.2 → 0.1.0

package/README.md

@@ -1,183 +1,83 @@
-![scanmate diff](./scanmate-diff.svg)
+![scanmate diff](./assets/scanmate-diff.svg)
 
 # `@scanmate/diff`
 
-> Visual change detection, form field verification, and unexpected modification analysis for aligned document pairs.
+What changed between an original and its aligned scan: which expected regions were filled in, what ink was added where nothing was expected, what printed ink the scan lost — and a picture of all three.
 
-`@scanmate/diff` analyzes differences between original digital document templates and aligned scanned pages. It verifies whether expected form regions (such as signature blocks, checkboxes, and fillable fields) were completed, measures added/removed ink quantities, isolates unexpected handwritten marks or edits using 2-pass connected components analysis, and renders color-coded visual difference overlays.
+![the original and the returned scan side by side, the signer fields outlined](./assets/side-by-side.jpg)
 
----
+*Green: a field that was filled in. Orange: the band around it where ink still counts as that field's. Made from the [IRS Form W-9](https://www.irs.gov/pub/irs-pdf/fw9.pdf) (a work of the United States government, in the public domain): filled in as a generator would, printed, signed by hand and scanned crooked.*
 
-## Features
-
-- ✍️ **Form Region Verification (`compareRegions`)**: Quantifies added ink inside specific bounding boxes in original canvas coordinates.
-- 🔍 **Sub-Pixel Dilation Tolerance**: Fattens original ink boundaries before subtraction to eliminate false-positive edge noise caused by minor printing/scanning shifts.
-- 🎨 **Color-Coded Visual Overlay (`renderDiff`)**: Produces RGBA difference overlays (Red = added ink / signature, Blue = removed ink, Grey = matching ink).
-- 🧩 **Unexpected Mark Isolation (`diffPage`)**: Uses 8-connectivity Connected Component Analysis (CCL) to group un-matched ink pixels into isolated bounding boxes.
-- 📦 **Automated Box Merging**: Consolidates adjacent connected components to present clean, readable change boxes around handwritten notes or stamps.
-
----
-
-## Installation
+## Install
 
 ```bash
-# Using npm
-npm install @scanmate/diff @scanmate/ink
-
-# Using pnpm
-pnpm add @scanmate/diff @scanmate/ink
-
-# Using yarn
-yarn add @scanmate/diff @scanmate/ink
+npm install @scanmate/diff @scanmate/extract @scanmate/align
 ```
 
----
-
-## Quick Start
-
 ```ts
-import { compareRegions } from '@scanmate/diff'
-import { decodeImage } from '@scanmate/ink'
-
-const original = await decodeImage(originalBuffer)
-const aligned = await decodeImage(alignedBuffer)
-
-const reports = compareRegions(original, aligned, [
-  { id: 'signature', rect: { x: 100, y: 750, width: 350, height: 80 } },
-])
+import { alignPages } from '@scanmate/align'
+import { diffPages } from '@scanmate/diff'
+import { extractPair } from '@scanmate/extract'
+
+const { pages } = await extractPair({ original: 'fw9-issued.pdf', scanned: 'fw9-returned.pdf' })
+// The W-9's signature row, measured off the form in points from the page's top-left.
+const changes = await diffPages(await alignPages(pages), [
+  { page: 1, id: 'signature', x: 120, y: 577, width: 262, height: 22 },
+  { page: 1, id: 'date',      x: 404, y: 577, width: 171, height: 22 },
+], { sideBySide: true })
+
+changes[0].expected          // [{ id, identified, addedInk, overfilled, ink: { ... } }]
+changes[0].unexpected        // [{ x, y, width, height, inkArea, pixels }]
+changes[0].missing           // printed ink the scan lost
+changes[0].sideBySideImage   // original and scan, boxed alike
+changes[0].diffImage         // the overlay: violet where the ink differs, grey where it agrees
 ```
 
----
-
-## Architecture & Algorithm Deep-Dive
-
-### 1. Dilation Masking & Sub-Pixel Tolerance
-
-Even when a scan is perfectly aligned, real-world printing and scanning artifacts (ink bleed, scanner MTF blur, rasterization anti-aliasing) create sub-pixel outline differences along text character edges. Subtracting raw ink maps directly produces false-positive "halos" around every letter on the page.
-
-To prevent this, `@scanmate/diff` applies **morphological dilation** with radius $r$ (default 2px) to the original template's ink map $M_{orig}$:
-
-$$M_{orig, dilated} = \text{dilate}(M_{orig}, r)$$
-
-$$\text{Ink}_{added}(x, y) = \max\left(0, \text{Ink}_{scan}(x, y) - M_{orig, dilated}(x, y)\right)$$
-
-```mermaid
-flowchart TD
-    A["Original Ink Map"] --> B["Morphological Dilation (Radius r = 2px)"]
-    B --> C["Dilated Original Mask M_dilated"]
-    D["Aligned Scan Ink Map M_scan"] --> E["Ink Subtraction:<br/>Added = max(0, M_scan - M_dilated)"]
-    C --> E
-    E --> F["Clean Added Ink Map<br/>(Character outline noise suppressed,<br/>Signatures & Checkmarks retained)"]
-```
-
----
-
-### 2. Connected Component Analysis & Unexpected Mark Grouping
-
-```mermaid
-sequenceDiagram
-    autonumber
-    participant Diff as diffPage()
-    participant Mask as Mask Engine
-    participant CCL as Connected Components
-    participant Merge as Box Merger
-
-    Diff->>Mask: Compute Added Ink Map & Mask expected regions
-    Mask-->>Diff: Un-matched Added Ink Map
-    Diff->>CCL: connectedComponents(binaryInkMap)
-    CCL->>CCL: Pass 1: Label 8-connected pixel clusters & track equivalences
-    CCL->>CCL: Pass 2: Resolve label equivalences & calculate component stats
-    CCL-->>Diff: Return raw pixel blob Components
-    Diff->>Merge: mergeBoxes(components, { maxGap: 15px })
-    Merge->>Merge: Calculate bounding box overlaps & expand by maxGap
-    Merge->>Merge: Merge intersecting bounding boxes into unified regions
-    Merge-->>Diff: Return MergedBox array
-    Diff-->>Diff: Annotate overlay & produce PageDiff report
-```
-
----
-
-## Comprehensive API Reference
-
-### 1. Region Comparison & Form Verification
-
-#### `compareRegions(original: Raster, aligned: Raster, regions: Region[], options?: RegionOptions): RegionReport[]`
-Evaluates specific rectangular form regions to check if signatures, checkboxes, or text boxes were filled in.
-- **Parameters**:
-  - `original`: Original template `Raster`.
-  - `aligned`: Aligned scan `Raster` (must match `original` canvas width/height).
-  - `regions`: Array of `Region` objects (`{ id: string, rect: Rect, threshold?: number }`).
-  - `options` *(optional)*: `RegionOptions` object (see breakdown below).
-- **Returns**: Array of `RegionReport` (`{ id, rect, filled, score, added, removed, addedPixels, totalPixels }`).
-
-##### Detailed Options Explanation (`RegionOptions`):
-
-| Option | Type | Default | Description & Impact |
-|---|---|---|---|
-| `tolerance` | `number` | `2` | Morphological dilation radius in pixels applied to original ink before subtraction. Absorbs minor sub-pixel rendering shifts. |
-| `threshold` | `number` | `0.02` | Ink ratio threshold (2% of region area) above which `filled` is set to `true`. |
-| `addedColor` | `Rgba` | `[239, 68, 68, 255]` | RGBA color (Red) for added ink in diff overlays. |
-| `removedColor` | `Rgba` | `[59, 130, 246, 255]` | RGBA color (Blue) for removed ink in diff overlays. |
-| `matchedColor` | `Rgba` | `[156, 163, 175, 255]`| RGBA color (Grey) for matching ink in diff overlays. |
-
----
-
-#### `diffDocument(original: Raster, aligned: Raster, regions?: Region[], options?: RegionOptions): DocumentDiff`
-Computes whole-page added/removed ink statistics plus per-region details in a single efficient pass.
-- **Returns**: `DocumentDiff` (`{ overallAdded, overallRemoved, overallAddedPixels, overallTotalPixels, regions: RegionReport[] }`).
-
-#### `renderDiff(original: Raster, aligned: Raster, options?: RegionOptions): Raster`
-Generates a 4-color RGBA overlay `Raster` suitable for visual inspection (Red = scan additions, Blue = template deletions, Grey = matched ink, White = paper background).
-
----
-
-### 2. High-Level Page & Document Diffing
-
-#### `diffPage(options: DiffOptions): Promise<PageDiff>`
-Full change detection pipeline for a single page, matching expected form regions and isolating unexpected handwritten edits using connected component analysis.
-- **Parameters (`DiffOptions`)**:
-  - `page`: Page number index.
-  - `original`: Original template `Raster`.
-  - `aligned`: Aligned scan `Raster`.
-  - `expectedRegions` *(optional)*: Array of expected form field bounding boxes.
-  - `minChangePixels` *(default: 20)*: Minimum area in pixels to consider a connected component a valid unexpected change box.
-  - `tolerance` *(default: 2)*: Dilation tolerance radius.
-  - `addedColor` / `removedColor`: Visual overlay colors.
-- **Returns**: `Promise<PageDiff>` (`{ page, expected: ExpectedResult[], unexpected: UnexpectedChange[], overlay: Raster }`).
+Rectangles are in PDF points from the page's top-left by default, the same frame `@scanmate/extract` reports text in; `units: 'pixels'` switches to the original's rendered pixels.
 
-#### `diffPages(alignedPages: AlignedPage[], expectedRegions: ExpectedRegion[], options?: DiffOptions): Promise<PageDiff[]>`
-Batch page diffing for multi-page document collections.
+## What it measures
 
----
+- **Ink, not brightness.** Both pages are divided by their own local background before anything is compared, so a shadow or a grey scanner lid takes no part.
+- **In square millimetres, not shares of a box.** A signature is a few tens of mm² whether its box is a stamp or the width of the page, and the same mark is four times the pixels at twice the resolution.
+- **Two thresholds on the scan.** A normal one for what counts as *added*, and a faint one at a quarter of it for what is *still there at all*. Printed ink counts as lost only where the scan shows nothing even at the lower bar: a pale photocopy has lighter ink, not missing ink.
+- **Tolerance for the alignment.** The original's ink is fattened by `tolerance` (2 px) before subtraction, so stroke edges do not become a halo of confetti. A change that stays inside that band cannot be seen here — a digit swapped for another digit is exactly such a change, which is why `@scanmate/ocr` matches figures glyph by glyph.
 
-### 3. Pipeline Building Blocks & Connected Components
+## Deciding a region
 
-#### `buildMasks(original: Raster, aligned: Raster, options?: RegionOptions): Masks`
-Computes intermediate Float32 ink maps and binary addition/subtraction masks.
-- **Returns**: `Masks` (`{ originalInk, alignedInk, addedMask, removedMask, width, height }`).
+A region is **identified** when it has at least `minFillArea` (2 mm²) of new ink and is not **overfilled** — covered or struck through, which `maxFill` (0.5) draws the line on. Form rules showing through a slight misregistration are discounted: a component spanning 90% of the region and no thicker than 0.6 mm is the box's own printed line.
 
-#### `measureRegion(masks: Masks, region: Region, options?: RegionOptions): RegionReport`
-Measures ink statistics inside a single `Region` using pre-computed `Masks`.
+People sign past the box they are given, so each region also claims the ink within `expectedMargin` (6 points) of it, and the regions claim it **together**, so one stroke running through two fields is not left over as an unexpected mark. What a region reports is still the rectangle it was given; the band is drawn in pink.
 
-#### `paintOverlay(masks: Masks, options?: RegionOptions): Raster`
-Paints RGBA overlay `Raster` from pre-computed `Masks`.
+Each region reports its shape too — how many separate changes, the largest, the bounds as a share of the box, how much ink touches the border — so a signature can be told from a stray line without looking at the picture.
 
-#### `connectedComponents(binary: BinaryImage, options?: ComponentOptions): Component[]`
-Executes 2-pass 8-connectivity Connected Component Analysis (CCL) to extract disjoint pixel blobs.
-- **Options**:
-  - `minPixels` *(default: 1)*: Ignore components with pixel count below this limit.
-- **Returns**: Array of `Component` (`{ id, minX, minY, maxX, maxY, pixelCount, width, height }`).
+## Options
 
-#### `mergeBoxes(boxes: MergedBox[], options?: MergeOptions): MergedBox[]`
-Consolidates overlapping or closely adjacent bounding boxes.
-- **Options**:
-  - `maxGap` *(default: 15)*: Maximum distance in pixels between box boundaries to trigger a box merge.
+| option | default | |
+|---|---|---|
+| `units` | `'points'` | Or `'pixels'`. |
+| `tolerance` | `2` px | How much misregistration is forgiven. |
+| `faintInk` | `0.25` | Fraction of the normal threshold for "still there". |
+| `minFillArea` | `2` mm² | New ink a region needs. |
+| `maxFill` | `0.5` | Above this the region is covered, not filled. |
+| `expectedMargin` | `6` pt | How far outside a region its ink may lie. |
+| `formLineSpan` | `0.9` | Span that makes a component a printed rule... |
+| `formLineThickness` | `0.6` mm | ...if it is no thicker than this. |
+| `minChangeArea` | `1` mm² | Smallest change reported. |
+| `minMissingArea` | `4` mm² | Smallest loss reported. |
+| `mergeGap` | `3` mm | Boxes closer than this become one. |
+| `regionOverlap` | `0.5` | Share of a change's ink that must fall inside a region. |
+| `maxChanges` | `50` | Cap on a confetti page; the report says it was capped. |
+| `assumeDpi` | `150` | Used when the page does not say. |
+| `output` | `'png'` | `'none'` keeps rasters only. |
+| `annotate` | `false` | Draw the report onto the overlay. |
+| `sideBySide` | `false` | Also compose the two pages side by side. |
+| `probes` | none | Rectangles to measure the ink at, changed or not - added, lost and shared, in mm². |
+| `keepMasks` | `false` | Keep the ink masks on the result, so `probeInk` can ask about places found later. |
 
-#### `annotateOverlay(overlay: Raster, annotations: Annotation[], options?: LabelOptions): Raster`
-Draws bounding box rectangles and text labels onto a diff overlay `Raster`.
+## Building blocks
 
----
+`buildMasks`, `connectedComponents`, `labelComponents`, `mergeBoxes`, `measureRegionInk`, `annotateOverlay`, `composeSideBySide`, `compareRegions`, `diffDocument` and `renderDiff` are exported for callers who want a stage on its own, along with the annotation colours.
 
-## License
+## How it decides
 
-MIT © [ScanMate Team](https://github.com/russoedu/scanmate)
+[`documentation/algorithms.md`](./documentation/algorithms.md) has the algorithms in full: what each step measures, the decision flows, every constant with the measurement behind it, and what the package deliberately does not do.

package/dist/index.esm.js

@@ -1,9 +1,18 @@
-import { decodeImage, binarize, inkMap, toGrayscale, otsuThreshold, dilate, coverage, createRaster, encodeImage } from '@scanmate/ink';
+import { createRaster, decodeImage, binarize, inkMap, toGrayscale, otsuThreshold, dilate, coverage, encodeImage } from '@scanmate/ink';
 
-const IDENTIFIED = [30, 160, 70, 255];
-const NOT_IDENTIFIED = [230, 150, 20, 255];
-const UNEXPECTED = [200, 30, 190, 255];
-const MISSING = [20, 90, 230, 255];
+/** The area in question, drawn on the original: a statement of where, not of what. */
+const REFERENCE = [0, 23, 252, 255];
+const IDENTIFIED = [0, 252, 17, 255];
+/** Empty, covered, or changed: the answers that need a person. */
+const NOT_IDENTIFIED = [252, 0, 39, 255];
+/**
+ * Neither comparison could settle it. Not red, because red is a statement that
+ * something is wrong and this is a statement that nobody knows.
+ */
+const UNSETTLED = [200, 160, 0, 255];
+const EXPECTED_MARGIN = [245, 0, 252, 255];
+const UNEXPECTED = [255, 138, 0, 255];
+const MISSING = [0, 200, 252, 255];
 function annotateOverlay(overlay, annotations, thickness = 2) {
   for (const {
     rect,
@@ -55,6 +64,70 @@ function fill(raster, rect, color) {
   }
 }
 
+/**
+ * The original and the aligned scan next to each other, boxed.
+ *
+ * The overlay answers "which pixels changed"; this answers "show me", for the
+ * person who has to agree with the verdict. Because the scan is aligned onto
+ * the original's canvas, a box drawn at the same place on each half surrounds
+ * the same part of the page in both, so the eye goes straight from the empty
+ * field on the left to the signature on the right.
+ *
+ * The two halves can say different things, and usually should: the original's
+ * half states *where* the question is, in one colour, and the scan's half
+ * states the answer, in the colour of that answer.
+ */
+/** Separator between the halves: mid grey, so it shows against white paper and a grey scan alike. */
+const GUTTER = [150, 150, 150, 255];
+/**
+ * Several images in a row on one canvas, each with its own boxes, separated by
+ * a grey gutter - the original, the scan, and the overlay of the two, say.
+ *
+ * @param panels - The images, left to right.
+ * @param thickness - Outline thickness, in pixels.
+ * @param gutter - Pixels between panels.
+ * @returns One image, as wide as the panels and their gutters.
+ */
+function composePanels(panels, thickness, gutter) {
+  const width = panels.reduce((sum, panel) => sum + panel.raster.width, 0) + gutter * Math.max(0, panels.length - 1);
+  const height = Math.max(...panels.map(panel => panel.raster.height));
+  const result = createRaster(width, height);
+  let left = 0;
+  for (const [index, panel] of panels.entries()) {
+    paste(result, panel.raster, left);
+    annotateOverlay(result, (panel.annotations ?? []).map(a => ({
+      ...a,
+      rect: {
+        ...a.rect,
+        x: a.rect.x + left
+      }
+    })), thickness);
+    left += panel.raster.width;
+    if (index < panels.length - 1) {
+      for (let y = 0; y < height; y++) for (let x = left; x < left + gutter; x++) result.data.set(GUTTER, (y * result.width + x) * 4);
+      left += gutter;
+    }
+  }
+  return result;
+}
+function composeSideBySide(original, aligned, annotations, thickness, gutter) {
+  const sides = Array.isArray(annotations) ? {
+    original: annotations,
+    scanned: annotations
+  } : annotations;
+  return composePanels([{
+    raster: original,
+    annotations: sides.original
+  }, {
+    raster: aligned,
+    annotations: sides.scanned
+  }], thickness, gutter);
+}
+function paste(target, source, left) {
+  const rowBytes = source.width * 4;
+  for (let y = 0; y < source.height; y++) target.data.set(source.data.subarray(y * rowBytes, (y + 1) * rowBytes), (y * target.width + left) * 4);
+}
+
 function connectedComponents(mask, options = {}) {
   return labelComponents(mask, options).components;
 }
@@ -161,6 +234,73 @@ function labelComponents(mask, options = {}) {
   };
 }
 
+/**
+ * What the ink does inside places the caller names, changed or not.
+ *
+ * The rest of this subfeature starts from a change and asks where it is. This
+ * asks the opposite question - *how much ink is here?* - and it exists to
+ * settle an argument. A reading that disagrees with the original is not by
+ * itself a change: OCR misreads small, faint and sideways print, and identical
+ * ink under a word means identical characters whatever they were read as.
+ *
+ * It is separate from `diffPage` because the places worth asking about are
+ * usually not known until the page has been read, which happens alongside the
+ * pixel comparison rather than before it.
+ *
+ * @param masks - The ink masks of the pair, from `diffPage` or `buildMasks`.
+ * @param rects - Where to measure, in `units`.
+ * @param options - How to read `rects`, and the resolution they are in.
+ * @returns One probe per rectangle, in the same order, in square millimetres.
+ */
+function probeInk(masks, rects, options = {}) {
+  const {
+    dpi = 150,
+    units = 'points'
+  } = options;
+  const toPixels = units === 'points' ? dpi / 72 : 1;
+  const pixelsPerMm = dpi / 25.4;
+  const mm2PerPixel = 1 / (pixelsPerMm * pixelsPerMm);
+  return rects.map(rect => {
+    const ink = inkWithin({
+      x: rect.x * toPixels,
+      y: rect.y * toPixels,
+      width: rect.width * toPixels,
+      height: rect.height * toPixels
+    }, masks);
+    return {
+      rect,
+      addedInk: ink.added * mm2PerPixel,
+      lostInk: ink.lost * mm2PerPixel,
+      sharedInk: ink.shared * mm2PerPixel
+    };
+  });
+}
+/** Added, lost and shared ink inside one rectangle of the page, in pixels. */
+function inkWithin(rect, masks) {
+  const left = Math.max(0, Math.floor(rect.x));
+  const top = Math.max(0, Math.floor(rect.y));
+  const right = Math.min(masks.width, Math.ceil(rect.x + rect.width));
+  const bottom = Math.min(masks.height, Math.ceil(rect.y + rect.height));
+  let added = 0;
+  let lost = 0;
+  let shared = 0;
+  for (let y = top; y < bottom; y++) {
+    const row = y * masks.width;
+    for (let x = left; x < right; x++) {
+      const scan = masks.scan.data[row + x] === 1;
+      const print = masks.original.data[row + x] === 1;
+      if (scan && masks.originalDilated.data[row + x] === 0) added++;
+      if (print && masks.scanDilated.data[row + x] === 0) lost++;
+      if (scan && print) shared++;
+    }
+  }
+  return {
+    added,
+    lost,
+    shared
+  };
+}
+
 /** Default for {@link buildMasks}' `faintInk`, measured against real scans. */
 const FAINT_INK = 0.25;
 async function buildMasks(original, aligned, ink, tolerance, faintInk = FAINT_INK) {
@@ -304,10 +444,15 @@ function measureRegion(region, masks, defaultThreshold) {
 /**
  * An RGBA overlay of the comparison, for looking at with your own eyes.
  *
- * Red is ink the scan added, blue is ink it lost, grey is ink both agree on.
- * A correctly aligned pair of a signed form is almost entirely grey with a red
- * signature; a misaligned one is red and blue confetti along every stroke,
- * which is the fastest way to tell the two failures apart.
+ * Violet is ink the two pages do not share - added by the scan or lost from the
+ * print, which the picture does not distinguish; grey is ink they agree on. A
+ * correctly aligned pair of a signed form is almost entirely grey with a violet
+ * signature; a misaligned one is violet confetti along every stroke, which is
+ * the fastest way to tell the two failures apart.
+ *
+ * Which side the ink came from is a question for the report, where it is
+ * measured separately and in millimetres. Here it would be a second colour
+ * carrying a distinction the eye does not need at this zoom.
  */
 async function renderDiff(original, aligned, options = {}) {
   const {
@@ -318,7 +463,15 @@ async function renderDiff(original, aligned, options = {}) {
   const masks = await buildMasks(original, aligned, ink, tolerance, faintInk);
   return paintOverlay(masks);
 }
-/** The overlay, from masks already built. Red added, blue lost, grey agreed, white paper. */
+/**
+ * Ink the two pages do not share. Violet rather than red, because red means
+ * "changed" on every annotated page in the pipeline and one colour cannot mean
+ * two things.
+ */
+const OVERLAY_DIFFERENT = [127, 0, 252, 255];
+/** Ink the two pages agree on. */
+const OVERLAY_SHARED = [110, 110, 110, 255];
+/** The overlay, from masks already built. Violet where they differ, grey where they agree, white paper. */
 function paintOverlay(masks) {
   const {
     width,
@@ -332,19 +485,9 @@ function paintOverlay(masks) {
     let r = 255;
     let g = 255;
     let b = 255;
-    if (inScan && !nearOriginal) {
-      r = 220;
-      g = 30;
-      b = 40;
-    } else if (inOriginal && masks.scanDilated.data[p] === 0) {
-      r = 40;
-      g = 90;
-      b = 220;
-    } else if (inOriginal || inScan) {
-      r = 110;
-      g = 110;
-      b = 110;
-    }
+    const added = inScan && !nearOriginal;
+    const lost = inOriginal && masks.scanDilated.data[p] === 0;
+    if (added || lost) [r, g, b] = OVERLAY_DIFFERENT;else if (inOriginal || inScan) [r, g, b] = OVERLAY_SHARED;
     data[i] = r;
     data[i + 1] = g;
     data[i + 2] = b;
@@ -508,39 +651,6 @@ function measureRegionInk(added, rect, options) {
   };
 }
 
-/**
- * The original and the aligned scan next to each other, with the same boxes on
- * both: what was expected, what changed, what went missing.
- *
- * The overlay answers "which pixels changed"; this answers "show me", for the
- * person who has to agree with the verdict. Because the scan is aligned onto
- * the original's canvas, a box drawn at the same place on each half surrounds
- * the same part of the page in both, so the eye goes straight from the empty
- * field on the left to the signature on the right.
- */
-/** Separator between the halves: mid grey, so it shows against white paper and a grey scan alike. */
-const GUTTER = [150, 150, 150, 255];
-function composeSideBySide(original, aligned, annotations, thickness, gutter) {
-  const offset = original.width + gutter;
-  const result = createRaster(offset + aligned.width, Math.max(original.height, aligned.height));
-  paste(result, original, 0);
-  paste(result, aligned, offset);
-  for (let y = 0; y < result.height; y++) for (let x = original.width; x < offset; x++) result.data.set(GUTTER, (y * result.width + x) * 4);
-  annotateOverlay(result, annotations, thickness);
-  annotateOverlay(result, annotations.map(a => ({
-    ...a,
-    rect: {
-      ...a.rect,
-      x: a.rect.x + offset
-    }
-  })), thickness);
-  return result;
-}
-function paste(target, source, left) {
-  const rowBytes = source.width * 4;
-  for (let y = 0; y < source.height; y++) target.data.set(source.data.subarray(y * rowBytes, (y + 1) * rowBytes), (y * target.width + left) * 4);
-}
-
 /**
  * What changed on each page, and whether it was supposed to.
  *
@@ -572,7 +682,10 @@ async function diffPages(pages, expected = [], options = {}) {
       index,
       total: pages.length
     });
-    const result = await diffPage(page, expected.filter(e => e.page === page.page), options);
+    const result = await diffPage(page, expected.filter(e => e.page === page.page), {
+      ...options,
+      probes: (options.probes ?? []).filter(probe => probe.page === undefined || probe.page === page.page)
+    });
     results.push(result);
     onProgress?.({
       stage: 'diff',
@@ -604,7 +717,10 @@ async function diffPage(page, expected = [], options = {}) {
     mergeGap = 3,
     assumeDpi = 150,
     regionOverlap = 0.5,
+    expectedMargin = 6,
     maxChanges = 50,
+    probes = [],
+    keepMasks = false,
     output = 'png',
     annotate = false,
     sideBySide = false,
@@ -615,9 +731,12 @@ async function diffPage(page, expected = [], options = {}) {
   const pixelsPerMm = dpi / 25.4;
   const mm2PerPixel = 1 / (pixelsPerMm * pixelsPerMm);
   const masks = await buildMasks(page.original.raster, page.aligned.raster, ink, tolerance, faintInk);
+  // People sign past the box they are given, so each region claims the ink a little
+  // way outside it too; what it reports is still the region it was given.
   const regions = expected.map(e => ({
     id: e.id,
-    rect: scaleRect(e, toPixels)
+    rect: scaleRect(e, toPixels),
+    claim: scaleRect(grow(e, expectedMargin), toPixels)
   }));
   const findChanges = (mask, minArea) => {
     const components = connectedComponents(mask).filter(c => c.pixels >= 2);
@@ -626,10 +745,11 @@ async function diffPage(page, expected = [], options = {}) {
   };
   const addedMask = difference(masks.scan, masks.originalDilated);
   const added = findChanges(addedMask, minChangeArea);
-  const outside = added.filter(box => regions.every(r => inkShareInside(box, r.rect, masks) < regionOverlap));
+  // Taken together, since one stroke can run through two fields at once.
+  const outside = added.filter(box => inkShareInside(box, regions.map(r => r.claim), masks) < regionOverlap);
   const lost = findChanges(difference(masks.original, masks.scanDilated), minMissingArea);
   const expectedResults = regions.map((region, i) => {
-    const measured = measureRegionInk(addedMask, region.rect, {
+    const measured = measureRegionInk(addedMask, region.claim, {
       mergeGap: Math.round(mergeGap * pixelsPerMm),
       minChangePixels: minChangeArea / mm2PerPixel,
       lineSpan: formLineSpan,
@@ -666,6 +786,11 @@ async function diffPage(page, expected = [], options = {}) {
       }
     };
   });
+  // What the ink does where the caller asked, changed or not.
+  const measured = probeInk(masks, probes, {
+    dpi,
+    units
+  });
   const truncated = outside.length > maxChanges || lost.length > maxChanges;
   const toChange = box => ({
     x: box.x / toPixels,
@@ -677,20 +802,37 @@ async function diffPage(page, expected = [], options = {}) {
   });
   const unexpected = outside.slice(0, maxChanges).map(box => toChange(box));
   const missing = lost.slice(0, maxChanges).map(box => toChange(box));
-  const reported = [...regions.map((region, i) => ({
+  // The band first, so a region's own outline draws over it where they meet.
+  const margins = expectedMargin > 0 ? regions.map(region => ({
+    rect: region.claim,
+    color: EXPECTED_MARGIN
+  })) : [];
+  const verdicts = regions.map((region, i) => ({
     rect: grow(region.rect, 2),
     color: expectedResults[i].identified ? IDENTIFIED : NOT_IDENTIFIED
-  })), ...outside.slice(0, maxChanges).map(box => ({
+  }));
+  const reported = [...margins, ...verdicts, ...outside.slice(0, maxChanges).map(box => ({
     rect: grow(box, 4),
     color: UNEXPECTED
   }))];
+  // On the original, every region is simply the area in question; the answers belong to the scan.
+  const asAsked = regions.map(region => ({
+    rect: grow(region.rect, 2),
+    color: REFERENCE
+  }));
   const diffRaster = paintOverlay(masks);
   if (annotate) annotateOverlay(diffRaster, reported);
   // Lines about a point thick at any dpi, so the boxes read the same on every page.
-  const sideBySideRaster = sideBySide ? composeSideBySide(page.original.raster, page.aligned.raster, [...reported, ...lost.slice(0, maxChanges).map(box => ({
+  const losses = lost.slice(0, maxChanges).map(box => ({
     rect: grow(box, 4),
     color: MISSING
-  }))], Math.max(2, Math.round(dpi / 72)), Math.max(4, Math.round(dpi / 12))) : null;
+  }));
+  const sideBySideRaster = sideBySide ? composeSideBySide(page.original.raster, page.aligned.raster, {
+    // Left: where the questions are, and the ink the scan lost, which is the original's.
+    original: [...asAsked, ...losses],
+    // Right: the answers.
+    scanned: [...reported, ...losses]
+  }, Math.max(2, Math.round(dpi / 72)), Math.max(4, Math.round(dpi / 12))) : null;
   const whole = measureRegion({
     id: '__page__',
     rect: {
@@ -712,6 +854,8 @@ async function diffPage(page, expected = [], options = {}) {
       format: output
     }),
     expected: expectedResults,
+    probes: measured,
+    masks: keepMasks ? masks : null,
     unexpected,
     missing,
     truncated,
@@ -741,16 +885,24 @@ function difference(a, b) {
  * Measured on ink, not on box area: a signature that overflows its box by a
  * flourish is still mostly inside it, while its bounding box may not be.
  */
-function inkShareInside(box, region, masks) {
-  const left = Math.max(box.x, Math.floor(region.x));
-  const top = Math.max(box.y, Math.floor(region.y));
-  const right = Math.min(box.x + box.width, Math.ceil(region.x + region.width));
-  const bottom = Math.min(box.y + box.height, Math.ceil(region.y + region.height));
-  if (right <= left || bottom <= top) return 0;
+function inkShareInside(box, regions, masks) {
   let inside = 0;
-  for (let y = top; y < bottom; y++) {
-    const row = y * masks.width;
-    for (let x = left; x < right; x++) if (masks.scan.data[row + x] === 1 && masks.originalDilated.data[row + x] === 0) inside++;
+  const counted = new Set();
+  for (const region of regions) {
+    const left = Math.max(box.x, Math.floor(region.x));
+    const top = Math.max(box.y, Math.floor(region.y));
+    const right = Math.min(box.x + box.width, Math.ceil(region.x + region.width));
+    const bottom = Math.min(box.y + box.height, Math.ceil(region.y + region.height));
+    if (right <= left || bottom <= top) continue;
+    for (let y = top; y < bottom; y++) {
+      const row = y * masks.width;
+      for (let x = left; x < right; x++) {
+        // Regions may overlap once grown, and a pixel belongs to the box only once.
+        if (masks.scan.data[row + x] !== 1 || masks.originalDilated.data[row + x] !== 0 || counted.has(row + x)) continue;
+        counted.add(row + x);
+        inside++;
+      }
+    }
   }
   // The count here includes isolated pixels the component filter dropped from
   // box.pixels, so it can nudge past one.
@@ -779,5 +931,5 @@ function grow(rect, by) {
   };
 }
 
-export { IDENTIFIED, MISSING, NOT_IDENTIFIED, UNEXPECTED, annotateOverlay, buildMasks, compareRegions, composeSideBySide, connectedComponents, diffDocument, diffPage, diffPages, labelComponents, measureRegion, measureRegionInk, mergeBoxes, paintOverlay, renderDiff };
+export { EXPECTED_MARGIN, IDENTIFIED, MISSING, NOT_IDENTIFIED, OVERLAY_DIFFERENT, OVERLAY_SHARED, REFERENCE, UNEXPECTED, UNSETTLED, annotateOverlay, buildMasks, compareRegions, composePanels, composeSideBySide, connectedComponents, diffDocument, diffPage, diffPages, labelComponents, measureRegion, measureRegionInk, mergeBoxes, paintOverlay, probeInk, renderDiff };
 //# sourceMappingURL=index.esm.js.map

package/dist/src/change-detection/annotate-overlay.use-case.d.ts

@@ -1,14 +1,32 @@
-import type { Raster, Rect } from '@scanmate/ink';
+import type { Raster, Rect, Rgba } from '@scanmate/ink';
 /**
  * Outline what the report says onto the overlay, so the picture and the numbers
- * can be checked against each other at a glance: green for an expected region
- * that was filled in, amber for one that was not, magenta around every change
- * nobody expected, and blue - the overlay's colour for lost ink - around ink
- * that went missing.
+ * can be checked against each other at a glance.
+ *
+ * One colour per answer, and the answers are few:
+ *
+ * | colour | |
+ * |---|---|
+ * | blue `#0017FC` | the area being asked about, as the original has it |
+ * | green `#00FC11` | an expected region that was filled in |
+ * | red `#FC0027` | one left empty, covered, or content that changed |
+ * | pink `#F500FC` | the band around a region where ink still counts as its own |
+ * | orange `#FF8A00` | ink added where nothing was expected |
+ * | cyan `#00C8FC` | printed ink the scan lost |
+ * | olive `#C8A000` | a disagreement nothing could settle |
  */
-export type Rgba = readonly [number, number, number, number];
+export type { Rgba } from '@scanmate/ink';
+/** The area in question, drawn on the original: a statement of where, not of what. */
+export declare const REFERENCE: Rgba;
 export declare const IDENTIFIED: Rgba;
+/** Empty, covered, or changed: the answers that need a person. */
 export declare const NOT_IDENTIFIED: Rgba;
+/**
+ * Neither comparison could settle it. Not red, because red is a statement that
+ * something is wrong and this is a statement that nobody knows.
+ */
+export declare const UNSETTLED: Rgba;
+export declare const EXPECTED_MARGIN: Rgba;
 export declare const UNEXPECTED: Rgba;
 export declare const MISSING: Rgba;
 export interface Annotation {

package/dist/src/change-detection/index.d.ts

@@ -1,12 +1,15 @@
 /** What changed on a page and whether it was supposed to: changed pixels grouped into reportable regions. */
-export { annotateOverlay, IDENTIFIED, MISSING, NOT_IDENTIFIED, UNEXPECTED } from './annotate-overlay.use-case.js';
+export { annotateOverlay, EXPECTED_MARGIN, IDENTIFIED, MISSING, NOT_IDENTIFIED, REFERENCE, UNEXPECTED, UNSETTLED } from './annotate-overlay.use-case.js';
+export { composePanels } from './side-by-side.use-case.js';
+export type { Panel, SideAnnotations } from './side-by-side.use-case.js';
 export type { Annotation, Rgba } from './annotate-overlay.use-case.js';
 export { connectedComponents, labelComponents } from './connected-components.use-case.js';
+export { inkWithin, probeInk } from './probe-ink.use-case.js';
 export type { Component, LabelledComponents, LabelOptions } from './connected-components.use-case.js';
 export { diffPage, diffPages } from './diff-pages.use-case.js';
 export { mergeBoxes } from './merge-boxes.use-case.js';
 export type { MergedBox } from './merge-boxes.use-case.js';
-export type { Change, CoordinateUnits, DiffOptions, ExpectedChange, ExpectedResult, PageDiff, RegionInkMetrics } from './page-diff.contract.js';
+export type { Change, CoordinateUnits, DiffOptions, ExpectedChange, ExpectedResult, InkProbe, PageDiff, ProbeRect, RegionInkMetrics } from './page-diff.contract.js';
 export { measureRegionInk } from './region-ink.use-case.js';
 export type { RegionInk, RegionInkOptions } from './region-ink.use-case.js';
 export { composeSideBySide } from './side-by-side.use-case.js';

package/dist/src/change-detection/page-diff.contract.d.ts

@@ -1,4 +1,5 @@
 import type { ImageFormat, InkOptions, ProgressCallback, Raster, Rect } from '@scanmate/ink';
+import type { Masks } from '../region-comparison/index.js';
 /**
  * Coordinates for regions going in and changes coming out.
  *
@@ -22,6 +23,37 @@ export interface ExpectedChange {
 export interface DiffOptions {
     /** Units of `ExpectedChange` rectangles and of every rectangle reported back. Default `'points'`. */
     units?: CoordinateUnits;
+    /**
+     * Places to measure the ink at, whether or not anything changed there.
+     *
+     * A reading that disagrees with the original is not by itself a change: OCR
+     * misreads small print, and sideways print, and print on a shaded bar. Asking
+     * what the ink does at the very place the reading disagrees settles it - ink
+     * that is identical there means the characters are identical, whatever was
+     * read. `@scanmate/audit` passes every text difference through here.
+     */
+    probes?: readonly ProbeRect[];
+    /**
+     * Keep the ink masks on the result, so the caller can probe places it does
+     * not know about yet.
+     *
+     * The places worth probing are the ones the reading disputes, and the reading
+     * runs alongside this rather than before it - so `probes` cannot name them.
+     * Masks are four binary images the size of the page, about 9 MB for A4 at
+     * 150 dpi, so a caller is expected to drop them as soon as it has asked.
+     */
+    keepMasks?: boolean;
+    /**
+     * How far outside an expected region its ink may still lie, in `units`. Default `6`
+     * (2 mm at 72 points to the inch).
+     *
+     * People sign past the box they are given - a descender below the rule, a flourish
+     * out to the side - and that is the signature, not a mark someone made elsewhere. The
+     * region claims the ink within this band and measures it, while still reporting the
+     * rectangle it was given. Ink inside the band of any expected region counts towards
+     * them all, so one stroke crossing two fields is not left over as unexpected.
+     */
+    expectedMargin?: number;
     /** Pixels the original's ink is fattened by before diffing, to absorb sub-pixel misalignment. Default `2`. */
     tolerance?: number;
     /**
@@ -163,6 +195,21 @@ export interface RegionInkMetrics {
     formLines: number;
 }
 /** A change found where nothing was expected, or ink that went missing. */
+/** A place to measure the ink at; `page` selects the page when several are compared. */
+export type ProbeRect = Rect & {
+    page?: number;
+};
+/** What the ink does inside one place that was asked about. */
+export interface InkProbe {
+    /** The place asked about, in `units`. */
+    rect: Rect;
+    /** New ink there, in square millimetres. */
+    addedInk: number;
+    /** Printed ink lost there, in square millimetres. */
+    lostInk: number;
+    /** Ink the two pages agree on there, in square millimetres. */
+    sharedInk: number;
+}
 export interface Change {
     /** Bounding box, in the requested units. */
     x: number;
@@ -176,13 +223,17 @@ export interface Change {
 }
 export interface PageDiff {
     page: number;
-    /** The overlay: red added, blue lost, grey agreed - annotated when asked. */
+    /** The overlay: violet where the ink differs, grey where it agrees - annotated when asked. */
     diffRaster: Raster;
     diffImage: Uint8Array | null;
     /** Original and aligned scan side by side with the report boxed on both, when `sideBySide` was asked for. */
     sideBySideRaster: Raster | null;
     sideBySideImage: Uint8Array | null;
     expected: ExpectedResult[];
+    /** The ink at each place `probes` asked about, in the same order. */
+    probes: InkProbe[];
+    /** The ink masks, when `keepMasks` asked for them: for `probeInk`, then dropped. */
+    masks: Masks | null;
     /** New ink outside every expected region, merged into one box per change. */
     unexpected: Change[];
     /**

package/dist/src/change-detection/probe-ink.use-case.d.ts

@@ -0,0 +1,32 @@
+import type { Rect } from '@scanmate/ink';
+import type { Masks } from '../region-comparison/index.js';
+import type { CoordinateUnits, InkProbe } from './page-diff.contract.js';
+/**
+ * What the ink does inside places the caller names, changed or not.
+ *
+ * The rest of this subfeature starts from a change and asks where it is. This
+ * asks the opposite question - *how much ink is here?* - and it exists to
+ * settle an argument. A reading that disagrees with the original is not by
+ * itself a change: OCR misreads small, faint and sideways print, and identical
+ * ink under a word means identical characters whatever they were read as.
+ *
+ * It is separate from `diffPage` because the places worth asking about are
+ * usually not known until the page has been read, which happens alongside the
+ * pixel comparison rather than before it.
+ *
+ * @param masks - The ink masks of the pair, from `diffPage` or `buildMasks`.
+ * @param rects - Where to measure, in `units`.
+ * @param options - How to read `rects`, and the resolution they are in.
+ * @returns One probe per rectangle, in the same order, in square millimetres.
+ */
+export declare function probeInk(masks: Masks, rects: readonly Rect[], options?: {
+    dpi?: number;
+    units?: CoordinateUnits;
+}): InkProbe[];
+/** Added, lost and shared ink inside one rectangle of the page, in pixels. */
+export declare function inkWithin(rect: Rect, masks: Masks): {
+    added: number;
+    lost: number;
+    shared: number;
+};
+//# sourceMappingURL=probe-ink.use-case.d.ts.map

package/dist/src/change-detection/side-by-side.use-case.d.ts

@@ -1,4 +1,24 @@
 import type { Raster } from '@scanmate/ink';
 import type { Annotation } from './annotate-overlay.use-case.js';
-export declare function composeSideBySide(original: Raster, aligned: Raster, annotations: readonly Annotation[], thickness: number, gutter: number): Raster;
+/** What to draw on each half, when the two differ. An array draws the same on both. */
+export interface SideAnnotations {
+    original?: readonly Annotation[];
+    scanned?: readonly Annotation[];
+}
+/** One panel of a composition: an image, and what to draw on it. */
+export interface Panel {
+    raster: Raster;
+    annotations?: readonly Annotation[];
+}
+/**
+ * Several images in a row on one canvas, each with its own boxes, separated by
+ * a grey gutter - the original, the scan, and the overlay of the two, say.
+ *
+ * @param panels - The images, left to right.
+ * @param thickness - Outline thickness, in pixels.
+ * @param gutter - Pixels between panels.
+ * @returns One image, as wide as the panels and their gutters.
+ */
+export declare function composePanels(panels: readonly Panel[], thickness: number, gutter: number): Raster;
+export declare function composeSideBySide(original: Raster, aligned: Raster, annotations: readonly Annotation[] | SideAnnotations, thickness: number, gutter: number): Raster;
 //# sourceMappingURL=side-by-side.use-case.d.ts.map

package/dist/src/index.d.ts

@@ -15,11 +15,11 @@
  * different part of the page in each image.
  */
 export { diffPage, diffPages } from './change-detection/index.js';
-export type { Change, CoordinateUnits, DiffOptions, ExpectedChange, ExpectedResult, PageDiff, RegionInkMetrics } from './change-detection/index.js';
+export type { Change, CoordinateUnits, DiffOptions, ExpectedChange, ExpectedResult, InkProbe, PageDiff, ProbeRect, RegionInkMetrics } from './change-detection/index.js';
 export { compareRegions, diffDocument, renderDiff } from './region-comparison/index.js';
 export type { DocumentDiff, Region, RegionOptions, RegionReport } from './region-comparison/index.js';
-export { annotateOverlay, composeSideBySide, connectedComponents, IDENTIFIED, labelComponents, measureRegionInk, mergeBoxes, MISSING, NOT_IDENTIFIED, UNEXPECTED } from './change-detection/index.js';
-export type { Annotation, Component, LabelledComponents, LabelOptions, MergedBox, RegionInk, RegionInkOptions, Rgba } from './change-detection/index.js';
-export { buildMasks, measureRegion, paintOverlay } from './region-comparison/index.js';
+export { annotateOverlay, composePanels, composeSideBySide, connectedComponents, EXPECTED_MARGIN, IDENTIFIED, labelComponents, measureRegionInk, mergeBoxes, MISSING, NOT_IDENTIFIED, probeInk, REFERENCE, UNEXPECTED, UNSETTLED } from './change-detection/index.js';
+export type { Annotation, Component, Panel, LabelledComponents, LabelOptions, MergedBox, RegionInk, RegionInkOptions, Rgba } from './change-detection/index.js';
+export { buildMasks, measureRegion, OVERLAY_DIFFERENT, OVERLAY_SHARED, paintOverlay } from './region-comparison/index.js';
 export type { Masks } from './region-comparison/index.js';
 //# sourceMappingURL=index.d.ts.map

package/dist/src/region-comparison/index.d.ts

@@ -4,5 +4,5 @@ export { compareRegions, diffDocument, measureRegion } from './compare-regions.u
 export { buildMasks } from './ink-masks.use-case.js';
 export type { Masks } from './ink-masks.use-case.js';
 export type { DocumentDiff, Region, RegionOptions, RegionReport } from './region.model.js';
-export { paintOverlay, renderDiff } from './render-diff.use-case.js';
+export { OVERLAY_DIFFERENT, OVERLAY_SHARED, paintOverlay, renderDiff } from './render-diff.use-case.js';
 //# sourceMappingURL=index.d.ts.map

package/dist/src/region-comparison/render-diff.use-case.d.ts

@@ -1,15 +1,28 @@
-import type { ImageInput, Raster } from '@scanmate/ink';
+import type { ImageInput, Raster, Rgba } from '@scanmate/ink';
 import type { Masks } from './ink-masks.use-case.js';
 import type { RegionOptions } from './region.model.js';
 /**
  * An RGBA overlay of the comparison, for looking at with your own eyes.
  *
- * Red is ink the scan added, blue is ink it lost, grey is ink both agree on.
- * A correctly aligned pair of a signed form is almost entirely grey with a red
- * signature; a misaligned one is red and blue confetti along every stroke,
- * which is the fastest way to tell the two failures apart.
+ * Violet is ink the two pages do not share - added by the scan or lost from the
+ * print, which the picture does not distinguish; grey is ink they agree on. A
+ * correctly aligned pair of a signed form is almost entirely grey with a violet
+ * signature; a misaligned one is violet confetti along every stroke, which is
+ * the fastest way to tell the two failures apart.
+ *
+ * Which side the ink came from is a question for the report, where it is
+ * measured separately and in millimetres. Here it would be a second colour
+ * carrying a distinction the eye does not need at this zoom.
  */
 export declare function renderDiff(original: ImageInput, aligned: ImageInput, options?: RegionOptions): Promise<Raster>;
-/** The overlay, from masks already built. Red added, blue lost, grey agreed, white paper. */
+/**
+ * Ink the two pages do not share. Violet rather than red, because red means
+ * "changed" on every annotated page in the pipeline and one colour cannot mean
+ * two things.
+ */
+export declare const OVERLAY_DIFFERENT: Rgba;
+/** Ink the two pages agree on. */
+export declare const OVERLAY_SHARED: Rgba;
+/** The overlay, from masks already built. Violet where they differ, grey where they agree, white paper. */
 export declare function paintOverlay(masks: Masks): Raster;
 //# sourceMappingURL=render-diff.use-case.d.ts.map

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@scanmate/diff",
-  "version": "0.0.2",
+  "version": "0.1.0",
   "description": "What changed between an original and its aligned scan: expected regions filled in, unexpected marks, lost ink, and a side-by-side evidence image.",
   "license": "MIT",
   "author": "Eduardo Russo",
@@ -39,12 +39,12 @@
     "!**/*.js.map"
   ],
   "dependencies": {
-    "@scanmate/ink": "^0.0.2"
+    "@scanmate/ink": "^0.1.0"
   },
   "publishConfig": {
     "access": "public"
   },
   "devDependencies": {
-    "@scanmate/align": "^0.0.2"
+    "@scanmate/align": "^0.1.0"
   }
 }