hazo_scrape 1.4.0 → 1.6.0

package/CHANGE_LOG.md

@@ -1,5 +1,47 @@
 # hazo_scrape — Change Log
 
+## 1.6.0 — 2026-07-25
+
+### Added — `parseNumber` tokenizes a trailing bare cent marker
+- `parseNumber('45c')` → `45`, `parseNumber('14.5c')` → `14.5`, `parseNumber('45¢')` → `45`. This is the symbol/abbreviation form of the `cents` word already stripped (many AU/UK IR tables write `45c` rather than `45 cents`), so it is tokenized identically — the cents→dollars **scaling stays the caller's job** (via a domain `amount_unit`). Anchored to a digit immediately before the marker and to end-of-string, so it never touches a `c` inside a word (`recent`) or a mid-string letter. Surfaced by Woolworths (`45c`) and Harvey Norman (`14.5c`) dividend tables.
+
+### Added — `parseDate` optional 2-digit-year support (`opts.yearPivot`)
+- `parseDate('10 Mar 26', { yearPivot: 2000 })` → `2026-03-10`. 2-digit years remain **rejected by default** (returning `null`, true to the "never guess the century" contract); a caller that knows the source's convention opts in with `yearPivot`, the base of a sliding 100-year window (`2000` reads `26`→2026/`99`→2099; `1950` reads `26`→2026/`99`→1999). Applies to the named-month formats (`D Mon YY`, `Mon D, YY`); slash dates stay 4-digit-only (a 2-digit slash year compounds day/month AND century ambiguity). Surfaced by Coles, whose live table writes `10 Mar 26`.
+- `mapRows` gains `opts.dateYearPivot?: number`, forwarded verbatim to `parseDate` for every date-typed cell.
+
+### Added — `extractTable` `cellIgnoreSelectors` (strip in-cell noise)
+- `extractTable(html, { cellIgnoreSelectors: ['sup'] })` removes matching descendants from every cell (and header) **before** its text is read, so `100<sup>4</sup>` reads as `100`, not `1004`. Off by default (cells read verbatim). Applies to real-table, grid, and combine modes alike; the removal is done on a per-cell clone, never mutating the shared document. Surfaced by Wesfarmers, whose franking cells staple a footnote superscript onto the number.
+
+### Added — `extractTable` `combine` mode (header-only table + per-section sibling tables)
+- `extractTable(html, { combine: { headerSelector, dropLabelRows? } })` reconstructs one logical table from a header-only `<table>` followed by one `<table>` per section (e.g. per year), each led by a section-label row — a common Computershare-style IR layout. Headers come from `headerSelector`'s first row; data rows are gathered from that table plus its immediately-following sibling `<table>`s **that share its column count** (stopping at the first that doesn't, so an unrelated later table can't leak in). `dropLabelRows: true` drops bare section-divider rows (only the first cell non-empty). Takes precedence over `select`; `grid` still wins over it. Surfaced by JB Hi-Fi, whose dividend history is a header table plus one table per year.
+
+## 1.5.2 — 2026-07-24
+
+### Fixed — `extractTableFromPdf` mid-word text-run splits
+- A word a PDF splits across two adjacent text runs (pdfjs reports the second run beginning, within 1px, exactly where the first ended, with no whitespace at either boundary — e.g. `"Septemb"` + `"er"`) is now glued back with **no space** (`"September"`) instead of joined with a space (`"Septemb er"`). Genuinely separate tokens on the same line — which render a visible gap and/or carry a boundary space in the run text — still get their single separating space. Surfaced by BHP's historical dividend PDF, where a split month name (`"4 Septemb er 2020"`) made a record-date cell unparseable. `lineToCells` now decides spacing per-item from each text run's `x + width` and boundary whitespace rather than blindly space-joining.
+
+## 1.5.1 — 2026-07-24
+
+### Fixed — `extractTableFromPdf` multi-page row merging
+- Visual lines are now clustered **per page** instead of over the concatenation of all pages' text items. Text-item y-coordinates repeat page to page (row N sits at the same y on every page), so the previous cross-page clustering collapsed each page's row N into a single garbled line — a multi-page table (e.g. BHP's 6-page historical dividend list) came out unusable. Column-band detection still spans all items (column geometry is page-independent); page order is preserved; a single-page document is unaffected.
+
+## 1.5.0 — 2026-07-24
+
+### Added — `extractTableFromPdf` (PDF text-layer → table)
+- `extractTableFromPdf(data, opts?)` reconstructs a `{ headers, rows }` grid — the same `ExtractTableResult` shape `extractTable(html)` produces — from a PDF's positioned text layer, via `pdfjs-dist`. It clusters text items into visual lines by y-position and into columns by x-band, so a caller's existing `mapColumns`/`mapRows` pipeline consumes HTML and PDF tables identically.
+- `opts.columnBands?: number[]` — left x-edges of each column; omit to auto-detect by x-gap (a 25px gap starts a new column).
+- `opts.headerLabels?: string[]` — verbatim headers positionally aligned to the bands; when set, every clustered line is treated as a data row (for PDFs whose header spans multiple visual lines or uses columns no keyword classifier should see).
+- `opts.headerRows?: number` — else merge the first N lines into headers (default 1); ignored when `headerLabels` is set.
+- `opts.mergeContinuationRows?: boolean` — a line whose leftmost band is empty is folded into the previous row (per-band, space-joined) instead of becoming its own row — for PDFs that wrap a cell's overflow (e.g. a dual-currency amount) onto a second line.
+- `opts.rowTolerance?: number` (default 3) and `opts.maxPages?: number` round out the tuning surface.
+- New export surface: `src/pdf/` re-exported from the package root (`.`) — SERVER-ONLY (pulls in `pdfjs-dist`), never from the client-safe `./parse` entry.
+- New dependency: `pdfjs-dist@^4.10.38` (imported from `pdfjs-dist/legacy/build/pdf.mjs`; no worker/font setup needed for text-only extraction).
+
+### Added — `fetchPdf` + `FetchOptions.binary` (raw-byte fetch)
+- `fetchPdf(url, opts?)` fetches a PDF (or any binary document) as raw bytes, reusing `fetchDocument`'s robots / rate-limit / retry / proxy machinery. On a 2xx response, `result.bytes` (`Uint8Array`) holds the payload.
+- `FetchOptions` gains `binary?: boolean`; `FetchResult` gains `bytes?: Uint8Array`. When `binary: true`, `fetchDocument` reads the response as bytes instead of text (`body` is `''`) and the response is never written to the on-disk cache.
+- Motivation: `extractTableFromPdf` needs raw bytes, not decoded text, and a caller (e.g. `extractor_asx`) should not have to hand-roll a second fetch path to get them.
+
 ## 1.4.0 — 2026-07-24
 
 ### Added — `parseDate` optional `opts.extractLeading` (+ `mapRows` `opts.dateExtractLeading`)

package/dist/fetch/index.d.ts

@@ -7,4 +7,10 @@ export type { ScrapeTableOptions, TableResult } from './scrape.js';
 /** Clears all cached ProxyPools. For tests. */
 export declare function resetProxyPools(): void;
 export declare function fetchDocument(url: string, opts?: FetchOptions): Promise<FetchResult>;
+/**
+ * Fetch a PDF (or any binary document) as raw bytes, reusing fetchDocument's
+ * robots / rate-limit / retry / proxy machinery. On a 2xx, `result.bytes` holds
+ * the payload; binary responses are never cached.
+ */
+export declare function fetchPdf(url: string, opts?: FetchOptions): Promise<FetchResult>;
 //# sourceMappingURL=index.d.ts.map

package/dist/fetch/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/fetch/index.ts"],"names":[],"mappings":"AA0BA,OAAO,KAAK,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAE5D,OAAO,EAAE,qBAAqB,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AAClD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAC9E,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,YAAY,EAAE,kBAAkB,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAOnE,+CAA+C;AAC/C,wBAAgB,eAAe,IAAI,IAAI,CAEtC;AA0GD,wBAAsB,aAAa,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,GAAE,YAAiB,GAAG,OAAO,CAAC,WAAW,CAAC,CAiN9F"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/fetch/index.ts"],"names":[],"mappings":"AA0BA,OAAO,KAAK,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AAE5D,OAAO,EAAE,qBAAqB,EAAE,gBAAgB,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AAClD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAC9E,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,YAAY,EAAE,kBAAkB,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAOnE,+CAA+C;AAC/C,wBAAgB,eAAe,IAAI,IAAI,CAEtC;AA0GD,wBAAsB,aAAa,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,GAAE,YAAiB,GAAG,OAAO,CAAC,WAAW,CAAC,CAoN9F;AAED;;;;GAIG;AACH,wBAAsB,QAAQ,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,GAAE,YAAiB,GAAG,OAAO,CAAC,WAAW,CAAC,CAEzF"}

package/dist/fetch/index.js

@@ -251,14 +251,16 @@ export async function fetchDocument(url, opts = {}) {
             }
             // Terminal: 2xx/3xx, or a terminal 4xx not in retryOn. Returned as a
             // FetchResult (ok:false for 4xx) — never thrown.
-            const body = await res.text();
+            const isBinary = opts.binary === true;
+            const bytes = isBinary ? new Uint8Array(await res.arrayBuffer()) : undefined;
+            const body = isBinary ? '' : await res.text();
             const finalUrl = res.url || url;
             const responseHeaders = collectHeaders(res);
             const contentType = res.headers.get('content-type') ?? undefined;
             // [Phase 5] Only 2xx responses are cache-worthy — terminal 4xx/etc are
             // never stored. Written under the same pre-redirect `url`-derived key
             // used for the lookup in step 3.
-            if (res.ok && cfg.cache.enabled && cacheKey) {
+            if (res.ok && !isBinary && cfg.cache.enabled && cacheKey) {
                 await writeCache(cfg.cache, cacheKey, {
                     finalUrl,
                     status: res.status,
@@ -277,6 +279,7 @@ export async function fetchDocument(url, opts = {}) {
                 ok: res.ok,
                 headers: responseHeaders,
                 body,
+                bytes,
                 contentType,
                 fromCache: false,
                 attempts: attempt,
@@ -324,3 +327,11 @@ export async function fetchDocument(url, opts = {}) {
         ...(lastError !== undefined ? { cause: lastError } : {}),
     });
 }
+/**
+ * Fetch a PDF (or any binary document) as raw bytes, reusing fetchDocument's
+ * robots / rate-limit / retry / proxy machinery. On a 2xx, `result.bytes` holds
+ * the payload; binary responses are never cached.
+ */
+export async function fetchPdf(url, opts = {}) {
+    return fetchDocument(url, { ...opts, binary: true });
+}

package/dist/fetch/types.d.ts

@@ -37,6 +37,8 @@ export interface FetchOptions {
     };
     /** Test seam — passed through to safeFetch's `deps.fetchImpl`. */
     fetchImpl?: typeof fetch;
+    /** When true, read the response as raw bytes into `FetchResult.bytes` instead of text. Not cached. */
+    binary?: boolean;
 }
 export interface FetchResult {
     url: string;
@@ -51,5 +53,7 @@ export interface FetchResult {
     attempts: number;
     timingMs: number;
     usedProxy?: string;
+    /** Raw response bytes — populated only when the request was made with `binary: true`. */
+    bytes?: Uint8Array;
 }
 //# sourceMappingURL=types.d.ts.map

package/dist/fetch/types.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../../src/fetch/types.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;AAKvD,MAAM,MAAM,eAAe,GAAG,mBAAmB,CAAC;AAElD,MAAM,WAAW,YAAY;IAC3B,kFAAkF;IAClF,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,MAAM,CAAC,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC;IAC5B,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACjC,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,KAAK,CAAC,EAAE;QACN,WAAW,CAAC,EAAE,MAAM,CAAC;QACrB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,aAAa,CAAC,EAAE,MAAM,CAAC;QACvB,MAAM,CAAC,EAAE,OAAO,CAAC;QACjB,OAAO,CAAC,EAAE,MAAM,EAAE,CAAC;KACpB,CAAC;IACF,SAAS,CAAC,EAAE;QACV,aAAa,CAAC,EAAE,MAAM,CAAC;QACvB,OAAO,CAAC,EAAE,OAAO,CAAC;KACnB,CAAC;IACF,KAAK,CAAC,EAAE;QACN,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC;QAChB,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,QAAQ,CAAC,EAAE,aAAa,CAAC;QACzB,UAAU,CAAC,EAAE,MAAM,CAAC;KACrB,CAAC;IACF,KAAK,CAAC,EAAE;QACN,OAAO,CAAC,EAAE,OAAO,CAAC;QAClB,GAAG,CAAC,EAAE,MAAM,CAAC;QACb,UAAU,CAAC,EAAE,MAAM,CAAC;KACrB,CAAC;IACF,MAAM,CAAC,EAAE;QACP,OAAO,CAAC,EAAE,OAAO,CAAC;KACnB,CAAC;IACF,kEAAkE;IAClE,SAAS,CAAC,EAAE,OAAO,KAAK,CAAC;CAC1B;AAED,MAAM,WAAW,WAAW;IAC1B,GAAG,EAAE,MAAM,CAAC;IACZ,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,MAAM,CAAC;IACnB,EAAE,EAAE,OAAO,CAAC;IACZ,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAChC,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,SAAS,EAAE,OAAO,CAAC;IACnB,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB"}
+{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../../src/fetch/types.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;AAKvD,MAAM,MAAM,eAAe,GAAG,mBAAmB,CAAC;AAElD,MAAM,WAAW,YAAY;IAC3B,kFAAkF;IAClF,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,MAAM,CAAC,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC;IAC5B,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACjC,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,KAAK,CAAC,EAAE;QACN,WAAW,CAAC,EAAE,MAAM,CAAC;QACrB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,aAAa,CAAC,EAAE,MAAM,CAAC;QACvB,MAAM,CAAC,EAAE,OAAO,CAAC;QACjB,OAAO,CAAC,EAAE,MAAM,EAAE,CAAC;KACpB,CAAC;IACF,SAAS,CAAC,EAAE;QACV,aAAa,CAAC,EAAE,MAAM,CAAC;QACvB,OAAO,CAAC,EAAE,OAAO,CAAC;KACnB,CAAC;IACF,KAAK,CAAC,EAAE;QACN,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC;QAChB,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,QAAQ,CAAC,EAAE,MAAM,CAAC;QAClB,QAAQ,CAAC,EAAE,aAAa,CAAC;QACzB,UAAU,CAAC,EAAE,MAAM,CAAC;KACrB,CAAC;IACF,KAAK,CAAC,EAAE;QACN,OAAO,CAAC,EAAE,OAAO,CAAC;QAClB,GAAG,CAAC,EAAE,MAAM,CAAC;QACb,UAAU,CAAC,EAAE,MAAM,CAAC;KACrB,CAAC;IACF,MAAM,CAAC,EAAE;QACP,OAAO,CAAC,EAAE,OAAO,CAAC;KACnB,CAAC;IACF,kEAAkE;IAClE,SAAS,CAAC,EAAE,OAAO,KAAK,CAAC;IACzB,sGAAsG;IACtG,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB;AAED,MAAM,WAAW,WAAW;IAC1B,GAAG,EAAE,MAAM,CAAC;IACZ,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,MAAM,CAAC;IACnB,EAAE,EAAE,OAAO,CAAC;IACZ,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAChC,IAAI,EAAE,MAAM,CAAC;IACb,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,SAAS,EAAE,OAAO,CAAC;IACnB,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,yFAAyF;IACzF,KAAK,CAAC,EAAE,UAAU,CAAC;CACpB"}

package/dist/index.d.ts

@@ -1,3 +1,4 @@
 export * from './lib/index.js';
 export * from './fetch/index.js';
+export * from './pdf/index.js';
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAQA,cAAc,gBAAgB,CAAC;AAC/B,cAAc,kBAAkB,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAQA,cAAc,gBAAgB,CAAC;AAC/B,cAAc,kBAAkB,CAAC;AACjC,cAAc,gBAAgB,CAAC"}

package/dist/index.js

@@ -7,3 +7,4 @@
 // client-safe, network-free entry (pure HTML parsing, no I/O).
 export * from './lib/index.js';
 export * from './fetch/index.js';
+export * from './pdf/index.js';

package/dist/parse/extract_table.d.ts

@@ -29,10 +29,45 @@ export type GridSelector = {
      */
     headerRow?: string;
 };
+/**
+ * Combine-mode selector — for pages that split one logical table across
+ * several sibling <table>s: a header-only <table> followed by one <table>
+ * per section (e.g. per year), each led by a section-label row. Common on
+ * Computershare-style IR pages (e.g. JB Hi-Fi's dividend history). The header
+ * comes from `headerSelector`'s first row; data rows are gathered from that
+ * table plus its immediately-following sibling <table>s that share its column
+ * count (stopping at the first sibling that doesn't), so an unrelated later
+ * table can't leak in.
+ */
+export type CombineSelector = {
+    /** Selector matching the ONE table whose first row supplies the shared headers. */
+    headerSelector: string;
+    /**
+     * Drop "section-label" rows — a row in which only the first cell holds text
+     * and every other cell is empty (e.g. a bare "2025" year divider). Off by
+     * default. Column-mapping/parsing would already null such a row out, but
+     * dropping it keeps the raw row set clean and honest.
+     */
+    dropLabelRows?: boolean;
+};
 export type ExtractTableOptions = {
     select?: string;
     /** Read a <div>-based pseudo-table instead of a real <table>. Takes precedence over `select`. */
     grid?: GridSelector;
+    /**
+     * Combine a header-only table with its per-section sibling tables into one
+     * table (see CombineSelector). Takes precedence over `select`; `grid` still
+     * wins over this.
+     */
+    combine?: CombineSelector;
+    /**
+     * CSS selectors whose matching descendants are removed from every cell
+     * BEFORE its text is read — for stripping in-cell noise that corrupts
+     * parsing, e.g. `['sup']` to drop footnote-reference superscripts so
+     * "100<sup>4</sup>" reads as "100" not "1004". Applies to real-table, grid,
+     * and combine modes alike. Omit to read cells verbatim (the default).
+     */
+    cellIgnoreSelectors?: string[];
 };
 export declare function extractTable(html: string, opts?: ExtractTableOptions): ExtractTableResult;
 //# sourceMappingURL=extract_table.d.ts.map

package/dist/parse/extract_table.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"extract_table.d.ts","sourceRoot":"","sources":["../../src/parse/extract_table.ts"],"names":[],"mappings":"AAsBA,MAAM,MAAM,kBAAkB,GAAG;IAC/B,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,IAAI,EAAE,MAAM,EAAE,EAAE,CAAC;IACjB,OAAO,CAAC,EAAE,MAAM,CAAC;CAClB,CAAC;AAEF;;;;;GAKG;AACH,MAAM,MAAM,YAAY,GAAG;IACzB;;;;;;OAMG;IACH,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,0FAA0F;IAC1F,GAAG,EAAE,MAAM,CAAC;IACZ,0DAA0D;IAC1D,IAAI,EAAE,MAAM,CAAC;IACb;;;;OAIG;IACH,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB,CAAC;AAEF,MAAM,MAAM,mBAAmB,GAAG;IAChC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,iGAAiG;IACjG,IAAI,CAAC,EAAE,YAAY,CAAC;CACrB,CAAC;AAwLF,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,mBAAmB,GAAG,kBAAkB,CA6BzF"}
+{"version":3,"file":"extract_table.d.ts","sourceRoot":"","sources":["../../src/parse/extract_table.ts"],"names":[],"mappings":"AAsBA,MAAM,MAAM,kBAAkB,GAAG;IAC/B,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,IAAI,EAAE,MAAM,EAAE,EAAE,CAAC;IACjB,OAAO,CAAC,EAAE,MAAM,CAAC;CAClB,CAAC;AAEF;;;;;GAKG;AACH,MAAM,MAAM,YAAY,GAAG;IACzB;;;;;;OAMG;IACH,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,0FAA0F;IAC1F,GAAG,EAAE,MAAM,CAAC;IACZ,0DAA0D;IAC1D,IAAI,EAAE,MAAM,CAAC;IACb;;;;OAIG;IACH,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB,CAAC;AAEF;;;;;;;;;GASG;AACH,MAAM,MAAM,eAAe,GAAG;IAC5B,mFAAmF;IACnF,cAAc,EAAE,MAAM,CAAC;IACvB;;;;;OAKG;IACH,aAAa,CAAC,EAAE,OAAO,CAAC;CACzB,CAAC;AAEF,MAAM,MAAM,mBAAmB,GAAG;IAChC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,iGAAiG;IACjG,IAAI,CAAC,EAAE,YAAY,CAAC;IACpB;;;;OAIG;IACH,OAAO,CAAC,EAAE,eAAe,CAAC;IAC1B;;;;;;OAMG;IACH,mBAAmB,CAAC,EAAE,MAAM,EAAE,CAAC;CAChC,CAAC;AAgQF,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,mBAAmB,GAAG,kBAAkB,CAkCzF"}

package/dist/parse/extract_table.js

@@ -25,6 +25,18 @@ const NO_TABLE_WARNING = 'No table-like element found on the page (it may be Jav
 function normalizeCell(raw) {
     return raw.replace(/\s+/g, ' ').trim();
 }
+// Read a cell's normalized text, optionally removing noise descendants
+// (`ignore`) first — e.g. footnote <sup>s — so they never reach the parsed
+// value. Clones the cell so the removal is local to this read and never
+// mutates the shared document (other cells / re-reads see the original).
+function readCellText(cellSel, ignore) {
+    if (ignore && ignore.length > 0) {
+        const clone = cellSel.clone();
+        clone.find(ignore.join(',')).remove();
+        return normalizeCell(clone.text());
+    }
+    return normalizeCell(cellSel.text());
+}
 // Rows that belong directly to THIS table — not to any table nested inside a
 // cell. `tr` may be a direct child of <table>, or a child of a direct
 // <thead>/<tbody>/<tfoot> child. Collecting via `.children()` at each level
@@ -68,19 +80,14 @@ function scoreTable(tableSel) {
     const consistencyRatio = numDataRows > 0 ? numConsistent / numDataRows : 0;
     return numHeaderCells * 2 + (hasTh ? 5 : 0) + numDataRows * 3 + consistencyRatio * 10;
 }
-function readTable(tableSel) {
-    const rows = tableRows(tableSel);
-    if (rows.length === 0)
-        return { headers: [], rows: [] };
-    const headerCells = rows.eq(0).children('th, td');
-    const headers = [];
-    for (let i = 0; i < headerCells.length; i++) {
-        headers.push(normalizeCell(headerCells.eq(i).text()));
-    }
-    const dataRowsSel = rows.slice(1);
+// Read the data rows (everything after the header row) of one real <table>,
+// aligning each cell against `headers` (for the repeated-in-cell-label strip)
+// and removing `ignore` descendants first. Shared by readTable and combine
+// mode so both handle ragged/label/responsive rows identically.
+function readTableDataRows(rows, headers, ignore, startIndex) {
     const dataRows = [];
-    for (let i = 0; i < dataRowsSel.length; i++) {
-        const rowSel = dataRowsSel.eq(i);
+    for (let i = startIndex; i < rows.length; i++) {
+        const rowSel = rows.eq(i);
         // A genuine data row has at least one <td>; a pure-<th> row (a section
         // divider spanning the table) is skipped — prototype behaviour. But the
         // cells themselves are then read as `th, td`, not `td` alone, so a
@@ -92,7 +99,7 @@ function readTable(tableSel) {
         const cellEls = rowSel.children('th, td');
         const cells = [];
         for (let j = 0; j < cellEls.length; j++) {
-            let text = normalizeCell(cellEls.eq(j).text());
+            let text = readCellText(cellEls.eq(j), ignore);
             // Responsive-table pattern: some IR pages repeat the column header
             // INSIDE each cell as a (visually-hidden on desktop) label, e.g. NAB's
             // "<div>Payment date</div><div>2 July 2026</div>", whose .text()
@@ -108,21 +115,80 @@ function readTable(tableSel) {
         }
         dataRows.push(cells);
     }
-    return { headers, rows: dataRows };
+    return dataRows;
+}
+function readTable(tableSel, ignore) {
+    const rows = tableRows(tableSel);
+    if (rows.length === 0)
+        return { headers: [], rows: [] };
+    const headerCells = rows.eq(0).children('th, td');
+    const headers = [];
+    for (let i = 0; i < headerCells.length; i++) {
+        headers.push(readCellText(headerCells.eq(i), ignore));
+    }
+    return { headers, rows: readTableDataRows(rows, headers, ignore, 1) };
+}
+// True when a row is a "section-label" row: exactly one non-empty cell, and
+// it's the first — e.g. a bare "2025" year divider between per-year tables.
+function isLabelRow(cells) {
+    if (cells.length === 0 || cells[0].trim() === '')
+        return false;
+    for (let i = 1; i < cells.length; i++) {
+        if (cells[i].trim() !== '')
+            return false;
+    }
+    return true;
+}
+// Combine mode — a header-only table plus its per-section sibling tables.
+function readCombined($, combine, ignore) {
+    const headerTable = $(combine.headerSelector).first();
+    if (headerTable.length === 0) {
+        return { headers: [], rows: [], warning: `The selector "${combine.headerSelector}" matched no element on the page.` };
+    }
+    const headerRows = tableRows(headerTable);
+    if (headerRows.length === 0)
+        return { headers: [], rows: [], warning: NO_TABLE_WARNING };
+    const headerCells = headerRows.eq(0).children('th, td');
+    const headers = [];
+    for (let i = 0; i < headerCells.length; i++)
+        headers.push(readCellText(headerCells.eq(i), ignore));
+    const allRows = [];
+    // Any data rows in the header table itself (a header-only table has none).
+    allRows.push(...readTableDataRows(headerRows, headers, ignore, 1));
+    // Following-sibling tables that share the header's column count; stop at the
+    // first sibling that doesn't (or a non-table sibling) so an unrelated later
+    // table with a coincidentally-equal width can't leak in.
+    const siblings = headerTable.nextAll('table');
+    for (let i = 0; i < siblings.length; i++) {
+        const sib = siblings.eq(i);
+        const sibRows = tableRows(sib);
+        if (sibRows.length === 0)
+            break;
+        const firstRowCols = sibRows.eq(0).children('th, td').length;
+        if (firstRowCols !== headers.length)
+            break;
+        // Every row of a section table is data here (its first row is a section
+        // label, NOT a repeated header) — read from index 0.
+        allRows.push(...readTableDataRows(sibRows, headers, ignore, 0));
+    }
+    const rows = combine.dropLabelRows ? allRows.filter((cells) => !isLabelRow(cells)) : allRows;
+    if (headers.length === 0 && rows.length === 0)
+        return { headers, rows, warning: NO_TABLE_WARNING };
+    return { headers, rows };
 }
 // Read one cell's normalized text list from a row selection, using a cell
 // sub-selector (div-grid mode — see GridSelector).
-function gridRowCells(rowSel, cellSelector) {
+function gridRowCells(rowSel, cellSelector, ignore) {
     const cellEls = rowSel.find(cellSelector);
     const cells = [];
     for (let j = 0; j < cellEls.length; j++)
-        cells.push(normalizeCell(cellEls.eq(j).text()));
+        cells.push(readCellText(cellEls.eq(j), ignore));
     return cells;
 }
 // Read a single div-grid wrapper into the same {headers, rows} shape a real
 // <table> yields, so every downstream stage (mapColumns/mapRows) is oblivious
 // to which markup the data came from.
-function readGridContainer(container, grid) {
+function readGridContainer(container, grid, ignore) {
     const allRows = container.find(grid.row);
     if (allRows.length === 0)
         return { headers: [], rows: [] };
@@ -131,10 +197,10 @@ function readGridContainer(container, grid) {
     if (grid.headerRow != null) {
         const headerRows = allRows.filter(grid.headerRow);
         if (headerRows.length > 0)
-            headers = gridRowCells(headerRows.eq(0), grid.cell);
+            headers = gridRowCells(headerRows.eq(0), grid.cell, ignore);
     }
     if (headers.length === 0) {
-        headers = gridRowCells(allRows.eq(0), grid.cell);
+        headers = gridRowCells(allRows.eq(0), grid.cell, ignore);
         firstRowIsHeader = true;
     }
     const dataRows = [];
@@ -144,14 +210,14 @@ function readGridContainer(container, grid) {
             continue; // never treat a header row as data
         if (grid.headerRow == null && firstRowIsHeader && i === 0)
             continue; // first row already consumed as header
-        const cells = gridRowCells(rowSel, grid.cell);
+        const cells = gridRowCells(rowSel, grid.cell, ignore);
         if (cells.length === 0)
             continue;
         dataRows.push(cells);
     }
     return { headers, rows: dataRows };
 }
-function readGrid($, grid) {
+function readGrid($, grid, ignore) {
     const containers = grid.container != null ? $(grid.container) : $.root();
     if (grid.container != null && containers.length === 0) {
         return { headers: [], rows: [], warning: `The selector "${grid.container}" matched no element on the page.` };
@@ -161,7 +227,7 @@ function readGrid($, grid) {
     // pickBestTable uses for real tables). Ties keep the first encountered.
     let best = null;
     for (let i = 0; i < containers.length; i++) {
-        const parsed = readGridContainer(containers.eq(i), grid);
+        const parsed = readGridContainer(containers.eq(i), grid, ignore);
         if (best === null || parsed.rows.length > best.rows.length)
             best = parsed;
     }
@@ -190,8 +256,12 @@ function pickBestTable($) {
 }
 export function extractTable(html, opts) {
     const $ = load(html);
+    const ignore = opts?.cellIgnoreSelectors;
     if (opts?.grid != null) {
-        return readGrid($, opts.grid);
+        return readGrid($, opts.grid, ignore);
+    }
+    if (opts?.combine != null) {
+        return readCombined($, opts.combine, ignore);
     }
     if (opts?.select != null) {
         const matched = $(opts.select);
@@ -202,7 +272,7 @@ export function extractTable(html, opts) {
                 warning: `The selector "${opts.select}" matched no element on the page.`,
             };
         }
-        const { headers, rows } = readTable(matched.eq(0));
+        const { headers, rows } = readTable(matched.eq(0), ignore);
         if (headers.length === 0 && rows.length === 0) {
             return { headers, rows, warning: NO_TABLE_WARNING };
         }
@@ -212,5 +282,5 @@ export function extractTable(html, opts) {
     if (best === null) {
         return { headers: [], rows: [], warning: NO_TABLE_WARNING };
     }
-    return readTable(best);
+    return readTable(best, ignore);
 }

package/dist/parse/map_rows.d.ts

@@ -2,6 +2,7 @@ import type { ColumnKeywords, ColumnMap, MappedRow } from './types.js';
 export interface MapRowsOptions {
     dateFormats?: string[];
     dateExtractLeading?: boolean;
+    dateYearPivot?: number;
 }
 export declare function mapRows(table: {
     headers: string[];

package/dist/parse/map_rows.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"map_rows.d.ts","sourceRoot":"","sources":["../../src/parse/map_rows.ts"],"names":[],"mappings":"AA2BA,OAAO,KAAK,EAAQ,cAAc,EAAE,SAAS,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AAI7E,MAAM,WAAW,cAAc;IAC7B,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;IAMvB,kBAAkB,CAAC,EAAE,OAAO,CAAC;CAC9B;AAmBD,wBAAgB,OAAO,CACrB,KAAK,EAAE;IAAE,OAAO,EAAE,MAAM,EAAE,CAAC;IAAC,IAAI,EAAE,MAAM,EAAE,EAAE,CAAA;CAAE,EAC9C,GAAG,EAAE,SAAS,EACd,QAAQ,EAAE,cAAc,EACxB,IAAI,CAAC,EAAE,cAAc,GACpB,SAAS,EAAE,CAyBb"}
+{"version":3,"file":"map_rows.d.ts","sourceRoot":"","sources":["../../src/parse/map_rows.ts"],"names":[],"mappings":"AA2BA,OAAO,KAAK,EAAQ,cAAc,EAAE,SAAS,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AAI7E,MAAM,WAAW,cAAc;IAC7B,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;IAMvB,kBAAkB,CAAC,EAAE,OAAO,CAAC;IAM7B,aAAa,CAAC,EAAE,MAAM,CAAC;CACxB;AAqBD,wBAAgB,OAAO,CACrB,KAAK,EAAE;IAAE,OAAO,EAAE,MAAM,EAAE,CAAC;IAAC,IAAI,EAAE,MAAM,EAAE,EAAE,CAAA;CAAE,EAC9C,GAAG,EAAE,SAAS,EACd,QAAQ,EAAE,cAAc,EACxB,IAAI,CAAC,EAAE,cAAc,GACpB,SAAS,EAAE,CA0Bb"}

package/dist/parse/map_rows.js

@@ -26,13 +26,14 @@
 // the exact prior behavior — this is purely additive.
 import { parseNumber } from './parse_number.js';
 import { parseDate } from './parse_date.js';
-function cellValue(type, raw, dateFormats, dateExtractLeading) {
+function cellValue(type, raw, dateFormats, dateExtractLeading, dateYearPivot) {
     if (type === 'number')
         return parseNumber(raw);
     if (type === 'date') {
         return parseDate(raw, {
             ...(dateFormats ? { formats: dateFormats } : {}),
             ...(dateExtractLeading ? { extractLeading: true } : {}),
+            ...(dateYearPivot != null ? { yearPivot: dateYearPivot } : {}),
         });
     }
     const trimmed = raw.trim();
@@ -42,6 +43,7 @@ export function mapRows(table, map, keywords, opts) {
     const keys = Object.keys(map.matched);
     const dateFormats = opts?.dateFormats;
     const dateExtractLeading = opts?.dateExtractLeading;
+    const dateYearPivot = opts?.dateYearPivot;
     return table.rows.map((row) => {
         const mappedRow = {};
         for (const key of keys) {
@@ -53,7 +55,7 @@ export function mapRows(table, map, keywords, opts) {
                     header: candidate.header,
                     index: candidate.index,
                     raw,
-                    value: cellValue(type, raw, dateFormats, dateExtractLeading),
+                    value: cellValue(type, raw, dateFormats, dateExtractLeading, dateYearPivot),
                 };
             });
         }

package/dist/parse/parse_date.d.ts

@@ -1,5 +1,6 @@
 export declare function parseDate(raw: string, opts?: {
     formats?: string[];
     extractLeading?: boolean;
+    yearPivot?: number;
 }): string | null;
 //# sourceMappingURL=parse_date.d.ts.map

package/dist/parse/parse_date.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"parse_date.d.ts","sourceRoot":"","sources":["../../src/parse/parse_date.ts"],"names":[],"mappings":"AAuFA,wBAAgB,SAAS,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE;IAAE,OAAO,CAAC,EAAE,MAAM,EAAE,CAAC;IAAC,cAAc,CAAC,EAAE,OAAO,CAAA;CAAE,GAAG,MAAM,GAAG,IAAI,CA4C7G"}
+{"version":3,"file":"parse_date.d.ts","sourceRoot":"","sources":["../../src/parse/parse_date.ts"],"names":[],"mappings":"AA6GA,wBAAgB,SAAS,CACvB,GAAG,EAAE,MAAM,EACX,IAAI,CAAC,EAAE;IAAE,OAAO,CAAC,EAAE,MAAM,EAAE,CAAC;IAAC,cAAc,CAAC,EAAE,OAAO,CAAC;IAAC,SAAS,CAAC,EAAE,MAAM,CAAA;CAAE,GAC1E,MAAM,GAAG,IAAI,CAoDf"}

package/dist/parse/parse_date.js

@@ -57,6 +57,29 @@ function monthFromName(name) {
     const key = name.toLowerCase().slice(0, 3);
     return MONTHS[key] ?? null;
 }
+/**
+ * Resolve a year token that may be 2- or 4-digit into a full year.
+ *
+ * 4-digit years are returned as-is. A 2-digit year is a GUESS (which century?)
+ * — so, true to the engine's "never guess" contract, it is only accepted when
+ * the caller opts in with `yearPivot`, and returns null otherwise. When opted
+ * in, `yearPivot` is the base of a sliding 100-year window: the 2-digit value
+ * maps to the single year in `[yearPivot, yearPivot + 99]`. E.g. `yearPivot:
+ * 2000` reads "26" -> 2026 and "99" -> 2099; `yearPivot: 1950` reads "26" ->
+ * 2026 and "99" -> 1999. A 3-digit token is treated as a literal year (padded
+ * on output), matching prior 4-digit-only strictness closely enough for the
+ * rare case.
+ */
+function resolveYear(raw, yearPivot) {
+    if (raw.length >= 3)
+        return Number(raw);
+    if (yearPivot == null)
+        return null; // 2-digit year without opt-in — never guess
+    const yy = Number(raw);
+    const base = yearPivot - (yearPivot % 100);
+    const candidate = base + yy;
+    return candidate < yearPivot ? candidate + 100 : candidate;
+}
 function pickSlashHint(formats) {
     if (!formats)
         return undefined;
@@ -90,6 +113,7 @@ export function parseDate(raw, opts) {
     const s = String(raw).trim();
     if (s === '')
         return null;
+    const yearPivot = opts?.yearPivot;
     // End-of-token anchor. Strict mode (default) anchors to end-of-string, so
     // any trailing text fails the match. `extractLeading` relaxes that to "no
     // further word char or slash follows the date" — enough to end the date
@@ -97,26 +121,34 @@ export function parseDate(raw, opts) {
     // without letting a partial number ("2024" out of "20245") slip through.
     const tail = opts?.extractLeading ? '(?![\\w/])' : '$';
     const re = (body) => new RegExp(`^${body}${tail}`);
-    // ISO: yyyy-mm-dd
+    // ISO: yyyy-mm-dd (always 4-digit year — never abbreviated in ISO form).
     let m = s.match(re('(\\d{4})-(\\d{2})-(\\d{2})'));
     if (m) {
         return toIsoIfValid(Number(m[1]), Number(m[2]), Number(m[3]));
     }
-    // "D Mon YYYY" / "D Month YYYY", e.g. "5 Mar 2026", "18 November 2025"
-    m = s.match(re('(\\d{1,2})\\s+([A-Za-z]{3,})\\s+(\\d{4})'));
+    // "D Mon YYYY" / "D Month YYYY", e.g. "5 Mar 2026", "18 November 2025".
+    // The year is 2–4 digits; a 2-digit year resolves only under `yearPivot`
+    // (see resolveYear), otherwise it stays null — preserving strict default.
+    m = s.match(re('(\\d{1,2})\\s+([A-Za-z]{3,})\\s+(\\d{2,4})'));
     if (m) {
         const month = monthFromName(m[2]);
         if (month == null)
             return null;
-        return toIsoIfValid(Number(m[3]), month, Number(m[1]));
+        const year = resolveYear(m[3], yearPivot);
+        if (year == null)
+            return null;
+        return toIsoIfValid(year, month, Number(m[1]));
     }
     // "Mon D, YYYY" / "Month D YYYY", comma optional, e.g. "Mar 5, 2026"
-    m = s.match(re('([A-Za-z]{3,})\\s+(\\d{1,2}),?\\s+(\\d{4})'));
+    m = s.match(re('([A-Za-z]{3,})\\s+(\\d{1,2}),?\\s+(\\d{2,4})'));
     if (m) {
         const month = monthFromName(m[1]);
         if (month == null)
             return null;
-        return toIsoIfValid(Number(m[3]), month, Number(m[2]));
+        const year = resolveYear(m[3], yearPivot);
+        if (year == null)
+            return null;
+        return toIsoIfValid(year, month, Number(m[2]));
     }
     // Slash dates: D/M/Y or M/D/Y — ambiguous without a hint or a proof by
     // calendar impossibility (see resolveSlashDate). Two-digit years are

package/dist/parse/parse_number.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"parse_number.d.ts","sourceRoot":"","sources":["../../src/parse/parse_number.ts"],"names":[],"mappings":"AAaA,wBAAgB,WAAW,CAAC,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CA4BtD"}
+{"version":3,"file":"parse_number.d.ts","sourceRoot":"","sources":["../../src/parse/parse_number.ts"],"names":[],"mappings":"AAaA,wBAAgB,WAAW,CAAC,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CA8CtD"}

package/dist/parse/parse_number.js

@@ -29,6 +29,22 @@ export function parseNumber(raw) {
     // header's no-cents-conversion note). Word-boundaried so it never touches
     // the digits or an unrelated word like "recent".
     s = s.replace(/\bcents?\b/gi, '');
+    // Strip a trailing per-unit annotation — "$2.05/share", "1.2/unit". The
+    // slash-plus-word denominator states the amount's unit basis (per share, per
+    // security); it's not part of the number. Letters required after the slash,
+    // so a bare fraction like "3/4" is left untouched (and then fails to parse,
+    // as before). Runs before the bare-cent strip so "2c/share"-style oddities
+    // reduce cleanly.
+    s = s.replace(/\/[A-Za-z]+\.?$/, '');
+    s = s.trim();
+    // Strip a trailing bare cent marker — "45c" / "14.5 c" / "45¢". This is the
+    // symbol/abbreviation form of the "cents" word stripped above (many AU/UK IR
+    // tables write "45c" rather than "45 cents"), so it's tokenized the same way:
+    // the SCALING to dollars stays the caller's job (amount_unit). Anchored to a
+    // digit immediately before it and to end-of-string, so it only ever removes a
+    // genuine trailing unit — never a 'c' inside a word ("recent" ends in 't', and
+    // "abc" has no leading digit) and never a mid-string letter.
+    s = s.replace(/(\d)\s*[c¢]$/i, '$1');
     s = s.trim();
     if (s === '')
         return null;

package/dist/pdf/extract_table_pdf.d.ts

@@ -0,0 +1,11 @@
+import type { ExtractTableResult } from '../parse/extract_table.js';
+export type ExtractTablePdfOptions = {
+    headerLabels?: string[];
+    headerRows?: number;
+    columnBands?: number[];
+    mergeContinuationRows?: boolean;
+    rowTolerance?: number;
+    maxPages?: number;
+};
+export declare function extractTableFromPdf(data: Uint8Array, opts?: ExtractTablePdfOptions): Promise<ExtractTableResult>;
+//# sourceMappingURL=extract_table_pdf.d.ts.map

package/dist/pdf/extract_table_pdf.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"extract_table_pdf.d.ts","sourceRoot":"","sources":["../../src/pdf/extract_table_pdf.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAEpE,MAAM,MAAM,sBAAsB,GAAG;IACnC,YAAY,CAAC,EAAE,MAAM,EAAE,CAAC;IACxB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,WAAW,CAAC,EAAE,MAAM,EAAE,CAAC;IACvB,qBAAqB,CAAC,EAAE,OAAO,CAAC;IAChC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB,CAAC;AA4EF,wBAAsB,mBAAmB,CACvC,IAAI,EAAE,UAAU,EAChB,IAAI,GAAE,sBAA2B,GAChC,OAAO,CAAC,kBAAkB,CAAC,CAwD7B"}

package/dist/pdf/extract_table_pdf.js

@@ -0,0 +1,141 @@
+// hazo_scrape/src/pdf/extract_table_pdf.ts — deterministic PDF text-layer → table.
+//
+// Reconstructs a {headers, rows} grid from a PDF's positioned text items by
+// clustering items into visual lines (by y) and columns (by x-band), mirroring
+// the ExtractTableResult shape of extract_table.ts so a caller's
+// mapColumns/mapRows pipeline consumes HTML and PDF tables identically.
+//
+// SERVER-ONLY: pulls in pdfjs-dist. Exported from the package root ('.'),
+// never from the client-safe './parse' entry.
+import { getDocument } from 'pdfjs-dist/legacy/build/pdf.mjs';
+/** Cluster items into visual lines by y within `tol`, ordered top→bottom. */
+function clusterLines(items, tol) {
+    const sorted = [...items].sort((a, b) => b.y - a.y);
+    const lines = [];
+    for (const it of sorted) {
+        const line = lines.find((l) => Math.abs(l.y - it.y) <= tol);
+        if (line)
+            line.items.push(it);
+        else
+            lines.push({ y: it.y, items: [it] });
+    }
+    return lines;
+}
+/** Auto-detect column left-edges by cutting the sorted x-starts at the widest gaps. */
+function detectBands(items) {
+    const xs = [...new Set(items.map((i) => Math.round(i.x)))].sort((a, b) => a - b);
+    if (xs.length === 0)
+        return [0];
+    const bands = [xs[0]];
+    for (let i = 1; i < xs.length; i++) {
+        if (xs[i] - xs[i - 1] > 25)
+            bands.push(xs[i]); // 25px gap = new column
+    }
+    return bands;
+}
+/** Assign x to the last band whose left-edge is <= x. */
+function bandIndex(bands, x) {
+    let idx = 0;
+    for (let i = 0; i < bands.length; i++)
+        if (x + 3 >= bands[i])
+            idx = i;
+    return idx;
+}
+// A word the PDF split across two text runs reports the second run beginning
+// (within GLUE_PX) exactly where the first ended, with NO whitespace at either
+// run's boundary — e.g. "Septemb"|"er". A real inter-word space renders a
+// visible gap (and/or a boundary space in the run text). GLUE_PX is the max
+// horizontal gap still considered "same word".
+const GLUE_PX = 1;
+/**
+ * Join a band's items (sorted by x) into one cell string. Consecutive items
+ * that are horizontally CONTIGUOUS and carry no whitespace at their shared
+ * boundary are glued with no space (they are pieces of one word split across
+ * text runs); otherwise a single space separates them. Without this, a
+ * mid-word split surfaces as "Septemb er 2020" and fails downstream date/number
+ * parsing.
+ */
+function joinCellItems(items) {
+    let out = '';
+    let prev = null;
+    for (const it of items) {
+        if (!it.str.trim())
+            continue;
+        if (prev != null) {
+            const contiguous = it.x - (prev.x + prev.w) < GLUE_PX;
+            const boundarySpace = /\s$/.test(prev.str) || /^\s/.test(it.str);
+            out += contiguous && !boundarySpace ? '' : ' ';
+        }
+        out += it.str.trim();
+        prev = it;
+    }
+    return out.replace(/\s+/g, ' ').trim();
+}
+/** Concatenate a line's items into one cell per band (items sorted by x). */
+function lineToCells(line, bands) {
+    const cells = bands.map(() => []);
+    for (const it of [...line.items].sort((a, b) => a.x - b.x)) {
+        if (!it.str.trim())
+            continue;
+        cells[bandIndex(bands, it.x)].push(it);
+    }
+    return cells.map(joinCellItems);
+}
+export async function extractTableFromPdf(data, opts = {}) {
+    const tol = opts.rowTolerance ?? 3;
+    const doc = await getDocument({ data, isEvalSupported: false, useSystemFonts: false }).promise;
+    const pageCount = opts.maxPages ? Math.min(doc.numPages, opts.maxPages) : doc.numPages;
+    const pages = [];
+    for (let p = 1; p <= pageCount; p++) {
+        const page = await doc.getPage(p);
+        const content = await page.getTextContent();
+        const items = [];
+        for (const it of content.items) {
+            if (typeof it.str !== 'string' || !it.str.trim())
+                continue;
+            items.push({ x: it.transform[4], y: it.transform[5], w: it.width ?? 0, str: it.str });
+        }
+        pages.push(items);
+    }
+    // Column geometry is page-independent, so detect bands from every item; but
+    // cluster visual lines PER PAGE. Text-item y-coordinates repeat page to page
+    // (row N sits at the same y on every page), so clustering the concatenation
+    // would merge each page's row N into a single garbled line. Page order is
+    // preserved, and a single-page document behaves exactly as before.
+    const bands = opts.columnBands ?? detectBands(pages.flat());
+    let grid = [];
+    for (const items of pages) {
+        for (const line of clusterLines(items, tol))
+            grid.push(lineToCells(line, bands));
+    }
+    // Continuation merge: a line whose leftmost cell is empty belongs to the row above.
+    if (opts.mergeContinuationRows) {
+        const merged = [];
+        for (const row of grid) {
+            if (merged.length > 0 && row[0].trim() === '') {
+                const prev = merged[merged.length - 1];
+                for (let i = 0; i < row.length; i++) {
+                    if (row[i].trim())
+                        prev[i] = [prev[i], row[i]].filter(Boolean).join(' ').trim();
+                }
+            }
+            else {
+                merged.push([...row]);
+            }
+        }
+        grid = merged;
+    }
+    let headers;
+    let rows;
+    if (opts.headerLabels) {
+        headers = opts.headerLabels;
+        rows = grid;
+    }
+    else {
+        const n = Math.max(1, opts.headerRows ?? 1);
+        const headerLines = grid.slice(0, n);
+        headers = bands.map((_, c) => headerLines.map((l) => l[c] ?? '').join(' ').replace(/\s+/g, ' ').trim());
+        rows = grid.slice(n);
+    }
+    return { headers, rows };
+}

package/dist/pdf/index.d.ts

@@ -0,0 +1,2 @@
+export * from './extract_table_pdf.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/pdf/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/pdf/index.ts"],"names":[],"mappings":"AACA,cAAc,wBAAwB,CAAC"}

package/dist/pdf/index.js

@@ -0,0 +1,2 @@
+// hazo_scrape/src/pdf/index.ts — server-only PDF table extraction surface.
+export * from './extract_table_pdf.js';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "hazo_scrape",
-  "version": "1.4.0",
+  "version": "1.6.0",
   "description": "Generic source-agnostic web scraping engine with a network-free parse core.",
   "type": "module",
   "module": "./dist/index.js",
@@ -62,6 +62,7 @@
     "cheerio": "^1.2.0",
     "hazo_config": "^2.4.1",
     "hazo_secure": "^1.4.0",
+    "pdfjs-dist": "^4.10.38",
     "robots-parser": "^3.0.1",
     "undici": "^7.28.0"
   }