@d-zero/page-cluster 0.2.0 → 0.3.0

package/dist/filter-first-party-stylesheet-hrefs.d.ts

@@ -1,7 +1,10 @@
 /**
  * Narrows every page's `stylesheetHrefs` down to just the hrefs whose host
- * matches the single most common host across the whole batch (the site's own
- * first-party domain), dropping every other host.
+ * matches that page's own `host` field (direct comparison), when the caller
+ * provides it. Falls back to the single most common host across the whole
+ * batch (the site's own first-party domain, inferred rather than given) for
+ * any page that omits `host` — see this function's own JSDoc further down
+ * for that fallback's known limitations.
  *
  * Confirmed on real crawl data (302 pages): a handful of articles embedding
  * a YouTube video pulled in `youtube.com`'s own player stylesheet plus a
@@ -20,20 +23,37 @@
  * first-party hrefs before blocking removes that false signal at the
  * source, rather than trying to recognize its effects downstream.
  *
- * Determining "first-party" from the batch's own href distribution (rather
- * than, say, comparing each href's host against each page's own URL) means
- * this needs no extra per-page input beyond what
- * {@link ./resolve-blocking-group-keys.js | resolveBlockingGroupKeys} already
- * takes — but it inherits that same function's "roughly homogeneous batch"
- * precondition (see `computeDocumentFrequency`'s own JSDoc): a batch that
- * mixes pages from more than one site in one call has no single genuine
- * first-party host to find, and this function has no way to detect that
- * it's been handed one — it will still confidently pick *a* dominant host
- * (whichever site contributes more stylesheet-bearing pages) and silently
- * strip every other site's real first-party hrefs. Splitting a
- * multi-site/section batch into homogeneous groups before calling this is
- * the caller's responsibility, same as it already is for
- * `resolveBlockingGroupKeys`.
+ * `host`, when provided, is compared directly against each of that page's
+ * own stylesheet hrefs — no batch-wide inference involved, so this path
+ * cannot mistake a third party for the first party regardless of how many
+ * pages happen to also embed it. Added after a real crawl (302 pages) hit
+ * the dominant-host fallback's tie case: every single page loaded both its
+ * own first-party stylesheet *and* the same `fonts.googleapis.com` webfont
+ * request (a common sitewide pattern, not a rare misconfiguration), so both
+ * hosts tied at "referenced by 100% of pages" and the fallback's `>`-only
+ * tie-break (see below) picked whichever host happened to be counted first
+ * — silently the wrong one on that crawl. A caller that already knows each
+ * page's own host (e.g. it has the page's URL on hand, as most crawlers do)
+ * should always provide it; the inferred fallback exists only for callers
+ * that don't have that information available.
+ *
+ * The dominant-host fallback (used per-page whenever that page omits `host`)
+ * determines "first-party" from the batch's own href distribution instead
+ * — comparing each href's host against each page's own URL wasn't possible
+ * for callers that only ever had `stylesheetHrefs` on hand, not full page
+ * URLs (this is still true for anything built directly on
+ * {@link ./resolve-blocking-group-keys.js | resolveBlockingGroupKeys}'s
+ * `PageBlockingSignals`, which never carried a host field until this
+ * `host` option was added). It inherits `resolveBlockingGroupKeys`'s own
+ * "roughly homogeneous batch" precondition (see `computeDocumentFrequency`'s
+ * own JSDoc): a batch that mixes pages from more than one site in one call
+ * has no single genuine first-party host to find, and this function has no
+ * way to detect that it's been handed one — it will still confidently pick
+ * a* dominant host (whichever site contributes more stylesheet-bearing
+ * pages) and silently strip every other site's real first-party hrefs.
+ * Splitting a multi-site/section batch into homogeneous groups before
+ * calling this (or simply providing `host` per page) is the caller's
+ * responsibility, same as it already is for `resolveBlockingGroupKeys`.
  *
  * The dominant host is picked by how many *pages* reference it at least
  * once, not by how many stylesheet `<link>` tags reference it — a page
@@ -45,19 +65,23 @@
  * unresolved protocol-relative URL) is still one site, not two competing
  * "hosts" splitting its own vote.
  *
- * The trade-off: a site that legitimately serves its own stylesheets from
- * more than one first-party host (e.g. a CDN subdomain alongside the main
- * domain) will have its non-dominant host's hrefs dropped too, same as any
- * genuinely-third-party host — not yet observed on real data, but a known
- * limitation of picking a single dominant host rather than a set.
+ * The fallback's remaining trade-offs: a site that legitimately serves its
+ * own stylesheets from more than one first-party host (e.g. a CDN subdomain
+ * alongside the main domain) will have its non-dominant host's hrefs
+ * dropped too, same as any genuinely-third-party host; and a tie between
+ * two equally-common hosts silently keeps whichever was counted first
+ * (confirmed above to include real, common sitewide third parties, not
+ * just a theoretical edge case) — both are avoided entirely by providing
+ * `host`.
  *
- * A batch where no page has any stylesheet href at all (or none of the
- * hrefs are parseable absolute URLs) has no host to detect; every page's
- * `stylesheetHrefs` is returned unchanged in that case, matching this
+ * A page with neither a provided `host` nor any batch-wide dominant host to
+ * fall back on (no page in the batch has any parseable stylesheet href at
+ * all) has its `stylesheetHrefs` returned unchanged, matching this
  * function's job of narrowing signal, not fabricating it.
  * @param pages
  * @example
  * ```ts
+ * // Without `host`: falls back to dominant-host inference (ties possible).
  * filterFirstPartyStylesheetHrefs([
  * 	{ stylesheetHrefs: ['https://example.com/a.css', 'https://example.com/b.css'] },
  * 	{ stylesheetHrefs: ['https://example.com/a.css', 'https://fonts.googleapis.com/css?family=x'] },
@@ -66,8 +90,18 @@
  * // 	{ stylesheetHrefs: ['https://example.com/a.css', 'https://example.com/b.css'] },
  * // 	{ stylesheetHrefs: ['https://example.com/a.css'] }, // fonts.googleapis.com dropped
  * // ]
+ *
+ * // With `host`: direct per-page comparison, immune to ties.
+ * filterFirstPartyStylesheetHrefs([
+ * 	{
+ * 		host: 'example.com',
+ * 		stylesheetHrefs: ['https://example.com/a.css', 'https://fonts.googleapis.com/css?family=x'],
+ * 	},
+ * ]);
+ * // [{ host: 'example.com', stylesheetHrefs: ['https://example.com/a.css'] }]
  * ```
  */
 export declare function filterFirstPartyStylesheetHrefs<T extends {
     stylesheetHrefs: readonly string[];
+    host?: string;
 }>(pages: readonly T[]): T[];

package/dist/filter-first-party-stylesheet-hrefs.js

@@ -16,8 +16,11 @@ function tryGetHost(href) {
 }
 /**
  * Narrows every page's `stylesheetHrefs` down to just the hrefs whose host
- * matches the single most common host across the whole batch (the site's own
- * first-party domain), dropping every other host.
+ * matches that page's own `host` field (direct comparison), when the caller
+ * provides it. Falls back to the single most common host across the whole
+ * batch (the site's own first-party domain, inferred rather than given) for
+ * any page that omits `host` — see this function's own JSDoc further down
+ * for that fallback's known limitations.
  *
  * Confirmed on real crawl data (302 pages): a handful of articles embedding
  * a YouTube video pulled in `youtube.com`'s own player stylesheet plus a
@@ -36,20 +39,37 @@ function tryGetHost(href) {
  * first-party hrefs before blocking removes that false signal at the
  * source, rather than trying to recognize its effects downstream.
  *
- * Determining "first-party" from the batch's own href distribution (rather
- * than, say, comparing each href's host against each page's own URL) means
- * this needs no extra per-page input beyond what
- * {@link ./resolve-blocking-group-keys.js | resolveBlockingGroupKeys} already
- * takes — but it inherits that same function's "roughly homogeneous batch"
- * precondition (see `computeDocumentFrequency`'s own JSDoc): a batch that
- * mixes pages from more than one site in one call has no single genuine
- * first-party host to find, and this function has no way to detect that
- * it's been handed one — it will still confidently pick *a* dominant host
- * (whichever site contributes more stylesheet-bearing pages) and silently
- * strip every other site's real first-party hrefs. Splitting a
- * multi-site/section batch into homogeneous groups before calling this is
- * the caller's responsibility, same as it already is for
- * `resolveBlockingGroupKeys`.
+ * `host`, when provided, is compared directly against each of that page's
+ * own stylesheet hrefs — no batch-wide inference involved, so this path
+ * cannot mistake a third party for the first party regardless of how many
+ * pages happen to also embed it. Added after a real crawl (302 pages) hit
+ * the dominant-host fallback's tie case: every single page loaded both its
+ * own first-party stylesheet *and* the same `fonts.googleapis.com` webfont
+ * request (a common sitewide pattern, not a rare misconfiguration), so both
+ * hosts tied at "referenced by 100% of pages" and the fallback's `>`-only
+ * tie-break (see below) picked whichever host happened to be counted first
+ * — silently the wrong one on that crawl. A caller that already knows each
+ * page's own host (e.g. it has the page's URL on hand, as most crawlers do)
+ * should always provide it; the inferred fallback exists only for callers
+ * that don't have that information available.
+ *
+ * The dominant-host fallback (used per-page whenever that page omits `host`)
+ * determines "first-party" from the batch's own href distribution instead
+ * — comparing each href's host against each page's own URL wasn't possible
+ * for callers that only ever had `stylesheetHrefs` on hand, not full page
+ * URLs (this is still true for anything built directly on
+ * {@link ./resolve-blocking-group-keys.js | resolveBlockingGroupKeys}'s
+ * `PageBlockingSignals`, which never carried a host field until this
+ * `host` option was added). It inherits `resolveBlockingGroupKeys`'s own
+ * "roughly homogeneous batch" precondition (see `computeDocumentFrequency`'s
+ * own JSDoc): a batch that mixes pages from more than one site in one call
+ * has no single genuine first-party host to find, and this function has no
+ * way to detect that it's been handed one — it will still confidently pick
+ * a* dominant host (whichever site contributes more stylesheet-bearing
+ * pages) and silently strip every other site's real first-party hrefs.
+ * Splitting a multi-site/section batch into homogeneous groups before
+ * calling this (or simply providing `host` per page) is the caller's
+ * responsibility, same as it already is for `resolveBlockingGroupKeys`.
  *
  * The dominant host is picked by how many *pages* reference it at least
  * once, not by how many stylesheet `<link>` tags reference it — a page
@@ -61,19 +81,23 @@ function tryGetHost(href) {
  * unresolved protocol-relative URL) is still one site, not two competing
  * "hosts" splitting its own vote.
  *
- * The trade-off: a site that legitimately serves its own stylesheets from
- * more than one first-party host (e.g. a CDN subdomain alongside the main
- * domain) will have its non-dominant host's hrefs dropped too, same as any
- * genuinely-third-party host — not yet observed on real data, but a known
- * limitation of picking a single dominant host rather than a set.
+ * The fallback's remaining trade-offs: a site that legitimately serves its
+ * own stylesheets from more than one first-party host (e.g. a CDN subdomain
+ * alongside the main domain) will have its non-dominant host's hrefs
+ * dropped too, same as any genuinely-third-party host; and a tie between
+ * two equally-common hosts silently keeps whichever was counted first
+ * (confirmed above to include real, common sitewide third parties, not
+ * just a theoretical edge case) — both are avoided entirely by providing
+ * `host`.
  *
- * A batch where no page has any stylesheet href at all (or none of the
- * hrefs are parseable absolute URLs) has no host to detect; every page's
- * `stylesheetHrefs` is returned unchanged in that case, matching this
+ * A page with neither a provided `host` nor any batch-wide dominant host to
+ * fall back on (no page in the batch has any parseable stylesheet href at
+ * all) has its `stylesheetHrefs` returned unchanged, matching this
  * function's job of narrowing signal, not fabricating it.
  * @param pages
  * @example
  * ```ts
+ * // Without `host`: falls back to dominant-host inference (ties possible).
  * filterFirstPartyStylesheetHrefs([
  * 	{ stylesheetHrefs: ['https://example.com/a.css', 'https://example.com/b.css'] },
  * 	{ stylesheetHrefs: ['https://example.com/a.css', 'https://fonts.googleapis.com/css?family=x'] },
@@ -82,6 +106,15 @@ function tryGetHost(href) {
  * // 	{ stylesheetHrefs: ['https://example.com/a.css', 'https://example.com/b.css'] },
  * // 	{ stylesheetHrefs: ['https://example.com/a.css'] }, // fonts.googleapis.com dropped
  * // ]
+ *
+ * // With `host`: direct per-page comparison, immune to ties.
+ * filterFirstPartyStylesheetHrefs([
+ * 	{
+ * 		host: 'example.com',
+ * 		stylesheetHrefs: ['https://example.com/a.css', 'https://fonts.googleapis.com/css?family=x'],
+ * 	},
+ * ]);
+ * // [{ host: 'example.com', stylesheetHrefs: ['https://example.com/a.css'] }]
  * ```
  */
 export function filterFirstPartyStylesheetHrefs(pages) {
@@ -106,13 +139,19 @@ export function filterFirstPartyStylesheetHrefs(pages) {
             dominantCount = count;
         }
     }
-    if (dominantHost === undefined) {
-        return [...pages];
-    }
-    return pageHrefHosts.map(({ page, hrefHosts }) => ({
-        ...page,
-        stylesheetHrefs: hrefHosts
-            .filter(({ host }) => host === dominantHost)
-            .map(({ href }) => href),
-    }));
+    return pageHrefHosts.map(({ page, hrefHosts }) => {
+        // A page that supplies its own host is judged against that host
+        // alone, bypassing the batch-wide dominant-host inference (and its
+        // tie-breaking pitfall) entirely — see this function's own JSDoc.
+        const expectedHost = page.host ?? dominantHost;
+        if (expectedHost === undefined) {
+            return { ...page };
+        }
+        return {
+            ...page,
+            stylesheetHrefs: hrefHosts
+                .filter(({ host }) => host === expectedHost)
+                .map(({ href }) => href),
+        };
+    });
 }

package/dist/find-shallowest-elements.d.ts

@@ -1,10 +1,29 @@
+/**
+ * One genuinely-closed match for type `T`, carrying the depth used for
+ * shallowest-wins selection and the element's whole-element and inner-content
+ * spans. `depth` is 0 for `<body>` itself, 1 for a direct child, and so on.
+ *
+ * Returned by {@link ./find-shallowest-elements.js | findMatchingElements},
+ * which is the "collect every candidate" primitive shared by every landmark-
+ * scanning use site. Callers that want a single winner per type consume the
+ * matches through {@link ./find-shallowest-elements.js | findShallowestElements}
+ * (which drops `depth` since it's an internal selection artifact once the
+ * shallowest has been chosen).
+ */
+export type MatchingElement<T extends string> = {
+    type: T;
+    depth: number;
+    startOffset: number;
+    endOffset: number;
+    contentStart: number;
+    contentEnd: number;
+};
 /**
  * One winning element for type `T`: the shallowest (fewest ancestors since
  * `<body>`) genuinely-closed match, ties broken by document order. Both the
  * whole-element span (`startOffset`/`endOffset`) and the inner-content span
  * (`contentStart`/`contentEnd`, excluding the element's own opening/closing
- * tags) are always computed — {@link ./extract-landmarks.js | extractLandmarks}
- * only needs the former, {@link ./cap-content-depth.js | capContentDepth}
+ * tags) are always computed — {@link ./cap-content-depth.js | capContentDepth}
  * only needs the latter, and computing both is cheap enough (two
  * `indexOf`/`lastIndexOf` calls) that carrying the unused half costs nothing
  * a caller need worry about.
@@ -17,23 +36,41 @@ export type ShallowestElementMatch<T extends string> = {
     contentEnd: number;
 };
 /**
- * Shared walk behind {@link ./extract-landmarks.js | extractLandmarks} (which
- * matches four landmark types per element in one pass) and
- * {@link ./cap-content-depth.js | capContentDepth} (which matches a single
- * landmark). Both need the identical "shallowest genuinely-closed match
- * wins" search — same `<body>`-scoping, same opaque-tag skip, same malformed-
- * markup discard via {@link ./is-genuine-close.js | isGenuineClose} — so a
- * fix to one (e.g. the body-scoping edge case already fixed once in
- * `extractLandmarks`) can't silently fail to apply to the other.
+ * The shared HTML walk behind every landmark-scanning use site: collects
+ * every genuinely-closed match for every requested type, without picking a
+ * winner. {@link ./find-shallowest-elements.js | findShallowestElements}
+ * layers shallowest-per-type selection on top of this; extractLandmarks
+ * consumes the full list so downstream data-driven frequency filters can
+ * decide which matches are chrome vs content instead of a hard-coded depth
+ * rule making that call up here.
  *
  * Only the first `<body>` is in scope, and nothing inside an opaque tag
  * (`script`/`style`/`noscript`/`svg`) is searched — see
  * `extractLandmarks`/`capContentDepth`'s own JSDoc for why.
+ *
+ * Results are sorted by `startOffset` ascending (document order of the
+ * opening tag), which is what extractLandmarks needs both for document-order
+ * concatenation and for the outer-before-inner sweep used to filter out
+ * nested landmarks so they aren't double-counted in shell tokens.
  * @param html
  * @param matchTypes Given an element's tag name and `role` attribute (already
  * normalized: an empty/absent `role` arrives as `undefined`), returns every
  * type `T` that element matches. Returning more than one lets a single
- * element (e.g. `<header role="navigation">`) win more than one type at
+ * element (e.g. `<header role="navigation">`) match more than one type at
  * once.
  */
+export declare function findMatchingElements<T extends string>(html: string, matchTypes: (tagName: string, role: string | undefined) => readonly T[]): MatchingElement<T>[];
+/**
+ * Picks the single shallowest (fewest ancestors since `<body>`) match per
+ * type from {@link ./find-shallowest-elements.js | findMatchingElements},
+ * ties broken by document order. Used by
+ * {@link ./cap-content-depth.js | capContentDepth} to locate the one `<main>`
+ * element per page (HTML spec discourages multiple `<main>`s, so shallowest-
+ * wins is semantically correct for that use).
+ *
+ * `depth` is dropped from the returned shape because it's an internal
+ * selection artifact that no consumer of the winner needs.
+ * @param html
+ * @param matchTypes
+ */
 export declare function findShallowestElements<T extends string>(html: string, matchTypes: (tagName: string, role: string | undefined) => readonly T[]): ShallowestElementMatch<T>[];

package/dist/find-shallowest-elements.js

@@ -2,28 +2,32 @@ import { Parser } from 'htmlparser2';
 import { isGenuineClose } from './is-genuine-close.js';
 import { isOpaqueTagName } from './opaque-tags.js';
 /**
- * Shared walk behind {@link ./extract-landmarks.js | extractLandmarks} (which
- * matches four landmark types per element in one pass) and
- * {@link ./cap-content-depth.js | capContentDepth} (which matches a single
- * landmark). Both need the identical "shallowest genuinely-closed match
- * wins" search — same `<body>`-scoping, same opaque-tag skip, same malformed-
- * markup discard via {@link ./is-genuine-close.js | isGenuineClose} — so a
- * fix to one (e.g. the body-scoping edge case already fixed once in
- * `extractLandmarks`) can't silently fail to apply to the other.
+ * The shared HTML walk behind every landmark-scanning use site: collects
+ * every genuinely-closed match for every requested type, without picking a
+ * winner. {@link ./find-shallowest-elements.js | findShallowestElements}
+ * layers shallowest-per-type selection on top of this; extractLandmarks
+ * consumes the full list so downstream data-driven frequency filters can
+ * decide which matches are chrome vs content instead of a hard-coded depth
+ * rule making that call up here.
  *
  * Only the first `<body>` is in scope, and nothing inside an opaque tag
  * (`script`/`style`/`noscript`/`svg`) is searched — see
  * `extractLandmarks`/`capContentDepth`'s own JSDoc for why.
+ *
+ * Results are sorted by `startOffset` ascending (document order of the
+ * opening tag), which is what extractLandmarks needs both for document-order
+ * concatenation and for the outer-before-inner sweep used to filter out
+ * nested landmarks so they aren't double-counted in shell tokens.
  * @param html
  * @param matchTypes Given an element's tag name and `role` attribute (already
  * normalized: an empty/absent `role` arrives as `undefined`), returns every
  * type `T` that element matches. Returning more than one lets a single
- * element (e.g. `<header role="navigation">`) win more than one type at
+ * element (e.g. `<header role="navigation">`) match more than one type at
  * once.
  */
-export function findShallowestElements(html, matchTypes) {
+export function findMatchingElements(html, matchTypes) {
     const stack = [];
-    const candidates = [];
+    const matches = [];
     let opaque = null;
     let bodyDone = false;
     let ignoredBodyOpens = 0;
@@ -83,7 +87,7 @@ export function findShallowestElements(html, matchTypes) {
                 const contentStart = html.indexOf('>', frame.startOffset) + 1;
                 const contentEnd = html.lastIndexOf('<', endOffset - 1);
                 for (const type of frame.matchedTypes) {
-                    candidates.push({
+                    matches.push({
                         type,
                         depth,
                         startOffset: frame.startOffset,
@@ -98,19 +102,35 @@ export function findShallowestElements(html, matchTypes) {
         },
     }, { decodeEntities: false });
     parser.end(html);
+    // Close-tag order is post-order (inner before outer); flip to
+    // startOffset-ascending so callers get document-order-of-opening.
+    matches.sort((a, b) => a.startOffset - b.startOffset);
+    return matches;
+}
+/**
+ * Picks the single shallowest (fewest ancestors since `<body>`) match per
+ * type from {@link ./find-shallowest-elements.js | findMatchingElements},
+ * ties broken by document order. Used by
+ * {@link ./cap-content-depth.js | capContentDepth} to locate the one `<main>`
+ * element per page (HTML spec discourages multiple `<main>`s, so shallowest-
+ * wins is semantically correct for that use).
+ *
+ * `depth` is dropped from the returned shape because it's an internal
+ * selection artifact that no consumer of the winner needs.
+ * @param html
+ * @param matchTypes
+ */
+export function findShallowestElements(html, matchTypes) {
+    const matches = findMatchingElements(html, matchTypes);
     const winners = new Map();
-    for (const candidate of candidates) {
-        const current = winners.get(candidate.type);
+    for (const match of matches) {
+        const current = winners.get(match.type);
         if (current === undefined ||
-            candidate.depth < current.depth ||
-            (candidate.depth === current.depth && candidate.startOffset < current.startOffset)) {
-            winners.set(candidate.type, candidate);
+            match.depth < current.depth ||
+            (match.depth === current.depth && match.startOffset < current.startOffset)) {
+            winners.set(match.type, match);
         }
     }
-    // `depth` (Candidate's own tie-break field) is deliberately not part of
-    // ShallowestElementMatch: it's an internal selection detail, not
-    // something either caller (extractLandmarks, capContentDepth) uses once
-    // the winner is chosen.
     return [...winners.values()].map(({ type, startOffset, endOffset, contentStart, contentEnd }) => ({
         type,
         startOffset,

package/dist/merge-cross-block-clusters.d.ts

@@ -0,0 +1,61 @@
+import type { PerPageLandmarkInstance } from './per-page-landmark-signatures.js';
+/**
+ * One cluster (post-Stage-A) entering cross-block comparison.
+ *
+ * ## Why `memberLandmarkInstances` rather than raw `ExtractLandmarksResult[]`
+ *
+ * Stage B never reads the full `ExtractLandmarksResult` — it only needs each
+ * member page's landmark instances (per-type, per-signature, per-token-set)
+ * for `shellQuorum`'s per-token page-frequency histogram. Callers pre-run
+ * {@link ./per-page-landmark-signatures.js | computePerPageLandmarkInstances}
+ * once at unit creation and hand Stage B the compact instance list instead
+ * of the ~10× larger raw landmark HTML strings. This is the single largest
+ * memory reduction the streaming path relies on: the difference between a
+ * 176k-page corpus fitting in an 8 GB heap and exhausting a 12 GB heap.
+ */
+export type CrossBlockUnit = {
+    readonly key: string;
+    readonly memberTokenSets: readonly ReadonlySet<string>[];
+    readonly memberLandmarkInstances: readonly (readonly PerPageLandmarkInstance[])[];
+};
+/**
+ * Merges cross-block clusters (Stage B) via recursive quorum-core comparison.
+ *
+ * Returns a `Map` from each input unit's `key` to its final root key. Units
+ * not absorbed into any other unit map to themselves.
+ *
+ * Three merge mechanisms run per round, in order:
+ * 1. **Fine stage** — complete-linkage at `CROSS_BLOCK_THRESHOLD` on quorum
+ *    cores, then containment assignment (0.9), then shape-Jaccard (0.9) for
+ *    class-name-only differences.
+ * 2. **L2 stage** (only when fine found nothing) — multiset containment on
+ *    `main`-anchored 2-level shape signatures, with shell corroboration
+ *    (header+nav+footer quorum Jaccard ≥ 0.8) required.
+ *
+ * Rounds continue until neither stage finds anything, or `MAX_ROUNDS` is hit.
+ * Each round re-derives quorum cores from pooled members of merged units.
+ *
+ * Why quorum cores instead of strict intersection or full union:
+ * strict intersection degenerated on real crawl data (89 articles → 1 shared
+ * distinctive token → jaccard 1.0 false merges). Full union was shell-dominated
+ * (298-page avalanche into 4 clusters). Both failure modes are documented in
+ * `@d-zero/page-cluster` source JSDoc; quorum 80% + page-frequency shell
+ * removal was validated on two real crawl corpora.
+ * @param units Post-Stage-A clusters.
+ * @param options Forwarded `similarityThreshold` (defaults to 0.8).
+ * @param options.similarityThreshold
+ * @param options.capMembers
+ */
+export declare function mergeCrossBlockClusters(units: readonly CrossBlockUnit[], options?: {
+    similarityThreshold?: number;
+    /**
+     * Opt-in post-merge cap on a group's retained member count. When
+     * set, each merged group is reservoir-sampled back down to this
+     * cap after every merge so a chain of merges cannot balloon past
+     * the per-unit cap Stage A applied at creation. Callers on the
+     * streaming path pass the same value as `capMembers` used in
+     * Stage A; the in-memory path omits it so validated corpora keep
+     * full-membership merge behavior unchanged.
+     */
+    capMembers?: number;
+}): Map<string, string>;