navu-catalog 4.14.2

package/.cursor/skills/catalog-site-review/SKILL.md

@@ -0,0 +1,715 @@
+---
+name: catalog-site-review
+description: >-
+  Onboard and review product-catalog site pipelines end-to-end: harvest, profile
+  staging specs, decide scope property and manifest properties, implement typed
+  product builder, category resolver when needed, verify three catalog tool
+  definitions and Mongo artifacts, trap facets, and manifest/processing version.
+  Use when adding a new catalog site, onboarding S10* sites, after initial harvest,
+  post-harvest profile, Phase D corrective review,   fixing harvest field typing,
+  checking catalog.toolDefinitions, scope/category gaps, structured measurements,
+  run length parsing, range and compound scalar expansion, semi-structured field parsing,
+  facet canonicalization,
+  facet quality gate, pound-capacity label sets, unit-in-value audit, unit-in-token
+  suffixes, image_url
+  population, Algolia secured-key pagination, facet bucket split, orphan pass,
+  nested category facets, two-pass PDP enrichment (__NEXT_DATA__), PDP 404 exclusion,
+  harvest progress logging for long-running operations,
+  Phase 0 site discovery from homepage URL (Shopify, Algolia, sitemap, __NEXT_DATA__),
+  property redundancy candidates (alias_of / allowlist),
+  or reviewing xtras.catalog staging / catalog.harvestProducts data.
+---
+
+# Catalog site review
+
+Workflow for **product-catalog** site modules on the **manifest-driven engine**. Read `docs/catalog-pipeline-guidelines.md`, [catalog-site-discovery-playbook.md](../../../docs/catalog-site-discovery-playbook.md), and `docs/catalog-search-architecture.md` first.
+
+**Progress checklist:** [onboarding-checklist.md](onboarding-checklist.md) — copy into chat for Phase 0→D tracking.
+
+**Legacy reference:** Pre-rebuild cleaners and EAV patterns live in `src/site-catalogs/` — reuse parser/resolver ideas, not `CatalogCleanProduct` / token-grammar tools.
+
+## When to use
+
+- **New site onboarding** (full Phase 0→D flow)
+- **Homepage URL only** — no vendor docs yet; run Phase 0 discovery before coding
+- After **first harvest** for a new site (Phase B profiling)
+- Before implementing or rewriting a **product builder**
+- When specs appear as strings but should be **numbers/booleans** in `fields`
+- When **Length/Span/Dimensions** labels should promote to structured harvest fields
+- When **ranges or compounds** still have units buried in string values
+- When **string fields embed multiple specs** (voltage + conductor count + size in one value) — semi-structured prose audit
+- When **duplicate unit columns** exist for one measurement (`length_ft` + `length_m`)
+- When **redundant same-grain properties** appear (`brand` / `brand_name`, dual labels with complementary coverage) — review redundancy candidates before allowlisting
+- When **facet values are prose-heavy** or need canonicalization before manifest `facet: "terms"`
+- When **`scope` property** is missing, empty, or high **`Other`** rate
+- When staging has images but **`image_url`** is missing on harvest products
+- When upstream is **Algolia + storefront PDP** (secured keys, pagination cap, facet split, two-pass harvest)
+- When Algolia **`nbHits` ≠ retrieved hits** or category parser returns zero products in a known subtree
+- When PDP fetches return **404** for Algolia-listed SKUs (exclude from catalog, not harvest failure)
+- When reviewing `xtras.catalog.<sitecode>.<catalogname>`, `catalog.harvestProducts`, `catalog.manifests`, **`catalog.openSearchDocuments`**, or `catalog.toolDefinitions`
+- After Phase C implementation, before considering a site merge-ready (**Phase D**)
+
+## Onboarding orchestration
+
+Run phases in order. **Do not skip decision gates.** **Do not skip Phase 0** when upstream is unknown.
+
+| Phase | Goal | Agent must produce |
+|-------|------|-------------------|
+| **0 — Discovery** | Detect catalog upstream from homepage URL | `docs/sites/<site>/RECON.md`, discovery fixtures, classified harvest pattern; **STOP for user review** |
+| **A — Harvest** | Populate staging / harvest rows | Stable `product_id`; sample `data`/`raw`; item count |
+| **B — Profile** | Understand data + **gates** | `spec-profile.json` + **categoryStrategy** + measurement/range/value-shape audits + gate decisions |
+| **C — Build** | Parser, product builder, manifest | Code, tests, version bump, **index document mirror** + index rebuild (skip embeddings while iterating) |
+| **D — Live review** | Audit code + Mongo; fix gaps | Findings report, facet/scope audits, corrective actions, `correctiveReview` in spec-profile |
+
+Standalone (when wired):
+
+```bash
+npm run harvest:vimvigr:v2
+npm run build-index-documents:vimvigr:v2   # Mongo mirror only; no OpenSearch required
+npm run rebuild-index:vimvigr:v2           # mirror + OpenSearch when elastic creds configured
+```
+
+Site harvest CLI: `--harvest-catalog`. Index mirror: `--build-index-documents`. Full rebuild: `--rebuild-catalog-index`. Add `--skip-embeddings` while iterating.
+
+---
+
+## Phase 0 — Site discovery (homepage URL only)
+
+**When:** New site and upstream is unknown, or user provides only a storefront URL.
+
+**Read:** [docs/catalog-site-discovery-playbook.md](../../../docs/catalog-site-discovery-playbook.md) — living registry of probes to run.
+
+### Agent workflow
+
+1. Fetch `robots.txt`, sitemap index, homepage HTML.
+2. Scan for platform fingerprints (Shopify CDN, Algolia scripts, `__NEXT_DATA__`, commercetools, HubSpot, JSON-LD).
+3. Follow nav to a **category/listing** page and one **PDP**.
+4. Run confirmatory probes in parallel:
+   - `GET {origin}/products.json?limit=1` (Shopify)
+   - Extract Algolia app id / secured key from category or SSR JSON; test query with `hitsPerPage: 0`
+   - Parse PDP for `__NEXT_DATA__`, JSON-LD Product, variant family shape
+   - Sample sitemap product URLs for path/locale patterns
+5. Classify **primary upstream pattern** and **enumeration strategy** (see playbook decision table).
+6. Write `docs/sites/<site>/RECON.md` using playbook template; save raw samples to `tests/fixtures/<site>/discovery/`.
+7. Record stub in spec-profile when helpful:
+
+```json
+"discoveryStrategy": {
+  "homepageUrl": "https://example.com",
+  "probesRun": ["robots", "sitemap", "shopify_products_json", "algolia_ssr_key", "next_data_pdp"],
+  "detectedPlatforms": ["algolia", "next_pages_router", "commercetools"],
+  "primaryPattern": "algolia_listing_plus_pdp",
+  "enumeration": "algolia_category_filters",
+  "escalationRequired": false
+}
+```
+
+8. **STOP** — present detection summary and proposed harvest architecture; wait for user approval before Phase A code.
+
+### Pass criteria
+
+| Check | Expected |
+|-------|----------|
+| RECON.md exists | Detection summary + chosen architecture + baseline counts or explicit “unknown count” |
+| Negative signals recorded | e.g. Shopify 404, no Algolia key — avoids re-probing |
+| Sample fixtures saved | At least category + PDP or one API response |
+| Escalation documented | If probes fail, next step named (auth API, feed URL, customer docs) — no HTML-scraper guess |
+
+Only after Phase 0 passes (or user approves escalation) proceed to gates 8–9 and Phase A harvest implementation.
+
+---
+
+## Decision gates (mandatory)
+
+Complete during **Phase B** (decisions) and **Phase C/D** (verification). **Ask the user** when the right choice is not obvious.
+
+### 1. Scope property gate
+
+Product catalogs usually need **one scope property** on the manifest (`scope: true`, e.g. `product_type` or `category`) with **15–25 values** for discovery partitioning.
+
+**Step 1 — Audit upstream taxonomy** (staging `data`, `raw`, URL patterns):
+
+| Source | If present |
+|--------|------------|
+| Breadcrumb / category path | Map level-0 → scope field on `product.fields` |
+| `product_type` or department codes | Map via resolver table |
+| Site nav / sitemap lines | Use as allowed enum values |
+| **None** (flat XML/SKU feed) | Go to Step 2 |
+
+**Step 2 — Can a scope dimension be constructed?**
+
+| Constructible from | Approach |
+|--------------------|----------|
+| Title + URL keywords | `*-category-resolver.ts` + manifest scope enum |
+| Single homogeneous line | Omit scope (`scope: false` on all properties); document in spec-profile |
+| Brand only | **Not** a substitute for scope — keep as separate facet token |
+
+Document in `spec-profile.json`:
+
+```json
+"categoryStrategy": {
+  "upstreamTaxonomy": "none | breadcrumbs | product_type | …",
+  "recommendation": "resolver | copy upstream | omit scope",
+  "fallback": "Other",
+  "scopeProperty": "product_type"
+},
+"imageStrategy": {
+  "upstreamSource": "shopify_images | algolia_thumbnail | api_image_path | none",
+  "availability": "present | absent | partial"
+}
+```
+
+**Step 3 — Implement or ask**
+
+- Feasible resolver → implement in Phase C; register values in manifest scope property
+- **`Other` > ~15%** → sample titles; propose keyword additions; ask user before new enum values
+- Truly homogeneous → record `"scopeProperty": null` and omit scope flag on manifest
+
+**Verify after index rebuild** — [Scope enum distribution audit](#4-scope-enum-distribution-audit-phase-d).
+
+### 2. Product / variant grain gate
+
+**One OpenSearch document per product** always. No `recordModel` / flat vs `parent_child` profile flag.
+
+| Data shape | Harvest shape |
+|------------|---------------|
+| One purchasable SKU per product row upstream | `variants[]` length 1 |
+| Product family + multiple SKUs (Shopify) | `variants[]` length 2+ |
+
+Query tool supports `type=product` (default) and `type=variant` + `product_id` for drill-down — same for all sites.
+
+### 3. Tool definition gate (after index rebuild)
+
+```javascript
+db.getCollection("catalog.toolDefinitions").findOne(
+  { siteId: ObjectId("<siteId>"), catalogName: "products" },
+  { tools: 1, lastUpdated: 1 },
+)
+```
+
+**Pass criteria:**
+
+| Check | Expected |
+|-------|----------|
+| Tool count | 3: `{base}-discovery`, `{base}-query`, `{base}-get-details` |
+| `toolKind` | `discovery`, `query`, `get_details` |
+| `ready` | `true` |
+| Filters | `{ token, op?, value }` in tool descriptions / skill instructions — not legacy token grammar |
+
+If checks fail: fix manifest/profile, bump processing version, rebuild index.
+
+### 4. Scope enum distribution audit (Phase D)
+
+**Mandatory** when a scope property is configured.
+
+**Agent workflow:**
+
+1. Load **declared enum** from manifest / product builder resolver
+2. Load **discovery facets** from a smoke `discovery` call or manifest docs
+3. Aggregate **`catalog.harvestProducts`** on scope field (see guidelines)
+4. Table: scope value | count | % | in enum?
+5. Compute `otherRate`, `emptyScopeCount`, `deadEnumValues`, `undeclaredValues`
+6. Flag anomalies (guidelines threshold table)
+7. Record in `spec-profile.json` → `scopeDistribution`
+
+**Pass targets:** `emptyScopeCount === 0`, `otherRate ≤ ~15%`, no undeclared values, ≤ ~25 scope values.
+
+### 5. Structured measurement gate
+
+Same as legacy pipeline — classify high-count dimension/length/mass labels in Phase B; promote to typed **`fields`** in Phase C **before** string normalizer. Reference `src/site-catalogs/cisco-eagle/`.
+
+### 6. Range and compound scalar gate
+
+Expand ranges and labeled compounds into **multiple numeric field names** — not single strings. Reference `src/site-catalogs/cisco-eagle/cisco-eagle-specification-scalar-parser.ts` and `src/site-catalogs/gantrade/gantrade-specification-scalar-parser.ts`.
+
+**Scalar parser checklist (Phase C):**
+
+| Pattern | Must emit | Must not leave as string |
+|---------|-----------|-------------------------|
+| `545-565` | `{base}_min`, `{base}_max` | `{base}: "545-565"` |
+| `1100-1700 (52°C)` | min/max only | base + min/max duplicate |
+| `≤5`, `<500`, `1 max` | `{base}_max` or `{base}_min` | `"1 max"`, `"<500"` |
+| `0.05% max` on bare column | `{base}_max_pct` | `"0.05% max"` |
+| `80±1` | `{base}: 80` | `"80±1"` |
+| Plain `315` on interval column | `{base}_min` = `{base}_max` = 315 | `{base}: 315` alone |
+
+**Mixed scalar / interval-range alignment (mandatory when a label mixes shapes):**
+
+1. Phase B: fingerprint each numeric label — plain number vs interval (`545-565`) vs bound (`1 max`, `<500`) vs tolerance (`80±1`).
+2. Classify columns: **interval-range** (dash/`to` expands to min+max), **bound-only**, **tolerance-only**, or **scalar-only**.
+3. Phase C: two-pass or pre-scan — detect `intervalRangeColumns` / `intervalRangeLabels` from eligible upstream rows before final parse.
+4. Plain numerics **and tolerance nominals** on interval-range columns → `{base}_min` = `{base}_max`.
+5. Manifest: declare `{base}_min` / `{base}_max` for interval columns (`facet: "range"`); omit `{base}` when min/max pair exists. Declare one-sided `{base}_max` / `{base}_min` tokens — do **not** skip all `_min`/`_max` suffixes.
+6. Phase D: verify no `{base}` alongside `{base}_min`/`{base}_max`; no leftover `"±"` strings on numeric columns.
+
+**Unit-in-token (HubDB / fixed schema):** when column names encode units, use column name as field token; no `{token}_unit` fields. See gate 7 for PIM/CVL sites.
+
+Record in `spec-profile.json`:
+
+```json
+"intervalRangeColumns": ["hydroxyl_value_mgkohg", "molecular_weight_gmol", "ph"],
+"boundColumns": ["color_gardner", "moisture", "water_ppm"],
+"toleranceColumns": ["isomer_ratio_24_tdi", "viscosity_mpa_s25_c"],
+"unitInToken": true,
+"scalarRangeAlignment": {
+  "pointValuesOnIntervalColumns": "min_max_degenerate",
+  "toleranceValues": "nominal_only",
+  "oneSidedBounds": "single_bound_field_only"
+}
+```
+
+Phase D: unit-in-value audit on `product.fields` / `variants[].fields`.
+
+### 6b. Semi-structured field gate (Phase B)
+
+**Mandatory** when profiling string fields from tables, PDF extraction, or matrix layouts (not only dash-range compounds).
+
+String values that **look like one attribute but encode several filterable facts** should not ship as a finished `facet: "terms"` axis without review.
+
+| Pattern | Signal | Typical action |
+|---------|--------|----------------|
+| Multi-fact prose blob | Commas/newlines; embedded `120V`, `3 Conductor`, `11mm`, AWG sizes | Parse into typed fields or **defer** with `semiStructuredFieldAudits` |
+| Duplicate unit columns | `length_ft` + `length_m`, `weight_lb` + `weight_kg` on same row | One canonical range facet; trap or demote the duplicate |
+| Product config on every variant | Identical string across all SKUs in a product | Move to **product** `fields`; parse once |
+| Display range string + min/max | `cord_od` prose when `cord_od_min`/`cord_od_max` exist | Trap display token |
+| Table row label | Row header verbatim; column headers already structured | Site-specific row-label parser or defer |
+| Geometry / slug fallback | Token looks auto-generated; value is prose or mis-typed | Trap or fix upstream extraction |
+
+**Phase B:** run semi-structured audit on top string fields; record `semiStructuredFieldAudits` in spec-profile (see guidelines).
+
+**Phase C:** implement parser expansion for **parse now** rows; do not block merge on **defer** items if documented in `correctiveReview`.
+
+**Phase D:** verify deferred items remain listed; confirm duplicate unit columns are not both advertised as range facets.
+
+Example defer (TPC Wire): `din_configuration` on DIN cordset pages — column splits (`connector_head`, lengths) are correct; row label prose deferred until Vol. 9 or a dedicated parser pass.
+
+### 7. Unit-in-token gate (Phase B/C)
+
+**Mandatory** for PIM/PDP catalogs and any site with structured unit metadata.
+
+| Check | Pass criteria |
+|-------|---------------|
+| PIM `features[].unit` | Parser appends normalized suffix when token lacks one (`tag_height` + `mm` → `tag_height_mm`) |
+| Commerce attributes | Read paired CVL fields (`unitOfWeight`, `unitOfDimension`); emit `weight_g`/`weight_kg`, `length_mm`, etc. |
+| Bare attributes | **Omit** numeric harvest fields when unit metadata missing — do not guess |
+| No unit metadata on site | **Ask user** to confirm inferred units before implementing suffixes |
+| Already suffixed ids | No double suffix (`pressure_bar`, `cts_weight_g`) |
+| Manifest bump | Bump `MANIFEST_VERSION` / processing version when token renames affect filters |
+
+Record in `spec-profile.json`:
+
+```json
+"unitStrategy": {
+  "pimFeatureUnitField": "features[].unit",
+  "commerceWeightUnitField": "attributes.unitOfWeight.values[0].key",
+  "commerceDimensionUnitField": "attributes.unitOfDimension.values[0].key",
+  "omitBareWhenUnitMissing": true,
+  "confirmedInferredUnits": false
+}
+```
+
+Reference: `src/site-catalogs/atlascopco/atlascopco-unit-support.ts`.
+
+### 8. Facet quality gate (Phase D)
+
+**Mandatory** before merge when manifest has `facet: "terms"` properties.
+
+For each terms facet: coverage %, raw distinct, canonical distinct. Classify: **keep** | **canonicalize** | **demote to `facet: "none"`** | **defer**. Ask user when trade-offs are unclear. Record `facetPriorities` in spec-profile.
+
+### 9. Algolia enumeration gate (Phase A — when upstream is Algolia)
+
+Complete during Phase 0 recon and verify on first harvest. Read `docs/catalog-pipeline-guidelines.md` → **Algolia storefront catalogs**.
+
+| Check | Pass criteria |
+|-------|---------------|
+| Secured key bootstrapped | Fresh key from category page / `__NEXT_DATA__` each harvest |
+| Base filter documented | Copied from live category SSR in `docs/sites/<site>/RECON.md` |
+| Live count baseline | `countHits(filter)` per scope — validator uses live counts, not prompt totals |
+| Pagination | `hitsPerPage` ≤ secured-key cap (often **500** when cap is **1,000**) — never `hitsPerPage === cap` with `nbHits > cap` |
+| Large subtrees | Facet bucket split + dedupe by `objectID`; orphan pass when bucket sum < `nbHits` |
+| Category parser | Matches **real hit JSON** (flat vs nested locale facets) |
+| Retrieved vs expected | `dedupe(hits).length === countHits(filter)` for each enumeration filter |
+
+Record in `spec-profile.json`:
+
+```json
+"algoliaStrategy": {
+  "indexName": "prd_products_GB10",
+  "baseFilter": "(NOT materialGroup:03 …) AND audience:B2C",
+  "categoryFacetShape": "nested | flat",
+  "maxHitsPerPage": 500,
+  "securedKeyCap": 1000,
+  "enumeration": "single_filter | facet_split | adaptive_split",
+  "orphanPass": true,
+  "liveBaselineCounts": { "products": 568, "spareParts": 7426 }
+}
+```
+
+Reference: `src/site-catalogs/atlascopco/`, `src/site-catalogs/hamilton/`.
+
+### 10. PDP enrichment gate (Phase A/C — when listing ≠ harvest product)
+
+When Algolia hits are SKU/material rows but **`CatalogHarvestProduct`** grain is a **variant family** or PIM specs require PDP HTML:
+
+| Decision | Guidance |
+|----------|----------|
+| Two-pass harvest | Pass 1 `pdpEnrichment: false` (counts + categories); Pass 2 `pdpEnrichment: true` |
+| Family key | From upstream (e.g. `hierarchy.model.id`) — **never** title-based grouping |
+| Payload cache | Write raw PDP JSON to `tests/fixtures/<site>/raw/` for resume |
+| Progress | Log every N items via `CatalogProgressLogger`; heartbeat when stalled; see **gate 11** |
+| `parsedItemCount` vs `processedCount` | Family dedup — `parsedItemCount` ≪ `processedCount` is normal |
+| **404** | Info log “excluded from catalog”; adjust expected validation counts; **not** `fetchFailures` |
+| Timeouts / 5xx | WARN + `fetchFailures`; long runs need sleep prevention (`caffeinate -i`) |
+
+Document in `spec-profile.json`:
+
+```json
+"pdpStrategy": {
+  "sourceFidelity": "next_data",
+  "passes": 2,
+  "familyKey": "pimProduct.hierarchy.model.id",
+  "cacheDir": "tests/fixtures/<site>/raw",
+  "notFoundPolicy": "exclude_from_catalog",
+  "expectedDurationNote": "hours for full SKU enumeration"
+}
+```
+
+### 11. Harvest progress logging gate (Phase A/C — long-running harvests)
+
+**Mandatory** when a harvest performs many sequential HTTP requests or is expected to run **> ~1 minute** (SSR pagination, PDP/detail enrichment, Algolia bucket splits, disk staging import).
+
+| Requirement | Guidance |
+|-------------|----------|
+| Shared utilities | `CatalogProgressLogger` + `CatalogProgressHeartbeat` in `src/catalog-manager/catalog-progress-logger.ts` |
+| Phase boundaries | Log **start** and **complete** for each major phase (listing fetch, grouping, detail/PDP enrichment, manifest persist) |
+| Count progress | Log every **N** items: `DEFAULT_INTERVAL` (50) for hundreds; `BULK_HARVEST_INTERVAL` (500) for thousands |
+| Progress fields | Include `processedCount`, `totalCount`, `remainingCount`, `percentComplete` via `CatalogProgressLogger.progressFields()` |
+| Heartbeat | `CatalogProgressHeartbeat` (30s) when counts stall but work continues (resume/skip paths, slow upstream) |
+| Domain counters | Add site-specific tallies (`detailFetchedCount`, `inScopeCount`, `fetchFailures`, `pdpNotFoundProducts`, …) |
+| Silence is a bug | If a phase can run minutes with no logs, add pagination or enrichment progress before merge |
+
+Record in `spec-profile.json` when duration is non-obvious:
+
+```json
+"harvestProgressStrategy": {
+  "expectedDurationNote": "~5 min listing + detail for 691 configs",
+  "progressInterval": 50,
+  "phases": ["listing_pages", "config_grouping", "detail_enrichment"]
+}
+```
+
+Reference: `src/site-catalogs/customtruck/customtruck-harvester.ts`, `src/site-catalogs/atlascopco/atlascopco-harvester.ts`, `src/site-catalogs/hamilton/hamilton-harvester.ts`.
+
+---
+
+## Phase A — Skeleton / harvest
+
+After Phase 0 approval (or when upstream is already documented):
+
+1. Implement upstream client per RECON chosen architecture.
+2. Save one raw upstream sample + parsed staging sample under `tests/fixtures/<site>/`.
+3. For **Algolia + PDP** sites: confirm secured-key pagination on largest filter before full harvester (gate 8).
+4. **Wire progress logging** (gate 11) for any multi-page listing or per-item detail/PDP loop before first full harvest.
+
+---
+
+## Four phases
+
+| Phase | Goal | Output |
+|-------|------|--------|
+| **0 — Discovery** | Detect upstream from homepage URL | `RECON.md`, discovery fixtures, classified pattern |
+| **A — Skeleton** | Harvest + pass-through or minimal builder | Staging / harvest rows populated |
+| **B — Profile** | Classify every spec label | `tests/fixtures/<site>/spec-profile.json` |
+| **C — Typed build** | Product builder + manifest + tests | Typed `fields`, manifest validated, index rebuild |
+| **D — Data review** | Audit + corrective actions | `correctiveReview`, before/after metrics |
+
+Do not skip Phase 0 (when upstream unknown), Phase B, or Phase D.
+
+---
+
+## Phase B — Profile staging data
+
+### 1. Confirm harvest
+
+Check Mongo (`navu-catalog-standalone` or configured db):
+
+```javascript
+db.getCollection("xtras.catalog.<sitecode>.<catalogname>").countDocuments()
+// and/or
+db.getCollection("catalog.harvestProducts").countDocuments({ siteId: ObjectId("<siteId>"), catalogName: "products" })
+```
+
+### 2. Aggregate specification labels
+
+```javascript
+db.getCollection("xtras.catalog.<sitecode>.<catalogname>").aggregate([
+  { $project: { specs: { $objectToArray: "$data.specifications" } } },
+  { $unwind: "$specs" },
+  { $group: { _id: "$specs.k", count: { $sum: 1 }, samples: { $addToSet: "$specs.v" } } },
+  { $project: { count: 1, samples: { $slice: ["$samples", 8] } } },
+  { $sort: { count: -1 } },
+  { $limit: 100 },
+])
+```
+
+### 3. Classify each label
+
+| Class | Harvest / manifest action |
+|-------|---------------------------|
+| **numeric_range** | `{base}_min` + `{base}_max`; point/tolerance on mixed column → min=max; manifest min/max pair |
+| **numeric_bound** | `{base}_max`, `{base}_min`, or `{base}_max_pct` — inequalities, `N max`/`N min`; do **not** min=max |
+| **numeric_tolerance** | `{base}` = nominal from `N±M`; drop ± band |
+| **numeric_compound** | per-axis field pair |
+| **numeric_scalar** | single number field |
+| **numeric_structured** | promoted `length_ft`, `width_in`, … |
+| **boolean** | boolean field |
+| **categorical_facet** | keyword + `facet: "terms"` |
+| **categorical_trap** | keyword + `facet: "none"` |
+| **categorical_prose** | narrative or `facet: "none"` |
+| **text** | narrative only |
+
+Run **value-shape audit** — per label/column count plain vs range vs bound vs tolerance vs prose (see guidelines fingerprint table). Record `valueShapeAudits`, `labelSetAudits`, `scalarExpansion`, `intervalRangeColumns`, `boundColumns` in spec-profile.
+
+### 4. Draft manifest property list
+
+Map each filterable axis to a `CatalogProperty` row (token, field_path, grain, facet, volatile). Validate with `CatalogManifestValidation`.
+
+---
+
+## Phase C — Implement product builder + manifest
+
+### Checklist
+
+1. **Structured measurements** on raw specs before normalizer
+2. **Scalar parser** — ranges, bounds, tolerance nominals, compounds (see gate 6 checklist)
+3. **Interval-range pre-scan** — detect mixed-shape columns; min=max for points and tolerance nominals
+4. **Product builder** — `CatalogHarvestProduct` with typed `fields` / `variants[].fields`
+4. **`narrative`** — rich prose on harvest row; copied to indexed doc **`narrative`** for BM25; no price/stock/facet duplicates
+5. **`image_url`** — product hero on `product.fields.image_url` / `CatalogHarvestProduct.image_url`; variant image on `variants[].fields.image_url` when upstream provides URLs (manifest: `filterable: false`, `facet: "none"`)
+6. **Site-specific variant fields** — e.g. Shopify `shopify_variant_id` in `variants[].fields` when upstream provides it; declare in site manifest (`filterable: false`, `facet: "none"`)
+7. **`CatalogManifest`** — scope property, facet flags, volatile on price/availability
+8. **Tests** — product builder + manifest validation + `live-sample.json`
+9. **Version bump** on profile / manifest
+10. **Index document build** — `--build-index-documents`; verify mirror fields; `--skip-embeddings` while iterating
+
+---
+
+## Phase D — Audit, corrective actions, verification
+
+### D1 findings areas
+
+| Area | Common issues |
+|------|----------------|
+| Scope | High `Other`, empty scope, dead enum values |
+| Tools | Missing tool, `ready: false` |
+| Manifest vs data | Missing fields, wrong types, string numerics |
+| Facets | Prose facets left as `facet: "terms"` |
+| Range/compound | Unit still in string values; **mixed scalar/range misalignment**; string `"80±1"` / `"1 max"` / range strings alongside min/max |
+| Semi-structured | Multi-fact prose blobs still on `facet: "terms"`; duplicate unit facets; product-level config duplicated on every variant |
+| Scalar typing | Numeric manifest properties with `facet: "terms"` instead of `range` |
+| Images | Missing product `image_url` or variant `fields.image_url` when upstream has images |
+| Narrative | Price/stock leaked into narrative |
+| **Harvest shape** | Opaque harvest tokens (`a`, `b`, `product`); bare unit tokens (`weight`, `length` without `_lb`/`_in`); flat-SKU duplicate weight; spec lines duplicated in narrative |
+| **Property redundancy** | Same-grain equivalent tokens with zero co-occurrence; dual Brand facets; leftover spec-table duplicates of product-grain fields |
+
+Present prioritized findings table before D2.
+
+### D1.5 — Sample product review (mandatory)
+
+Before D2 coding, manually review **3–5 harvest products** across distinct `product_type` lines against `docs/catalog-pipeline-guidelines.md` product-builder rules:
+
+1. Pick one dense-spec SKU, one sparse-spec SKU, and one with cryptic upstream labels.
+2. Check each for: unit-in-token suffixes, typed numerics, opaque tokens, product/variant dedup on flat SKUs, narrative dedup, semi-structured leftovers.
+3. Record gaps in findings table and `spec-profile.json` (`unitStrategy`, `semiStructuredFieldAudits`, deferred items).
+4. **Do not call D complete** when aggregate audit passes but sample products fail guideline shape.
+
+### D1 harvest-shape audit (automate in `scripts/audit-<site>-phase-d.mjs`)
+
+| Check | Fail when |
+|-------|-----------|
+| Opaque variant tokens | `variants[].fields` contains `a`, `b`, `product`, or single-letter keys |
+| Bare unit tokens | `weight`, `item_weight`, or dimension tokens without `_lb` / `_in` / `_inches` suffix |
+| Flat-SKU weight dup | `product.fields.weight_lb` and `variants[].fields.item_weight` both present |
+| Narrative spec dup | Narrative contains `Label: value` lines for specs already mapped to `fields` |
+| Unit strategy documented | `spec-profile.json` missing `unitStrategy` when site emits mass/dimension fields |
+| Property redundancy | `npm run audit:catalog-redundancy:v2` (or site phase-d audit) shows unreviewed candidates; two live Brand facets; variant brand leftover when product `brand_name` covers all SKUs |
+
+### D2 typical fixes
+
+| Type | Where |
+|------|--------|
+| Scope resolver keywords | `*-category-resolver.ts` |
+| Scalar/range parsing | `*-specification-scalar-parser.ts`, `*-interval-range-*-detector.ts` |
+| Label aliases + unit suffixes | `*-specification-field-mapper.ts` |
+| Interval-range alignment | row parser / product builder min=max; manifest property builder |
+| Facet canonicalization | `*-facet-value-support.ts` |
+| Demote trap facets | manifest `facet: "none"` |
+| Exclude redundant spec labels | product builder skip list; narrative trim |
+| Promote flat-SKU duplicates | product builder: weight/spec once at product grain |
+| Same-grain property merge | Path A unit-only auto-merge; Path B allowlist only; scope-qualifier crossing hard-stopped unless override |
+| Cross-grain / leftover duplicate | Prefer harvest skip + `TRAP_TOKENS` (e.g. variant Brand when product `brand_name` exists) over allowlist |
+| Opt out of unit auto-merge | Denylist the pair in `resolveRedundancyConfig` |
+
+Bump version → rebuild index. Do **not** allowlist non-unit pairs with conflicting values without inspection. Never allowlist across scope qualifiers (`capacity_tons` / `ring_clutch_tonnage_*`) without `qualifierCrossingOverrides` + justification.
+
+### D3 verification
+
+Re-run affected audits; write `correctiveReview`, `scopeDistribution`, `facetPriorities` to spec-profile.
+
+### Audit queries (harvest products)
+
+```javascript
+const siteId = ObjectId("<siteId>");
+
+// Scope distribution (adjust field name)
+db.getCollection("catalog.harvestProducts").aggregate([
+  { $match: { siteId, catalogName: "products" } },
+  { $group: { _id: "$product.fields.product_type", count: { $sum: 1 } } },
+  { $sort: { count: -1 } },
+])
+
+// String values that look numeric on manifest numeric tokens — sample review
+db.getCollection("catalog.harvestProducts").aggregate([
+  { $match: { siteId } },
+  { $project: { product: 1 } },
+  { $limit: 50 },
+])
+```
+
+---
+
+## Verify (after Phase C or D)
+
+```javascript
+db.getCollection("catalog.toolDefinitions").findOne(
+  { siteId: ObjectId("<siteId>"), catalogName: "products" },
+  { tools: 1 },
+)
+
+db.getCollection("catalog.manifests").findOne(
+  { siteId: ObjectId("<siteId>"), catalogName: "products" },
+)
+
+db.getCollection("catalog.harvestProducts").findOne(
+  { siteId: ObjectId("<siteId>"), catalogName: "products" },
+)
+```
+
+Smoke path: **discovery** (facets) → **query** (product rows + `matching_variants`) → **get-details** (`product_ids`).
+
+---
+
+## Prompt templates
+
+### Phase 0 — discovery from homepage URL
+
+```
+Discover how to harvest the product catalog for site <SITE_CODE> starting from homepage <URL>.
+
+Follow Phase 0 in .cursor/skills/catalog-site-review/SKILL.md and docs/catalog-site-discovery-playbook.md.
+
+Run automated probes (robots, sitemap, Shopify /products.json, Algolia fingerprints, __NEXT_DATA__, JSON-LD, category + PDP samples).
+Classify upstream pattern and enumeration strategy.
+Write docs/sites/<site>/RECON.md and save fixtures under tests/fixtures/<site>/discovery/.
+
+STOP after RECON — present findings and proposed harvest architecture. Do not write harvester code until I approve.
+```
+
+### Full new-site onboarding
+
+```
+Onboard catalog site <SITE_CODE> in navu-catalog (manifest-driven engine).
+
+Follow .cursor/skills/catalog-site-review/SKILL.md, docs/catalog-pipeline-guidelines.md, docs/catalog-site-discovery-playbook.md, and docs/catalog-search-architecture.md.
+Use onboarding-checklist.md for progress.
+
+0. Phase 0: discovery from homepage URL if upstream unknown; RECON.md + fixtures; stop for review.
+1. Phase A: confirm harvest / catalog.harvestProducts populated.
+2. Phase B: profile specs; gates (scope property, variant grain); draft CatalogManifest; write spec-profile.json.
+3. Phase C: product builder, manifest, tests; rebuild index (skip embeddings until stable).
+4. Phase D: D1 findings → D2 fixes → D3 verify spec-profile.
+
+Three AI tools per catalog: discovery, query, get-details. Filters use { token, op?, value }.
+Do not use legacy recordModel, EAV attributes, or token-grammar tools.
+
+If Algolia + PDP: secured-key pagination check, two-pass harvest when needed, 404 → exclude from catalog.
+```
+
+### After Phase A harvest (profiling)
+
+```
+Profile staging specifications for site <SITE_CODE>.
+
+1. Read docs/catalog-pipeline-guidelines.md Phase B.
+2. Run Mongo aggregations on xtras.catalog.<sitecode>.<catalogname>.
+3. Classify labels; draft CatalogProperty list.
+4. Write tests/fixtures/<site>/spec-profile.json (categoryStrategy, scopeProperty recommendation).
+5. Run scope and facet gates; ask user to confirm scope enum if resolver needed.
+
+Profile only — no product builder changes unless I ask.
+```
+
+### Phase D — full catalog audit
+
+```
+Run Phase D for site <SITE_CODE>.
+
+D1: scope distribution, tool definitions (3 tools), manifest vs harvestProducts typing, facet quality, range/compound strings, semi-structured field audit, images, narrative leaks, harvest-shape audit (opaque tokens, bare unit tokens, flat-SKU dupes, narrative spec dup), property redundancy candidates (npm run audit:catalog-redundancy:v2 → tests/fixtures/<site>/redundancy-candidates.json). Findings table first.
+D1.5: manually review 3–5 sample harvest products across product lines against product-builder rules before coding fixes.
+D2: approved fixes (non-unit allowlist merges or harvest-trap leftovers; unit-only pairs already auto-merge), version bump, rebuild index.
+D3: correctiveReview + metrics in spec-profile.json; re-run harvest-shape + redundancy audit until sample products pass.
+```
+
+---
+
+## Reference implementations
+
+| Pattern | Location |
+|---------|----------|
+| Phase 0 discovery playbook | [docs/catalog-site-discovery-playbook.md](../../../docs/catalog-site-discovery-playbook.md) |
+| Phase 0 RECON example | [docs/sites/atlascopco/RECON.md](../../../docs/sites/atlascopco/RECON.md) |
+| HubDB table harvest + unit-in-token | `src/site-catalogs/gantrade/` — [docs/sites/gantrade/RECON.md](../../../docs/sites/gantrade/RECON.md) |
+| Algolia flat feed | `src/site-catalogs/hamilton/` |
+| Algolia + PDP + facet split | `src/site-catalogs/atlascopco/` |
+| Scalar/range/compound (label-based) | `src/site-catalogs/cisco-eagle/` |
+| Mixed scalar/interval-range alignment | `gantrade-interval-range-column-detector.ts`, `cisco-eagle-interval-range-label-detector.ts` |
+| SSR listing + detail enrichment progress | `src/site-catalogs/customtruck/` |
+| Category resolver | `src/site-catalogs/hamilton/`, `src/site-catalogs/cisco-eagle/` |
+| Spec normalizer | `src/site-catalogs/catalog-specification-value-normalizer.ts` |
+| Engine (mapping + derivation) | `src/manifest/`, `src/catalog-index/catalog-product-derivation.ts` |
+| Property redundancy merger | `src/manifest/catalog-property-redundancy-merger.ts`, `catalog-property-alias-value-converter.ts`, `SiteCatalogModuleRegistry.resolveRedundancyConfig`, `npm run audit:catalog-redundancy:v2` |
+| Demo manifest fixture | `tests/fixtures/new-model/demo-catalog.json` |
+
+---
+
+## Common mistakes
+
+- Emitting EAV `{ name, value, value_num }` instead of named **`fields`**
+- Omitting **`CatalogManifest`** or drifting mapping from manifest
+- **`scope: true`** on more than one property
+- **`facet: "terms"`** on price or prose specs
+- Per-variant **embeddings** (one narrative per product only)
+- Leaving ranges as single strings (`400-4,500 lb.`)
+- **Shipping multi-fact prose blobs** (`din_configuration`, `configuration`) as finished facets without Phase B audit
+- **Duplicate unit facets** — `length_ft` and `length_m` both `facet: "range"` for the same measurement
+- **Mixed scalar/range misalignment** — some products on `{base}`, others on `{base}_min`/`{base}_max`; manifest declares `{base}` but skips `_min`/`_max`
+- Leaving **bounds** as strings (`"1 max"`, `"<500"`) or **tolerance** as strings (`"80±1"`) when nominal/bound parsing applies
+- Keeping **base range string** alongside `{base}_min`/`{base}_max` (e.g. `"20000-25000 (25℃)"` after expansion)
+- **`facet: "terms"`** on numeric harvest fields — use `facet: "range"` for numbers
+- Emitting **`{token}_unit`** when unit is already in the field token (HubDB column-name sites)
+- **`category: []`** or empty scope on a diverse catalog without documenting omission
+- Expecting legacy **`find-products`** monolith tool instead of three tools
+- Quoting **`price_indicative`** as live price (use lookup-products when enabled)
+- Skipping Phase D facet audit on large catalogs
+- Skipping Phase D **property redundancy** review (`redundancy-candidates.json`) or allowlisting conflicting-value non-unit pairs without inspection
+- Expecting dual-unit pairs (`length_in` / `length_ft`) to stay as two live facets — they auto-merge to the higher-coverage unit unless denylisted
+- Using legacy EAV cleaners directly without mapping to `CatalogHarvestProduct`
+- **`hitsPerPage: 1000`** with secured Algolia key capped at 1,000 retrievable hits (`nbPages: 1`, silent under-harvest)
+- Parsing **flat** category facet keys when hits use **nested** locale objects (or vice versa)
+- Treating PDP **404** as harvest failure — Algolia-listed SKUs may have no live page; exclude and adjust expected counts
+- Hardcoding Algolia **`nbHits`** instead of live `countHits()` at harvest start
+- Inferring variant **family** from product title instead of upstream family id
+- Failing on **`imageUrl: null`** — coerce null to missing before string checks
+- **Skipping Phase 0** and guessing HTML scrapers when Shopify/Algolia/public APIs were not probed
+- Treating sitemap URL count as catalog scope without locale/market filters (Algolia may differ)
+- **Long harvest with no progress logs** — multi-minute SSR pagination or detail enrichment must log every N items + phase boundaries (gate 11)