cornhsu-polymigrate 2.2.0 → 2.3.0

package/README.md

@@ -13,7 +13,7 @@ PolyMigrate turns legacy dynamic sites (old PHP sites and the like) into clean,
 Markdown — and it treats multilingual content as a first-class concern, not an afterthought.
 Config-driven, fully offline-rerunnable, built on .NET.
 
-**Status: 2.2.** The extraction pipeline, pairing, verification, thumbnails and orphan-page
+**Status: 2.3.** The extraction pipeline, pairing, verification, thumbnails and orphan-page
 recovery are complete and validated against a real full-site migration (see below). The **CLI
 surface and the Phase output contracts remain stable** (unchanged since 1.0): new features bump
 the minor version, fixes bump the patch. 2.0 was an engineering release — it narrowed the
@@ -21,8 +21,9 @@ the minor version, fixes bump the patch. 2.0 was an engineering release — it n
 config fields; 2.1 fixed up the release pipeline (arm64 npm packages, symbol packages).
 2.2 closed a blind spot in `verify` — pages nothing links to were reported as fine, because a
 link-following audit never reaches them — and added `slugs` for keeping up with a source site
-that is still being updated. **Upgrading to 2.2 can turn a passing `verify` into exit 1**;
-see the [CHANGELOG](CHANGELOG.md) for that and the other migration notes.
+that is still being updated. **Upgrading to 2.2 can turn a passing `verify` into exit 1**.
+2.3 reads the source site's own `hreflang` declarations (see below) and reports them in a new
+`hreflang_map.csv`. See the [CHANGELOG](CHANGELOG.md) for the migration notes.
 
 ## Why
 
@@ -33,14 +34,48 @@ the pairing to you. PolyMigrate:
 
 - **pairs automatically** where filenames are symmetric (`/ch/news/x` ↔ `/en/news/x` share a
   `translation_key`),
-- **suggests pairs heuristically** where they are not — shared photo albums, normalized dates
-  hidden in slugs (`20240121` vs `01212024`), title similarity,
+- **reads the site's own `hreflang` declarations** — `<link rel="alternate" hreflang>` is the one
+  pairing signal the site's authors *stated* rather than something a tool inferred, so it is
+  tried first and can pair across sections (`/ch/news/` ↔ `/en/press/`),
+- **suggests pairs heuristically** where neither applies — slugs that differ only in separators
+  (`2025-light-offering` vs `2025_light_offering`), shared photo albums, normalized dates hidden
+  in slugs (`20240121` vs `01212024`), title similarity,
 - **honestly reports what it cannot pair**, producing a review-ready gap inventory instead of
   guessing wrong.
 
+Note what PolyMigrate deliberately does *not* do: match language versions by URL string
+similarity. `/products/`, `/produkte/` and `/產品/` have a similarity of zero and are the same
+page; symmetric-path pairing is an exact match after the language prefix is stripped, never a
+fuzzy one.
+
 Languages are not limited to two: declare any number in `lang_map` and every output
 (frontmatter, inventories, pairing) expands accordingly. All locale output is standard BCP-47.
 
+### About `hreflang`
+
+Legacy sites are exactly the sites whose `hreflang` is missing or wrong — usually bolted on by
+an SEO contractor years later. So PolyMigrate treats it as strong evidence, not as gospel.
+A declaration is only used for pairing if it:
+
+- isn't `x-default` (that's "where to go when nothing matches", not a language version),
+- doesn't point at the page itself,
+- resolves to a page actually in your mirror — **same host included**, since plenty of sites put
+  the English version on `en.example.org`, where a path-only comparison would make a page its
+  own translation,
+- and **isn't one of many same-language pages claiming the same target**. This is the classic
+  failure: one `<link>` pasted into the site-wide template, so every Chinese page declares the
+  English homepage as its English version. One page cannot be the translation of three others,
+  so all of those claims are dropped rather than turned into a confident-looking wrong pair.
+
+Everything declared — including the rejected ones — is written to `hreflang_map.csv` with
+`in_mirror` / `reciprocal` / `usable` and a `reject_reason` naming which rule it failed, because
+"can I trust this site's hreflang?" is a question you want answered with data, not a total.
+
+Pairs are still only ever *suggested*: `hreflang` raises the evidence quality, it does not
+switch the tool into merging pages on its own. That is a deliberate call, not an unfinished
+one — the measurement behind it, and what would have to change to revisit it, are recorded in
+[docs/hreflang_量測與決策.md](docs/hreflang_量測與決策.md).
+
 ## Battle-tested defaults
 
 The extraction pipeline bakes in fixes for real problems found during a real migration —
@@ -55,6 +90,7 @@ things generic tools and LLM extractors silently get wrong:
 | `<title>` polluted with dates and site name | body-first title extraction + configurable cleanup |
 | Mixed date formats (`YYYYMMDD` / `MMDDYYYY` / `DDMMYYYY`) | all recognized and normalized |
 | Old articles removed from indexes but still served | orphan probing (per-day URL candidates + suffix variants) |
+| `hreflang` pointing at dead URLs, other hosts, or itself | validated against the mirror; unusable ones recorded, not applied |
 | Broken images on the source site | detected, recorded, never blocks the run |
 | Bot protection (JS cookie challenge → 409) | declarative cookie workaround in config |
 | Legacy encodings (Big5, GB2312, …) | declared or defaulted per site |
@@ -105,6 +141,12 @@ polymigrate fetch-orphans site.yaml --section news
 polymigrate slugs . --section news          # slugs already mirrored under ./raw (pipeable)
 ```
 
+> **`raw/` is the one thing you cannot regenerate.** Re-crawling a site takes hours at a
+> polite request rate, and the source site may be switched off the moment you finish
+> migrating — at which point your mirror is the only copy left. No PolyMigrate command
+> ever deletes or overwrites `raw/` or `media/`; keep them until the migration is signed
+> off, and back them up if the source is going away.
+
 > No .NET? Every command above also runs with **zero install** via npm — just prefix it:
 > `npx cornhsu-polymigrate extract site.yaml`, `npx cornhsu-polymigrate verify out/`, and so on.
 
@@ -122,7 +164,7 @@ url_pattern:
 extract:
   content: "section[id]:not(#header):not(#footer)"
 pairing:
-  fallback: [shared_media, date, title_similarity]
+  fallback: [hreflang, slug_normalized, shared_media, date, title_similarity]
 ```
 
 ## Use as a library
@@ -153,6 +195,7 @@ Console.WriteLine($"{verify.Errors} errors, {verify.Warnings} warnings");
 | `src/PolyMigrate.Cli` | the `polymigrate` CLI (NuGet tool package: `Cornhsu.PolyMigrate`) |
 | `tests/` | unit/integration tests + an offline fixture site with golden-file baselines |
 | `docs/contracts.md` | file-format contracts between pipeline phases |
+| `docs/hreflang_量測與決策.md` | why `hreflang` only ever *suggests* — the measurement behind that call |
 | `docs/搬遷工具_評估與規劃書.md` | the original design/planning doc (the `§X.Y` references throughout the source point here) |
 
 ## Development

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cornhsu-polymigrate",
-  "version": "2.2.0",
+  "version": "2.3.0",
   "description": "The i18n-first static-site migrator — the only one that pairs your multilingual pages automatically. Mirrors a legacy site into frontmatter Markdown, auto-matches language versions, and honestly lists what it could not pair.",
   "keywords": [
     "i18n",
@@ -36,11 +36,11 @@
     "node": ">=18"
   },
   "optionalDependencies": {
-    "@cornhsu/polymigrate-win32-x64": "2.2.0",
-    "@cornhsu/polymigrate-win32-arm64": "2.2.0",
-    "@cornhsu/polymigrate-linux-x64": "2.2.0",
-    "@cornhsu/polymigrate-linux-arm64": "2.2.0",
-    "@cornhsu/polymigrate-darwin-x64": "2.2.0",
-    "@cornhsu/polymigrate-darwin-arm64": "2.2.0"
+    "@cornhsu/polymigrate-win32-x64": "2.3.0",
+    "@cornhsu/polymigrate-win32-arm64": "2.3.0",
+    "@cornhsu/polymigrate-linux-x64": "2.3.0",
+    "@cornhsu/polymigrate-linux-arm64": "2.3.0",
+    "@cornhsu/polymigrate-darwin-x64": "2.3.0",
+    "@cornhsu/polymigrate-darwin-arm64": "2.3.0"
   }
 }