@goflag/cli 0.2.13 → 0.2.15

package/README.md

@@ -37,21 +37,21 @@ here and carries no code.
 
 ## What it does
 
-- **Broken links** — scrapes every crawled page, dedupes targets globally (a footer link on 500 pages is probed once), and checks each with `HEAD`→`GET` fallback, redirect-chain/loop detection, soft-404 and anti-bot (403/429) triage. Broken links are mapped back to the pages that reference them.
+- **Broken links** — reads the links of every crawled page from the copy the crawl kept, dedupes targets globally (a footer link on 500 pages is probed once, and a link to a page the crawl fetched without a redirect is judged from that answer rather than asked again), and checks each with `HEAD`→`GET` fallback, redirect-chain/loop detection, soft-404 and anti-bot (403/429) triage. Broken links are mapped back to the pages that reference them.
 - **Missing translation pages** — builds a `route × locale` matrix and flags every route that exists in one locale but is missing in another, plus hreflang reciprocity gaps (`A → B` with no `B → A`), missing `x-default`, and locale tags naming a language, script or region that does not exist. Rows are keyed by pathname, unless the site declares its translation clusters — either with `xhtml:link` in the sitemap, or with reciprocal `hreflang` between two crawled pages' `<head>`. Then those declarations decide which URLs are one page, whatever their slugs, and the run reports `diagnostics.declaredClusters`. The sitemap source survives `--coverage structural`, where the two locales of a slug-translating family are rarely both sampled; the `<head>` source needs both pages in hand and is what fixes the common site that declares properly there and nothing in its sitemap.
 - **robots.txt policy** — the file is parsed to RFC 9309, not scanned: every group, every rule, every line it could not read, each with its number. goflag flags a site that forbids crawling while its pages ask to be indexed — site-wide, and now per path, which is the quiet version nothing was watching. It also reports the file that errors (a 5xx there is read as a site-wide ban), one over the 500 KiB a parser must honour, a typo that silently deletes the rule you meant to write, a redirect handing your crawl policy to another origin, and a `Sitemap:` that is not a full URL.
 - **SEO metadata** — lints each page's `<head>` for the handful of mistakes that actually hurt in search and social and are invisible in a browser: missing/oversized `<title>` and description, missing/relative canonical, missing `og:title`/`og:description`/`og:image`, missing viewport, and contradictory `robots`/`googlebot`/`X-Robots-Tag` directives.
 - **Open Graph, past its presence** — a declared `og:image` is judged as well as counted: relative URLs no crawler can resolve, an undeclared size (so the first share of a URL renders without the image), a shape far from the 1.91:1 the card is cropped to, a missing `og:image:alt`, and — the one no tag check can reach — whether the URL actually serves an image at all. On a translated page, `og:locale` and `og:locale:alternate` are checked against the hreflang cluster — two vocabularies for one fact, which nothing in a build keeps in step.
 - **Icons** — whether anything declares one at all (nothing in a spec requires it, which is why nothing complains), whether iOS has an `apple-touch-icon` to use instead of screenshotting the page, whether the Web App Manifest and the `<head>` contradict each other about the same file, whether every declared icon answers with an image, whether a `sizes` attribute describes the file it points at (a `.ico` carrying 16, 32 and 48 declared as `48x48` advertises a third of itself), and whether the origin actually serves `/favicon.ico` — a 200 of HTML from a catch-all route does not count. Two icon lists that merely differ are the normal case and are not reported.
 - **Sitemap** — whether there is one, whether it parses (an HTML error page served with a 200 reads as healthy to anything that only checks the status), whether it lists anything, and whether an index can reach its children. Each entry is judged too: a `<loc>` that is not absolute, entries on another host or another protocol, a `<lastmod>` that is not a W3C Datetime or is dated in the future, a `changefreq` or `priority` outside the values the protocol defines.
-- **Where the two files meet the crawl** — the contradictions no single artefact shows. A sitemap entry that `robots.txt` forbids fetching, one whose page declares `noindex`, one whose canonical points elsewhere, one that 404s or redirects, and the indexable pages the crawl found that the sitemap never lists. Entry probing answers from the crawl and the link audit before it fetches anything, so a well-built site costs almost no extra requests — and when the caps stop it short, the finding says how many went unchecked rather than implying the rest are fine. Nothing is broken when you look at either half alone, which is why nobody finds these by reading.
+- **Where the two files meet the crawl** — the contradictions no single artefact shows. A sitemap entry that `robots.txt` forbids fetching, one whose page declares `noindex`, one whose canonical points elsewhere, one that 404s or redirects, and the indexable pages the crawl found that the sitemap never lists — judged only once every sitemap the site declares has been read, since a page listed where goflag could not read looks exactly like one the site forgot. Entry probing answers from the crawl and the link audit before it fetches anything, so a well-built site costs almost no extra requests — and when the caps stop it short, the finding says how many went unchecked rather than implying the rest are fine. Nothing is broken when you look at either half alone, which is why nobody finds these by reading.
 - **Unreachable pages** — a page the crawl reached that answered non-2xx, or that never answered at all (timeout, reset, DNS failure — recorded as `status: 0`), is reported in `unreachablePages`. goflag asks a second time, from the back of the queue, before calling a page unreachable. Each one is an `error`-severity finding: it turns the verdict red, counts toward `--max-debt`, and fails the build even under `--fail-on error`. A page that silently dropped out of the audited set is what poisons a baseline, so it is reported rather than warned about.
 
 **Which pages get audited.** When goflag finds a sitemap it does not audit every URL in it. It groups URLs that share a path shape — and therefore a template — into families, audits every page that stands alone, and samples three pages per family. A cap answers "how many" and never "which": on a site of thousands of pages built from thirty templates, the first 200 pages a crawl reaches are four templates out of thirty.
 
 The trade is stated in every report rather than hidden. Template rules (`canonical.*`, `hreflang.*`, `og.image.missing`) are conclusive on a sampled family; copy rules (`title.length`, `description.length`), broken links and the translation matrix are only conclusive on the pages that were drawn. The terminal prints a `COVERAGE` line, and `diagnostics.coverage` carries `mode`, `considered`, `selected` and every sampled family with its ratio. Pass `--coverage all` to audit what the sitemap lists in order, up to `--max-pages`, which is the pre-0.2.3 behaviour.
 
-Pages that declare a `<link rel="canonical">` pointing at another crawled page are excluded from the checks — the site has said they are duplicates, so linting them would multiply every finding by the number of filtered variants. The count is reported as `diagnostics.duplicatePages`.
+Pages that declare a `<link rel="canonical">` pointing at another crawled page are excluded from the page checks — the site has said they are duplicates, so linting them would multiply every finding by the number of filtered variants. The count is reported as `diagnostics.duplicatePages`. They still count for what the sitemap lists: a variant the sitemap names is the defect `sitemap.entry.non-canonical` reports.
 
 **Client-rendered pages.** By default, a page whose `<head>` looks empty — no
 title, no description, no canonical, no OG, no JSON-LD — is re-rendered in
@@ -156,8 +156,8 @@ npx @goflag/cli rules > rules.json
 ```
 
 `rules` answers a question about goflag rather than about a site: no URL, no
-crawl, no network. It ships fifty-eight rules — twenty-five page rules,
-twenty-eight site rules and five prose rules — and every entry carries its
+crawl, no network. It ships sixty-four rules — twenty-five page rules,
+thirty-four site rules and five prose rules — and every entry carries its
 scope, severity and summary. All but one also carry a rigor, the documents they
 cite and, where a remedy is a line of code, a fix snippet. The exception is
 `hreflang.sitemap-mismatch`, which emits `rigor: null` with an empty `sources`
@@ -558,7 +558,7 @@ Three rules, whichever you use:
 stages: [build, deploy, audit]
 
 variables:
-  GOFLAG_VERSION: "0.2.13"
+  GOFLAG_VERSION: "0.2.15"
 
 # Before the merge: build the branch, let goflag boot it, audit that.
 seo:mr:
@@ -610,7 +610,7 @@ jobs:
       - run: corepack enable && pnpm install --frozen-lockfile
       - run: pnpm build
       - run: |
-          npx --yes @goflag/cli@0.2.13 http://localhost:3000 \
+          npx --yes @goflag/cli@0.2.15 http://localhost:3000 \
             --start "pnpm start" --static --no-external \
             --baseline .goflag/baseline.json --regressions-only --max-debt 41 \
             --report goflag-report.json