pagegraph 0.5.1 → 0.6.1

package/README.md

@@ -22,22 +22,26 @@ flowchart LR
 
 ```bash
 bun add pagegraph
-bun add -D effect @effect/platform-bun   # only if you use the CLI
-bun add -D lighthouse                    # only for `pagegraph audit` performance evidence
+bun add -D lighthouse   # only for `pagegraph audit` performance evidence
 ```
 
 | Entry                        | Exports                                                                                        | Peers                                       |
 | ---------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------- |
-| `pagegraph`        | `buildSeoGraph`, `renderSitemap`, `renderRobots`, `contentSignal`, `checkGraph`, `inspectHtml` | —                                           |
+| `pagegraph`        | `buildSeoGraph`, `renderSitemap`, `renderRobots`, `contentSignal`, `checkGraph`, `checkRenderedCoverage`, `decodeRenderedEdgeArtifact`, `inspectHtml` | —                                           |
 | `pagegraph/react`  | `createSeo` → `seoHead`, `Breadcrumbs`, JSON-LD generators                                     | `react`, `@tanstack/react-router`           |
 | `pagegraph/vite`   | `seoRouteConfig` coverage gate                                                                 | `vite`                                      |
 | `pagegraph/config` | `defineSeoConfig`, `viteGraphLoader`                                                           | `vite`                                      |
 | `pagegraph/audit`  | Audit services, scanner protocol, rules, and report schemas                                    | `effect`                                    |
-| `pagegraph` bin                    | CLI over the same graph                                                                        | `effect`, `@effect/platform-bun` (optional) |
+| `pagegraph` bin                   | CLI over the same graph                                                                        | bundled — runs on Bun                        |
 
 The core and React entries have **zero runtime dependencies** — everything above is a
 peer, and only the entries you import need theirs installed.
 
+The CLI **bundles Effect and the TypeSafe provider**, so it needs no Effect peers and does not
+depend on the app's Effect RC; it runs on [Bun](https://bun.sh) (`bunx pagegraph`). `vite` stays a
+peer — the graph commands load your app through Vite at runtime — and `lighthouse` is only needed by
+`pagegraph audit`. Importing `pagegraph/audit` programmatically still needs `effect`.
+
 ## Quick start
 
 **1. Bind your site identity once.** Route files never see an origin or a brand name.
@@ -245,7 +249,7 @@ severities:
 - **structural** — internally broken declarations; these fail `pagegraph check` (exit 1):
   dead or duplicate edges, path collisions, a redirect in the sitemap, a
   sitemap/noindex contradiction, a `related` target with no `link` card, an
-  instance without a title.
+  instance without a title, or an unmet contextual-link coverage rule.
 - **editorial** — quality smells, reported but non-failing: duplicate or mis-sized
   titles and descriptions.
 
@@ -285,6 +289,8 @@ export default defineSeoConfig({
   origin: "https://example.com",
   disallow: routeConfig.robotsExclusions,
   contentSignal: "search=yes, ai-input=yes, ai-train=yes",
+  // Fail `check` unless each named money page has enough contextual links.
+  coverage: [{ path: "/pricing", minInbound: 2 }, { path: "/features/*", minInbound: 1 }],
   loadGraph: viteGraphLoader({
     root: import.meta.dirname,
     entry: "/lib/seo/graph.ts",
@@ -298,12 +304,143 @@ pagegraph check                 # CI gate — exit 1 on structural violations
 pagegraph graph                 # the graph as a tree · --format mermaid | json
 pagegraph inspect /pricing      # one node: policy, sitemap status, in/out edges
 pagegraph inspect <url> --live  # fetch a deployed page, validate its rendered <head>
+pagegraph links verify <url>    # crawl served HTML: depth, orphans, declared-vs-rendered
+pagegraph links verify <url> --assert-coverage  # also assert seo.config.ts coverage on served anchors
+pagegraph links verify <url> --emit-rendered <file>  # save the rendered edge set
+pagegraph links candidates      # propose contextual links from the declared graph
+pagegraph links decide <file>   # answer typed link questions with Jev (TYPESAFE_API_KEY)
 pagegraph sitemap               # print sitemap.xml
 pagegraph robots                # print robots.txt
 ```
 
 Stdout is data, stderr is status — `pagegraph check --json | jq` just works.
 
+### Verify the rendered link graph
+
+`pagegraph links verify` crawls a site's served HTML — through the same DNS-pinned,
+private-IP-blocked HTTP path as `pagegraph audit` — and reports what a crawler
+actually receives: the real homepage depth, the pages with no incoming internal
+edge (rendered orphans), and, when the app has a `seo.config.ts`, the
+declared-vs-rendered link diff. It is bounded with `--limit` and needs no
+framework or config:
+
+```bash
+pagegraph links verify https://example.com
+pagegraph links verify https://example.com --limit 25 --json | jq
+```
+
+`--emit-rendered <path>` writes the crawl's raw edge set as a versioned artifact,
+and `--rendered <path>` replays one instead of crawling — so a single crawl can be
+asserted repeatedly in CI without re-fetching:
+
+```bash
+pagegraph links verify https://example.com --emit-rendered .seo/rendered.json
+pagegraph links verify --rendered .seo/rendered.json --json | jq
+```
+
+The artifact preserves every anchor's `region` (`body` vs `nav`/`footer`/`header`)
+and the provenance a gate needs to trust it:
+
+```jsonc
+{
+  "kind": "links-rendered",
+  "schemaVersion": 1,
+  "origin": "https://example.com",
+  "seed": "https://example.com/",
+  "crawl": {
+    "pages": 42,
+    "root": "/",
+    "limit": 100,
+    "truncated": false,
+    "bodyTruncated": false,
+    "truncatedPages": [],
+    "failures": [],
+  },
+  "nodes": ["/", "/pricing"],
+  "edges": [{ "from": "/", "to": "/pricing", "region": "body" }],
+}
+```
+
+It is also a drop-in input for `pagegraph links candidates --rendered`.
+
+### Assert coverage on the rendered graph
+
+`pagegraph check` evaluates the `seo.config.ts` `coverage` rules against the
+**declared** graph; `pagegraph links verify --assert-coverage` evaluates the same
+rules against the anchors a crawler **actually receives**. Only same-origin
+body-region anchors count — nav, header, and footer links are chrome — which is
+the served analogue of `check` counting `related` edges alone. The rule's target
+universe is still the declared graph's sitemap-eligible pages.
+
+```bash
+pagegraph links verify https://example.com --assert-coverage
+pagegraph links verify --rendered .seo/rendered.json --assert-coverage --json | jq
+```
+
+The command exits non-zero when any rule is unmet, and **refuses to assert** (also
+non-zero) rather than reporting a false pass when the crawl outran `--limit`, a
+page failed to fetch (its anchors are missing), a body hit `--max-body-bytes`, the
+config declares no `coverage`, or the artifact's origin differs from
+`seo.config.ts`. Under `--assert-coverage` the report gains a `coverage` block
+(`{ rules, ok, violations }`); the default `--json` summary is unchanged.
+
+### Propose contextual links
+
+`pagegraph links candidates` reads the declared graph and proposes
+`(source, destination)` pairs that are plausible contextual links and not already
+connected. Pages are grouped into clusters — a shared top-level section, or the
+same `kind` for root-level pages — and only sitemap-eligible pages are proposed.
+Pairs already declared as a `related` edge are excluded, and `--rendered` accepts
+a JSON dump of already-served anchors (`[{ from, to }]` or `{ edges: [...] }`) to
+exclude those too.
+
+The output is a reviewable plan: human text by default, versioned JSON with
+`--json`. Nothing is applied. `--limit` and `--cluster` bound the plan, and
+`--decide` optionally hands the candidates to Jev for a recommendation and
+confidence per pair (requires `TYPESAFE_API_KEY`):
+
+```bash
+pagegraph links candidates
+pagegraph links candidates --cluster blog --limit 20
+pagegraph links candidates --rendered rendered.json --json | jq
+pagegraph links candidates --decide
+```
+
+### Decide with Jev
+
+`pagegraph links decide` answers two typed questions per candidate — *does the source have a genuine
+reason to link to the destination?* and *is descriptive anchor text already in the copy?* — and
+routes anything inside the confidence band `(1−t, t)` to a `review` bucket instead of auto-applying.
+
+```bash
+pagegraph links decide candidates.json --threshold 0.9
+pagegraph links decide candidates.json --threshold 0.9 --json | jq
+cat candidates.json | pagegraph links decide
+```
+
+Decisions run through [Jev](https://typesafe.ai) via `@effect/ai-typesafe` (model `jev-latest`),
+bundled into the CLI. The API key is read from the **`TYPESAFE_API_KEY` environment variable** — it
+is deliberately not a `seo.config.ts` field, so keys never live in the repo; `seo.config.ts` carries
+non-secret decision settings only. Without a key the command exits 1 with a clear message and an
+empty stdout.
+
+### Contextual-link coverage
+
+Declare a contextual-link coverage policy in `seo.config.ts` (a `coverage` array
+of `{ path, minInbound }`), or pass repeatable `--require-inbound "<path-glob>=<n>"`
+flags to override it for one run. `pagegraph check` then fails (exit 1) unless
+every sitemap-eligible page matching the glob has at least `minInbound` incoming
+`related` edges. `*` matches within a path segment and `**` crosses segments; a
+rule that matches no sitemap-eligible page is itself a violation.
+
+```bash
+pagegraph check --require-inbound "/pricing=2" --require-inbound "/features/*=1"
+```
+
+`pagegraph check` asserts the policy on the declared graph. To prove the links are
+actually served, assert the same policy on the rendered graph — see
+[Assert coverage on the rendered graph](#assert-coverage-on-the-rendered-graph).
+
 ### Audit any website
 
 `pagegraph audit` is framework-independent and does not need `seo.config.ts`. It

package/dist/audit.d.ts

@@ -98,6 +98,11 @@ interface ProbeOptions {
   readonly timeoutMs: number;
   readonly maxBodyBytes: number;
   readonly allowPrivate: boolean;
+  /**
+   * Capture resolved anchors from HTML bodies. Opt-in because it is the only
+   * probe field that grows with document size; the audit report leaves it unset.
+   */
+  readonly captureAnchors?: boolean | undefined;
 }
 interface AuditOptions {
   readonly concurrency?: number;

package/dist/audit.js

@@ -1,3 +1,4 @@
+import { a as extractAnchors } from "./links-BFJc-pJ9.js";
 import { mkdir, mkdtemp, readFile, rename, rm, writeFile } from "node:fs/promises";
 import { join } from "node:path";
 import * as Context from "effect/Context";
@@ -976,6 +977,7 @@ const probeHttp = async (request, options) => {
 		} : await readBoundedBody(response, options.maxBodyBytes);
 		const contentType = typeof response.headers["content-type"] === "string" ? response.headers["content-type"] : "";
 		const body = new TextDecoder().decode(bounded.bytes);
+		const anchors = options.captureAnchors === true && contentType.includes("text/html") ? extractAnchors(body, currentUrl.href) : void 0;
 		return {
 			kind: request.kind,
 			method: request.method,
@@ -993,6 +995,7 @@ const probeHttp = async (request, options) => {
 			bodyTruncated: bounded.truncated,
 			bodyExcerpt: bodyExcerpt(body, contentType),
 			document: body.length === 0 ? null : extractDocumentSignals(body, contentType, currentUrl),
+			anchors,
 			error: null
 		};
 	} catch (error) {