@eightmedia/flightcheck 1.0.0 → 1.1.0

package/docs/analyze.md

@@ -36,6 +36,8 @@ Without `--output`, paths are derived from the input: `-raw.json` becomes
 ```json
 {
   "version": 1,
+  "flightcheckVersion": "1.0.1",
+  "flags": ["https://example.com", "--quiet"],
   "generatedAt": "2026-10-27T12:00:00.000Z",
   "source": "./flightcheck/example-raw.json",
   "redirectOnly": false,
@@ -67,7 +69,8 @@ Without `--output`, paths are derived from the input: `-raw.json` becomes
       "category": "infra",
       "scope": "origin",
       "title": "Missing Hsts",
-      "message": "Strict-Transport-Security header is missing on HTTPS."
+      "message": "Strict-Transport-Security header is missing on HTTPS.",
+      "fix": "Send `Strict-Transport-Security: max-age=31536000; includeSubDomains` on HTTPS responses (hosting or app headers)."
     }
   ]
 }
@@ -82,22 +85,31 @@ includes metadata so consumers can filter or regroup.
 | `category` | `infra` \| `seo` \| `a11y` \| `best-practices` \| `security` \| `performance` |
 | `scope` | `site-wide` \| `template` \| `page` \| `origin` |
 | `title` | Short human title |
-| `message` | What to do / what was found |
+| `message` | What was found |
+| `fix` | Concrete remediation suggestion when available |
 | `template` | Present when `scope` is `template` |
 | `urls` / `count` | Affected pages when relevant |
 
+Header fields:
+
+| Field | Meaning |
+| --- | --- |
+| `version` | Report JSON schema version |
+| `flightcheckVersion` | npm package version that produced the report |
+| `flags` | CLI arguments used for this run (empty when not invoked via CLI) |
+
 ### `*-report.md`
 
 Generated from the JSON. Structure:
 
-1. **At a glance** — page counts and finding totals
-2. **Stack / DNS / redirect** — factual context when available
-3. **Findings** — grouped by severity for reading order, with plain-language titles
+1. Header — generated time, Flightcheck version, flags, source path
+2. **At a glance** — page counts and finding totals
+3. **Stack / DNS / redirect** — factual context when available
+4. **Findings** — grouped by severity for reading order, with plain-language titles and a **Fix** line when available
 
 ## Notes
 
 - The script validates crawl input against the expected flightcheck report schema.
 - On schema errors, the script stops with a clear error message.
 - Fix-manifest and agent-jobs output are no longer generated. Prefer `*-report.json`
-  for machine consumers. See the design note in
-  [superpowers/specs/2026-10-27-structured-report-design.md](./superpowers/specs/2026-10-27-structured-report-design.md).
+  for machine consumers.

package/docs/crawler.md

@@ -34,7 +34,7 @@ Key flags:
 - `--errors-only` include only critical/error findings in the Markdown report
 - `--checks <ids>` comma-separated checks; default: all implemented checks (`crawl`, `security-headers`, `dns`, `robots-txt`, `sitemap`, `analyze`)
 - `--parallel` run independent checks in the same wave concurrently
-- `--max-pages <n>`
+- `--max-pages <n>` (default **100**) — hard crawl budget with smart URL selection (see below)
 - `--delay-ms <n>`
 - `--timeout-ms <n>`
 - `--concurrency <n>`
@@ -149,7 +149,20 @@ Additionally:
 
 - Only internal links are crawled. Links are discovered via `extractInternalLinks` and added to the queue; Crawlee's `enqueueLinks` is intentionally not used.
 - Sitemap URLs are added to the crawl queue by default so pages from `sitemap.xml` are visited even when they are not reachable via internal links.
-- `--max-pages` is the hard stop (`maxRequestsPerCrawl`). The crawl stops once this count is processed, even if unvisited links remain. Increase `--max-pages` to crawl the entire site.
+- `--max-pages` (default **100**) is the hard stop (`maxRequestsPerCrawl`). The crawl stops once this count is processed, even if unvisited links remain. Raise `--max-pages` (for example `1000` or higher) when you need broader coverage of a large site.
+
+### Smart page selection
+
+Large sitemaps often list thousands of archive detail URLs. Selection is based on **URL-tree shape**, not slug names (so `/stories`, `/nieuws`, or `/whatever` all work the same):
+
+1. Build a path tree from known URLs (sitemap seeds and later discoveries)
+2. Treat a path as a **list parent** when it has many direct children that are mostly **leaves** (no further subpages) — typical archive indexes
+3. Prefer **home**, other **structural** shallow pages, and those **list** indexes
+4. Under each list parent, take only a **round-robin sample** of leaf URLs until `--max-pages` is filled
+
+Seed order follows that ranking (FIFO). Structural links discovered from HTML are pushed to the front of the queue (`forefront`); leaf URLs under list parents are capped so one archive cannot flood the budget.
+
+Selection lives in `src/crawler/smartPageSelection.ts`.
 - Query parameters and hashes do not count toward URL identity.
 - Non-HTML responses are reported with `note`.
 - Links to files with a known non-HTML extension (such as `.pdf`, `.zip`, images, video, fonts) are not added to the queue, so Crawlee does not generate unnecessary retries and errors.

package/docs/index.md

@@ -11,14 +11,12 @@
 - [QC checks plan](./qc-checks-plan.md)
 - [Agent fixes (deprecated)](./agent-fixes.md)
 
-## Specs
-
-- [Structured report](./superpowers/specs/2026-10-27-structured-report-design.md)
-
 ## Changelog
 
 - Package releases: [CHANGELOG.md](../CHANGELOG.md)
 - Branch notes: [changelog/](./changelog/)
+  - [2026-10-28 — smart-max-pages](./changelog/2026-10-28--smart-max-pages.md)
+  - [2026-10-28 — report-version-flags](./changelog/2026-10-28--report-version-flags.md)
   - [2026-10-27 — npm-public-package](./changelog/2026-10-27--npm-public-package.md)
   - [2026-10-27 — structured-report-json](./changelog/2026-10-27--structured-report-json.md)
   - [2026-09-28 — agent-ready-check](./changelog/2026-09-28--agent-ready-check.md)

package/docs/qc-checks-plan.md

@@ -6,7 +6,7 @@ image alt text, broken links, accessibility, security headers, DNS, and optional
 performance auditing.
 
 **Status:** in progress — Phase 1 shipped, Phase 2 mostly shipped  
-**Last updated:** 2026-09-28
+**Last updated:** 2026-10-27
 
 ---
 
@@ -18,8 +18,9 @@ Add the checks below as modular, exportable functions (see `docs/check-api.md`)
 - Run independently via API endpoints in a consuming project
 - Execute in parallel where dependencies allow
 
-Each check produces structured JSON output and feeds into the Markdown TODO report via
-`analyze`.
+Each check produces structured JSON output and feeds into analyze, which writes
+`*-report.json` (`analysis` + `findings[]`) and a plain-language `*-report.md` from that
+model.
 
 ---
 
@@ -126,7 +127,7 @@ Wave 3 (optional, after crawl):
   performance (Lighthouse or PageSpeed API on same URL sample)
 
 Wave 4:
-  analyze (merges all check outputs into TODO report)
+  analyze (merges check outputs into *-report.json + *-report.md)
 ```
 
 Wave 2 and 3 share a URL sampler (`src/checks/sampleUrls.ts`):

package/docs/stack.md

@@ -1,7 +1,7 @@
 # Stack detection
 
-Flightcheck fingerprints the technology stack of a crawled site for handover docs and agent
-routing. Results appear in `analysis.stack` in `*-report.json` and under **Stack** in the Markdown report.
+Flightcheck fingerprints the technology stack of a crawled site for handover docs and tooling.
+Results appear in `analysis.stack` in `*-report.json` and under **Stack** in the Markdown report.
 
 Detection uses **curated flightcheck rules** in `src/crawler/stackDetect.ts` — not the full
 Wappalyzer / webappanalyzer database. Rules only fire on strong signals (script URLs, distinctive
@@ -49,26 +49,19 @@ during analyze.
 
 ## Analyze output
 
-```markdown
-## Stack
-
-### Platform
-- **Node.js** (medium, site-wide)
-  - implied by Next.js
-
-### Framework
-- **Next.js** (high, site-wide)
-  - html: `data-next-head`
+In `*-report.json`, stack detections live under `analysis.stack` (name, categories,
+confidence, evidence, pageCount). The Markdown report renders them as:
 
-### Hosting
-- **Vercel** (high, origin)
-  - header: x-vercel-id
+```markdown
+### Stack
 
-### Tracking (flightcheck crawl)
-- Tag managers:
-  - Google Tag Manager: `GTM-XXXX`
+- **Next.js** (JavaScript frameworks, high) — Next.js runtime markers in HTML
+- **Vercel** (Hosting, high) — header: x-vercel-id
 ```
 
+Tracking IDs from the crawl are still aggregated in the stack profile for tooling; the
+human Markdown report focuses on stack detections in the **At a glance** section.
+
 Run with infra checks for the fullest picture:
 
 ```bash

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@eightmedia/flightcheck",
-  "version": "1.0.0",
+  "version": "1.1.0",
   "description": "Pre-launch website audit: crawl SEO/a11y/security checks and emit structured JSON + Markdown reports.",
   "license": "GPL-3.0",
   "type": "module",