@eightmedia/flightcheck 0.0.0-stage → 1.0.0

package/docs/stack.md

@@ -0,0 +1,99 @@
+# Stack detection
+
+Flightcheck fingerprints the technology stack of a crawled site for handover docs and agent
+routing. Results appear in `analysis.stack` in `*-report.json` and under **Stack** in the Markdown report.
+
+Detection uses **curated flightcheck rules** in `src/crawler/stackDetect.ts` — not the full
+Wappalyzer / webappanalyzer database. Rules only fire on strong signals (script URLs, distinctive
+HTML markers, response headers, DNS). This avoids false positives from loose substring matches.
+
+## Categories
+
+| Category | Examples |
+| --- | --- |
+| Platform | PHP, Node.js, Ruby on Rails, Django, ASP.NET |
+| Framework | Next.js, Nuxt, Vue, Angular, Astro, Gatsby, Remix, SvelteKit |
+| CMS | WordPress, Sanity, Contentful, Storyblok, Shopify, Drupal, Webflow, Prismic |
+| Search | Algolia, Meilisearch, Typesense |
+| Hosting | Vercel, Netlify |
+| CDN | Cloudflare, AWS CloudFront |
+
+Tracking (Google Tag Manager, Google Analytics (GA4), Google Analytics (Universal), Facebook Pixel, Matomo, Plausible) is
+detected separately by the `tracking` crawl group and shown under **Tracking (flightcheck crawl)**
+in the stack section.
+
+## Signal sources
+
+| Source | When collected | Module |
+| --- | --- | --- |
+| HTML crawl | `stack` group (default on) | `detectStackFromHtml()` |
+| Response headers | `security-headers` check | `detectStackFromHeaders()` |
+| DNS records | `dns` check | `detectStackFromDns()` |
+| Tracking IDs | `tracking` crawl group | `extractTracking()` in `src/crawler/tracking.ts` |
+
+Without a headless browser, stack detection is limited to static HTML and infrastructure signals.
+Late-loaded scripts (GTM after consent, client-only frameworks) may be missed unless they appear in
+the initial HTML.
+
+## Crawl group
+
+The `stack` group runs during crawl by default:
+
+```bash
+pnpm run flightcheck -- https://example.com --skip stack
+```
+
+Per-page detections are stored as `stack.detections` on each crawl record. Detections are tagged
+with `detector: "flightcheck-v1"` so older crawl JSON from the webappanalyzer era is ignored
+during analyze.
+
+## Analyze output
+
+```markdown
+## Stack
+
+### Platform
+- **Node.js** (medium, site-wide)
+  - implied by Next.js
+
+### Framework
+- **Next.js** (high, site-wide)
+  - html: `data-next-head`
+
+### Hosting
+- **Vercel** (high, origin)
+  - header: x-vercel-id
+
+### Tracking (flightcheck crawl)
+- Tag managers:
+  - Google Tag Manager: `GTM-XXXX`
+```
+
+Run with infra checks for the fullest picture:
+
+```bash
+pnpm run flightcheck -- https://example.com --checks crawl,security-headers,dns,robots-txt,analyze --parallel
+```
+
+## Architecture
+
+```text
+src/crawler/stackDetect.ts   (curated fingerprint rules)
+        ↓
+src/crawler/stack.ts         (re-exports detect helpers)
+        ↓
+src/crawler/runCrawler.ts    (per-page detect during crawl)
+        ↓
+src/analyze/stack.ts         (aggregate site-wide + analyze report)
+```
+
+## Adding fingerprints
+
+Add a rule to `HTML_RULES`, `HEADER_RULES`, or `DNS_RULES` in `src/crawler/stackDetect.ts`.
+Each rule must match on a **specific** signal — never run loose patterns against the full HTML
+document.
+
+Use `IMPLIED` for safe parent technology inference (e.g. Next.js → Node.js).
+Use `EXCLUDES` to suppress redundant detections (e.g. Next.js excludes generic React).
+
+Add tests in `src/crawler/stackDetect.test.ts`.

package/package.json

@@ -1,6 +1,88 @@
 {
   "name": "@eightmedia/flightcheck",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "1.0.0",
+  "description": "Pre-launch website audit: crawl SEO/a11y/security checks and emit structured JSON + Markdown reports.",
+  "license": "GPL-3.0",
+  "type": "module",
+  "author": "EightMedia <info@eight.nl>",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/EightMedia/flightcheck.git"
+  },
+  "bugs": {
+    "url": "https://github.com/EightMedia/flightcheck/issues"
+  },
+  "homepage": "https://github.com/EightMedia/flightcheck#readme",
+  "keywords": [
+    "flightcheck",
+    "crawler",
+    "seo",
+    "accessibility",
+    "security",
+    "audit",
+    "pre-launch",
+    "cli"
+  ],
+  "engines": {
+    "node": ">=22.0.0"
+  },
+  "publishConfig": {
+    "access": "public",
+    "registry": "https://registry.npmjs.org/"
+  },
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    },
+    "./checks": {
+      "types": "./dist/checks/index.d.ts",
+      "import": "./dist/checks/index.js"
+    },
+    "./package.json": "./package.json"
+  },
+  "bin": {
+    "flightcheck": "dist/cli.js",
+    "flightcheck-analyze": "dist/analyze.js"
+  },
+  "files": [
+    "dist",
+    "!dist/orchestrate.js",
+    "!dist/orchestrate/**",
+    "!dist/orchestrate.d.ts",
+    "!dist/orchestrate.d.ts.map",
+    "docs/*.md",
+    "README.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "dev": "tsx src/cli.ts --help",
+    "flightcheck": "tsx src/cli.ts",
+    "crawl": "tsx src/cli.ts",
+    "analyze": "tsx src/analyze.ts",
+    "build": "tsc",
+    "prepublishOnly": "pnpm test && pnpm build",
+    "start": "node dist/cli.js",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc --noEmit"
+  },
+  "dependencies": {
+    "cheerio": "^1.1.2",
+    "commander": "^14.0.1",
+    "crawlee": "^3.15.0",
+    "pino": "^10.1.0",
+    "robots-parser": "^3.0.1",
+    "zod": "^4.1.11"
+  },
+  "devDependencies": {
+    "@types/node": "^24.5.2",
+    "tsx": "^4.20.6",
+    "typescript": "^5.9.3",
+    "vitest": "^4.1.9"
+  },
+  "packageManager": "pnpm@11.5.2+sha512.71c631e382066efc25625d5cf029075de07b61b37f6e27350fbd84b1bda5864c8c1967adc280776b45c30a715c0359a3be08fef42d5bb09e2b99029979692916"
+}