designlang 7.0.0 → 7.1.0

package/docs/superpowers/specs/2026-04-19-designlang-v7-1-design.md

@@ -0,0 +1,111 @@
+# designlang v7.1 — Designer Toolkit
+
+**Date:** 2026-04-19
+**Status:** Approved for implementation
+
+## Theme
+
+*From extractor to designer's toolkit.* v7.0 shipped the full extraction surface. v7.1 makes the output composable, auditable, and complete — pull one component at a time, get every brand asset alongside the tokens, drop into any stack (Tailwind v4, Emotion, Stitches, Vanilla Extract, Panda), and fail your CI when production drifts from your design system.
+
+## Scope (7 features)
+
+### 1. Single-component extraction
+New flag: `designlang <url> --component <css-selector>`.
+- Playwright walks to the matched element, captures its own computed styles + its children's styles + its bounding box + its text content.
+- Emits a tiny extraction: `*-component.md` (a specimen-style card), `*-component.json` (the DTCG subset actually used by this component), `*-component.tsx` / `*-component.vue` / `*-component.svelte` based on `--framework` (or all three by default).
+- `--component` can be combined with `--platforms all` and `--emit-agent-rules`.
+
+### 2. Asset / brand-kit extraction
+New flag: `designlang <url> --assets` (also implied by `--full`).
+- Crawler enumerates every `<img>`, `<picture>`, `<source>`, `<svg>` (reuse existing icons work), `<link rel="icon">`, `<meta property="og:image">`, and CSS `background-image` URLs.
+- Downloads each asset with a timeout and size cap (50MB total per extraction).
+- Classifies: `logo`, `icon`, `illustration`, `photo`, `hero`, `og`, `favicon`, `avatar`, `screenshot`, `other`.
+- Emits to `<out>/assets/<kind>/<original-filename-or-hash.ext>` plus `assets-manifest.json` with the type, source URL, dimensions, byte size, and an `srcset`-style variant list for `<picture>` elements.
+- Size-aware: don't download a 20MB hero if it exceeds a per-asset cap (default 5MB). Log skipped assets in the manifest with a `skipped: reason` field.
+
+### 3. Illustration detection (folded into #2)
+- SVGs ≥ 200px or with >30 path elements → classified `illustration`, not `icon`.
+- Inline-SVG illustrations captured alongside external file references.
+- Raster illustrations (transparent PNG with flat-color regions) classified separately from photos via dominant-color entropy heuristic.
+
+### 4. Tailwind v4 `@theme` formatter
+New output file `<prefix>-tailwind.css` (separate from the existing `*-tailwind.config.js`, which stays for v3 users).
+- Uses the Tailwind v4 `@theme { … }` block syntax inside a `@import "tailwindcss"` wrapper.
+- Maps DTCG semantic tokens → Tailwind v4 CSS custom properties.
+- Respects `--tokens-legacy` (v4 file still emitted but from primitive-only JSON).
+
+### 5. CSS-in-JS emitters
+Four new formatters under `src/formatters/css-in-js/`:
+- `emotion.js` → `<prefix>-emotion.theme.ts`
+- `stitches.js` → `<prefix>-stitches.config.ts`
+- `vanilla-extract.js` → `<prefix>-vanilla.css.ts`
+- `panda.js` → `<prefix>-panda.config.ts`
+
+Selectable via new flag: `--framework emotion|stitches|vanilla-extract|panda|tailwind4`. Multi-select: `--framework emotion,stitches`. Default behavior unchanged: emit React theme + shadcn theme.
+
+### 6. Drift detection + GitHub Action
+New command: `designlang drift <url> <tokens-path> [--out <report-path>] [--fail-on <category>] [--tolerance <n>]`.
+- Extracts from `<url>`, loads DTCG JSON at `<tokens-path>`, walks both trees, and produces a report:
+  - `added` tokens (in extraction, not in file)
+  - `removed` (in file, not in extraction)
+  - `changed` (path matches, `$value` differs beyond tolerance)
+- Tolerance: colors use ΔE threshold (default 3), dimensions use px delta (default 1), others exact match.
+- `--fail-on color|dimension|typography|all` controls exit code (default exits 0 for a report-only run; exit 1 when any matching drift found with `--fail-on`).
+- Emits `drift-report.md` + `drift-report.json`.
+- **GitHub Action:** new `.github/workflows/designlang-drift.yml` template committed to this repo under `templates/github-action/designlang-drift.yml` with a README snippet. Runs on PR + daily cron, posts the report as a PR comment when drift is detected.
+
+### 7. Release
+- Bump to `7.1.0`, CHANGELOG entry, README updates, npm publish, git tag.
+
+## Non-goals (deferred to v7.2+)
+
+- **Remix Mode** / interactive web editor — substantial Next.js work; needs its own spec.
+- **Figma plugin**, **Chrome extension**, **VS Code extension** — each is a separate package with its own store/review cycle.
+- **Brand-voice extraction** (tone, reading level) — out of scope for a token release.
+- **Token marketplace / shared workspaces** — hosted SaaS territory.
+
+## Architecture
+
+### New files
+- `src/extractors/component.js` — selector-scoped extraction
+- `src/extractors/assets.js` — classifier + downloader for images and illustrations
+- `src/formatters/tailwind-v4.js`
+- `src/formatters/css-in-js/emotion.js`
+- `src/formatters/css-in-js/stitches.js`
+- `src/formatters/css-in-js/vanilla-extract.js`
+- `src/formatters/css-in-js/panda.js`
+- `src/formatters/component-snippet.js` — emits JSX / Vue / Svelte / HTML snippet from a component extraction
+- `src/formatters/drift.js` — produces drift-report.md + drift-report.json
+- `src/drift.js` — orchestrator for the `drift` subcommand
+- `templates/github-action/designlang-drift.yml`
+- `templates/github-action/README.md`
+
+### Modified files
+- `src/crawler.js` — add `fetchAssets()` with per-asset timeout + cap; add `page.$(selector).then(el => el.evaluate(...))` path for `--component`
+- `src/index.js` — wire new extractors + output files
+- `bin/design-extract.js` — new flags + `drift` subcommand
+- `src/config.js` — thread new options (`component`, `assets`, `framework`, driftTolerance)
+- `tests/extractors.test.js` + `tests/formatters.test.js` — new tests per extractor/formatter
+- `package.json` — version bump, potentially add `sharp` for optional raster resize (deferred decision — only if it stays zero-runtime-cost)
+- `README.md`
+- `CHANGELOG.md`
+
+### Dependencies
+- No new runtime deps. Image downloads use Node 20's native `fetch`. Classifier is heuristic only (no ML).
+
+### Security / safety
+- Asset downloader respects the same URL-safety layer as the website API (no private IPs, http(s) only, byte cap).
+- Same-origin preferred; cross-origin assets downloaded with an `Accept` header and a 10s timeout each.
+
+## Tests
+
+- Unit tests per extractor and formatter, snapshot-style for golden outputs.
+- Integration smoke: `designlang <url> --component "header nav" --assets --framework emotion,stitches,vanilla-extract,panda,tailwind4` produces all expected files.
+- Drift integration test: run against a fixture tokens file and a live URL, confirm report contents.
+
+## Release plan
+
+1. Implement via subagents in 3 parallel chunks (component+assets, emitters, drift+action).
+2. Tests green.
+3. README + CHANGELOG.
+4. Version bump → PR → merge → npm publish → tag.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "designlang",
-  "version": "7.0.0",
+  "version": "7.1.0",
   "description": "Extract the complete design language from any website — colors, typography, spacing, shadows, and more. Outputs AI-optimized markdown, W3C design tokens, Tailwind config, and CSS variables.",
   "type": "module",
   "bin": {

package/src/config.js

@@ -35,6 +35,9 @@ export function mergeConfig(cliOpts, config) {
     tokensLegacy: cliOpts.tokensLegacy || config.tokensLegacy || false,
     platforms: parsePlatforms(cliOpts.platforms ?? config.platforms ?? 'web'),
     emitAgentRules: cliOpts.emitAgentRules || config.emitAgentRules || false,
+    cookieFile: cliOpts.cookieFile || config.cookieFile,
+    insecure: cliOpts.insecure || config.insecure || false,
+    userAgent: cliOpts.userAgent || config.userAgent,
   };
 }
 

package/src/crawler.js

@@ -17,17 +17,35 @@ async function gotoWithRetry(page, url, opts, retries = 3) {
 }
 
 export async function crawlPage(url, options = {}) {
-  const { width = 1280, height = 800, wait = 0, dark = false, depth = 0, screenshots = false, outDir = '', executablePath, browserArgs, cookies, headers, ignore } = options;
+  const {
+    width = 1280, height = 800, wait = 0, dark = false, depth = 0,
+    screenshots = false, outDir = '', executablePath, browserArgs,
+    cookies, headers, ignore,
+    insecure = false,
+    userAgent,
+  } = options;
+
+  const launchArgs = [
+    ...(browserArgs || []),
+    // Common flags that help with dev environments and CI. Insecure-only flags
+    // are added below when the user opts in.
+    '--disable-dev-shm-usage',
+  ];
+  if (insecure) {
+    launchArgs.push('--ignore-certificate-errors', '--ignore-ssl-errors');
+  }
 
   const browser = await chromium.launch({
     headless: true,
     ...(executablePath && { executablePath }),
-    ...(browserArgs && { args: browserArgs }),
+    args: launchArgs,
   });
   try {
     const context = await browser.newContext({
       viewport: { width, height },
       colorScheme: 'light',
+      ignoreHTTPSErrors: insecure,
+      ...(userAgent && { userAgent }),
       ...(headers && { extraHTTPHeaders: headers }),
     });
 

package/src/utils-cookies.js

@@ -0,0 +1,73 @@
+// Cookie file loaders. Supports three formats so users can paste whatever
+// their existing tooling exports:
+//   - JSON array of Playwright cookie objects: [{name, value, domain, path, …}]
+//   - Playwright storageState JSON: { cookies: [...], origins: [...] }
+//   - Netscape cookies.txt: tab-separated lines (curl / wget / browser extensions)
+//
+// Returned shape is always the Playwright cookie array.
+
+import { readFileSync } from 'fs';
+
+function parseNetscape(text, targetUrl) {
+  const cookies = [];
+  const lines = text.split(/\r?\n/);
+  for (const raw of lines) {
+    const line = raw.trim();
+    if (!line) continue;
+    // Skip comment lines, but keep the Netscape `#HttpOnly_<domain>` prefix
+    // that browsers use to mark HttpOnly cookies — those are real entries.
+    if (line.startsWith('#') && !/^#HttpOnly_/i.test(line)) continue;
+    const parts = raw.split('\t');
+    if (parts.length < 7) continue;
+    const [domain, , path, secure, expires, name, value] = parts;
+    if (!name) continue;
+    const cookie = {
+      name,
+      value: value ?? '',
+      domain: domain.replace(/^#HttpOnly_/i, ''),
+      path: path || '/',
+      secure: secure === 'TRUE',
+      httpOnly: /^#HttpOnly_/i.test(domain),
+    };
+    const exp = Number(expires);
+    if (Number.isFinite(exp) && exp > 0) cookie.expires = exp;
+    if (!cookie.domain && targetUrl) cookie.url = targetUrl;
+    cookies.push(cookie);
+  }
+  return cookies;
+}
+
+function parseJson(text) {
+  const parsed = JSON.parse(text);
+  // Playwright storageState: { cookies: [...], origins: [...] }
+  if (parsed && Array.isArray(parsed.cookies)) return parsed.cookies;
+  // Raw array
+  if (Array.isArray(parsed)) return parsed;
+  throw new Error('cookie file: JSON must be a cookie array or Playwright storageState');
+}
+
+export function loadCookiesFromFile(filePath, targetUrl) {
+  const text = readFileSync(filePath, 'utf-8');
+  const trimmed = text.trimStart();
+  if (trimmed.startsWith('{') || trimmed.startsWith('[')) {
+    return parseJson(text);
+  }
+  return parseNetscape(text, targetUrl);
+}
+
+// Merge CLI-provided cookies (name=value strings) with file cookies.
+// Later entries (file) override earlier entries (CLI) at the same name+domain.
+export function mergeCookies(cliCookies = [], fileCookies = [], targetUrl) {
+  const seen = new Map();
+  const parseCli = (c) => {
+    if (typeof c !== 'string') return c;
+    const [name, ...rest] = c.split('=');
+    return { name, value: rest.join('='), url: targetUrl };
+  };
+  for (const c of [...cliCookies.map(parseCli), ...fileCookies]) {
+    if (!c || !c.name) continue;
+    const key = `${c.name}|${c.domain || c.url || ''}`;
+    seen.set(key, c);
+  }
+  return [...seen.values()];
+}

package/tests/cookies.test.js

@@ -0,0 +1,98 @@
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { writeFileSync, mkdtempSync } from 'fs';
+import { tmpdir } from 'os';
+import { join } from 'path';
+import { loadCookiesFromFile, mergeCookies } from '../src/utils-cookies.js';
+
+const tmp = mkdtempSync(join(tmpdir(), 'dl-cookies-'));
+
+function tmpFile(name, text) {
+  const p = join(tmp, name);
+  writeFileSync(p, text, 'utf-8');
+  return p;
+}
+
+describe('loadCookiesFromFile', () => {
+  it('loads JSON array', () => {
+    const f = tmpFile('c.json', JSON.stringify([
+      { name: 'session', value: 'abc', domain: '.example.com', path: '/' },
+    ]));
+    const out = loadCookiesFromFile(f, 'https://example.com');
+    assert.equal(out.length, 1);
+    assert.equal(out[0].name, 'session');
+    assert.equal(out[0].domain, '.example.com');
+  });
+
+  it('loads Playwright storageState', () => {
+    const f = tmpFile('state.json', JSON.stringify({
+      cookies: [{ name: 'csrf', value: 'xyz', domain: 'example.com', path: '/' }],
+      origins: [],
+    }));
+    const out = loadCookiesFromFile(f, 'https://example.com');
+    assert.equal(out.length, 1);
+    assert.equal(out[0].name, 'csrf');
+  });
+
+  it('loads Netscape cookies.txt with tab-separated lines and comment', () => {
+    const netscape = [
+      '# Netscape HTTP Cookie File',
+      '# comment',
+      '.example.com\tTRUE\t/\tFALSE\t1765000000\tsid\tabc123',
+      '#HttpOnly_.example.com\tTRUE\t/\tTRUE\t0\tauth\ttoken-value',
+    ].join('\n');
+    const f = tmpFile('c.txt', netscape);
+    const out = loadCookiesFromFile(f, 'https://example.com');
+    assert.equal(out.length, 2);
+    const sid = out.find((c) => c.name === 'sid');
+    assert.equal(sid.value, 'abc123');
+    assert.equal(sid.domain, '.example.com');
+    assert.equal(sid.secure, false);
+    assert.equal(sid.expires, 1765000000);
+    const auth = out.find((c) => c.name === 'auth');
+    assert.equal(auth.httpOnly, true);
+    assert.equal(auth.secure, true);
+  });
+
+  it('throws on invalid JSON shape', () => {
+    const f = tmpFile('bad.json', '{"not":"array"}');
+    assert.throws(() => loadCookiesFromFile(f, 'https://example.com'));
+  });
+});
+
+describe('mergeCookies', () => {
+  it('parses CLI name=value strings into cookie objects', () => {
+    const out = mergeCookies(['session=abc'], [], 'https://example.com');
+    assert.equal(out.length, 1);
+    assert.equal(out[0].name, 'session');
+    assert.equal(out[0].value, 'abc');
+    assert.equal(out[0].url, 'https://example.com');
+  });
+
+  it('lets file cookies override CLI cookies with the same name+domain', () => {
+    const out = mergeCookies(
+      [{ name: 'session', value: 'cli', domain: '.example.com' }],
+      [{ name: 'session', value: 'file', domain: '.example.com' }],
+      'https://example.com',
+    );
+    assert.equal(out.length, 1);
+    assert.equal(out[0].value, 'file');
+  });
+
+  it('keeps cookies with different domains separately', () => {
+    const out = mergeCookies(
+      [],
+      [
+        { name: 'x', value: '1', domain: 'a.example.com' },
+        { name: 'x', value: '2', domain: 'b.example.com' },
+      ],
+      'https://example.com',
+    );
+    assert.equal(out.length, 2);
+  });
+
+  it('strips entries without a name', () => {
+    const out = mergeCookies([], [{ value: 'orphan' }], 'https://example.com');
+    assert.equal(out.length, 0);
+  });
+});

package/website/app/api/extract/route.js

@@ -1,18 +1,50 @@
+// POST /api/extract
+// Streaming NDJSON: one JSON event per line.
+// Events:
+//   { type:'cache', cached:true }               — if served from Blob
+//   { type:'stage', name:'crawl'|... }           — progress markers
+//   { type:'token', category, path, value, $type } — one per semantic token
+//   { type:'summary', summary }
+//   { type:'files', files }                      — final, full file map
+//   { type:'error', error }                      — terminal failure
+
 import { extractDesignLanguage } from '../../../../src/index.js';
 import { formatMarkdown } from '../../../../src/formatters/markdown.js';
-import { formatTokens } from '../../../../src/formatters/tokens.js';
 import { formatTailwind } from '../../../../src/formatters/tailwind.js';
 import { formatCssVars } from '../../../../src/formatters/css-vars.js';
 import { formatPreview } from '../../../../src/formatters/preview.js';
 import { formatFigma } from '../../../../src/formatters/figma.js';
 import { formatReactTheme, formatShadcnTheme } from '../../../../src/formatters/theme.js';
+import { formatWordPress, formatWordPressTheme } from '../../../../src/formatters/wordpress.js';
+import { formatDtcgTokens } from '../../../../src/formatters/dtcg-tokens.js';
+import { formatIosSwiftUI } from '../../../../src/formatters/ios-swiftui.js';
+import { formatAndroidCompose } from '../../../../src/formatters/android-compose.js';
+import { formatFlutterDart } from '../../../../src/formatters/flutter-dart.js';
+import { formatAgentRules } from '../../../../src/formatters/agent-rules.js';
 import { nameFromUrl } from '../../../../src/utils.js';
 
-export const maxDuration = 60;
+import { validateTargetUrl } from '../../../../website/lib/url-safety.js';
+import { checkRate } from '../../../../website/lib/rate-limit.js';
+import { cacheKey, getCached, putCached } from '../../../../website/lib/cache.js';
+
+export const runtime = 'nodejs';
 export const dynamic = 'force-dynamic';
+export const maxDuration = 60;
+
+const STAGES = [
+  'crawl',
+  'colors',
+  'typography',
+  'spacing',
+  'shadows',
+  'borders',
+  'components',
+  'regions',
+  'a11y',
+  'score',
+];
 
 async function getBrowserOptions() {
-  // On Vercel/Lambda, use @sparticuz/chromium; locally, use playwright's bundled browser
   if (process.env.VERCEL || process.env.AWS_LAMBDA_FUNCTION_NAME) {
     const chromium = (await import('@sparticuz/chromium')).default;
     return {
@@ -23,63 +55,191 @@ async function getBrowserOptions() {
   return {};
 }
 
+function ndjson(obj) {
+  return new TextEncoder().encode(JSON.stringify(obj) + '\n');
+}
+
+// Walk a DTCG token tree and yield every leaf ({ $value, $type }).
+function* walkDtcgTokens(tree, path = []) {
+  if (!tree || typeof tree !== 'object') return;
+  if (tree.$value !== undefined && tree.$type !== undefined) {
+    yield { path: path.join('.'), value: tree.$value, $type: tree.$type };
+    return;
+  }
+  for (const key of Object.keys(tree)) {
+    if (key.startsWith('$')) continue;
+    yield* walkDtcgTokens(tree[key], [...path, key]);
+  }
+}
+
+function buildSummary(design) {
+  return {
+    url: design.meta?.url,
+    title: design.meta?.title,
+    colors: design.colors?.all?.length ?? 0,
+    colorList: (design.colors?.all || []).slice(0, 20).map((c) => c.hex),
+    fonts: design.typography?.families?.map((f) => f.name).join(', ') || 'none detected',
+    spacingCount: design.spacing?.scale?.length ?? 0,
+    spacingBase: design.spacing?.base ?? null,
+    shadowCount: design.shadows?.values?.length ?? 0,
+    radiiCount: design.borders?.radii?.length ?? 0,
+    componentCount: Object.keys(design.components || {}).length,
+    cssVarCount: Object.values(design.variables || {}).reduce((s, v) => s + Object.keys(v).length, 0),
+    a11yScore: design.accessibility?.score ?? null,
+    a11yFailCount: design.accessibility?.failCount ?? 0,
+    score: design.score,
+  };
+}
+
+function buildFiles(design, targetUrl) {
+  const prefix = nameFromUrl(targetUrl);
+  const dtcg = formatDtcgTokens(design);
+  const dtcgJson = JSON.stringify(dtcg, null, 2);
+
+  const files = {
+    [`${prefix}-design-language.md`]: formatMarkdown(design),
+    [`${prefix}-design-tokens.json`]: dtcgJson,
+    [`${prefix}-tailwind.config.js`]: formatTailwind(design),
+    [`${prefix}-variables.css`]: formatCssVars(design),
+    [`${prefix}-preview.html`]: formatPreview(design),
+    [`${prefix}-figma-variables.json`]: formatFigma(design),
+    [`${prefix}-theme.js`]: formatReactTheme(design),
+    [`${prefix}-shadcn-theme.css`]: formatShadcnTheme(design),
+    [`${prefix}-wordpress-theme.json`]: formatWordPress(design),
+  };
+
+  // MCP companion JSON — same subset the CLI writes.
+  files[`${prefix}-mcp.json`] = JSON.stringify({
+    colors: { all: design.colors?.all || [] },
+    regions: design.regions || [],
+    componentClusters: design.componentClusters || [],
+    accessibility: { remediation: design.accessibility?.remediation || [] },
+    cssHealth: design.cssHealth || null,
+  }, null, 2);
+
+  // iOS
+  files['ios/DesignTokens.swift'] = formatIosSwiftUI(dtcg);
+
+  // Android (returns { filename: content })
+  const android = formatAndroidCompose(dtcg);
+  for (const name of Object.keys(android)) {
+    files[`android/${name}`] = android[name];
+  }
+
+  // Flutter
+  files['flutter/design_tokens.dart'] = formatFlutterDart(dtcg);
+
+  // WordPress block theme (5 files)
+  const wpTheme = formatWordPressTheme(dtcg, design);
+  for (const name of Object.keys(wpTheme)) {
+    files[`wordpress-theme/${name}`] = wpTheme[name];
+  }
+
+  // Agent rules
+  const agentFiles = formatAgentRules({ design, tokens: dtcg, url: targetUrl });
+  for (const name of Object.keys(agentFiles)) {
+    files[name] = agentFiles[name];
+  }
+
+  return { files, dtcg };
+}
+
+function extractIp(request) {
+  const xff = request.headers.get('x-forwarded-for');
+  if (xff) return xff.split(',')[0].trim();
+  const real = request.headers.get('x-real-ip');
+  if (real) return real.trim();
+  return 'unknown';
+}
+
+// Emit cached payload as a simulated stream so the hero paints consistently.
+async function streamCached(controller, cached, targetUrl) {
+  controller.enqueue(ndjson({ type: 'cache', cached: true }));
+  for (const stage of STAGES) {
+    controller.enqueue(ndjson({ type: 'stage', name: stage }));
+    await new Promise((r) => setTimeout(r, 40));
+  }
+  // Re-derive DTCG tokens from the cached design so the token-by-token paint
+  // still happens on a cache hit.
+  const { files, dtcg } = buildFiles(cached.design, targetUrl);
+  for (const { path, value, $type } of walkDtcgTokens(dtcg)) {
+    const category = path.split('.')[1] || 'misc';
+    controller.enqueue(ndjson({ type: 'token', category, path, value, $type }));
+  }
+  controller.enqueue(ndjson({ type: 'summary', summary: buildSummary(cached.design) }));
+  controller.enqueue(ndjson({ type: 'files', files }));
+}
+
 export async function POST(request) {
-  try {
-    const { url } = await request.json();
-
-    if (!url) {
-      return Response.json({ error: 'URL is required' }, { status: 400 });
-    }
-
-    let targetUrl = url;
-    if (!targetUrl.startsWith('http')) targetUrl = `https://${targetUrl}`;
-
-    // Validate URL
-    try {
-      new URL(targetUrl);
-    } catch {
-      return Response.json({ error: 'Invalid URL' }, { status: 400 });
-    }
-
-    const browserOpts = await getBrowserOptions();
-    const design = await extractDesignLanguage(targetUrl, browserOpts);
-
-    const prefix = nameFromUrl(targetUrl);
-
-    const files = {
-      [`${prefix}-design-language.md`]: formatMarkdown(design),
-      [`${prefix}-design-tokens.json`]: formatTokens(design),
-      [`${prefix}-tailwind.config.js`]: formatTailwind(design),
-      [`${prefix}-variables.css`]: formatCssVars(design),
-      [`${prefix}-preview.html`]: formatPreview(design),
-      [`${prefix}-figma-variables.json`]: formatFigma(design),
-      [`${prefix}-theme.js`]: formatReactTheme(design),
-      [`${prefix}-shadcn-theme.css`]: formatShadcnTheme(design),
-    };
+  let body;
+  try { body = await request.json(); } catch {
+    return Response.json({ error: 'Invalid JSON body' }, { status: 400 });
+  }
 
-    const summary = {
-      url: design.meta.url,
-      title: design.meta.title,
-      colors: design.colors.all.length,
-      colorList: design.colors.all.slice(0, 20).map(c => c.hex),
-      fonts: design.typography.families.map(f => f.name).join(', ') || 'none detected',
-      spacingCount: design.spacing.scale.length,
-      spacingBase: design.spacing.base,
-      shadowCount: design.shadows.values.length,
-      radiiCount: design.borders.radii.length,
-      componentCount: Object.keys(design.components).length,
-      cssVarCount: Object.values(design.variables).reduce((s, v) => s + Object.keys(v).length, 0),
-      a11yScore: design.accessibility?.score ?? null,
-      a11yFailCount: design.accessibility?.failCount ?? 0,
-      score: design.score,
-    };
+  const validation = validateTargetUrl(body?.url);
+  if (!validation.ok) {
+    return Response.json({ error: validation.reason }, { status: validation.status });
+  }
+  const targetUrl = validation.url;
 
-    return Response.json({ summary, files });
-  } catch (err) {
-    console.error('Extraction failed:', err);
+  const ip = extractIp(request);
+  const rate = checkRate(`extract:${ip}`);
+  if (!rate.allowed) {
     return Response.json(
-      { error: err.message || 'Extraction failed' },
-      { status: 500 }
+      { error: 'Rate limit — 3 extractions per day. Try again later.', resetAt: rate.resetAt },
+      { status: 429 }
     );
   }
+
+  const stream = new ReadableStream({
+    async start(controller) {
+      try {
+        const key = cacheKey(targetUrl);
+        const cached = await getCached(key);
+        if (cached) {
+          await streamCached(controller, cached, targetUrl);
+          controller.close();
+          return;
+        }
+
+        // Pre-stage markers — best-effort progress since extraction is atomic.
+        controller.enqueue(ndjson({ type: 'stage', name: 'crawl' }));
+
+        const browserOpts = await getBrowserOptions();
+        const design = await extractDesignLanguage(targetUrl, browserOpts);
+
+        // Post-stage markers once extraction resolves.
+        for (const stage of STAGES.slice(1)) {
+          controller.enqueue(ndjson({ type: 'stage', name: stage }));
+        }
+
+        const { files, dtcg } = buildFiles(design, targetUrl);
+
+        // Token-by-token paint — every DTCG leaf becomes its own event.
+        for (const { path, value, $type } of walkDtcgTokens(dtcg)) {
+          const category = path.split('.')[1] || 'misc';
+          controller.enqueue(ndjson({ type: 'token', category, path, value, $type }));
+        }
+
+        controller.enqueue(ndjson({ type: 'summary', summary: buildSummary(design) }));
+        controller.enqueue(ndjson({ type: 'files', files }));
+
+        // Fire-and-forget cache write.
+        putCached(key, { design }).catch(() => {});
+      } catch (err) {
+        console.error('[extract] failed', { url: targetUrl, ip, message: err?.message });
+        controller.enqueue(ndjson({ type: 'error', error: err?.message || 'Extraction failed' }));
+      } finally {
+        controller.close();
+      }
+    },
+  });
+
+  return new Response(stream, {
+    headers: {
+      'content-type': 'application/x-ndjson; charset=utf-8',
+      'cache-control': 'no-store',
+      'x-accel-buffering': 'no',
+    },
+  });
 }