@opendatalabs/darshana 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,162 @@
1
+ # darshana
2
+
3
+ Crawl any web app and generate a labeled PDF, HTML viewer, or image set for AI-assisted design review.
4
+
5
+ *Darśana* — Sanskrit for "the act of seeing clearly."
6
+
7
+ ## Install
8
+
9
+ ```bash
10
+ npm install -g @opendatalabs/darshana
11
+ # or use directly:
12
+ npx @opendatalabs/darshana --config review.config.json
13
+ ```
14
+
15
+ After installing, set up Playwright's browser:
16
+
17
+ ```bash
18
+ npx playwright install chromium
19
+ ```
20
+
21
+ ## Quick start
22
+
23
+ **1. Create a config file** (`review.config.json`):
24
+
25
+ ```json
26
+ {
27
+ "title": "My App",
28
+ "url": "https://myapp.example.com",
29
+ "start": "/dashboard",
30
+ "public": false,
31
+ "authStorage": "./auth.json",
32
+ "crawl": {
33
+ "include": ["^/dashboard"],
34
+ "exclude": ["logout", "delete"],
35
+ "maxDepth": 3,
36
+ "maxPages": 50,
37
+ "extraRoutes": []
38
+ },
39
+ "capture": {
40
+ "themes": ["dark"],
41
+ "viewports": ["desktop", "mobile"],
42
+ "delay": 400
43
+ },
44
+ "outputs": ["pdf", "html"],
45
+ "outputDir": "./output"
46
+ }
47
+ ```
48
+
49
+ **2. Authenticate** (opens a browser — log in, press Enter):
50
+
51
+ ```bash
52
+ npx darshana --config review.config.json --auth-only
53
+ ```
54
+
55
+ Or provide an `authScript` for headless login (see [examples/auth-example.mjs](examples/auth-example.mjs)).
56
+
57
+ **3. Generate the review**:
58
+
59
+ ```bash
60
+ npx darshana --config review.config.json
61
+ ```
62
+
63
+ ## Config reference
64
+
65
+ | Field | Type | Default | Description |
66
+ |---|---|---|---|
67
+ | `title` | string | `"Design Review"` | Title shown on cover page and HTML header |
68
+ | `url` | string | required | Base URL of the app |
69
+ | `start` | string | required | Path to start crawling from |
70
+ | `public` | boolean | `false` | Skip auth entirely for public sites |
71
+ | `authStorage` | string | `"./auth.json"` | Path to saved Playwright storageState |
72
+ | `authScript` | string | — | Path to a JS file that handles login programmatically |
73
+ | `outputs` | string[] | `["pdf"]` | Any of `"pdf"`, `"html"`, `"images"` |
74
+ | `outputDir` | string | same dir as config | Directory for generated output files |
75
+
76
+ ### `crawl`
77
+
78
+ | Field | Type | Default | Description |
79
+ |---|---|---|---|
80
+ | `include` | string[] | `[]` | Regex patterns — URL pathname must match all |
81
+ | `exclude` | string[] | `[]` | Regex patterns — URL pathname must not match any |
82
+ | `maxDepth` | number | `5` | Max BFS depth from start URL |
83
+ | `maxPages` | number | `100` | Hard cap on total pages crawled |
84
+ | `extraRoutes` | string[] | `[]` | Additional paths to capture (not crawled for links) |
85
+ | `routes` | Route[] | `[]` | Per-pattern sampling rules (see Routes DSL) |
86
+
87
+ ### `capture`
88
+
89
+ | Field | Type | Default | Description |
90
+ |---|---|---|---|
91
+ | `themes` | string[] | `["dark"]` | Theme names to capture — injected as `data-theme` + CSS class |
92
+ | `viewports` | string[] | `["desktop"]` | `"desktop"` (1440×900) or `"mobile"` (390×844) |
93
+ | `fullPage` | boolean | `true` | Capture full scrollable page height |
94
+ | `delay` | number | `400` | ms to wait after page load before capture |
95
+ | `waitFor` | string | — | CSS selector (prefix `$`) or JS expression to wait for |
96
+ | `overrides` | Override[] | `[]` | Per-route capture overrides |
97
+ | `contextOptions` | object | `{}` | Passed directly to `browser.newContext()` |
98
+ | `launchOptions` | object | `{}` | Passed directly to `chromium.launch()` |
99
+ | `playwrightOptions` | object | `{}` | Passed directly to `page.screenshot()` |
100
+ | `routeOptions` | object | — | `{ blockPatterns: string[] }` — abort matching network requests |
101
+
102
+ ### Routes DSL
103
+
104
+ Limit how many pages of each "shape" are captured. Uses Express-style `:param` notation.
105
+
106
+ ```json
107
+ "routes": [
108
+ { "pattern": "/dashboard/records/:id", "sample": 1, "follow": false },
109
+ { "pattern": "/dashboard/runs/:id", "sample": 2, "follow": false },
110
+ { "pattern": "/dashboard/**", "follow": true }
111
+ ]
112
+ ```
113
+
114
+ | Field | Type | Default | Description |
115
+ |---|---|---|---|
116
+ | `pattern` | string | required | Path pattern using `:param` and `/**` |
117
+ | `sample` | number | unlimited | Max pages to visit matching this pattern |
118
+ | `follow` | boolean | `true` | Whether to BFS-follow links on matching pages |
119
+
120
+ Patterns are matched in order — first match wins.
121
+
122
+ ## Auth options
123
+
124
+ **Headed handover** (default when no `authScript`): darshana opens a browser, you log in manually, press Enter — session is saved to `authStorage`. Sessions are reused for 12 hours.
125
+
126
+ **Headless auth script**: Create a JS file that exports a default function:
127
+
128
+ ```javascript
129
+ export default async function login(browser) {
130
+ const context = await browser.newContext();
131
+ const page = await context.newPage();
132
+ await page.goto(process.env.APP_URL + '/login');
133
+ await page.fill('#password', process.env.APP_PASSWORD);
134
+ await page.click('button[type="submit"]');
135
+ await page.waitForURL(/\/dashboard/);
136
+ const storagePath = './auth.json';
137
+ await context.storageState({ path: storagePath });
138
+ await context.close();
139
+ return storagePath;
140
+ }
141
+ ```
142
+
143
+ Set `"authScript": "./my-auth.mjs"` in config.
144
+
145
+ ## CLI
146
+
147
+ ```bash
148
+ darshana --config <path> # run full pipeline
149
+ darshana --config <path> --dry-run # discover URLs without capturing
150
+ darshana --config <path> --route /dashboard # capture one route only
151
+ darshana --config <path> --auth-only # save auth session and exit
152
+ ```
153
+
154
+ ## Outputs
155
+
156
+ - **`pdf`** — `<outputDir>/console-review.pdf` — labeled pages, cover page, one page per capture
157
+ - **`html`** — `<outputDir>/console-review.html` — self-contained HTML with sidebar nav, filters, keyboard navigation, viewport-correct sizing
158
+ - **`images`** — `<outputDir>/images/<viewport>/NNN-slug-theme.png` — individual screenshots grouped by viewport
159
+
160
+ ## License
161
+
162
+ MIT
@@ -0,0 +1,32 @@
1
+ /**
2
+ * Example authScript for a password-protected app.
3
+ * Configure in review.config.json as: "authScript": "./auth.mjs"
4
+ *
5
+ * This script receives a Playwright Browser instance, logs in,
6
+ * saves storageState to auth.json, and returns the path.
7
+ */
8
+ import path from 'node:path';
9
+ import { fileURLToPath } from 'node:url';
10
+
11
+ const __dir = path.dirname(fileURLToPath(import.meta.url));
12
+
13
+ export default async function login(browser) {
14
+ const pw = process.env.APP_PASSWORD;
15
+ if (!pw) throw new Error('APP_PASSWORD env var required');
16
+
17
+ const context = await browser.newContext();
18
+ const page = await context.newPage();
19
+
20
+ // Customize these for your app:
21
+ await page.goto(process.env.APP_URL + '/login', { waitUntil: 'domcontentloaded', timeout: 30000 });
22
+ await page.fill('#password', pw); // update selector for your login form
23
+ await Promise.all([
24
+ page.waitForURL(/\/dashboard/, { timeout: 30000 }),
25
+ page.click('button[type="submit"]'),
26
+ ]);
27
+
28
+ const storagePath = path.join(__dir, 'auth.json');
29
+ await context.storageState({ path: storagePath });
30
+ await context.close();
31
+ return storagePath;
32
+ }
@@ -0,0 +1,46 @@
1
+ {
2
+ "title": "PDPP Console",
3
+ "url": "https://pdpp.vivid.fish",
4
+ "start": "/dashboard",
5
+ "public": false,
6
+ "authScript": "./auth.mjs",
7
+ "crawl": {
8
+ "include": ["^/dashboard"],
9
+ "exclude": ["delete", "revoke", "logout", "disconnect"],
10
+ "maxDepth": 4,
11
+ "maxPages": 80,
12
+ "extraRoutes": ["/design-system"],
13
+ "routes": [
14
+ { "pattern": "/dashboard/records/add/static-secret/:connector", "sample": 1, "follow": false },
15
+ { "pattern": "/dashboard/records/add/browser-session/:connector", "sample": 1, "follow": false },
16
+ { "pattern": "/dashboard/connect/manual-upload/:connector", "sample": 1, "follow": false },
17
+ { "pattern": "/dashboard/records/:connector/:recordId", "sample": 1, "follow": false },
18
+ { "pattern": "/dashboard/records/:connector", "sample": 3, "follow": true },
19
+ { "pattern": "/dashboard/runs/:runId", "sample": 2, "follow": false },
20
+ { "pattern": "/dashboard/traces/:traceId", "sample": 2, "follow": false },
21
+ { "pattern": "/dashboard/grants/request", "follow": false },
22
+ { "pattern": "/dashboard/grants/packages/:packageId", "sample": 1, "follow": false },
23
+ { "pattern": "/dashboard/grants/:grantId", "sample": 2, "follow": false },
24
+ { "pattern": "/dashboard/**", "follow": true }
25
+ ]
26
+ },
27
+ "capture": {
28
+ "themes": ["dark", "light"],
29
+ "viewports": ["desktop", "mobile"],
30
+ "fullPage": true,
31
+ "delay": 400,
32
+ "routeOptions": {
33
+ "blockPatterns": ["analytics", "tracking", "hotjar", "intercom", "segment"]
34
+ },
35
+ "overrides": [
36
+ { "route": "^/design-system$", "themes": ["dark"], "viewports": ["desktop"] },
37
+ { "route": "/dashboard/records/", "delay": 1000 }
38
+ ]
39
+ },
40
+ "outputs": ["pdf", "html"],
41
+ "outputDir": "./output",
42
+ "pdf": {
43
+ "output": "./console-review.pdf",
44
+ "pageSize": "A4"
45
+ }
46
+ }
package/package.json ADDED
@@ -0,0 +1,50 @@
1
+ {
2
+ "name": "@opendatalabs/darshana",
3
+ "version": "1.0.0",
4
+ "description": "Crawl any web app and generate a labeled PDF, HTML viewer, or image set for design review.",
5
+ "type": "module",
6
+ "bin": {
7
+ "darshana": "./src/index.mjs"
8
+ },
9
+ "files": [
10
+ "src/",
11
+ "examples/",
12
+ "README.md"
13
+ ],
14
+ "dependencies": {
15
+ "playwright": "^1.60.0",
16
+ "pdf-lib": "^1.17.1",
17
+ "path-to-regexp": "^8.0.0"
18
+ },
19
+ "devDependencies": {
20
+ "semantic-release": "^25.0.0",
21
+ "@semantic-release/commit-analyzer": "^13.0.0",
22
+ "@semantic-release/release-notes-generator": "^14.0.0",
23
+ "@semantic-release/npm": "^12.0.0",
24
+ "@semantic-release/github": "^10.0.0",
25
+ "conventional-changelog-conventionalcommits": "^8.0.0"
26
+ },
27
+ "engines": {
28
+ "node": ">=18"
29
+ },
30
+ "publishConfig": {
31
+ "access": "public",
32
+ "provenance": false,
33
+ "registry": "https://registry.npmjs.org/",
34
+ "tag": "latest"
35
+ },
36
+ "keywords": [
37
+ "playwright",
38
+ "screenshot",
39
+ "pdf",
40
+ "html",
41
+ "design-review",
42
+ "crawler",
43
+ "darshana"
44
+ ],
45
+ "license": "MIT",
46
+ "repository": {
47
+ "type": "git",
48
+ "url": "https://github.com/vana-com/darshana.git"
49
+ }
50
+ }
package/src/auth.mjs ADDED
@@ -0,0 +1,72 @@
1
+ import fs from 'node:fs';
2
+ import readline from 'node:readline';
3
+ import { chromium } from 'playwright';
4
+
5
+ const AUTH_MAX_AGE_MS = 12 * 60 * 60 * 1000;
6
+
7
+ export async function ensureAuth(config) {
8
+ if (config.public === true) {
9
+ console.log('[auth] Public app — skipping auth.');
10
+ return null;
11
+ }
12
+
13
+ const storagePath = config.authStorage;
14
+
15
+ if (fs.existsSync(storagePath)) {
16
+ const stat = fs.statSync(storagePath);
17
+ const ageMs = Date.now() - stat.mtimeMs;
18
+ if (ageMs < AUTH_MAX_AGE_MS) {
19
+ const ageMin = Math.round(ageMs / 60000);
20
+ console.log(`[auth] Using cached auth (${ageMin}m old): ${storagePath}`);
21
+ return storagePath;
22
+ }
23
+ console.log('[auth] Cached auth is stale (>12h) — re-authenticating.');
24
+ }
25
+
26
+ if (config.authScript) {
27
+ console.log(`[auth] Running authScript: ${config.authScript}`);
28
+ const mod = await import(config.authScript);
29
+ const fn = mod.default;
30
+ if (typeof fn !== 'function') {
31
+ throw new Error(`authScript must export a default function, got: ${typeof fn}`);
32
+ }
33
+ const browser = await chromium.launch({ headless: true });
34
+ try {
35
+ const result = await fn(browser);
36
+ if (typeof result !== 'string') {
37
+ throw new Error('authScript must return a storageState file path string');
38
+ }
39
+ return result;
40
+ } finally {
41
+ await browser.close();
42
+ }
43
+ }
44
+
45
+ console.log('\n[auth] Launching headed browser for manual login...');
46
+ console.log(`[auth] Navigate to: ${config.url}`);
47
+ console.log('[auth] Log in, then press ENTER here to capture session.\n');
48
+
49
+ const browser = await chromium.launch({ headless: false });
50
+ const context = await browser.newContext();
51
+ const page = await context.newPage();
52
+ await page.goto(config.url);
53
+
54
+ await waitForEnter();
55
+
56
+ console.log(`[auth] Saving session to: ${storagePath}`);
57
+ await context.storageState({ path: storagePath });
58
+ await browser.close();
59
+
60
+ console.log('[auth] Session saved.\n');
61
+ return storagePath;
62
+ }
63
+
64
+ function waitForEnter() {
65
+ return new Promise((resolve) => {
66
+ const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
67
+ rl.question('', () => {
68
+ rl.close();
69
+ resolve();
70
+ });
71
+ });
72
+ }
@@ -0,0 +1,121 @@
1
+ import { VIEWPORT_PRESETS } from './config.mjs';
2
+
3
+ const NEXTJS_HIDE_STYLE =
4
+ 'nextjs-portal,[data-nextjs-toast],[data-nextjs-dialog],#__next-build-watcher{display:none!important}';
5
+
6
+ export async function captureAll(browser, config, urls) {
7
+ const results = [];
8
+ const themes = config.capture.themes;
9
+ const viewportNames = config.capture.viewports;
10
+
11
+ for (const viewportName of viewportNames) {
12
+ for (const theme of themes) {
13
+ const vpPreset = VIEWPORT_PRESETS[viewportName] ?? { width: 1440, height: 900, deviceScaleFactor: 1 };
14
+ const storageStatePath = config._storageStatePath ?? null;
15
+
16
+ const contextOpts = {
17
+ viewport: { width: vpPreset.width, height: vpPreset.height },
18
+ deviceScaleFactor: vpPreset.deviceScaleFactor ?? 1,
19
+ ...(config.capture.contextOptions ?? {}),
20
+ ...(storageStatePath ? { storageState: storageStatePath } : {}),
21
+ };
22
+
23
+ console.log(`\n[capture] Segment: ${viewportName} / ${theme} (${urls.length} URLs)`);
24
+ const context = await browser.newContext(contextOpts);
25
+
26
+ for (const url of urls) {
27
+ let pathname;
28
+ try { pathname = new URL(url).pathname; } catch { pathname = url; }
29
+
30
+ const override = resolveOverride(config.capture.overrides, pathname);
31
+
32
+ const effectiveThemes = override?.themes;
33
+ if (effectiveThemes && !effectiveThemes.includes(theme)) {
34
+ console.log(` [capture] skip ${pathname} [${theme}] (override)`);
35
+ continue;
36
+ }
37
+ const effectiveViewports = override?.viewports;
38
+ if (effectiveViewports && !effectiveViewports.includes(viewportName)) {
39
+ console.log(` [capture] skip ${pathname} [${viewportName}] (override)`);
40
+ continue;
41
+ }
42
+
43
+ const delay = override?.delay ?? config.capture.delay ?? 400;
44
+ const waitFor = override?.waitFor ?? config.capture.waitFor ?? null;
45
+ const label = makeLabel(pathname, viewportName, theme);
46
+
47
+ console.log(` [capture] ${label}`);
48
+ const page = await context.newPage();
49
+
50
+ try {
51
+ const blockPatterns = config.capture.routeOptions?.blockPatterns ?? [];
52
+ if (blockPatterns.length > 0) {
53
+ await page.route('**/*', (route) => {
54
+ const reqUrl = route.request().url();
55
+ if (blockPatterns.some(pat => reqUrl.includes(pat))) {
56
+ route.abort();
57
+ } else {
58
+ route.continue();
59
+ }
60
+ });
61
+ }
62
+
63
+ await page.goto(url, { waitUntil: 'networkidle', timeout: 60000 });
64
+ await page.addStyleTag({ content: NEXTJS_HIDE_STYLE }).catch(() => {});
65
+
66
+ await page.evaluate((t) => {
67
+ const html = document.documentElement;
68
+ html.setAttribute('data-theme', t);
69
+ if (t === 'dark') {
70
+ html.classList.add('dark');
71
+ html.classList.remove('light');
72
+ } else {
73
+ html.classList.add('light');
74
+ html.classList.remove('dark');
75
+ }
76
+ }, theme);
77
+
78
+ if (waitFor) {
79
+ if (waitFor.startsWith('$')) {
80
+ await page.waitForSelector(waitFor.slice(1), { timeout: 15000 });
81
+ } else {
82
+ await page.waitForFunction(waitFor, { timeout: 15000 });
83
+ }
84
+ }
85
+
86
+ if (delay > 0) await page.waitForTimeout(delay);
87
+
88
+ const imageBuffer = await page.screenshot({
89
+ fullPage: config.capture.fullPage ?? true,
90
+ type: 'png',
91
+ ...(config.capture.playwrightOptions ?? {}),
92
+ });
93
+
94
+ results.push({ url, pathname, theme, viewport: viewportName, imageBuffer, label });
95
+ } catch (err) {
96
+ console.error(` [capture] FAILED ${pathname}: ${err.message}`);
97
+ } finally {
98
+ await page.close();
99
+ }
100
+ }
101
+
102
+ await context.close();
103
+ console.log(`[capture] Segment done: ${viewportName} / ${theme}`);
104
+ }
105
+ }
106
+
107
+ return results;
108
+ }
109
+
110
+ function resolveOverride(overrides, pathname) {
111
+ if (!overrides?.length) return null;
112
+ for (const ov of overrides) {
113
+ if (new RegExp(ov.route).test(pathname)) return ov;
114
+ }
115
+ return null;
116
+ }
117
+
118
+ function makeLabel(pathname, viewport, theme) {
119
+ const humanPath = pathname.replace(/^\//, '').replace(/-/g, ' ') || '/';
120
+ return `${humanPath} · ${viewport} · ${theme}`;
121
+ }
package/src/config.mjs ADDED
@@ -0,0 +1,72 @@
1
+ import fs from 'node:fs';
2
+ import path from 'node:path';
3
+
4
+ export const VIEWPORT_PRESETS = {
5
+ desktop: { width: 1440, height: 900, deviceScaleFactor: 1 },
6
+ mobile: { width: 390, height: 844, deviceScaleFactor: 2 },
7
+ };
8
+
9
+ export function loadConfig(configPath) {
10
+ const absConfigPath = path.resolve(configPath);
11
+ const configDir = path.dirname(absConfigPath);
12
+
13
+ if (!fs.existsSync(absConfigPath)) {
14
+ throw new Error(`Config file not found: ${absConfigPath}`);
15
+ }
16
+
17
+ const raw = JSON.parse(fs.readFileSync(absConfigPath, 'utf8'));
18
+
19
+ if (!raw.url) throw new Error('Config missing required field: url');
20
+ if (!raw.start) throw new Error('Config missing required field: start');
21
+
22
+ const config = {
23
+ title: raw.title ?? 'Design Review',
24
+ url: raw.url.replace(/\/$/, ''),
25
+ start: raw.start,
26
+ public: raw.public ?? false,
27
+ authStorage: raw.authStorage ?? './auth.json',
28
+ authScript: raw.authScript ?? null,
29
+
30
+ crawl: {
31
+ include: raw.crawl?.include ?? [],
32
+ exclude: raw.crawl?.exclude ?? [],
33
+ maxDepth: raw.crawl?.maxDepth ?? 5,
34
+ maxPages: raw.crawl?.maxPages ?? 100,
35
+ extraRoutes: raw.crawl?.extraRoutes ?? [],
36
+ routes: raw.crawl?.routes ?? [],
37
+ },
38
+
39
+ capture: {
40
+ themes: raw.capture?.themes ?? ['dark'],
41
+ viewports: raw.capture?.viewports ?? ['desktop'],
42
+ fullPage: raw.capture?.fullPage ?? true,
43
+ delay: raw.capture?.delay ?? 400,
44
+ waitFor: raw.capture?.waitFor ?? null,
45
+ contextOptions: raw.capture?.contextOptions ?? {},
46
+ launchOptions: raw.capture?.launchOptions ?? {},
47
+ playwrightOptions: raw.capture?.playwrightOptions ?? {},
48
+ routeOptions: raw.capture?.routeOptions ?? null,
49
+ overrides: raw.capture?.overrides ?? [],
50
+ },
51
+
52
+ pdf: {
53
+ output: raw.pdf?.output ?? './console-review.pdf',
54
+ pageSize: raw.pdf?.pageSize ?? 'A4',
55
+ },
56
+
57
+ outputs: raw.outputs ?? ['pdf'],
58
+ outputDir: raw.outputDir ? path.resolve(configDir, raw.outputDir) : null,
59
+ };
60
+
61
+ config.authStorage = path.resolve(configDir, config.authStorage);
62
+ if (config.authScript) {
63
+ config.authScript = path.resolve(configDir, config.authScript);
64
+ }
65
+ config.pdf.output = path.resolve(configDir, config.pdf.output);
66
+
67
+ if (!config.outputDir) {
68
+ config.outputDir = path.dirname(config.pdf.output);
69
+ }
70
+
71
+ return config;
72
+ }
package/src/crawl.mjs ADDED
@@ -0,0 +1,116 @@
1
+ import { match } from 'path-to-regexp';
2
+
3
+ // path-to-regexp v8 uses {*path} for catch-all wildcards; config uses /** for readability
4
+ function toRegexpPattern(pattern) {
5
+ return pattern.replace(/\/\*\*$/, '/{*path}');
6
+ }
7
+
8
+ function compileRoutes(routes) {
9
+ return routes.map(route => ({
10
+ pattern: route.pattern,
11
+ matchFn: match(toRegexpPattern(route.pattern), { decode: decodeURIComponent }),
12
+ sample: route.sample ?? null,
13
+ follow: route.follow ?? true,
14
+ }));
15
+ }
16
+
17
+ export async function crawl(context, config) {
18
+ const origin = new URL(config.url).origin;
19
+ const startUrl = config.url + config.start;
20
+
21
+ const includePatterns = (config.crawl.include ?? []).map(r => new RegExp(r));
22
+ const excludePatterns = (config.crawl.exclude ?? []).map(r => new RegExp(r));
23
+ const compiledRoutes = compileRoutes(config.crawl.routes ?? []);
24
+ const seenShapes = new Map();
25
+
26
+ const visitedPathnames = new Set();
27
+ const queue = [{ url: startUrl, depth: 0 }];
28
+ const result = [];
29
+
30
+ function passesFilters(url) {
31
+ let pathname;
32
+ try { pathname = new URL(url).pathname; } catch { return false; }
33
+ if (includePatterns.length > 0 && !includePatterns.every(re => re.test(pathname))) return false;
34
+ if (excludePatterns.some(re => re.test(pathname))) return false;
35
+ return true;
36
+ }
37
+
38
+ function normalizeKey(url) {
39
+ try { return new URL(url).pathname; } catch { return url; }
40
+ }
41
+
42
+ function findRoute(url) {
43
+ let pathname;
44
+ try { pathname = new URL(url).pathname; } catch { return null; }
45
+ for (const route of compiledRoutes) {
46
+ if (route.matchFn(pathname)) return route;
47
+ }
48
+ return null;
49
+ }
50
+
51
+ while (queue.length > 0 && result.length < config.crawl.maxPages) {
52
+ const { url, depth } = queue.shift();
53
+ const key = normalizeKey(url);
54
+
55
+ if (visitedPathnames.has(key)) continue;
56
+ visitedPathnames.add(key);
57
+
58
+ if (!passesFilters(url)) continue;
59
+
60
+ const matchedRoute = findRoute(url);
61
+ let shouldFollow = true;
62
+
63
+ if (matchedRoute !== null) {
64
+ const visitCount = seenShapes.get(matchedRoute.pattern) ?? 0;
65
+ if (matchedRoute.sample !== null && visitCount >= matchedRoute.sample) {
66
+ // Over sample limit — skip entirely, don't visit, don't follow
67
+ continue;
68
+ }
69
+ result.push(url);
70
+ console.log(` [crawl] ${url} (depth ${depth}) [${matchedRoute.pattern}]`);
71
+ seenShapes.set(matchedRoute.pattern, visitCount + 1);
72
+ shouldFollow = matchedRoute.follow;
73
+ } else {
74
+ // No route matched — visit and follow (existing behavior)
75
+ result.push(url);
76
+ console.log(` [crawl] ${url} (depth ${depth})`);
77
+ }
78
+
79
+ // Don't load the page if we're not following links from it
80
+ if (!shouldFollow) continue;
81
+ if (depth >= config.crawl.maxDepth || result.length >= config.crawl.maxPages) continue;
82
+
83
+ const page = await context.newPage();
84
+ try {
85
+ await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
86
+ const hrefs = await page.$$eval('a[href]', els =>
87
+ els.map(el => el.getAttribute('href')).filter(Boolean)
88
+ );
89
+ for (const href of hrefs) {
90
+ let resolved;
91
+ try { resolved = new URL(href, url).href; } catch { continue; }
92
+ if (!resolved.startsWith(origin)) continue;
93
+ const childKey = normalizeKey(resolved);
94
+ if (!visitedPathnames.has(childKey)) {
95
+ queue.push({ url: resolved, depth: depth + 1 });
96
+ }
97
+ }
98
+ } catch (err) {
99
+ console.warn(` [crawl] Failed to load ${url}: ${err.message}`);
100
+ } finally {
101
+ await page.close();
102
+ }
103
+ }
104
+
105
+ for (const route of (config.crawl.extraRoutes ?? [])) {
106
+ const fullUrl = route.startsWith('http') ? route : config.url + route;
107
+ const key = normalizeKey(fullUrl);
108
+ if (!visitedPathnames.has(key)) {
109
+ visitedPathnames.add(key);
110
+ result.push(fullUrl);
111
+ console.log(` [crawl] extra: ${fullUrl}`);
112
+ }
113
+ }
114
+
115
+ return result;
116
+ }
package/src/html.mjs ADDED
@@ -0,0 +1,164 @@
1
+ import fs from 'node:fs';
2
+ import path from 'node:path';
3
+ import { VIEWPORT_PRESETS } from './config.mjs';
4
+
5
+ export async function assembleHtml(pages, config, outputDir) {
6
+ const date = new Date().toISOString().slice(0, 10);
7
+ const title = config.title ?? 'Design Review';
8
+
9
+ // Generate viewport-specific CSS widths from actual config
10
+ const viewportCss = config.capture.viewports.map(vp => {
11
+ const preset = VIEWPORT_PRESETS[vp] ?? { width: 1440 };
12
+ const cssWidth = vp === 'desktop' ? '100%' : `${preset.width}px`;
13
+ return `img[data-viewport="${vp}"] { width: ${cssWidth}; max-width: ${preset.width}px; }`;
14
+ }).join('\n ');
15
+
16
+ const navItems = [];
17
+ const pageSections = [];
18
+
19
+ pages.forEach((capture, i) => {
20
+ const idx = i + 1;
21
+ const pageId = `page-${idx}`;
22
+ const base64 = capture.imageBuffer.toString('base64');
23
+
24
+ navItems.push(
25
+ `<li data-theme="${escHtml(capture.theme)}" data-viewport="${escHtml(capture.viewport)}"><a href="#${pageId}" data-theme="${escHtml(capture.theme)}" data-viewport="${escHtml(capture.viewport)}">${escHtml(capture.label)}</a></li>`
26
+ );
27
+
28
+ pageSections.push(`<div class="page" id="${pageId}" data-theme="${escHtml(capture.theme)}" data-viewport="${escHtml(capture.viewport)}">
29
+ <div class="label"><span class="idx">${idx}</span>${escHtml(capture.label)}</div>
30
+ <img src="data:image/png;base64,${base64}" alt="${escHtml(capture.label)}" data-viewport="${escHtml(capture.viewport)}" loading="lazy">
31
+ </div>`);
32
+ });
33
+
34
+ const themes = [...new Set(pages.map(p => p.theme))];
35
+ const viewports = [...new Set(pages.map(p => p.viewport))];
36
+
37
+ const themeCheckboxes = themes.map(t =>
38
+ `<label><input type="checkbox" data-filter="theme" value="${escHtml(t)}" checked> ${escHtml(t)}</label>`
39
+ ).join('\n ');
40
+
41
+ const viewportCheckboxes = viewports.map(v =>
42
+ `<label><input type="checkbox" data-filter="viewport" value="${escHtml(v)}" checked> ${escHtml(v)}</label>`
43
+ ).join('\n ');
44
+
45
+ const html = `<!DOCTYPE html>
46
+ <html lang="en">
47
+ <head>
48
+ <meta charset="utf-8">
49
+ <meta name="viewport" content="width=device-width, initial-scale=1">
50
+ <title>${escHtml(title)}</title>
51
+ <style>
52
+ *, *::before, *::after { box-sizing: border-box; }
53
+ body { background: #1a1a1a; color: #fff; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; margin: 0; display: flex; min-height: 100vh; }
54
+
55
+ #sidebar { width: 240px; min-width: 240px; background: #111; border-right: 1px solid #2a2a2a; position: fixed; top: 0; left: 0; height: 100vh; overflow-y: auto; display: flex; flex-direction: column; z-index: 100; }
56
+ .nav-header { padding: 16px; font-size: 13px; font-weight: 600; color: #fff; border-bottom: 1px solid #2a2a2a; }
57
+ .nav-meta { padding: 8px 16px; font-size: 11px; color: #555; border-bottom: 1px solid #2a2a2a; }
58
+ .filters { padding: 12px 16px; border-bottom: 1px solid #2a2a2a; }
59
+ .filters label { display: block; font-size: 12px; color: #aaa; margin: 4px 0; cursor: pointer; }
60
+ .filters label input { margin-right: 6px; accent-color: #4a9eff; }
61
+ .filter-group-label { font-size: 10px; color: #555; text-transform: uppercase; letter-spacing: 0.08em; margin: 8px 0 4px; }
62
+ #nav-list { list-style: none; margin: 0; padding: 8px 0; flex: 1; }
63
+ #nav-list li a { display: block; padding: 5px 16px; font-size: 11px; color: #666; text-decoration: none; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
64
+ #nav-list li a:hover, #nav-list li a.active { background: #1e1e1e; color: #fff; }
65
+ #nav-list li[data-hidden] { display: none; }
66
+
67
+ #content { margin-left: 240px; flex: 1; padding: 32px; min-width: 0; }
68
+ .cover { padding: 48px 0 40px; border-bottom: 1px solid #2a2a2a; margin-bottom: 48px; }
69
+ .cover h1 { font-size: 1.75rem; margin: 0 0 8px; font-weight: 600; }
70
+ .cover p { color: #666; margin: 4px 0; font-size: 13px; }
71
+
72
+ .page { margin-bottom: 56px; }
73
+ .page[data-hidden] { display: none; }
74
+ .label { background: #0d0d0d; border: 1px solid #2a2a2a; border-bottom: none; padding: 8px 14px; font-size: 12px; color: #ccc; font-family: 'SF Mono', 'Fira Code', monospace; display: flex; align-items: center; gap: 10px; border-radius: 6px 6px 0 0; }
75
+ .label .idx { background: #2a2a2a; color: #888; padding: 1px 6px; border-radius: 3px; font-size: 10px; min-width: 24px; text-align: center; }
76
+ .page img { display: block; border: 1px solid #2a2a2a; border-radius: 0 0 6px 6px; box-shadow: 0 4px 24px rgba(0,0,0,0.4); }
77
+ img[data-viewport="mobile"] { border-radius: 12px; box-shadow: 0 8px 32px rgba(0,0,0,0.5); }
78
+
79
+ ${viewportCss}
80
+
81
+ @media (max-width: 600px) {
82
+ #sidebar { display: none; }
83
+ #content { margin-left: 0; }
84
+ }
85
+ </style>
86
+ </head>
87
+ <body>
88
+ <nav id="sidebar">
89
+ <div class="nav-header">${escHtml(title)}</div>
90
+ <div class="nav-meta">${escHtml(date)} · ${pages.length} pages</div>
91
+ <div class="filters">
92
+ <div class="filter-group-label">Theme</div>
93
+ ${themeCheckboxes}
94
+ <div class="filter-group-label" style="margin-top:10px">Viewport</div>
95
+ ${viewportCheckboxes}
96
+ </div>
97
+ <ul id="nav-list">
98
+ ${navItems.join('\n ')}
99
+ </ul>
100
+ </nav>
101
+ <main id="content">
102
+ <div class="cover">
103
+ <h1>${escHtml(title)}</h1>
104
+ <p>${escHtml(config.url)}</p>
105
+ <p>${escHtml(date)} · ${pages.length} pages</p>
106
+ </div>
107
+ ${pageSections.join('\n ')}
108
+ </main>
109
+ <script>
110
+ function applyFilters() {
111
+ const checked = { theme: new Set(), viewport: new Set() };
112
+ document.querySelectorAll('input[data-filter]').forEach(cb => {
113
+ if (cb.checked) checked[cb.dataset.filter].add(cb.value);
114
+ });
115
+ document.querySelectorAll('.page').forEach((page, i) => {
116
+ const visible = checked.theme.has(page.dataset.theme) && checked.viewport.has(page.dataset.viewport);
117
+ page.toggleAttribute('data-hidden', !visible);
118
+ document.querySelectorAll('#nav-list li')[i]?.toggleAttribute('data-hidden', !visible);
119
+ });
120
+ }
121
+ document.querySelectorAll('input[data-filter]').forEach(cb => cb.addEventListener('change', applyFilters));
122
+
123
+ let currentIdx = 0;
124
+ function visiblePages() { return [...document.querySelectorAll('.page:not([data-hidden])')]; }
125
+ document.addEventListener('keydown', e => {
126
+ const ps = visiblePages();
127
+ if (!ps.length) return;
128
+ if (e.key === 'ArrowRight' || e.key === 'ArrowDown') currentIdx = Math.min(currentIdx + 1, ps.length - 1);
129
+ else if (e.key === 'ArrowLeft' || e.key === 'ArrowUp') currentIdx = Math.max(currentIdx - 1, 0);
130
+ else return;
131
+ ps[currentIdx].scrollIntoView({ behavior: 'smooth', block: 'start' });
132
+ document.querySelectorAll('#nav-list li a').forEach(a => a.classList.remove('active'));
133
+ const link = document.querySelector('#nav-list li a[href="#' + ps[currentIdx].id + '"]');
134
+ if (link) { link.classList.add('active'); link.scrollIntoView({ block: 'nearest' }); }
135
+ });
136
+
137
+ const observer = new IntersectionObserver(entries => {
138
+ entries.forEach(entry => {
139
+ if (entry.isIntersecting && !entry.target.hasAttribute('data-hidden')) {
140
+ const id = entry.target.id;
141
+ document.querySelectorAll('#nav-list li a').forEach(a => {
142
+ a.classList.toggle('active', a.getAttribute('href') === '#' + id);
143
+ });
144
+ }
145
+ });
146
+ }, { threshold: 0.1 });
147
+ document.querySelectorAll('.page').forEach(p => observer.observe(p));
148
+ </script>
149
+ </body>
150
+ </html>`;
151
+
152
+ const outputPath = path.join(outputDir, 'console-review.html');
153
+ fs.writeFileSync(outputPath, html, 'utf8');
154
+ const sizeMB = (fs.statSync(outputPath).size / 1024 / 1024).toFixed(1);
155
+ console.log(`\n[html] Wrote ${pages.length} pages (${sizeMB} MB) → ${outputPath}`);
156
+ }
157
+
158
+ function escHtml(str) {
159
+ return String(str)
160
+ .replace(/&/g, '&amp;')
161
+ .replace(/</g, '&lt;')
162
+ .replace(/>/g, '&gt;')
163
+ .replace(/"/g, '&quot;');
164
+ }
package/src/index.mjs ADDED
@@ -0,0 +1,141 @@
1
+ #!/usr/bin/env node
2
+ import fs from 'node:fs';
3
+ import path from 'node:path';
4
+ import { chromium } from 'playwright';
5
+ import { loadConfig } from './config.mjs';
6
+ import { ensureAuth } from './auth.mjs';
7
+ import { crawl } from './crawl.mjs';
8
+ import { captureAll } from './capture.mjs';
9
+ import { assemblePdf } from './pdf.mjs';
10
+ import { assembleHtml } from './html.mjs';
11
+
12
+ function parseArgs(argv) {
13
+ const args = { config: null, dryRun: false, route: null, authOnly: false };
14
+ for (let i = 0; i < argv.length; i++) {
15
+ if (argv[i] === '--config' && argv[i + 1]) { args.config = argv[++i]; continue; }
16
+ if (argv[i] === '--dry-run') { args.dryRun = true; continue; }
17
+ if (argv[i] === '--route' && argv[i + 1]) { args.route = argv[++i]; continue; }
18
+ if (argv[i] === '--auth-only') { args.authOnly = true; continue; }
19
+ }
20
+ return args;
21
+ }
22
+
23
+ const args = parseArgs(process.argv.slice(2));
24
+
25
+ if (!args.config) {
26
+ console.error('Usage: darshana --config <path> [--dry-run] [--route <path>] [--auth-only]');
27
+ process.exit(1);
28
+ }
29
+
30
+ async function main() {
31
+ const config = loadConfig(args.config);
32
+ console.log(`[darshana] ${config.title} — ${config.url}`);
33
+
34
+ const storageStatePath = await ensureAuth(config);
35
+ config._storageStatePath = storageStatePath;
36
+
37
+ if (args.authOnly) {
38
+ console.log('[darshana] --auth-only done.');
39
+ process.exit(0);
40
+ }
41
+
42
+ let urls;
43
+ if (args.route) {
44
+ const fullUrl = args.route.startsWith('http') ? args.route : config.url + args.route;
45
+ urls = [fullUrl];
46
+ console.log(`[darshana] --route mode: ${fullUrl}`);
47
+ } else {
48
+ const crawlBrowser = await chromium.launch({ headless: true });
49
+ const crawlContextOpts = storageStatePath ? { storageState: storageStatePath } : {};
50
+ const crawlContext = await crawlBrowser.newContext(crawlContextOpts);
51
+ try {
52
+ console.log(`[darshana] Crawling from ${config.url}${config.start} ...`);
53
+ urls = await crawl(crawlContext, config);
54
+ } finally {
55
+ await crawlContext.close();
56
+ await crawlBrowser.close();
57
+ }
58
+ console.log(`[darshana] Crawl complete: ${urls.length} URLs found.`);
59
+ }
60
+
61
+ if (args.dryRun) {
62
+ console.log('\n[darshana] --dry-run: discovered URLs:');
63
+ for (const u of urls) console.log(` ${u}`);
64
+ process.exit(0);
65
+ }
66
+
67
+ const browser = await chromium.launch({
68
+ headless: true,
69
+ args: ['--no-sandbox', '--disable-setuid-sandbox'],
70
+ ...(config.capture.launchOptions ?? {}),
71
+ });
72
+
73
+ let captures;
74
+ try {
75
+ captures = await captureAll(browser, config, urls);
76
+ } finally {
77
+ await browser.close();
78
+ }
79
+
80
+ console.log(`\n[darshana] Captured ${captures.length} page(s).`);
81
+
82
+ if (captures.length === 0) {
83
+ console.error('[darshana] No pages captured. Exiting.');
84
+ process.exit(1);
85
+ }
86
+
87
+ const outputDir = config.outputDir;
88
+ fs.mkdirSync(outputDir, { recursive: true });
89
+
90
+ const outputs = config.outputs ?? ['pdf'];
91
+
92
+ if (outputs.includes('pdf')) {
93
+ await assemblePdf(captures, config, outputDir);
94
+ }
95
+
96
+ if (outputs.includes('html')) {
97
+ await assembleHtml(captures, config, outputDir);
98
+ }
99
+
100
+ if (outputs.includes('images')) {
101
+ await writeImages(captures, outputDir);
102
+ }
103
+
104
+ console.log('\n[darshana] Done.');
105
+ }
106
+
107
+ async function writeImages(captures, outputDir) {
108
+ const byViewport = {};
109
+ for (const capture of captures) {
110
+ (byViewport[capture.viewport] = byViewport[capture.viewport] || []).push(capture);
111
+ }
112
+ let total = 0;
113
+ for (const [viewport, vpCaptures] of Object.entries(byViewport)) {
114
+ const vpDir = path.join(outputDir, 'images', viewport);
115
+ fs.mkdirSync(vpDir, { recursive: true });
116
+ vpCaptures.forEach((capture, i) => {
117
+ const slug = slugifyPathname(capture.pathname);
118
+ const filename = `${String(i + 1).padStart(3, '0')}-${slug}-${capture.theme}.png`;
119
+ fs.writeFileSync(path.join(vpDir, filename), capture.imageBuffer);
120
+ console.log(` [images] ${viewport}/${filename}`);
121
+ total++;
122
+ });
123
+ }
124
+ console.log(`\n[images] Wrote ${total} PNG(s) → ${path.join(outputDir, 'images')}`);
125
+ }
126
+
127
+ function slugifyPathname(pathname) {
128
+ return (pathname ?? '/')
129
+ .replace(/^\//, '')
130
+ .replace(/\//g, '-')
131
+ .replace(/\s+/g, '-')
132
+ .replace(/[^a-zA-Z0-9-]/g, '')
133
+ .replace(/-+/g, '-')
134
+ .replace(/^-|-$/g, '')
135
+ || 'root';
136
+ }
137
+
138
+ main().catch(err => {
139
+ console.error('[darshana] Fatal:', err);
140
+ process.exit(1);
141
+ });
package/src/pdf.mjs ADDED
@@ -0,0 +1,69 @@
1
+ import fs from 'node:fs';
2
+ import path from 'node:path';
3
+ import { PDFDocument, rgb, StandardFonts } from 'pdf-lib';
4
+
5
+ const HEADER_HEIGHT = 28;
6
+ const HEADER_BG = rgb(0, 0, 0);
7
+ const HEADER_FG = rgb(1, 1, 1);
8
+ const FONT_SIZE = 11;
9
+
10
+ export async function assemblePdf(pages, config, outputDir) {
11
+ const masterDoc = await PDFDocument.create();
12
+ const font = await masterDoc.embedFont(StandardFonts.Helvetica);
13
+ const date = new Date().toISOString().slice(0, 10);
14
+
15
+ // Cover page — sized to first capture's dimensions, or 1440×900 fallback
16
+ let coverWidth = 1440, coverHeight = 900;
17
+ if (pages.length > 0) {
18
+ try {
19
+ const firstImg = await masterDoc.embedPng(pages[0].imageBuffer);
20
+ const dims = firstImg.scale(1);
21
+ coverWidth = dims.width;
22
+ coverHeight = dims.height;
23
+ } catch (_) {}
24
+ }
25
+
26
+ const coverPage = masterDoc.addPage([coverWidth, coverHeight + HEADER_HEIGHT]);
27
+ coverPage.drawRectangle({ x: 0, y: 0, width: coverWidth, height: coverHeight + HEADER_HEIGHT, color: rgb(0.05, 0.05, 0.05) });
28
+ coverPage.drawText(config.title ?? 'Design Review', {
29
+ x: 60, y: coverHeight / 2 + 60, font, size: 36, color: rgb(1, 1, 1),
30
+ });
31
+ coverPage.drawText(config.url, {
32
+ x: 60, y: coverHeight / 2 + 10, font, size: 16, color: rgb(0.7, 0.7, 0.7),
33
+ });
34
+ coverPage.drawText(`${date} · ${pages.length} pages`, {
35
+ x: 60, y: coverHeight / 2 - 30, font, size: 14, color: rgb(0.5, 0.5, 0.5),
36
+ });
37
+
38
+ // Embed each screenshot
39
+ for (const capture of pages) {
40
+ console.log(` [pdf] embedding: ${capture.label}`);
41
+ try {
42
+ const img = await masterDoc.embedPng(capture.imageBuffer);
43
+ const { width: imgWidth, height: imgHeight } = img.scale(1);
44
+ const pgWidth = imgWidth;
45
+ const pgHeight = imgHeight + HEADER_HEIGHT;
46
+ const page = masterDoc.addPage([pgWidth, pgHeight]);
47
+
48
+ // Draw screenshot filling below the header
49
+ page.drawImage(img, { x: 0, y: 0, width: pgWidth, height: imgHeight });
50
+
51
+ // Header bar at top (y=0 is bottom in pdf-lib, so header sits at y=imgHeight)
52
+ page.drawRectangle({
53
+ x: 0, y: imgHeight, width: pgWidth, height: HEADER_HEIGHT, color: HEADER_BG,
54
+ });
55
+ page.drawText(capture.label, {
56
+ x: 8, y: imgHeight + 8, font, size: FONT_SIZE, color: HEADER_FG, maxWidth: pgWidth - 16,
57
+ });
58
+ } catch (err) {
59
+ console.warn(` [pdf] WARNING: failed to embed ${capture.label}: ${err.message}`);
60
+ }
61
+ }
62
+
63
+ const outputPath = path.join(outputDir, 'console-review.pdf');
64
+ const pdfBytes = await masterDoc.save();
65
+ fs.writeFileSync(outputPath, pdfBytes);
66
+
67
+ const sizeMB = (fs.statSync(outputPath).size / 1024 / 1024).toFixed(1);
68
+ console.log(`\n[pdf] Wrote ${masterDoc.getPageCount()} pages (${sizeMB} MB) → ${outputPath}`);
69
+ }