dembrandt 0.2.0 → 0.4.0

package/README.md

@@ -4,403 +4,101 @@
 [![npm downloads](https://img.shields.io/npm/dm/dembrandt.svg)](https://www.npmjs.com/package/dembrandt)
 [![license](https://img.shields.io/npm/l/dembrandt.svg)](https://github.com/thevangelist/dembrandt/blob/main/LICENSE)
 
-A CLI tool for extracting design tokens and brand assets from any website. Powered by Playwright with advanced bot detection avoidance.
+Extract any website’s design system into design tokens in a few seconds: logo, colors, typography, borders, and more. One command.
 
 ![Dembrandt Demo](showcase.png)
 
-## Quick Start
+## Install
 
 ```bash
-npx dembrandt stripe.com
+npx dembrandt bmw.de
 ```
 
-No installation required! Extract design tokens from any website in seconds. Or install globally with `npm install -g dembrandt`.
+Or install globally: `npm install -g dembrandt` then run `dembrandt bmw.de`
 
-## What It Does
+Requires Node.js 18+
 
-Dembrandt analyzes live websites and extracts their complete design system:
+## What to expect from extraction?
 
-- **Logo** — Logo detection (img/svg) with dimensions and source URL
-- **Favicons** — All favicon variants with sizes and types
-- **Colors** — Semantic colors, color palette with confidence scoring, CSS variables (both hex and RGB formats)
-- **Typography** — Font families, sizes, weights, line heights, font sources (Google Fonts, Adobe Fonts, custom)
-- **Spacing** — Margin and padding scales with grid system detection (4px/8px/custom)
-- **Border Radius** — Corner radius patterns with usage frequency
-- **Shadows** — Box shadow values for elevation systems
-- **Buttons** — Component styles with variants and states
-- **Inputs** — Form field styles (input, textarea, select)
-- **Links** — Link styles with hover states and decorations
-- **Breakpoints** — Responsive design breakpoints from media queries
-- **Icons** — Icon system detection (Font Awesome, Material Icons, SVG)
-- **Frameworks** — CSS framework detection (Tailwind, Bootstrap, Material-UI, Chakra)
-
-Perfect for competitive analysis, brand audits, or rebuilding a brand when you don't have design guidelines.
-
-## Why It Matters
-
-**Designers** — Analyze competitor systems, document production tokens, audit brand consistency
-
-**Developers** — Migrate design tokens, reverse engineer components, validate implementations
-
-**Product Managers** — Track competitor evolution, quantify design debt, evaluate vendors
-
-**Marketing** — Audit competitor brands, plan rebrands, monitor brand compliance
-
-**Engineering Leaders** — Measure technical debt, plan migrations, assess acquisition targets
-
-## How It Works
-
-Uses Playwright to render the page, extracts computed styles from the DOM, analyzes color usage and confidence, groups similar typography, detects spacing patterns, and returns actionable design tokens.
-
-### Extraction Process
-
-1. **Browser Launch** - Launches Chromium with stealth configuration
-2. **Anti-Detection** - Injects scripts to bypass bot detection
-3. **Navigation** - Navigates to target URL with retry logic
-4. **Hydration** - Waits for SPAs to fully load (8s initial + 4s stabilization)
-5. **Content Validation** - Verifies page content is substantial (>500 chars)
-6. **Parallel Extraction** - Runs all extractors concurrently for speed
-7. **Analysis** - Analyzes computed styles, DOM structure, and CSS variables
-8. **Scoring** - Assigns confidence scores based on context and usage
-
-### Color Confidence
-
-- **High** — Logo, brand elements, primary buttons
-- **Medium** — Interactive elements, icons, navigation
-- **Low** — Generic UI components (filtered from display)
-
-Only shows high and medium confidence colors in terminal. Full palette in JSON.
-
-### Typography Detection
-
-Samples all heading levels (h1-h6), body text, buttons, links. Groups by font family, size, and weight. Detects Google Fonts, Adobe Fonts, custom @font-face.
-
-### Framework Detection
-
-Recognizes Tailwind CSS, Bootstrap, Material-UI, and others by class patterns and CDN links.
-
-## Installation
-
-### Using npx (Recommended)
-
-No installation needed! Run directly with `npx`:
-
-```bash
-npx dembrandt stripe.com
-```
-
-The first run will automatically install Chromium (~170MB).
-
-### Global Installation
-
-Install globally for repeated use:
-
-```bash
-npm install -g dembrandt
-dembrandt stripe.com
-```
-
-### Prerequisites
-
-- Node.js 18 or higher
-
-### Development Setup
-
-For contributors who want to work on dembrandt:
-
-```bash
-git clone https://github.com/thevangelist/dembrandt.git
-cd dembrandt
-npm install
-npm link
-```
+- Colors (semantic, palette, CSS variables)
+- Typography (fonts, sizes, weights, sources)
+- Spacing (margin/padding scales)
+- Borders (radius, widths, styles, colors)
+- Shadows
+- Components (buttons, inputs, links)
+- Breakpoints
+- Icons & frameworks
 
 ## Usage
 
-### Basic Usage
-
-```bash
-# Using npx (no installation)
-npx dembrandt <url>
-
-# Or if installed globally
-dembrandt <url>
-
-# Examples
-dembrandt stripe.com
-dembrandt https://github.com
-dembrandt tailwindcss.com
-```
-
-### Options
-
-**`--json-only`** - Output raw JSON to stdout instead of formatted terminal display
-
-```bash
-dembrandt stripe.com --json-only > tokens.json
-```
-
-Note: JSON is automatically saved to `output/domain.com/` regardless of this flag.
-
-**`-d, --debug`** - Run with visible browser and detailed logs
-
-```bash
-dembrandt stripe.com --debug
-```
-
-Useful for troubleshooting bot detection, timeouts, or extraction issues.
-
-**`--verbose-colors`** - Show medium and low confidence colors in terminal output
-
-```bash
-dembrandt stripe.com --verbose-colors
-```
-
-By default, only high-confidence colors are shown. Use this flag to see all detected colors.
-
-**`--dark-mode`** - Extract colors from dark mode
-
-```bash
-dembrandt stripe.com --dark-mode
-```
-
-Enables dark mode preference detection for sites that support it.
-
-**`--mobile`** - Extract from mobile viewport
-
-```bash
-dembrandt stripe.com --mobile
-```
-
-Simulates a mobile device viewport for responsive design token extraction.
-
-## Output
-
-### Automatic JSON Saves
-
-Every extraction is automatically saved to `output/domain.com/YYYY-MM-DDTHH-MM-SS.json` with:
-
-- Complete design token data
-- Timestamped for version tracking
-- Organized by domain
-
-Example: `output/stripe.com/2025-11-22T14-30-45.json`
-
-### Terminal Output
-
-Clean, formatted tables showing:
-
-- Color palette with confidence ratings (with visual swatches)
-- CSS variables with color previews
-- Typography hierarchy with context
-- Spacing scale (4px/8px grid detection)
-- Shadow system
-- Button variants
-- Component style breakdowns
-- Framework and icon system detection
-
-### JSON Output Format
-
-Complete extraction data for programmatic use:
-
-```json
-{
-  "url": "https://example.com",
-  "extractedAt": "2025-11-22T...",
-  "logo": { "source": "img", "url": "...", "width": 120, "height": 40 },
-  "colors": {
-    "semantic": { "primary": "#3b82f6", ... },
-    "palette": [{ "color": "#3b82f6", "confidence": "high", "count": 45, "sources": [...] }],
-    "cssVariables": { "--color-primary": "#3b82f6", ... }
-  },
-  "typography": {
-    "styles": [{ "fontFamily": "Inter", "fontSize": "16px", "fontWeight": "400", ... }],
-    "sources": { "googleFonts": [...], "adobeFonts": false, "customFonts": [...] }
-  },
-  "spacing": { "scaleType": "8px", "commonValues": [{ "px": "16px", "rem": "1rem", "count": 42 }, ...] },
-  "borderRadius": { "values": [{ "value": "8px", "count": 15, "confidence": "high" }, ...] },
-  "shadows": [{ "shadow": "0 2px 4px rgba(0,0,0,0.1)", "count": 8, "confidence": "high" }, ...],
-  "components": {
-    "buttons": [{ "backgroundColor": "...", "color": "...", "padding": "...", ... }],
-    "inputs": [{ "type": "input", "border": "...", "borderRadius": "...", ... }]
-  },
-  "breakpoints": [{ "px": "768px" }, ...],
-  "iconSystem": [{ "name": "Font Awesome", "type": "icon-font" }, ...],
-  "frameworks": [{ "name": "Tailwind CSS", "confidence": "high", "evidence": "class patterns" }]
-}
-```
-
-## Examples
-
-### Extract Design Tokens
-
 ```bash
-# Analyze a single site (auto-saves JSON to output/stripe.com/)
-dembrandt stripe.com
-
-# View saved JSON files
-ls output/stripe.com/
-
-# Output to stdout for piping
-dembrandt stripe.com --json-only | jq '.colors.semantic'
-
-# Debug mode for difficult sites
-dembrandt example.com --debug
+dembrandt <url>                    # Basic extraction
+dembrandt bmw.de --save-output     # Save JSON to output folder
+dembrandt bmw.de --json-only       # JSON output only (no save)
+dembrandt bmw.de --debug           # Visible browser
+dembrandt bmw.de --dark-mode       # Dark mode
+dembrandt bmw.de --mobile          # Mobile viewport
+dembrandt bmw.de --slow            # 3x timeouts
 ```
 
-### Compare Competitors
-
-```bash
-# Extract tokens from multiple competitors (auto-saved to output/)
-for site in stripe.com square.com paypal.com; do
-  dembrandt $site
-done
-
-# Compare color palettes from most recent extractions
-jq '.colors.palette[] | select(.confidence=="high")' output/stripe.com/2025-11-22T*.json output/square.com/2025-11-22T*.json
-
-# Compare semantic colors across competitors
-jq '.colors.semantic' output/*/2025-11-22T*.json
-```
-
-### Integration with Design Tools
-
-```bash
-# Extract and convert to custom config format
-dembrandt mysite.com --json-only | jq '{
-  colors: .colors.semantic,
-  fontFamily: .typography.sources,
-  spacing: .spacing.commonValues
-}' > design-tokens.json
-```
+By default, results display in terminal only. Use `--save-output` to save JSON to `output/bmw.de/YYYY-MM-DDTHH-MM-SS.json`
 
 ## Use Cases
 
-### Brand Audits
-
-Extract and document your company's current design system from production websites.
-
-### Competitive Analysis
+- Brand audits & competitive analysis
+- Design system documentation
+- Reverse engineering brands
+- Multi-site brand consolidation
 
-Compare design systems across competitors to identify trends and opportunities.
-
-### Design System Migration
-
-Document legacy design tokens before migrating to a new system.
-
-### Reverse Engineering
-
-Rebuild a brand when original design guidelines are unavailable.
-
-### Quality Assurance
-
-Verify design consistency across different pages and environments.
-
-## Advanced Features
-
-### Bot Detection Avoidance
-
-- Stealth mode with anti-detection scripts
-- Automatic fallback to visible browser on detection
-- Human-like interaction simulation (mouse movement, scrolling)
-- Custom user agent and browser fingerprinting
-
-### Smart Retry Logic
-
-- Automatic retry on navigation failures (up to 2 attempts)
-- SPA hydration detection and waiting
-- Content validation to ensure page is fully loaded
-- Detailed progress logging at each step
-
-### Comprehensive Logging
-
-- Real-time spinner with step-by-step progress
-- Detailed extraction metrics (colors found, styles detected, etc.)
-- Error context with URL, stage, and attempt information
-- Debug mode with full stack traces
-
-## Troubleshooting
-
-### Bot Detection Issues
-
-If you encounter timeouts or network errors:
-
-```bash
-dembrandt example.com --debug
-```
-
-This will automatically retry with a visible browser.
-
-### Page Not Loading
-
-Some sites require longer load times. The tool waits 8 seconds for SPA hydration, but you can modify this in the source.
+## How It Works
 
-### Empty Content
+Uses Playwright to render the page, extracts computed styles from the DOM, analyzes color usage and confidence, groups similar typography, detects spacing patterns, and returns actionable design tokens.
 
-If content length is < 500 chars, the tool will automatically retry (up to 2 attempts).
+### Extraction Process
 
-### Debug Mode
+1. Browser Launch - Launches Chromium with stealth configuration
+2. Anti-Detection - Injects scripts to bypass bot detection
+3. Navigation - Navigates to target URL with retry logic
+4. Hydration - Waits for SPAs to fully load (8s initial + 4s stabilization)
+5. Content Validation - Verifies page content is substantial (>500 chars)
+6. Parallel Extraction - Runs all extractors concurrently for speed
+7. Analysis - Analyzes computed styles, DOM structure, and CSS variables
+8. Scoring - Assigns confidence scores based on context and usage
 
-Use `--debug` to see:
+### Color Confidence
 
-- Browser launch confirmation
-- Step-by-step progress logs
-- Full error stack traces
-- Extraction metrics
+- High — Logo, brand elements, primary buttons
+- Medium — Interactive elements, icons, navigation
+- Low — Generic UI components (filtered from display)
+- Only shows high and medium confidence colors in terminal. Full palette in JSON.
 
 ## Limitations
 
-- Dark mode requires `--dark-mode` flag (not automatically detected)
+- Dark mode requires --dark-mode flag (not automatically detected)
 - Hover/focus states extracted from CSS (not fully interactive)
 - Canvas/WebGL-rendered sites cannot be analyzed (e.g., Tesla, Apple Vision Pro demos)
 - JavaScript-heavy sites require hydration time (8s initial + 4s stabilization)
 - Some dynamically-loaded content may be missed
-- Default viewport is 1920x1080 (use `--mobile` for responsive analysis)
-
-## Architecture
-
-```
-dembrandt/
-├── index.js              # CLI entry point, command handling
-├── lib/
-│   ├── extractors.js     # Core extraction logic with stealth mode
-│   └── display.js        # Terminal output formatting
-├── output/               # Auto-saved JSON extractions (gitignored)
-│   ├── stripe.com/
-│   │   ├── 2025-11-22T14-30-45.json
-│   │   └── 2025-11-22T15-12-33.json
-│   └── github.com/
-│       └── 2025-11-22T14-35-12.json
-├── package.json
-└── README.md
-```
+- Default viewport is 1920x1080 (use --mobile for responsive analysis)
 
 ## Ethics & Legality
 
 Dembrandt extracts publicly available design information (colors, fonts, spacing) from website DOMs for analysis purposes. This falls under fair use in most jurisdictions (USA's DMCA § 1201(f), EU Software Directive 2009/24/EC) when used for competitive analysis, documentation, or learning.
 
-**Legal:** Analyzing public HTML/CSS is generally legal. Does not bypass protections or violate copyright. Check site ToS before mass extraction.
+Legal: Analyzing public HTML/CSS is generally legal. Does not bypass protections or violate copyright. Check site ToS before mass extraction.
 
-**Ethical:** Use for inspiration and analysis, not direct copying. Respect servers (no mass crawling), give credit to sources, be transparent about data origin.
+Ethical: Use for inspiration and analysis, not direct copying. Respect servers (no mass crawling), give credit to sources, be transparent about data origin.
 
 ## Contributing
 
-Issues and pull requests welcome. Please include:
+Bugs you found? Weird websites that make it cry? Pull requests (even one-liners make me happy)?
 
-- Clear description of the issue/feature
-- Example URLs that demonstrate the problem
-- Expected vs actual behavior
+Spam me in [Issues](https://github.com/thevangelist/dembrandt/issues) or PRs. I reply to everything.
 
-## License
+Let's keep the light alive together.
 
-MIT
+@thevangelist
 
-## Roadmap
+---
 
-- [x] Dark mode extraction (via `--dark-mode` flag)
-- [x] Mobile viewport support (via `--mobile` flag)
-- [x] Clickable terminal links for modern terminals
-- [ ] Animation/transition detection
-- [ ] Interactive state capture (hover, focus, active)
-- [ ] Multi-page analysis
-- [ ] Configuration file support
+MIT — do whatever you want with it.

package/index.js

@@ -10,7 +10,7 @@
 import { program } from "commander";
 import chalk from "chalk";
 import ora from "ora";
-import { chromium } from "playwright";
+import { chromium } from "playwright-core";
 import { extractBranding } from "./lib/extractors.js";
 import { displayResults } from "./lib/display.js";
 import { writeFileSync, mkdirSync } from "fs";
@@ -23,9 +23,11 @@ program
   .argument("<url>")
   .option("--json-only", "Output raw JSON")
   .option("-d, --debug", "Force visible browser")
+  .option("--save-output", "Save JSON file to output folder")
   .option("--verbose-colors", "Show medium and low confidence colors")
   .option("--dark-mode", "Extract colors from dark mode")
   .option("--mobile", "Extract from mobile viewport")
+  .option("--slow", "3x longer timeouts for slow-loading sites")
   .action(async (input, opts) => {
     let url = input;
     if (!url.match(/^https?:\/\//)) url = "https://" + url;
@@ -64,6 +66,7 @@ program
             navigationTimeout: 90000,
             darkMode: opts.darkMode,
             mobile: opts.mobile,
+            slow: opts.slow,
           });
           break;
         } catch (err) {
@@ -89,10 +92,10 @@ program
         }
       }
 
-      spinner.succeed("Done!");
+      console.log();
 
-      // Save JSON output automatically (unless --json-only)
-      if (!opts.jsonOnly) {
+      // Save JSON output only if --save-output is specified
+      if (opts.saveOutput && !opts.jsonOnly) {
         try {
           const domain = new URL(url).hostname.replace("www.", "");
           const timestamp = new Date()
@@ -109,14 +112,14 @@ program
 
           console.log(
             chalk.dim(
-              `\n💾 JSON saved to: ${chalk.cyan(
+              `💾 JSON saved to: ${chalk.hex('#8BE9FD')(
                 `output/${domain}/${filename}`
               )}`
             )
           );
         } catch (err) {
           console.log(
-            chalk.yellow(`⚠ Could not save JSON file: ${err.message}`)
+            chalk.hex('#FFB86C')(`⚠ Could not save JSON file: ${err.message}`)
           );
         }
       }
@@ -125,6 +128,7 @@ program
       if (opts.jsonOnly) {
         console.log(JSON.stringify(result, null, 2));
       } else {
+        console.log();
         displayResults(result, { verboseColors: opts.verboseColors });
       }
     } catch (err) {
@@ -140,7 +144,7 @@ program
 
       if (!opts.debug) {
         console.log(
-          chalk.yellow(
+          chalk.hex('#FFB86C')(
             "\nTip: Try with --debug flag for tough sites and detailed error logs"
           )
         );