framer-export 5.0.0-beta.1 → 5.0.0-beta.3

package/README.md

@@ -1,269 +1,173 @@
+<h1 align="center">F-EXPORT</h1>
+
 <p align="center">
-  <img src="https://img.shields.io/badge/TypeScript-3178C6?style=for-the-badge&logo=typescript&logoColor=white" alt="TypeScript" />
-  <img src="https://img.shields.io/badge/Node.js-339933?style=for-the-badge&logo=node.js&logoColor=white" alt="Node.js" />
-  <img src="https://img.shields.io/badge/Puppeteer-40B5A4?style=for-the-badge&logo=puppeteer&logoColor=white" alt="Puppeteer" />
-  <img src="https://img.shields.io/badge/version-5.0.0--beta.1-blue?style=for-the-badge" alt="Version" />
-  <img src="https://img.shields.io/badge/license-MIT-green?style=for-the-badge" alt="License" />
+  <img src="https://img.shields.io/badge/version-5.0.0--beta.3-blue" alt="Version" />
+  <img src="https://img.shields.io/badge/license-MIT-green" alt="License" />
+  <img src="https://img.shields.io/badge/node-%3E%3D20-339933" alt="Node" />
 </p>
 
-<h1 align="center">
-  <br />
-  F-EXPORT
-  <br />
-  <sub>Multi-Platform Website Exporter &middot; 25+ platforms</sub>
-</h1>
+<p align="center">Export a site built on Framer, Webflow, Wix or 22 other platforms into a local folder that actually works.</p>
 
-<p align="center">
-  <b>Export 25+ website platforms into a fully working local mirror.</b><br/>
-  All assets, animations, fonts, videos, and scripts — downloaded, rewritten, and pretty-printed.<br/>
-  Badges and tracking removed. Ready to serve.
-</p>
+## What this is
 
-<p align="center">
-  <a href="#-features">Features</a> &middot;
-  <a href="#-quick-start">Quick Start</a> &middot;
-  <a href="#-platforms">Platforms</a> &middot;
-  <a href="#-cli-reference">CLI</a> &middot;
-  <a href="#-architecture">Architecture</a> &middot;
-  <a href="#-contributing">Contributing</a> &middot;
-  <a href="#-license">License</a>
-</p>
+You have a site on a hosted platform. You want the real files: the HTML as the browser sees it, the CSS, the scripts, the images, the fonts, the videos. Not a screenshot, not a broken "save page as" folder, an actual mirror you can open locally, host anywhere, or hand to an AI agent to rebuild as a clean codebase.
 
----
+That is what framer-export does. It loads the page in a headless browser, waits for the framework to hydrate, captures the rendered result, downloads every asset it can find, rewrites all the CDN URLs to local relative paths, strips the platform badges and trackers, and writes a folder with a small `serve.js` so you can preview it immediately.
 
-## &#x2728; Features
+It knows 25 platforms and detects the right one from the URL, and when the URL is a custom domain it falls back to reading the HTML. If a site sits behind Cloudflare or a captcha, the export fails loudly with an explanation instead of giving you an empty folder.
 
-| Feature | Description |
-|---------|-------------|
-| **Multi-Platform** | Auto-detects 25+ platforms (Framer, Webflow, Wix, Shopify, Notion, WordPress, Squarespace, Ghost, …) |
-| **One File Per Platform** | Each exporter is a self-contained handler — adding one never touches the others |
-| **Anti-Bot Aware** | Cloudflare / captcha challenges fail loudly with a clear message; never an empty export |
-| **Full Mirror** | Downloads HTML, CSS, JS, images, fonts, and videos |
-| **Badge Removal** | Strips "Made in Webflow", Wix ads banner, Framer badge |
-| **URL Rewriting** | All CDN URLs rewritten to local relative paths |
-| **Pretty-Print** | Minified JS/MJS files reformatted with Prettier |
-| **Integrity Strip** | Removes `integrity` and `crossorigin` attributes for local serving |
-| **SEO Optimization** | Injects canonical, OG tags, and robots meta if missing |
-| **Interactive Setup** | Arrow-key wizard with platform detection and progress animation |
-| **Cooking Animation** | Shimmer gradient progress indicator during export |
-| **Local Server** | Built-in serve.js with SPA fallback and CORS headers |
-| **Smart Naming** | Output folder auto-named from URL (e.g. `webflow-mysite/`) |
+## Installation
 
-## &#x1F680; Quick Start
+```bash
+npm install -g framer-export
+```
+
+Or from source:
 
 ```bash
-# Clone and install
 git clone https://github.com/danbenba/FramerExport.git
 cd FramerExport
 npm install
+```
+
+Node 20 or newer. The first run downloads a Chromium build for Puppeteer, which takes a moment.
 
-# Interactive mode (recommended)
-npm run dev
+## Three ways to run it
 
-# Direct export
-npm run dev -- https://mysite.framer.app
-npm run dev -- https://mysite.webflow.io
-npm run dev -- https://user.wixsite.com/my-site
+### The web interface
 
-# Serve the exported site
-cd webflow-mysite && node serve.js
+```bash
+framer-export ui
 ```
 
-## &#x1F310; Platforms
+This starts a local server on port 4400 and opens your browser. You get a gallery of every supported platform. Click one (or pick auto-detect), paste the URL, adjust the options, and watch the export run with live logs on the left and a summary panel on the right showing the current phase, asset counters and the last files written. When it finishes you can copy the full log or the serve command with one click.
 
-### Framer
-- Auto-detected via `.framer.app`, `.framer.website`, `.framer.ai`
-- Handles framerusercontent.com, framerstatic.com, framercanvas.com
-- Strips Framer badge, analytics events, bootstrap scripts
-- Waits for SPA hydration (`#main` element)
+The server only listens on 127.0.0.1 and rejects cross-origin requests, so nothing on the network or in another browser tab can trigger exports on your machine. Use `--port <n>` to change the port and `--no-open` if you do not want the browser to open by itself.
 
-### Webflow
-- Auto-detected via `.webflow.io`, `.webflow.com`
-- Handles cdn.prod.website-files.com, cloudfront CDN, GSAP plugins
-- Strips "Made in Webflow" badge, "Powered by" footer, `data-wf-*` attributes
-- Removes generator meta tag and HTML comments
-- Downloads videos (MP4/WebM) and responsive image variants
+### The terminal wizard
 
-### Wix
-- Auto-detected via `.wixsite.com`, `.wix.com`
-- Handles static.wixstatic.com, static.parastorage.com, video.wixstatic.com
-- Strips WIX_ADS banner (nested div with SVG), wix-ads class, wix-badge
-- Blocks frog.wix.com, panorama.wixapps.net analytics
-- Preserves 40+ inline `<style>` tags (Wix architecture)
+```bash
+framer-export
+```
 
-## &#x1F4BB; CLI Reference
+Running it with no arguments opens the interactive setup. First you pick the tool from a scrollable list grouped by category, and you can just start typing to filter it. Then you enter the URL, then the output directory, and finally a single options panel with checkboxes for pretty-printing and sub-pages, plus a concurrency setting. If the URL looks like a different platform than the one you picked, it asks before continuing.
 
-```
-Usage:
-  framer-export <url> [output-dir]
-  framer-export --setup
-  framer-export --setup --legacy-mode
-
-Options:
-  --setup            Launch interactive setup wizard
-  --platform <name>  Force platform: framer | webflow | wix
-  --legacy-mode      Use y/n text input instead of arrow selection
-  --help, -h         Show help message
-```
+While the export runs, terminals wider than 100 columns get a live sidebar on the right with the phase, elapsed time, download counters and recent files. The log stream itself stays untouched.
 
-### Examples
+If arrow keys do not work in your terminal, `framer-export --setup --legacy-mode` falls back to plain text prompts.
+
+### The direct command
 
 ```bash
-# Auto-detect platform from URL
 framer-export https://mysite.framer.app
-
-# Force platform for custom domains
-framer-export --platform webflow https://custom-domain.com
-
-# Specify output directory
 framer-export https://mysite.webflow.io ./my-export
+framer-export --platform webflow https://my-custom-domain.com
+framer-export --subpages https://mysite.framer.app
+```
 
-# Interactive wizard with arrow-key selection
-framer-export --setup
+Auto-detection covers the hosted domains (`.framer.app`, `.webflow.io`, `.wixsite.com` and so on). For custom domains, pass `--platform` with the platform id, or let the HTML detection figure it out.
 
-# Legacy mode (y/n prompts)
-framer-export --setup --legacy-mode
+## CLI reference
+
+```
+framer-export <url> [output-dir]      export a site
+framer-export ui [--port <n>]         launch the web interface
+framer-export --setup                 launch the terminal wizard
+
+--platform <id>    force a platform (framer, webflow, wix, shopify, notion, ...)
+--subpages         crawl internal links and export every page
+--dpr <number>     capture device pixel ratio, default 1
+--legacy-mode      with --setup, use plain text prompts
+--no-open          with ui, do not open the browser
+--about            version and package information
+--version, -v      version number
+--help, -h         full help with the platform list
 ```
 
-### npm Scripts
+## Supported platforms
 
-```bash
-npm run dev       # Run with tsx (development)
-npm start         # Same as dev
-npm run build     # Bundle with tsup
-npm run typecheck # Type-check with tsc
-npm run format    # Format with Prettier
-```
+Website builders: Framer, Webflow, Wix, Bubble, Carrd, Strikingly, Duda, Squarespace, Weebly, Tilda.
 
-## &#x1F4C1; Output Structure
+Landing page tools: ClickFunnels, Instapage, Unbounce, Systeme.io, Elementor.
 
-```
-webflow-mysite/
-├── index.html           # Main page (URLs rewritten, badges stripped)
-├── serve.js            # Local HTTP server with SPA fallback
-├── styles/              # CSS files
-├── scripts/
-│   ├── vendor/          # Platform JS modules (pretty-printed)
-│   └── modules/         # Component modules
-├── assets/
-│   ├── images/          # PNG, JPG, SVG, WebP, AVIF
-│   ├── videos/          # MP4, WebM
-│   ├── fonts/           # WOFF2, WOFF, TTF, OTF
-│   └── misc/            # Other assets
-└── data/                # CMS data, JSON, search index
-```
+CMS: Notion, Ghost, WordPress.
+
+Course platforms: Kajabi, Teachable, Thinkific, Podia.
+
+E-commerce: Gumroad, Shopify.
+
+AI builders: Gamma.
+
+Framer, Webflow and Wix are the stable trio, tested on real production sites. Everything else carries a beta badge in the gallery and in `--help`: the handlers work, but they have seen fewer real-world sites, so expect the occasional rough edge and report what you find.
 
-## &#x1F3D7; Architecture
+Each platform is a single self-contained handler in `src/platforms/`. It declares how to detect the platform, which domains and selectors to strip, how long to wait for hydration, how to route assets into folders, and optional hooks that run before capture, after capture and after the build. Adding a platform never touches the others.
+
+## What you get
 
 ```
-src/
-├── cli/                 # Command-line interface
-│   ├── index.ts         # Entry point, flag parsing
-│   ├── banner.ts        # ASCII art display
-│   ├── help.ts          # Help text with examples
-│   ├── setup.ts         # Interactive wizard
-│   ├── select.ts        # Arrow-key selection component
-│   └── cooking.ts       # Shimmer gradient animation
-├── platforms/           # Platform-specific handlers
-│   ├── types.ts         # PlatformHandler interface
-│   ├── framer.ts        # Framer detection, mapping, stripping
-│   ├── webflow.ts       # Webflow detection, mapping, stripping
-│   ├── wix.ts           # Wix detection, mapping, stripping
-│   ├── detect.ts        # Auto-detection by URL and HTML
-│   └── index.ts         # Barrel exports
-├── exporter/            # Export engine
-│   ├── index.ts         # FramerExporter orchestrator
-│   ├── capture.ts       # Puppeteer browser capture
-│   ├── download.ts      # Parallel asset downloader
-│   ├── output.ts        # HTML processing and file output
-│   └── summary.ts       # Export report
-├── assets/
-│   └── asset-map.ts     # URL-to-local-path mapping
-├── network/
-│   ├── download.ts      # HTTP download with retries
-│   └── pool.ts          # Concurrency pool
-├── formatter/
-│   └── prettify.ts      # Prettier-based JS formatter
-├── logger/
-│   └── index.ts         # Colored logging with cooking integration
-├── server/
-│   └── template.ts      # Embedded HTTP server template
-├── config/
-│   └── index.ts         # Global configuration
-└── types.ts             # Shared TypeScript interfaces
+framer-mysite-fresh-build-a1b2/
+  index.html          the rendered page, URLs rewritten to local paths
+  serve.js            a small static server with SPA fallback
+  package.json        so "npm run serve" works
+  export.log          the complete log of the run
+  styles/             CSS files
+  scripts/vendor/     third-party bundles
+  scripts/modules/    page modules and lazy chunks
+  assets/images/      images, responsive variants included
+  assets/fonts/       font files
+  assets/videos/      video files
+  assets/misc/        everything else
+  data/               JSON and data files
+  subpages/           one HTML file per crawled page, when --subpages is on
 ```
 
-## &#x1F527; How It Works
+To preview it:
 
-1. **SSR Fetch** - Raw HTTP GET of the page HTML (before JavaScript execution)
-2. **Browser Capture** - Puppeteer loads the page, intercepts all network responses
-3. **Hydration Wait** - Platform-specific wait for SPA rendering (Framer) or static render (Webflow/Wix)
-4. **Lazy Load Scroll** - Full-page scroll triggers lazy-loaded images and videos
-5. **Asset Download** - All intercepted resources downloaded with concurrency pool
-6. **Badge Strip** - Platform-specific badges, ads, and tracking removed
-7. **Integrity Strip** - SHA integrity hashes and CORS attributes removed for local serving
-8. **URL Rewrite** - All CDN URLs in HTML, CSS, and JS rewritten to local paths
-9. **Pretty-Print** - Minified JS files reformatted with Prettier for readability
-10. **Output** - Clean `index.html` + `serve.js` written to output directory
+```bash
+cd framer-mysite-fresh-build-a1b2
+node serve.js
+```
 
-## &#x1F91D; Contributing
+The site has to be served over HTTP because module scripts do not load from `file://`. The bundled server handles MIME types, CORS headers and the sub-page fallback, so a route like `/about` resolves to `subpages/about.html` automatically.
 
-Contributions are welcome! Here's how to get started:
+Every run also writes `export.log` with the full untruncated log history. The finish panel in the terminal and the web interface both offer to copy it to the clipboard, which is handy when you want to report a problem or feed the context to an AI tool.
 
-1. Fork the repository
-2. Create a feature branch: `git checkout -b feat/my-feature`
-3. Make your changes
-4. Run type-check: `npm run typecheck`
-5. Format code: `npm run format`
-6. Commit with conventional commits: `git commit -m "feat(platform): add Squarespace support"`
-7. Push and open a PR
+## The AI conversion assistant
 
-### Adding a New Platform
+After a terminal export you can generate a conversion brief for an AI coding agent. Pick a target stack (React with Vite, Next.js, Vue, SvelteKit or Astro), pick the tool you use (Claude Code, Codex, OpenCode or another agent), pick a goal (clean rebuild, pixel-perfect migration, component system, or performance and SEO), and it writes a detailed prompt file into `ai/` inside the export. The prompt references the real files and counts from your export, so the agent starts from facts instead of guesses.
 
-1. Create `src/platforms/yourplatform.ts` implementing `PlatformHandler`
-2. Add it to `src/platforms/detect.ts` and `src/platforms/index.ts`
-3. Test with a real site URL
+## How it works
 
-### Commit Convention
+The pipeline has six phases. First it fetches the server-rendered HTML over plain HTTP, which is what search engines see and what gives the cleanest markup. Then it launches Puppeteer, blocks the analytics domains the platform handler lists, navigates, waits for the hydration selector, scrolls through the page to trigger lazy loading, and records every network response into an asset map. Sub-page crawling reuses the same browser session when enabled.
 
-```
-feat(scope):     New feature
-fix(scope):      Bug fix
-perf(scope):     Performance improvement
-refactor(scope): Code refactoring
-chore(scope):    Tooling, deps, config
-```
+Once the browser closes, the downloader writes all unique assets to disk with a configurable concurrency, then follows `import` statements inside downloaded JS chunks to resolve lazily loaded modules the browser never requested. The build step rewrites every URL to a local relative path, strips badges, trackers and integrity attributes, injects canonical and Open Graph tags when they are missing, pretty-prints the JavaScript unless you turned that off, and writes `index.html`, `serve.js` and `export.log`.
 
-## &#x1F4DC; License
+## When something goes wrong
 
-MIT License - see [LICENSE](LICENSE) for details.
+If a site is protected by Cloudflare, hCaptcha or a similar challenge, the export stops with a clear message rather than saving the challenge page as if it were the site. There is no bypass built in; run the export from a network the site trusts, or use a platform-hosted URL instead of the proxied custom domain.
 
-## &#x1F4CB; Code of Conduct
+If the page comes out incomplete, try `--dpr 2` for sharper image variants, or check the log for `Download failed` lines. Some platforms serve assets from session-bound URLs that expire; re-running the export usually resolves it.
 
-### Our Pledge
+If detection picks the wrong platform on a custom domain, pass `--platform` explicitly. Detection priorities are conservative on purpose: a hosted domain always wins over a generator meta tag.
 
-We are committed to providing a friendly, safe, and welcoming environment for all, regardless of level of experience, gender identity, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, or nationality.
+## Development
 
-### Our Standards
+```bash
+npm run dev          # run the CLI from source
+npm test             # 150 unit and integration tests, node test runner
+npm run typecheck    # tsc --noEmit
+npm run build        # bundle with tsup into dist/
+npm run format       # prettier over src/
+```
 
-**Positive behavior includes:**
-- Using welcoming and inclusive language
-- Being respectful of differing viewpoints and experiences
-- Gracefully accepting constructive criticism
-- Focusing on what is best for the community
+The test suite covers platform detection for all 25 handlers against recorded research profiles, asset mapping, URL rewriting, the logger, the generated serve.js (spawned for real and probed over HTTP), the progress state and every route of the UI server, including its origin and host checks.
 
-**Unacceptable behavior includes:**
-- Trolling, insulting, or derogatory comments
-- Public or private harassment
-- Publishing others' private information without permission
-- Other conduct which could reasonably be considered inappropriate
+To add a platform, create one file in `src/platforms/<category>/`, implement the `PlatformHandler` interface, register it in `src/platforms/registry.ts`, and add a research profile in `tests/research/` so the detection tests cover it. Look at `src/platforms/builder/carrd.ts` for a small example and `src/platforms/framer.ts` for a complete one.
 
-### Enforcement
+Commits follow the conventional format: `feat(scope):`, `fix(scope):`, `test:`, `docs:`, `chore:`.
 
-Project maintainers are responsible for clarifying standards of acceptable behavior and will take appropriate action in response to unacceptable behavior. Violations may be reported by opening an issue or contacting the maintainers.
+## License
 
----
+MIT. See [LICENSE](LICENSE).
 
-<p align="center">
-  Made with &#x2764; by <a href="https://github.com/danbenba">Dany</a>
-</p>
+Built by [Dany (danbenba)](https://github.com/danbenba).