smippo 0.0.9 → 0.1.1

package/README.md

@@ -27,45 +27,6 @@
 
 📚 **[View complete documentation →](https://smippo.com)**
 
-## Table of Contents
-
-- [Table of Contents](#table-of-contents)
-- [Features](#features)
-- [Quick Start](#quick-start)
-- [Installation](#installation)
-  - [Requirements](#requirements)
-  - [npm (Global)](#npm-global)
-  - [Homebrew (Coming soon)](#homebrew-coming-soon)
-- [Usage](#usage)
-  - [Basic Usage](#basic-usage)
-  - [Interactive Mode](#interactive-mode)
-  - [Filtering](#filtering)
-  - [Scope Control](#scope-control)
-  - [Browser Options](#browser-options)
-  - [Screenshots](#screenshots)
-  - [Authentication](#authentication)
-  - [Output Options](#output-options)
-  - [Performance \& Parallelism: The Vacuum Architecture](#performance--parallelism-the-vacuum-architecture)
-  - [Continue/Update](#continueupdate)
-  - [Serve](#serve)
-  - [Static Mode](#static-mode)
-- [Structured Output](#structured-output)
-- [Programmatic API](#programmatic-api)
-- [Contributing](#contributing)
-- [License](#license)
-- [Acknowledgments](#acknowledgments)
-
-## Features
-
-- **🚀 Vacuum Architecture** — Parallel workers consume sites rapidly, just like hippos vacuum up everything in their path
-- **📸 Structured Mirroring** — Every page, every resource, every network request captured in organized, structured output
-- **🔍 Complete Fidelity** — Gets the page exactly as you see it, including CSS-in-JS, dynamic content, and lazy-loaded images
-- **🎯 Smart Consumption** — Respects robots.txt, filters by URL patterns, MIME types, and file sizes
-- **📦 Structured Output** — Organized mirror structure preserves original paths for seamless offline browsing
-- **🎨 Beautiful CLI** — Interactive guided mode, progress bars, and elegant terminal output
-- **🌐 Built-in Server** — Serve captured sites locally with directory browsing
-- **📊 HAR Files** — Generates HTTP Archive files for debugging and replay
-
 ## Quick Start
 
 Install globally:
@@ -92,336 +53,48 @@ Or use without installing:
 npx smippo https://example.com
 ```
 
-> 📖 **For complete documentation, guides, and API reference, visit [smippo.com](https://smippo.com)**
-
-## Installation
-
-### Requirements
-
-- Node.js 18 or later
-- Chromium (automatically downloaded on first install)
-
-### npm (Global)
-
-```bash
-npm install -g smippo
-```
-
-### Homebrew (Coming soon)
-
-```bash
-brew install smippo
-```
-
-## Usage
-
-### Basic Usage
-
-```bash
-# Capture a single page with all assets
-smippo https://example.com
-
-# Mirror a site with depth control
-smippo https://example.com --depth 3
-
-# Save to custom directory
-smippo https://example.com --output ./my-mirror
-```
-
-### Interactive Mode
-
-Just run `smippo` with no arguments to start the guided wizard:
-
-```bash
-smippo
-```
-
-This will walk you through:
-
-- URL to capture
-- Crawl depth
-- Scope settings
-- Asset options
-- Advanced configuration
-
-Perfect for beginners or when you want to explore options!
-
-### Filtering
-
-```bash
-# Include only specific patterns
-smippo https://example.com --include "*.html" --include "*.css"
-
-# Exclude patterns
-smippo https://example.com --exclude "*tracking*" --exclude "*ads*"
-
-# Filter by MIME type
-smippo https://example.com --mime-include "image/*" --mime-exclude "video/*"
-
-# Filter by file size
-smippo https://example.com --max-size 5MB --min-size 1KB
-```
-
-### Scope Control
-
-```bash
-# Stay on same subdomain (default)
-smippo https://www.example.com --scope subdomain
-
-# Allow all subdomains
-smippo https://www.example.com --scope domain
-
-# Go everywhere (use with caution!)
-smippo https://example.com --scope all --depth 2
-```
-
-### Browser Options
+## Commands
 
-```bash
-# Wait for specific condition
-smippo https://example.com --wait networkidle
-smippo https://example.com --wait domcontentloaded
-
-# Add extra wait time for slow sites
-smippo https://example.com --wait-time 5000
-
-# Custom user agent
-smippo https://example.com --user-agent "Mozilla/5.0..."
+Smippo provides several commands for different use cases:
 
-# Custom viewport
-smippo https://example.com --viewport 1280x720
-
-# Emulate device
-smippo https://example.com --device "iPhone 13"
-```
-
-### Screenshots
-
-Take quick screenshots without mirroring the full site:
-
-```bash
-# Basic screenshot
-smippo capture https://example.com
+- **`smippo <url>`** — Capture and mirror websites with full fidelity
+- **`smippo capture <url>`** — Take screenshots of web pages
+- **`smippo serve <directory>`** — Serve captured sites locally
+- **`smippo continue`** — Resume an interrupted capture
+- **`smippo update`** — Update an existing mirror
 
-# Full-page screenshot (captures entire scrollable page)
-smippo capture https://example.com --full-page
+Run `smippo` with no arguments to start the interactive guided mode.
 
-# Save to specific file
-smippo capture https://example.com -O ./screenshots/example.png
-
-# Mobile device screenshot
-smippo capture https://example.com --device "iPhone 13" -O mobile.png
-
-# Screenshot with dark mode
-smippo capture https://example.com --dark-mode
-
-# Capture specific element
-smippo capture https://example.com --selector ".hero-section"
-
-# JPEG format with quality
-smippo capture https://example.com --format jpeg --quality 90
-```
-
-### Authentication
-
-```bash
-# Basic auth
-smippo https://user:pass@example.com
-
-# Cookie-based auth
-smippo https://example.com --cookies cookies.json
-
-# Interactive login (opens browser window)
-smippo https://example.com --capture-auth
-```
-
-### Output Options
-
-```bash
-# Generate screenshots
-smippo https://example.com --screenshot
-
-# Generate PDFs
-smippo https://example.com --pdf
-
-# Skip HAR file
-smippo https://example.com --no-har
-
-# Output structure
-smippo https://example.com --structure original  # URL paths (default)
-smippo https://example.com --structure flat      # All in one directory
-smippo https://example.com --structure domain    # Organized by domain
-```
-
-### Performance & Parallelism: The Vacuum Architecture
-
-Smippo's parallel worker architecture mirrors how hippos consume everything in their path—rapidly and efficiently. Multiple workers operate simultaneously, each vacuuming up pages, resources, and network requests in parallel.
-
-```bash
-# Default: 8 parallel workers (8 hippos vacuuming simultaneously)
-smippo https://example.com
-
-# Limit to 4 workers (for rate-limited sites)
-smippo https://example.com --workers 4
-
-# Single worker (sequential, safest)
-smippo https://example.com --workers 1
-
-# Maximum speed (use with caution)
-smippo https://example.com --workers 16
-
-# Limit total pages
-smippo https://example.com --max-pages 100
-
-# Limit total time
-smippo https://example.com --max-time 300  # 5 minutes
-
-# Rate limiting (delay between requests per worker)
-smippo https://example.com --rate-limit 1000  # 1 second between requests
-```
-
-**The Vacuum Architecture:**
-
-Each worker operates like an independent hippo, vacuuming up:
-
-- Fully rendered pages (after JavaScript execution)
-- All network resources (images, fonts, stylesheets, API responses)
-- Network metadata (captured in HAR files)
-- Link structures (for recursive crawling)
-
-All captured content is then **structured** into organized mirrors that preserve original paths and relationships.
-
-**Tips for optimal performance:**
-
-- Use `--workers 1` for sites with strict rate limiting
-- Use `--workers 4-8` for most sites (default: 8)
-- Use `--workers 16` only for fast servers you control
-- Combine `--workers` with `--rate-limit` for polite crawling
-
-### Continue/Update
-
-```bash
-# Continue an interrupted capture
-smippo continue
-
-# Update an existing mirror
-smippo update
-```
-
-### Serve
-
-Serve captured sites locally with a built-in web server:
-
-```bash
-# Serve with auto port detection
-smippo serve ./site
-
-# Specify port
-smippo serve ./site --port 3000
-
-# Open browser automatically
-smippo serve ./site --open
-
-# Show all requests
-smippo serve ./site --verbose
-```
-
-The server provides:
-
-- **Auto port detection** — Finds next available port if default is busy
-- **Proper MIME types** — Correct content-type headers for all file types
-- **CORS support** — Enabled by default for local development
-- **Nice terminal UI** — Shows clickable URL and request logs
-
-### Static Mode
-
-For any site, use `--static` to strip scripts for true offline viewing:
-
-```bash
-# Capture as static HTML (removes JS, keeps rendered content)
-smippo https://example.com --static --external-assets
-
-# Then serve
-smippo serve ./site --open
-```
-
-## Structured Output
-
-Smippo creates **structured mirrors** that preserve the original URL structure and relationships. Every page, every resource, every network request is organized and stored in a logical hierarchy:
-
-```
-site/
-├── example.com/
-│   ├── index.html
-│   ├── about/
-│   │   └── index.html
-│   └── assets/
-│       ├── style.css
-│       └── logo.png
-├── .smippo/
-│   ├── cache.json          # Metadata cache
-│   ├── network.har         # HAR file
-│   ├── manifest.json       # Capture manifest
-│   └── log.txt             # Capture log
-└── index.html              # Entry point
-```
-
-## Programmatic API
-
-```javascript
-import {capture, Crawler, createServer} from 'smippo';
-
-// Simple capture
-const result = await capture('https://example.com', {
-  output: './mirror',
-  depth: 2,
-});
-
-console.log(`Captured ${result.stats.pagesCapt} pages`);
-
-// Advanced usage with events
-const crawler = new Crawler({
-  url: 'https://example.com',
-  output: './mirror',
-  depth: 3,
-  scope: 'domain',
-});
-
-crawler.on('page:complete', ({url, size}) => {
-  console.log(`Captured: ${url} (${size} bytes)`);
-});
+## Features
 
-crawler.on('error', ({url, error}) => {
-  console.error(`Failed: ${url} - ${error.message}`);
-});
+- **🚀 Vacuum Architecture** — Parallel workers consume sites rapidly
+- **📸 Complete Fidelity** — Captures pages exactly as rendered, including CSS-in-JS, dynamic content, and lazy-loaded images
+- **🎯 Smart Filtering** — Filter by URL patterns, MIME types, and file sizes. Respects robots.txt
+- **🌐 Built-in Server** — Serve captured sites locally with directory browsing
+- **📊 HAR Files** — Generates HTTP Archive files for debugging and replay
+- **💻 Programmatic API** — Use Smippo in your Node.js applications
 
-await crawler.start();
+## Documentation
 
-// Start a server programmatically
-const server = await createServer({
-  directory: './mirror',
-  port: 8080,
-  open: true, // Opens browser automatically
-});
+For complete documentation, guides, and API reference, visit **[smippo.com](https://smippo.com)**:
 
-console.log(`Server running at ${server.url}`);
+- **[Installation Guide](https://smippo.com/getting-started/installation)** — Detailed installation instructions
+- **[Commands Reference](https://smippo.com/commands)** — All available commands and options
+- **[Configuration](https://smippo.com/configuration)** — Filtering, scope control, performance tuning
+- **[Guides](https://smippo.com/guides)** — Output structure, link rewriting, troubleshooting
+- **[Programmatic API](https://smippo.com/api/programmatic)** — Use Smippo in your Node.js code
+- **[Examples](https://smippo.com/getting-started/examples)** — Real-world use cases
 
-// Later: stop the server
-await server.close();
-```
+## Requirements
 
-> 📖 **For complete API documentation, see the [Programmatic API guide](https://smippo.com/api-reference/programmatic-api) on smippo.com**
+- Node.js 18 or later
+- Chromium (automatically downloaded on first install)
 
 ## Contributing
 
 Contributions are welcome! Whether it's bug reports, feature requests, or pull requests — all contributions help make Smippo better.
 
-Please read our [Contributing Guide](CONTRIBUTING.md) for details on:
-
-- Development setup
-- Code style guidelines
-- Pull request process
-- Testing requirements
+Please read our [Contributing Guide](CONTRIBUTING.md) for details on development setup, code style guidelines, and the pull request process.
 
 Quick start:
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "smippo",
-  "version": "0.0.9",
+  "version": "0.1.1",
   "description": "S.M.I.P.P.O. — Structured Mirroring of Internet Pages and Public Objects. Modern website copier that captures sites exactly as they appear in your browser.",
   "main": "src/index.js",
   "bin": {

package/src/cli.js

@@ -10,6 +10,12 @@ import {
   runInteractiveCapture,
   shouldRunInteractive,
 } from './interactive.js';
+import {
+  getSiteDir,
+  getDomainFromUrl,
+  addSiteToGlobalManifest,
+  ensureSmippoHome,
+} from './utils/home.js';
 
 const program = new Command();
 
@@ -52,7 +58,10 @@ export function run() {
   // Main capture command
   program
     .argument('[url]', 'URL to capture')
-    .option('-o, --output <dir>', 'Output directory', './site')
+    .option(
+      '-o, --output <dir>',
+      'Output directory (default: ~/.smippo/sites/[domain])',
+    )
     .option('-d, --depth <n>', 'Recursion depth (0 = single page)', '0')
     .option('--no-crawl', 'Disable link following (same as -d 0)')
     .option('--dry-run', 'Show what would be captured without downloading')
@@ -80,8 +89,47 @@ export function run() {
       'Wait strategy: networkidle|load|domcontentloaded',
       'networkidle',
     )
-    .option('--wait-time <ms>', 'Additional wait time after network idle', '0')
+    .option(
+      '--wait-time <ms>',
+      'Additional wait time after network idle',
+      '500',
+    )
     .option('--timeout <ms>', 'Page load timeout', '30000')
+
+    // Scroll and reveal options (for capturing dynamic content)
+    .option(
+      '--scroll',
+      'Pre-scroll page to trigger lazy content (default: true)',
+    )
+    .option('--no-scroll', 'Disable pre-scroll behavior')
+    .option(
+      '--scroll-wait <ms>',
+      'Wait time after scrolling for animations',
+      '1000',
+    )
+    .option(
+      '--scroll-step <px>',
+      'Pixels per scroll increment (default: 200)',
+      '200',
+    )
+    .option(
+      '--scroll-delay <ms>',
+      'Delay between scroll steps (default: 50)',
+      '50',
+    )
+    .option(
+      '--scroll-behavior <type>',
+      'Scroll behavior: smooth|instant (default: smooth)',
+      'smooth',
+    )
+    .option(
+      '--reveal-all',
+      'Force reveal scroll-triggered content like GSAP, AOS (default: true)',
+    )
+    .option(
+      '--no-reveal-all',
+      'Disable force-reveal of scroll-triggered content',
+    )
     .option('--user-agent <string>', 'Custom user agent')
     .option('--viewport <WxH>', 'Viewport size', '1920x1080')
     .option('--device <name>', 'Emulate device (e.g., "iPhone 13")')
@@ -179,7 +227,7 @@ export function run() {
   // Serve command
   program
     .command('serve [directory]')
-    .description('Serve a captured site locally')
+    .description('Serve a captured site locally (default: ~/.smippo/sites/)')
     .option(
       '-p, --port <port>',
       'Port to serve on (auto-finds available)',
@@ -192,8 +240,13 @@ export function run() {
     .option('-q, --quiet', 'Minimal output')
     .action(async (directory, options) => {
       const {serve} = await import('./server.js');
+      const {getSitesDir} = await import('./utils/home.js');
+
+      // If no directory specified, use global smippo sites directory
+      const serveDir = directory || getSitesDir();
+
       await serve({
-        directory: directory || './site',
+        directory: serveDir,
         port: options.port,
         host: options.host,
         open: options.open,
@@ -269,6 +322,14 @@ export function run() {
 }
 
 async function capture(url, options) {
+  // Compute output directory based on URL domain if not specified
+  let outputDir = options.output;
+  if (!outputDir) {
+    const domain = getDomainFromUrl(url);
+    outputDir = getSiteDir(domain);
+    await ensureSmippoHome();
+  }
+
   const spinner = ora({
     text: 'Initializing browser...',
     isSilent: options.quiet,
@@ -276,7 +337,7 @@ async function capture(url, options) {
 
   const crawler = new Crawler({
     url,
-    output: options.output,
+    output: outputDir,
     depth: parseInt(options.depth, 10),
     scope: options.scope,
     stayInDir: options.stayInDir,
@@ -290,6 +351,12 @@ async function capture(url, options) {
     wait: options.wait,
     waitTime: parseInt(options.waitTime, 10),
     timeout: parseInt(options.timeout, 10),
+    scroll: options.scroll,
+    scrollWait: parseInt(options.scrollWait, 10),
+    scrollStep: parseInt(options.scrollStep, 10),
+    scrollDelay: parseInt(options.scrollDelay, 10),
+    scrollBehavior: options.scrollBehavior,
+    revealAll: options.revealAll,
     userAgent: options.userAgent,
     viewport: parseViewport(options.viewport),
     device: options.device,
@@ -356,7 +423,22 @@ async function capture(url, options) {
     console.log(chalk.yellow(`    Errors:          ${result.stats.errors}`));
   }
   console.log('');
-  console.log(`  Output: ${chalk.underline(options.output)}`);
+  console.log(`  Output: ${chalk.underline(outputDir)}`);
+
+  // Update global manifest with this capture (tracks all sites regardless of location)
+  try {
+    const domain = getDomainFromUrl(url);
+    await addSiteToGlobalManifest({
+      domain,
+      rootUrl: url,
+      outputDir: outputDir,
+      title: result.pages?.[0]?.title || domain,
+      pagesCount: result.stats.pagesCapt,
+      assetsCount: result.stats.assetsCapt,
+    });
+  } catch {
+    // Silently ignore manifest errors
+  }
 }
 
 async function continueCapture(options) {

package/src/crawler.js

@@ -270,6 +270,12 @@ export class Crawler extends EventEmitter {
         mimeExclude: this.options.mimeExclude,
         maxSize: this.options.maxSize,
         minSize: this.options.minSize,
+        scroll: this.options.scroll,
+        scrollWait: this.options.scrollWait,
+        scrollStep: this.options.scrollStep,
+        scrollDelay: this.options.scrollDelay,
+        scrollBehavior: this.options.scrollBehavior,
+        revealAll: this.options.revealAll,
       });
 
       const result = await capture.capture(url);

package/src/page-capture.js

@@ -35,9 +35,26 @@ export class PageCapture {
       }
     }
 
-    // Additional wait time if specified
-    if (this.options.waitTime > 0) {
-      await this.page.waitForTimeout(this.options.waitTime);
+    // Additional wait time if specified (default 500ms for animations to start)
+    const waitTime = this.options.waitTime ?? 500;
+    if (waitTime > 0) {
+      await this.page.waitForTimeout(waitTime);
+    }
+
+    // Step 1: Force reveal all scroll-triggered content
+    if (this.options.revealAll !== false) {
+      await this._revealAllContent();
+    }
+
+    // Step 2: Pre-scroll the page to trigger scroll animations
+    if (this.options.scroll !== false) {
+      await this._scrollPage();
+    }
+
+    // Step 3: Additional wait after scroll for animations to complete
+    const scrollWait = this.options.scrollWait ?? 1000;
+    if (scrollWait > 0 && this.options.scroll !== false) {
+      await this.page.waitForTimeout(scrollWait);
     }
 
     // Get the rendered HTML
@@ -148,4 +165,301 @@ export class PageCapture {
       return type === filter || type.startsWith(filter + ';');
     });
   }
+
+  /**
+   * Pre-scroll the page to trigger scroll-based animations and lazy loading
+   * Performs smooth, incremental scrolling to trigger all scroll-based content
+   */
+  async _scrollPage() {
+    const scrollBehavior = this.options.scrollBehavior || 'smooth';
+    const scrollStep = this.options.scrollStep || 200; // pixels per step
+    const scrollDelay = this.options.scrollDelay || 50; // ms between steps
+
+    /* eslint-disable no-undef */
+    await this.page.evaluate(
+      async ({step, delay, behavior}) => {
+        // Helper for smooth scrolling with requestAnimationFrame
+        const smoothScroll = (targetY, duration = 300) => {
+          return new Promise(resolve => {
+            const startY = window.scrollY;
+            const distance = targetY - startY;
+            const startTime = performance.now();
+
+            const animate = currentTime => {
+              const elapsed = currentTime - startTime;
+              const progress = Math.min(elapsed / duration, 1);
+
+              // Easing function (ease-out-cubic)
+              const eased = 1 - Math.pow(1 - progress, 3);
+
+              window.scrollTo(0, startY + distance * eased);
+
+              if (progress < 1) {
+                requestAnimationFrame(animate);
+              } else {
+                resolve();
+              }
+            };
+
+            requestAnimationFrame(animate);
+          });
+        };
+
+        // Get initial page height
+        let lastHeight = document.body.scrollHeight;
+        let currentY = 0;
+
+        // Phase 1: Scroll down incrementally
+        while (currentY < document.body.scrollHeight) {
+          const targetY = Math.min(currentY + step, document.body.scrollHeight);
+
+          if (behavior === 'smooth') {
+            await smoothScroll(targetY, delay * 2);
+          } else {
+            window.scrollTo(0, targetY);
+          }
+
+          currentY = targetY;
+          await new Promise(r => setTimeout(r, delay));
+
+          // Check if page height increased (lazy content loaded)
+          const newHeight = document.body.scrollHeight;
+          if (newHeight > lastHeight) {
+            lastHeight = newHeight;
+          }
+        }
+
+        // Phase 2: Wait at bottom for any pending lazy loads
+        await new Promise(r => setTimeout(r, 500));
+
+        // Check if more content loaded while waiting
+        if (document.body.scrollHeight > lastHeight) {
+          // Scroll to the new bottom
+          if (behavior === 'smooth') {
+            await smoothScroll(document.body.scrollHeight, 300);
+          } else {
+            window.scrollTo(0, document.body.scrollHeight);
+          }
+          await new Promise(r => setTimeout(r, 300));
+        }
+
+        // Phase 3: Scroll back up slowly (some sites have scroll-up animations)
+        const scrollUpStep = step * 2; // Faster on the way up
+        currentY = window.scrollY;
+
+        while (currentY > 0) {
+          const targetY = Math.max(currentY - scrollUpStep, 0);
+
+          if (behavior === 'smooth') {
+            await smoothScroll(targetY, delay);
+          } else {
+            window.scrollTo(0, targetY);
+          }
+
+          currentY = targetY;
+          await new Promise(r => setTimeout(r, delay / 2));
+        }
+
+        // Phase 4: Return to top and wait
+        window.scrollTo(0, 0);
+        await new Promise(r => setTimeout(r, 200));
+      },
+      {step: scrollStep, delay: scrollDelay, behavior: scrollBehavior},
+    );
+    /* eslint-enable no-undef */
+  }
+
+  /**
+   * Force reveal all scroll-triggered content by disabling/triggering
+   * common animation libraries like GSAP ScrollTrigger, AOS, etc.
+   */
+  async _revealAllContent() {
+    /* eslint-disable no-undef */
+    await this.page.evaluate(() => {
+      // Helper to safely access nested properties
+      const safeGet = (obj, path) => {
+        try {
+          return path.split('.').reduce((o, k) => o?.[k], obj);
+        } catch {
+          return undefined;
+        }
+      };
+
+      // 1. GSAP ScrollTrigger - kill all triggers and show content
+      const ScrollTrigger = safeGet(window, 'ScrollTrigger');
+      if (ScrollTrigger) {
+        try {
+          // Get all ScrollTrigger instances
+          const triggers = ScrollTrigger.getAll?.() || [];
+          triggers.forEach(trigger => {
+            try {
+              // Kill the trigger to prevent it from hiding content
+              trigger.kill?.();
+            } catch (e) {
+              /* ignore */
+            }
+          });
+          // Refresh to ensure proper state
+          ScrollTrigger.refresh?.();
+        } catch (e) {
+          /* ignore */
+        }
+      }
+
+      // Also check for gsap.ScrollTrigger
+      const gsapScrollTrigger = safeGet(window, 'gsap.ScrollTrigger');
+      if (gsapScrollTrigger && gsapScrollTrigger !== ScrollTrigger) {
+        try {
+          const triggers = gsapScrollTrigger.getAll?.() || [];
+          triggers.forEach(trigger => trigger.kill?.());
+        } catch (e) {
+          /* ignore */
+        }
+      }
+
+      // 2. AOS (Animate On Scroll) - reveal all elements
+      const AOS = safeGet(window, 'AOS');
+      if (AOS) {
+        try {
+          // Disable AOS and show all elements
+          document.querySelectorAll('[data-aos]').forEach(el => {
+            el.classList.add('aos-animate');
+            el.style.opacity = '1';
+            el.style.transform = 'none';
+            el.style.visibility = 'visible';
+          });
+        } catch (e) {
+          /* ignore */
+        }
+      }
+
+      // 3. WOW.js - reveal all elements
+      document.querySelectorAll('.wow').forEach(el => {
+        el.classList.add('animated');
+        el.style.visibility = 'visible';
+        el.style.opacity = '1';
+        el.style.animationName = 'none';
+      });
+
+      // 4. ScrollReveal - reveal all elements
+      const ScrollReveal = safeGet(window, 'ScrollReveal');
+      if (ScrollReveal) {
+        try {
+          document.querySelectorAll('[data-sr-id]').forEach(el => {
+            el.style.visibility = 'visible';
+            el.style.opacity = '1';
+            el.style.transform = 'none';
+          });
+        } catch (e) {
+          /* ignore */
+        }
+      }
+
+      // 5. Intersection Observer based lazy loading - trigger all observers
+      // This is tricky since we can't access observers directly,
+      // but we can trigger the elements they're watching
+
+      // 6. Generic fixes for common hidden patterns
+      // Elements with opacity: 0 that are meant to fade in
+      document
+        .querySelectorAll('[style*="opacity: 0"], [style*="opacity:0"]')
+        .forEach(el => {
+          // Only reveal if it seems intentionally hidden for animation
+          const computedStyle = window.getComputedStyle(el);
+          if (
+            computedStyle.opacity === '0' &&
+            !el.hasAttribute('aria-hidden')
+          ) {
+            el.style.opacity = '1';
+          }
+        });
+
+      // Elements with visibility: hidden that may animate in
+      document
+        .querySelectorAll(
+          '[style*="visibility: hidden"], [style*="visibility:hidden"]',
+        )
+        .forEach(el => {
+          el.style.visibility = 'visible';
+        });
+
+      // Elements with transform: translateY that slide in
+      document.querySelectorAll('[style*="translateY"]').forEach(el => {
+        const style = el.getAttribute('style') || '';
+        // Only fix if it looks like a scroll animation starting position
+        if (
+          style.includes('translateY(') &&
+          (style.includes('opacity') || el.classList.length > 0)
+        ) {
+          el.style.transform = 'none';
+        }
+      });
+
+      // 7. Lazy-loaded images - force load
+      document
+        .querySelectorAll('img[data-src], img[data-lazy], img[loading="lazy"]')
+        .forEach(img => {
+          const src =
+            img.getAttribute('data-src') || img.getAttribute('data-lazy');
+          if (src && !img.src) {
+            img.src = src;
+          }
+          // Remove lazy loading to ensure images load
+          img.removeAttribute('loading');
+        });
+
+      // 8. Lazy-loaded iframes
+      document.querySelectorAll('iframe[data-src]').forEach(iframe => {
+        const src = iframe.getAttribute('data-src');
+        if (src && !iframe.src) {
+          iframe.src = src;
+        }
+      });
+
+      // 9. Picture elements with lazy loading
+      document
+        .querySelectorAll('picture source[data-srcset]')
+        .forEach(source => {
+          const srcset = source.getAttribute('data-srcset');
+          if (srcset) {
+            source.srcset = srcset;
+          }
+        });
+
+      // 10. Background images in data attributes
+      document.querySelectorAll('[data-bg], [data-background]').forEach(el => {
+        const bg =
+          el.getAttribute('data-bg') || el.getAttribute('data-background');
+        if (bg && !el.style.backgroundImage) {
+          el.style.backgroundImage = `url(${bg})`;
+        }
+      });
+
+      // 11. Lottie animations - try to advance to final state
+      const lottieElements = document.querySelectorAll(
+        'lottie-player, [data-lottie]',
+      );
+      lottieElements.forEach(el => {
+        try {
+          if (el.goToAndStop) {
+            el.goToAndStop(el.totalFrames - 1, true);
+          }
+        } catch (e) {
+          /* ignore */
+        }
+      });
+
+      // 12. Force all CSS animations to complete
+      document.querySelectorAll('*').forEach(el => {
+        const style = window.getComputedStyle(el);
+        if (style.animationName && style.animationName !== 'none') {
+          // Set animation to end state
+          el.style.animationPlayState = 'paused';
+          el.style.animationDelay = '0s';
+          el.style.animationDuration = '0.001s';
+        }
+      });
+    });
+    /* eslint-enable no-undef */
+  }
 }

package/src/server.js

@@ -6,6 +6,7 @@ import chalk from 'chalk';
 import {exec} from 'child_process';
 import * as p from '@clack/prompts';
 import {readManifest} from './manifest.js';
+import {getAllCapturedSites, getSitesDir} from './utils/home.js';
 
 // MIME type mapping
 const MIME_TYPES = {
@@ -256,7 +257,7 @@ async function generateDirectoryListing(dirPath, urlPath, rootDir) {
           : ''
       }
 		</div>
-		
+
 		${
       !isRoot
         ? `
@@ -273,7 +274,7 @@ async function generateDirectoryListing(dirPath, urlPath, rootDir) {
 		</div>`
         : ''
     }
-		
+
 		<div class="listing">
 			<div class="listing-header">
 				${isRoot ? 'Captured Sites' : `Contents of ${urlPath}`}
@@ -317,7 +318,7 @@ async function generateDirectoryListing(dirPath, urlPath, rootDir) {
           : ''
       }
 		</div>
-		
+
 		<div class="footer">
 			Powered by Smippo • Modern Website Copier
 		</div>
@@ -577,41 +578,103 @@ function truncatePath(p, maxLen) {
 }
 
 /**
- * Get captured sites from a smippo output directory
+ * Get captured sites - either from global manifest or a specific directory
+ * @param {string|null} directory - Specific directory to serve, or null for global
  */
 async function getCapturedSites(directory) {
   const sites = [];
-  const smippoDir = path.join(directory, '.smippo');
 
+  // If no directory specified or it's the global sites dir, use global manifest
+  const globalSitesDir = getSitesDir();
+  const isGlobalMode =
+    !directory || path.resolve(directory) === path.resolve(globalSitesDir);
+
+  if (isGlobalMode) {
+    // Use global manifest to get all captured sites
+    const globalSites = await getAllCapturedSites();
+    for (const site of globalSites) {
+      // Find the domain subdirectory within the site path
+      const domainDir = path.join(site.path, site.domain);
+      const indexInDomain = path.join(domainDir, 'index.html');
+      const indexInRoot = path.join(site.path, 'index.html');
+
+      let hasIndex = false;
+
+      if (await fs.pathExists(indexInDomain)) {
+        hasIndex = true;
+      } else if (await fs.pathExists(indexInRoot)) {
+        hasIndex = true;
+      }
+
+      sites.push({
+        domain: site.domain,
+        fullPath: site.path, // Absolute path to serve from
+        domainPath: site.domain, // Domain subdirectory
+        hasIndex,
+        rootUrl: site.rootUrl,
+        title: site.title || site.domain,
+        pagesCount: site.pagesCount || 0,
+        assetsCount: site.assetsCount || 0,
+        lastUpdated: site.updated || null,
+      });
+    }
+    return sites;
+  }
+
+  // Specific directory mode: check for .smippo directory
+  const smippoDir = path.join(directory, '.smippo');
   if (!(await fs.pathExists(smippoDir))) {
+    // Check if directory itself is a site directory (has index.html)
+    const indexPath = path.join(directory, 'index.html');
+    if (await fs.pathExists(indexPath)) {
+      const dirName = path.basename(directory);
+      sites.push({
+        domain: dirName,
+        fullPath: directory,
+        domainPath: '',
+        hasIndex: true,
+        rootUrl: null,
+        title: dirName,
+        pagesCount: 0,
+        assetsCount: 0,
+        lastUpdated: null,
+      });
+    }
     return sites;
   }
 
-  // Read manifest for site info
+  // Read local manifest for site info
   const manifest = await readManifest(directory);
 
-  // Find domain directories
-  const entries = await fs.readdir(directory);
-  for (const entry of entries) {
-    if (entry.startsWith('.')) continue;
-    const entryPath = path.join(directory, entry);
-    const stat = await fs.stat(entryPath);
-    if (stat.isDirectory()) {
-      // Check if this directory has an index.html
-      const indexPath = path.join(entryPath, 'index.html');
+  if (!manifest?.rootUrl) {
+    return sites;
+  }
+
+  // Extract domain from rootUrl
+  try {
+    const url = new URL(manifest.rootUrl);
+    const mainDomain = url.hostname;
+
+    // Check if the main domain directory exists
+    const domainPath = path.join(directory, mainDomain);
+    if (await fs.pathExists(domainPath)) {
+      const indexPath = path.join(domainPath, 'index.html');
       const hasIndex = await fs.pathExists(indexPath);
 
       sites.push({
-        domain: entry,
-        path: entry,
+        domain: mainDomain,
+        fullPath: directory,
+        domainPath: mainDomain,
         hasIndex,
-        rootUrl: manifest?.rootUrl || null,
-        title: manifest?.pages?.[0]?.title || entry,
-        pagesCount: manifest?.stats?.pagesCapt || 0,
-        assetsCount: manifest?.stats?.assetsCapt || 0,
-        lastUpdated: manifest?.updated || null,
+        rootUrl: manifest.rootUrl,
+        title: manifest.pages?.[0]?.title || mainDomain,
+        pagesCount: manifest.stats?.pagesCapt || 0,
+        assetsCount: manifest.stats?.assetsCapt || 0,
+        lastUpdated: manifest.updated || null,
       });
     }
+  } catch {
+    // Invalid URL in manifest, fall back to directory scan
   }
 
   return sites;
@@ -622,12 +685,15 @@ async function getCapturedSites(directory) {
  */
 export async function serve(options) {
   try {
-    const directory = options.output || options.directory || './site';
-    const resolvedDir = path.resolve(directory);
+    const directory = options.output || options.directory;
+    const globalSitesDir = getSitesDir();
+
+    // Get captured sites (from global manifest if no directory specified)
+    const sites = await getCapturedSites(directory);
 
-    // Check if this is a smippo capture directory
-    const sites = await getCapturedSites(resolvedDir);
-    let sitePath = null;
+    let serveDir = directory ? path.resolve(directory) : null;
+    let openPath = null;
+    let selectedSite = null;
 
     if (sites.length > 0 && process.stdin.isTTY && !options.quiet) {
       // Show interactive site selection
@@ -636,22 +702,30 @@ export async function serve(options) {
 
       if (sites.length === 1) {
         // Single site - auto-select but show info
-        const site = sites[0];
+        selectedSite = sites[0];
         console.log(
-          chalk.dim('  Found captured site: ') + chalk.bold(site.domain),
+          chalk.dim('  Found captured site: ') +
+            chalk.bold(selectedSite.domain),
         );
-        if (site.pagesCount > 0) {
+        if (selectedSite.pagesCount > 0) {
           console.log(
-            chalk.dim(`  ${site.pagesCount} pages, ${site.assetsCount} assets`),
+            chalk.dim(
+              `  ${selectedSite.pagesCount} pages, ${selectedSite.assetsCount} assets`,
+            ),
           );
         }
-        sitePath = site.path;
+        if (selectedSite.fullPath) {
+          console.log(chalk.dim(`  Location: ${selectedSite.fullPath}`));
+        }
       } else {
         // Multiple sites - let user choose
         const siteOptions = sites.map(site => ({
-          value: site.path,
+          value: site,
           label: site.domain,
-          hint: site.pagesCount > 0 ? `${site.pagesCount} pages` : undefined,
+          hint:
+            site.pagesCount > 0
+              ? `${site.pagesCount} pages - ${site.fullPath}`
+              : site.fullPath,
         }));
 
         const selected = await p.select({
@@ -664,20 +738,34 @@ export async function serve(options) {
           process.exit(0);
         }
 
-        sitePath = selected;
+        selectedSite = selected;
       }
       console.log('');
     } else if (sites.length === 1) {
       // Non-interactive mode with single site
-      sitePath = sites[0].path;
+      selectedSite = sites[0];
+    } else if (sites.length === 0 && !directory) {
+      console.log(chalk.yellow('No captured sites found.'));
+      console.log(
+        chalk.dim('  Capture a site first: ') + chalk.cyan('smippo <url>'),
+      );
+      process.exit(0);
+    }
+
+    // Determine serve directory and open path
+    if (selectedSite) {
+      serveDir = selectedSite.fullPath;
+      openPath = selectedSite.domainPath || null;
+    } else if (!serveDir) {
+      serveDir = globalSitesDir;
     }
 
     const serverInfo = await createServer({
-      directory: resolvedDir,
+      directory: serveDir,
       port: options.port || 8080,
       host: options.host || '127.0.0.1',
       open: options.open,
-      openPath: sitePath, // Pass the site path to open
+      openPath: openPath,
       cors: options.cors !== false,
       verbose: options.verbose,
       quiet: options.quiet,

package/src/utils/home.js

@@ -0,0 +1,175 @@
+// @flow
+import fs from 'fs-extra';
+import path from 'path';
+import os from 'os';
+
+const SMIPPO_HOME_DIR = '.smippo';
+const SITES_DIR = 'sites';
+const GLOBAL_MANIFEST_FILE = 'manifest.json';
+
+/**
+ * Get the global smippo home directory (~/.smippo/)
+ */
+export function getSmippoHome() {
+  return path.join(os.homedir(), SMIPPO_HOME_DIR);
+}
+
+/**
+ * Get the sites directory (~/.smippo/sites/)
+ */
+export function getSitesDir() {
+  return path.join(getSmippoHome(), SITES_DIR);
+}
+
+/**
+ * Get the output directory for a specific domain
+ * @param {string} domain - The domain name (e.g., 'example.com')
+ * @returns {string} The full path to the site directory
+ */
+export function getSiteDir(domain) {
+  return path.join(getSitesDir(), domain);
+}
+
+/**
+ * Extract domain from a URL
+ * @param {string} url - The URL to extract domain from
+ * @returns {string} The domain (hostname)
+ */
+export function getDomainFromUrl(url) {
+  try {
+    const parsed = new URL(url);
+    return parsed.hostname;
+  } catch {
+    // If URL parsing fails, return the original string
+    return url;
+  }
+}
+
+/**
+ * Ensure the smippo home directory exists
+ */
+export async function ensureSmippoHome() {
+  const homeDir = getSmippoHome();
+  const sitesDir = getSitesDir();
+
+  await fs.ensureDir(homeDir);
+  await fs.ensureDir(sitesDir);
+
+  return homeDir;
+}
+
+/**
+ * Get the global manifest path (~/.smippo/manifest.json)
+ */
+export function getGlobalManifestPath() {
+  return path.join(getSmippoHome(), GLOBAL_MANIFEST_FILE);
+}
+
+/**
+ * Read the global manifest (list of all captured sites)
+ */
+export async function readGlobalManifest() {
+  const manifestPath = getGlobalManifestPath();
+
+  if (!(await fs.pathExists(manifestPath))) {
+    return {
+      version: '1.0.0',
+      sites: [],
+    };
+  }
+
+  try {
+    const content = await fs.readFile(manifestPath, 'utf8');
+    return JSON.parse(content);
+  } catch {
+    return {
+      version: '1.0.0',
+      sites: [],
+    };
+  }
+}
+
+/**
+ * Write the global manifest
+ */
+export async function writeGlobalManifest(manifest) {
+  await ensureSmippoHome();
+  const manifestPath = getGlobalManifestPath();
+  await fs.writeFile(manifestPath, JSON.stringify(manifest, null, 2), 'utf8');
+}
+
+/**
+ * Add a site to the global manifest
+ * @param {Object} siteInfo - Site information
+ * @param {string} siteInfo.domain - Domain name
+ * @param {string} siteInfo.rootUrl - Original URL
+ * @param {string} siteInfo.outputDir - Directory where site was saved
+ * @param {string} [siteInfo.title] - Page title
+ * @param {number} [siteInfo.pagesCount] - Number of pages captured
+ * @param {number} [siteInfo.assetsCount] - Number of assets captured
+ */
+export async function addSiteToGlobalManifest(siteInfo) {
+  const manifest = await readGlobalManifest();
+
+  // Use outputDir as the unique key (same domain can be saved to different dirs)
+  const outputPath = path.resolve(siteInfo.outputDir);
+  const existingIndex = manifest.sites.findIndex(s => s.path === outputPath);
+
+  const siteEntry = {
+    domain: siteInfo.domain,
+    rootUrl: siteInfo.rootUrl,
+    title: siteInfo.title || siteInfo.domain,
+    path: outputPath,
+    created: siteInfo.created || new Date().toISOString(),
+    updated: new Date().toISOString(),
+    pagesCount: siteInfo.pagesCount || 0,
+    assetsCount: siteInfo.assetsCount || 0,
+  };
+
+  if (existingIndex >= 0) {
+    // Update existing entry (preserve created date)
+    manifest.sites[existingIndex] = {
+      ...siteEntry,
+      created: manifest.sites[existingIndex].created,
+    };
+  } else {
+    // Add new entry
+    manifest.sites.push(siteEntry);
+  }
+
+  await writeGlobalManifest(manifest);
+  return manifest;
+}
+
+/**
+ * Remove a site from the global manifest
+ */
+export async function removeSiteFromGlobalManifest(domain) {
+  const manifest = await readGlobalManifest();
+  manifest.sites = manifest.sites.filter(s => s.domain !== domain);
+  await writeGlobalManifest(manifest);
+  return manifest;
+}
+
+/**
+ * Get all captured sites from the global manifest
+ */
+export async function getAllCapturedSites() {
+  const manifest = await readGlobalManifest();
+
+  // Verify each site still exists
+  const validSites = [];
+  for (const site of manifest.sites) {
+    if (await fs.pathExists(site.path)) {
+      validSites.push(site);
+    }
+  }
+
+  // Update manifest if some sites were removed
+  if (validSites.length !== manifest.sites.length) {
+    manifest.sites = validSites;
+    await writeGlobalManifest(manifest);
+  }
+
+  return validSites;
+}