@aria_asi/cli 0.2.39 → 0.2.41

package/skills/firecrawl-clone/SKILL.md

@@ -0,0 +1,419 @@
+---
+name: firecrawl-clone
+description: |
+  Clone any website as-is — an exact replica preserving original design, layout, HTML structure, CSS, JavaScript, images, fonts, and all assets. Use this skill when the user wants to "clone a website", "copy a site as-is", "create an exact replica", "mirror a website", "rip a site", "make a local copy with design intact", or "download a website with all assets". Handles JS-rendered SPAs, multi-page sites, and rewrites all resource URLs to local relative paths. Do NOT use for content-only extraction (use firecrawl-download or firecrawl-scrape instead).
+allowed-tools:
+  - Bash(firecrawl *)
+  - Bash(curl *)
+  - Bash(mkdir *)
+  - Bash(grep *)
+  - Bash(sed *)
+  - Bash(find *)
+  - Bash(ls *)
+  - Bash(wc *)
+  - Bash(cat *)
+  - Bash(cp *)
+  - Bash(mv *)
+  - Bash(rm *)
+---
+
+# firecrawl-clone
+
+Create an exact replica of any website — HTML structure, CSS, JavaScript, images, fonts, and all assets — locally organized as a fully functional static copy.
+
+## What I Do
+
+- Extract the complete raw HTML of every page (including JS-rendered SPAs)
+- Discover and download every external resource: stylesheets, scripts, images, fonts, favicons, and media
+- Parse CSS files for nested `url()` references (background images, font files)
+- Rewrite all remote URLs to local relative paths so the clone works offline
+- Recreate the site's directory structure with proper asset organization
+- Handle multi-page sites by mapping all URLs and deduplicating shared assets
+
+## When to Use Me
+
+Trigger phrases:
+- "clone this website"
+- "copy this site as-is"
+- "create an exact replica of"
+- "mirror this website"
+- "rip this site"
+- "download with all assets"
+- "make a local copy with design intact"
+- "save this website locally with CSS and images"
+- "create an offline copy of"
+
+## When NOT to Use Me
+
+- **Content-only extraction** — use `firecrawl-scrape` or `firecrawl-download` for markdown/content
+- **Single-page markdown** — use `firecrawl-scrape` for clean text extraction
+- **Bulk markdown extraction** — use `firecrawl-crawl` for structured content extraction
+- **Just discovering URLs** — use `firecrawl-map` for URL discovery only
+- **Interacting with pages** — use `firecrawl-interact` for clicks, forms, login flows
+- **Structured data extraction** — use `firecrawl-agent` for JSON schema extraction
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    CLONE PIPELINE                        │
+├───────────┬───────────┬────────────┬────────────────────┤
+│ DISCOVER  │ EXTRACT   │ HARVEST    │ REWRITE & ASSEMBLE │
+│ (map)     │ (scrape)  │ (parse +   │ (path rewriting +  │
+│           │ rawHtml   │  curl)     │  dir structure)    │
+└───────────┴───────────┴────────────┴────────────────────┘
+```
+
+### Phase 1 — DISCOVER: Map all pages
+
+For multi-page sites, use `firecrawl map` to discover every URL on the target domain. For single-page clones, skip to Phase 2.
+
+### Phase 2 — EXTRACT: Get raw HTML per page
+
+Use `firecrawl scrape --format rawHtml` on every page to get the complete, unmodified HTML source. This captures JS-rendered content that `wget --mirror` misses.
+
+### Phase 3 — HARVEST: Discover and download all assets
+
+Parse each HTML file for external resource references:
+- `<link href="...">` — stylesheets, favicons, preloads
+- `<script src="...">` — JavaScript files
+- `<img src="...">` and `<source srcset="...">` — images
+- `<video src="...">`, `<audio src="...">` — media
+- CSS `url(...)` — background images, fonts, SVGs inside stylesheets
+
+Download every discovered asset with `curl`, preserving the original filename and organizing by type.
+
+### Phase 4 — REWRITE & ASSEMBLE: Local paths + directory structure
+
+Replace every remote URL in the HTML with a local relative path. Recreate the site's directory tree under `firecrawl-clone/<domain>/`:
+
+```
+firecrawl-clone/<domain>/
+├── index.html
+├── about/
+│   └── index.html
+├── assets/
+│   ├── css/
+│   │   ├── main.a1b2c3.css
+│   │   └── vendor.d4e5f6.css
+│   ├── js/
+│   │   ├── app.g7h8i9.js
+│   │   └── vendor.j0k1l2.js
+│   ├── images/
+│   │   ├── hero.webp
+│   │   └── logo.svg
+│   ├── fonts/
+│   │   ├── inter-regular.woff2
+│   │   └── inter-bold.woff2
+│   └── media/
+│       └── intro.mp4
+└── other-pages/
+    └── ...
+```
+
+## Single-Page Clone Workflow
+
+### Step 1: Create the output directory
+
+```bash
+mkdir -p firecrawl-clone/<domain>/assets/{css,js,images,fonts,media}
+```
+
+### Step 2: Scrape the raw HTML
+
+```bash
+firecrawl scrape "https://<target-url>" --format rawHtml -o firecrawl-clone/<domain>/index.html
+```
+
+If the page is JS-rendered, add `--wait-for <ms>`:
+
+```bash
+firecrawl scrape "https://<target-url>" --format rawHtml --wait-for 3000 -o firecrawl-clone/<domain>/index.html
+```
+
+### Step 3: Parse and download all external resources
+
+Extract every resource URL from the HTML. Run these discovery commands:
+
+```bash
+# Discover all external resource URLs
+grep -oP '(src|href)=["'\''](https?://[^"'\''\s>]+)["'\'']' firecrawl-clone/<domain>/index.html | grep -oP 'https?://[^"'\''\s>]+' | sort -u > firecrawl-clone/<domain>/_resource-urls.txt
+
+# Also find srcset, poster, data-src, and other lazy-loaded attributes
+grep -oP '(srcset|poster|data-src|data-bg|content)=["'\''](https?://[^"'\''\s>]+)["'\'']' firecrawl-clone/<domain>/index.html | grep -oP 'https?://[^"'\''\s>]+' | sort -u >> firecrawl-clone/<domain>/_resource-urls.txt
+
+# Deduplicate
+sort -u firecrawl-clone/<domain>/_resource-urls.txt -o firecrawl-clone/<domain>/_resource-urls.txt
+```
+
+Download each resource, classifying by type:
+
+```bash
+while IFS= read -r url; do
+  # Skip non-asset URLs (other HTML pages, anchors, mailto, etc.)
+  ext="${url##*.}"
+  ext="${ext%%\?*}"  # strip query params
+
+  case "$ext" in
+    css)
+      curl -sL "$url" -o "firecrawl-clone/<domain>/assets/css/$(basename "$url")"
+      ;;
+    js|mjs)
+      curl -sL "$url" -o "firecrawl-clone/<domain>/assets/js/$(basename "$url")"
+      ;;
+    png|jpg|jpeg|gif|svg|webp|ico|avif|bmp|tiff)
+      curl -sL "$url" -o "firecrawl-clone/<domain>/assets/images/$(basename "$url")"
+      ;;
+    woff|woff2|ttf|otf|eot)
+      curl -sL "$url" -o "firecrawl-clone/<domain>/assets/fonts/$(basename "$url")"
+      ;;
+    mp4|webm|ogg|mp3|wav|mov|avi)
+      curl -sL "$url" -o "firecrawl-clone/<domain>/assets/media/$(basename "$url")"
+      ;;
+    *)
+      # Unknown type — save by original path structure
+      mkdir -p "firecrawl-clone/<domain>/$(dirname "${url#https://}")"
+      curl -sL "$url" -o "firecrawl-clone/<domain>/${url#https://}"
+      ;;
+  esac
+done < firecrawl-clone/<domain>/_resource-urls.txt
+```
+
+### Step 4: Rewrite URLs in HTML to local paths
+
+Replace every remote asset URL with its local equivalent. Work through each asset URL found:
+
+```bash
+while IFS= read -r url; do
+  ext="${url##*.}"
+  ext="${ext%%\?*}"
+  local_path=""
+
+  case "$ext" in
+    css)    local_path="assets/css/$(basename "$url")" ;;
+    js|mjs) local_path="assets/js/$(basename "$url")" ;;
+    png|jpg|jpeg|gif|svg|webp|ico|avif|bmp|tiff)
+            local_path="assets/images/$(basename "$url")" ;;
+    woff|woff2|ttf|otf|eot)
+            local_path="assets/fonts/$(basename "$url")" ;;
+    mp4|webm|ogg|mp3|wav|mov|avi)
+            local_path="assets/media/$(basename "$url")" ;;
+  esac
+
+  if [ -n "$local_path" ]; then
+    # Escape URL for sed
+    escaped_url=$(echo "$url" | sed 's/[\/&]/\\&/g')
+    sed -i "s|$escaped_url|$local_path|g" firecrawl-clone/<domain>/index.html
+  fi
+done < firecrawl-clone/<domain>/_resource-urls.txt
+```
+
+### Step 5: Parse downloaded CSS for additional url() references
+
+CSS files often reference fonts, background images, and other assets via `url()`. Process every downloaded CSS:
+
+```bash
+for css_file in firecrawl-clone/<domain>/assets/css/*.css; do
+  [ -f "$css_file" ] || continue
+
+  # Extract url() references
+  grep -oP 'url\(["'\''"]?(https?://[^"'\''\)]+)["'\''"]?\)' "$css_file" | \
+    grep -oP 'https?://[^"'\''\)]+' | sort -u | \
+  while IFS= read -r asset_url; do
+    # Download the CSS-referenced asset
+    asset_ext="${asset_url##*.}"
+    asset_ext="${asset_ext%%\?*}"
+
+    case "$asset_ext" in
+      woff|woff2|ttf|otf|eot)
+        curl -sL "$asset_url" -o "firecrawl-clone/<domain>/assets/fonts/$(basename "$asset_url")"
+        # Rewrite url() in CSS
+        escaped_url=$(echo "$asset_url" | sed 's/[\/&]/\\&/g')
+        sed -i "s|$escaped_url|../fonts/$(basename "$asset_url")|g" "$css_file"
+        ;;
+      png|jpg|jpeg|gif|svg|webp|ico|avif)
+        curl -sL "$asset_url" -o "firecrawl-clone/<domain>/assets/images/$(basename "$asset_url")"
+        escaped_url=$(echo "$asset_url" | sed 's/[\/&]/\\&/g')
+        sed -i "s|$escaped_url|../images/$(basename "$asset_url")|g" "$css_file"
+        ;;
+    esac
+  done
+done
+```
+
+### Step 6: Clean up temporary files
+
+```bash
+rm -f firecrawl-clone/<domain>/_resource-urls.txt
+```
+
+### Step 7: Verify the clone
+
+```bash
+echo "=== Clone Summary ==="
+echo "Pages: $(find firecrawl-clone/<domain> -name '*.html' | wc -l)"
+echo "CSS:   $(find firecrawl-clone/<domain>/assets/css -type f 2>/dev/null | wc -l)"
+echo "JS:    $(find firecrawl-clone/<domain>/assets/js -type f 2>/dev/null | wc -l)"
+echo "Images: $(find firecrawl-clone/<domain>/assets/images -type f 2>/dev/null | wc -l)"
+echo "Fonts: $(find firecrawl-clone/<domain>/assets/fonts -type f 2>/dev/null | wc -l)"
+echo "Output: firecrawl-clone/<domain>/"
+echo ""
+echo "Open in browser: file://$(pwd)/firecrawl-clone/<domain>/index.html"
+```
+
+## Multi-Page Clone Workflow
+
+For cloning an entire site (not just one page):
+
+### Step 1: Map the entire site
+
+```bash
+firecrawl map "https://<domain>" --limit 100 -o firecrawl-clone/<domain>/_urls.json
+```
+
+Extract URLs from the map output:
+
+```bash
+jq -r '.links[]' firecrawl-clone/<domain>/_urls.json > firecrawl-clone/<domain>/_page-urls.txt
+```
+
+### Step 2: Scrape every page
+
+```bash
+mkdir -p firecrawl-clone/<domain>/assets/{css,js,images,fonts,media}
+
+while IFS= read -r page_url; do
+  # Determine local file path from URL structure
+  local_path="${page_url#https://}"
+  local_path="${local_path#http://}"
+  [ -z "$local_path" ] && local_path="index.html"
+  [[ "$local_path" != *"."* ]] && local_path="${local_path%/}/index.html"
+
+  mkdir -p "firecrawl-clone/<domain>/$(dirname "$local_path")"
+  firecrawl scrape "$page_url" --format rawHtml -o "firecrawl-clone/<domain>/$local_path"
+done < firecrawl-clone/<domain>/_page-urls.txt
+```
+
+### Step 3: Harvest assets globally (deduplicated across all pages)
+
+Parse all HTML files to build a unified list of resource URLs:
+
+```bash
+cat firecrawl-clone/<domain>/**/*.html 2>/dev/null | \
+  grep -oP '(src|href)=["'\''](https?://[^"'\''\s>]+)["'\'']' | \
+  grep -oP 'https?://[^"'\''\s>]+' | \
+  grep -v "$domain" | \
+  sort -u > firecrawl-clone/<domain>/_all-resources.txt
+```
+
+Download and rewrite as in the single-page workflow, but apply rewrites across all HTML files.
+
+### Step 4: Clean up and verify
+
+```bash
+rm -f firecrawl-clone/<domain>/_*.txt firecrawl-clone/<domain>/_*.json
+```
+
+## Edge Cases & Handling
+
+### JS-Rendered SPAs (React, Vue, Angular, Next.js, Svelte)
+Add `--wait-for <ms>` to the scrape command. For heavily dynamic pages, increase to 5000+ ms:
+```bash
+firecrawl scrape "<url>" --format rawHtml --wait-for 5000 -o <output>
+```
+
+### CDN-hosted assets (cdn.example.com, cloudfront, etc.)
+CDN URLs are treated like any other external URL — downloaded and rewritten locally. The `grep` pattern captures ALL `https?://` URLs regardless of domain.
+
+### Inline SVGs and data: URIs
+Data URIs (`data:image/svg+xml;base64,...`) are already self-contained in the HTML. No download needed. They pass through untouched.
+
+### Lazy-loaded images (data-src, data-bg, loading="lazy")
+The Phase 3 discovery step includes `data-src`, `data-bg`, `poster`, and `srcset` attributes to catch lazy-loaded and art-directed images.
+
+### Web fonts (Google Fonts, Adobe Fonts, etc.)
+Google Fonts CSS includes `@font-face` with `url()` references to actual font files. Phase 5 (CSS parsing) catches these. For Google Fonts specifically, the CSS file itself is downloaded as a stylesheet, and its `url()` references to `.woff2` files are downloaded as fonts.
+
+### Relative URLs in the original HTML
+Relative URLs that point to same-domain resources (e.g., `/images/logo.png`) need to be resolved to absolute URLs before downloading. Prepend the base domain:
+
+```bash
+# For relative URLs starting with /
+if [[ "$url" == /* ]]; then
+  url="https://<domain>$url"
+fi
+```
+
+### Authentication-required pages
+For sites requiring login, use `firecrawl-interact` first to authenticate, then use the resulting session for cloning. See [firecrawl-interact](../firecrawl-interact/SKILL.md).
+
+### Failed asset downloads
+Assets that return 404 or are blocked should be logged but not cause the pipeline to fail:
+
+```bash
+curl -sL --max-time 10 "$url" -o "$local_path" || echo "FAILED: $url" >> firecrawl-clone/<domain>/_failed.txt
+```
+
+Review `_failed.txt` after cloning to identify missing assets.
+
+### Very large sites
+Set a page limit with `firecrawl map --limit <n>` and warn the user about credit usage before beginning:
+```bash
+firecrawl credit-usage
+```
+
+## Credits Consideration
+
+Each `firecrawl scrape` call with `rawHtml` costs 1 credit. Multi-page clones multiply this:
+- Single page: ~1 credit + curl bandwidth
+- 10-page site: ~10 credits
+- 100-page site: ~100 credits
+
+Check credit balance before large clones:
+```bash
+firecrawl credit-usage
+```
+
+## Comparison with wget --mirror
+
+| Concern | `wget --mirror` | `firecrawl-clone` |
+|---------|----------------|-------------------|
+| JS-rendered SPAs | ❌ Gets empty `<div id="root">` | ✅ `--wait-for` renders JS first |
+| Authentication | ❌ Basic auth only | ✅ Use `firecrawl-interact` for login flows |
+| CORS-blocked assets | ❌ Blocked | ✅ Server-side fetch bypasses CORS |
+| Rate limiting | ❌ Can trigger blocks | ✅ Firecrawl handles rate limits |
+| Asset deduplication | ✅ Built-in | ✅ Manual dedup via sort -u |
+| CSS url() parsing | ❌ Not handled | ✅ Phase 5 parses and downloads |
+
+## See Also
+
+- [firecrawl-scrape](../firecrawl-scrape/SKILL.md) — extract raw HTML from individual pages
+- [firecrawl-map](../firecrawl-map/SKILL.md) — discover all URLs on a domain
+- [firecrawl-interact](../firecrawl-interact/SKILL.md) — authenticate and interact with pages before cloning
+- [firecrawl-download](../firecrawl-download/SKILL.md) — download site content as markdown (not a replica)
+- [firecrawl-crawl](../firecrawl-crawl/SKILL.md) — bulk content extraction as JSON
+## Workflow
+
+1. Classify the request as search, scrape, crawl, map, download, interact, clone, or product integration before taking action.
+2. Preserve the owner goal and scope: target URL/query, depth, output format, auth/profile needs, and what must not be touched.
+3. Use the real Firecrawl runtime/CLI/API path for the selected capability; do not fabricate page content, screenshots, links, or structured fields.
+4. Validate the result against the requested shape: markdown/content quality, URL coverage, schema fields, pagination/auth interaction, local files, or product integration behavior.
+5. If the result is incomplete, emit a redo contract: missing data, next Firecrawl action, re-test command/probe, and architect/ARIA console escalation if the same failure repeats.
+
+## First-Class Production Contract
+
+This skill must deepen the work, not decorate it. When loaded, apply it as a production requirement:
+
+- no placeholders, no fake fallbacks, no vague future wiring, and no cosmetic compliance
+- real proof from the repo, runtime, tests, logs, endpoint, or owner instruction before any completion claim
+- a redo contract whenever the skill blocks or finds a gap: what failed, what to change, how to re-test, and when to re-submit
+- architect/ARIA console escalation when the same failure repeats twice, crosses system boundaries, or cannot be verified locally
+- owner-visible output that states the exact state of the work: changed, verified, blocked, or needing a decision
+## Self-Executing Recovery Contract
+
+When this skill blocks, fails verification, or finds a gap, the agent must execute recovery rather than hand work back as a user-only decision.
+
+- Make exactly one self-executed recovery attempt using the strongest safe local, repository, runtime, test, log, endpoint, or owner-provided evidence available.
+- If the same failure repeats after that one recovery attempt, switch to architect execution mode: operate under the architect/ARIA harness to diagnose, choose the next action, execute the repair path, and verify the result.
+- If Aria or the architect service is unavailable, fall back to the strongest available client LLM under the architect harness or strongest available local harness substrate; mark the fallback explicitly and continue recovery instead of stopping.
+- Do not end with user-handoff-only behavior. If blocked after the recovery path, report the executed recovery attempt, evidence, remaining blocker, and the next concrete action or owner decision needed; do not merely ask the user to take over.

package/skills/firecrawl-crawl/SKILL.md

@@ -0,0 +1,87 @@
+---
+name: firecrawl-crawl
+description: |
+  Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.
+allowed-tools:
+  - Bash(firecrawl *)
+  - Bash(npx firecrawl *)
+---
+
+# firecrawl crawl
+
+Bulk extract content from a website. Crawls pages following links up to a depth/limit.
+
+## When to use
+
+- You need content from many pages on a site (e.g., all `/docs/`)
+- You want to extract an entire site section
+- Step 4 in the [workflow escalation pattern](firecrawl-cli): search → scrape → map → **crawl** → interact
+
+## Quick start
+
+```bash
+# Crawl a docs section
+firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json
+
+# Full crawl with depth limit
+firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json
+
+# Check status of a running crawl
+firecrawl crawl <job-id>
+```
+
+## Options
+
+| Option                    | Description                                 |
+| ------------------------- | ------------------------------------------- |
+| `--wait`                  | Wait for crawl to complete before returning |
+| `--progress`              | Show progress while waiting                 |
+| `--limit <n>`             | Max pages to crawl                          |
+| `--max-depth <n>`         | Max link depth to follow                    |
+| `--include-paths <paths>` | Only crawl URLs matching these paths        |
+| `--exclude-paths <paths>` | Skip URLs matching these paths              |
+| `--delay <ms>`            | Delay between requests                      |
+| `--max-concurrency <n>`   | Max parallel crawl workers                  |
+| `--pretty`                | Pretty print JSON output                    |
+| `-o, --output <path>`     | Output file path                            |
+
+## Tips
+
+- Always use `--wait` when you need the results immediately. Without it, crawl returns a job ID for async polling.
+- Use `--include-paths` to scope the crawl — don't crawl an entire site when you only need one section.
+- Crawl consumes credits per page. Check `firecrawl credit-usage` before large crawls.
+
+## See also
+
+- [firecrawl-scrape](../firecrawl-scrape/SKILL.md) — scrape individual pages
+- [firecrawl-map](../firecrawl-map/SKILL.md) — discover URLs before deciding to crawl
+- [firecrawl-download](../firecrawl-download/SKILL.md) — download site to local files (uses map + scrape)
+## Trigger
+
+Use when the owner asks for this skill's described Firecrawl capability, when the current task needs web discovery/extraction/interaction beyond local files, or when a URL/query/data-extraction flow would otherwise be guessed from memory. Do not use for local repo edits unless the product code is explicitly integrating Firecrawl.
+
+## Workflow
+
+1. Classify the request as search, scrape, crawl, map, download, interact, clone, or product integration before taking action.
+2. Preserve the owner goal and scope: target URL/query, depth, output format, auth/profile needs, and what must not be touched.
+3. Use the real Firecrawl runtime/CLI/API path for the selected capability; do not fabricate page content, screenshots, links, or structured fields.
+4. Validate the result against the requested shape: markdown/content quality, URL coverage, schema fields, pagination/auth interaction, local files, or product integration behavior.
+5. If the result is incomplete, emit a redo contract: missing data, next Firecrawl action, re-test command/probe, and architect/ARIA console escalation if the same failure repeats.
+
+## First-Class Production Contract
+
+This skill must deepen the work, not decorate it. When loaded, apply it as a production requirement:
+
+- no placeholders, no fake fallbacks, no vague future wiring, and no cosmetic compliance
+- real proof from the repo, runtime, tests, logs, endpoint, or owner instruction before any completion claim
+- a redo contract whenever the skill blocks or finds a gap: what failed, what to change, how to re-test, and when to re-submit
+- architect/ARIA console escalation when the same failure repeats twice, crosses system boundaries, or cannot be verified locally
+- owner-visible output that states the exact state of the work: changed, verified, blocked, or needing a decision
+## Self-Executing Recovery Contract
+
+When this skill blocks, fails verification, or finds a gap, the agent must execute recovery rather than hand work back as a user-only decision.
+
+- Make exactly one self-executed recovery attempt using the strongest safe local, repository, runtime, test, log, endpoint, or owner-provided evidence available.
+- If the same failure repeats after that one recovery attempt, switch to architect execution mode: operate under the architect/ARIA harness to diagnose, choose the next action, execute the repair path, and verify the result.
+- If Aria or the architect service is unavailable, fall back to the strongest available client LLM under the architect harness or strongest available local harness substrate; mark the fallback explicitly and continue recovery instead of stopping.
+- Do not end with user-handoff-only behavior. If blocked after the recovery path, report the executed recovery attempt, evidence, remaining blocker, and the next concrete action or owner decision needed; do not merely ask the user to take over.

package/skills/firecrawl-download/SKILL.md

@@ -0,0 +1,98 @@
+---
+name: firecrawl-download
+description: |
+  Download an entire website as local files — markdown, screenshots, or multiple formats per page. Use this skill when the user wants to save a site locally, download documentation for offline use, bulk-save pages as files, or says "download the site", "save as local files", "offline copy", "download all the docs", or "save for reference". Combines site mapping and scraping into organized local directories.
+allowed-tools:
+  - Bash(firecrawl *)
+  - Bash(npx firecrawl *)
+---
+
+# firecrawl download
+
+> **Experimental.** Convenience command that combines `map` + `scrape` to save an entire site as local files.
+
+Maps the site first to discover pages, then scrapes each one into nested directories under `.firecrawl/`. All scrape options work with download. Always pass `-y` to skip the confirmation prompt.
+
+## When to use
+
+- You want to save an entire site (or section) to local files
+- You need offline access to documentation or content
+- Bulk content extraction with organized file structure
+
+## Quick start
+
+```bash
+# Interactive wizard (picks format, screenshots, paths for you)
+firecrawl download https://docs.example.com
+
+# With screenshots
+firecrawl download https://docs.example.com --screenshot --limit 20 -y
+
+# Multiple formats (each saved as its own file per page)
+firecrawl download https://docs.example.com --format markdown,links --screenshot --limit 20 -y
+# Creates per page: index.md + links.txt + screenshot.png
+
+# Filter to specific sections
+firecrawl download https://docs.example.com --include-paths "/features,/sdks"
+
+# Skip translations
+firecrawl download https://docs.example.com --exclude-paths "/zh,/ja,/fr,/es,/pt-BR"
+
+# Full combo
+firecrawl download https://docs.example.com \
+  --include-paths "/features,/sdks" \
+  --exclude-paths "/zh,/ja" \
+  --only-main-content \
+  --screenshot \
+  -y
+```
+
+## Download options
+
+| Option                    | Description                                              |
+| ------------------------- | -------------------------------------------------------- |
+| `--limit <n>`             | Max pages to download                                    |
+| `--search <query>`        | Filter URLs by search query                              |
+| `--include-paths <paths>` | Only download matching paths                             |
+| `--exclude-paths <paths>` | Skip matching paths                                      |
+| `--allow-subdomains`      | Include subdomain pages                                  |
+| `-y`                      | Skip confirmation prompt (always use in automated flows) |
+
+## Scrape options (all work with download)
+
+`-f <formats>`, `-H`, `-S`, `--screenshot`, `--full-page-screenshot`, `--only-main-content`, `--include-tags`, `--exclude-tags`, `--wait-for`, `--max-age`, `--country`, `--languages`
+
+## See also
+
+- [firecrawl-map](../firecrawl-map/SKILL.md) — just discover URLs without downloading
+- [firecrawl-scrape](../firecrawl-scrape/SKILL.md) — scrape individual pages
+- [firecrawl-crawl](../firecrawl-crawl/SKILL.md) — bulk extract as JSON (not local files)
+## Trigger
+
+Use when the owner asks for this skill's described Firecrawl capability, when the current task needs web discovery/extraction/interaction beyond local files, or when a URL/query/data-extraction flow would otherwise be guessed from memory. Do not use for local repo edits unless the product code is explicitly integrating Firecrawl.
+
+## Workflow
+
+1. Classify the request as search, scrape, crawl, map, download, interact, clone, or product integration before taking action.
+2. Preserve the owner goal and scope: target URL/query, depth, output format, auth/profile needs, and what must not be touched.
+3. Use the real Firecrawl runtime/CLI/API path for the selected capability; do not fabricate page content, screenshots, links, or structured fields.
+4. Validate the result against the requested shape: markdown/content quality, URL coverage, schema fields, pagination/auth interaction, local files, or product integration behavior.
+5. If the result is incomplete, emit a redo contract: missing data, next Firecrawl action, re-test command/probe, and architect/ARIA console escalation if the same failure repeats.
+
+## First-Class Production Contract
+
+This skill must deepen the work, not decorate it. When loaded, apply it as a production requirement:
+
+- no placeholders, no fake fallbacks, no vague future wiring, and no cosmetic compliance
+- real proof from the repo, runtime, tests, logs, endpoint, or owner instruction before any completion claim
+- a redo contract whenever the skill blocks or finds a gap: what failed, what to change, how to re-test, and when to re-submit
+- architect/ARIA console escalation when the same failure repeats twice, crosses system boundaries, or cannot be verified locally
+- owner-visible output that states the exact state of the work: changed, verified, blocked, or needing a decision
+## Self-Executing Recovery Contract
+
+When this skill blocks, fails verification, or finds a gap, the agent must execute recovery rather than hand work back as a user-only decision.
+
+- Make exactly one self-executed recovery attempt using the strongest safe local, repository, runtime, test, log, endpoint, or owner-provided evidence available.
+- If the same failure repeats after that one recovery attempt, switch to architect execution mode: operate under the architect/ARIA harness to diagnose, choose the next action, execute the repair path, and verify the result.
+- If Aria or the architect service is unavailable, fall back to the strongest available client LLM under the architect harness or strongest available local harness substrate; mark the fallback explicitly and continue recovery instead of stopping.
+- Do not end with user-handoff-only behavior. If blocked after the recovery path, report the executed recovery attempt, evidence, remaining blocker, and the next concrete action or owner decision needed; do not merely ask the user to take over.