@ulpi/browse 2.3.4 → 2.4.0

package/skill/browse-seo/SKILL.md

@@ -0,0 +1,188 @@
+---
+name: browse-seo
+version: 1.0.0
+description: |
+  SEO auditing for AI agents using the browse CLI. Extracts meta tags, headings, structured data,
+  performance metrics, link structure, and mobile rendering. Generates actionable audit reports
+  with findings and recommendations. Works with any URL.
+allowed-tools:
+  - Bash
+  - Read
+argument-hint: "[URL to audit]"
+arguments:
+  - request
+when_to_use: |
+  Use when the user asks for an SEO audit, SEO check, SEO analysis, on-page SEO review,
+  or says /browse-seo. Signs you should use this skill:
+  - User wants to check meta tags, structured data, or heading hierarchy
+  - User wants a technical SEO report for a page or site
+  - User asks about Open Graph tags, Twitter cards, or schema markup
+  - User wants to check mobile-friendliness or Core Web Vitals
+  Do NOT use for general browsing, form filling, or non-SEO tasks.
+effort: medium
+---
+
+# browse-seo: SEO Audit for AI Agents
+
+## Goal
+
+Run a complete on-page SEO audit for a target URL using the `browse` CLI. Extract technical
+SEO signals, analyze them, and produce a report with findings and recommendations.
+
+## Step 0: Consider stealth setup (optional)
+
+If the audit target is behind bot detection (Cloudflare, competitor sites), use camoufox:
+
+```bash
+browse --runtime camoufox --headed goto <url>
+```
+
+For most public pages, the default Chromium runtime is sufficient. Skip unless you hit blocks.
+
+## Step 1: Navigate to the target URL
+
+```bash
+browse goto <url>
+browse wait --network-idle
+```
+
+Verify the page loaded. Note any redirects -- redirect chains matter for SEO. If the page
+returns 403 or a challenge, switch to camoufox (Step 0).
+
+## Step 2: Extract meta tags
+
+```bash
+browse meta
+```
+
+Returns title, description, canonical, OG tags, Twitter Card, robots, and viewport. Check:
+
+- **Title**: present, under 60 chars, contains target keywords
+- **Description**: present, 120-160 chars, compelling and unique
+- **Canonical**: present, correct URL (trailing slashes, www vs non-www)
+- **Robots**: no accidental `noindex` or `nofollow`
+- **Viewport**: `width=device-width, initial-scale=1` for mobile
+- **OG tags**: `og:title`, `og:description`, `og:image`, `og:url` all present
+- **Twitter Card**: `twitter:card`, `twitter:title`, `twitter:description` present
+
+## Step 3: Check heading structure
+
+```bash
+browse headings
+```
+
+Returns H1-H6 hierarchy with counts. Check:
+
+- **Exactly one H1**: multiple H1s dilute the primary topic signal
+- **H1 matches topic**: should contain the primary keyword
+- **Logical nesting**: H2 under H1, H3 under H2 -- no skipped levels
+- **Keyword presence**: target keywords in H1 and H2 tags
+
+## Step 4: Extract structured data
+
+```bash
+browse schema
+```
+
+Extracts JSON-LD, Microdata, and RDFa from the page. Check:
+
+- **Presence**: at least one schema type (Article, Product, Organization, etc.)
+- **Required properties**: each type needs required fields per Google's docs
+- **Valid JSON-LD**: no syntax errors or malformed objects
+- **Correct @type**: matches actual page content
+- **Missing opportunities**: FAQ, HowTo, Breadcrumb, Review where content supports it
+
+## Step 5: Check performance
+
+```bash
+browse perf
+```
+
+Returns navigation timing (DNS, TTFB, DOM load, full load). Check:
+
+- **TTFB**: under 200ms good, over 600ms problem
+- **DOM Content Loaded**: under 1.5s acceptable
+- **Full load**: under 3s for good UX
+- **DNS**: near zero with proper prefetching
+
+Use `browse network` to inspect render-blocking resources, large assets, and request counts.
+
+## Step 6: Analyze link structure
+
+```bash
+browse links
+```
+
+Returns all anchor elements with text and href. Check:
+
+- **Internal links**: sufficient linking for crawlability
+- **Anchor text**: descriptive, not "click here" or bare URLs
+- **External links**: appropriate outbound links, nofollow where needed
+- **Broken links**: anything pointing to 404 or error pages
+
+## Step 7: Check mobile rendering
+
+```bash
+browse responsive
+```
+
+Captures screenshots at mobile (375px), tablet (768px), and desktop (1440px). Review for:
+
+- **Mobile layout**: no horizontal scrolling, readable without zooming
+- **Tap targets**: 48px minimum for touch
+- **Content parity**: mobile has same critical content as desktop
+- **Font size**: 16px minimum base font
+
+Use `browse screenshot` for a single full-page capture if needed.
+
+## Step 8: Custom checks via JavaScript
+
+Run targeted checks the built-in commands do not cover:
+
+```bash
+# Image alt tags
+browse js "const imgs=[...document.querySelectorAll('img')]; const missing=imgs.filter(i=>!i.alt&&!i.getAttribute('role')); JSON.stringify({total:imgs.length,missingAlt:missing.length,examples:missing.slice(0,5).map(i=>i.src.slice(-60))})"
+
+# Duplicate meta tags
+browse js "JSON.stringify({titles:document.querySelectorAll('title').length,descriptions:document.querySelectorAll('meta[name=description]').length,canonicals:document.querySelectorAll('link[rel=canonical]').length})"
+
+# Hreflang tags
+browse js "JSON.stringify([...document.querySelectorAll('link[rel=alternate][hreflang]')].map(t=>({lang:t.hreflang,href:t.href})))"
+
+# Lazy images without dimensions
+browse js "const lazy=[...document.querySelectorAll('img[loading=lazy]')]; const noDims=lazy.filter(i=>!i.width&&!i.height&&!i.getAttribute('width')); JSON.stringify({lazyImages:lazy.length,missingDimensions:noDims.length})"
+```
+
+Check robots.txt and sitemap:
+
+```bash
+browse goto <origin>/robots.txt
+browse text
+browse goto <origin>/sitemap.xml
+browse text
+```
+
+## Step 9: Generate the audit report
+
+After collecting data from Steps 2-8, produce a structured report:
+
+1. **Summary**: one-paragraph overview with pass/fail/warning counts
+2. **Meta Tags**: title, description, canonical, robots, OG, Twitter -- each rated
+3. **Heading Structure**: H1 count, nesting issues, keyword presence
+4. **Structured Data**: schema types found, missing opportunities, validation issues
+5. **Performance**: TTFB, load times, blocking resources
+6. **Link Analysis**: internal/external ratio, broken links, anchor text quality
+7. **Mobile Friendliness**: viewport tag, responsive screenshots, tap target issues
+8. **Images**: alt tag coverage, lazy loading, dimension attributes
+9. **Recommendations**: prioritized fixes (critical, important, nice-to-have)
+
+Rate each section as PASS, WARNING, or FAIL. Prioritize recommendations by impact.
+
+## Important Rules
+
+- The browser persists between commands. Cookies, tabs, and session state carry over.
+- After `goto`, always `wait --network-idle` before extracting data.
+- The agent interprets all results. Browse extracts raw data; analysis is your job.
+- Do not install anything automatically.
+- Save screenshots under `.browse/` directories.
+- For sites that block bots, use `--runtime camoufox --headed`.

package/skill/browse-stealth/SKILL.md

@@ -0,0 +1,246 @@
+---
+name: browse-stealth
+version: 2.0.0
+description: |
+  Stealth browsing with the camoufox runtime for AI agents. Covers Cloudflare Turnstile bypass,
+  Google anti-bot evasion, authenticated sessions, and proxy rotation. Uses the browse CLI with
+  --runtime camoufox and named camoufox profiles.
+allowed-tools:
+  - Bash
+  - Read
+argument-hint: "[URL or site that blocks normal browsers]"
+arguments:
+  - request
+when_to_use: |
+  Use when the user asks for stealth browsing, anti-detection, bot bypass, Turnstile, or says
+  /browse-stealth. Signs you need this:
+  - Page shows CAPTCHA or "verify you are human" challenge
+  - Page returns blank/403/challenge page instead of content
+  - Google shows "unusual traffic" block
+  - Site uses Cloudflare, DataDome, PerimeterX, or similar bot protection
+  Do NOT use for sites that load fine with regular browse -- camoufox is slower to start.
+effort: high
+---
+
+# browse-stealth: Stealth Browsing with Camoufox
+
+## Step 0: Setup via /browse-config
+
+Before launching a stealth session, generate a camoufox configuration profile.
+
+Run `/browse-config` to generate a stealth profile, or use the Google-safe preset.
+
+Presets available in /browse-config:
+- **Stealth browsing** -- geoip + humanize for general anti-detection
+- **Google-safe** -- geoip + humanize + random OS for Google searches
+- **Fast scraping** -- blocks images, WebRTC, WebGL; enables cache
+- **Custom** -- interactive walkthrough of all options
+
+The `--camoufox-profile` flag only applies when starting a NEW server. If a browse server is already running, stop it first with `browse stop` before switching profiles.
+
+## Step 1: Launch with camoufox
+
+Stop any existing server, then launch with the camoufox runtime:
+
+```bash
+browse stop                                          # stop any running server
+browse --runtime camoufox --camoufox-profile <name> goto <url>
+```
+
+Replace `<name>` with the profile created in Step 0 (e.g. "stealth", "google"). If no named profile exists, the `camoufox` section in `browse.json` is used automatically:
+
+```bash
+browse stop
+browse --runtime camoufox goto <url>
+```
+
+After navigation, stabilize the page before reading or interacting:
+
+```bash
+browse wait --network-idle
+browse snapshot -i
+```
+
+## Step 2: Cloudflare Turnstile bypass
+
+### Navigate and identify the Turnstile widget
+
+```bash
+browse --runtime camoufox --camoufox-profile <name> goto https://target-site.com/login
+browse snapshot -i
+```
+
+Look in the snapshot output for a Turnstile iframe or checkbox. Common patterns:
+- An iframe with `challenges.cloudflare.com` in the src
+- A checkbox labeled "Verify you are human"
+- Elements with `cf-turnstile` in their class or ID
+
+### Wait for Turnstile to auto-resolve
+
+Many Turnstile challenges resolve automatically when humanize is enabled in the profile. Wait for it:
+
+```bash
+browse wait --network-idle
+browse snapshot -i
+```
+
+Check if the page has advanced past the challenge. If the form or main content is now visible, Turnstile passed.
+
+### Click the checkbox if Turnstile persists
+
+If the challenge is still showing after the wait:
+
+```bash
+browse click @eN                                     # use the @ref from snapshot
+```
+
+Replace `@eN` with the ref pointing to the Turnstile checkbox from the snapshot.
+
+### Verify and retry
+
+```bash
+browse snapshot -i
+```
+
+If Turnstile still blocks, increase humanize delay in the profile (set humanize to a higher value like 2.0), stop the server, and relaunch:
+
+```bash
+browse stop
+browse --runtime camoufox --camoufox-profile <name> goto https://target-site.com/login
+```
+
+## Step 3: Google anti-bot bypass
+
+### Configure the profile
+
+Use the Google-safe preset from /browse-config, which enables:
+- `geoip: true` -- spoofs location based on exit IP
+- `humanize: true` -- adds random delays to interactions
+- `os: ["windows", "macos", "linux"]` -- random OS fingerprint per launch
+
+### Navigate to Google
+
+```bash
+browse stop
+browse --runtime camoufox --camoufox-profile google goto "https://www.google.com/search?q=your+query"
+```
+
+Or use the search macro:
+
+```bash
+browse --runtime camoufox --camoufox-profile google goto @google "your query"
+```
+
+### Check for blocks
+
+```bash
+browse snapshot -i
+```
+
+Look for CAPTCHA screens or consent dialogs. If blocked:
+
+```bash
+browse text
+```
+
+Confirm the block message (e.g. "unusual traffic from your computer network").
+
+### Handle consent screens
+
+Google often shows a consent/cookie dialog before search results. Dismiss it:
+
+```bash
+browse snapshot -i                                   # find the Accept button ref
+browse click @eN                                     # click Accept/Agree
+```
+
+### Add delays between requests
+
+When making multiple Google searches, add waits between them to avoid rate limiting:
+
+```bash
+browse wait --network-idle
+```
+
+If still blocked after multiple attempts, the IP may be flagged. Switch to a different proxy (see Step 5).
+
+## Step 4: Authenticated sessions
+
+### Persistent browser profiles
+
+Use `--profile` for a persistent browser identity. Cookies, localStorage, and session data survive server restarts:
+
+```bash
+browse stop
+browse --runtime camoufox --camoufox-profile stealth --profile mysite goto https://target-site.com/login
+```
+
+Login once through the normal flow (fill credentials, click submit). On subsequent launches with the same `--profile`, cookies persist and you skip the login:
+
+```bash
+browse stop
+browse --runtime camoufox --camoufox-profile stealth --profile mysite goto https://target-site.com/dashboard
+```
+
+### Credential vault
+
+For automated login flows, save credentials to the encrypted vault:
+
+```bash
+browse auth save mysite https://target-site.com/login myuser --password-stdin <<< "mypassword"
+```
+
+Then login automatically:
+
+```bash
+browse auth login mysite
+```
+
+The vault uses AES-256-GCM encryption. Passwords are stored in `.browse/auth/` with mode 0o600.
+
+## Step 5: Proxy rotation
+
+### Configure proxy in the camoufox profile
+
+Add proxy settings to the profile JSON (via /browse-config custom mode or by editing the file directly):
+
+```json
+{
+  "geoip": true,
+  "humanize": true,
+  "proxy": {
+    "server": "http://proxy:8080",
+    "username": "user",
+    "password": "pass"
+  }
+}
+```
+
+Save this as a named profile, e.g. `.browse/camoufox-profiles/proxy-us.json`.
+
+### Switch proxies by switching profiles
+
+Create multiple profiles with different proxy endpoints:
+
+- `.browse/camoufox-profiles/proxy-us.json` -- US proxy
+- `.browse/camoufox-profiles/proxy-eu.json` -- EU proxy
+- `.browse/camoufox-profiles/proxy-asia.json` -- Asia proxy
+
+To rotate, stop the server and relaunch with a different profile:
+
+```bash
+browse stop
+browse --runtime camoufox --camoufox-profile proxy-eu goto <url>
+```
+
+Each profile switch requires a server restart because the proxy is configured at browser launch time.
+
+## Key Rules
+
+1. **`--camoufox-profile` is server-spawn-only.** Switching profiles requires `browse stop` first. The profile is read once at server startup.
+2. **`--runtime camoufox` selects the Firefox-based anti-detection engine.** It replaces the default Chromium with a hardened Firefox that spoofs fingerprints at the C++ level.
+3. **humanize adds random delays to clicks and typing.** Set to `true` for default delays, or a number (0.5-2.0) to control speed. Higher values mean slower, more human-like behavior.
+4. **geoip spoofs location based on exit IP.** When using a proxy, geoip derives timezone, locale, and geolocation from the proxy's IP address.
+5. **Never mix `--profile` (persistent browser data) with `--session` (shared Chromium multiplexing).** They serve different purposes. `--profile` persists cookies across restarts. `--session` isolates parallel agents within a single server.
+6. **IP reputation matters.** Fingerprint spoofing alone does not bypass IP-based blocks. Use residential proxies for heavily protected sites.
+7. **Headed mode is recommended.** Headless Firefox is easier to detect. Use `--headed` when anti-detection is critical, or set `headless: false` in the profile.