opensteer 0.5.1 → 0.5.3

package/skills/opensteer/SKILL.md

@@ -0,0 +1,168 @@
+---
+name: opensteer
+description: "Browser automation, web scraping, and structured data extraction using Opensteer CLI and SDK. Use when the agent needs to: navigate web pages, interact with elements (click, type, select, hover), extract structured data from pages, take snapshots or screenshots, manage browser tabs and cookies, or generate scraper/automation scripts. Also use when the user asks to create a scraper, automation script, or replay a browsing session as code."
+---
+
+# Opensteer Browser Automation
+
+Opensteer provides persistent browser automation via a CLI and TypeScript SDK. It maintains browser sessions across calls and caches resolved element paths for deterministic replay.
+
+## CRITICAL: Always Use Opensteer Methods Over Playwright
+
+Opensteer methods are optimized for scraping — they handle waiting, element resolution, and selector caching automatically. **Never use raw Playwright when an Opensteer method exists.**
+
+| Wrong (raw Playwright)                                                        | Right (Opensteer)                                              |
+| ----------------------------------------------------------------------------- | -------------------------------------------------------------- |
+| `page.evaluate(() => [...document.querySelectorAll('.item')].map(...))`       | `opensteer.extract({ description: "product listing" })`        |
+| `page.click('.submit')`                                                       | `opensteer.click({ description: "the submit button" })`        |
+| `page.fill('#search', 'query')`                                              | `opensteer.input({ description: "search input", text: "q" })`  |
+
+**Why:** `opensteer.extract()` caches structural selectors that work across pages sharing the same template. Raw `querySelectorAll` is brittle, non-replayable, and bypasses the caching system. The only valid use of `opensteer.page.evaluate()` is calling `fetch()` for API-based extraction when a site has internal REST/GraphQL endpoints.
+
+## Default Workflow
+
+**Always use the CLI for exploration first. Only write scripts when the user asks.**
+
+1. **Explore with CLI** — Open pages, snapshot, interact with elements interactively
+2. **Cache selectors** — Re-run actions with `--description` flags to cache element paths for replay
+3. **Cache extractions** — Run `extract` with `--description` for every page type the scraper will visit
+4. **Generate script** — Use cached descriptions in TypeScript (no counters needed)
+
+**Namespace links CLI and SDK.** The `--name` flag on `opensteer open` defines the cache namespace. `new Opensteer({ name: "..." })` in the SDK reads from the same cache. These must match.
+
+## CLI Exploration
+
+```bash
+# 1. Set session once per shell
+export OPENSTEER_SESSION=my-session
+
+# 2. Open page with namespace
+opensteer open https://example.com/products --name "product-scraper"
+
+# 3. Snapshot for interactions or data
+opensteer snapshot action       # Interactive elements with counters
+opensteer snapshot extraction   # Data-oriented HTML with counters
+
+# 4. Interact using counter numbers from snapshot
+opensteer click 3
+opensteer input 5 "laptop" --pressEnter
+
+# 5. Cache actions with --description for replay
+opensteer click 3 --description "the products link"
+opensteer input 5 "laptop" --pressEnter --description "the search input"
+
+# 6. Extract data: snapshot extraction → identify counters → extract with schema
+opensteer snapshot extraction
+opensteer extract '{"products":[{"name":{"element":11},"price":{"element":12}},{"name":{"element":25},"price":{"element":26}}]}' \
+  --description "product listing with name and price"
+
+# 7. Cache extractions for ALL page types the scraper will visit
+opensteer click 11 --description "first product link"
+opensteer snapshot extraction
+opensteer extract '{"title":{"element":3},"price":{"element":7}}' \
+  --description "product detail page"
+
+# 8. Always close when done
+opensteer close
+```
+
+**Key rules:**
+
+- Set `--name` on `open` to define cache namespace
+- Specify snapshot mode explicitly: `action` (interactions) or `extraction` (data)
+- `snapshot extraction` shows structure; `extract` produces JSON — never parse snapshot HTML manually
+- Use `--description` to cache selectors for replay (one character difference = cache miss)
+- For arrays, include all items in the schema — Opensteer caches the structural pattern and finds all matches on replay
+- `open` does raw `page.goto()`; use `navigate` for subsequent pages (includes stability wait)
+- Re-snapshot after navigation or significant page changes
+
+## Writing Scraper Scripts
+
+Read [sdk-reference.md](references/sdk-reference.md) for exact method signatures before writing any script.
+
+### Template
+
+```typescript
+import { Opensteer } from "opensteer";
+
+async function run() {
+  const opensteer = new Opensteer({
+    name: "product-scraper", // MUST match --name from CLI exploration
+    storage: { rootDir: process.cwd() },
+  });
+
+  await opensteer.launch({ headless: false });
+
+  try {
+    await opensteer.goto("https://example.com/products");
+
+    await opensteer.input({
+      text: "laptop",
+      description: "the search input", // exact match to CLI --description
+    });
+
+    // Use extract with description — no schema needed when cache exists
+    const data = await opensteer.extract({
+      description: "product listing with name and price",
+    });
+
+    console.log(JSON.stringify(data, null, 2));
+  } finally {
+    await opensteer.close();
+  }
+}
+
+run().catch((err) => {
+  console.error(err);
+  process.exit(1);
+});
+```
+
+### Script Rules
+
+- No top-level `await` — wrap in `async function run()` + `run().catch(...)`
+- Default to `headless: false` (many sites block headless)
+- Use cached `description` strings for all interactions and extractions
+- Do NOT add wait calls before SDK actions — they handle waiting internally
+- Use `opensteer.waitForText("literal text")` or `page.waitForSelector("css")` only for page transitions or confirming SPA content loaded
+- Run with: `npx tsx scraper.ts`
+
+## Browser Connection
+
+- **Sandbox (default):** `opensteer open <url>` — fresh Chromium, no user sessions
+- **Connect (existing browser):** `opensteer open --connect-url http://localhost:9222` — attach to a running CDP-enabled browser. Verify CDP: `curl -s http://127.0.0.1:9222/json/version`
+
+## Element Targeting (preference order)
+
+1. **Counter** (from snapshot): `click 5` — fast, needs fresh snapshot
+2. **Description** (cached): `click --description "the submit button"` — replayable
+3. **CSS selector**: `click --selector "#btn"` — explicit but brittle
+
+## Snapshot Modes
+
+```bash
+opensteer snapshot action      # Interactable elements (default)
+opensteer snapshot extraction  # Flattened HTML for data extraction
+opensteer snapshot clickable   # Only clickable elements
+opensteer snapshot scrollable  # Only scrollable containers
+opensteer snapshot full        # Raw HTML — only for debugging
+```
+
+All modes except `full` are intelligently filtered to show only relevant elements with counters.
+
+## Debugging
+
+When a scraper produces wrong or missing data, diagnose in this order:
+
+1. **Timing** — SPA content not rendered. Add `waitForSelector` or `waitForText` before extraction.
+2. **Missing cache** — Forgot to cache extraction during CLI exploration for a page type.
+3. **Obstacles** — Cookie banners, modals, or login walls blocking the target.
+4. **Missing data** — Some pages genuinely lack certain fields. Handle with null checks.
+
+**Do NOT replace `opensteer.extract()` with `page.evaluate()` + `querySelectorAll` when debugging.** The extraction logic is not the problem — fix timing, caching, or obstacles instead.
+
+## Reference
+
+- CLI commands: [cli-reference.md](references/cli-reference.md)
+- SDK API: [sdk-reference.md](references/sdk-reference.md)
+- Full examples: [examples.md](references/examples.md)

package/skills/opensteer/references/cli-reference.md

@@ -0,0 +1,154 @@
+# Opensteer CLI Command Reference
+
+All commands output JSON. Set session once per shell:
+
+```bash
+export OPENSTEER_SESSION=my-session
+# Or for non-interactive runners:
+export OPENSTEER_CLIENT_ID=agent-1
+```
+
+Global flags: `--session <id>`, `--name <namespace>`, `--headless`, `--description <text>`.
+
+## Navigation
+
+```bash
+opensteer open <url>                                # Open browser, navigate to URL
+opensteer open <url> --name "my-scraper"            # Set selector cache namespace
+opensteer open <url> --headless                     # Headless mode
+opensteer open --connect-url http://localhost:9222   # Connect to running browser
+opensteer navigate <url>                            # Navigate with visual stability wait
+opensteer navigate <url> --timeout 60000            # Custom timeout (default 30s)
+opensteer back                                      # Go back in history
+opensteer forward                                   # Go forward in history
+opensteer reload                                    # Reload page
+opensteer close                                     # Close browser and stop server
+opensteer close --all                               # Close all active sessions
+opensteer sessions                                  # List active sessions
+opensteer status                                    # Show resolved session/name state
+```
+
+`open` does raw `page.goto()` (no stability wait). `navigate` includes `waitForVisualStability`. Use `open` once to start, then `navigate` for subsequent pages.
+
+## Observation
+
+```bash
+opensteer snapshot action           # Same as above (explicit)
+opensteer snapshot extraction       # Flattened HTML for data scraping
+opensteer snapshot clickable        # Only clickable elements
+opensteer snapshot scrollable       # Only scrollable containers
+opensteer snapshot full             # Minimal cleaning, full HTML
+opensteer state                     # URL + title + cleaned HTML
+opensteer screenshot                # Save screenshot to screenshot.png
+opensteer screenshot output.png     # Save to specific file
+opensteer screenshot --fullPage     # Full page screenshot
+```
+
+## Actions
+
+First positional argument is element counter (`c="N"` from snapshot).
+
+```bash
+opensteer click 5                                   # Click by counter
+opensteer click --description "the submit button"   # By cached description
+opensteer click 5 --button right                    # Right-click
+opensteer click 5 --clickCount 2                    # Double-click
+opensteer hover 4                                   # Hover over element
+opensteer hover --description "the user menu"       # Hover by description
+```
+
+## Input
+
+```bash
+opensteer input 3 "Hello"                           # Type into element (clears first)
+opensteer input 3 "Hello" --clear false             # Append text
+opensteer input 3 "query" --pressEnter              # Type and press Enter
+opensteer input --description "the search input" --text "query" --pressEnter
+```
+
+## Select / Scroll
+
+```bash
+opensteer select 9 --label "Option A"              # Select by visible label
+opensteer select 9 --value "opt-a"                  # Select by value attribute
+opensteer select 9 --index 2                        # Select by index
+opensteer scroll                                    # Scroll page down (default)
+opensteer scroll --direction up                     # Scroll up
+opensteer scroll --direction down --amount 1000
+opensteer scroll 12                                 # Scroll within container element
+```
+
+## Keyboard
+
+```bash
+opensteer press Enter
+opensteer press Tab
+opensteer press Escape
+opensteer press "Control+a"
+opensteer type "Hello World"                        # Type into focused element
+```
+
+## Element Info
+
+```bash
+opensteer get-text 5                                # Get element text content
+opensteer get-text --description "the heading"
+opensteer get-value 3                               # Get input/textarea value
+opensteer get-attrs 5                               # Get all HTML attributes
+opensteer get-html                                  # Full page HTML
+opensteer get-html "main"                           # HTML of element matching selector
+```
+
+## Tabs
+
+```bash
+opensteer tabs                                      # List open tabs with indices
+opensteer tab-new                                   # Open new blank tab
+opensteer tab-new https://example.com               # Open URL in new tab
+opensteer tab-switch 0                              # Switch to tab by index
+opensteer tab-close                                 # Close current tab
+opensteer tab-close 2                               # Close specific tab
+```
+
+## Cookies
+
+```bash
+opensteer cookies                                   # Get all cookies
+opensteer cookies --url https://example.com         # Cookies for specific URL
+opensteer cookie-set --name token --value abc123
+opensteer cookies-clear                             # Clear all cookies
+opensteer cookies-export /tmp/cookies.json          # Export to file
+opensteer cookies-import /tmp/cookies.json          # Import from file
+```
+
+## Utility
+
+```bash
+opensteer eval "document.title"                     # Execute JS in page
+opensteer wait-for "Success"                        # Wait for text to appear
+opensteer wait-for "Success" --timeout 5000
+opensteer wait-selector "h1"                        # Wait for selector to appear
+```
+
+## Data Extraction
+
+### Counter-based (preferred)
+
+```bash
+opensteer snapshot extraction
+# Read counters from output, then:
+opensteer extract '{"title":{"element":3},"price":{"element":7}}'
+opensteer extract '{"url":{"element":5,"attribute":"href"}}'
+opensteer extract '{"pageUrl":{"source":"current_url"},"title":{"element":3}}'
+
+# Arrays: include multiple items to identify the pattern
+opensteer extract '{"results":[{"title":{"element":11},"url":{"element":10,"attribute":"href"}},{"title":{"element":16},"url":{"element":15,"attribute":"href"}}]}'
+```
+
+### AI-based (limited and requires LLM API keys)
+
+```bash
+opensteer extract '{"title":"","price":""}' --description "product details"
+```
+
+Always prefer counter-based. AI extraction requires `@ai-sdk/*` packages and does NOT work from workspace root scripts.

package/skills/opensteer/references/examples.md

@@ -0,0 +1,116 @@
+# Opensteer Examples
+
+## Full Workflow: CLI Exploration to Scraper Script
+
+### Step 1: Explore and cache with CLI
+
+```bash
+export OPENSTEER_SESSION=eures-session
+
+opensteer open https://europa.eu/eures/portal/jv-se/home --name "eures-jobs"
+
+opensteer snapshot action
+opensteer input 5 "software engineer" --pressEnter --description "the job search input"
+opensteer click 12 --description "the search button"
+
+# Wait for results, then extract job listings
+opensteer snapshot extraction
+opensteer extract '{"jobs":[{"title":{"element":20},"company":{"element":22},"url":{"element":20,"attribute":"href"}},{"title":{"element":35},"company":{"element":37},"url":{"element":35,"attribute":"href"}},{"title":{"element":50},"company":{"element":52},"url":{"element":50,"attribute":"href"}}]}' \
+  --description "job listing with title company and url"
+
+# Cache detail page extraction too
+opensteer click 20 --description "first job link"
+opensteer snapshot extraction
+opensteer extract '{"title":{"element":3},"company":{"element":7},"location":{"element":12},"description":{"element":18}}' \
+  --description "job detail page"
+
+opensteer close
+```
+
+### Step 2: Generate replay script (same namespace, same descriptions)
+
+```typescript
+import { Opensteer } from "opensteer";
+
+async function run() {
+  const opensteer = new Opensteer({
+    name: "eures-jobs",
+    storage: { rootDir: process.cwd() },
+  });
+
+  await opensteer.launch({ headless: false });
+
+  try {
+    await opensteer.goto("https://europa.eu/eures/portal/jv-se/home");
+
+    await opensteer.input({
+      text: "software engineer",
+      description: "the job search input",
+    });
+    await opensteer.click({ description: "the search button" });
+
+    await opensteer.waitForText("Showing 1 to 10");
+
+    // Extract all job listings using cached description — no schema needed
+    const listings = await opensteer.extract({
+      description: "job listing with title company and url",
+    });
+
+    // Visit each detail page and extract using cached description
+    for (const job of listings.jobs) {
+      await opensteer.goto(job.url);
+      await opensteer.page.waitForSelector("h1");
+
+      const detail = await opensteer.extract({
+        description: "job detail page",
+      });
+      console.log(JSON.stringify(detail, null, 2));
+    }
+  } finally {
+    await opensteer.close();
+  }
+}
+
+run().catch((err) => {
+  console.error(err);
+  process.exit(1);
+});
+```
+
+## API-Based Extraction
+
+When a site has internal APIs (REST, GraphQL, Algolia), navigate first for cookies, then use `fetch()` inside `page.evaluate()`. This is the only valid use of `page.evaluate()` for data.
+
+```typescript
+import { Opensteer } from "opensteer";
+
+async function run() {
+  const opensteer = new Opensteer({
+    name: "api-scraper",
+    storage: { rootDir: process.cwd() },
+  });
+
+  await opensteer.launch({ headless: false });
+
+  try {
+    // Navigate first to establish session cookies
+    await opensteer.goto("https://example.com");
+
+    const data = await opensteer.page.evaluate(async () => {
+      const res = await fetch("https://api.example.com/search?q=shoes&limit=100", {
+        headers: { "Content-Type": "application/json" },
+      });
+      return res.json();
+    });
+
+    console.log(JSON.stringify(data, null, 2));
+  } finally {
+    await opensteer.close();
+  }
+}
+
+run().catch((err) => {
+  console.error(err);
+  process.exit(1);
+});
+```

package/skills/opensteer/references/sdk-reference.md

@@ -0,0 +1,143 @@
+# Opensteer SDK API Reference
+
+The SDK is the `Opensteer` class imported from `'opensteer'`. **Only the methods listed below exist.** Do NOT call CLI command names as SDK methods.
+
+## Construction and Lifecycle
+
+```typescript
+const opensteer = new Opensteer({
+  name: "my-scraper",
+  storage: { rootDir: process.cwd() },
+});
+await opensteer.launch({ headless: false });
+await opensteer.close();
+
+// Or wrap an existing Playwright page:
+const opensteer = Opensteer.from(existingPage, { name: "my-scraper" });
+```
+
+## Properties
+
+```typescript
+opensteer.page;    // Raw Playwright Page — only for page.evaluate(fetch), page.waitForSelector, page.waitForTimeout
+opensteer.context; // Raw Playwright BrowserContext
+```
+
+## Navigation
+
+```typescript
+await opensteer.goto(url);                      // Navigate + waitForVisualStability
+await opensteer.goto(url, { timeout: 60000 });  // Custom timeout
+```
+
+## Observation
+
+```typescript
+const html = await opensteer.snapshot();                          // Action mode (default)
+const html = await opensteer.snapshot({ mode: "extraction" });    // Extraction mode
+const state = await opensteer.state();                            // { url, title, html }
+const buffer = await opensteer.screenshot();                      // PNG buffer
+const jpeg = await opensteer.screenshot({ type: "jpeg", fullPage: true });
+```
+
+## Interactions
+
+```typescript
+await opensteer.click({ element: 5 });
+await opensteer.click({ description: "the submit button" });
+await opensteer.click({ selector: "#btn" });
+await opensteer.dblclick({ element: 7 });
+await opensteer.rightclick({ element: 7 });
+await opensteer.hover({ element: 4 });
+await opensteer.input({ element: 3, text: "Hello" });
+await opensteer.input({ description: "search", text: "q", pressEnter: true });
+await opensteer.select({ element: 9, label: "Option A" });
+await opensteer.scroll();
+await opensteer.scroll({ direction: "up", amount: 500 });
+```
+
+## Data Extraction
+
+```typescript
+// Replay from cached descriptions (preferred in scraper scripts)
+const data = await opensteer.extract({
+  description: "product details",
+});
+
+// Counter-based (during exploration or when no cache exists)
+const data = await opensteer.extract({
+  schema: { title: { element: 3 }, price: { element: 7 } },
+  description: "product details",
+});
+```
+
+Schema field types: `{ element: N }`, `{ element: N, attribute: "href" }`, `{ selector: ".price" }`, `{ source: "current_url" }`.
+
+For arrays, include multiple items in the schema. Opensteer caches the structural pattern and expands to all matching items on replay.
+
+## Keyboard
+
+```typescript
+await opensteer.pressKey("Enter");
+await opensteer.pressKey("Control+a");
+await opensteer.type("Hello World");
+```
+
+## Element Info
+
+```typescript
+const text = await opensteer.getElementText({ element: 5 });
+const value = await opensteer.getElementValue({ element: 3 });
+const attrs = await opensteer.getElementAttributes({ element: 5 });
+const box = await opensteer.getElementBoundingBox({ element: 5 });
+const html = await opensteer.getHtml();
+const html = await opensteer.getHtml("main");
+const title = await opensteer.getTitle();
+```
+
+## Wait
+
+**Do NOT use wait calls before SDK actions** — each action handles waiting internally. Only use explicit waits for page transitions or confirming SPA content loaded.
+
+```typescript
+await opensteer.waitForText("Success");                              // Literal text on page
+await opensteer.waitForText("Success", { timeout: 5000 });
+await opensteer.page.waitForSelector("article");                     // CSS selector
+await opensteer.page.waitForSelector(".loading", { state: "hidden" });
+```
+
+## Tabs
+
+```typescript
+const tabs = await opensteer.tabs();
+await opensteer.newTab("https://example.com");
+await opensteer.switchTab(0);
+await opensteer.closeTab(1);
+```
+
+## Cookies
+
+```typescript
+const cookies = await opensteer.getCookies();
+await opensteer.setCookie({ name: "token", value: "abc" });
+await opensteer.clearCookies();
+await opensteer.exportCookies("/tmp/cookies.json");
+await opensteer.importCookies("/tmp/cookies.json");
+```
+
+## File Upload
+
+```typescript
+await opensteer.uploadFile({ element: 5, paths: ["/path/to/file.pdf"] });
+```
+
+## Methods That DO NOT Exist
+
+| Wrong (throws)                   | Correct                                |
+| -------------------------------- | -------------------------------------- |
+| `opensteer.evaluate(...)`        | `opensteer.page.evaluate(...)`         |
+| `opensteer.waitForSelector(...)` | `opensteer.page.waitForSelector(...)`  |
+| `opensteer.waitForLoad(...)`     | `opensteer.page.waitForLoadState(...)` |
+| `opensteer.navigate(...)`        | `opensteer.goto(...)`                  |
+| `opensteer.browser_launch(...)`  | `opensteer.launch(...)`                |
+| `opensteer.browser_close(...)`   | `opensteer.close(...)`                 |