opensteer 0.8.18 → 0.9.0

package/skills/opensteer/SKILL.md

@@ -1,179 +1,142 @@
 ---
 name: opensteer
-description: "Handles Opensteer browser automation, structured DOM extraction, and browser-backed API reverse engineering. Use when the user mentions Opensteer, browser automation, real Chromium sessions, DOM extraction, network capture, reverse-engineering a site API, or browser-grade fetch."
+description: "Use when the task needs real browser automation, DOM exploration, browser session state, network capture, or browser-backed request replay with Opensteer. The default pattern is: explore with the CLI first, then write the final code with the SDK."
 argument-hint: "[goal]"
 ---
 
 # Opensteer
 
-Opensteer gives agents a real Chromium browser for two jobs normal code cannot do:
+Use Opensteer when normal code is not enough because the task depends on a real browser:
 
-1. **DOM automation** — interact with pages and extract structured data via snapshot-based element targeting.
-2. **API reverse engineering** — capture real browser traffic, identify APIs, test transport portability, and write reusable code.
+- interact with a real page
+- inspect browser state like cookies or storage
+- capture real network traffic from real browser actions
+- replay requests through the browser session
+- turn the discovery into plain TypeScript
 
-The workflow is always: **explore with CLI, then write reusable code with SDK.**
+The default workflow is:
 
-## When To Use
+1. Use the CLI to explore the site.
+2. Figure out the page or API behavior.
+3. Save stable targets with `persist` when useful.
+4. Write the final reusable code with the SDK.
 
-- Task involves a website's API, network traffic, auth headers, or replay → **API workflow**
-- Task involves page content, forms, clicking, typing, or extracting visible data → **DOM workflow**
-- Task involves browser profiles, attaching to Chrome, or workspace setup → **Browser management**
-- Unsure → Start with the API workflow. Capture traffic first, then decide.
-
-## Rules
-
-1. Set `--workspace <id>` on every command, or export `OPENSTEER_WORKSPACE`.
-2. Re-snapshot after every navigation before using element numbers.
-3. Import as `import { Opensteer } from "opensteer"` — never a relative path.
-4. SDK constructor needs only two fields:
-   ```ts
-   const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
-   ```
-5. `persist` is the only naming mechanism for reusable targets and extractions.
-6. `--capture-network <label>` is opt-in. Add it to any action when you need traffic.
-7. `opensteer.fetch()` works without a page open — it uses the session cookie jar and transport stack directly.
-8. Element numbers come from `c="N"` attributes in snapshot HTML. Always snapshot first, then act.
+Do not stop at manual exploration if the user needs automation. Explore first, then convert the result into SDK code.
 
----
+## When To Use
 
-## DOM Automation
+- The task needs a real browser session.
+- The task involves clicks, forms, DOM extraction, or navigation.
+- The task involves cookies, localStorage, sessionStorage, or auth state.
+- The task involves reverse-engineering a site API from real browser traffic.
+- The task needs browser-backed `fetch()` instead of plain Node HTTP.
+- The task needs coordinate-based fallback because DOM targeting is not enough.
 
-### Step 1: Open and snapshot
+If the user wants to manually drive a browser and record the flow, use the `recorder` skill instead.
 
-```bash
-opensteer open https://example.com --workspace demo
-opensteer snapshot action --workspace demo
-```
+## Core Rules
 
-Read the `html` output. Find `c="N"` markers — these are your element IDs.
+1. Always use a workspace for stateful commands: `--workspace <id>` or `OPENSTEER_WORKSPACE`.
+2. In this repo, prefer `pnpm run opensteer:local -- <command>` instead of bare `opensteer ...`.
+3. Re-snapshot after navigation or big UI changes before reusing element numbers.
+4. Use the CLI to discover. Use the SDK for the final implementation.
+5. Use `persist` for stable reusable targets and extraction payloads.
+6. Use `exec` for SDK code and API experiments. Use `evaluate` only for page-context JavaScript.
+7. If `fetch()` fails with auth errors, inspect `state()`, `cookies()`, and `storage()` before changing transport.
+8. Keep the final output simple. Prefer ordinary TypeScript with `Opensteer`, not extra abstraction unless the user asks for it.
+9. Close the browser when you are done. Do not leave headed browser windows running on the user's machine. Use `opensteer browser delete --workspace <id>` or the matching SDK cleanup when the session does not need to stay open.
 
-### Step 2: Interact
+## Choose A Path
 
-```bash
-opensteer click 7 --workspace demo --persist "search button"
-opensteer input 5 "laptop" --workspace demo --press-enter --persist "search input"
-opensteer hover 3 --workspace demo --persist "menu trigger"
-opensteer scroll down 400 --workspace demo
-```
+- Page interaction or extraction: use the DOM path.
+- API discovery or replay: use the network path.
+- Canvas, WebGL, or hard-to-target UI: use computer-use.
+- Browser profile, attach mode, or workspace management: use browser management.
+- Unsure: start by capturing network traffic.
 
-`--persist <key>` saves the element's structural DOM path for deterministic SDK replay. Element number is always required on CLI — persist is save-only, not a targeting mode.
+## DOM Path
 
-### Step 3: Extract
+Use this when the goal is clicking, typing, navigating, or extracting visible data.
 
-Re-snapshot, then extract with a JSON schema referencing element numbers:
+### CLI exploration
 
 ```bash
+opensteer open https://example.com --workspace demo
+opensteer snapshot action --workspace demo
+opensteer input 5 "laptop" --workspace demo --press-enter --persist "search input"
+opensteer click 7 --workspace demo --persist "search button"
 opensteer snapshot extraction --workspace demo
 opensteer extract '{"items":[{"name":{"element":13},"price":{"element":14}}]}' \
-  --persist "search results" --workspace demo
+  --workspace demo \
+  --persist "search results"
 ```
 
-The positional argument is the JSON schema. `--persist` names it for SDK replay.
-
-### Step 4: Close
-
-```bash
-opensteer close --workspace demo
-```
+Element numbers come from `c="N"` markers in the snapshot HTML.
 
-### SDK Replay
+### SDK implementation
 
 ```ts
 import { Opensteer } from "opensteer";
-const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
-
-try {
-  await opensteer.open("https://example.com");
 
-  // Replay by persist key — no element numbers or snapshots needed
-  await opensteer.input({ persist: "search input", text: "laptop", pressEnter: true });
-  await opensteer.click({ persist: "search button" });
+const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
 
-  // Extract using cached schema
-  const data = await opensteer.extract({ persist: "search results" });
-  console.log(data);
-} finally {
-  await opensteer.close();
-}
+await opensteer.open("https://example.com");
+await opensteer.input({ persist: "search input", text: "laptop", pressEnter: true });
+await opensteer.click({ persist: "search button" });
+const data = await opensteer.extract({ persist: "search results" });
 ```
 
-When `persist` is used with `element`, it **saves** the path. When used alone, it **resolves** from cache.
+Use `selector` in SDK code only when a stable CSS selector is cleaner than `persist`.
 
----
+## Network Path
 
-## API Reverse Engineering
+Use this when the goal is to find or replay a site API.
 
-### Step 1: Capture traffic
-
-Open the site and trigger the real browser action with `--capture-network`:
+### CLI exploration
 
 ```bash
 opensteer open https://example.com --workspace demo
 opensteer goto https://example.com/search --workspace demo --capture-network page-load
 opensteer input 5 "laptop" --workspace demo --press-enter --capture-network search
+opensteer network query --workspace demo --capture search --json
+opensteer network detail rec_123 --workspace demo --probe
 ```
 
-### Step 2: Find the API
-
-```bash
-opensteer network query --workspace demo --capture search
-opensteer network query --workspace demo --capture search --hostname api.example.com --json
-```
-
-`--json` filters to JSON and GraphQL responses only. Other filters: `--url`, `--path`, `--method`, `--status`, `--type`, `--before`, `--after`, `--limit`.
-
-### Step 3: Inspect and probe transport
-
-```bash
-opensteer network detail rec_123 --workspace demo
-opensteer network detail rec_123 --probe --workspace demo
-```
-
-The first command shows: URL, method, request headers, cookies sent, request/response body preview, GraphQL metadata, redirect chain.
-
-Add `--probe` to also test which transport works for this API:
+Use `network detail --probe` to learn which transport works.
 
-| Transport     | Meaning                        | SDK usage                                       |
-| ------------- | ------------------------------ | ----------------------------------------------- |
-| `direct-http` | Plain HTTP works               | `this.fetch(url)` (default)                     |
-| `matched-tls` | Needs TLS fingerprint matching | `this.fetch(url, { transport: "matched-tls" })` |
-| `page-http`   | Needs a live browser page      | `this.fetch(url, { transport: "page" })`        |
+### Session state checks
 
-### Step 4: Check browser state (if 401/403)
+Use these when auth or browser state matters:
 
 ```bash
 opensteer state example.com --workspace demo
 ```
 
-Look for session cookies, CSRF tokens, or storage-backed auth that the request depends on.
-
-### Step 5: Test and write code with exec
+```ts
+const cookies = await opensteer.cookies("example.com");
+const localStorage = await opensteer.storage("example.com", "local");
+const sessionStorage = await opensteer.storage("example.com", "session");
+const state = await opensteer.state("example.com");
+```
 
-Use `exec` to test the API call with the SDK. The code you write here is the same code that goes in your final script:
+### Prove the request with `exec`
 
 ```bash
 opensteer exec "
-  const r = await this.fetch('https://api.example.com/search', {
+  const response = await this.fetch('https://api.example.com/search', {
     method: 'POST',
     headers: { 'content-type': 'application/json' },
     body: JSON.stringify({ keyword: 'laptop', count: 24 }),
   });
-  return { status: r.status, data: await r.json() };
+  return { status: response.status, data: await response.json() };
 " --workspace demo
 ```
 
-`exec` runs JavaScript with the Opensteer SDK bound as `this`. It supports `await` and has access to `this.fetch()`, `this.cookies()`, `this.evaluate()`, and all other SDK methods.
-
-IMPORTANT: Use `exec` instead of `evaluate` for API calls. `evaluate` runs inside the browser page where anti-bot scripts can detect and block programmatic requests. `exec` runs through the framework's transport stack which automatically finds the least detectable path.
-
-### Step 6: Write the final SDK script
-
-```bash
-opensteer close --workspace demo
-```
+### SDK implementation
 
 ```ts
 import { Opensteer } from "opensteer";
+
 const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
 
 export async function search(keyword: string) {
@@ -186,161 +149,75 @@ export async function search(keyword: string) {
 }
 ```
 
-`opensteer.fetch()` accepts standard Web Fetch API syntax (`body` as string, `headers` as `Record<string, string>`). It auto-selects the best transport and includes browser cookies by default.
+Use ordinary `fetch()` syntax. Only set `transport` explicitly if probing showed you need it.
 
----
-
-## Browser Management
+## Computer-Use
 
-### Import a Chrome profile
-
-Copy cookies, localStorage, and session state from an existing Chrome installation:
+Use this only when DOM targeting is not enough.
 
 ```bash
-opensteer browser clone --workspace my-site \
-  --source-user-data-dir "$HOME/Library/Application Support/Google/Chrome" \
-  --source-profile-directory Default
+opensteer computer click 245 380 --workspace demo --capture-network action
+opensteer computer type "search query" --workspace demo
+opensteer computer key Enter --workspace demo
+opensteer computer screenshot --workspace demo
 ```
 
-### Attach to a running browser
-
-Start Chrome with remote debugging, then connect Opensteer:
-
-```bash
-# Terminal 1: launch Chrome
-/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222
-
-# Terminal 2: attach Opensteer
-opensteer open https://example.com --workspace demo --attach-endpoint http://localhost:9222
+```ts
+await opensteer.computerExecute({
+  action: { type: "click", x: 245, y: 380 },
+});
 ```
 
-### Headful mode
+After coordinate-based actions, switch back to normal extraction or request analysis as soon as possible.
 
-```bash
-opensteer open https://example.com --workspace demo --headless false
-```
+## Browser Management
 
-### Workspace lifecycle
+Use these when the task is about browser setup rather than page logic.
 
 ```bash
+opensteer browser discover
+opensteer browser inspect --attach-endpoint http://localhost:9222
+opensteer browser clone --workspace demo \
+  --source-user-data-dir "$HOME/Library/Application Support/Google/Chrome" \
+  --source-profile-directory Default
 opensteer browser status --workspace demo
-opensteer browser reset --workspace demo     # reset browser data, keep workspace
-opensteer browser delete --workspace demo    # delete workspace entirely
-```
-
----
-
-## Tabs
-
-```bash
-opensteer tab list --workspace demo
-opensteer tab new https://other-page.com --workspace demo
-opensteer tab 2 --workspace demo             # switch to tab 2
-opensteer tab close 2 --workspace demo
-```
-
-## Run JavaScript
-
-Use `exec` for API calls and SDK operations (runs in Node.js, avoids bot detection):
-
-```bash
-opensteer exec "await this.fetch('https://api.example.com/data').then(r => r.json())" --workspace demo
-```
-
-Use `evaluate` only for DOM inspection (runs inside the browser page):
-
-```bash
-opensteer evaluate "document.title" --workspace demo
+opensteer browser reset --workspace demo
+opensteer browser delete --workspace demo
 ```
 
-## Computer-Use (Coordinate-Based)
-
-For canvas, WebGL, or complex iframes where DOM element targeting fails:
+Attach to an existing browser:
 
 ```bash
-opensteer computer click 245 380 --workspace demo --capture-network action
-opensteer computer type "search query" --workspace demo
-opensteer computer key Enter --workspace demo
-opensteer computer scroll 400 300 --dx 0 --dy -200 --workspace demo
-opensteer computer screenshot --workspace demo
+opensteer open https://example.com --workspace demo --attach-endpoint http://localhost:9222
 ```
 
----
+Cloud mode:
 
-## CLI Quick Reference
-
-| Command                                                    | Positional args      | Key flags                                                               |
-| ---------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------- |
-| `open <url>`                                               | url                  | `--headless`, `--provider`, `--attach-endpoint`, `--attach-header`      |
-| `close`                                                    | —                    | —                                                                       |
-| `status`                                                   | —                    | —                                                                       |
-| `goto <url>`                                               | url                  | `--capture-network`                                                     |
-| `snapshot [mode]`                                          | action \| extraction | —                                                                       |
-| `click <element>`                                          | element number       | `--persist`, `--capture-network`, `--button`                            |
-| `hover <element>`                                          | element number       | `--persist`, `--capture-network`                                        |
-| `input <element> <text>`                                   | element, text        | `--persist`, `--press-enter`, `--capture-network`                       |
-| `scroll <dir> <amount>`                                    | direction, amount    | `--element`, `--persist`, `--capture-network`                           |
-| `extract <schema>`                                         | JSON schema          | `--persist`                                                             |
-| `evaluate <script>`                                        | JS expression        | —                                                                       |
-| `network query`                                            | —                    | `--capture`, `--url`, `--hostname`, `--json`, `--limit`, +6 filters     |
-| `network detail <id>`                                      | recordId             | `--probe`                                                               |
-| `fetch <url>`                                              | url                  | `--method`, `--header`, `--query`, `--body`, `--transport`, `--cookies` |
-| `state [domain]`                                           | domain (optional)    | —                                                                       |
-| `exec <expression>`                                        | JS expression        | —                                                                       |
-| `tab list / new / <n> / close`                             | varies               | —                                                                       |
-| `computer click/type/key/scroll/move/drag/screenshot/wait` | varies               | `--capture-network`                                                     |
-
-## SDK Quick Reference
+- Use `--provider cloud` on CLI when needed.
+- Common env vars are `OPENSTEER_BASE_URL`, `OPENSTEER_API_KEY`, and `OPENSTEER_CLOUD_APP_BASE_URL`.
 
-```ts
-// Browser lifecycle
-await opensteer.open(url);
-await opensteer.goto(url, { captureNetwork?: "label" });
-await opensteer.close();
-
-// DOM actions — save to cache
-await opensteer.click({ element: 7, persist: "name" });
-await opensteer.input({ element: 5, text: "...", persist: "name", pressEnter: true });
-await opensteer.hover({ element: 3, persist: "name" });
-await opensteer.scroll({ direction: "down", amount: 400 });
-
-// DOM actions — resolve from cache
-await opensteer.click({ persist: "name" });
-await opensteer.input({ persist: "name", text: "..." });
-
-// Extraction
-await opensteer.extract({ persist: "name" });                    // cached schema
-await opensteer.extract({ persist: "name", schema: { ... } });   // inline schema
-
-// Network discovery
-const records = await opensteer.network.query({ capture: "label", limit: 20 });
-const detail = await opensteer.network.detail(recordId);
-
-// Fetch — standard Web Fetch API syntax, auto-selects transport
-const response = await opensteer.fetch(url, {
-  method: "POST",
-  headers: { "content-type": "application/json" },
-  body: JSON.stringify({ keyword: "laptop" }),
-});
+## Useful SDK Surface
 
-// Browser state
-const cookies = await opensteer.cookies("domain.com"); // .has(), .get(), .serialize()
-const state = await opensteer.state("domain.com");
+Use these often:
 
-// Snapshots
-const html = await opensteer.snapshot("action");
-const html = await opensteer.snapshot("extraction");
-```
-
----
+- `open(url)`
+- `goto(url, { captureNetwork? })`
+- `snapshot("action" | "extraction")`
+- `click()`, `hover()`, `input()`, `scroll()`
+- `extract()`
+- `network.query()`
+- `network.detail()`
+- `waitForNetwork()`, `waitForResponse()`, `waitForPage()`
+- `cookies()`, `storage()`, `state()`
+- `fetch()`
+- `computerExecute()`
+- `addInitScript()`
+- `browser.status()`, `browser.clone()`, `browser.reset()`, `browser.delete()`
 
-## Common Issues
+## Guardrails
 
-| Symptom                                      | Fix                                                                                                |
-| -------------------------------------------- | -------------------------------------------------------------------------------------------------- |
-| Element numbers wrong after navigation       | Re-snapshot before using element numbers                                                           |
-| `fetch()` returns 401/403                    | Check `state` — request depends on session cookies or tokens                                       |
-| Direct HTTP blocked, browser transport works | Site uses TLS fingerprinting — use `transport: "matched-tls"`                                      |
-| Extract returns empty data                   | Element numbers changed — re-snapshot and rebuild the schema                                       |
-| `fetch()` fails with no session              | Call `opensteer.goto(url)` first to establish cookies, then `fetch()`                              |
-| Using `evaluate` for API calls               | Use `exec` instead — `evaluate` runs inside the page where anti-bot scripts can intercept requests |
+- Snapshot before using element numbers.
+- Snapshot again after UI changes.
+- Do not use `evaluate` for API work.
+- Do not keep the result as a manual-only workflow if the user needs reusable automation.
+- Prefer a small final script over a large framework.

package/skills/recorder/SKILL.md

@@ -81,4 +81,4 @@ Cloud recording requires:
 ## References
 
 - [Recorder Reference](references/recorder-reference.md)
-- [Opensteer SDK Reference](../opensteer/references/sdk-reference.md)
+- [Opensteer Skill](../opensteer/SKILL.md)