opensteer 0.8.18 → 0.9.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. package/README.md +5 -5
  2. package/dist/chunk-4LP7QP2O.js +4336 -0
  3. package/dist/chunk-4LP7QP2O.js.map +1 -0
  4. package/dist/{chunk-GQ3PGLYQ.js → chunk-6PGXWW3X.js} +5142 -9634
  5. package/dist/chunk-6PGXWW3X.js.map +1 -0
  6. package/dist/chunk-BMPUL66S.js +1170 -0
  7. package/dist/chunk-BMPUL66S.js.map +1 -0
  8. package/dist/{chunk-T6TG4WO2.js → chunk-L4FWHBQJ.js} +4 -3
  9. package/dist/chunk-L4FWHBQJ.js.map +1 -0
  10. package/dist/chunk-Z53HNZ7Z.js +1800 -0
  11. package/dist/chunk-Z53HNZ7Z.js.map +1 -0
  12. package/dist/cli/bin.cjs +3533 -499
  13. package/dist/cli/bin.cjs.map +1 -1
  14. package/dist/cli/bin.js +130 -11
  15. package/dist/cli/bin.js.map +1 -1
  16. package/dist/index.cjs +1382 -487
  17. package/dist/index.cjs.map +1 -1
  18. package/dist/index.d.cts +20 -27
  19. package/dist/index.d.ts +20 -27
  20. package/dist/index.js +4 -2
  21. package/dist/local-view/public/assets/app.css +770 -0
  22. package/dist/local-view/public/assets/app.js +2053 -0
  23. package/dist/local-view/public/index.html +235 -0
  24. package/dist/local-view/serve-entry.cjs +7436 -0
  25. package/dist/local-view/serve-entry.cjs.map +1 -0
  26. package/dist/local-view/serve-entry.d.cts +1 -0
  27. package/dist/local-view/serve-entry.d.ts +1 -0
  28. package/dist/local-view/serve-entry.js +23 -0
  29. package/dist/local-view/serve-entry.js.map +1 -0
  30. package/dist/opensteer-KZCRP425.js +6 -0
  31. package/dist/{opensteer-XMCWYUH3.js.map → opensteer-KZCRP425.js.map} +1 -1
  32. package/dist/session-control-VGBFOH3Y.js +39 -0
  33. package/dist/session-control-VGBFOH3Y.js.map +1 -0
  34. package/package.json +8 -8
  35. package/skills/README.md +12 -6
  36. package/skills/opensteer/SKILL.md +275 -217
  37. package/skills/recorder/SKILL.md +1 -1
  38. package/dist/chunk-GQ3PGLYQ.js.map +0 -1
  39. package/dist/chunk-T6TG4WO2.js.map +0 -1
  40. package/dist/opensteer-XMCWYUH3.js +0 -4
@@ -1,179 +1,150 @@
1
1
  ---
2
2
  name: opensteer
3
- description: "Handles Opensteer browser automation, structured DOM extraction, and browser-backed API reverse engineering. Use when the user mentions Opensteer, browser automation, real Chromium sessions, DOM extraction, network capture, reverse-engineering a site API, or browser-grade fetch."
3
+ description: "Use when the task needs real browser automation, DOM exploration, browser session state, network capture, or browser-backed request replay with Opensteer. The default pattern is: explore with the CLI first, then write the final code with the SDK."
4
4
  argument-hint: "[goal]"
5
5
  ---
6
6
 
7
7
  # Opensteer
8
8
 
9
- Opensteer gives agents a real Chromium browser for two jobs normal code cannot do:
9
+ Opensteer gives AI agents a real Chromium browser local or cloud. Use it when normal code is not enough because the task depends on a live browser session.
10
10
 
11
- 1. **DOM automation** — interact with pages and extract structured data via snapshot-based element targeting.
12
- 2. **API reverse engineering** — capture real browser traffic, identify APIs, test transport portability, and write reusable code.
11
+ Default workflow:
13
12
 
14
- The workflow is always: **explore with CLI, then write reusable code with SDK.**
13
+ 1. CLI to explore the site and discover behavior.
14
+ 2. Save stable targets with `persist`.
15
+ 3. SDK to write the final reusable TypeScript.
15
16
 
16
- ## When To Use
17
-
18
- - Task involves a website's API, network traffic, auth headers, or replay → **API workflow**
19
- - Task involves page content, forms, clicking, typing, or extracting visible data → **DOM workflow**
20
- - Task involves browser profiles, attaching to Chrome, or workspace setup → **Browser management**
21
- - Unsure → Start with the API workflow. Capture traffic first, then decide.
22
-
23
- ## Rules
24
-
25
- 1. Set `--workspace <id>` on every command, or export `OPENSTEER_WORKSPACE`.
26
- 2. Re-snapshot after every navigation before using element numbers.
27
- 3. Import as `import { Opensteer } from "opensteer"` — never a relative path.
28
- 4. SDK constructor needs only two fields:
29
- ```ts
30
- const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
31
- ```
32
- 5. `persist` is the only naming mechanism for reusable targets and extractions.
33
- 6. `--capture-network <label>` is opt-in. Add it to any action when you need traffic.
34
- 7. `opensteer.fetch()` works without a page open — it uses the session cookie jar and transport stack directly.
35
- 8. Element numbers come from `c="N"` attributes in snapshot HTML. Always snapshot first, then act.
36
-
37
- ---
17
+ Do not stop at manual exploration if the user needs automation.
38
18
 
39
- ## DOM Automation
19
+ ## Setup
40
20
 
41
- ### Step 1: Open and snapshot
21
+ Install the Opensteer skill so the coding agent can use it:
42
22
 
43
23
  ```bash
44
- opensteer open https://example.com --workspace demo
45
- opensteer snapshot action --workspace demo
24
+ opensteer skills install
46
25
  ```
47
26
 
48
- Read the `html` output. Find `c="N"` markers these are your element IDs.
27
+ This registers the skill with the agent's tool system. Only needed once per environment.
49
28
 
50
- ### Step 2: Interact
29
+ ## When To Use
51
30
 
52
- ```bash
53
- opensteer click 7 --workspace demo --persist "search button"
54
- opensteer input 5 "laptop" --workspace demo --press-enter --persist "search input"
55
- opensteer hover 3 --workspace demo --persist "menu trigger"
56
- opensteer scroll down 400 --workspace demo
57
- ```
31
+ - Real browser session needed (clicks, forms, DOM extraction, navigation).
32
+ - Cookies, localStorage, sessionStorage, or auth state involved.
33
+ - Reverse-engineering a site API from real browser traffic.
34
+ - Browser-backed `fetch()` instead of plain Node HTTP.
35
+ - Coordinate-based interaction (canvas, WebGL, hard-to-target UI).
36
+ - Need to reuse a real user's logged-in browser profile.
58
37
 
59
- `--persist <key>` saves the element's structural DOM path for deterministic SDK replay. Element number is always required on CLI — persist is save-only, not a targeting mode.
38
+ If the user wants to manually drive a browser and record the flow, use the `recorder` skill instead.
60
39
 
61
- ### Step 3: Extract
40
+ ## Core Rules
62
41
 
63
- Re-snapshot, then extract with a JSON schema referencing element numbers:
42
+ 1. Always use a workspace for stateful commands: `--workspace <id>` or `OPENSTEER_WORKSPACE`.
43
+ 2. In this repo, prefer `pnpm run opensteer:local -- <command>` instead of bare `opensteer ...`.
44
+ 3. Re-snapshot after navigation or big UI changes before reusing element numbers.
45
+ 4. CLI to discover, SDK for the final implementation.
46
+ 5. Use `persist` for stable reusable targets and extraction payloads.
47
+ 6. Use `exec` for SDK code and API experiments. Use `evaluate` only for page-context JavaScript.
48
+ 7. If `fetch()` fails with auth errors, inspect `state()`, `cookies()`, and `storage()` before changing transport.
49
+ 8. Keep output simple. Prefer ordinary TypeScript with `Opensteer`, no extra abstraction.
50
+ 9. Close the browser when done. Do not leave headed browsers running. Use `opensteer browser delete --workspace <id>` or SDK cleanup when the session does not need to stay open.
64
51
 
65
- ```bash
66
- opensteer snapshot extraction --workspace demo
67
- opensteer extract '{"items":[{"name":{"element":13},"price":{"element":14}}]}' \
68
- --persist "search results" --workspace demo
52
+ ## Choose A Path
53
+
54
+ ```
55
+ What does the task need?
56
+ ├─ Click, type, navigate, extract visible data → DOM path
57
+ ├─ Find or replay a site API → Network path
58
+ ├─ Analyze, deobfuscate, or sandbox page JavaScript → Scripts analysis
59
+ ├─ Canvas, WebGL, or hard-to-target UI → Computer-use
60
+ ├─ Work with multiple tabs or popups → Tab management
61
+ ├─ Set up browser profile, clone, or attach → Browser sessions
62
+ ├─ Run browser in the cloud → Cloud mode
63
+ ├─ Watch what a headless browser is doing → Local view
64
+ └─ Unsure → start by capturing network traffic
69
65
  ```
70
66
 
71
- The positional argument is the JSON schema. `--persist` names it for SDK replay.
67
+ ## DOM Path
72
68
 
73
- ### Step 4: Close
69
+ Use this when the goal is clicking, typing, navigating, or extracting visible data.
70
+
71
+ ### CLI exploration
74
72
 
75
73
  ```bash
76
- opensteer close --workspace demo
74
+ opensteer open https://example.com --workspace demo
75
+ opensteer snapshot action --workspace demo
76
+ opensteer input 5 "laptop" --workspace demo --press-enter --persist "search input"
77
+ opensteer click 7 --workspace demo --persist "search button"
78
+ opensteer snapshot extraction --workspace demo
79
+ opensteer extract '{"items":[{"name":{"element":13},"price":{"element":14}}]}' \
80
+ --workspace demo \
81
+ --persist "search results"
77
82
  ```
78
83
 
79
- ### SDK Replay
84
+ Element numbers come from `c="N"` markers in the snapshot HTML.
85
+
86
+ ### SDK implementation
80
87
 
81
88
  ```ts
82
89
  import { Opensteer } from "opensteer";
83
- const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
84
-
85
- try {
86
- await opensteer.open("https://example.com");
87
90
 
88
- // Replay by persist key no element numbers or snapshots needed
89
- await opensteer.input({ persist: "search input", text: "laptop", pressEnter: true });
90
- await opensteer.click({ persist: "search button" });
91
+ const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
91
92
 
92
- // Extract using cached schema
93
- const data = await opensteer.extract({ persist: "search results" });
94
- console.log(data);
95
- } finally {
96
- await opensteer.close();
97
- }
93
+ await opensteer.open("https://example.com");
94
+ await opensteer.input({ persist: "search input", text: "laptop", pressEnter: true });
95
+ await opensteer.click({ persist: "search button" });
96
+ const data = await opensteer.extract({ persist: "search results" });
98
97
  ```
99
98
 
100
- When `persist` is used with `element`, it **saves** the path. When used alone, it **resolves** from cache.
99
+ Use `selector` in SDK code only when a stable CSS selector is cleaner than `persist`.
101
100
 
102
- ---
103
-
104
- ## API Reverse Engineering
101
+ ## Network Path
105
102
 
106
- ### Step 1: Capture traffic
103
+ Use this when the goal is to find or replay a site API.
107
104
 
108
- Open the site and trigger the real browser action with `--capture-network`:
105
+ ### CLI exploration
109
106
 
110
107
  ```bash
111
108
  opensteer open https://example.com --workspace demo
112
109
  opensteer goto https://example.com/search --workspace demo --capture-network page-load
113
110
  opensteer input 5 "laptop" --workspace demo --press-enter --capture-network search
111
+ opensteer network query --workspace demo --capture search --json
112
+ opensteer network detail rec_123 --workspace demo --probe
114
113
  ```
115
114
 
116
- ### Step 2: Find the API
117
-
118
- ```bash
119
- opensteer network query --workspace demo --capture search
120
- opensteer network query --workspace demo --capture search --hostname api.example.com --json
121
- ```
122
-
123
- `--json` filters to JSON and GraphQL responses only. Other filters: `--url`, `--path`, `--method`, `--status`, `--type`, `--before`, `--after`, `--limit`.
124
-
125
- ### Step 3: Inspect and probe transport
126
-
127
- ```bash
128
- opensteer network detail rec_123 --workspace demo
129
- opensteer network detail rec_123 --probe --workspace demo
130
- ```
131
-
132
- The first command shows: URL, method, request headers, cookies sent, request/response body preview, GraphQL metadata, redirect chain.
133
-
134
- Add `--probe` to also test which transport works for this API:
115
+ Use `network detail --probe` to learn which transport works.
135
116
 
136
- | Transport | Meaning | SDK usage |
137
- | ------------- | ------------------------------ | ----------------------------------------------- |
138
- | `direct-http` | Plain HTTP works | `this.fetch(url)` (default) |
139
- | `matched-tls` | Needs TLS fingerprint matching | `this.fetch(url, { transport: "matched-tls" })` |
140
- | `page-http` | Needs a live browser page | `this.fetch(url, { transport: "page" })` |
141
-
142
- ### Step 4: Check browser state (if 401/403)
117
+ ### Session state checks
143
118
 
144
119
  ```bash
145
120
  opensteer state example.com --workspace demo
146
121
  ```
147
122
 
148
- Look for session cookies, CSRF tokens, or storage-backed auth that the request depends on.
149
-
150
- ### Step 5: Test and write code with exec
123
+ ```ts
124
+ const cookies = await opensteer.cookies("example.com");
125
+ const localStorage = await opensteer.storage("example.com", "local");
126
+ const sessionStorage = await opensteer.storage("example.com", "session");
127
+ const state = await opensteer.state("example.com");
128
+ ```
151
129
 
152
- Use `exec` to test the API call with the SDK. The code you write here is the same code that goes in your final script:
130
+ ### Prove the request with `exec`
153
131
 
154
132
  ```bash
155
133
  opensteer exec "
156
- const r = await this.fetch('https://api.example.com/search', {
134
+ const response = await this.fetch('https://api.example.com/search', {
157
135
  method: 'POST',
158
136
  headers: { 'content-type': 'application/json' },
159
137
  body: JSON.stringify({ keyword: 'laptop', count: 24 }),
160
138
  });
161
- return { status: r.status, data: await r.json() };
139
+ return { status: response.status, data: await response.json() };
162
140
  " --workspace demo
163
141
  ```
164
142
 
165
- `exec` runs JavaScript with the Opensteer SDK bound as `this`. It supports `await` and has access to `this.fetch()`, `this.cookies()`, `this.evaluate()`, and all other SDK methods.
166
-
167
- IMPORTANT: Use `exec` instead of `evaluate` for API calls. `evaluate` runs inside the browser page where anti-bot scripts can detect and block programmatic requests. `exec` runs through the framework's transport stack which automatically finds the least detectable path.
168
-
169
- ### Step 6: Write the final SDK script
170
-
171
- ```bash
172
- opensteer close --workspace demo
173
- ```
143
+ ### SDK implementation
174
144
 
175
145
  ```ts
176
146
  import { Opensteer } from "opensteer";
147
+
177
148
  const opensteer = new Opensteer({ workspace: "demo", rootDir: process.cwd() });
178
149
 
179
150
  export async function search(keyword: string) {
@@ -186,161 +157,248 @@ export async function search(keyword: string) {
186
157
  }
187
158
  ```
188
159
 
189
- `opensteer.fetch()` accepts standard Web Fetch API syntax (`body` as string, `headers` as `Record<string, string>`). It auto-selects the best transport and includes browser cookies by default.
160
+ Use ordinary `fetch()` syntax. Only set `transport` explicitly if probing showed you need it.
190
161
 
191
- ---
162
+ ## Scripts Analysis
163
+
164
+ Use this when you need to understand what JavaScript a page is running — reverse-engineering obfuscated code, finding hidden API calls, or testing script behavior in isolation.
192
165
 
193
- ## Browser Management
166
+ ### Capture scripts from the page
194
167
 
195
- ### Import a Chrome profile
168
+ ```bash
169
+ opensteer scripts capture --workspace demo
170
+ opensteer scripts capture --workspace demo --url-filter "api" --external --dynamic
171
+ ```
196
172
 
197
- Copy cookies, localStorage, and session state from an existing Chrome installation:
173
+ Flags: `--inline`, `--external`, `--dynamic`, `--workers` to filter by source type. `--persist` to save as an artifact. `--url-filter <pattern>` to match script URLs.
174
+
175
+ ### Beautify and deobfuscate
198
176
 
199
177
  ```bash
200
- opensteer browser clone --workspace my-site \
201
- --source-user-data-dir "$HOME/Library/Application Support/Google/Chrome" \
202
- --source-profile-directory Default
178
+ # Format minified code
179
+ opensteer scripts beautify <artifactId> --workspace demo --persist
180
+
181
+ # Deobfuscate packed/obfuscated code
182
+ opensteer scripts deobfuscate <artifactId> --workspace demo --persist
203
183
  ```
204
184
 
205
- ### Attach to a running browser
185
+ ### Sandbox execution
206
186
 
207
- Start Chrome with remote debugging, then connect Opensteer:
187
+ Run captured JavaScript in isolation with controlled inputs:
208
188
 
209
189
  ```bash
210
- # Terminal 1: launch Chrome
211
- /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222
190
+ opensteer scripts sandbox <artifactId> --workspace demo \
191
+ --fidelity standard \
192
+ --timeout 5000 \
193
+ --cookies '{"session":"abc123"}' \
194
+ --globals '{"window.API_KEY":"test"}' \
195
+ --ajax-routes '[{"url":"*/api/*","response":{"data":[]}}]'
196
+ ```
212
197
 
213
- # Terminal 2: attach Opensteer
214
- opensteer open https://example.com --workspace demo --attach-endpoint http://localhost:9222
198
+ Fidelity levels: `minimal` (fast, no DOM), `standard` (basic DOM), `full` (complete browser emulation).
199
+
200
+ ### Typical workflow
201
+
202
+ ```bash
203
+ opensteer scripts capture --workspace demo --persist --external
204
+ opensteer artifact read art_abc123 --workspace demo # inspect raw
205
+ opensteer scripts beautify art_abc123 --workspace demo --persist
206
+ opensteer scripts deobfuscate art_def456 --workspace demo --persist
207
+ opensteer scripts sandbox art_ghi789 --workspace demo
215
208
  ```
216
209
 
217
- ### Headful mode
210
+ ## Computer-Use
211
+
212
+ Use this only when DOM targeting is not enough.
213
+
214
+ ```bash
215
+ opensteer computer click 245 380 --workspace demo --capture-network action
216
+ opensteer computer type "search query" --workspace demo
217
+ opensteer computer key Enter --workspace demo
218
+ opensteer computer screenshot --workspace demo
219
+ ```
220
+
221
+ ```ts
222
+ await opensteer.computerExecute({
223
+ action: { type: "click", x: 245, y: 380 },
224
+ });
225
+ ```
226
+
227
+ After coordinate-based actions, switch back to normal extraction or request analysis as soon as possible.
228
+
229
+ ## Tab Management
230
+
231
+ Use when handling OAuth popups, multi-page flows, or any task that opens new tabs.
232
+
233
+ ```bash
234
+ opensteer tab list --workspace demo # List all open tabs
235
+ opensteer tab new https://example.com --workspace demo # Open new tab
236
+ opensteer tab 2 --workspace demo # Switch to tab 2
237
+ opensteer tab close 3 --workspace demo # Close tab 3
238
+ opensteer tab close --workspace demo # Close current tab
239
+ ```
240
+
241
+ ```ts
242
+ const tabs = await opensteer.listPages();
243
+ await opensteer.newPage("https://example.com");
244
+ await opensteer.activatePage(2);
245
+ await opensteer.closePage(3);
246
+ ```
247
+
248
+ Re-snapshot after switching tabs — element numbers are per-page.
249
+
250
+ ## Browser Sessions
251
+
252
+ Each workspace has one browser. Three modes:
253
+
254
+ | Mode | What it does | Data persists? |
255
+ | ------------------------ | ----------------------------------------------- | --------------------------------------------------------------------- |
256
+ | **Persistent** (default) | Browser tied to workspace, survives restarts | Yes — cookies, localStorage, logins, history, extensions all retained |
257
+ | **Temporary** | Headless browser in `/tmp`, cleaned up on close | No |
258
+ | **Attach** | Connects to an already-running browser via CDP | Depends on that browser |
259
+
260
+ ### Headless vs headed
261
+
262
+ Browsers launch headless by default. Use `--headless false` to see the browser window:
218
263
 
219
264
  ```bash
220
265
  opensteer open https://example.com --workspace demo --headless false
221
266
  ```
222
267
 
268
+ Use headed mode for debugging or when the user wants to watch. For hands-free automation, keep headless and use `opensteer view` if a human needs to observe.
269
+
270
+ ### Persistent sessions
271
+
272
+ When you `opensteer open` with a workspace, the browser's full Chrome user-data directory lives at `~/.opensteer/workspaces/<id>/browser/user-data/`. Everything Chrome normally persists (cookies, localStorage, IndexedDB, history, extensions) survives between runs.
273
+
274
+ Re-running `opensteer open --workspace demo` reconnects to the existing browser if it's still alive, or launches a fresh one with the same profile if it died.
275
+
276
+ ### Profile cloning
277
+
278
+ Clone a real user's Chrome profile to start a workspace with their logins already active:
279
+
280
+ ```bash
281
+ # Discover available local browsers and profiles
282
+ opensteer browser discover
283
+
284
+ # Clone a profile into a workspace
285
+ opensteer browser clone --workspace demo \
286
+ --source-user-data-dir "$HOME/Library/Application Support/Google/Chrome" \
287
+ --source-profile-directory Default
288
+ ```
289
+
290
+ This copies cookies, localStorage, extensions, and settings from the source browser. Caches and lock files are skipped. The source browser does not need to be closed — cloning while running is safe.
291
+
292
+ ### Attach to an existing browser
293
+
294
+ ```bash
295
+ opensteer open https://example.com --workspace demo --attach-endpoint http://localhost:9222
296
+ ```
297
+
223
298
  ### Workspace lifecycle
224
299
 
225
300
  ```bash
226
- opensteer browser status --workspace demo
227
- opensteer browser reset --workspace demo # reset browser data, keep workspace
228
- opensteer browser delete --workspace demo # delete workspace entirely
301
+ opensteer browser status --workspace demo # Check if browser is running
302
+ opensteer browser reset --workspace demo # Wipe browser data, keep workspace
303
+ opensteer browser delete --workspace demo # Delete workspace entirely
229
304
  ```
230
305
 
231
- ---
306
+ ## Cloud Mode
307
+
308
+ Run the browser on Opensteer's cloud infrastructure instead of locally. Use cloud mode when you need browsers that run 24/7, or when the local machine should not run Chromium.
232
309
 
233
- ## Tabs
310
+ ### Setup
234
311
 
235
312
  ```bash
236
- opensteer tab list --workspace demo
237
- opensteer tab new https://other-page.com --workspace demo
238
- opensteer tab 2 --workspace demo # switch to tab 2
239
- opensteer tab close 2 --workspace demo
313
+ export OPENSTEER_API_KEY=osk_your_key_here # Required
314
+ export OPENSTEER_PROVIDER=cloud # Or use --provider cloud per command
240
315
  ```
241
316
 
242
- ## Run JavaScript
317
+ ### Usage
243
318
 
244
- Use `exec` for API calls and SDK operations (runs in Node.js, avoids bot detection):
319
+ All CLI commands work the same with `--provider cloud`:
245
320
 
246
321
  ```bash
247
- opensteer exec "await this.fetch('https://api.example.com/data').then(r => r.json())" --workspace demo
322
+ opensteer open https://example.com --workspace demo --provider cloud
323
+ opensteer snapshot action --workspace demo
324
+ opensteer click 5 --workspace demo
248
325
  ```
249
326
 
250
- Use `evaluate` only for DOM inspection (runs inside the browser page):
327
+ ### Export local browser profile to cloud
328
+
329
+ Sync a local browser's cookies to a cloud browser profile so the cloud session starts logged in:
251
330
 
252
331
  ```bash
253
- opensteer evaluate "document.title" --workspace demo
332
+ # Reads cookies from local Chrome, decrypts them, uploads to cloud
333
+ opensteer browser clone --workspace demo \
334
+ --source-user-data-dir "$HOME/Library/Application Support/Google/Chrome" \
335
+ --source-profile-directory Default \
336
+ --provider cloud
254
337
  ```
255
338
 
256
- ## Computer-Use (Coordinate-Based)
339
+ The cookies are extracted from the local SQLite database, decrypted, packaged into a portable format, and uploaded. The cloud browser then starts with those cookies applied.
340
+
341
+ ## Local View
257
342
 
258
- For canvas, WebGL, or complex iframes where DOM element targeting fails:
343
+ When Opensteer runs headless, a human cannot see what the browser is doing. Local view streams live screenshots from headless sessions to a browser-based viewer.
259
344
 
260
345
  ```bash
261
- opensteer computer click 245 380 --workspace demo --capture-network action
262
- opensteer computer type "search query" --workspace demo
263
- opensteer computer key Enter --workspace demo
264
- opensteer computer scroll 400 300 --dx 0 --dy -200 --workspace demo
265
- opensteer computer screenshot --workspace demo
346
+ opensteer view # Start viewer service, print URL
347
+ opensteer view stop # Stop the viewer service
348
+ opensteer view --auto # Auto-start viewer on every browser launch
349
+ opensteer view --no-auto # Only start viewer when manually requested
266
350
  ```
267
351
 
268
- ---
352
+ The viewer is a local web UI that shows:
269
353
 
270
- ## CLI Quick Reference
271
-
272
- | Command | Positional args | Key flags |
273
- | ---------------------------------------------------------- | -------------------- | ----------------------------------------------------------------------- |
274
- | `open <url>` | url | `--headless`, `--provider`, `--attach-endpoint`, `--attach-header` |
275
- | `close` | — | — |
276
- | `status` | — | — |
277
- | `goto <url>` | url | `--capture-network` |
278
- | `snapshot [mode]` | action \| extraction | — |
279
- | `click <element>` | element number | `--persist`, `--capture-network`, `--button` |
280
- | `hover <element>` | element number | `--persist`, `--capture-network` |
281
- | `input <element> <text>` | element, text | `--persist`, `--press-enter`, `--capture-network` |
282
- | `scroll <dir> <amount>` | direction, amount | `--element`, `--persist`, `--capture-network` |
283
- | `extract <schema>` | JSON schema | `--persist` |
284
- | `evaluate <script>` | JS expression | — |
285
- | `network query` | — | `--capture`, `--url`, `--hostname`, `--json`, `--limit`, +6 filters |
286
- | `network detail <id>` | recordId | `--probe` |
287
- | `fetch <url>` | url | `--method`, `--header`, `--query`, `--body`, `--transport`, `--cookies` |
288
- | `state [domain]` | domain (optional) | — |
289
- | `exec <expression>` | JS expression | — |
290
- | `tab list / new / <n> / close` | varies | — |
291
- | `computer click/type/key/scroll/move/drag/screenshot/wait` | varies | `--capture-network` |
292
-
293
- ## SDK Quick Reference
354
+ - Live JPEG stream of the active browser tab
355
+ - Tab bar with switching
356
+ - Navigation controls (back, forward, reload, URL bar)
294
357
 
295
- ```ts
296
- // Browser lifecycle
297
- await opensteer.open(url);
298
- await opensteer.goto(url, { captureNetwork?: "label" });
299
- await opensteer.close();
300
-
301
- // DOM actions — save to cache
302
- await opensteer.click({ element: 7, persist: "name" });
303
- await opensteer.input({ element: 5, text: "...", persist: "name", pressEnter: true });
304
- await opensteer.hover({ element: 3, persist: "name" });
305
- await opensteer.scroll({ direction: "down", amount: 400 });
306
-
307
- // DOM actions — resolve from cache
308
- await opensteer.click({ persist: "name" });
309
- await opensteer.input({ persist: "name", text: "..." });
310
-
311
- // Extraction
312
- await opensteer.extract({ persist: "name" }); // cached schema
313
- await opensteer.extract({ persist: "name", schema: { ... } }); // inline schema
314
-
315
- // Network discovery
316
- const records = await opensteer.network.query({ capture: "label", limit: 20 });
317
- const detail = await opensteer.network.detail(recordId);
318
-
319
- // Fetch — standard Web Fetch API syntax, auto-selects transport
320
- const response = await opensteer.fetch(url, {
321
- method: "POST",
322
- headers: { "content-type": "application/json" },
323
- body: JSON.stringify({ keyword: "laptop" }),
324
- });
358
+ Local view is a passive observer — it connects to the browser's existing CDP endpoint. Starting or stopping it has zero impact on running browser sessions.
359
+
360
+ ## Interaction Capture & Replay
325
361
 
326
- // Browser state
327
- const cookies = await opensteer.cookies("domain.com"); // .has(), .get(), .serialize()
328
- const state = await opensteer.state("domain.com");
362
+ Record a trace of browser interactions (clicks, typing, network, DOM changes) and replay them deterministically. Useful for building repeatable test flows or comparing behavior across runs.
329
363
 
330
- // Snapshots
331
- const html = await opensteer.snapshot("action");
332
- const html = await opensteer.snapshot("extraction");
364
+ ```bash
365
+ opensteer interaction capture --workspace demo --key "login-flow" --duration 30000
366
+ opensteer interaction get <traceId> --workspace demo
367
+ opensteer interaction replay <traceId> --workspace demo
368
+ opensteer interaction diff <traceA> <traceB> --workspace demo
333
369
  ```
334
370
 
335
- ---
371
+ ## Artifacts
336
372
 
337
- ## Common Issues
373
+ Commands that use `--persist` save artifacts to the workspace. Read them back with:
374
+
375
+ ```bash
376
+ opensteer artifact read <artifactId> --workspace demo
377
+ ```
338
378
 
339
- | Symptom | Fix |
340
- | -------------------------------------------- | -------------------------------------------------------------------------------------------------- |
341
- | Element numbers wrong after navigation | Re-snapshot before using element numbers |
342
- | `fetch()` returns 401/403 | Check `state` — request depends on session cookies or tokens |
343
- | Direct HTTP blocked, browser transport works | Site uses TLS fingerprinting — use `transport: "matched-tls"` |
344
- | Extract returns empty data | Element numbers changed — re-snapshot and rebuild the schema |
345
- | `fetch()` fails with no session | Call `opensteer.goto(url)` first to establish cookies, then `fetch()` |
346
- | Using `evaluate` for API calls | Use `exec` instead — `evaluate` runs inside the page where anti-bot scripts can intercept requests |
379
+ Artifacts are created by `extract --persist`, `scripts capture --persist`, `scripts beautify --persist`, and other persist-enabled commands.
380
+
381
+ ## Useful SDK Surface
382
+
383
+ - `open(url)`, `goto(url, { captureNetwork? })`, `close()`
384
+ - `snapshot("action" | "extraction")`
385
+ - `click()`, `hover()`, `input()`, `scroll()`
386
+ - `extract()`
387
+ - `listPages()`, `newPage()`, `activatePage()`, `closePage()`
388
+ - `network.query()`, `network.detail()`
389
+ - `waitForNetwork()`, `waitForResponse()`, `waitForPage()`
390
+ - `cookies()`, `storage()`, `state()`
391
+ - `fetch()`
392
+ - `evaluate()`, `addInitScript()`
393
+ - `route()` — intercept and modify network requests
394
+ - `computerExecute()`
395
+ - `browser.status()`, `browser.clone()`, `browser.reset()`, `browser.delete()`
396
+
397
+ ## Guardrails
398
+
399
+ - Snapshot before using element numbers. Snapshot again after UI changes.
400
+ - Do not use `evaluate` for API work — use `exec` or `fetch`.
401
+ - Do not keep the result as a manual-only workflow if the user needs reusable automation.
402
+ - Prefer a small final script over a large framework.
403
+ - Close browsers when done. Do not leave headed browser windows open.
404
+ - When cloning profiles, verify the source path exists with `opensteer browser discover` first.
@@ -81,4 +81,4 @@ Cloud recording requires:
81
81
  ## References
82
82
 
83
83
  - [Recorder Reference](references/recorder-reference.md)
84
- - [Opensteer SDK Reference](../opensteer/references/sdk-reference.md)
84
+ - [Opensteer Skill](../opensteer/SKILL.md)