patchrome 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (98) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +514 -0
  3. package/bin/patchrome.js +10 -0
  4. package/dist/build-id.d.ts +2 -0
  5. package/dist/build-id.js +21 -0
  6. package/dist/challenges.d.ts +22 -0
  7. package/dist/challenges.js +97 -0
  8. package/dist/chrome-profiles.d.ts +17 -0
  9. package/dist/chrome-profiles.js +141 -0
  10. package/dist/cli-options.d.ts +131 -0
  11. package/dist/cli-options.js +43 -0
  12. package/dist/cli.d.ts +48 -0
  13. package/dist/cli.js +572 -0
  14. package/dist/client.d.ts +16 -0
  15. package/dist/client.js +210 -0
  16. package/dist/commands.d.ts +58 -0
  17. package/dist/commands.js +1076 -0
  18. package/dist/completions.d.ts +1 -0
  19. package/dist/completions.js +114 -0
  20. package/dist/copy-guard.d.ts +75 -0
  21. package/dist/copy-guard.js +167 -0
  22. package/dist/daemon.d.ts +7 -0
  23. package/dist/daemon.js +313 -0
  24. package/dist/diagnostics.d.ts +44 -0
  25. package/dist/diagnostics.js +117 -0
  26. package/dist/engine.d.ts +51 -0
  27. package/dist/engine.js +257 -0
  28. package/dist/events.d.ts +41 -0
  29. package/dist/events.js +106 -0
  30. package/dist/extract.d.ts +27 -0
  31. package/dist/extract.js +62 -0
  32. package/dist/focus.d.ts +1 -0
  33. package/dist/focus.js +44 -0
  34. package/dist/glob.d.ts +4 -0
  35. package/dist/glob.js +63 -0
  36. package/dist/har.d.ts +105 -0
  37. package/dist/har.js +88 -0
  38. package/dist/history.d.ts +35 -0
  39. package/dist/history.js +277 -0
  40. package/dist/host-platform.d.ts +5 -0
  41. package/dist/host-platform.js +19 -0
  42. package/dist/host-prompts-macos.d.ts +2 -0
  43. package/dist/host-prompts-macos.js +102 -0
  44. package/dist/host-prompts-wsl.d.ts +6 -0
  45. package/dist/host-prompts-wsl.js +64 -0
  46. package/dist/host-prompts.d.ts +3 -0
  47. package/dist/host-prompts.js +25 -0
  48. package/dist/index.d.ts +17 -0
  49. package/dist/index.js +47 -0
  50. package/dist/network.d.ts +54 -0
  51. package/dist/network.js +204 -0
  52. package/dist/origin-storage.d.ts +31 -0
  53. package/dist/origin-storage.js +82 -0
  54. package/dist/paths.d.ts +17 -0
  55. package/dist/paths.js +52 -0
  56. package/dist/pipe.d.ts +9 -0
  57. package/dist/pipe.js +73 -0
  58. package/dist/profile-mode.d.ts +10 -0
  59. package/dist/profile-mode.js +42 -0
  60. package/dist/protocol-help.d.ts +34 -0
  61. package/dist/protocol-help.js +66 -0
  62. package/dist/protocol.d.ts +49 -0
  63. package/dist/protocol.js +89 -0
  64. package/dist/refs.d.ts +9 -0
  65. package/dist/refs.js +46 -0
  66. package/dist/routes.d.ts +20 -0
  67. package/dist/routes.js +106 -0
  68. package/dist/runner.d.ts +20 -0
  69. package/dist/runner.js +81 -0
  70. package/dist/session-name.d.ts +9 -0
  71. package/dist/session-name.js +50 -0
  72. package/dist/session-store.d.ts +5 -0
  73. package/dist/session-store.js +58 -0
  74. package/dist/sessions.d.ts +47 -0
  75. package/dist/sessions.js +171 -0
  76. package/dist/tab-groups.d.ts +9 -0
  77. package/dist/tab-groups.js +13 -0
  78. package/dist/targets.d.ts +43 -0
  79. package/dist/targets.js +229 -0
  80. package/dist/validate.d.ts +3 -0
  81. package/dist/validate.js +31 -0
  82. package/dist/wait.d.ts +24 -0
  83. package/dist/wait.js +88 -0
  84. package/examples/go/go.mod +3 -0
  85. package/examples/go/main.go +104 -0
  86. package/examples/hn-front-page.sh +18 -0
  87. package/examples/hn-front-page.ts +24 -0
  88. package/examples/hn_front_page.py +56 -0
  89. package/extension/tab-groups/manifest.json +8 -0
  90. package/extension/tab-groups/service-worker.js +41 -0
  91. package/package.json +60 -0
  92. package/skills/patchrome/SKILL.md +74 -0
  93. package/skills/patchrome/references/commands.md +130 -0
  94. package/skills/patchrome/references/debugging.md +20 -0
  95. package/skills/patchrome/references/hard-pages.md +49 -0
  96. package/skills/patchrome/references/logins.md +46 -0
  97. package/skills/patchrome/references/scraping.md +51 -0
  98. package/skills/patchrome/references/scripting.md +79 -0
@@ -0,0 +1,49 @@
1
+ # Hard pages: iframes, shadow roots, CAPTCHAs, bot walls
2
+
3
+ ## iframes
4
+
5
+ Snapshots include iframes, cross-site ones too; their refs carry a different frame prefix, like
6
+ `@f2e4`. Locators need the frame: `click --role checkbox --name Verify --frame 'iframe[title*=widget]'`.
7
+
8
+ ## Closed shadow roots
9
+
10
+ A widget inside a closed shadow root shows up empty in the snapshot. CSS selectors pierce it:
11
+
12
+ ```bash
13
+ patchrome click --selector '#cb' --frame 'iframe[src*="challenges"]'
14
+ patchrome fill --selector 'input#code' --frame 'iframe' 123456
15
+ ```
16
+
17
+ ## Canvas and anything without an element
18
+
19
+ Take a screenshot, read the pixel position, then:
20
+
21
+ ```bash
22
+ patchrome click --at 120,340 # trusted click at viewport pixels, into whatever iframe is under it
23
+ patchrome type "hello" # trusted keystrokes into the focused element
24
+ ```
25
+
26
+ ## CAPTCHAs
27
+
28
+ Never solve one yourself: not by clicking its checkbox, reading its images or typing its characters.
29
+
30
+ ```bash
31
+ patchrome challenge # none | pending | solved, with vendor and box
32
+ patchrome challenge --handoff # raises the tab for the user, waits up to 10 min
33
+ ```
34
+
35
+ When `challenge` reports `pending`, tell the user, run `challenge --handoff`, and carry on once it
36
+ prints `solved` or `none`.
37
+
38
+ ## Bot-wall interstitials
39
+
40
+ "Just a moment", "Prove your humanity" and similar pages often clear by themselves within seconds. One
41
+ early read reports a block that is not there:
42
+
43
+ ```bash
44
+ patchrome open https://www.reddit.com/r/programming/
45
+ patchrome wait --title "Prove your humanity" --gone --timeout-ms 20000
46
+ patchrome snapshot
47
+ ```
48
+
49
+ Only a wait that times out means blocked. Then run `challenge`, and hand off if it reports `pending`.
@@ -0,0 +1,46 @@
1
+ # Logins and state
2
+
3
+ Cookies and logins are shared by every session in a profile: sign in once and every agent is signed
4
+ in. Never log out of a site another agent may be using.
5
+
6
+ ## Pick the way in
7
+
8
+ 1. The user says they are signed in to the site in their everyday Chrome: `state import <site>`.
9
+ 2. Otherwise, a person signs in: `login <url> --until <url-glob>`.
10
+ 3. A saved state file from earlier: `state load <file>`.
11
+ 4. A different account, or a clean cookie jar: `open --isolated` as your session's first tab.
12
+
13
+ ## `state import`
14
+
15
+ ```bash
16
+ patchrome state import github.com --from "Profile 1"
17
+ ```
18
+
19
+ `--from` takes the name in Chrome's profile menu, the folder (`Default`, `Profile 1`) or the signed-in
20
+ email; without it, the last used profile. Ask which profile when the user has several; a wrong
21
+ `--from` lists them. The site covers its subdomains. Google accounts use device-bound sessions and
22
+ may refuse an imported login; use `login` for those.
23
+
24
+ ## `login`
25
+
26
+ A visible tab for a person. Tell the user a Chrome window is waiting, and pass `--until` with a URL
27
+ the site reaches after sign-in. Without `--until` it returns when the user closes the tab. It waits
28
+ up to 10 minutes.
29
+
30
+ ## Approvals
31
+
32
+ `state import` and `state load` wait up to a minute for the user to approve with Touch ID (Windows
33
+ Hello on WSL). Tell the user a prompt is coming before you run one. `copy_denied` means they refused:
34
+ do not retry, and never copy cookies or profile files another way. `audit` lists past copies.
35
+
36
+ ## `state save` and `state load`
37
+
38
+ `state save <file>` writes cookies and localStorage for the sites your session visited, in
39
+ Playwright's storageState format. The file holds live session tokens: keep it out of git.
40
+ `state load <file>` adds to the shared profile without clearing it, so every session sees the cookies.
41
+
42
+ ## Isolated sessions
43
+
44
+ `open --isolated` on your session's first tab gives it an in-memory cookie jar. Its cookies vanish
45
+ when the session closes or the daemon restarts, and its tabs are not grouped in Chrome. `state load`
46
+ and `state import` in an isolated session fill only its own jar.
@@ -0,0 +1,51 @@
1
+ # Scraping
2
+
3
+ ## Read the API response first
4
+
5
+ Most shops and apps render from JSON. The response the page already fetched beats parsing HTML:
6
+
7
+ ```bash
8
+ patchrome open https://shop.example/product/42 --wait networkidle
9
+ patchrome network list --type xhr,fetch # n17 t2 200 fetch GET https://shop.example/api/product/42 88ms
10
+ patchrome network get n17 --body # JSON inline, or a file path past 2 KB
11
+ patchrome network get --url '*/api/product/*' --body --inline # by URL: the newest match
12
+ ```
13
+
14
+ A response body lives only as long as its page. Read it before navigating away, or `tab_gone` follows.
15
+
16
+ ## Rows from HTML
17
+
18
+ ```bash
19
+ patchrome extract '{"rows": "li.product", "fields": {"name": "h2", "url": {"selector": "a", "attr": "href"}, "tags": {"selector": ".tag", "all": true}}, "limit": 50}'
20
+ ```
21
+
22
+ - A field is a selector string (its visible text), or `{selector, attr, all}`.
23
+ - No `selector` means the row itself. `href` and `src` come back absolute. A missing element is `null`.
24
+ - Pass a file path instead of inline JSON for long schemas.
25
+ - `--role`, `--selector` or `--ref` limits rows to inside one element.
26
+
27
+ ## Globs
28
+
29
+ Globs match the whole URL: `*` is any run of characters, `?` one character. `*/api/items` does not
30
+ match `/api/items?page=2`; write `*/api/items*`. Quote them in the shell.
31
+
32
+ ## Recording and shaping traffic
33
+
34
+ ```bash
35
+ patchrome network har start
36
+ # ... browse ...
37
+ patchrome network har stop --out traffic.har # your session's requests, with text bodies
38
+ patchrome route block '*/analytics/*'
39
+ patchrome route mock '*/api/flags*' flags.json
40
+ patchrome route clear
41
+ ```
42
+
43
+ Routes intercept every request on your tabs while any rule exists, which turns off the cache. Clear
44
+ them when done.
45
+
46
+ ## Pages that change after load
47
+
48
+ - `click` returns once the click lands; `wait --url` or `wait --text` for the next page before reading.
49
+ - Paginate with a locator: `click --role link --name Next --exact`, then `wait --url '*page=2*'`.
50
+ - Keep volume low on protected sites: one page at a time, no tight `goto` loops.
51
+ - For a scrape the user will rerun, pass `--inline` or `--out <file>` so output keeps one shape.
@@ -0,0 +1,79 @@
1
+ # Scripting a flow
2
+
3
+ ## Contents
4
+ - Export what worked
5
+ - Output that keeps its shape
6
+ - `patchrome pipe`
7
+ - The Node library
8
+ - Sessions in scripts
9
+
10
+ ## Export what worked
11
+
12
+ Explore with refs, then export the successful commands as a script. The daemon rewrites each ref as
13
+ the role and name from the snapshot it came from.
14
+
15
+ ```bash
16
+ patchrome session history clear # before exploring
17
+ # ... open, snapshot, fill @e5, click @e6, extract ...
18
+ patchrome session history --out scrape.sh # runnable sh
19
+ patchrome session history --format jsonl --out scrape.jsonl
20
+ sh scrape.sh # or: patchrome pipe --bail < scrape.jsonl
21
+ ```
22
+
23
+ History leaves out snapshots, tabs, session commands and failed commands. Read every `# check:` line
24
+ before handing the script over:
25
+
26
+ - a repeated role and name got `--nth` from its position on the recorded page
27
+ - a ref inside an iframe needs `--frame <iframe-css>`
28
+ - text typed into a password field became `$PATCHROME_SECRET` (sh) or `<secret>` (jsonl)
29
+ - tab ids and `login`, `challenge --handoff`, `state import` steps need a person or may differ
30
+
31
+ Run the export once in a fresh session (`PATCHROME_SESSION=try-1 sh scrape.sh`) before calling it done.
32
+
33
+ ## Output that keeps its shape
34
+
35
+ Without flags, `text`, `eval`, `extract`, `cookies`, `network list` and `network get --body` print a
36
+ value up to 2 KB and a path past that. Scripts pass `--inline` (always the value) or `--out <file>`
37
+ (always the file). With `--json`, values keep their JSON type: `eval` answers `{"value": ...}`,
38
+ `extract` answers `{"count", "rows": [...]}`. Use `network get --url <glob>`, not request ids.
39
+
40
+ ## `patchrome pipe`
41
+
42
+ One request per line on stdin: the words after `patchrome` as a JSON array, or
43
+ `{"id": ..., "argv": [...]}`. One response per line on stdout: the `--json` object plus `id` (the line
44
+ number when none was given).
45
+
46
+ ```bash
47
+ printf '%s\n' '["open", "https://news.ycombinator.com"]' \
48
+ '{"id": "top", "argv": ["extract", "{\"rows\": \"tr.athing\", \"fields\": {\"title\": \".titleline > a\"}}", "--inline"]}' \
49
+ | patchrome pipe
50
+ # {"id":1,"ok":true,"data":{"tab":"t1","url":"https://news.ycombinator.com/","title":"Hacker News"}}
51
+ # {"id":"top","ok":true,"data":{"count":30,"rows":[{"title":"..."}]}}
52
+ ```
53
+
54
+ - Requests for one session run in order; `--session` inside `argv` runs other sessions at once.
55
+ - `watch` and `console --follow` send `{"id", "stream"}` lines first and do not block later requests.
56
+ - `--bail` stops at the first failure. Exit code 0 when every request succeeded, 1 otherwise.
57
+ - Kept open as a coprocess, one pipe serves a whole Python or Go program with no process per step.
58
+
59
+ A Python caller writes `json.dumps({"id": n, "argv": [...]})` to the pipe's stdin and reads lines
60
+ until one has that `id` and no `stream` key. The package's `examples/` folder has sh, Python, Node and
61
+ Go versions.
62
+
63
+ ## The Node library
64
+
65
+ ```ts
66
+ import { CommandError, connect } from "patchrome";
67
+
68
+ const browser = connect({ session: "prices" });
69
+ await browser.run("open", "https://shop.example");
70
+ const { rows } = await browser.run("extract", "schema.json", "--inline");
71
+ await browser.stream(["watch", "--events", "response", "--count", "3"], (event) => console.log(event));
72
+ ```
73
+
74
+ `run` resolves to the `--json` `data` and throws `CommandError` with `code`.
75
+
76
+ ## Sessions in scripts
77
+
78
+ A script started from your shell shares your session and tabs. Give each run its own:
79
+ `PATCHROME_SESSION=scrape-$$` in sh, `connect({ session })` in Node. End with `session close`.