obxcura 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +205 -0
- data/README.md +98 -105
- data/lib/obxcura/browser.rb +37 -10
- data/lib/obxcura/client.rb +8 -5
- data/lib/obxcura/cookies.rb +363 -0
- data/lib/obxcura/frame/dom.rb +3 -4
- data/lib/obxcura/frame/runtime.rb +0 -26
- data/lib/obxcura/headers.rb +164 -0
- data/lib/obxcura/node.rb +40 -16
- data/lib/obxcura/page.rb +340 -40
- data/lib/obxcura/version.rb +1 -1
- data/lib/obxcura.rb +4 -1
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7f74cd065bb29071f0bc6468f6b1809c6b900f9c563b11a72afa9ed905b8ae3d
|
|
4
|
+
data.tar.gz: 2a5894fd3d306f10171e7f955e7142375b8354efe528e06d72816b6e44062e82
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5997307fc7e40606dfeb0f086b66ea2073dd5730be22b787fe0dff226c3a10f1873aae498544161502d37d45dcb50e374ef7cbb008ffe3e3807f9ccea5b192ca
|
|
7
|
+
data.tar.gz: b10406aa827442b519f41b7f130ad51464776edf3c31fb3bc2d7172e49f771b9043bc7309ae2135ec0ee671c0ce7c90dad5919ee902edc6a93ad41232341ddf8
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,210 @@
|
|
|
1
1
|
## [Unreleased]
|
|
2
2
|
|
|
3
|
+
## [0.4.0] - 2026-08-18
|
|
4
|
+
|
|
5
|
+
Catches the client up with Obscura 0.2.0 — which grew a native render engine —
|
|
6
|
+
and gives extra headers and cookies proper read/write collections instead of raw
|
|
7
|
+
CDP calls. As always, every behaviour below was measured against the binary
|
|
8
|
+
rather than taken from the protocol docs, and the places where Obscura departs
|
|
9
|
+
from Chrome are documented instead of papered over: cookie reads that ignore the
|
|
10
|
+
URL you ask about, deletes that compare the path exactly, expired cookies that
|
|
11
|
+
linger in the jar, and a PDF engine that draws text rather than embedding it.
|
|
12
|
+
|
|
13
|
+
One breaking change, in `Page#cookies` — see below.
|
|
14
|
+
|
|
15
|
+
Also fixes a way to kill the browser outright from the public API.
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
|
|
19
|
+
- **`Page#cookies` returns a collection, not an `Array`.** It used to hand back
|
|
20
|
+
the connection's whole jar as an array of hashes; it now returns an
|
|
21
|
+
`Obxcura::Cookies`, enumerable over the cookies the page's URL would actually
|
|
22
|
+
send. `#map`, `#select`, `#find` and friends keep working, and the old value is
|
|
23
|
+
`page.cookies.all`.
|
|
24
|
+
|
|
25
|
+
What breaks quietly is integer indexing: `page.cookies[0]` used to be the first
|
|
26
|
+
cookie hash and now looks up the cookie *named* `"0"`, so it returns `nil` —
|
|
27
|
+
and `page.cookies[0]["value"]` becomes a `NoMethodError` on nil. Reach for
|
|
28
|
+
`page.cookies.to_a[0]`, or better, `page.cookies["session"]`.
|
|
29
|
+
|
|
30
|
+
- CI pins Obscura `v0.2.0` and now installs the **render** build, since the
|
|
31
|
+
screenshot specs would otherwise skip rather than fail. It also triggered on
|
|
32
|
+
pushes to `master` while the default branch is `main`, so the push trigger
|
|
33
|
+
could never fire; both are fixed.
|
|
34
|
+
|
|
35
|
+
### Added
|
|
36
|
+
|
|
37
|
+
- **`Page#cookies`** — the browser's cookie jar seen from one page: `Enumerable`
|
|
38
|
+
over the cookies that page's URL would actually send, and writable. `#all`,
|
|
39
|
+
`#[]`, `#for_url`, `#size`, `#empty?` for reading; `#set`, `#remove`, `#clear`
|
|
40
|
+
for writing. Cookies stay raw CDP hashes with string keys, so what comes out of
|
|
41
|
+
a read goes straight back into `#set` — `fresh.cookies.set(saved)` moves a
|
|
42
|
+
logged-in session into another browser. Includes the `HttpOnly` cookies
|
|
43
|
+
`document.cookie` cannot see, which is the whole reason to read them over CDP.
|
|
44
|
+
|
|
45
|
+
`#set` takes `url:`, `domain:`, `path:`, `secure:`, `http_only:`, `same_site:`
|
|
46
|
+
and `expires:` (a `Time` or epoch seconds), defaulting to the page's current
|
|
47
|
+
URL because the browser refuses a cookie that belongs nowhere. Unknown options
|
|
48
|
+
and a bad `same_site:` raise `ArgumentError` rather than reaching the browser,
|
|
49
|
+
which accepts both silently — a bogus `sameSite` is stored as `Lax` without
|
|
50
|
+
complaint.
|
|
51
|
+
|
|
52
|
+
Four measured browser behaviours shaped this, all documented in
|
|
53
|
+
[`docs/cookies.md`](docs/cookies.md):
|
|
54
|
+
|
|
55
|
+
- **Cookie reads ignore the URL you ask about.** `Network.getCookies` accepts a
|
|
56
|
+
`urls:` parameter and Obscura ignores it — that call, `Storage.getCookies`
|
|
57
|
+
and `Network.getAllCookies` all return the whole connection-wide jar, from
|
|
58
|
+
any page's session. So the scoping is done in Ruby, following RFC 6265
|
|
59
|
+
§5.1.3–5.1.4: domain, path, `Secure`, expiry. Ports are not part of cookie
|
|
60
|
+
scope.
|
|
61
|
+
- **An expired cookie stays in the jar** and is simply never sent, so `#all`
|
|
62
|
+
can show what will never go on the wire while the scoped view drops it.
|
|
63
|
+
- **`Network.deleteCookies` compares the path exactly** — a delete aimed at
|
|
64
|
+
`/cookies` leaves a cookie set on `/` alone, silently. `#remove` therefore
|
|
65
|
+
looks the cookie up first and deletes at the domain and path the jar reports,
|
|
66
|
+
and returns whether the jar actually changed.
|
|
67
|
+
- **Host-only cookies cannot be told apart.** Chrome stores `Domain=example.com`
|
|
68
|
+
as `.example.com`; Obscura stores it verbatim, leaving nothing to distinguish
|
|
69
|
+
it from a host-only cookie. Domain matching is applied to every entry — over
|
|
70
|
+
reporting beats losing the session cookie you came for.
|
|
71
|
+
|
|
72
|
+
- **`Page#headers`** — a small mutable collection for extra HTTP headers, in the
|
|
73
|
+
shape Ferrum uses: `#get`/`#to_h`, `#set`, `#add`, `#clear`, plus `#[]`,
|
|
74
|
+
`#[]=`, `#delete`, `#key?`, `#empty?` and `#size`. Names match
|
|
75
|
+
case-insensitively per RFC 9110, so writing `x-token` after `X-Token` replaces
|
|
76
|
+
it instead of sending both. Names and values are stringified, as CDP requires.
|
|
77
|
+
|
|
78
|
+
`#get` reports what the object last sent, because that is the only readable
|
|
79
|
+
record there is: CDP has no getter, and Obscura answers
|
|
80
|
+
`Network.getExtraHTTPHeaders` with `Unknown Network method`.
|
|
81
|
+
|
|
82
|
+
Two measured behaviours the documentation calls out rather than papering over.
|
|
83
|
+
**The scope is the connection, not the page** — the API hangs off `Page`
|
|
84
|
+
because that is the only session the command works from (sent without one it is
|
|
85
|
+
accepted and silently does nothing), but Obscura keeps one table per
|
|
86
|
+
connection, so two pages on a browser cannot carry different headers and the
|
|
87
|
+
last write wins. And **they cover navigation only**: script-initiated requests
|
|
88
|
+
never carry them, so `#post` still needs its own `headers` argument.
|
|
89
|
+
|
|
90
|
+
- **`Page#pdf`** — print the page, now that Obscura 0.2.0 can. Returns the raw
|
|
91
|
+
PDF bytes, or writes them and returns the path with `path:`. Supports
|
|
92
|
+
`landscape:`, `print_background:`, `scale:` (0.1..2.0), `page_ranges:`,
|
|
93
|
+
`margin:` (one number or per side, in inches), and paper as either a name
|
|
94
|
+
(`:letter`, `:legal`, `:tabloid`, `:a3`, `:a4`, `:a5`) or explicit
|
|
95
|
+
`paper_width:`/`paper_height:` in inches.
|
|
96
|
+
|
|
97
|
+
Verified against the binary by reading the generated PDFs, not by trusting
|
|
98
|
+
byte sizes — an A4 render of the test page weighs *exactly* as much as the
|
|
99
|
+
Letter one, so only the `/MediaBox` reveals that paper size applied at all
|
|
100
|
+
(612x792pt vs 595x842pt). Landscape swaps the box, `scale:` changes the page
|
|
101
|
+
count, and `page_ranges:` trims it.
|
|
102
|
+
|
|
103
|
+
**Output is raster-backed.** Obscura reports `print-media-raster`, and a
|
|
104
|
+
generated document has zero font objects and no extractable text — measured.
|
|
105
|
+
It cannot be searched, selected, or read by a screen reader; use `#html` when
|
|
106
|
+
you need the text. Header/footer rendering and CSS `@page` sizing are not
|
|
107
|
+
implemented upstream, so those options are deliberately not exposed.
|
|
108
|
+
|
|
109
|
+
Needs a build with the render feature, like `#screenshot`; the same refusal is
|
|
110
|
+
mapped to an `Obxcura::Error` naming the asset to install. An unknown paper
|
|
111
|
+
name, `paper:` combined with explicit dimensions, and a `scale:` outside
|
|
112
|
+
0.1..2.0 all raise `ArgumentError` before any CDP round trip.
|
|
113
|
+
|
|
114
|
+
- **`Page#screenshot`** — real rasterisation, now that Obscura 0.2.0 ships a
|
|
115
|
+
native render engine. Returns the raw image bytes (base64 is a CDP transport
|
|
116
|
+
detail and is decoded here), or writes them and returns the path when given
|
|
117
|
+
`path:`. Supports `:png` / `:jpeg` / `:webp`, `quality:` for `:jpeg`,
|
|
118
|
+
`full_page:` for the whole document, and `clip:` for a region with optional
|
|
119
|
+
`scale:`. Format is inferred from the `path:` extension when not given.
|
|
120
|
+
|
|
121
|
+
Every option was verified against the 0.2.0 binary: `full_page:` really does
|
|
122
|
+
capture past the viewport (1280x720 → 1280x2400 on the test page), `clip:` with
|
|
123
|
+
`scale: 2` really doubles the raster, and `quality:` really changes the jpeg
|
|
124
|
+
encode. Obscura's webp encoder is lossless and rejects `quality:` at any value,
|
|
125
|
+
so that combination is refused client-side rather than round-tripped.
|
|
126
|
+
|
|
127
|
+
Requires a build carrying the render feature. The `-no-render` archives of the
|
|
128
|
+
same version refuse `Page.captureScreenshot`; that refusal surfaces as an
|
|
129
|
+
`Obxcura::Error` naming the asset to install, not a bare `ProtocolError`.
|
|
130
|
+
Unsupported formats and contradictory options (`full_page:` with `clip:`) raise
|
|
131
|
+
`ArgumentError` before any CDP round trip.
|
|
132
|
+
|
|
133
|
+
### Fixed
|
|
134
|
+
|
|
135
|
+
- **`Browser#create_page(url)` no longer kills the browser.** Handing a URL to
|
|
136
|
+
`Target.createTarget` works exactly once: the *second* such call on a
|
|
137
|
+
connection takes `obscura serve` down outright — every command after it fails
|
|
138
|
+
with `connection closed: end of file reached`, and the process is gone, not
|
|
139
|
+
just the socket.
|
|
140
|
+
|
|
141
|
+
```ruby
|
|
142
|
+
browser.create_page("https://www.google.com.mx") # fine
|
|
143
|
+
browser.create_page("https://example.com") # browser dies here
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Measured on 0.2.0 and fully deterministic, whatever the URLs are — two local
|
|
147
|
+
ones crash it just the same. One URL-loaded target is safe, any number of
|
|
148
|
+
`about:blank` targets is safe, navigation is safe, and two URL-loaded targets
|
|
149
|
+
on *separate* connections are safe. It is specifically the second
|
|
150
|
+
`createTarget` carrying a URL.
|
|
151
|
+
|
|
152
|
+
`#create_page` now always creates the target blank and navigates, so the
|
|
153
|
+
parameter is honoured without handing callers a way to kill the browser. It
|
|
154
|
+
blocks until the load event fires, which makes `#go_to` simply its expressive
|
|
155
|
+
name. The workaround note on `#go_to` was also slightly wrong: the crash needs
|
|
156
|
+
no `evaluate`, it lands on the `create_page` call itself.
|
|
157
|
+
|
|
158
|
+
## [0.3.0] - 2026-07-29
|
|
159
|
+
|
|
160
|
+
Realigns the client with Obscura 0.1.11, which lifted several of the browser
|
|
161
|
+
limits this gem was built around. Every change below was verified against the
|
|
162
|
+
0.1.11 binary rather than taken from the release notes — two claims in those
|
|
163
|
+
notes did not hold, and are documented as still-standing constraints.
|
|
164
|
+
|
|
165
|
+
### Changed
|
|
166
|
+
|
|
167
|
+
- **Breaking:** `Page#xhr_post` is now `Page#post` and uses `fetch`, which
|
|
168
|
+
Obscura routes as of 0.1.11. `xhr_post` remains as a deprecated alias.
|
|
169
|
+
`timeout:` is now enforced *in the page* by racing the request against a
|
|
170
|
+
timer, so a tarpitting server fails at roughly `timeout:` instead of waiting
|
|
171
|
+
out the CDP reply timeout. It races rather than using `AbortSignal.timeout`
|
|
172
|
+
because an abort's rejection is swallowed and surfaces as `undefined`.
|
|
173
|
+
- **Breaking:** `Node#type` dispatches real key events via
|
|
174
|
+
`Input.dispatchKeyEvent` instead of assigning `value` directly, so the browser
|
|
175
|
+
performs the insertion and `keydown`/`input`/`keyup` fire as they would from a
|
|
176
|
+
keyboard. `change` no longer fires per keystroke — real browsers only fire it
|
|
177
|
+
on blur, so listeners depending on the old behaviour should use `input`.
|
|
178
|
+
`Input.insertText` is still unimplemented, so insertion is per character.
|
|
179
|
+
- **Breaking:** `Frame#read_string` and `Runtime::EVALUATE_CHUNK` are removed,
|
|
180
|
+
along with the `window.__obxcura_read` global. Obscura's single-message ceiling
|
|
181
|
+
went from ~500–700 KB to 64 MiB, so `Page#html` reads `outerHTML` in one round
|
|
182
|
+
trip. Verified with 8 MiB and 16 MiB strings.
|
|
183
|
+
- `Node#submit` always uses `requestSubmit` and no longer falls back to
|
|
184
|
+
`submit()`. In 0.1.11 the two genuinely differ — `submit()` bypasses the
|
|
185
|
+
cancelable submit event — so the fallback would have quietly skipped both
|
|
186
|
+
constraint validation and any registered listener.
|
|
187
|
+
|
|
188
|
+
### Added
|
|
189
|
+
|
|
190
|
+
- `Page#network_log` — the request log has existed as internal state since
|
|
191
|
+
0.1.0, but `Network.enable` was never sent, so no events ever arrived and
|
|
192
|
+
nothing could read it. The domain is now enabled per page and the log is
|
|
193
|
+
exposed. Scope is navigation-driven requests only: script-initiated requests
|
|
194
|
+
emit no CDP network events, so `#post` traffic does not appear. This
|
|
195
|
+
contradicts the 0.1.11 note for #415, and was confirmed by measurement.
|
|
196
|
+
- `Browser#clear_cookies` — listed in the 0.1.0 changelog and referenced in the
|
|
197
|
+
docs, but never actually implemented; the call site was commented out in
|
|
198
|
+
`Page#close`. Now a real method. Note cookies no longer leak between
|
|
199
|
+
connections in 0.1.11, so this only resets state within one connection.
|
|
200
|
+
|
|
201
|
+
### Performance
|
|
202
|
+
|
|
203
|
+
- `Client::READ_CHUNK` raised from 512 B to 64 KiB. With frames now reaching
|
|
204
|
+
64 MiB, a 512-byte read turned a single reply into tens of thousands of
|
|
205
|
+
syscalls. Measured on a 16 MiB reply: 0.188s → 0.105s. Past 64 KiB the curve
|
|
206
|
+
flattens.
|
|
207
|
+
|
|
3
208
|
## [0.2.0] - 2026-07-20
|
|
4
209
|
|
|
5
210
|
### Added
|
data/README.md
CHANGED
|
@@ -15,132 +15,122 @@ own attached session. One connection, many pages.
|
|
|
15
15
|
bundle add obxcura
|
|
16
16
|
```
|
|
17
17
|
|
|
18
|
-
You also need the
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
## Usage
|
|
22
|
-
|
|
23
|
-
Start the browser first (defaults to port 9222):
|
|
18
|
+
You also need the **`obscura` browser binary**, which is a separate project. The
|
|
19
|
+
repo ships an installer for it:
|
|
24
20
|
|
|
25
21
|
```bash
|
|
26
|
-
obscura
|
|
22
|
+
scripts/obscura/install.sh
|
|
27
23
|
```
|
|
28
24
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
require "obxcura"
|
|
25
|
+
That pulls the latest release for your platform into `/usr/local/bin` (asking for
|
|
26
|
+
`sudo` only if the prefix is not writable), then verifies the install and prints
|
|
27
|
+
the checksums.
|
|
33
28
|
|
|
34
|
-
|
|
35
|
-
|
|
29
|
+
```bash
|
|
30
|
+
scripts/obscura/install.sh --version v0.2.0 # pin a release
|
|
31
|
+
scripts/obscura/install.sh --prefix ~/.local/bin # no sudo
|
|
32
|
+
scripts/obscura/install.sh --no-stealth # plain build
|
|
33
|
+
scripts/obscura/install.sh --no-render # smaller, no screenshot/pdf
|
|
34
|
+
scripts/obscura/install.sh --help
|
|
35
|
+
```
|
|
36
36
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
37
|
+
Prefer it to a manual download, because the packaging has sharp edges:
|
|
38
|
+
|
|
39
|
+
- **Since 0.2.0 there are four builds per platform**, and the suffix decides what
|
|
40
|
+
works:
|
|
41
|
+
|
|
42
|
+
| Asset suffix | Render engine | Stealth |
|
|
43
|
+
|---|---|---|
|
|
44
|
+
| *(none)* | yes | no |
|
|
45
|
+
| `-stealth` | yes | yes |
|
|
46
|
+
| `-no-render` | no | no |
|
|
47
|
+
| `-no-render-stealth` | no | yes |
|
|
48
|
+
|
|
49
|
+
The script defaults to **`-stealth`**: it carries the render engine that
|
|
50
|
+
`#screenshot` and `#pdf` need, *and* it is the build that gets past commercial
|
|
51
|
+
bot detection. `--no-render` trades both away for a smaller download.
|
|
52
|
+
- **`--version` cannot tell the builds apart** — all four of a release report the
|
|
53
|
+
same string. That is why the script prints checksums: they are the only way to
|
|
54
|
+
identify later which build is actually installed.
|
|
55
|
+
- **The archive holds two binaries**, `obscura` and `obscura-worker`, and `serve`
|
|
56
|
+
looks for the worker as a *sibling*. The script always moves the pair, since
|
|
57
|
+
installing one alone silently mixes builds.
|
|
58
|
+
- **On macOS it removes the target before copying.** Overwriting a binary in
|
|
59
|
+
place leaves a stale cached code signature on that inode, and the kernel then
|
|
60
|
+
`SIGKILL`s it on exec — even though the bytes are fine.
|
|
61
|
+
|
|
62
|
+
Driving anything on localhost also needs `--allow-private-network`, which is off
|
|
63
|
+
by default:
|
|
41
64
|
|
|
42
|
-
|
|
65
|
+
```bash
|
|
66
|
+
obscura serve --allow-private-network
|
|
43
67
|
```
|
|
44
68
|
|
|
45
|
-
|
|
69
|
+
## Usage
|
|
46
70
|
|
|
47
|
-
|
|
71
|
+
Start the browser first (defaults to port 9222):
|
|
48
72
|
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
page.at_css("a")["href"] # attribute value, or nil
|
|
52
|
-
page.at_css("h1").outer_html # "<h1>Example Domain</h1>"
|
|
53
|
-
page.css("p").map(&:text) # every <p>'s text
|
|
54
|
-
page.at_css("#missing") # => nil
|
|
73
|
+
```bash
|
|
74
|
+
obscura serve
|
|
55
75
|
```
|
|
56
76
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
Pass Ruby values as arguments — they cross into the page as real values
|
|
60
|
-
(`arguments[0]`, `arguments[1]`, …), never string-interpolated into source:
|
|
77
|
+
Then:
|
|
61
78
|
|
|
62
79
|
```ruby
|
|
63
|
-
|
|
80
|
+
require "obxcura"
|
|
64
81
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
JS
|
|
69
|
-
```
|
|
82
|
+
browser = Obxcura::Browser.new # or Obxcura.start
|
|
83
|
+
begin
|
|
84
|
+
page = browser.go_to("https://example.com")
|
|
70
85
|
|
|
71
|
-
|
|
86
|
+
page.title # => "Example Domain"
|
|
87
|
+
page.current_url # => "https://example.com/"
|
|
88
|
+
page.html # rendered DOM, post-JS
|
|
89
|
+
page.evaluate("1 + 2") # => 3
|
|
72
90
|
|
|
73
|
-
|
|
74
|
-
|
|
91
|
+
page.at_css("h1").text # => "Example Domain"
|
|
92
|
+
page.css("a").map { |a| a["href"] }
|
|
75
93
|
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
"
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
result["status"] # => 200
|
|
85
|
-
result["ok"] # => true (2xx)
|
|
86
|
-
result["body"] # response text
|
|
94
|
+
page.at_css("#username").type("guillermo")
|
|
95
|
+
page.at_css("form").submit
|
|
96
|
+
|
|
97
|
+
page.screenshot(path: "shot.png")
|
|
98
|
+
page.pdf(path: "page.pdf", paper: :a4)
|
|
99
|
+
ensure
|
|
100
|
+
browser.quit # always, so targets don't leak
|
|
101
|
+
end
|
|
87
102
|
```
|
|
88
103
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
`
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
`Obxcura.start
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
These are properties of the **Obscura browser**, not of this gem. They shape the
|
|
121
|
-
API, so they're worth knowing:
|
|
122
|
-
|
|
123
|
-
- **No paint engine.** There is no screenshot API, and there never will be one
|
|
124
|
-
here. `Page.captureScreenshot` is unusable.
|
|
125
|
-
- **~500–700 KB message ceiling.** Obscura won't send a single CDP message
|
|
126
|
-
larger than that. `Page#html` works around it by snapshotting `outerHTML` into
|
|
127
|
-
a page global and pulling it back in 400 KB slices. Don't return giant values
|
|
128
|
-
from `#evaluate` directly.
|
|
129
|
-
- **DOM nodes don't serialize.** A node returned by value comes back as an
|
|
130
|
-
internal stub, so `#at_css` / `#css` resolve it (via `DOM.resolveNode`) to a
|
|
131
|
-
live handle and read it with `Runtime.callFunctionOn`.
|
|
132
|
-
- **XHR, not fetch.** Obscura routes `XMLHttpRequest` but not `fetch`, so
|
|
133
|
-
`#xhr_post` uses XHR from the page context. It also ignores
|
|
134
|
-
`XMLHttpRequest#timeout`, so the effective bound is the CDP-level `timeout:`.
|
|
135
|
-
- **Persistent cookies.** `obscura serve` is long-lived and its cookie jar
|
|
136
|
-
survives `#quit` (that only drops the WebSocket). Read them with
|
|
137
|
-
`page.cookies`; closing a page clears the browser's cookie jar.
|
|
138
|
-
- **Private networks are blocked by default.** To drive a local site, start the
|
|
139
|
-
browser with `obscura serve --allow-private-network`.
|
|
140
|
-
|
|
141
|
-
The transport is built directly on `websocket-driver` rather than
|
|
142
|
-
`websocket-client-simple`, which reads a byte at a time and wedges on large
|
|
143
|
-
frames.
|
|
104
|
+
That is the whole shape of it: a `Browser` owns the connection, a `Page` is a
|
|
105
|
+
target on it, and `#at_css` / `#css` hand back live `Node` handles you can read
|
|
106
|
+
from and act on.
|
|
107
|
+
|
|
108
|
+
## Documentation
|
|
109
|
+
|
|
110
|
+
Full reference in [`docs/`](docs/README.md), one page per area:
|
|
111
|
+
|
|
112
|
+
| | |
|
|
113
|
+
|---|---|
|
|
114
|
+
| [Connecting](docs/connecting.md) | `Obxcura.start`, `Browser.new`, ports, timeouts, shutdown |
|
|
115
|
+
| [Pages](docs/pages.md) | `#goto`, `#refresh`, `#close`, raw `#command` |
|
|
116
|
+
| [Reading content](docs/content.md) | `#html`, `#title`, `#current_url` |
|
|
117
|
+
| [Querying the DOM](docs/dom.md) | `#at_css`, `#css`, and every `Node` method |
|
|
118
|
+
| [Running JavaScript](docs/javascript.md) | `#evaluate`, `#evaluate_func`, `#call_on` |
|
|
119
|
+
| [Forms and input](docs/forms.md) | `#focus`, `#type`, `#submit`, `#value` |
|
|
120
|
+
| [HTTP from the page](docs/http.md) | `#post`, `#network_log` |
|
|
121
|
+
| [Cookies](docs/cookies.md) | `#cookies` — read, set, remove, replay a session |
|
|
122
|
+
| [Headers](docs/headers.md) | `#headers` — read and set extra HTTP headers |
|
|
123
|
+
| [Screenshots](docs/screenshots.md) | `#screenshot` and every option |
|
|
124
|
+
| [PDF](docs/pdf.md) | `#pdf` and every option |
|
|
125
|
+
| [Errors](docs/errors.md) | The exception hierarchy, and what does *not* raise |
|
|
126
|
+
| [Browser constraints](docs/constraints.md) | Obscura limits that shape this API |
|
|
127
|
+
|
|
128
|
+
Runnable examples live in [`examples/`](examples/).
|
|
129
|
+
|
|
130
|
+
Two constraints are worth knowing before you write anything, both covered in
|
|
131
|
+
[Browser constraints](docs/constraints.md): Obscura stops running background
|
|
132
|
+
JavaScript shortly after `load`, so `goto` + `sleep` does not do what you expect;
|
|
133
|
+
and an in-page `throw` comes back as `nil` rather than raising.
|
|
144
134
|
|
|
145
135
|
## Development
|
|
146
136
|
|
|
@@ -159,6 +149,9 @@ OBSCURA_BIN=/path/to/obscura bundle exec rspec
|
|
|
159
149
|
```
|
|
160
150
|
|
|
161
151
|
Without the binary those specs skip cleanly, so `bundle exec rspec` still passes.
|
|
152
|
+
The `#screenshot` and `#pdf` specs skip the same way on a `-no-render` build, so
|
|
153
|
+
run them against a build that has the render engine — either the default from
|
|
154
|
+
`scripts/obscura/install.sh` or the unsuffixed asset — if you touch either.
|
|
162
155
|
|
|
163
156
|
## Contributing
|
|
164
157
|
|
data/lib/obxcura/browser.rb
CHANGED
|
@@ -18,6 +18,10 @@ module Obxcura
|
|
|
18
18
|
DEFAULT_HOST = "127.0.0.1"
|
|
19
19
|
# @return [Integer] default CDP port for `obscura serve`.
|
|
20
20
|
DEFAULT_PORT = 9222
|
|
21
|
+
# The only URL a target may safely be created at; see {#create_page}.
|
|
22
|
+
#
|
|
23
|
+
# @return [String]
|
|
24
|
+
BLANK_PAGE = "about:blank"
|
|
21
25
|
|
|
22
26
|
# @return [Obxcura::Client] the underlying CDP transport.
|
|
23
27
|
# @return [Array<Obxcura::Page>] the pages currently open.
|
|
@@ -41,29 +45,38 @@ module Obxcura
|
|
|
41
45
|
@client = Client.new(browser_ws_url, timeout: @timeout)
|
|
42
46
|
end
|
|
43
47
|
|
|
44
|
-
# Open a fresh page (a CDP target) and attach to it.
|
|
48
|
+
# Open a fresh page (a CDP target) and attach to it. With a `url`, the page
|
|
49
|
+
# is navigated there and the call blocks until its load event fires.
|
|
50
|
+
#
|
|
51
|
+
# The target is always created blank and then navigated, never opened at
|
|
52
|
+
# `url` directly. Handing a URL to `Target.createTarget` works exactly once:
|
|
53
|
+
# the *second* such call on a connection kills `obscura serve` outright —
|
|
54
|
+
# every command after it fails with `connection closed: end of file
|
|
55
|
+
# reached`, and the process is gone, not just the socket. Measured on 0.2.0
|
|
56
|
+
# and deterministic, whatever the URLs are; two blank targets are fine, and
|
|
57
|
+
# so is any amount of navigation. Since the parameter cannot be honoured
|
|
58
|
+
# literally without handing callers a way to kill the browser, it is honoured
|
|
59
|
+
# by navigating.
|
|
45
60
|
#
|
|
46
|
-
# @param url [String] URL to open the
|
|
61
|
+
# @param url [String] URL to open the page at (defaults to a blank page).
|
|
47
62
|
# @return [Obxcura::Page] the new, tracked page.
|
|
48
|
-
def create_page(url =
|
|
49
|
-
target_id = command("Target.createTarget", { url:
|
|
63
|
+
def create_page(url = BLANK_PAGE)
|
|
64
|
+
target_id = command("Target.createTarget", { url: BLANK_PAGE })["targetId"]
|
|
50
65
|
session_id = command("Target.attachToTarget", { targetId: target_id, flatten: true })["sessionId"]
|
|
51
66
|
|
|
52
67
|
page = Page.new(self, target_id: target_id, session_id: session_id)
|
|
53
68
|
@pages << page
|
|
69
|
+
page.goto(url) unless url == BLANK_PAGE
|
|
54
70
|
page
|
|
55
71
|
end
|
|
56
72
|
|
|
57
|
-
# Open a page and navigate to `url` in one call
|
|
58
|
-
#
|
|
59
|
-
# Navigates from a blank target rather than passing the URL straight to
|
|
60
|
-
# Target.createTarget: creating two URL-loaded targets and then evaluating
|
|
61
|
-
# crashes `obscura serve` (connection closed: end of file reached).
|
|
73
|
+
# Open a page and navigate to `url` in one call — the expressive name for
|
|
74
|
+
# {#create_page} with a URL.
|
|
62
75
|
#
|
|
63
76
|
# @param url [String] URL to navigate to.
|
|
64
77
|
# @return [Obxcura::Page] the navigated page (load event fired).
|
|
65
78
|
def go_to(url)
|
|
66
|
-
create_page
|
|
79
|
+
create_page(url)
|
|
67
80
|
end
|
|
68
81
|
alias goto go_to
|
|
69
82
|
|
|
@@ -82,6 +95,20 @@ module Obxcura
|
|
|
82
95
|
JSON.parse(Net::HTTP.get(uri))
|
|
83
96
|
end
|
|
84
97
|
|
|
98
|
+
# Drop every cookie held on this connection.
|
|
99
|
+
#
|
|
100
|
+
# Since Obscura 0.1.11 each connection owns its own browser context, so cookies
|
|
101
|
+
# no longer leak between `Browser` instances and this is only about resetting
|
|
102
|
+
# state *within* one connection — between logical sessions on the same socket,
|
|
103
|
+
# say. `obscura serve` is still long-lived and {#quit} only drops the socket, so
|
|
104
|
+
# a fresh `Browser` is the other way to get a clean jar.
|
|
105
|
+
#
|
|
106
|
+
# @return [void]
|
|
107
|
+
def clear_cookies
|
|
108
|
+
command("Network.clearBrowserCookies")
|
|
109
|
+
nil
|
|
110
|
+
end
|
|
111
|
+
|
|
85
112
|
# Stop tracking a page. Called by {Obxcura::Page#close}.
|
|
86
113
|
#
|
|
87
114
|
# @param page [Obxcura::Page] the page to forget.
|
data/lib/obxcura/client.rb
CHANGED
|
@@ -25,17 +25,20 @@ module Obxcura
|
|
|
25
25
|
# @return [Integer] default seconds to wait for a command reply.
|
|
26
26
|
DEFAULT_TIMEOUT = 30
|
|
27
27
|
|
|
28
|
-
# Obscura
|
|
29
|
-
#
|
|
28
|
+
# Matches Obscura's own 64 MiB frame ceiling, so anything the browser is
|
|
29
|
+
# willing to send, the driver is willing to assemble.
|
|
30
30
|
#
|
|
31
31
|
# @return [Integer] largest CDP frame (bytes) the driver will assemble.
|
|
32
32
|
MAX_MESSAGE_SIZE = 64 * 1024 * 1024
|
|
33
33
|
|
|
34
|
-
# Bytes pulled per read syscall.
|
|
35
|
-
#
|
|
34
|
+
# Bytes pulled per read syscall. This has to be large: since Obscura raised
|
|
35
|
+
# its frame ceiling to 64 MiB, multi-megabyte replies are routine, and a
|
|
36
|
+
# small value turns one reply into tens of thousands of syscalls. Measured
|
|
37
|
+
# on a 16 MiB reply: 512B → 0.188s, 64KiB → 0.105s. Past 64KiB the curve
|
|
38
|
+
# flattens, so the extra buffer buys nothing.
|
|
36
39
|
#
|
|
37
40
|
# @return [Integer]
|
|
38
|
-
READ_CHUNK =
|
|
41
|
+
READ_CHUNK = 64 * 1024
|
|
39
42
|
|
|
40
43
|
# @return [String] the WebSocket URL of the browser endpoint.
|
|
41
44
|
attr_reader :url
|