wrangle 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +87 -0
- data/LICENSE.txt +48 -0
- data/README.md +473 -0
- data/exe/wrangle +374 -0
- data/lib/wrangle/action_space.rb +67 -0
- data/lib/wrangle/decider.rb +167 -0
- data/lib/wrangle/errors.rb +53 -0
- data/lib/wrangle/jev.rb +87 -0
- data/lib/wrangle/js/bridge.js +390 -0
- data/lib/wrangle/js/page.js +127 -0
- data/lib/wrangle/js/snapshot.js +118 -0
- data/lib/wrangle/jxa_bridge.rb +194 -0
- data/lib/wrangle/mcp_bridge.rb +300 -0
- data/lib/wrangle/observation.rb +50 -0
- data/lib/wrangle/run_loop.rb +298 -0
- data/lib/wrangle/safari.rb +404 -0
- data/lib/wrangle/session_server.rb +447 -0
- data/lib/wrangle/version.rb +5 -0
- data/lib/wrangle.rb +19 -0
- data/skills/wrangle/SKILL.md +206 -0
- metadata +69 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: c71a3ad462a532ec7f6ddea4621ff5620bfde1085cbdbcb5ec0f04995b2707e3
|
|
4
|
+
data.tar.gz: de242cf8fa5fb7b3c21c03af57d8f8287db282c487fabc20b61f52459756d113
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: f295f56bb4e7553dcac794b08ef3d63295f57a2f18922a6fc1d7142dfde7ece183f0971340db080d7a87ce55fb671ef8c431a8e57cd0fd930d0c710a0c051c5f
|
|
7
|
+
data.tar.gz: e4d1eaffa60d2360f513aaf20f2a4a97255bc387e24ff3e1e563a1d2c189912c37eba1fb022e25dd4931540a37b8f6f9a74c5fdd6d96361f1d8ce9700b5b145c
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## [0.1.0] - 2026-09-18
|
|
4
|
+
|
|
5
|
+
- `wrangle run --goal "..." --execute`: a decision loop driven by Jev, a typed-choice model that
|
|
6
|
+
picks one of the actions Wrangle observed. It never writes text — `--literal LABEL=VALUE` supplies
|
|
7
|
+
it, and a fill with no literal stops and asks.
|
|
8
|
+
- `--plan GOAL` (repeatable): ordered sub-goals advanced internally on DONE, so a whole form is one
|
|
9
|
+
command instead of one agent turn per click. Measured on Google Flights: 29 steps in ~14s, against
|
|
10
|
+
ten turns and 129s hand-stepping the same task.
|
|
11
|
+
- A confidence floor (`--min-confidence`, default `0.5`) that looks again once before handing back,
|
|
12
|
+
because a page half-rendered when the model looked reads as ambiguity. `DONE` and `BLOCKED` are
|
|
13
|
+
held to it too — an uncertain `DONE` still advances a plan, an uncertain `BLOCKED` must earn its
|
|
14
|
+
handoff, since being wrong about them costs very different amounts.
|
|
15
|
+
- `--side left|right|top|bottom` parks the window on half a display.
|
|
16
|
+
- `--backend mcp` drives `safaridriver --mcp` instead of Apple Events: ~3ms per action against
|
|
17
|
+
~120ms, behind ~4s of startup, so it only pays off past roughly 28 actions. Experimental — its
|
|
18
|
+
automation tab is backgrounded, and menus animated on `requestAnimationFrame` do not respond.
|
|
19
|
+
- `BLOCKED` is held to its own floor (0.6) that `--min-confidence` can raise but not lower, and a
|
|
20
|
+
weak one gets two further looks with a widening pause before it is believed. A control that has
|
|
21
|
+
not rendered yet is not a dead end, and lowering the floor to help an underconfident click should
|
|
22
|
+
not make it easier to abandon the run.
|
|
23
|
+
- Every request carries a verification question — is this goal's outcome visible on the page? —
|
|
24
|
+
answered independently of the one choosing actions, and read only to dispute a `DONE`. A confident
|
|
25
|
+
disagreement keeps the leg working; an unsure one is ignored; one that survives three widening
|
|
26
|
+
pauses hands back. `DONE` is the model reporting on its own work and it is optimistic: a plan
|
|
27
|
+
reported success with a requested filter never applied.
|
|
28
|
+
- `UNPROVEN` in the transcript marks a claim that did not check out, and does not spend the budget.
|
|
29
|
+
- A dispute only ends a run when the leg it disputes has done nothing. A leg that acted is taken at
|
|
30
|
+
its word once it has looked again: the verifier reads one snapshot, the actor knows what it did,
|
|
31
|
+
and abandoning the remaining legs is the more expensive way to be wrong.
|
|
32
|
+
- A leg that has not acted yet cannot report `BLOCKED` without confirming it, however sure it is.
|
|
33
|
+
A leg starts the instant the last one ends, so it often looks at a document that is still loading,
|
|
34
|
+
and a half-loaded page does not read as ambiguous — it reads as definite, and reads surer the
|
|
35
|
+
second time. Up to three further looks, widening to 2.5s.
|
|
36
|
+
- A second look appears in the transcript as `RELOOK` rather than being silently discarded.
|
|
37
|
+
- A `select` is no longer hit-tested before it runs. Styled dropdowns hide the native control under
|
|
38
|
+
an overlay, which made Amazon's sort permanently unactionable; a select is driven by assigning
|
|
39
|
+
value and dispatching input/change, so nothing depends on it being the topmost element.
|
|
40
|
+
- A blocked action says which of the four reasons stopped it — gone, read-only, scrolled out of
|
|
41
|
+
view, or behind something else — instead of one message covering all of them.
|
|
42
|
+
- The step budget counts work done, not attempts made; stale retries and second looks no longer
|
|
43
|
+
spend a leg's allowance, with a separate spin cap for loops making no progress.
|
|
44
|
+
- Waiting is an operation the model can choose rather than a fixed pause, and the settle poll starts
|
|
45
|
+
impatient and backs off only when the page proves it is churning.
|
|
46
|
+
- The page is watched while Jev is thinking rather than before the request, so proving it has gone
|
|
47
|
+
still is free: it fits inside a wait the step was making anyway. A rejected decision then has a
|
|
48
|
+
fresh read already in hand and costs one more request instead of a request and a read.
|
|
49
|
+
- Fixed: the settle budget had never applied. The CLI always sent the key, so `fetch("steady", 0.2)`
|
|
50
|
+
returned the `nil` that was there, `nil.to_f` is zero, and the backoff after a stale retry was
|
|
51
|
+
multiplying that zero by two. No decision had waited for a page in months.
|
|
52
|
+
- A rejected decision says which part of the guard moved — the document, the route, the viewport, a
|
|
53
|
+
field elsewhere, the target itself, or the content around it — instead of reporting that something,
|
|
54
|
+
somewhere, changed.
|
|
55
|
+
- The route comparison drops the query string. Sites rewrite it constantly to hold state (a flights
|
|
56
|
+
form puts the itinerary in `?tfs=` and rewrites it on every keystroke), and comparing the full URL
|
|
57
|
+
rejected decisions about controls that had not moved. Removed that rejection reason entirely.
|
|
58
|
+
- A fill is checked against its target like a click, instead of falling back to the whole-page
|
|
59
|
+
marker, which compares the title, every word of text and every action on the page.
|
|
60
|
+
- On the flights form: **0.67 wall seconds per decision, against 1.04**; a rejected decision costs
|
|
61
|
+
428ms against 695ms; reading the page costs 38ms per decision against 120ms. The rejection rate
|
|
62
|
+
barely moved (27% to 25%) — what remains is the page genuinely changing, and re-deciding is the
|
|
63
|
+
right answer to that. Removing the retry backoff was tried and reverted: rejections went from 4.3
|
|
64
|
+
to 7.3 a run.
|
|
65
|
+
- Unit tests for the three pieces that had only been covered through the loop: the action space, the
|
|
66
|
+
decider, and the MCP bridge. The bridge's run against a fake `safaridriver --mcp` speaking real
|
|
67
|
+
JSON-RPC over a real pipe, so a silent server, a dead one, and a document that took the installed
|
|
68
|
+
runtime with it are all exercised rather than described. Nine deliberate mutations — folding
|
|
69
|
+
actions by node, per-option select targets, the argmax and sum checks on a distribution, reading
|
|
70
|
+
an unnamed head, retrying a mutation whose runtime vanished — were each confirmed to fail a test.
|
|
71
|
+
|
|
72
|
+
- Initial Safari backend: scoped windows, snapshot observations, guarded actions.
|
|
73
|
+
- Interactive sessions: a background server holds one window behind a socket in `~/.wrangle`, so
|
|
74
|
+
`observe` and `act` are separate shell commands against the same page.
|
|
75
|
+
- Every action reports what moved — appeared/gone controls, text and scroll deltas, and an explicit
|
|
76
|
+
"nothing changed" — so a caller can tell a click that worked from one that did not.
|
|
77
|
+
- `--settle` polls until the page stops changing rather than sleeping a guessed interval, and
|
|
78
|
+
waits out a quiet floor first so a click that navigates is not mistaken for one that did nothing.
|
|
79
|
+
- Distinct exit codes separate "observe and retry" from "this session is over".
|
|
80
|
+
- `skills/wrangle`: an Agent Skill so an AI agent drives Safari by shell command, not by script.
|
|
81
|
+
- Fixed the test fixture emitting `scroll` as an integer where Safari emits `{y, height}`.
|
|
82
|
+
- Tests cover 98% of lines and 95% of branches, measured and enforced with Ruby's own `Coverage`
|
|
83
|
+
module — no coverage gem, since adding a dependency to check that Wrangle has none would undercut
|
|
84
|
+
the claim. Getting there found three real defects: a client that raised a raw `Errno::EPIPE` when a
|
|
85
|
+
session died mid-write instead of saying the session had gone, a `--version` flag that did not
|
|
86
|
+
exist despite the `version` subcommand doing, and a test suite that could reach the real Jev
|
|
87
|
+
endpoint and spend money whenever a test forgot to pass a fake.
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Eric Boehs
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
lib/wrangle/js/snapshot.js is vendored from browser-use/jev-ultrafast and is
|
|
26
|
+
used under the MIT License:
|
|
27
|
+
|
|
28
|
+
MIT License
|
|
29
|
+
|
|
30
|
+
Copyright (c) 2026 Browser Use
|
|
31
|
+
|
|
32
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
33
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
34
|
+
in the Software without restriction, including without limitation the rights
|
|
35
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
36
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
37
|
+
furnished to do so, subject to the following conditions:
|
|
38
|
+
|
|
39
|
+
The above copyright notice and this permission notice shall be included in all
|
|
40
|
+
copies or substantial portions of the Software.
|
|
41
|
+
|
|
42
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
43
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
44
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
45
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
46
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
47
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
48
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,473 @@
|
|
|
1
|
+
# wrangle
|
|
2
|
+
|
|
3
|
+
[](https://github.com/ericboehs/wrangle/actions/workflows/ci.yml)
|
|
4
|
+
[](https://rubygems.org/gems/wrangle)
|
|
5
|
+
|
|
6
|
+
**Hand one Safari window to a program, and no more than that.**
|
|
7
|
+
|
|
8
|
+
Wrangle drives an ordinary Safari window through Apple Events. There is no automation session, no
|
|
9
|
+
extension, and no native helper — so the window stays a real one you can see, keep, and take back at
|
|
10
|
+
any moment. It is pure Ruby with **no runtime dependencies**: everything it needs ships with Ruby and
|
|
11
|
+
macOS.
|
|
12
|
+
|
|
13
|
+
The usual way to automate Safari is `safaridriver --enable`, which gives you a quarantined browser
|
|
14
|
+
with a banner across the top, none of your cookies, and none of your sessions. That is the right tool
|
|
15
|
+
for testing a site. It is the wrong tool for doing something *in* a browser you are already logged
|
|
16
|
+
into. Wrangle is for the second case.
|
|
17
|
+
|
|
18
|
+
```ruby
|
|
19
|
+
require "wrangle"
|
|
20
|
+
|
|
21
|
+
Wrangle::Safari.open("https://example.com", display: 1) do |session|
|
|
22
|
+
page = session.observe
|
|
23
|
+
link = page["actions"].find { |a| a["label"] == "Learn more" }
|
|
24
|
+
session.act(link, page)
|
|
25
|
+
puts session.observe["title"] # => "Example Domains"
|
|
26
|
+
end
|
|
27
|
+
# the window Wrangle opened is closed here; a window you handed it never would be
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Install
|
|
31
|
+
|
|
32
|
+
```ruby
|
|
33
|
+
gem "wrangle"
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Requires macOS, Safari, and Ruby 3.2+. You also need two things switched on once, by hand:
|
|
37
|
+
|
|
38
|
+
- **Safari → Settings → Advanced → Allow JavaScript from Apple Events.**
|
|
39
|
+
- The first run raises an Apple Events permission prompt. Approve it, or find it later under
|
|
40
|
+
**System Settings → Privacy & Security → Automation**.
|
|
41
|
+
|
|
42
|
+
Wrangle will not change either setting for you.
|
|
43
|
+
|
|
44
|
+
## Two ways in
|
|
45
|
+
|
|
46
|
+
**A window Wrangle opens.** It owns this one, so it may close it.
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
session = Wrangle::Safari.open("https://example.com", display: 1)
|
|
50
|
+
session = Wrangle::Safari.open("https://example.com", bounds: [0, 0, 1200, 900])
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
**A window you hand over.** Already open, already signed in, already where you left it. Wrangle will
|
|
54
|
+
never close it and never navigate it.
|
|
55
|
+
|
|
56
|
+
```ruby
|
|
57
|
+
Wrangle::Safari.windows(titles: true)
|
|
58
|
+
# => [{"window_id"=>26081, "tabs"=>4, "display"=>0, "title"=>"Home / X", ...}]
|
|
59
|
+
|
|
60
|
+
session = Wrangle::Safari.attach(window_id: 26081)
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Titles and URLs identify a tab to whoever owns it, so `windows` omits them unless you ask.
|
|
64
|
+
|
|
65
|
+
## Observing
|
|
66
|
+
|
|
67
|
+
`observe` returns one snapshot of the scoped tab:
|
|
68
|
+
|
|
69
|
+
```ruby
|
|
70
|
+
page = session.observe
|
|
71
|
+
|
|
72
|
+
page["url"] # "https://example.com/"
|
|
73
|
+
page["title"] # "Example Domain"
|
|
74
|
+
page["text"] # visible text, capped at 6000 characters
|
|
75
|
+
page["scroll"] # {"y" => 0, "height" => 997}
|
|
76
|
+
page["fingerprint"] # sha256 over url + text + actions + scroll
|
|
77
|
+
page["actions"]
|
|
78
|
+
# [{"id"=>"e1", "kind"=>"click", "node"=>1, "role"=>"link", "label"=>"Learn more",
|
|
79
|
+
# "rect"=>{"x"=>384, "y"=>227.2, "w"=>82, "h"=>18.8}, "value"=>""},
|
|
80
|
+
# {"id"=>"wait", "kind"=>"wait", "label"=>"Wait for the page to update"}]
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
The five action kinds are `click`, `fill`, `select`, `scroll`, and `wait`. A `select` is expanded into
|
|
84
|
+
one candidate per option, so choosing an option means choosing an action rather than supplying a
|
|
85
|
+
string the page never offered.
|
|
86
|
+
|
|
87
|
+
## Acting
|
|
88
|
+
|
|
89
|
+
```ruby
|
|
90
|
+
session.act(action, page) # click, select, scroll, wait
|
|
91
|
+
session.act(action, page, text: "Lisbon") # fill
|
|
92
|
+
session.fresh?(page, action) # check without mutating
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
`act` takes the action **and** the observation it came from. That pairing is the whole point: an
|
|
96
|
+
action must be an exact candidate from a page that is still the page you saw.
|
|
97
|
+
|
|
98
|
+
## What it refuses to do
|
|
99
|
+
|
|
100
|
+
This is the interesting part. Wrangle fails closed, and every refusal below is a distinct exception
|
|
101
|
+
rather than a return value you can forget to check.
|
|
102
|
+
|
|
103
|
+
**It will not act on an action you made up.** An action must be an exact, unambiguous candidate from
|
|
104
|
+
the observation you pass alongside it. Change one field — a node id, a value — and it raises
|
|
105
|
+
`ArgumentError` without dispatching anything. If two candidates share an id, it refuses rather than
|
|
106
|
+
picks. This is the boundary that keeps model output from becoming instructions: a decision may only
|
|
107
|
+
name something the page was just seen to offer. Model output never becomes JavaScript, a selector, a
|
|
108
|
+
coordinate, a key command, or an option value that was not observed.
|
|
109
|
+
|
|
110
|
+
**It will not act on a page that moved.** Before every mutation, Wrangle re-checks the target: a
|
|
111
|
+
`guard` for clicks and selects, a `marker` for fills and scrolls. If the page shifted between your
|
|
112
|
+
observation and your decision, you get `StalePage` and nothing was dispatched. Observe again.
|
|
113
|
+
|
|
114
|
+
**It will not lose track of which window it was given.** Scope is one window id, one tab position, one
|
|
115
|
+
expected URL, and one document epoch. If the window you handed over gains a tab, or a different tab
|
|
116
|
+
becomes current, or a handed-over tab replaces its document, you get `ScopeLost` — and the session is
|
|
117
|
+
finished. It does not search for a replacement window, because the window it was given is the only
|
|
118
|
+
window it was given. A window Wrangle opened *is* allowed to navigate; a window you lent it is not.
|
|
119
|
+
|
|
120
|
+
**It will not retry a mutation to find out whether it landed.** Every action carries a nonce and the
|
|
121
|
+
page records a phase against it. If the reply goes missing, Wrangle reads the nonce back and resolves
|
|
122
|
+
what actually happened. It never repeats the action. If the outcome still cannot be established you
|
|
123
|
+
get `DeliveryUnknown`, the session is poisoned, and — since it can no longer describe what it would be
|
|
124
|
+
closing — it leaves the window open.
|
|
125
|
+
|
|
126
|
+
**It will not close a window it did not open.** `attach` sessions never close anything. Even an owned
|
|
127
|
+
window is left alone if it gained tabs in the meantime.
|
|
128
|
+
|
|
129
|
+
**It will not start when window ids are ambiguous.** Window ids are only unique within one Safari
|
|
130
|
+
process, and `safaridriver` is fond of leaving extra ones behind. More than one running Safari and
|
|
131
|
+
Wrangle refuses to start, rather than address the wrong browser. Override with
|
|
132
|
+
`allow_multiple_safari: true` if you know what you are doing.
|
|
133
|
+
|
|
134
|
+
```ruby
|
|
135
|
+
Wrangle::Error # base
|
|
136
|
+
├─ Wrangle::BridgeError # the osascript bridge failed
|
|
137
|
+
│ ├─ Wrangle::BridgeTimeout
|
|
138
|
+
│ └─ Wrangle::BridgeCallError # #code, #scope?
|
|
139
|
+
├─ Wrangle::ScopeLost # terminal: the window/tab/document is not the one you gave
|
|
140
|
+
├─ Wrangle::DeliveryUnknown # terminal: an action may or may not have landed
|
|
141
|
+
└─ Wrangle::StalePage # recoverable: observe again
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
## Speed
|
|
145
|
+
|
|
146
|
+
Measured against a real Safari on an M-series Mac, median of five runs:
|
|
147
|
+
|
|
148
|
+
| Step | Time |
|
|
149
|
+
|---|---:|
|
|
150
|
+
| Start the bridge, open a window, bind the document | 718 ms |
|
|
151
|
+
| First observation | 34 ms |
|
|
152
|
+
| Subsequent observation | 33 ms |
|
|
153
|
+
| One action | 84 ms |
|
|
154
|
+
| Close | 140 ms |
|
|
155
|
+
|
|
156
|
+
So a decision loop costs about **117 ms per act-and-observe step**. For comparison, the same work over
|
|
157
|
+
Safari's MCP server measured 9,059 ms to start and 1,327 ms per action — roughly 12× the startup and
|
|
158
|
+
16× the per-action cost.
|
|
159
|
+
|
|
160
|
+
The reason is that an Apple Event costs about **17 ms flat**, no matter how much data it carries, so
|
|
161
|
+
the only optimisation that matters is sending fewer of them. A read costs two events (one scope guard,
|
|
162
|
+
one evaluation) and a mutation costs four. The page scripts are shipped once at startup instead of
|
|
163
|
+
~12 KB per call, unresolved specifiers are addressed rather than resolved, and nothing reads window
|
|
164
|
+
bounds on the hot path.
|
|
165
|
+
|
|
166
|
+
## CLI
|
|
167
|
+
|
|
168
|
+
```
|
|
169
|
+
$ wrangle windows --titles
|
|
170
|
+
26081 display 0 4 tabs Home / X
|
|
171
|
+
33389 display 1 1 tab Are we the Krell? - YouTube
|
|
172
|
+
|
|
173
|
+
$ wrangle displays
|
|
174
|
+
0 x=0 y=31 1440x2529
|
|
175
|
+
1 x=-1920 y=351 1920x1080
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## Interactive sessions
|
|
179
|
+
|
|
180
|
+
A browser session is only useful if it survives between commands, and a shell gives you one process
|
|
181
|
+
per command. `wrangle open` starts a background server holding one window behind a socket in
|
|
182
|
+
`~/.wrangle`, so every later command drives the same page.
|
|
183
|
+
|
|
184
|
+
```
|
|
185
|
+
$ wrangle open "https://www.google.com/travel/flights?q=Flights+from+OKC+to+DEN" --display 1 --settle 8
|
|
186
|
+
page "Oklahoma City to Denver | Google Flights" <https://...>
|
|
187
|
+
state 54da85dcabb5 scroll 0/1623 1218 chars 72 actions
|
|
188
|
+
13 click button 1 passenger, change number of passengers.
|
|
189
|
+
|
|
190
|
+
$ wrangle act 13
|
|
191
|
+
acted click "1 passenger, change number of passengers."
|
|
192
|
+
changed text -1139
|
|
193
|
+
appeared "Add adult", "Add child aged 2 to 11", "Done", "Cancel"
|
|
194
|
+
gone "Change ticket type. Round trip", "Where from? Oklahoma City OKC"
|
|
195
|
+
1 click button Add adult
|
|
196
|
+
5 click button Done
|
|
197
|
+
|
|
198
|
+
$ wrangle act 1
|
|
199
|
+
acted click "Add adult"
|
|
200
|
+
appeared "Remove adult"
|
|
201
|
+
|
|
202
|
+
$ wrangle text --match 'adult'
|
|
203
|
+
Prices include required taxes + fees for 2 adults.
|
|
204
|
+
|
|
205
|
+
$ wrangle close
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Every action answers the only question that matters next: **what moved?** `"Remove adult"` appearing
|
|
209
|
+
is the proof the count went 1 → 2. When nothing moves, it says so outright — `changed nothing — the
|
|
210
|
+
page is byte-identical` — instead of leaving you to diff two page dumps.
|
|
211
|
+
|
|
212
|
+
The numbers are refs into the last observation and they shift after every action. Acting on a stale
|
|
213
|
+
one is refused rather than mis-clicked. `--settle N` polls until the page stops changing instead of
|
|
214
|
+
sleeping a guessed interval; without it you will read half-loaded pages and believe them.
|
|
215
|
+
|
|
216
|
+
| Command | |
|
|
217
|
+
|---|---|
|
|
218
|
+
| `open <url>` / `attach <id>` | start a session; `--display N`, `--side left`, `--session NAME` |
|
|
219
|
+
| `observe` | look; `--match RE`, `--all`, `--settle S` |
|
|
220
|
+
| `act <ref>` | one action; `--text STR`, `--settle S` |
|
|
221
|
+
| `run` | decide and act in a loop; `--goal`, `--plan`, `--execute` |
|
|
222
|
+
| `text` | page text; `--match RE` |
|
|
223
|
+
| `status` / `close` | |
|
|
224
|
+
|
|
225
|
+
Exit codes: `0` ok, `2` usage, `3` stale (observe and retry), `4` the session is over, `5` no
|
|
226
|
+
session. Add `--json` to any command for the raw reply.
|
|
227
|
+
|
|
228
|
+
`--side left|right|top|bottom` parks the window on half a display, which is usually where you want
|
|
229
|
+
an agent's browser: big enough to render a real page, out of the way of yours.
|
|
230
|
+
|
|
231
|
+
## Letting it decide
|
|
232
|
+
|
|
233
|
+
Stepping by hand costs a full model turn per click. Measured against a real agent driving this same
|
|
234
|
+
form, each turn was **5–6 seconds** — roughly fifteen times the cost of the decision itself. `wrangle
|
|
235
|
+
run` closes that loop in-process using [Jev](https://typesafe.ai), a typed-choice model that picks
|
|
236
|
+
one of the actions Wrangle already observed.
|
|
237
|
+
|
|
238
|
+
```bash
|
|
239
|
+
wrangle run --goal "Set the origin to OKC and the destination to DEN" \
|
|
240
|
+
--literal 'where from=OKC' --literal 'where to=DEN' --execute
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
It never invents text. `--literal LABEL=VALUE` supplies it; a fill with no matching literal stops the
|
|
244
|
+
run and asks. Nothing touches the page without `--execute`.
|
|
245
|
+
|
|
246
|
+
### Plans
|
|
247
|
+
|
|
248
|
+
One sub-goal per leg, advancing internally when a leg reports itself done:
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
wrangle run --execute --min-confidence 0.4 \
|
|
252
|
+
--literal 'where from=OKC' --literal 'where to=DEN' \
|
|
253
|
+
--plan "Set the origin to OKC and the destination to DEN, choosing from each autocomplete list." \
|
|
254
|
+
--plan "Open the Departure field, then click the day Monday, October 12, 2026." \
|
|
255
|
+
--plan "Click Thursday, October 15, 2026, then click Done to confirm the dates." \
|
|
256
|
+
--plan "Open the passenger selector, add a second adult, then click Done." \
|
|
257
|
+
--plan "Click Search." \
|
|
258
|
+
--expect 'taxes \+ fees for 2 adults'
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
```
|
|
262
|
+
== 1. Set the origin to OKC and the destination to DEN, choosing from each autocomplete list.
|
|
263
|
+
1. did Where from? (89% sure/93% target, jev 364ms, step 483ms)
|
|
264
|
+
typed: "OKC"
|
|
265
|
+
3. would The page moved while deciding; looked again (1) (0% sure, step 415ms)
|
|
266
|
+
4. did Will Rogers International Airport (OKC) (87% sure/92% target, jev 282ms, step 474ms)
|
|
267
|
+
...
|
|
268
|
+
proven the page shows "taxes \+ fees for 2 adults"
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
That run filled the whole form through the site's own calendar and passenger UI in **29 steps and
|
|
272
|
+
~14 seconds of one command**, where hand-stepping the same task took ten agent turns and 129
|
|
273
|
+
seconds. Narrow legs beat one broad goal: a leg that names the next concrete step gets 90%+
|
|
274
|
+
confidence, where a whole-task goal leaves the model weighing whether it is already finished.
|
|
275
|
+
|
|
276
|
+
A leg that cannot finish ends the plan. Later legs assume the earlier ones happened, so guessing
|
|
277
|
+
past a failure is how a run types a date into a passenger field.
|
|
278
|
+
|
|
279
|
+
### When it stops
|
|
280
|
+
|
|
281
|
+
| | |
|
|
282
|
+
|---|---|
|
|
283
|
+
| `did` | acted, with how sure it was and where the time went |
|
|
284
|
+
| `would` | proposed only (no `--execute`) |
|
|
285
|
+
| `ask` | handed back — below the confidence floor, or needs a literal |
|
|
286
|
+
| `done` / `blocked` | the leg reported itself finished, or stuck |
|
|
287
|
+
|
|
288
|
+
Below `--min-confidence` (default `0.5`) it looks again once — a page half-rendered when the model
|
|
289
|
+
looked reads as ambiguity, and a second decision costs ~350ms against 5–6s for a handoff — then
|
|
290
|
+
hands back rather than acting. This is load-bearing: an early build executed a **5%-confidence**
|
|
291
|
+
target and typed the origin into Google's "Where else?" multi-city field.
|
|
292
|
+
|
|
293
|
+
Stopping gets the floor too, but `DONE` and `BLOCKED` are not symmetric. Inside a plan an uncertain
|
|
294
|
+
`DONE` is cheap to be wrong about — the next leg simply does the work — while an uncertain `BLOCKED`
|
|
295
|
+
abandons every remaining leg. So both get a second look, `DONE` then passes, and `BLOCKED` must earn
|
|
296
|
+
its handoff. A leg that has not acted yet goes further and confirms a `BLOCKED` however sure it is:
|
|
297
|
+
it started the instant the last leg ended, so it is often reading a page that is still loading, and
|
|
298
|
+
a half-loaded page does not read as ambiguous — it reads as definite, and reads surer on the second
|
|
299
|
+
look.
|
|
300
|
+
|
|
301
|
+
### Checking its own work
|
|
302
|
+
|
|
303
|
+
`DONE` is the model reporting on its own work, chosen from the same look that proposed the actions,
|
|
304
|
+
and it is optimistic. An Amazon plan reported success with the filter it had been asked for never
|
|
305
|
+
applied; a flights leg finished a passenger dialog it had not finished.
|
|
306
|
+
|
|
307
|
+
So every request carries a second question — *is this goal's outcome actually visible on the page?*
|
|
308
|
+
— answered independently of the one choosing actions. It costs no extra round trip, because Jev
|
|
309
|
+
evaluates the heads in parallel, and it has no action to gain by agreeing. A confident `no`
|
|
310
|
+
overrules the claim and the leg carries on working:
|
|
311
|
+
|
|
312
|
+
```
|
|
313
|
+
3. would Said done, but the page does not show "In the left sidebar, click..." (76% sure)
|
|
314
|
+
4. did Apply 4 Stars & Up filter to narrow results (98% sure/100% target)
|
|
315
|
+
5. done DONE (79% sure)
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
An unsure `no` is ignored — a verifier that is merely uncertain is noise, not evidence. And a
|
|
319
|
+
dispute is worth more than the claim, not final: the verifier reads one snapshot, while the actor
|
|
320
|
+
knows what it did. A leg that applied an Amazon filter was disputed at 80% five runs in a row,
|
|
321
|
+
because the only proof on the page was a single link offering to remove the filter again.
|
|
322
|
+
|
|
323
|
+
So a dispute is decisive only against a leg that has **not done anything** — the premature-`DONE`
|
|
324
|
+
case it was built for, where there is no work to weigh against the page's reading. A leg that acted
|
|
325
|
+
looks again, then is taken at its word with the doubt written into the transcript:
|
|
326
|
+
|
|
327
|
+
```
|
|
328
|
+
3. done Done, but the page does not show "...the 4 Stars & Up filter" (79% sure)
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
A whole-task check at the end of a plan was tried and dropped: asked whether a filtered, sorted list
|
|
332
|
+
had been achieved, it scored **lower on runs that succeeded than on runs that failed**. Absence is
|
|
333
|
+
much harder to see than presence, and a check that cannot tell the two apart is worse than none.
|
|
334
|
+
|
|
335
|
+
### Deciding and waiting at the same time
|
|
336
|
+
|
|
337
|
+
A decision is made about a snapshot, and the page is free to move while the model is reading it. If
|
|
338
|
+
it does, the guard refuses the action and the whole decision is thrown away. On Google Flights that
|
|
339
|
+
was about a quarter of all decisions.
|
|
340
|
+
|
|
341
|
+
The obvious answer is to wait for the page to go still *before* asking. That is what `STEADY_BUDGET`
|
|
342
|
+
was for, and it had never once run: the CLI always sent the key, so `fetch("steady", 0.2)` returned
|
|
343
|
+
the `nil` that was there and `nil.to_f` is zero. The backoff after a stale retry was multiplying
|
|
344
|
+
that zero by two. Nothing had waited for a page in months.
|
|
345
|
+
|
|
346
|
+
Restoring it would have cost roughly what it saved — proving a page is still takes two looks about
|
|
347
|
+
200ms apart, and a decision only takes ~350ms. So the looks happen **underneath** the request
|
|
348
|
+
instead. Jev is asked on one thread while the page is watched on another, and the watching stops at
|
|
349
|
+
the first pair of reads that agree. A quiet page now costs nothing to confirm, and when a decision
|
|
350
|
+
is rejected the fresh read is already in hand, so the retry is one request rather than a request and
|
|
351
|
+
a read.
|
|
352
|
+
|
|
353
|
+
That left the question of why decisions were being rejected at all, which the transcript could not
|
|
354
|
+
answer because every rejection said the same thing. The guard now reports which of its parts moved:
|
|
355
|
+
|
|
356
|
+
```
|
|
357
|
+
4. RESTALE The page moved while deciding; looked again [The target is gone]
|
|
358
|
+
7. RESTALE The page moved while deciding; looked again [The page's address changed]
|
|
359
|
+
```
|
|
360
|
+
|
|
361
|
+
The second one was the bug. The guard compared `location.href` byte for byte, and Google Flights
|
|
362
|
+
puts the whole itinerary in a `?tfs=` parameter and rewrites it on every keystroke. Comparing the
|
|
363
|
+
origin and path instead — a real navigation, not a query string — removed that reason entirely. A
|
|
364
|
+
genuine route change still shows up, three times over: the path changes, or the document does, or
|
|
365
|
+
the DOM is rebuilt and the element's own identity changes with it.
|
|
366
|
+
|
|
367
|
+
A fill was also being checked against the whole-page marker, which compares the title, every word of
|
|
368
|
+
text and every action on the page, so any price or banner arriving anywhere rejected a decision
|
|
369
|
+
about a search box that had not moved. Anything aimed at an element is now checked against that
|
|
370
|
+
element.
|
|
371
|
+
|
|
372
|
+
Measured on the flights form, against the same task and page:
|
|
373
|
+
|
|
374
|
+
| | before | after |
|
|
375
|
+
|---|---|---|
|
|
376
|
+
| wall seconds per decision | 1.04 | **0.67** |
|
|
377
|
+
| cost of one rejected decision | 695ms | **428ms** |
|
|
378
|
+
| reading the page, per decision | 120ms | **38ms** |
|
|
379
|
+
| rejected decisions | 27% | 25% |
|
|
380
|
+
|
|
381
|
+
The rejection *rate* barely moved, and that is the honest result: what remains is the page genuinely
|
|
382
|
+
changing under the decision — an autocomplete list re-rendering, a calendar day growing a price —
|
|
383
|
+
and re-deciding is the right response to both. What got cheaper is being wrong.
|
|
384
|
+
|
|
385
|
+
One change was tried and reverted. With the watch in place the backoff after a stale retry looked
|
|
386
|
+
redundant, since the page has been settling during the request anyway. Removing it took rejected
|
|
387
|
+
decisions from 4.3 to 7.3 per run and the run got slower, so it stayed.
|
|
388
|
+
|
|
389
|
+
What is left is mostly Jev itself: about 350ms a call, flat regardless of how much page it is sent,
|
|
390
|
+
and a little over half of a run.
|
|
391
|
+
|
|
392
|
+
### Backends
|
|
393
|
+
|
|
394
|
+
`--backend jxa` (default) drives the running Safari through Apple Events. `--backend mcp` uses
|
|
395
|
+
`safaridriver --mcp` instead: per-action latency drops from ~120ms to ~3ms, but it pays ~4s of
|
|
396
|
+
startup, so it only amortises past roughly 28 actions. It is also **experimental** — its automation
|
|
397
|
+
tab is backgrounded, and pages whose menus animate on `requestAnimationFrame` (Google's ticket-type
|
|
398
|
+
selector, for one) never respond to a click that works fine under JXA.
|
|
399
|
+
|
|
400
|
+
## Use it from an AI agent
|
|
401
|
+
|
|
402
|
+
`skills/wrangle` is an [Agent Skill](https://agentskills.io) teaching the loop above, so an agent
|
|
403
|
+
drives Safari with shell commands and never writes Ruby. For [pi](https://github.com/badlogic/pi):
|
|
404
|
+
|
|
405
|
+
```bash
|
|
406
|
+
pi package add git:github.com/ericboehs/wrangle
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
Or point any harness at `skills/wrangle/SKILL.md`.
|
|
410
|
+
|
|
411
|
+
## How it works
|
|
412
|
+
|
|
413
|
+
One `osascript -l JavaScript` process stays alive for the session and speaks newline-delimited JSON
|
|
414
|
+
over a pipe. Spawning it costs ~56 ms; talking to one that already exists costs ~0.3 ms.
|
|
415
|
+
|
|
416
|
+
Three pieces of JavaScript do the actual work. `bridge.js` runs in JXA and owns the Safari objects,
|
|
417
|
+
scope checks, and window lifecycle. `page.js` and `snapshot.js` run in the page: one installs the
|
|
418
|
+
document epoch and executes guarded actions, the other builds the observation.
|
|
419
|
+
|
|
420
|
+
Page requests are JSON, passed as a **single argument** to a fixed function that was installed at
|
|
421
|
+
startup:
|
|
422
|
+
|
|
423
|
+
```js
|
|
424
|
+
(pageScript)({"op":"act","action":{...},"nonce":"..."}, () => (snapshotScript))
|
|
425
|
+
```
|
|
426
|
+
|
|
427
|
+
Nothing a caller supplies is ever interpolated into the program's structure.
|
|
428
|
+
|
|
429
|
+
One honest caveat: actions are dispatched as synthetic DOM events. They are not OS-level trusted
|
|
430
|
+
input, and a page that checks `event.isTrusted` will know the difference.
|
|
431
|
+
|
|
432
|
+
## Limitations
|
|
433
|
+
|
|
434
|
+
- macOS and Safari only.
|
|
435
|
+
- No navigation API — to visit a different URL, open another window.
|
|
436
|
+
- No screenshots, no file uploads, no credential entry, no arbitrary JavaScript from the caller.
|
|
437
|
+
- Only one Safari process may be running.
|
|
438
|
+
- Observed page text is capped at 6000 characters; fill text at 2000.
|
|
439
|
+
- Requires *Allow JavaScript from Apple Events*, which is a real privilege. Grant it deliberately.
|
|
440
|
+
|
|
441
|
+
## Development
|
|
442
|
+
|
|
443
|
+
No bundle, no compiler, no native extensions:
|
|
444
|
+
|
|
445
|
+
```
|
|
446
|
+
rake # test + node --check on the JavaScript (+ rubocop if installed)
|
|
447
|
+
rake test
|
|
448
|
+
```
|
|
449
|
+
|
|
450
|
+
RuboCop is optional and commented out of the `Gemfile`, because it wants a native `json` build and a
|
|
451
|
+
checkout should not need a working compiler.
|
|
452
|
+
|
|
453
|
+
The suite runs against a fake bridge subprocess that speaks the real protocol, so it exercises the
|
|
454
|
+
transport, the scope rules, and all four delivery-resolution outcomes without touching a browser.
|
|
455
|
+
Jev's client runs against a real HTTP server on a real socket for the same reason: everything worth
|
|
456
|
+
checking there — a 429 that has to be retried, a connection that opens and then says nothing — is a
|
|
457
|
+
property of the transport, not of a stub.
|
|
458
|
+
|
|
459
|
+
```
|
|
460
|
+
COVERAGE=1 rake test # per-file lines and branches
|
|
461
|
+
COVERAGE=1 COVERAGE_DETAIL=1 rake test # and which ones are missing
|
|
462
|
+
```
|
|
463
|
+
|
|
464
|
+
Coverage is measured with Ruby's own `Coverage` module rather than a gem: adding a dependency to
|
|
465
|
+
check that Wrangle has none would be a strange way to keep the claim. CI enforces a floor of 95% on
|
|
466
|
+
branches as well as lines, because line coverage says very little about a codebase whose main job is
|
|
467
|
+
refusing things — a guard runs on every happy path and never once refuses.
|
|
468
|
+
|
|
469
|
+
## License
|
|
470
|
+
|
|
471
|
+
MIT. `lib/wrangle/js/snapshot.js` is vendored from
|
|
472
|
+
[browser-use/jev-ultrafast](https://github.com/browser-use/jev-ultrafast) and used under the MIT
|
|
473
|
+
License, Copyright (c) 2026 Browser Use. See [LICENSE.txt](LICENSE.txt).
|