ego-jev 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -0
- package/README.md +63 -29
- package/bin/ego-jev.mjs +49 -8
- package/docs/AGENT_GUIDE.md +27 -0
- package/package.json +2 -2
- package/src/ego-jev.mjs +41 -8
- package/src/jev/jev.mjs +35 -10
- package/src/proxy.mjs +64 -0
- package/src/run.mjs +5 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,32 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.1 (2026-10-12)
|
|
4
|
+
|
|
5
|
+
- `ego-jev proxy` receives the key on stdin instead of its environment. A process's initial environment stays
|
|
6
|
+
readable with `ps eww` by the same user for the process's whole life; 0.3.0 put the key there.
|
|
7
|
+
|
|
8
|
+
## 0.3.0 (2026-10-12)
|
|
9
|
+
|
|
10
|
+
- Security: the API key no longer goes into the ego runtime's `process.env`. The runtime keeps one `process.env` for
|
|
11
|
+
its whole life and shares it with every script, so 0.2.0 left the key readable by later ego scripts. It now lives
|
|
12
|
+
only in the Jev client's module scope.
|
|
13
|
+
- `ego-jev proxy [--idle MINUTES]`, `proxy status`, `proxy stop`: a background process holds the key and forwards Jev
|
|
14
|
+
requests over a unix socket (mode 600). While it runs, `do`, `check` and `read` need no key, which saves the 2 to 4 s
|
|
15
|
+
a secret manager takes per call.
|
|
16
|
+
- Settle waits for same-origin `fetch`/XHR requests as well as DOM mutations, so single-page apps that re-render
|
|
17
|
+
after their own request are not read too early.
|
|
18
|
+
- Custom checkboxes whose transparent input covers the label's left edge are clicked on the input.
|
|
19
|
+
- Requirements count as reached only in order: a later one is not latched while an earlier one is still open (a
|
|
20
|
+
"filter is selected" score on the home page no longer counts as progress).
|
|
21
|
+
- With requirements, a confident target (0.6 or more) is acted on when the focused requirement scores under 0.2, even
|
|
22
|
+
if Jev's tool answer is "no action".
|
|
23
|
+
- Enter in the field just typed into no longer stops as `ambiguous` when its target confidence is low.
|
|
24
|
+
- `docs/AGENT_GUIDE.md`: when an agent should hand work to ego-jev, from six rounds of agent experiments.
|
|
25
|
+
- `experiments/agent/`: agent-with-ego-jev against agent-alone runs (tasks, grader, summariser, 122 recorded runs).
|
|
26
|
+
With the final guide: 20.5 s against 29.0 s on average over six tasks, 25 s against 75 s on Danawa.
|
|
27
|
+
- `experiments/`: G마켓 and YES24 tasks; Danawa checks read all three filter layouts. With requirements, 29 of 30
|
|
28
|
+
regression runs were correct and none reported `done` for a wrong result.
|
|
29
|
+
|
|
3
30
|
## 0.2.0 (2026-10-12)
|
|
4
31
|
|
|
5
32
|
- `--require "<page state>"` (repeatable): states the run has to reach, in order. Each round Jev scores them and works
|
package/README.md
CHANGED
|
@@ -13,20 +13,23 @@ goal and the values.
|
|
|
13
13
|
Your LLM agent stays in charge of what to do and why. ego-jev takes the clicking off its hands and keeps page
|
|
14
14
|
snapshots out of its context.
|
|
15
15
|
|
|
16
|
+
**Measured with Claude Opus 5.5 as the agent** (fresh subagents, paired runs, all 24 correct,
|
|
17
|
+
[details](experiments/agent/README.md)): on a 1,000-element shopping page with filters and sorting the agent finished
|
|
18
|
+
in 25 s with ego-jev against 75 s alone; across six tasks, 20.5 s against 29.0 s on average, with 40 % fewer tool
|
|
19
|
+
calls. On tasks where the agent did not hand anything to ego-jev, both took the same time.
|
|
20
|
+
|
|
16
21
|
## Requirements
|
|
17
22
|
|
|
18
23
|
- macOS with ego lite installed and onboarded (`ego-browser` on your PATH or in `~/.local/bin`)
|
|
19
24
|
- Node.js 20 or later
|
|
20
|
-
- A TypeSafe API key in `TYPESAFE_API_KEY`
|
|
25
|
+
- A TypeSafe API key in `TYPESAFE_API_KEY`, or an `ego-jev proxy` started with it
|
|
21
26
|
|
|
22
27
|
No other dependencies: no Playwright, no bundled browser.
|
|
23
28
|
|
|
24
29
|
## Install
|
|
25
30
|
|
|
26
31
|
```bash
|
|
27
|
-
npm install -g
|
|
28
|
-
# or the release tarball
|
|
29
|
-
npm install -g https://github.com/kimjisub/ego-jev/releases/download/v0.2.0/ego-jev-0.2.0.tgz
|
|
32
|
+
npm install -g ego-jev
|
|
30
33
|
# or from source
|
|
31
34
|
git clone https://github.com/kimjisub/ego-jev.git && cd ego-jev && npm link
|
|
32
35
|
```
|
|
@@ -61,6 +64,7 @@ EGO_JEV_RESULT {"status":"likely_done","url":".../checkout-complete.html","actio
|
|
|
61
64
|
ego-jev do "<goal>" [key=value ...] [--require "<page state>" ...] [--url U] [--space N] [--page p1] [--max 10] [--allow-irreversible] [--finish] [--name NAME]
|
|
62
65
|
ego-jev check "<yes/no question>" [--space N] [--page p1]
|
|
63
66
|
ego-jev read "<question>" [--space N] [--page p1]
|
|
67
|
+
ego-jev proxy [--idle MINUTES] | ego-jev proxy status | ego-jev proxy stop
|
|
64
68
|
```
|
|
65
69
|
|
|
66
70
|
- `do` works toward a goal. `key=value` pairs are the only strings Jev may type or select.
|
|
@@ -69,7 +73,33 @@ ego-jev read "<question>" [--space N] [--page p1]
|
|
|
69
73
|
- `space` in the result is the ego TaskSpace id. Pass it to `--space` to continue in the same tab, or open it from
|
|
70
74
|
your own `ego-browser nodejs` script with `taskSpace(12)` and take over by hand.
|
|
71
75
|
- `--finish` closes the TaskSpace when the command ends. Otherwise close it with `task.finish({ keep: [] })`.
|
|
72
|
-
- With a secret manager,
|
|
76
|
+
- With a secret manager, start the proxy once under it and call ego-jev without the key afterwards:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
op run --env-file keys.env -- ego-jev proxy # holds the key for 2 hours of inactivity (--idle MINUTES)
|
|
80
|
+
ego-jev do ... # no TYPESAFE_API_KEY needed while the proxy runs
|
|
81
|
+
ego-jev proxy status
|
|
82
|
+
ego-jev proxy stop
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The proxy receives the key on stdin (not in its environment, which `ps eww` would show), keeps it in memory and
|
|
86
|
+
forwards Jev requests that arrive on a unix socket (`~/.cache/ego-jev/jev.sock`, mode 600, in a mode 700
|
|
87
|
+
directory). It never sends the key back. Any process of your
|
|
88
|
+
user can send requests through it while it runs. A secret manager call costs 2 to 4 s, so this matters when an
|
|
89
|
+
agent calls ego-jev many times.
|
|
90
|
+
|
|
91
|
+
## Using it from an agent
|
|
92
|
+
|
|
93
|
+
Give your agent [docs/AGENT_GUIDE.md](docs/AGENT_GUIDE.md) (1.8 KB), ideally inside the skill or instructions it
|
|
94
|
+
already reads for browser work. The rule it teaches came out of six rounds of agent experiments:
|
|
95
|
+
|
|
96
|
+
> On a site whose markup you do not know, hand filters, sorting and multi-screen flows (cart, checkout, multi-step
|
|
97
|
+
> forms) to ego-jev from your first browser call. Do everything else yourself: sites you know, URLs you have seen
|
|
98
|
+
> work, a single form or click, and reading the result.
|
|
99
|
+
|
|
100
|
+
Two rules that sounded right and lost time: "use ego-jev whenever you would have to look at the page first" (agents
|
|
101
|
+
handed it demo sites they already knew), and "fall back to ego-jev when your own attempt fails" (agents keep
|
|
102
|
+
debugging instead of handing over).
|
|
73
103
|
|
|
74
104
|
## How to phrase a task
|
|
75
105
|
|
|
@@ -116,8 +146,9 @@ Rules that came out of the runs:
|
|
|
116
146
|
|
|
117
147
|
Each round:
|
|
118
148
|
|
|
119
|
-
1. Wait until the DOM has been quiet for 400 ms
|
|
120
|
-
not block
|
|
149
|
+
1. Wait until the DOM has been quiet for 400 ms and no same-origin `fetch` or XHR is in flight (pages open at
|
|
150
|
+
`domcontentloaded`, so slow third-party resources do not block; single-page apps that re-render after their own
|
|
151
|
+
request are waited for).
|
|
121
152
|
2. List the interactive elements of the top document with their label, state (checked, value, expanded, sorted,
|
|
122
153
|
selected) and whether they only open a popup or help text, and tag them with `data-jev-i`.
|
|
123
154
|
3. Ask Jev one request with several questions: is the goal done, is the page blocked, is an error shown, is this a
|
|
@@ -133,44 +164,47 @@ Before returning `done` without requirements, ego-jev asks a stricter question:
|
|
|
133
164
|
of the goal in effect, either shown (a selected filter, an entered value) or evidenced (a confirmation, the page after
|
|
134
165
|
submitting)? A goal that orders or submits is not in effect while the form is merely filled.
|
|
135
166
|
|
|
136
|
-
Clicks on checkboxes and radios land on the left edge of their label, because labels often hold links in the middle
|
|
167
|
+
Clicks on checkboxes and radios land on the left edge of their label, because labels often hold links in the middle;
|
|
168
|
+
when a custom checkbox lays its transparent input over that edge, the input itself is clicked.
|
|
137
169
|
A blocked click is retried after scrolling the target to the centre and once more after Escape; if it stays blocked,
|
|
138
170
|
Jev is told what covers it (`dialog "..." covers the target; close or dismiss it first`).
|
|
139
171
|
|
|
140
|
-
The API key reaches the ego runtime through a one-shot FIFO (mode 600)
|
|
141
|
-
|
|
172
|
+
The API key reaches the ego runtime through a one-shot FIFO (mode 600), or stays in the `ego-jev proxy` process. It
|
|
173
|
+
is never written to disk and never put into the runtime's `process.env`, which the ego runtime shares between all
|
|
174
|
+
scripts it runs.
|
|
142
175
|
|
|
143
176
|
## When to use it
|
|
144
177
|
|
|
145
|
-
|
|
146
|
-
([
|
|
147
|
-
|
|
148
|
-
| Task | Agent drives ego | Agent + ego-jev |
|
|
149
|
-
|---|---|---|
|
|
150
|
-
| Danawa: two filters + sort on a 900-element page, read top 5 | 31.6 s | **22.8 s** |
|
|
151
|
-
| SauceDemo: log in, add 2 items, check out, finish | 19.3 s | **16.7 s** |
|
|
152
|
-
| httpbin form: 7 fields + submit | **10.2 s** | 15.8 s |
|
|
153
|
-
| Wikipedia: search, open article, read one value | **7.7 s** | 15.9 s |
|
|
178
|
+
Fresh Claude Opus 5.5 subagents, the same task with and without ego-jev, two paired runs each, wall clock including
|
|
179
|
+
the agent's own thinking ([agent experiments](experiments/agent/README.md)):
|
|
154
180
|
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
181
|
+
| Task | Agent alone | Agent + ego-jev | ego-jev used |
|
|
182
|
+
|---|---|---|---|
|
|
183
|
+
| Danawa: search, two filters, sort, top 3 (about 1,000 elements, hidden checkbox inputs) | 75.2 s | **25.2 s** | yes |
|
|
184
|
+
| G마켓: search, brand and delivery filters, sort, top 3 (single-page app) | 38.9 s | **31.0 s** | yes |
|
|
185
|
+
| YES24: search, category, sort, three titles | 22.3 s | 23.8 s | yes |
|
|
186
|
+
| httpbin form: 7 fields, submit | 13.1 s | 12.5 s | no |
|
|
187
|
+
| Wikipedia: one value from an article | 10.0 s | 10.8 s | no |
|
|
188
|
+
| 국가법령정보센터: one definition from a statute in an iframe | 14.6 s | 19.9 s | no (agent variance) |
|
|
189
|
+
|
|
190
|
+
- **Use ego-jev** for filters, sorting and multi-screen flows on sites your agent does not know, pages with hundreds of
|
|
191
|
+
elements or awkward markup, and when you want to keep page snapshots out of your agent's context.
|
|
192
|
+
- **Drive ego directly** for sites the model already knows (popular demo apps are in its training data), a URL you
|
|
193
|
+
can build, and one or two actions. An ego-jev call costs about 0.5 s to start plus 1 to 3 s per action.
|
|
194
|
+
- **Read results with your own script** in the same TaskSpace after ego-jev; `ego-jev read` ranks blocks by relevance
|
|
195
|
+
and can surface recommendation boxes above the list you sorted.
|
|
163
196
|
|
|
164
197
|
## Limits
|
|
165
198
|
|
|
166
199
|
- Only the top document is read; elements inside iframes are not listed.
|
|
167
|
-
- Settling watches DOM mutations
|
|
200
|
+
- Settling watches DOM mutations and same-origin `fetch`/XHR requests, not WebSockets or cross-origin data calls.
|
|
168
201
|
- New tabs are not followed automatically.
|
|
169
202
|
- Controls whose selected state is not in the DOM (no `checked`, `aria-pressed`, `aria-selected` or a
|
|
170
203
|
`selected`/`active` class) cannot be scored as requirements reliably.
|
|
171
204
|
- The irreversible-step guard is a probability with a 0.6 threshold. On sites with real payments, deletes or
|
|
172
205
|
messages, keep `--max` small and split the goal into steps you can check.
|
|
173
|
-
- A requirement counts as reached at a score of 0.5
|
|
206
|
+
- A requirement counts as reached at a score of 0.5, once every earlier requirement has been reached, and stays
|
|
207
|
+
reached. If a later action undoes an earlier filter,
|
|
174
208
|
the run ends as `likely_done` and lists it; it does not go back to fix it.
|
|
175
209
|
|
|
176
210
|
## Credits
|
package/bin/ego-jev.mjs
CHANGED
|
@@ -5,6 +5,7 @@
|
|
|
5
5
|
// ego-jev do "<goal>" [key=value ...] [--url U] [--space N] [--page p1] [--max 10] [--require R ...] [--allow-irreversible] [--finish] [--name NAME]
|
|
6
6
|
// ego-jev check "<yes/no question>" [--space N] [--page p1]
|
|
7
7
|
// ego-jev read "<question>" [--space N] [--page p1]
|
|
8
|
+
// ego-jev proxy [--idle MINUTES] | proxy stop | proxy status
|
|
8
9
|
//
|
|
9
10
|
// The last line printed is `EGO_JEV_RESULT {json}`. Pass its `space` to --space to continue in the same TaskSpace.
|
|
10
11
|
//
|
|
@@ -17,14 +18,20 @@ import { join, dirname, delimiter } from "node:path";
|
|
|
17
18
|
import { randomBytes } from "node:crypto";
|
|
18
19
|
import { fileURLToPath, pathToFileURL } from "node:url";
|
|
19
20
|
import { createInterface } from "node:readline";
|
|
21
|
+
import { request } from "node:http";
|
|
22
|
+
import { SOCKET, proxyAlive } from "../src/proxy.mjs";
|
|
20
23
|
|
|
21
24
|
const ROOT = join(dirname(realpathSync(fileURLToPath(import.meta.url))), "..");
|
|
22
25
|
|
|
23
26
|
const USAGE = `usage: ego-jev do|check|read "<text>" [key=value ...] [--url U] [--space N] [--page p1] [--max N] [--require R ...] [--allow-irreversible] [--finish] [--name NAME]
|
|
27
|
+
ego-jev proxy [--idle MINUTES] | proxy status | proxy stop
|
|
24
28
|
|
|
25
29
|
do work toward a goal; key=value pairs are the values Jev may type or select
|
|
26
30
|
check yes/no question about the current page, answered as a probability
|
|
27
31
|
read the page text relevant to a question (Jev ranks blocks, code keeps the best)
|
|
32
|
+
proxy start a background process that holds TYPESAFE_API_KEY and forwards Jev requests over a unix socket
|
|
33
|
+
(mode 600). While it runs, do/check/read need no key. It exits after --idle minutes without requests
|
|
34
|
+
(default 120). "proxy stop" ends it, "proxy status" reports it
|
|
28
35
|
|
|
29
36
|
--url U open U first (otherwise continue on the current page)
|
|
30
37
|
--space N reuse TaskSpace N (default: create one named --name or "ego-jev")
|
|
@@ -37,12 +44,14 @@ const USAGE = `usage: ego-jev do|check|read "<text>" [key=value ...] [--url U] [
|
|
|
37
44
|
--no-merge-require score requirements in a separate request every round instead of with the action
|
|
38
45
|
--finish close the TaskSpace afterwards
|
|
39
46
|
|
|
40
|
-
environment: TYPESAFE_API_KEY (required), EGO_BROWSER (path to ego-browser, optional)
|
|
47
|
+
environment: TYPESAFE_API_KEY (required unless a proxy runs), EGO_BROWSER (path to ego-browser, optional),
|
|
48
|
+
EGO_JEV_SOCKET (proxy socket, default ~/.cache/ego-jev/jev.sock)`;
|
|
41
49
|
|
|
42
50
|
const fail = msg => { console.error(msg); process.exit(2); };
|
|
43
51
|
const argv = process.argv.slice(2);
|
|
44
52
|
if (!argv.length || ["-h", "--help", "help"].includes(argv[0])) { console.log(USAGE); process.exit(0); }
|
|
45
53
|
const cmd = argv.shift();
|
|
54
|
+
if (cmd === "proxy") await proxyCommand(argv);
|
|
46
55
|
if (!["do", "check", "read"].includes(cmd)) fail(USAGE);
|
|
47
56
|
const A = { cmd, values: {} };
|
|
48
57
|
for (let k = 0; k < argv.length; k++) {
|
|
@@ -64,15 +73,20 @@ for (let k = 0; k < argv.length; k++) {
|
|
|
64
73
|
if (!A.goal) fail(USAGE);
|
|
65
74
|
|
|
66
75
|
const key = process.env.TYPESAFE_API_KEY;
|
|
67
|
-
if (!key)
|
|
76
|
+
if (!key) {
|
|
77
|
+
if (await proxyAlive()) A.proxy = SOCKET;
|
|
78
|
+
else fail("TYPESAFE_API_KEY is not set and no ego-jev proxy is running. Export the key, wrap the call with your secret manager, or start `ego-jev proxy` once under it.");
|
|
79
|
+
}
|
|
68
80
|
|
|
69
81
|
const which = name => (process.env.PATH ?? "").split(delimiter).map(d => join(d, name)).find(p => existsSync(p));
|
|
70
82
|
const egoBrowser = [process.env.EGO_BROWSER, which("ego-browser"), join(homedir(), ".local/bin/ego-browser")].find(p => p && existsSync(p));
|
|
71
83
|
if (!egoBrowser) fail("ego-browser was not found. Install ego lite (https://github.com/citrolabs/ego-lite) and finish its onboarding.");
|
|
72
84
|
|
|
73
|
-
A.
|
|
74
|
-
|
|
75
|
-
|
|
85
|
+
if (!A.proxy) {
|
|
86
|
+
A.keyFifo = join(tmpdir(), `ego-jev-${randomBytes(6).toString("hex")}.fifo`);
|
|
87
|
+
if (spawnSync("mkfifo", ["-m", "600", A.keyFifo]).status !== 0) fail("could not create a FIFO for the API key");
|
|
88
|
+
}
|
|
89
|
+
const cleanup = () => A.keyFifo && rmSync(A.keyFifo, { force: true });
|
|
76
90
|
|
|
77
91
|
const script = [
|
|
78
92
|
`globalThis.EGO_JEV_ARGS = ${JSON.stringify(A)};`,
|
|
@@ -90,8 +104,35 @@ for (const s of [child.stdout, child.stderr]) {
|
|
|
90
104
|
createInterface({ input: s }).on("line", l => (OURS.test(l) ? process.stdout : process.stderr).write(l + "\n"));
|
|
91
105
|
}
|
|
92
106
|
// Opening a FIFO for writing waits until the runtime opens it for reading.
|
|
93
|
-
|
|
94
|
-
w
|
|
95
|
-
w.
|
|
107
|
+
if (A.keyFifo) {
|
|
108
|
+
const w = createWriteStream(A.keyFifo);
|
|
109
|
+
w.on("error", () => {});
|
|
110
|
+
w.end(key);
|
|
111
|
+
}
|
|
96
112
|
child.on("close", code => { cleanup(); process.exitCode = code ?? 1; });
|
|
97
113
|
for (const sig of ["SIGINT", "SIGTERM"]) process.on(sig, () => { child.kill(sig); cleanup(); process.exit(130); });
|
|
114
|
+
|
|
115
|
+
async function proxyCommand(args) {
|
|
116
|
+
const sub = args[0] && !args[0].startsWith("--") ? args.shift() : "start";
|
|
117
|
+
const alive = await proxyAlive();
|
|
118
|
+
if (sub === "status") { console.log(alive ? `running on ${SOCKET}` : "not running"); process.exit(alive ? 0 : 1); }
|
|
119
|
+
if (sub === "stop") {
|
|
120
|
+
if (!alive) { console.log("not running"); process.exit(0); }
|
|
121
|
+
await new Promise(res => { const r = request({ socketPath: SOCKET, path: "/stop", method: "POST" }, s => { s.resume(); s.on("end", res); }); r.on("error", res); r.end(); });
|
|
122
|
+
console.log("stopped"); process.exit(0);
|
|
123
|
+
}
|
|
124
|
+
if (sub !== "start") fail(USAGE);
|
|
125
|
+
if (alive) { console.log(`already running on ${SOCKET}`); process.exit(0); }
|
|
126
|
+
if (!process.env.TYPESAFE_API_KEY) fail("TYPESAFE_API_KEY is not set; start the proxy under your secret manager, e.g. op run --env-file keys.env -- ego-jev proxy");
|
|
127
|
+
const k = args.indexOf("--idle");
|
|
128
|
+
const idle = (k >= 0 ? Number(args[k + 1]) : 120) * 60_000;
|
|
129
|
+
const code = `const { serveFromStdin } = await import(${JSON.stringify(pathToFileURL(join(ROOT, "src/proxy.mjs")).href)}); await serveFromStdin({ idle: ${idle} });`;
|
|
130
|
+
// the key goes over stdin; the child's environment (readable with ps eww) does not carry it
|
|
131
|
+
const env = { ...process.env }; delete env.TYPESAFE_API_KEY;
|
|
132
|
+
const child = spawn(process.execPath, ["--input-type=module", "-e", code], { detached: true, stdio: ["pipe", "ignore", "ignore"], env });
|
|
133
|
+
child.stdin.end(process.env.TYPESAFE_API_KEY);
|
|
134
|
+
child.unref();
|
|
135
|
+
for (let i = 0; i < 30 && !(await proxyAlive()); i++) await new Promise(r => setTimeout(r, 100));
|
|
136
|
+
if (!(await proxyAlive())) fail("the proxy did not start");
|
|
137
|
+
console.log(`running on ${SOCKET}`); process.exit(0);
|
|
138
|
+
}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# ego-jev for agents
|
|
2
|
+
|
|
3
|
+
ego-jev drives a stretch of a site inside your ego TaskSpace: you give the goal, the strings to type and the page
|
|
4
|
+
states to reach; Jev picks every click (about 1 to 3 s per action, no tokens of yours).
|
|
5
|
+
|
|
6
|
+
**Use it for one thing: on a site whose markup you do not know, filters and sorting on a results page, or a flow
|
|
7
|
+
over several screens (cart, checkout, multi-step forms).** Make it your first browser call, with `--url`, instead of
|
|
8
|
+
looking at the page yourself. Do everything else with your own `ego-browser nodejs` scripts: sites you know, a URL
|
|
9
|
+
you have seen work, a single form or click on one page, and reading the result afterwards.
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
ego-jev do "<goal>" key=value ... --url <start url> --name <taskspace> --max <actions + 3> \
|
|
13
|
+
--require "<page state 1>" --require "<page state 2>" ... [--allow-irreversible]
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
- `key=value`: every string to type (search terms, names). Jev cannot write text.
|
|
17
|
+
- `--require`: what the page shows when each part is done, in order ("the 로지텍 brand filter is selected",
|
|
18
|
+
"the list is sorted by lowest price"). Not clicks.
|
|
19
|
+
- `--allow-irreversible` only when the task says to order, submit, send or delete.
|
|
20
|
+
- `--url`: start as close to the work as you can. If you know the site's search URL pattern, start on the results
|
|
21
|
+
page; every search Jev does from the home page costs two or three rounds.
|
|
22
|
+
- `--name` is the TaskSpace your scripts open with `taskSpace("<name>")`.
|
|
23
|
+
|
|
24
|
+
The last line is `EGO_JEV_RESULT {json}`. On `done`, the page is in that state: read what the task asks for and
|
|
25
|
+
finish, or finish right away if there is nothing to read. On anything else, the `requirements` scores show which
|
|
26
|
+
part is missing (under 0.5); the page is often already right, so check that part with one script and do only what
|
|
27
|
+
is left.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ego-jev",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.1",
|
|
4
4
|
"description": "Let Jev (TypeSafe System One) drive multi-step flows inside ego lite, your real browser. The agent supplies goals, values and target page states; Jev picks every click.",
|
|
5
5
|
"keywords": ["ego-lite", "jev", "typesafe", "browser-automation", "agent", "claude-code"],
|
|
6
6
|
"license": "MIT",
|
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
"type": "module",
|
|
12
12
|
"bin": { "ego-jev": "bin/ego-jev.mjs" },
|
|
13
13
|
"exports": { ".": "./src/ego-jev.mjs" },
|
|
14
|
-
"files": ["bin", "src", "README.md", "CHANGELOG.md", "LICENSE"],
|
|
14
|
+
"files": ["bin", "src", "docs", "README.md", "CHANGELOG.md", "LICENSE"],
|
|
15
15
|
"engines": { "node": ">=20" },
|
|
16
16
|
"os": ["darwin"]
|
|
17
17
|
}
|
package/src/ego-jev.mjs
CHANGED
|
@@ -104,8 +104,22 @@ export class EgoJev {
|
|
|
104
104
|
window.__jevMut = performance.now();
|
|
105
105
|
window.__jevMO = new MutationObserver(recs => { if (recs.some(r => r.attributeName !== "data-jev-i")) window.__jevMut = performance.now(); });
|
|
106
106
|
window.__jevMO.observe(document, { subtree: true, childList: true, attributes: true, characterData: true });
|
|
107
|
+
// Single-page apps change the URL at once and re-render only when their fetch returns, often after more
|
|
108
|
+
// than a second of DOM silence. Count same-origin requests in flight (analytics and ads go to other
|
|
109
|
+
// origins); ones older than 2.5 s (long polls) do not count.
|
|
110
|
+
const live = window.__jevNet = new Map(); let n = 0;
|
|
111
|
+
const own = u => { try { return new URL(String(u), location.href).origin === location.origin; } catch { return false; } };
|
|
112
|
+
// a finished request counts as a change too, so the render that follows it is waited for
|
|
113
|
+
const track = p => { const id = ++n; live.set(id, performance.now()); const done = () => { if (live.delete(id)) window.__jevMut = Math.max(window.__jevMut, performance.now()); }; p?.then?.(done, done); return done; };
|
|
114
|
+
const f = window.fetch;
|
|
115
|
+
if (f) window.fetch = function (...a) { const p = f.apply(this, a); if (own(a[0]?.url ?? a[0])) track(p); return p; };
|
|
116
|
+
const open = XMLHttpRequest.prototype.open, send = XMLHttpRequest.prototype.send;
|
|
117
|
+
XMLHttpRequest.prototype.open = function (m, u, ...r) { this.__jevOwn = own(u); return open.call(this, m, u, ...r); };
|
|
118
|
+
XMLHttpRequest.prototype.send = function (...a) { if (this.__jevOwn) this.addEventListener("loadend", track(), { once: true }); return send.apply(this, a); };
|
|
107
119
|
}
|
|
108
|
-
|
|
120
|
+
const now = performance.now();
|
|
121
|
+
if ([...window.__jevNet.values()].some(t => now - t < 2500)) return 0;
|
|
122
|
+
return document.readyState === "loading" ? 0 : now - window.__jevMut;
|
|
109
123
|
});
|
|
110
124
|
} catch { idle = 0; }
|
|
111
125
|
if (idle >= quiet) break;
|
|
@@ -281,11 +295,24 @@ export class EgoJev {
|
|
|
281
295
|
if (!l) return null;
|
|
282
296
|
l.scrollIntoView({ block: "center", inline: "nearest" });
|
|
283
297
|
l.setAttribute("data-jev-label", "1");
|
|
284
|
-
|
|
298
|
+
const r = l.getBoundingClientRect();
|
|
299
|
+
// the transparent input of a custom checkbox often sits over the label's left edge
|
|
300
|
+
return { h: r.height, inputOnEdge: isToggle && document.elementFromPoint(r.x + 4, r.y + r.height / 2) === el };
|
|
285
301
|
}, s);
|
|
286
302
|
const clear = () => this.ep.evaluate(() => document.querySelectorAll("[data-jev-label]").forEach(l => l.removeAttribute("data-jev-label")));
|
|
303
|
+
if (box?.inputOnEdge) {
|
|
304
|
+
await clear();
|
|
305
|
+
return this.ep.click(s, opts);
|
|
306
|
+
}
|
|
287
307
|
if (box) {
|
|
288
|
-
|
|
308
|
+
// Custom checkboxes often lay the transparent input over the label's left edge, so the click lands on the very
|
|
309
|
+
// control we want but ego reports it as covered. Then click the input itself, and last the label's centre.
|
|
310
|
+
try { return await this.ep.click("[data-jev-label]", { ...opts, timeout: 1500, position: { x: 4, y: Math.max(1, Math.floor(box.h / 2)) } }); }
|
|
311
|
+
catch (e) {
|
|
312
|
+
if (!/intercepts pointer events/i.test(String(e))) throw e;
|
|
313
|
+
try { return await this.ep.click(s, { ...opts, timeout: 1500 }); } catch {}
|
|
314
|
+
try { return await this.ep.click("[data-jev-label]", { ...opts, timeout: 1500 }); } catch { throw e; }
|
|
315
|
+
}
|
|
289
316
|
finally { await clear(); }
|
|
290
317
|
}
|
|
291
318
|
try { return await this.ep.click(s, { ...opts, timeout: 1500 }); }
|
|
@@ -435,12 +462,14 @@ export class EgoJev {
|
|
|
435
462
|
// been reached and the last is in effect now; done if every one is still in effect, likely_done otherwise.
|
|
436
463
|
// With mergeRequire, the scores come with the action in one request: the round guesses that the focus is the
|
|
437
464
|
// same as last round's, and asks again with the new focus only if the scores show that guess was already reached.
|
|
438
|
-
let focus = goal, extra = {}, req, a;
|
|
465
|
+
let focus = goal, focusP = 1, extra = {}, req, a;
|
|
439
466
|
const hist = history.slice(-12), diff = round ? pageDiff(prevPage, page) : undefined;
|
|
440
467
|
if (require.length) {
|
|
441
468
|
const firstOpen = () => require.findIndex((_, k) => !reached.has(k));
|
|
442
469
|
const extraFor = () => ({ overall_goal: goal, already_reached: require.filter((_, k) => reached.has(k)) });
|
|
443
|
-
|
|
470
|
+
// Requirements are given in order, so one counts as reached only once every earlier one has been: a high score
|
|
471
|
+
// for "the free-delivery filter is selected" on the home page, before any search, is not taken as progress.
|
|
472
|
+
const score = sc => { req = sc.map(p => +p.toFixed(2)); for (let k = 0; k < req.length; k++) { if (req[k] >= REQUIRE_MET && (k === 0 || reached.has(k - 1))) reached.add(k); } };
|
|
444
473
|
const guess = firstOpen();
|
|
445
474
|
// only pages asked in one stage can carry the scores; on two-stage pages the guess would cost a request
|
|
446
475
|
if (mergeRequire && guess >= 0 && page.elements.length <= MAX_SINGLE && fits(page)) {
|
|
@@ -459,12 +488,14 @@ export class EgoJev {
|
|
|
459
488
|
break;
|
|
460
489
|
}
|
|
461
490
|
if (next < 0) next = last;
|
|
462
|
-
focus = require[next];
|
|
491
|
+
focus = require[next]; focusP = req[next];
|
|
463
492
|
extra = extraFor();
|
|
464
493
|
if (!a) a = await this.decide(page, focus, values, hist, diff, extra);
|
|
465
494
|
} else a = await this.decide(page, goal, values, hist, diff);
|
|
466
495
|
prevPage = page;
|
|
467
|
-
|
|
496
|
+
// With requirements, "no action" next to a confident target is a contradiction when the tool answer is weak, or
|
|
497
|
+
// when Jev itself scores the focused requirement as clearly not in effect (under 0.2): act on the target then.
|
|
498
|
+
if (req && a.tool.choice === "none" && a.target && (a.tool.probabilities.none < 0.6 || focusP < 0.2)) {
|
|
468
499
|
const pt = Math.max(...Object.values(a.target.probabilities));
|
|
469
500
|
const alt = Object.entries(a.tool.probabilities).filter(([t]) => TARGETED.has(t)).sort((x, y) => y[1] - x[1])[0];
|
|
470
501
|
if (pt >= 0.6 && alt) a.tool = { ...a.tool, choice: alt[0], overridden_none: true };
|
|
@@ -514,7 +545,9 @@ export class EgoJev {
|
|
|
514
545
|
if (++waits > 6) { status = "stuck"; info = "page never finished loading"; break; }
|
|
515
546
|
await this.settle({ max: 4000 }); await sleep(600); history.push({ action: "wait" }); continue;
|
|
516
547
|
}
|
|
517
|
-
|
|
548
|
+
// Enter in the field just typed into needs no confident pick: that is where a value waits to be submitted
|
|
549
|
+
const enterAfterTyping = act.tool === "press_enter" && act.el && brief(act.el) === history.findLast(h => h.action === "type" && !h.error)?.element;
|
|
550
|
+
if (TARGETED.has(act.tool) && act.p_target < minTarget && !enterAfterTyping) { status = "ambiguous"; info = "target confidence too low; refine the goal or act on a candidate directly"; break; }
|
|
518
551
|
const seenKey = `${act.tool}|${brief(act.el)}|${["type", "select", "upload"].includes(act.tool) ? act.valueKey : ""}|${page.url}|${page.text}|${JSON.stringify(page.elements)}`;
|
|
519
552
|
seen.set(seenKey, (seen.get(seenKey) ?? 0) + 1);
|
|
520
553
|
if (seen.get(seenKey) >= 3) { status = "stuck"; info = "repeating the same action on the same page without progress"; break; }
|
package/src/jev/jev.mjs
CHANGED
|
@@ -1,30 +1,55 @@
|
|
|
1
1
|
// Minimal client for the Typesafe System One API (model: Jev).
|
|
2
2
|
// Jev answers typed questions over a state object; it never generates text.
|
|
3
|
-
// From jev-browser 0.1.2 (MIT, Ying-Kai Liao). Changed: the key comes
|
|
3
|
+
// From jev-browser 0.1.2 (MIT, Ying-Kai Liao). Changed: the key comes from setApiKey() or TYPESAFE_API_KEY (no .env lookup).
|
|
4
4
|
export const API_URL = process.env.JEV_API_URL || "https://api.typesafe.ai/v1/systemone";
|
|
5
5
|
export const MODEL = process.env.JEV_MODEL || "jev-latest";
|
|
6
6
|
const sleep = ms => new Promise(r => setTimeout(r, ms));
|
|
7
7
|
|
|
8
|
+
// Inside the ego runtime, process.env is shared by every script for the life of the runtime process, so the CLI
|
|
9
|
+
// hands the key over with setApiKey() and never puts it there.
|
|
10
|
+
let KEY = null;
|
|
11
|
+
export function setApiKey(k) { KEY = k; }
|
|
12
|
+
// With a local proxy (src/proxy.mjs) the requests go to its unix socket and no key is needed here.
|
|
13
|
+
let PROXY = null;
|
|
14
|
+
export function setProxy(socketPath) { PROXY = socketPath; }
|
|
15
|
+
|
|
16
|
+
async function viaProxy(payload, timeout) {
|
|
17
|
+
const { request } = await import("node:http");
|
|
18
|
+
return new Promise((res, rej) => {
|
|
19
|
+
const r = request({ socketPath: PROXY, path: "/", method: "POST", headers: { "Content-Type": "application/json" }, timeout }, s => {
|
|
20
|
+
const chunks = [];
|
|
21
|
+
s.on("data", c => chunks.push(c));
|
|
22
|
+
s.on("end", () => { let body = {}; try { body = JSON.parse(Buffer.concat(chunks).toString()); } catch {} res({ ok: s.statusCode < 300, status: s.statusCode, body }); });
|
|
23
|
+
});
|
|
24
|
+
r.on("error", rej); r.on("timeout", () => r.destroy(new Error("Jev proxy timeout")));
|
|
25
|
+
r.end(payload);
|
|
26
|
+
});
|
|
27
|
+
}
|
|
28
|
+
|
|
8
29
|
export function apiKey() {
|
|
9
|
-
const k = process.env.TYPESAFE_API_KEY;
|
|
30
|
+
const k = KEY ?? process.env.TYPESAFE_API_KEY;
|
|
10
31
|
if (!k) throw new Error("No TypeSafe API key: set TYPESAFE_API_KEY");
|
|
11
32
|
return k;
|
|
12
33
|
}
|
|
13
34
|
|
|
14
35
|
// questions: { name: { type: "noul" | "choice" | "score", instructions, criteria? } }
|
|
15
36
|
export async function jev(state, questions, { retries = 2, timeout = 60_000 } = {}) {
|
|
16
|
-
const key = apiKey();
|
|
37
|
+
const key = PROXY ? null : apiKey();
|
|
38
|
+
const payload = JSON.stringify({ state, model: MODEL, questions });
|
|
17
39
|
for (let attempt = 0; ; attempt++) {
|
|
18
40
|
const t = performance.now();
|
|
19
41
|
let res, body;
|
|
20
42
|
try {
|
|
21
|
-
res = await
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
43
|
+
if (PROXY) ({ body, ...res } = await viaProxy(payload, timeout));
|
|
44
|
+
else {
|
|
45
|
+
res = await fetch(API_URL, {
|
|
46
|
+
method: "POST",
|
|
47
|
+
headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
|
|
48
|
+
body: payload,
|
|
49
|
+
signal: AbortSignal.timeout(timeout),
|
|
50
|
+
});
|
|
51
|
+
body = await res.json().catch(() => ({}));
|
|
52
|
+
}
|
|
28
53
|
} catch (e) {
|
|
29
54
|
if (attempt < retries) { await sleep(800 * (attempt + 1)); continue; }
|
|
30
55
|
throw e;
|
package/src/proxy.mjs
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
// A local Jev proxy. It holds the API key in its own process and forwards Jev requests that arrive on a unix socket
|
|
2
|
+
// (mode 600, in a mode 700 directory), so later ego-jev calls need no key and skip the secret manager. The key is
|
|
3
|
+
// never sent back. Any process of the same user can send requests through it while it runs; it exits after
|
|
4
|
+
// `idle` ms without requests. The key arrives on stdin, not in the environment: a process's initial environment
|
|
5
|
+
// stays readable with `ps eww` for its whole life.
|
|
6
|
+
import { createServer, request } from "node:http";
|
|
7
|
+
import { mkdirSync, rmSync, chmodSync, existsSync } from "node:fs";
|
|
8
|
+
import { homedir } from "node:os";
|
|
9
|
+
import { join } from "node:path";
|
|
10
|
+
import { API_URL } from "./jev/jev.mjs";
|
|
11
|
+
|
|
12
|
+
export const SOCKET = process.env.EGO_JEV_SOCKET ?? join(homedir(), ".cache", "ego-jev", "jev.sock");
|
|
13
|
+
|
|
14
|
+
// Resolves true when a proxy answers on the socket.
|
|
15
|
+
export function proxyAlive(socket = SOCKET) {
|
|
16
|
+
if (!existsSync(socket)) return Promise.resolve(false);
|
|
17
|
+
return new Promise(res => {
|
|
18
|
+
const r = request({ socketPath: socket, path: "/ping", method: "GET", timeout: 1000 }, s => { s.resume(); res(s.statusCode === 200); });
|
|
19
|
+
r.on("error", () => res(false)); r.on("timeout", () => { r.destroy(); res(false); });
|
|
20
|
+
r.end();
|
|
21
|
+
});
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
export function serve({ key, socket = SOCKET, idle = 2 * 3600_000 }) {
|
|
25
|
+
mkdirSync(join(socket, ".."), { recursive: true, mode: 0o700 });
|
|
26
|
+
chmodSync(join(socket, ".."), 0o700);
|
|
27
|
+
rmSync(socket, { force: true });
|
|
28
|
+
let timer;
|
|
29
|
+
const bump = () => { clearTimeout(timer); timer = setTimeout(() => { server.close(); rmSync(socket, { force: true }); process.exit(0); }, idle); };
|
|
30
|
+
const server = createServer(async (req, res) => {
|
|
31
|
+
bump();
|
|
32
|
+
if (req.method === "GET" && req.url === "/ping") { res.end("ok"); return; }
|
|
33
|
+
if (req.method === "POST" && req.url === "/stop") { rmSync(socket, { force: true }); res.end("bye", () => process.exit(0)); return; }
|
|
34
|
+
if (req.method !== "POST") { res.statusCode = 405; res.end(); return; }
|
|
35
|
+
const chunks = [];
|
|
36
|
+
for await (const c of req) chunks.push(c);
|
|
37
|
+
try {
|
|
38
|
+
const up = await fetch(API_URL, {
|
|
39
|
+
method: "POST",
|
|
40
|
+
headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
|
|
41
|
+
body: Buffer.concat(chunks),
|
|
42
|
+
signal: AbortSignal.timeout(60_000),
|
|
43
|
+
});
|
|
44
|
+
res.statusCode = up.status;
|
|
45
|
+
res.setHeader("Content-Type", "application/json");
|
|
46
|
+
res.end(Buffer.from(await up.arrayBuffer()));
|
|
47
|
+
} catch (e) {
|
|
48
|
+
res.statusCode = 502;
|
|
49
|
+
res.end(JSON.stringify({ error: String(e.message ?? e) }));
|
|
50
|
+
}
|
|
51
|
+
});
|
|
52
|
+
server.listen(socket, () => { chmodSync(socket, 0o600); bump(); });
|
|
53
|
+
for (const sig of ["SIGINT", "SIGTERM"]) process.on(sig, () => { rmSync(socket, { force: true }); process.exit(0); });
|
|
54
|
+
return server;
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
// Reads the key from stdin until EOF, then serves.
|
|
58
|
+
export async function serveFromStdin(opts = {}) {
|
|
59
|
+
let key = "";
|
|
60
|
+
for await (const c of process.stdin) key += c;
|
|
61
|
+
key = key.trim();
|
|
62
|
+
if (!key) throw new Error("no API key on stdin");
|
|
63
|
+
return serve({ ...opts, key });
|
|
64
|
+
}
|
package/src/run.mjs
CHANGED
|
@@ -1,11 +1,14 @@
|
|
|
1
1
|
// Runs inside `ego-browser nodejs`. The short script written by the ego-jev CLI passes the arguments (EGO_JEV_ARGS)
|
|
2
|
-
// and ego's global functions (EGO_JEV_EGO), then imports this file. The API key is read once from a FIFO
|
|
2
|
+
// and ego's global functions (EGO_JEV_EGO), then imports this file. The API key is read once from a FIFO
|
|
3
|
+
// and handed to the Jev client directly: process.env is shared by every script in the runtime process.
|
|
3
4
|
import { readFileSync } from "node:fs";
|
|
4
5
|
import { EgoJev } from "./ego-jev.mjs";
|
|
6
|
+
import { setApiKey, setProxy } from "./jev/jev.mjs";
|
|
5
7
|
|
|
6
8
|
const A = globalThis.EGO_JEV_ARGS;
|
|
7
9
|
const { taskSpace } = globalThis.EGO_JEV_EGO;
|
|
8
|
-
|
|
10
|
+
if (A.proxy) setProxy(A.proxy);
|
|
11
|
+
else setApiKey(readFileSync(A.keyFifo, "utf8").trim());
|
|
9
12
|
|
|
10
13
|
const task = A.space != null ? await taskSpace(Number(A.space)) : await taskSpace(A.name ?? "ego-jev");
|
|
11
14
|
const page = task.page(A.page ?? "p1");
|