dirpush 0.1.5 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -23,7 +23,7 @@ You are a **worker**, not the source of truth. SQLite (via the `dirpush` CLI) ow
|
|
|
23
23
|
Something the human should do that doesn't stop the job (verify an email later, check a listing) → `dirpush todo add --job <job> --message "…" [--url …]`. It shows in the dashboard's Actions tab.
|
|
24
24
|
10. **Hold your lease.** Checkpoints extend it; on long pages call `dirpush work heartbeat <job>` at least every 10 minutes. Stopping cleanly: `dirpush work release <job>`.
|
|
25
25
|
11. **Browser = `dirpush ab <agent-browser command>`.** It drives the agents' own Chrome profile (logged into the human's Google account; Gmail, Drive and account pages blocked), never the human's everyday browser. Any agent that can run shell commands can use it. Never call `agent-browser` directly on that profile, never `network unroute`, never another browser tool or extension. Google sign-in rules in [authentication.md](references/authentication.md) are hard rules.
|
|
26
|
-
Habits: `dirpush ab open <url>`, `dirpush ab snapshot -i` (refs like `@e3`), `dirpush ab fill @e3 "text"`, `dirpush ab select`, `dirpush ab check`, `dirpush ab click`. Uploads: `dirpush ab upload '<input[type=file] selector>' <absolute path>`, never a button ref; sign in before uploading. Every call has a 90 s limit (`AB_TIMEOUT`); exit 142 = it hung: `dirpush ab close
|
|
26
|
+
Habits: `dirpush ab open <url>`, `dirpush ab snapshot -i` (refs like `@e3`), `dirpush ab fill @e3 "text"`, `dirpush ab select`, `dirpush ab check`, `dirpush ab click`. Uploads: `dirpush ab upload '<input[type=file] selector>' <absolute path>`, never a button ref; sign in before uploading. Every call has a 90 s limit (`AB_TIMEOUT`); exit 142 = it hung: close that tab (`dirpush ab tab close`), reopen, continue from your last checkpoint. If every call hangs while one site is open (some pages freeze any automated browser on purpose), close that tab and `work block --type bot_protection` saying the human must submit it in their own browser; never close the whole browser for this.
|
|
27
27
|
12. **Bot checks: wait, don't fight.** Cloudflare / Turnstile usually clears by itself in this Chrome: wait ~10 s and re-snapshot. Never click a CAPTCHA or Turnstile checkbox yourself and never use a solving service. Still there → `work block --type captcha` with the page left open. A plain arithmetic question in a form is part of the form: answer it.
|
|
28
28
|
13. **Close your browser** (`dirpush ab close`) when you stop, unless a job of yours is blocked on the human in that window (CAPTCHA, login) — leave it for them. A throwaway profile (`AB_PROFILE`) loses the filled form when closed: in the block message, list exactly what to re-enter.
|
|
29
29
|
|
|
@@ -13,14 +13,14 @@ Always drive the browser through `dirpush ab` (agent-browser on the agents' own
|
|
|
13
13
|
|
|
14
14
|
## Creating a password without ever seeing it
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
With the sign-up (or reset) page open in `dirpush ab`, one command generates the password, types it into every field you name, checks the site got it, and saves it to the vault. It never appears in your context or any file:
|
|
17
17
|
|
|
18
18
|
```bash
|
|
19
|
-
|
|
20
|
-
|
|
19
|
+
dirpush browser password <directory-id>-<product-id> --url <this page's url> --username <email> \
|
|
20
|
+
[--login-url <the site's sign-in page>] '<password selector>' ['<confirm selector>']
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
-
|
|
23
|
+
Passwords always have upper, lower, digit and symbol (24 chars). Pass `--login-url` whenever the page isn't the sign-in page (a sign-up form, a reset link): the vault opens that URL for every later `browser login`. Never generate passwords yourself (`openssl rand -base64` can lack a symbol, and sites reject it).
|
|
24
24
|
|
|
25
25
|
## Continue with Google (`accounts.oauth_google`)
|
|
26
26
|
|
package/package.json
CHANGED
package/scripts/ab
CHANGED
|
@@ -19,7 +19,14 @@ AB="agent-browser --headed --session ${AB_SESSION:-default} --profile ${AB_PROFI
|
|
|
19
19
|
# `launch browser start` already runs this profile (Google services blocked by its PAC script): attach to it
|
|
20
20
|
# instead of launching a second Chrome on the same profile.
|
|
21
21
|
PORT="${LAUNCH_BROWSER_PORT:-9222}"
|
|
22
|
-
|
|
22
|
+
up() { curl -s -m 1 "http://127.0.0.1:$PORT/json/version" >/dev/null 2>&1; }
|
|
23
|
+
# The real profile always runs as that dedicated Chrome. Launching it through agent-browser instead relaunches Chrome
|
|
24
|
+
# on every call (losing tabs and filled forms), so start it here when it's down (e.g. after a Google sign-in).
|
|
25
|
+
if [ -z "$AB_PROFILE" ] && ! up && [ "$1" != "close" ]; then
|
|
26
|
+
agent-browser --session "${AB_SESSION:-default}" close >/dev/null 2>&1 || true
|
|
27
|
+
bun "$PKG/src/cli.ts" browser start >/dev/null 2>&1 || { echo "couldn't start the agents' Chrome: run 'dirpush browser start' and read its error" >&2; exit 1; }
|
|
28
|
+
fi
|
|
29
|
+
if [ -z "$AB_PROFILE" ] && up; then
|
|
23
30
|
exec perl -e 'alarm shift; exec @ARGV or die "cannot run $ARGV[0]: $!\n"' "${AB_TIMEOUT:-90}" agent-browser --session "${AB_SESSION:-default}" --cdp "$PORT" "$@"
|
|
24
31
|
fi
|
|
25
32
|
for host in mail.google.com gmail.com www.gmail.com inbox.google.com \
|
package/scripts/google-login
CHANGED
|
@@ -13,5 +13,5 @@ bun "$PKG/src/cli.ts" browser stop >/dev/null 2>&1
|
|
|
13
13
|
# so `dirpush checks` can carry on. A separate --user-data-dir is its own Chrome process; the human's Chrome is untouched.
|
|
14
14
|
for i in 1 2 3 4 5 6 7 8 9 10; do [ -L data/browser-profile/SingletonLock ] || break; sleep 0.5; done
|
|
15
15
|
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --user-data-dir="$PWD/data/browser-profile" \
|
|
16
|
-
--use-mock-keychain --password-store=basic --no-first-run --no-default-browser-check \
|
|
16
|
+
--use-mock-keychain --password-store=basic --no-first-run --no-default-browser-check --disable-extensions --disable-sync \
|
|
17
17
|
"https://accounts.google.com/ServiceLogin?continue=https%3A%2F%2Fwww.google.com%2F" >/dev/null 2>&1
|
package/src/browser.ts
CHANGED
|
@@ -46,6 +46,9 @@ export async function browserStatus() {
|
|
|
46
46
|
}
|
|
47
47
|
}
|
|
48
48
|
|
|
49
|
+
/** Flags every Chrome on the agents' profile gets (scripts/google-login repeats them). */
|
|
50
|
+
export const AGENT_CHROME_FLAGS = ["--use-mock-keychain", "--password-store=basic", "--no-first-run", "--no-default-browser-check", "--disable-extensions", "--disable-sync"];
|
|
51
|
+
|
|
49
52
|
export async function startBrowser() {
|
|
50
53
|
const now = await browserStatus();
|
|
51
54
|
if (now.running) return { ...now, started: false };
|
|
@@ -53,8 +56,9 @@ export async function startBrowser() {
|
|
|
53
56
|
const pac = `data:application/x-ns-proxy-autoconfig;base64,${Buffer.from(pacScript()).toString("base64")}`;
|
|
54
57
|
// Plain Chrome, headed, no automation flags: Google sign-in and bot checks see a normal browser. The keychain
|
|
55
58
|
// flags match agent-browser and scripts/google-login: the profile's cookies are encrypted with that key, so
|
|
56
|
-
// without them every saved session would be unreadable.
|
|
57
|
-
|
|
59
|
+
// without them every saved session would be unreadable. No extensions and no sync: signing into Google here turns on
|
|
60
|
+
// Chrome sync, which copies the human's own extensions in (one with debugger access hijacked every agent session).
|
|
61
|
+
Bun.spawn([CHROME, `--user-data-dir=${PROFILE}`, `--remote-debugging-port=${BROWSER_PORT}`, `--proxy-pac-url=${pac}`, ...AGENT_CHROME_FLAGS], {
|
|
58
62
|
stdout: "ignore", stderr: "ignore", stdin: "ignore",
|
|
59
63
|
}).unref();
|
|
60
64
|
for (let i = 0; i < 40; i++) {
|
|
@@ -136,7 +140,7 @@ export function strongPassword() {
|
|
|
136
140
|
* New account password: generated here, typed into every given field of the open tab on `url`'s site, checked,
|
|
137
141
|
* then saved to the vault as `name`. Returns no secret.
|
|
138
142
|
*/
|
|
139
|
-
export async function newPassword(name: string, url: string, username: string, selectors: string[]) {
|
|
143
|
+
export async function newPassword(name: string, url: string, username: string, selectors: string[], loginUrl = url) {
|
|
140
144
|
if (!selectors.length) throw new Error("pass the password field selector(s), e.g. 'input[type=password]'");
|
|
141
145
|
const page = await pageOn(url);
|
|
142
146
|
const pw = strongPassword();
|
|
@@ -145,6 +149,7 @@ export async function newPassword(name: string, url: string, username: string, s
|
|
|
145
149
|
const lengths = JSON.parse(ab(["eval", `JSON.stringify(${JSON.stringify(selectors)}.map((s) => document.querySelector(s)?.value.length ?? -1))`]));
|
|
146
150
|
const got = (typeof lengths === "string" ? JSON.parse(lengths) : lengths) as number[];
|
|
147
151
|
if (got.some((n) => n !== pw.length)) throw new Error(`password fields not filled (lengths ${got.join(", ")}); nothing saved`);
|
|
148
|
-
|
|
152
|
+
// The vault keeps the sign-in page: `browser login` opens on it later, so a reset or sign-up URL there breaks logins.
|
|
153
|
+
ab(["auth", "save", name, "--url", loginUrl, "--username", username, "--password-stdin"], pw);
|
|
149
154
|
return { vault: name, username, fields: selectors.length, page: page.url.slice(0, 200), saved: true };
|
|
150
155
|
}
|
package/src/cli.ts
CHANGED
|
@@ -79,6 +79,7 @@ const { values: flags, positionals } = parseArgs({
|
|
|
79
79
|
source: { type: "string" },
|
|
80
80
|
job: { type: "string" },
|
|
81
81
|
url: { type: "string" },
|
|
82
|
+
"login-url": { type: "string" },
|
|
82
83
|
username: { type: "string" },
|
|
83
84
|
rel: { type: "string" }, // registry set-link
|
|
84
85
|
samples: { type: "string" },
|
|
@@ -655,8 +656,8 @@ const commands: Record<string, (() => Promise<unknown> | unknown) | undefined> =
|
|
|
655
656
|
"browser stop": () => stopBrowser(),
|
|
656
657
|
"browser login": () => vaultLogin(arg(0, "vault-name")),
|
|
657
658
|
"browser password": () => {
|
|
658
|
-
if (!flags.url || !flags.username) throw new Error("browser password <vault-name> --url <
|
|
659
|
-
return newPassword(arg(0, "vault-name"), flags.url, flags.username, args.slice(1));
|
|
659
|
+
if (!flags.url || !flags.username) throw new Error("browser password <vault-name> --url <page-url> --username <email> [--login-url <sign-in page>] <field-selector…>");
|
|
660
|
+
return newPassword(arg(0, "vault-name"), flags.url, flags.username, args.slice(1), flags["login-url"]);
|
|
660
661
|
},
|
|
661
662
|
async "serve"() {
|
|
662
663
|
const { serve } = await import("./server");
|
|
@@ -730,7 +731,7 @@ const HELP: [string, [string, string][]][] = [
|
|
|
730
731
|
["google-login", "the human signs into Google once in the agents' Chrome profile"],
|
|
731
732
|
["browser start | status | stop", "the dedicated Chrome (real profile, Google services blocked) agents drive"],
|
|
732
733
|
["browser login <vault-name>", "fill a saved login into the open tab on that site (password never shown)"],
|
|
733
|
-
["browser password <vault-name> --url u --username e <selector…>", "new
|
|
734
|
+
["browser password <vault-name> --url u --username e [--login-url u] <selector…>", "new password (sign-up or reset): generate, fill, verify and save it (never shown); --login-url when the page isn't the sign-in page (e.g. a reset link)"],
|
|
734
735
|
]],
|
|
735
736
|
["Other", [
|
|
736
737
|
["dr show | dr refresh [--all]", "Domain Rating history (refresh needs AHREFS_API_KEY)"],
|