dirpush 0.1.7 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/dirpush/SKILL.md +19 -3
- package/README.md +19 -11
- package/package.json +1 -1
- package/src/cli.ts +17 -4
- package/src/handoff.ts +120 -0
- package/src/jobs.ts +5 -0
- package/src/walk.ts +92 -0
- package/src/checks.ts +0 -153
|
@@ -23,8 +23,8 @@ You are a **worker**, not the source of truth. SQLite (via the `dirpush` CLI) ow
|
|
|
23
23
|
Something the human should do that doesn't stop the job (verify an email later, check a listing) → `dirpush todo add --job <job> --message "…" [--url …]`. It shows in the dashboard's Actions tab.
|
|
24
24
|
10. **Hold your lease.** Checkpoints extend it; on long pages call `dirpush work heartbeat <job>` at least every 10 minutes. Stopping cleanly: `dirpush work release <job>`.
|
|
25
25
|
11. **Browser = `dirpush ab <agent-browser command>`.** It drives the agents' own Chrome profile (logged into the human's Google account; Gmail, Drive and account pages blocked), never the human's everyday browser. Any agent that can run shell commands can use it. Never call `agent-browser` directly on that profile, never `network unroute`, never another browser tool or extension. Google sign-in rules in [authentication.md](references/authentication.md) are hard rules.
|
|
26
|
-
Habits: `dirpush ab open <url>`, `dirpush ab snapshot -i` (refs like `@e3`), `dirpush ab fill @e3 "text"`, `dirpush ab select`, `dirpush ab check`, `dirpush ab click`. Uploads: `dirpush ab upload '<input[type=file] selector>' <absolute path>`, never a button ref; sign in before uploading. Every call has a 90 s limit (`AB_TIMEOUT`); exit 142 = it hung: close that tab (`dirpush ab tab close`), reopen, continue from your last checkpoint. If every call hangs while one site is open (some pages freeze any automated browser on purpose), close that tab and `work block --type bot_protection
|
|
27
|
-
12. **Bot checks: wait, don't fight.** Cloudflare / Turnstile usually clears by itself in this Chrome: wait ~10 s and re-snapshot. Never click a CAPTCHA or Turnstile checkbox yourself and never use a solving service. Still there → `work block --type captcha` with the page left open. A plain arithmetic question in a form is part of the form: answer it.
|
|
26
|
+
Habits: `dirpush ab open <url>`, `dirpush ab snapshot -i` (refs like `@e3`), `dirpush ab fill @e3 "text"`, `dirpush ab select`, `dirpush ab check`, `dirpush ab click`. Uploads: `dirpush ab upload '<input[type=file] selector>' <absolute path>`, never a button ref; sign in before uploading. Every call has a 90 s limit (`AB_TIMEOUT`); exit 142 = it hung: close that tab (`dirpush ab tab close`), reopen, continue from your last checkpoint. If every call hangs while one site is open (some pages freeze any automated browser on purpose), close that tab and `work block --type bot_protection --message "<Site> breaks under automation. Do it in your own browser: <the steps, with the URL>"` (that phrase makes it a by-hand hand-off); never close the whole browser for this.
|
|
27
|
+
12. **Bot checks: wait, don't fight.** Cloudflare / Turnstile usually clears by itself in this Chrome: wait ~10 s and re-snapshot. Never click a CAPTCHA or Turnstile checkbox yourself and never use a solving service. Still there → `work block --type captcha` with the page left open; it becomes a hand-off (below). A plain arithmetic question in a form is part of the form: answer it.
|
|
28
28
|
13. **Close your browser** (`dirpush ab close`) when you stop, unless a job of yours is blocked on the human in that window (CAPTCHA, login) — leave it for them. A throwaway profile (`AB_PROFILE`) loses the filled form when closed: in the block message, list exactly what to re-enter.
|
|
29
29
|
|
|
30
30
|
## Loop
|
|
@@ -61,14 +61,30 @@ Workers read `dirpush capabilities` at the start of a session: email verificatio
|
|
|
61
61
|
- **"Set up dirpush"** / **"set up this product"** / **"add a product"**: `dirpush init <product-id>`, run the Setup check above if `dirpush capabilities` says it hasn't been done, then help the user fill `workspace/products/<id>/product.yaml` (and the shared `workspace/identity.yaml`, `permissions.yaml` the first time). Validate with `dirpush product show --product <id>`.
|
|
62
62
|
- **"Find N free directories I haven't submitted to and submit"**: `dirpush directories recommend --free --unsubmitted --limit N --product <id>`, show the list, `dirpush submissions enqueue <dir-id…> --product <id>`, then run the loop.
|
|
63
63
|
- **"Continue my launch"**: the whole routine, no other instructions needed (nothing from a previous conversation is either):
|
|
64
|
-
1. `dirpush actions` —
|
|
64
|
+
1. `dirpush actions` — note what's waiting on the human; if they're here, do the **hand-offs** below with them now (before step 3, so their jobs join the loop).
|
|
65
65
|
2. `dirpush submissions check` — marks pending listings live (skip if run in the last day).
|
|
66
66
|
3. Run the loop until `work next` is empty. Jobs still showing another worker with an expired lease are abandoned: `work next` hands them to you; continue from their last step.
|
|
67
67
|
4. Queue empty → `dirpush directories recommend --free --unsubmitted --limit 10 --product <id>`, show the list, enqueue only what the human approves.
|
|
68
68
|
5. `dirpush capabilities` has no email tool, or you don't have that tool this session → verification steps are a `work block --type email_verification` with what to click, never a wait.
|
|
69
|
+
- **"What do you need from me?"** / the human is around and `dirpush handoff list` isn't empty: do the hand-offs below.
|
|
69
70
|
- **"Add a directory I submitted to myself"**: `dirpush submissions add <directory-id or URL> --listing-url <url> [--outcome live|pending]`.
|
|
70
71
|
- New directories go upstream: tell the human to report them on the dirpush repository; the registry is not researched locally.
|
|
71
72
|
|
|
73
|
+
## Hand-offs: the human's part, done with you in chat
|
|
74
|
+
|
|
75
|
+
The human never needs the CLI. Some steps only they can do: sign into Google once, tick a bot check, or (rarely) submit by hand on a site that breaks under automation. You walk them through these one at a time, in chat:
|
|
76
|
+
|
|
77
|
+
1. `dirpush handoff next` opens the page itself (Google's sign-in window, the agents' Chrome on the check, or their own browser for a by-hand site) and returns `ask`: one sentence. Say exactly that. For a by-hand site it also returns `paste`: give them those values to copy.
|
|
78
|
+
2. Wait for their reply. Never solve a CAPTCHA or bot check yourself, and don't poll the page while they work.
|
|
79
|
+
3. Their answer:
|
|
80
|
+
- "done" → `dirpush handoff done <job>` (`google` for the sign-in, which hands back every Google job), then **claim that job and finish it right away**, in the same tab, before the page goes stale. Then the next hand-off.
|
|
81
|
+
- submitted it themselves → `dirpush handoff mine <job> [--listing-url <url>]`.
|
|
82
|
+
- "not now" / "skip" → `dirpush handoff skip <job>` (hidden from `next` for 12 hours).
|
|
83
|
+
- "stop" → stop asking; the rest stays in `dirpush handoff list` for next time.
|
|
84
|
+
4. Keep it short: one hand-off per message, no lists of commands, no internal job ids unless they ask. If the human isn't around, don't start hand-offs; mention how many are waiting at the end.
|
|
85
|
+
|
|
86
|
+
A human who prefers the terminal can run plain `dirpush handoff` to walk the same queue themselves; then `continue my launch` finishes what they cleared.
|
|
87
|
+
|
|
72
88
|
## Command reference
|
|
73
89
|
|
|
74
90
|
`dirpush help` lists every command with a one-line description. Output is compact JSON on stdout: list rows are short (ids, states, 200-char messages, ISO times), and only `work next`, `work show` and `wait` include a job's saved `data`. Lists stop at 50 rows. When something is cut short or empty, a `note:` line on stderr says why and how to narrow it; read it. `--full` returns raw rows, `--pretty` indents. The most used:
|
package/README.md
CHANGED
|
@@ -15,23 +15,31 @@ npm i -g dirpush agent-browser # or run everything below with npx
|
|
|
15
15
|
dirpush init acme # sets up ~/.dirpush and installs the agent skill
|
|
16
16
|
```
|
|
17
17
|
|
|
18
|
-
`init` keeps your launch in `~/.dirpush` (database, product files, the agents' Chrome profile) and installs the dirpush skill for your agents in `~/.claude/skills` and `~/.agents/skills`.
|
|
18
|
+
`init` keeps your launch in `~/.dirpush` (database, product files, the agents' Chrome profile) and installs the dirpush skill for your agents in `~/.claude/skills` and `~/.agents/skills`. That's the only command you need.
|
|
19
19
|
|
|
20
|
-
##
|
|
20
|
+
## Talk to your agent
|
|
21
|
+
|
|
22
|
+
| Say | What happens |
|
|
23
|
+
| --- | --- |
|
|
24
|
+
| **"set up dirpush"** | It fills in your product's facts with you and checks its tools (an email tool for verification links, the browser). |
|
|
25
|
+
| **"continue my launch"** | It finds directories that fit, submits, verifies emails, re-checks pending listings, and records everything. |
|
|
26
|
+
| **"what do you need from me?"** | It walks you through the few things only you can do, one at a time: sign into Google once, tick a bot check in a window it opens, rarely submit a site by hand. You answer "done" and it carries on. |
|
|
27
|
+
| **"how is my launch going?"** | Live, pending and rejected listings, and what's waiting. |
|
|
28
|
+
|
|
29
|
+
Agents never solve CAPTCHAs, never pay, and ask before anything your `permissions.yaml` marks as "confirm". `dirpush serve` opens a local dashboard if you want to look around.
|
|
30
|
+
|
|
31
|
+
## The CLI, if you want it
|
|
32
|
+
|
|
33
|
+
Agents drive everything below; you never have to. `dirpush help` lists it all.
|
|
21
34
|
|
|
22
35
|
| Command | Purpose |
|
|
23
36
|
| --- | --- |
|
|
24
|
-
| `dirpush
|
|
37
|
+
| `dirpush actions` | What's waiting on you |
|
|
38
|
+
| `dirpush handoff` | Walk your part yourself in the terminal instead of in chat |
|
|
25
39
|
| `dirpush submissions list --product <id>` | Every submission and its outcome |
|
|
26
|
-
| `dirpush
|
|
27
|
-
| `dirpush actions` | What needs you: CAPTCHAs, verification emails, questions |
|
|
28
|
-
| `dirpush checks` | Your part, guided: sign into Google, then clear CAPTCHAs and bot checks one by one; agents pick each job back up |
|
|
40
|
+
| `dirpush directories recommend --free --unsubmitted --product <id>` | Directories that fit and haven't been tried |
|
|
29
41
|
| `dirpush registry updates [--apply]` | Update your registry from the published one on dirpush.com |
|
|
30
|
-
| `dirpush
|
|
31
|
-
| `dirpush submissions add <id or URL>` | Record a listing you made yourself |
|
|
32
|
-
| `dirpush dr refresh` / `dirpush dr show` | Optional Ahrefs Domain Rating history (needs `AHREFS_API_KEY`) |
|
|
33
|
-
|
|
34
|
-
`dirpush help` lists everything.
|
|
42
|
+
| `dirpush dr show` | Domain Rating history (optional, needs `AHREFS_API_KEY`) |
|
|
35
43
|
|
|
36
44
|
## Working on dirpush
|
|
37
45
|
|
package/package.json
CHANGED
package/src/cli.ts
CHANGED
|
@@ -19,7 +19,8 @@ import { auditDirectory, auditDirectoryBrowser } from "./audit";
|
|
|
19
19
|
import { browserStatus, newPassword, startBrowser, stopBrowser, vaultLogin } from "./browser";
|
|
20
20
|
import { accountsFor, backfillAccounts, listAccounts } from "./accounts";
|
|
21
21
|
import { detect, loadCapabilities, saveCapabilities } from "./capabilities";
|
|
22
|
-
import {
|
|
22
|
+
import { handoffDone, handoffMine, handoffNext, handoffQueue, handoffSkip } from "./handoff";
|
|
23
|
+
import { walk } from "./walk";
|
|
23
24
|
import { ratingDomain, ratingHistory, refreshRatings } from "./ratings";
|
|
24
25
|
import { fetchRegistryFile, registrySource, registryUpdates } from "./registry-updates";
|
|
25
26
|
import { loadIdentity, loadPermissions, loadProduct, permission, productIds, resolveProductId, PRODUCTS, utm, WORKSPACE } from "./workspace";
|
|
@@ -37,7 +38,7 @@ if (first === "--version" || first === "-v") {
|
|
|
37
38
|
}
|
|
38
39
|
if (first === "--help" || first === "-h") process.argv.splice(2, 1, "help");
|
|
39
40
|
// Names people type for the human-facing commands.
|
|
40
|
-
const ALIASES: Record<string, string> = { check: "
|
|
41
|
+
const ALIASES: Record<string, string> = { checks: "handoff", check: "handoff", captchas: "handoff", needs: "actions", todos: "actions" };
|
|
41
42
|
if (first && ALIASES[first]) process.argv.splice(2, 1, ALIASES[first]);
|
|
42
43
|
/** Agents read JSON errors; a person at a terminal gets a plain sentence. */
|
|
43
44
|
function fail(error: string, extra: { didYouMean?: string[] } = {}): never {
|
|
@@ -537,7 +538,17 @@ const commands: Record<string, (() => Promise<unknown> | unknown) | undefined> =
|
|
|
537
538
|
},
|
|
538
539
|
"assets capture-complete": () => consumeProductApproval(db, Number(arg(0, "approval-id"))),
|
|
539
540
|
/** The human's part, interactive: Google sign-in, then CAPTCHAs and bot checks one by one (src/checks.ts). */
|
|
540
|
-
|
|
541
|
+
// Hand-offs (src/handoff.ts): agents call next/done/mine/skip and talk the human through it in chat;
|
|
542
|
+
// a human at a terminal can run plain `dirpush handoff` to walk the same queue.
|
|
543
|
+
"handoff": async () => {
|
|
544
|
+
if (process.stdin.isTTY && process.stdout.isTTY) { await walk(db, flags.product); process.exit(0); }
|
|
545
|
+
return handoffQueue(db, flags.product);
|
|
546
|
+
},
|
|
547
|
+
"handoff list": () => handoffQueue(db, flags.product, { includeSkipped: flags.all }),
|
|
548
|
+
"handoff next": () => handoffNext(db, flags.product),
|
|
549
|
+
"handoff done": () => handoffDone(db, arg(0, "job or google")),
|
|
550
|
+
"handoff mine": () => handoffMine(db, arg(0, "job"), flags["listing-url"]),
|
|
551
|
+
"handoff skip": () => handoffSkip(db, arg(0, "job")),
|
|
541
552
|
|
|
542
553
|
/** Everything waiting on the human, grouped. --product narrows it. */
|
|
543
554
|
"actions": () => {
|
|
@@ -677,7 +688,9 @@ const HELP: [string, [string, string][]][] = [
|
|
|
677
688
|
["products list", "all products in this workspace"],
|
|
678
689
|
["permission <group.action>", "automatic | confirm | never for an action, e.g. accounts.oauth_google"],
|
|
679
690
|
["actions [--product id]", "everything waiting on the human, grouped"],
|
|
680
|
-
["
|
|
691
|
+
["handoff next [--product id]", "the next thing only the human can do (Google sign-in, a bot check, a by-hand site): opens it and returns the sentence to say in chat"],
|
|
692
|
+
["handoff done <job|google> | mine <job> [--listing-url u] | skip <job>", "the human answered: hand the job back (then finish it now), they submitted it themselves, or not now"],
|
|
693
|
+
["handoff list [--all]", "everything waiting on the human (a person at a terminal: plain 'dirpush handoff' walks it)"],
|
|
681
694
|
]],
|
|
682
695
|
["Work loop", [
|
|
683
696
|
["work next [--claim]", "next submission job (unknown ones first); --claim leases it to you"],
|
package/src/handoff.ts
ADDED
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Hand-offs: the bits of a submission only the human can do (sign into Google, tick a bot check, or fill a form on a
|
|
3
|
+
* site that breaks under automation). Agent-first: the agent calls `handoff next`, which opens the page and returns
|
|
4
|
+
* one sentence to relay in chat; when the human answers, `handoff done` hands the job back and the agent finishes it
|
|
5
|
+
* right away, in the same window. `dirpush handoff` in a terminal walks a human through the same queue (src/walk.ts).
|
|
6
|
+
*/
|
|
7
|
+
import { join } from "node:path";
|
|
8
|
+
import { and, eq, gt, inArray } from "drizzle-orm";
|
|
9
|
+
import { HOME, PKG, type Db } from "./db";
|
|
10
|
+
import { events, jobs, type Job } from "./db/schema";
|
|
11
|
+
import * as J from "./jobs";
|
|
12
|
+
import { loadDirectory } from "./registry";
|
|
13
|
+
import { loadProduct, utm } from "./workspace";
|
|
14
|
+
|
|
15
|
+
export type Kind = "google_login" | "tick" | "own_browser";
|
|
16
|
+
/** `ask`: what the agent says in chat. `todo`: the same, short, for the terminal walk. */
|
|
17
|
+
export type Handoff = { kind: Kind; jobs: string[]; directory: string; name: string; url?: string; ask: string; todo: string };
|
|
18
|
+
|
|
19
|
+
const TICK = ["captcha", "bot_protection"];
|
|
20
|
+
const SKIP_HOURS = 12;
|
|
21
|
+
const env = { ...process.env, LAUNCH_HOME: HOME };
|
|
22
|
+
export const script = (name: string, ...args: string[]) =>
|
|
23
|
+
Bun.spawnSync([join(PKG, "scripts", name), ...args], { env, stdin: "ignore", stdout: "ignore", stderr: "ignore" }).exitCode;
|
|
24
|
+
|
|
25
|
+
/** Sites that break under automation: the agent's block message says the human must use their own browser. */
|
|
26
|
+
export const ownBrowser = (job: Job) => /your own browser/i.test(job.blockMessage ?? "");
|
|
27
|
+
const firstUrl = (s?: string | null) => s?.match(/https?:\/\/[^\s)'"”]+/)?.[0]?.replace(/[.,;:]$/, "");
|
|
28
|
+
/** The human's instruction from a block message written for them ("… in your own browser: <this>, then press s …"). */
|
|
29
|
+
const instruction = (job: Job) =>
|
|
30
|
+
(job.blockMessage ?? "").split(/your own browser:?/i)[1]?.trim().replace(/,?\s*then press s in dirpush (checks|handoff)\.?$/i, "").replace(/\.$/, "") || "submit it";
|
|
31
|
+
|
|
32
|
+
function skippedRecently(db: Db) {
|
|
33
|
+
const since = Date.now() - SKIP_HOURS * 3600_000;
|
|
34
|
+
return new Set(db.select().from(events).where(and(eq(events.type, "handoff_skipped"), gt(events.at, since))).all().map((e) => e.jobId));
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
/** What's waiting on the human, in order: one Google sign-in for all directories that need it, then each check. */
|
|
38
|
+
export async function handoffQueue(db: Db, product?: string, opts: { includeSkipped?: boolean } = {}): Promise<Handoff[]> {
|
|
39
|
+
const skipped = opts.includeSkipped ? new Set<string | null>() : skippedRecently(db);
|
|
40
|
+
const blocked = db.select().from(jobs)
|
|
41
|
+
.where(and(eq(jobs.kind, "submission"), eq(jobs.status, "blocked"), inArray(jobs.blockType, ["google_login", ...TICK]))).all()
|
|
42
|
+
.filter((j) => (!product || j.productId === product) && !skipped.has(j.id));
|
|
43
|
+
const out: Handoff[] = [];
|
|
44
|
+
const google = blocked.filter((j) => j.blockType === "google_login");
|
|
45
|
+
if (google.length) out.push({
|
|
46
|
+
kind: "google_login", jobs: google.map((j) => j.id), directory: "google", name: "Google sign-in",
|
|
47
|
+
todo: "A Chrome window opens: sign in, then quit it with ⌘Q.",
|
|
48
|
+
ask: `A Chrome window is open on Google's sign-in page. Sign in, then quit that window with ⌘Q and tell me. (${google.length} ${google.length === 1 ? "directory needs" : "directories need"} it.)`,
|
|
49
|
+
});
|
|
50
|
+
for (const job of blocked.filter((j) => TICK.includes(j.blockType ?? ""))) {
|
|
51
|
+
const dir = await loadDirectory(job.subject).then((d) => d.directory).catch(() => null);
|
|
52
|
+
const name = dir?.name ?? job.subject;
|
|
53
|
+
const url = firstUrl(job.blockMessage) ?? dir?.submission_url ?? dir?.website;
|
|
54
|
+
out.push(ownBrowser(job)
|
|
55
|
+
? { kind: "own_browser", jobs: [job.id], directory: job.subject, name, url, todo: `Do this one in your own browser: ${instruction(job)}.`,
|
|
56
|
+
ask: `${name} breaks when an agent drives the browser, so this one's by hand. I opened it in your browser: ${instruction(job)}. Tell me when it's submitted, with the listing link if there is one.` }
|
|
57
|
+
: { kind: "tick", jobs: [job.id], directory: job.subject, name, url, todo: "Tick the box in the window that opened. No box? Just press enter.",
|
|
58
|
+
ask: `A Chrome window is open on ${name}. Tick the box if you see one (no box is fine too), then tell me "done".` });
|
|
59
|
+
}
|
|
60
|
+
return out;
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
/** Opens the page for a hand-off: plain Chrome for Google, the agents' Chrome for a check, the human's browser otherwise. */
|
|
64
|
+
export function openHandoff(h: Handoff) {
|
|
65
|
+
if (h.kind === "google_login") {
|
|
66
|
+
// Blocks until the human quits that window, so detach: the agent waits for the human's reply in chat instead.
|
|
67
|
+
Bun.spawn([join(PKG, "scripts/google-login")], { env, stdin: "ignore", stdout: "ignore", stderr: "ignore" }).unref();
|
|
68
|
+
return true;
|
|
69
|
+
}
|
|
70
|
+
if (!h.url) return false;
|
|
71
|
+
if (h.kind === "own_browser") return Bun.spawnSync(["open", h.url]).exitCode === 0;
|
|
72
|
+
if (script("ab", "open", h.url) !== 0) return false;
|
|
73
|
+
script("show-browser");
|
|
74
|
+
return true;
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/** For the agent: the next hand-off, opened, with the sentence to relay and (by hand) the facts to paste. */
|
|
78
|
+
export async function handoffNext(db: Db, product?: string) {
|
|
79
|
+
const [h, ...rest] = await handoffQueue(db, product);
|
|
80
|
+
if (!h) return { next: null, note: "nothing waiting on the human" };
|
|
81
|
+
const opened = openHandoff(h);
|
|
82
|
+
const paste = h.kind === "own_browser" ? await pasteFacts(db, h.jobs[0]!) : undefined;
|
|
83
|
+
return {
|
|
84
|
+
...h, opened, ...(paste && { paste }), remaining: rest.length,
|
|
85
|
+
then: h.kind === "own_browser"
|
|
86
|
+
? `when they say it's submitted: dirpush handoff mine ${h.jobs[0]} [--listing-url <url>]`
|
|
87
|
+
: `when they say done: dirpush handoff done ${h.kind === "google_login" ? "google" : h.jobs[0]}, then claim and finish ${h.kind === "google_login" ? "those jobs" : "it now, in the same tab"}`,
|
|
88
|
+
...(!opened && h.url && { note: `couldn't open the page; give the human this link: ${h.url}` }),
|
|
89
|
+
};
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
async function pasteFacts(db: Db, jobId: string) {
|
|
93
|
+
const job = db.select().from(jobs).where(eq(jobs.id, jobId)).get();
|
|
94
|
+
if (!job) return undefined;
|
|
95
|
+
const p = await loadProduct(job.productId).catch(() => null);
|
|
96
|
+
if (!p) return undefined;
|
|
97
|
+
return {
|
|
98
|
+
name: p.name, website: utm(p.url, job.subject), tagline: p.tagline,
|
|
99
|
+
short: p.descriptions?.short, description: p.descriptions?.medium, categories: p.categories,
|
|
100
|
+
logo: p.assets?.icon ? join(p.dir, p.assets.icon) : undefined,
|
|
101
|
+
};
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
/** The human did their part: hand the job(s) back to the agents. `google` hands back every Google sign-in job. */
|
|
105
|
+
export function handoffDone(db: Db, id: string) {
|
|
106
|
+
const ids = id === "google"
|
|
107
|
+
? db.select().from(jobs).where(and(eq(jobs.status, "blocked"), eq(jobs.blockType, "google_login"))).all().map((j) => j.id)
|
|
108
|
+
: [id];
|
|
109
|
+
const note = id === "google" ? "human signed into Google in the agents' Chrome profile" : "human cleared the check in the agents' Chrome; continue in the same tab";
|
|
110
|
+
return { handedBack: ids.map((j) => J.unblock(db, j, note)).map((j) => ({ id: j.id, directory: j.subject })), then: "claim each and finish it now" };
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
/** The human submitted it themselves. */
|
|
114
|
+
export const handoffMine = (db: Db, id: string, listingUrl?: string) => J.recordManual(db, id, "pending", listingUrl);
|
|
115
|
+
|
|
116
|
+
/** Not now: leave it blocked, out of `handoff next` for a while. */
|
|
117
|
+
export function handoffSkip(db: Db, id: string) {
|
|
118
|
+
J.noteEvent(db, id, "handoff_skipped");
|
|
119
|
+
return { skipped: id, until: new Date(Date.now() + SKIP_HOURS * 3600_000).toISOString() };
|
|
120
|
+
}
|
package/src/jobs.ts
CHANGED
|
@@ -16,6 +16,11 @@ function logEvent(tx: Tx | Db, jobId: string | null, type: string, worker: strin
|
|
|
16
16
|
tx.insert(events).values({ jobId, type, worker, data }).run();
|
|
17
17
|
}
|
|
18
18
|
|
|
19
|
+
/** A plain event on a job (no state change), e.g. a hand-off the human skipped. */
|
|
20
|
+
export function noteEvent(db: Db, jobId: string, type: string, data: Record<string, unknown> = {}) {
|
|
21
|
+
logEvent(db, jobId, type, "human", data);
|
|
22
|
+
}
|
|
23
|
+
|
|
19
24
|
function get(tx: Tx, id: string): Job {
|
|
20
25
|
const job = tx.select().from(jobs).where(eq(jobs.id, id)).get();
|
|
21
26
|
if (!job) throw new JobError(`no such job: ${id}`);
|
package/src/walk.ts
ADDED
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* `dirpush handoff` in a terminal: the human walks the hand-off queue themselves (src/handoff.ts), for anyone who
|
|
3
|
+
* prefers that to their agent walking them through it in chat. Same queue, same effect on the jobs.
|
|
4
|
+
*/
|
|
5
|
+
import { createInterface } from "node:readline/promises";
|
|
6
|
+
import type { Db } from "./db";
|
|
7
|
+
import * as J from "./jobs";
|
|
8
|
+
import { loadCapabilities } from "./capabilities";
|
|
9
|
+
import { handoffDone, handoffQueue, handoffSkip, openHandoff, script } from "./handoff";
|
|
10
|
+
|
|
11
|
+
const color = process.stdout.isTTY && !process.env.NO_COLOR;
|
|
12
|
+
const paint = (code: string) => (s: string) => (color ? `\x1b[${code}m${s}\x1b[0m` : s);
|
|
13
|
+
const bold = paint("1"), dim = paint("2"), green = paint("32"), yellow = paint("33"), cyan = paint("36");
|
|
14
|
+
|
|
15
|
+
/** One keypress, no Enter needed. Enter comes back as "\r"; Ctrl-C quits. */
|
|
16
|
+
async function key(prompt: string, keys: string): Promise<string> {
|
|
17
|
+
process.stdout.write(prompt);
|
|
18
|
+
process.stdin.setRawMode(true);
|
|
19
|
+
process.stdin.resume();
|
|
20
|
+
try {
|
|
21
|
+
for (;;) {
|
|
22
|
+
const k = await new Promise<string>((r) => process.stdin.once("data", (b: Buffer) => r(b.toString())));
|
|
23
|
+
if (k === "\x03") { process.stdout.write("\n"); process.exit(130); }
|
|
24
|
+
const ch = k === "\r" || k === "\n" ? "\r" : k.toLowerCase();
|
|
25
|
+
if (keys.includes(ch)) { process.stdout.write(ch === "\r" ? "\n" : `${ch}\n`); return ch; }
|
|
26
|
+
}
|
|
27
|
+
} finally {
|
|
28
|
+
process.stdin.setRawMode(false);
|
|
29
|
+
process.stdin.pause();
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
async function ask(prompt: string) {
|
|
34
|
+
const rl = createInterface({ input: process.stdin, output: process.stdout });
|
|
35
|
+
try { return (await rl.question(prompt)).trim(); } finally { rl.close(); }
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
export async function walk(db: Db, product?: string) {
|
|
39
|
+
if (!process.stdin.isTTY) throw new Error("this is the interactive version: run 'dirpush handoff' in a terminal, or let your agent use 'handoff next'");
|
|
40
|
+
const items = await handoffQueue(db, product, { includeSkipped: true });
|
|
41
|
+
if (!items.length) return console.log("Nothing is waiting on you.");
|
|
42
|
+
console.log(`${bold("dirpush handoff")} ${dim(`· ${items.length} to clear · an agent finishes each one after you`)}`);
|
|
43
|
+
const host = (u: string) => u.replace(/^https?:\/\/(www\.)?/, "").replace(/\/$/, "");
|
|
44
|
+
const keys = (pairs: [string, string][]) => ` ${pairs.map(([k, l]) => `${cyan(k)} ${l}`).join(" ")} `;
|
|
45
|
+
let back = 0, mine = 0, skipped = 0;
|
|
46
|
+
|
|
47
|
+
for (const [i, h] of items.entries()) {
|
|
48
|
+
const where = h.kind === "google_login" ? `${h.jobs.length} ${h.jobs.length === 1 ? "directory" : "directories"}` : h.url ? host(h.url) : "";
|
|
49
|
+
console.log(`\n${dim(`${i + 1}/${items.length}`)} ${bold(h.name)} ${dim(where)}\n ${h.todo}`);
|
|
50
|
+
if (h.kind === "google_login") {
|
|
51
|
+
const k = await key(keys([["enter", "sign in"], ["n", "later"], ["q", "quit"]]), "\rnq");
|
|
52
|
+
if (k === "q") break;
|
|
53
|
+
if (k === "n") { skipped++; continue; }
|
|
54
|
+
console.log(dim(" Waiting for you to quit that Chrome window…"));
|
|
55
|
+
script("google-login"); // returns when that Chrome quits
|
|
56
|
+
if ((await key(` Signed in? ${cyan("y")} yes ${cyan("n")} no `, "yn")) === "y") { back += handoffDone(db, "google").handedBack.length; } else skipped++;
|
|
57
|
+
continue;
|
|
58
|
+
}
|
|
59
|
+
if (!openHandoff(h) && h.url) console.log(yellow(` Couldn't open it; go to ${h.url}`));
|
|
60
|
+
const byHand = h.kind === "own_browser";
|
|
61
|
+
const k = await key(keys(byHand ? [["s", "done"], ["n", "skip"], ["q", "quit"]] : [["enter", "done"], ["s", "I submitted it myself"], ["n", "skip"], ["q", "quit"]]), byHand ? "snq" : "\rsnq");
|
|
62
|
+
if (k === "q") break;
|
|
63
|
+
const id = h.jobs[0]!;
|
|
64
|
+
if (k === "n") { handoffSkip(db, id); skipped++; }
|
|
65
|
+
else if (k === "\r") { handoffDone(db, id); back++; }
|
|
66
|
+
else { const link = await ask(` Listing link ${dim("(enter to skip)")}: `); J.recordManual(db, id, "pending", link || undefined); mine++; }
|
|
67
|
+
}
|
|
68
|
+
const parts = [back && `${back} back to the agents`, mine && `${mine} done by you`, skipped && `${skipped} skipped`].filter(Boolean);
|
|
69
|
+
console.log(`\n${green("✓")} ${bold("Done.")} ${parts.join(" · ") || "Nothing changed."}`);
|
|
70
|
+
if (back) await startAgent(back);
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
/** The agent CLIs we can open on "continue my launch", most likely first: the one that recorded capabilities, then any on PATH. */
|
|
74
|
+
async function startAgent(n: number) {
|
|
75
|
+
const caps = await loadCapabilities().catch(() => null);
|
|
76
|
+
const known = [
|
|
77
|
+
{ id: "claude-code", name: "Claude Code", bin: "claude" },
|
|
78
|
+
{ id: "codex", name: "Codex", bin: "codex" },
|
|
79
|
+
].filter((a) => Bun.which(a.bin));
|
|
80
|
+
const agent = known.find((a) => caps?.checked_by?.startsWith(a.id)) ?? known[0];
|
|
81
|
+
const prompt = "continue my launch";
|
|
82
|
+
if (!agent || process.platform !== "darwin") {
|
|
83
|
+
console.log(`Next: tell your agent ${bold(`“${prompt}”`)} to finish them.`);
|
|
84
|
+
return;
|
|
85
|
+
}
|
|
86
|
+
const k = await key(` ${cyan("enter")} open ${agent.name} to finish them ${cyan("n")} later `, "\rn");
|
|
87
|
+
if (k === "n") return console.log(dim(` Later: tell your agent “${prompt}”.`));
|
|
88
|
+
// A new Terminal window, so the human watches it work and approves what it asks for. Nothing runs unattended.
|
|
89
|
+
const cmd = `cd ${JSON.stringify(process.cwd())} && ${agent.bin} ${JSON.stringify(prompt)}`;
|
|
90
|
+
const ok = Bun.spawnSync(["osascript", "-e", `tell application "Terminal" to do script ${JSON.stringify(cmd)}`, "-e", `tell application "Terminal" to activate`], { stdin: "ignore", stdout: "ignore", stderr: "ignore" }).exitCode === 0;
|
|
91
|
+
console.log(ok ? `${green("✓")} Opened ${agent.name} in a new Terminal window.` : `Couldn't open Terminal. Run this yourself: ${cmd}`);
|
|
92
|
+
}
|
package/src/checks.ts
DELETED
|
@@ -1,153 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* `dirpush checks`: the human's part of a launch, in the terminal. Google sign-in first, in plain Chrome with no
|
|
3
|
-
* automation attached (Google refuses sign-in in a window software is driving); then each CAPTCHA or bot check,
|
|
4
|
-
* opened in the agents' Chrome. Agents never solve these: this hands each job back once the human has.
|
|
5
|
-
*/
|
|
6
|
-
import { createInterface } from "node:readline/promises";
|
|
7
|
-
import { join } from "node:path";
|
|
8
|
-
import { and, eq, gt, inArray } from "drizzle-orm";
|
|
9
|
-
import { HOME, PKG, type Db } from "./db";
|
|
10
|
-
import { jobs } from "./db/schema";
|
|
11
|
-
import * as J from "./jobs";
|
|
12
|
-
import { loadCapabilities } from "./capabilities";
|
|
13
|
-
import { loadDirectory } from "./registry";
|
|
14
|
-
|
|
15
|
-
const color = process.stdout.isTTY && !process.env.NO_COLOR;
|
|
16
|
-
const paint = (code: string) => (s: string) => (color ? `\x1b[${code}m${s}\x1b[0m` : s);
|
|
17
|
-
const bold = paint("1"), dim = paint("2"), green = paint("32"), yellow = paint("33"), cyan = paint("36");
|
|
18
|
-
const env = { ...process.env, LAUNCH_HOME: HOME };
|
|
19
|
-
const script = (name: string, ...args: string[]) =>
|
|
20
|
-
Bun.spawnSync([join(PKG, "scripts", name), ...args], { env, stdin: "ignore", stdout: "ignore", stderr: "ignore" }).exitCode;
|
|
21
|
-
|
|
22
|
-
/** One keypress, no Enter needed. Enter comes back as "\r"; Ctrl-C quits. */
|
|
23
|
-
async function key(prompt: string, keys: string): Promise<string> {
|
|
24
|
-
process.stdout.write(prompt);
|
|
25
|
-
process.stdin.setRawMode(true);
|
|
26
|
-
process.stdin.resume();
|
|
27
|
-
try {
|
|
28
|
-
for (;;) {
|
|
29
|
-
const k = await new Promise<string>((r) => process.stdin.once("data", (b: Buffer) => r(b.toString())));
|
|
30
|
-
if (k === "\x03") { process.stdout.write("\n"); process.exit(130); }
|
|
31
|
-
const ch = k === "\r" || k === "\n" ? "\r" : k.toLowerCase();
|
|
32
|
-
if (keys.includes(ch)) { process.stdout.write(ch === "\r" ? "\n" : `${ch}\n`); return ch; }
|
|
33
|
-
}
|
|
34
|
-
} finally {
|
|
35
|
-
process.stdin.setRawMode(false);
|
|
36
|
-
process.stdin.pause();
|
|
37
|
-
}
|
|
38
|
-
}
|
|
39
|
-
|
|
40
|
-
async function ask(prompt: string) {
|
|
41
|
-
const rl = createInterface({ input: process.stdin, output: process.stdout });
|
|
42
|
-
try { return (await rl.question(prompt)).trim(); } finally { rl.close(); }
|
|
43
|
-
}
|
|
44
|
-
|
|
45
|
-
const WHAT: Record<string, string> = {
|
|
46
|
-
captcha: "Tick the CAPTCHA, then do whatever the note above asks.",
|
|
47
|
-
bot_protection: "Wait for the “checking your browser” page to pass, or tick its box if it shows one.",
|
|
48
|
-
};
|
|
49
|
-
|
|
50
|
-
export async function checks(db: Db, opts: { product?: string; types: string[] }) {
|
|
51
|
-
if (!process.stdin.isTTY) throw new Error("checks is interactive: run `dirpush checks` yourself in a terminal");
|
|
52
|
-
const blocked = (types: string[]) => db.select().from(jobs)
|
|
53
|
-
.where(and(eq(jobs.kind, "submission"), eq(jobs.status, "blocked"), inArray(jobs.blockType, types))).all()
|
|
54
|
-
.filter((j) => !opts.product || j.productId === opts.product);
|
|
55
|
-
const google = blocked(["google_login"]);
|
|
56
|
-
const queue = blocked(opts.types);
|
|
57
|
-
const handedBack: string[] = [], byYou: string[] = [], skipped: string[] = [];
|
|
58
|
-
let signedIn = false;
|
|
59
|
-
|
|
60
|
-
const summary = async () => {
|
|
61
|
-
console.log(`\n${bold("Done for now.")}`);
|
|
62
|
-
if (handedBack.length) console.log(`${green("✓")} Ready for the agents to finish: ${handedBack.join(", ")}`);
|
|
63
|
-
if (byYou.length) console.log(`${green("✓")} Submitted by you: ${byYou.join(", ")}`);
|
|
64
|
-
if (skipped.length) console.log(`${dim("·")} Skipped, still waiting: ${skipped.join(", ")}`);
|
|
65
|
-
if (handedBack.length) await startAgent(handedBack.length);
|
|
66
|
-
};
|
|
67
|
-
|
|
68
|
-
const googleSignIn = async () => {
|
|
69
|
-
const busy = db.select().from(jobs).where(and(eq(jobs.status, "running"), gt(jobs.leaseExpiresAt, Date.now()))).all();
|
|
70
|
-
if (busy.length) {
|
|
71
|
-
console.log(yellow(`An agent is using the browser right now (${busy.map((j) => j.subject).join(", ")}). Signing in closes that window.`));
|
|
72
|
-
if ((await key(`${cyan("[enter]")} close it and sign in ${cyan("[n]")} not now `, "\rn")) === "n") return false;
|
|
73
|
-
}
|
|
74
|
-
console.log(`A normal Chrome window opens on Google's sign-in page (the agents' own profile, nothing automated).`);
|
|
75
|
-
console.log(`Sign in, then ${bold("quit that window with ⌘Q")} to come back here.`);
|
|
76
|
-
script("google-login"); // returns when that Chrome quits
|
|
77
|
-
const ok = (await key(`Signed in? ${cyan("[y]")} yes ${cyan("[n]")} no `, "yn")) === "y";
|
|
78
|
-
if (ok) signedIn = true;
|
|
79
|
-
else console.log(dim("No problem: run `dirpush checks` again whenever you want to retry."));
|
|
80
|
-
return ok;
|
|
81
|
-
};
|
|
82
|
-
|
|
83
|
-
console.log(`${bold("dirpush checks")} ${dim(`${google.length} need Google sign-in · ${queue.length} CAPTCHAs or bot checks`)}`);
|
|
84
|
-
if (!google.length && !queue.length) return console.log("Nothing is waiting on you.");
|
|
85
|
-
console.log(dim("Agents never solve these. You clear each one, then an agent finishes the submission (at the end I can start yours)."));
|
|
86
|
-
|
|
87
|
-
if (google.length) {
|
|
88
|
-
console.log(`\n${bold("1. Sign into Google")} ${dim(google.map((j) => j.subject).join(", "))}`);
|
|
89
|
-
console.log("These directories use “Sign in with Google”. Sign in once and the agents can use it from then on.");
|
|
90
|
-
const k = await key(`${cyan("[enter]")} sign in now ${cyan("[n]")} later ${cyan("[q]")} quit `, "\rnq");
|
|
91
|
-
if (k === "q") return await summary();
|
|
92
|
-
if (k === "\r" && (await googleSignIn())) {
|
|
93
|
-
for (const j of google) { J.unblock(db, j.id, "human signed into Google in the agents' Chrome profile"); handedBack.push(j.subject); }
|
|
94
|
-
} else skipped.push(...google.map((j) => j.subject));
|
|
95
|
-
}
|
|
96
|
-
|
|
97
|
-
for (const [i, job] of queue.entries()) {
|
|
98
|
-
const dir = await loadDirectory(job.subject).then((d) => d.directory).catch(() => null);
|
|
99
|
-
const url = job.blockMessage?.match(/https?:\/\/[^\s)'"”]+/)?.[0]?.replace(/[.,;:]$/, "") ?? dir?.submission_url ?? dir?.website;
|
|
100
|
-
const open = () => {
|
|
101
|
-
if (!url) return console.log(yellow("No page saved for this one; open the directory yourself."));
|
|
102
|
-
if (script("ab", "open", url) === 0) script("show-browser");
|
|
103
|
-
else console.log(yellow(`Couldn't open the agents' Chrome. Open it yourself: ${url}`));
|
|
104
|
-
};
|
|
105
|
-
console.log(`\n${bold(`${google.length ? i + 2 : i + 1}. ${dir?.name ?? job.subject}`)} ${dim(`${i + 1} of ${queue.length}`)}`);
|
|
106
|
-
if (job.blockMessage) console.log(dim(job.blockMessage));
|
|
107
|
-
if (url) console.log(`${dim("Page:")} ${url}`);
|
|
108
|
-
console.log(WHAT[job.blockType ?? ""] ?? "Do what the note above asks.");
|
|
109
|
-
console.log(dim("Asked to sign in with Google? Press g: Google refuses sign-in in the agents' window."));
|
|
110
|
-
console.log(dim("Check never passes there? Submit it from your own browser instead, then press s."));
|
|
111
|
-
open();
|
|
112
|
-
for (;;) {
|
|
113
|
-
const k = await key(`${cyan("enter")} cleared, agent finishes it ${cyan("s")} I submitted it ${cyan("o")} reopen ${cyan("g")} Google sign-in ${cyan("n")} skip ${cyan("q")} quit › `, "\rsognq");
|
|
114
|
-
if (k === "o") { open(); continue; }
|
|
115
|
-
if (k === "g") { if (!signedIn) await googleSignIn(); open(); continue; }
|
|
116
|
-
if (k === "q") return await summary();
|
|
117
|
-
if (k === "n") skipped.push(job.subject);
|
|
118
|
-
else if (k === "\r") {
|
|
119
|
-
J.unblock(db, job.id, "human cleared the check in the agents' Chrome; continue in the same browser session");
|
|
120
|
-
handedBack.push(job.subject);
|
|
121
|
-
} else {
|
|
122
|
-
const link = await ask(`Listing URL, if there is one ${dim("(enter to skip)")}: `);
|
|
123
|
-
J.recordManual(db, job.id, "pending", link || undefined);
|
|
124
|
-
byYou.push(job.subject);
|
|
125
|
-
}
|
|
126
|
-
break;
|
|
127
|
-
}
|
|
128
|
-
}
|
|
129
|
-
await summary();
|
|
130
|
-
}
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
/** The agent CLIs we can open on "continue my launch", most likely first: the one that recorded capabilities, then any on PATH. */
|
|
134
|
-
async function startAgent(n: number) {
|
|
135
|
-
const caps = await loadCapabilities().catch(() => null);
|
|
136
|
-
const known = [
|
|
137
|
-
{ id: "claude-code", name: "Claude Code", bin: "claude" },
|
|
138
|
-
{ id: "codex", name: "Codex", bin: "codex" },
|
|
139
|
-
].filter((a) => Bun.which(a.bin));
|
|
140
|
-
const agent = known.find((a) => caps?.checked_by?.startsWith(a.id)) ?? known[0];
|
|
141
|
-
const prompt = "continue my launch";
|
|
142
|
-
if (!agent || process.platform !== "darwin") {
|
|
143
|
-
console.log(`\nNext: open your agent and say ${bold(`“${prompt}”`)}; it finishes the ${n} job(s) above.`);
|
|
144
|
-
return;
|
|
145
|
-
}
|
|
146
|
-
console.log(`\nThe ${n} job(s) above are waiting for an agent to finish them.`);
|
|
147
|
-
const k = await key(`${cyan("[enter]")} open ${agent.name} on “${prompt}” now ${cyan("[n]")} I'll do it later `, "\rn");
|
|
148
|
-
if (k === "n") return console.log(dim(`Whenever you're ready: open your agent and say “${prompt}”.`));
|
|
149
|
-
// A new Terminal window, so the human watches it work and approves what it asks for. Nothing runs unattended.
|
|
150
|
-
const cmd = `cd ${JSON.stringify(process.cwd())} && ${agent.bin} ${JSON.stringify(prompt)}`;
|
|
151
|
-
const ok = Bun.spawnSync(["osascript", "-e", `tell application "Terminal" to do script ${JSON.stringify(cmd)}`, "-e", `tell application "Terminal" to activate`], { stdin: "ignore", stdout: "ignore", stderr: "ignore" }).exitCode === 0;
|
|
152
|
-
console.log(ok ? `${green("✓")} Opened ${agent.name} in a new Terminal window.` : `Couldn't open Terminal. Run this yourself: ${cmd}`);
|
|
153
|
-
}
|