gemcatch 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +110 -1
- package/README.md +137 -7
- package/db.js +60 -3
- package/gemini.js +125 -10
- package/index.js +576 -56
- package/package.json +7 -2
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,113 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.5.0] - 2026-09-04
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- **Collaborative planning.** Until now the only agent flow was a blind one-shot
|
|
15
|
+
bet: gemcatch showed a cost band, asked y/N, and fired. The cost band was
|
|
16
|
+
everything you knew before spending, and there was no way to check the agent
|
|
17
|
+
had read the prompt the way you meant it. A 20-line `batch` against
|
|
18
|
+
`deep-research-max` is a $60–$140 command placed sight unseen. The Deep
|
|
19
|
+
Research docs have always documented a second mode for exactly this, and
|
|
20
|
+
gemcatch could not reach it. Now it can:
|
|
21
|
+
|
|
22
|
+
- `--plan` on `research` and `batch` submits with
|
|
23
|
+
`agent_config.collaborative_planning: true`, so **the agent returns a
|
|
24
|
+
research plan instead of a report**. Still `background: true`, still stored,
|
|
25
|
+
still collected by the daemon. The row is stored with `kind='plan'`.
|
|
26
|
+
- `gemcatch get <id>` on a plan prints the plan and then the literal next
|
|
27
|
+
command, `Approve with: gemcatch approve 8f3a1c04 · Refine with: gemcatch
|
|
28
|
+
refine 8f3a1c04 "..."`, on stderr, so `get <plan> > plan.md` still captures
|
|
29
|
+
just the plan.
|
|
30
|
+
- `gemcatch refine <id> "<instruction>"` sends the instruction back with
|
|
31
|
+
`previous_interaction_id` and planning still on, and stores the revised plan
|
|
32
|
+
linked to the one it came from, inheriting its agent and tag.
|
|
33
|
+
- `gemcatch approve <id>` is the turn that commits: `previous_interaction_id`
|
|
34
|
+
with `collaborative_planning: false`, stored as `kind='report'` linked to the
|
|
35
|
+
plan. It shows the cost band and asks, honours `--yes` when stdin is not a
|
|
36
|
+
TTY, and honours `--dry-run`.
|
|
37
|
+
|
|
38
|
+
**Planning is not cheaper.** The docs give one band per task and do not price a
|
|
39
|
+
planning turn separately, so a planning turn is quoted at the same band and the
|
|
40
|
+
line says so: *"the docs price per task and do not price a planning turn
|
|
41
|
+
separately"*. What you get for it is a look at the plan before you commit to
|
|
42
|
+
the research run, not a discount.
|
|
43
|
+
- `gemcatch list` renders a chain indented under its root, in submission order,
|
|
44
|
+
and grows a `KIND` column when a listing contains plans or reports (a store
|
|
45
|
+
with neither keeps the layout it had). `gemcatch export` follows a chain to its
|
|
46
|
+
report and leaves the intermediate plans out unless `--include-plans` is
|
|
47
|
+
passed; the Markdown sections and the JSON rows now name the turn.
|
|
48
|
+
- `gemcatch stats` now tallies plan and report turns, and totals the estimated
|
|
49
|
+
spend across the agent turns that were actually billed, from the same
|
|
50
|
+
published bands the guard quotes before each one. A chain bills per turn, so a
|
|
51
|
+
release that turns one submission into three owes the user a running total. It
|
|
52
|
+
is the documented bands applied to what was submitted, not a reading of your
|
|
53
|
+
bill, and it errs toward telling you rather than flattering you: only runs
|
|
54
|
+
that reached the server are priced (a submit that failed before it left the
|
|
55
|
+
machine is counted as an attempt but costs nothing), and an agent with no
|
|
56
|
+
published band totals to `unknown`, never to `$0.00`. A store with no agent
|
|
57
|
+
runs prints neither line.
|
|
58
|
+
- Additive schema migration: `collaborative_planning`, `previous_interaction_id`,
|
|
59
|
+
`kind` (defaulting to `'task'`) and `parent_id`. A 0.4.0 `tasks.db` upgrades in
|
|
60
|
+
place, keeps every row, and behaves exactly as it did.
|
|
61
|
+
|
|
62
|
+
### Fixed
|
|
63
|
+
|
|
64
|
+
- `approve` and `refine` fail before sending anything when the target is not a
|
|
65
|
+
plan, or is a plan that has not completed. A plan whose interaction the free
|
|
66
|
+
tier has already dropped names the retention window (1 day free, 55 days paid)
|
|
67
|
+
instead of sending a `previous_interaction_id` the server would reject. A plan
|
|
68
|
+
that expires *after* it completed locally gets the same explanation rather
|
|
69
|
+
than a bare 404 for an id the user never typed.
|
|
70
|
+
- `--plan` without `--agent` (or with `--model`) is a clean error rather than a
|
|
71
|
+
flag that quietly does nothing: collaborative planning is an agent feature.
|
|
72
|
+
- Approving the same plan twice submits a second run rather than silently
|
|
73
|
+
reusing the first, and `list` shows both under their plan.
|
|
74
|
+
|
|
75
|
+
## [0.4.0] - 2026-08-08
|
|
76
|
+
|
|
77
|
+
### Added
|
|
78
|
+
|
|
79
|
+
- **Research agents.** `-a, --agent <id>` on `research` and `batch` submits to a
|
|
80
|
+
Gemini Deep Research agent instead of a model — `interactions.create` is sent
|
|
81
|
+
`agent` *instead of* `model` (they are mutually exclusive, and passing both is
|
|
82
|
+
a clean error). Aliases resolve through one table: `deep-research` →
|
|
83
|
+
`deep-research-preview-04-2026`, `deep-research-max` →
|
|
84
|
+
`deep-research-max-preview-04-2026`; any other value passes through unchanged,
|
|
85
|
+
so a future agent id works without a gemcatch release. Agents *require*
|
|
86
|
+
background execution, which gemcatch has always set — and on the free tier the
|
|
87
|
+
finished report is dropped after 1 day, which is exactly the race the daemon
|
|
88
|
+
exists to win. The agent is recorded per task, shown in `list` (the AGENT
|
|
89
|
+
column appears when a listing contains agent runs) and tallied in `stats`.
|
|
90
|
+
- **Spend guard.** Deep Research is documented at $1.00–$3.00 per task and Deep
|
|
91
|
+
Research Max at $3.00–$7.00 (estimates based on preview rates, per the docs,
|
|
92
|
+
and subject to change). Every agent submission prints its band first —
|
|
93
|
+
`batch` prints N × the band as a total — and asks for an interactive `y/N`
|
|
94
|
+
confirmation. When stdin is not a TTY, `--yes` is required and anything else
|
|
95
|
+
is refused before a row is written; declining writes nothing and exits
|
|
96
|
+
non-zero. `--dry-run` (now on `research` too) prints the full projected spend
|
|
97
|
+
and submits nothing.
|
|
98
|
+
- **Citations.** Agent runs return citations alongside the report; the docs say
|
|
99
|
+
to review them to verify the sources, so they are persisted (new `citations`
|
|
100
|
+
column, JSON) rather than discarded, printed under the result as a `Sources:`
|
|
101
|
+
list, and carried in `--json` output.
|
|
102
|
+
- Result extraction now takes the **final answer-bearing step** — where the
|
|
103
|
+
docs place an agent's completed report (`steps[-1].content[0].text`) and
|
|
104
|
+
where a model run's `model_output` already sits — with a fall-back to the old
|
|
105
|
+
collect-everything behaviour if that step carries no text, so an unexpected
|
|
106
|
+
shape can never silently blank a result. No special-casing on the agent id.
|
|
107
|
+
- Additive schema migration: `agent` and `citations` columns. A pre-0.4.0
|
|
108
|
+
`tasks.db` upgrades in place, keeps every row, and reports `agent` as NULL
|
|
109
|
+
for them.
|
|
110
|
+
|
|
111
|
+
### Changed
|
|
112
|
+
|
|
113
|
+
- The default model is now **`gemini-3.5-flash-lite`** (GA on 2026-07-21),
|
|
114
|
+
replacing the older `gemini-3.1-flash-lite`. Override with `GEMCATCH_MODEL`
|
|
115
|
+
or `--model` as before.
|
|
116
|
+
|
|
10
117
|
## [0.3.0] - 2026-07-19
|
|
11
118
|
|
|
12
119
|
### Added
|
|
@@ -162,7 +269,9 @@ seen a task complete, the text is cached locally and survives that expiry — bu
|
|
|
162
269
|
something has to poll inside that window for it to be seen at all, which is what
|
|
163
270
|
`gemcatch daemon` exists to do.
|
|
164
271
|
|
|
165
|
-
[Unreleased]: https://github.com/Booyaka101/gemcatch/compare/v0.
|
|
272
|
+
[Unreleased]: https://github.com/Booyaka101/gemcatch/compare/v0.5.0...HEAD
|
|
273
|
+
[0.5.0]: https://github.com/Booyaka101/gemcatch/compare/v0.4.0...v0.5.0
|
|
274
|
+
[0.4.0]: https://github.com/Booyaka101/gemcatch/compare/v0.3.0...v0.4.0
|
|
166
275
|
[0.3.0]: https://github.com/Booyaka101/gemcatch/compare/v0.2.0...v0.3.0
|
|
167
276
|
[0.2.0]: https://github.com/Booyaka101/gemcatch/compare/v0.1.1...v0.2.0
|
|
168
277
|
[0.1.1]: https://github.com/Booyaka101/gemcatch/compare/v0.1.0...v0.1.1
|
package/README.md
CHANGED
|
@@ -24,7 +24,7 @@ The EU AI Act's high-risk obligations phase in from August 2026, whereas...
|
|
|
24
24
|
|
|
25
25
|
## Setup
|
|
26
26
|
|
|
27
|
-
Needs Node.js 22+ and a Gemini API key. **Getting a key needs no billing account and no card.** `gemini-3.
|
|
27
|
+
Needs Node.js 22+ and a Gemini API key. **Getting a key needs no billing account and no card.** `gemini-3.5-flash-lite` (the default model, GA since July 2026) runs free within the [free tier's](https://ai.google.dev/gemini-api/docs/pricing) daily quota; past that, paid rates apply.
|
|
28
28
|
|
|
29
29
|
1. Get a key at **<https://aistudio.google.com/apikey>**
|
|
30
30
|
2. Put it in your environment:
|
|
@@ -81,8 +81,10 @@ This week in AI: ...
|
|
|
81
81
|
| `gemcatch research "<prompt>"` | Submits with `background: true`, stores the interaction ID, exits immediately. |
|
|
82
82
|
| `gemcatch batch <file>` | Submits many prompts from a file at once, tagged as one collectable batch. |
|
|
83
83
|
| `gemcatch status <id>` | Polls the API and prints the current state. |
|
|
84
|
-
| `gemcatch get <id>` | Prints the full response if complete, otherwise the current status. |
|
|
85
|
-
| `gemcatch
|
|
84
|
+
| `gemcatch get <id>` | Prints the full response if complete, otherwise the current status. On a [plan](#see-the-plan-before-you-pay-for-the-run), the plan plus the approve/refine commands. |
|
|
85
|
+
| `gemcatch refine <id> "<instruction>"` | Sends an instruction back to a plan and returns a revised plan. |
|
|
86
|
+
| `gemcatch approve <id>` | Approves a plan and submits the research run it describes. |
|
|
87
|
+
| `gemcatch list` | All tasks, newest first: id, age, status, prompt. Plan chains are indented under their root. |
|
|
86
88
|
| `gemcatch export` | Concatenates finished results, each under its prompt, to stdout or a file (Markdown or JSON). |
|
|
87
89
|
| `gemcatch digest` | Feeds a tag's completed results through one Gemini call into a single summary. |
|
|
88
90
|
| `gemcatch watch <id>` | Polls until the task finishes, then prints the result. |
|
|
@@ -91,7 +93,7 @@ This week in AI: ...
|
|
|
91
93
|
| `gemcatch cancel <id>` | Asks the API to stop an in-flight task. |
|
|
92
94
|
| `gemcatch rm <ids...>` | Forgets tasks locally. `--remote` deletes them server-side too. |
|
|
93
95
|
| `gemcatch prune` | Drops finished tasks older than `--days` (default 30). |
|
|
94
|
-
| `gemcatch stats` | Where the store lives
|
|
96
|
+
| `gemcatch stats` | Where the store lives, what's in it, and what the agent runs have plausibly cost. |
|
|
95
97
|
|
|
96
98
|
Useful flags:
|
|
97
99
|
|
|
@@ -99,6 +101,9 @@ Useful flags:
|
|
|
99
101
|
| --- | --- | --- |
|
|
100
102
|
| `--json` | most commands | Machine-readable output. |
|
|
101
103
|
| `-m, --model <id>` | `research`, `batch` | Override the model. |
|
|
104
|
+
| `-a, --agent <id>` | `research`, `batch` | Submit to a [research agent](#research-agents) instead of a model. Mutually exclusive with `--model`. |
|
|
105
|
+
| `--plan` | `research`, `batch` | Ask the agent for a [research plan](#see-the-plan-before-you-pay-for-the-run) first, to refine and approve. Needs `--agent`. |
|
|
106
|
+
| `--yes` | `research`, `batch`, `refine`, `approve` | Confirm the agent cost without asking. Required for `--agent` when stdin is not a TTY. |
|
|
102
107
|
| `-s, --system <text>` | `research`, `batch` | Set a system instruction. |
|
|
103
108
|
| `-f, --file <path>` | `research` | Read the prompt from a file. |
|
|
104
109
|
| `-t, --tag <tag>` | `research`, `batch`, `list` | Label tasks and filter them. |
|
|
@@ -110,7 +115,8 @@ Useful flags:
|
|
|
110
115
|
| `-n, --limit <n>` | `list` | Cap the rows (non-negative; `0` shows none). |
|
|
111
116
|
| `--format <md\|json>` | `export` | Output format. Default `md`. |
|
|
112
117
|
| `-o, --out <file>` | `export` | Write to a file instead of stdout. |
|
|
113
|
-
| `--
|
|
118
|
+
| `--include-plans` | `export` | Also emit a chain's plan turns, not just its report. |
|
|
119
|
+
| `--dry-run` | `research`, `batch`, `refine`, `approve`, `prune` | Show what would go — including the projected agent spend; submit/delete nothing. |
|
|
114
120
|
| `--raw` | `get` | Dump the raw interaction JSON. |
|
|
115
121
|
|
|
116
122
|
IDs are the first 8 characters of a UUID. Any unique prefix works, so `gemcatch get 8f3a` is fine.
|
|
@@ -153,6 +159,129 @@ $ id=$(gemcatch research "..." --json | jq -r .id)
|
|
|
153
159
|
$ gemcatch watch "$id" --json | jq -r .result
|
|
154
160
|
```
|
|
155
161
|
|
|
162
|
+
## Research agents
|
|
163
|
+
|
|
164
|
+
The [Gemini Deep Research agents](https://ai.google.dev/gemini-api/docs/deep-research) are reachable only through the Interactions API, and the docs are explicit: *"You must use background execution (set `background=true`) to run the agent asynchronously and poll for results or stream updates."* That is precisely the half of the job `gemcatch` already does — it always sets `background: true`, owns the polling, and its daemon collects results before the free tier drops interactions after **1 day** (paid tier: 55 days). A Deep Research run takes minutes and you were never going to sit there holding the connection; submit it, and let the daemon catch it.
|
|
165
|
+
|
|
166
|
+
```console
|
|
167
|
+
$ gemcatch research "map the EU AI Act high-risk obligations against the UK approach" --agent deep-research
|
|
168
|
+
Agent deep-research-preview-04-2026 — estimated $1.00–$3.00 for this task (preview rates, subject to change).
|
|
169
|
+
Submit? [y/N] y
|
|
170
|
+
Task 8f3a1c04 submitted. Run: gemcatch get 8f3a1c04 when ready.
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
`--agent` takes an alias or a raw agent id:
|
|
174
|
+
|
|
175
|
+
| You type | Sent to the API |
|
|
176
|
+
| --- | --- |
|
|
177
|
+
| `deep-research` | `deep-research-preview-04-2026` |
|
|
178
|
+
| `deep-research-max` | `deep-research-max-preview-04-2026` |
|
|
179
|
+
| anything else | passed through unchanged (future agent ids work without a gemcatch release; a bad id fails fast with the API's own 4xx) |
|
|
180
|
+
|
|
181
|
+
An agent is sent **instead of** a model — the agent picks its own models — so `--model` and `--agent` together is an error, and nothing is submitted.
|
|
182
|
+
|
|
183
|
+
**These agents cost real money, per task.** The docs put Deep Research at **$1.00–$3.00 per task** and Deep Research Max at **$3.00–$7.00 per task** — with their own hedge attached: *"These figures are estimates based on preview rates and are subject to change."* Because `gemcatch batch` fires a whole file at once, a 20-line file against `deep-research-max` is a **$60–$140 command**, so every agent submission shows its band and asks first. In a script (stdin not a TTY) you must pass `--yes`; `--dry-run` prints the full projected spend and submits nothing:
|
|
184
|
+
|
|
185
|
+
```console
|
|
186
|
+
$ gemcatch batch questions.txt --agent deep-research-max --dry-run
|
|
187
|
+
20 prompts × deep-research-max-preview-04-2026 — estimated $60.00–$140.00 total. Nothing submitted (--dry-run).
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
The report lands like any other result — final answer only, none of the agent's interim plan — and its **citations** come with it. The docs tell you to review them to verify the sources, so `gemcatch get` prints them under the report as a `Sources:` list, `--json` carries them as an array, and they live in the store alongside the result.
|
|
191
|
+
|
|
192
|
+
An agent run can also come back `incomplete` — that is what a `max_total_tokens` budget cap produces when the run "safely pauses" — which `gemcatch` treats as terminal, exactly like the API does: the daemon retires it and moves on.
|
|
193
|
+
|
|
194
|
+
### See the plan before you pay for the run
|
|
195
|
+
|
|
196
|
+
A cost band tells you what a run will cost. It tells you nothing about whether the agent understood the question. Add `--plan` and it doesn't start researching: with `agent_config.collaborative_planning: true`, *"the agent returns a research plan instead of a full report"*. You read it, push back on it, and approve it when it's right.
|
|
197
|
+
|
|
198
|
+
```console
|
|
199
|
+
$ gemcatch research "map the EU AI Act high-risk obligations against the UK approach" --agent deep-research --plan -t euuk
|
|
200
|
+
Agent deep-research-preview-04-2026 (planning turn) — estimated $1.00–$3.00 for this task (preview rates, subject to change; the docs price per task and do not price a planning turn separately).
|
|
201
|
+
Submit? [y/N] y
|
|
202
|
+
Plan task d014e21b submitted. Run: gemcatch get d014e21b when ready.
|
|
203
|
+
|
|
204
|
+
$ gemcatch get d014e21b
|
|
205
|
+
Research plan
|
|
206
|
+
|
|
207
|
+
1. Scope the EU AI Act high-risk regime: Annex III use cases, Article 6 classification,
|
|
208
|
+
and the Chapter III obligations (risk management, data governance, logging, human
|
|
209
|
+
oversight, conformity assessment) with their August 2026 / August 2027 dates.
|
|
210
|
+
2. Scope the UK approach: the five cross-sector principles, the regulator-led model
|
|
211
|
+
(ICO, FCA, MHRA, Ofcom), and what is guidance rather than statute.
|
|
212
|
+
3. Build an obligation-by-obligation comparison table: EU requirement, nearest UK
|
|
213
|
+
equivalent, whether it is binding, and who enforces it.
|
|
214
|
+
4. Flag the gaps in both directions and the compliance implications for a firm
|
|
215
|
+
operating in both.
|
|
216
|
+
Approve with: gemcatch approve d014e21b · Refine with: gemcatch refine d014e21b "..."
|
|
217
|
+
|
|
218
|
+
$ gemcatch refine d014e21b "cut the history, and add enforcement penalties on both sides"
|
|
219
|
+
Plan task 2140e699 submitted (refines d014e21b). Run: gemcatch get 2140e699 when ready.
|
|
220
|
+
|
|
221
|
+
$ gemcatch approve 2140e699
|
|
222
|
+
Agent deep-research-preview-04-2026 — estimated $1.00–$3.00 for this task (preview rates, subject to change).
|
|
223
|
+
Submit? [y/N] y
|
|
224
|
+
Task 96e8209b submitted (approves plan 2140e699).
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Every turn is an ordinary background task: stored, polled, and collected by the daemon before the 1-day expiry, same as everything else. `refine` chains as many times as you like; each revision inherits the agent and tag of the plan it came from.
|
|
228
|
+
|
|
229
|
+
**A planning turn is a task, and it is billed as one.** The docs publish one band per task and price no planning turn separately, so `gemcatch` quotes the same band for it and says exactly that on the line. What `--plan` buys you is a look at the plan before you commit to the research run, not a discount. Budget for the plan, each refinement, and the run.
|
|
230
|
+
|
|
231
|
+
Since a chain bills per turn, `gemcatch stats` keeps a running total across all of them:
|
|
232
|
+
|
|
233
|
+
```console
|
|
234
|
+
$ gemcatch stats
|
|
235
|
+
Store: ~/.gemcatch/tasks.db
|
|
236
|
+
Tasks: 3
|
|
237
|
+
completed 3
|
|
238
|
+
Agent runs:
|
|
239
|
+
deep-research-preview-04-2026 3
|
|
240
|
+
Plan chains: 2 plan, 1 report
|
|
241
|
+
Estimated spend: $3.00–$9.00 across 3 billed task(s) (preview rates, subject to change).
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
That's the chain above: one plan, one refinement, one run, three tasks at the same band. It's the published bands applied to what you actually submitted, not a reading of your bill, and it errs toward telling you rather than flattering you:
|
|
245
|
+
|
|
246
|
+
- Only runs that **reached the server** are priced. A submit that failed before it left the machine (bad key, rejected agent id) is still counted under "Agent runs" as an attempt, but it costs nothing and isn't billed.
|
|
247
|
+
- An agent with **no published band** totals to `unknown`, never to `$0.00`. Quoting zero for a run that costs real money is the one thing a spend guard must not do.
|
|
248
|
+
|
|
249
|
+
`list` shows the chain as one thing, and `export` follows it to the report:
|
|
250
|
+
|
|
251
|
+
```console
|
|
252
|
+
$ gemcatch list
|
|
253
|
+
ID AGE STATUS KIND AGENT PROMPT
|
|
254
|
+
d014e21b 22s completed plan deep-research map the EU AI Act high-risk obligations against the UK ap...
|
|
255
|
+
2140e699 12s completed plan deep-research └─ cut the history, and add enforcement penalties on both sides
|
|
256
|
+
96e8209b 7s completed report deep-research └─ map the EU AI Act high-risk obligations against the UK ap...
|
|
257
|
+
|
|
258
|
+
$ gemcatch export --tag euuk -o report.md
|
|
259
|
+
Wrote 1 result(s) to report.md.
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
The plans are working notes on the way to the report, so `export` leaves them out; pass `--include-plans` if you want the whole chain in the document. The report is filed under the question that started the chain rather than the one-line approval actually sent to the API, so an exported document reads as research.
|
|
263
|
+
|
|
264
|
+
A few things that will bite otherwise:
|
|
265
|
+
|
|
266
|
+
- `--plan` needs `--agent`. Collaborative planning is an agent feature, so `--plan --model ...` (or `--plan` on its own) is an error, not a no-op.
|
|
267
|
+
- `approve` only works on a plan that has completed. Anything else fails before a request goes out.
|
|
268
|
+
- On the free tier the plan's interaction is dropped after a day. Once that happens the chain cannot be continued. `approve` says so and names the window instead of sending a `previous_interaction_id` the server will reject. Run the daemon, or approve the same day.
|
|
269
|
+
- Approving twice submits twice. There is no dedupe, and both runs show up under the plan in `list`.
|
|
270
|
+
- `--plan` and `--watch` work together: `gemcatch research "..." --agent deep-research --plan -w` waits for the plan and then prints it.
|
|
271
|
+
|
|
272
|
+
The agent recipe, end to end:
|
|
273
|
+
|
|
274
|
+
```bash
|
|
275
|
+
$ gemcatch batch questions.txt --agent deep-research --plan --yes # a plan per prompt, N × band quoted
|
|
276
|
+
$ gemcatch daemon --exit-when-idle # catch the plans
|
|
277
|
+
$ gemcatch list --tag batch-1a2b3c # read them, approve the good ones
|
|
278
|
+
$ gemcatch approve 8f3a1c04 --yes
|
|
279
|
+
$ gemcatch daemon --exit-when-idle # catch the reports before the 1-day expiry
|
|
280
|
+
$ gemcatch export --tag batch-1a2b3c -o reports.md # every report, with its sources
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
Drop `--plan` and the first two lines become the 0.4.0 one-shot flow, which still works exactly as it did.
|
|
284
|
+
|
|
156
285
|
## How it works
|
|
157
286
|
|
|
158
287
|
Tasks live in SQLite at `~/.gemcatch/tasks.db` (override with `GEMCATCH_HOME`):
|
|
@@ -160,7 +289,8 @@ Tasks live in SQLite at `~/.gemcatch/tasks.db` (override with `GEMCATCH_HOME`):
|
|
|
160
289
|
```sql
|
|
161
290
|
CREATE TABLE tasks (id TEXT PRIMARY KEY, prompt TEXT, interaction_id TEXT,
|
|
162
291
|
status TEXT DEFAULT 'pending', result TEXT, created_at INTEGER);
|
|
163
|
-
-- plus model, system_instruction, tag, error, usage, updated_at
|
|
292
|
+
-- plus model, system_instruction, tag, error, usage, updated_at, agent, citations,
|
|
293
|
+
-- collaborative_planning, previous_interaction_id, kind, parent_id
|
|
164
294
|
```
|
|
165
295
|
|
|
166
296
|
`research` calls `interactions.create({model, input, background: true})` via [`@google/genai`](https://www.npmjs.com/package/@google/genai) and keeps the returned `id`. The polling commands call `interactions.get(id)` and write the status back. Once a task completes, the text is cached in the `result` column — `gemcatch get` then answers from disk without touching the network.
|
|
@@ -208,7 +338,7 @@ Transient failures are retried with exponential backoff and full jitter, honouri
|
|
|
208
338
|
| --- | --- |
|
|
209
339
|
| `GEMINI_API_KEY` | Your API key. `GOOGLE_API_KEY` also works. |
|
|
210
340
|
| `GEMCATCH_HOME` | Where `tasks.db` lives. Default `~/.gemcatch`. |
|
|
211
|
-
| `GEMCATCH_MODEL` | Default model. Default `gemini-3.
|
|
341
|
+
| `GEMCATCH_MODEL` | Default model. Default `gemini-3.5-flash-lite`. |
|
|
212
342
|
| `GEMCATCH_POLL_MS` | `watch` poll interval in ms. Default `10000`. |
|
|
213
343
|
| `GEMCATCH_DAEMON_S` | `daemon` interval in seconds. Default `300`. |
|
|
214
344
|
| `GEMCATCH_RPM` | Requests/minute ceiling. Default `15` (the free tier). `0` disables pacing. |
|
package/db.js
CHANGED
|
@@ -26,6 +26,22 @@ const MIGRATIONS = [
|
|
|
26
26
|
['error', 'TEXT'],
|
|
27
27
|
['usage', 'TEXT'],
|
|
28
28
|
['updated_at', 'INTEGER'],
|
|
29
|
+
// 0.4.0: agent runs. `agent` is the resolved agent id the task was submitted
|
|
30
|
+
// with (NULL for model runs, including every pre-0.4.0 row); `citations` is
|
|
31
|
+
// the JSON array of sources an agent run returned alongside its report.
|
|
32
|
+
['agent', 'TEXT'],
|
|
33
|
+
['citations', 'TEXT'],
|
|
34
|
+
// 0.5.0: collaborative planning. `collaborative_planning` is the agent_config
|
|
35
|
+
// flag the row was submitted with (1 plan turn, 0 report turn, NULL for every
|
|
36
|
+
// run that sent no agent_config at all, including every pre-0.5.0 row);
|
|
37
|
+
// `previous_interaction_id` is the interaction this turn continues;
|
|
38
|
+
// `kind` is 'task' | 'plan' | 'report'; `parent_id` is the local task this
|
|
39
|
+
// turn continues. The DEFAULT backfills 'task' for older rows, so a 0.4.0
|
|
40
|
+
// store keeps behaving exactly as it did.
|
|
41
|
+
['collaborative_planning', 'INTEGER'],
|
|
42
|
+
['previous_interaction_id', 'TEXT'],
|
|
43
|
+
['kind', "TEXT DEFAULT 'task'"],
|
|
44
|
+
['parent_id', 'TEXT'],
|
|
29
45
|
];
|
|
30
46
|
|
|
31
47
|
let _db = null;
|
|
@@ -59,8 +75,10 @@ function createTask(fields) {
|
|
|
59
75
|
const now = Date.now();
|
|
60
76
|
db()
|
|
61
77
|
.prepare(
|
|
62
|
-
'INSERT INTO tasks (id, prompt, status, created_at, updated_at, model, system_instruction, tag
|
|
63
|
-
'
|
|
78
|
+
'INSERT INTO tasks (id, prompt, status, created_at, updated_at, model, system_instruction, tag, agent, ' +
|
|
79
|
+
'kind, parent_id, collaborative_planning, previous_interaction_id) ' +
|
|
80
|
+
'VALUES (@id, @prompt, @status, @now, @now, @model, @system_instruction, @tag, @agent, ' +
|
|
81
|
+
'@kind, @parent_id, @collaborative_planning, @previous_interaction_id)'
|
|
64
82
|
)
|
|
65
83
|
.run({
|
|
66
84
|
id,
|
|
@@ -70,6 +88,13 @@ function createTask(fields) {
|
|
|
70
88
|
model: t.model || null,
|
|
71
89
|
system_instruction: t.systemInstruction || null,
|
|
72
90
|
tag: t.tag || null,
|
|
91
|
+
agent: t.agent || null,
|
|
92
|
+
kind: t.kind || 'task',
|
|
93
|
+
parent_id: t.parentId || null,
|
|
94
|
+
// Presence, not truthiness: `false` is the report turn's real flag and
|
|
95
|
+
// must be stored as 0, while a run that sends no agent_config stores NULL.
|
|
96
|
+
collaborative_planning: t.collaborativePlanning === undefined ? null : Number(!!t.collaborativePlanning),
|
|
97
|
+
previous_interaction_id: t.previousInteractionId || null,
|
|
73
98
|
});
|
|
74
99
|
return id;
|
|
75
100
|
}
|
|
@@ -101,7 +126,7 @@ function setStatus(id, status, extra) {
|
|
|
101
126
|
const e = extra || {};
|
|
102
127
|
const sets = ['status = @status', 'updated_at = @now'];
|
|
103
128
|
const params = { id, status, now: Date.now() };
|
|
104
|
-
for (const key of ['result', 'error', 'usage']) {
|
|
129
|
+
for (const key of ['result', 'error', 'usage', 'citations']) {
|
|
105
130
|
if (e[key] !== undefined) {
|
|
106
131
|
sets.push(`${key} = @${key}`);
|
|
107
132
|
params[key] = e[key];
|
|
@@ -167,6 +192,35 @@ function counts() {
|
|
|
167
192
|
return db().prepare('SELECT status, COUNT(*) AS n FROM tasks GROUP BY status').all();
|
|
168
193
|
}
|
|
169
194
|
|
|
195
|
+
// Per-agent totals for `stats`. Model runs (agent IS NULL) are not a row here;
|
|
196
|
+
// they are already accounted for in counts().
|
|
197
|
+
function agentCounts() {
|
|
198
|
+
return db()
|
|
199
|
+
.prepare('SELECT agent, COUNT(*) AS n FROM tasks WHERE agent IS NOT NULL GROUP BY agent')
|
|
200
|
+
.all();
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
// Agent runs that actually reached the server, keyed by agent. A submit that
|
|
204
|
+
// never got an interaction_id (a bad key, a rejected agent id, a network
|
|
205
|
+
// failure) was never billed, so it must not appear in a spend total -- unlike
|
|
206
|
+
// agentCounts(), which tallies every attempt.
|
|
207
|
+
function billedAgentCounts() {
|
|
208
|
+
return db()
|
|
209
|
+
.prepare(
|
|
210
|
+
'SELECT agent, COUNT(*) AS n FROM tasks WHERE agent IS NOT NULL AND interaction_id IS NOT NULL GROUP BY agent'
|
|
211
|
+
)
|
|
212
|
+
.all();
|
|
213
|
+
}
|
|
214
|
+
|
|
215
|
+
// Plan/report totals for `stats`. Ordinary tasks are not a row here; they are
|
|
216
|
+
// already accounted for in counts(), and a store that has never planned reports
|
|
217
|
+
// nothing at all.
|
|
218
|
+
function kindCounts() {
|
|
219
|
+
return db()
|
|
220
|
+
.prepare("SELECT kind, COUNT(*) AS n FROM tasks WHERE kind IS NOT NULL AND kind != 'task' GROUP BY kind")
|
|
221
|
+
.all();
|
|
222
|
+
}
|
|
223
|
+
|
|
170
224
|
function close() {
|
|
171
225
|
if (_db) _db.close();
|
|
172
226
|
_db = null;
|
|
@@ -185,5 +239,8 @@ module.exports = {
|
|
|
185
239
|
removeMany,
|
|
186
240
|
prunableTasks,
|
|
187
241
|
counts,
|
|
242
|
+
agentCounts,
|
|
243
|
+
billedAgentCounts,
|
|
244
|
+
kindCounts,
|
|
188
245
|
close,
|
|
189
246
|
};
|
package/gemini.js
CHANGED
|
@@ -3,7 +3,47 @@
|
|
|
3
3
|
const { isDone, isSuccess } = require('./status');
|
|
4
4
|
|
|
5
5
|
// Free of charge on the Gemini free tier; override per-call with --model.
|
|
6
|
-
|
|
6
|
+
// gemini-3.5-flash-lite went GA on 2026-07-21 (it replaced 3.1 as the
|
|
7
|
+
// low-latency free-tier workhorse in the same release that deprecated the
|
|
8
|
+
// sampling parameters).
|
|
9
|
+
const DEFAULT_MODEL = process.env.GEMCATCH_MODEL || 'gemini-3.5-flash-lite';
|
|
10
|
+
|
|
11
|
+
// --- agents ---------------------------------------------------------------
|
|
12
|
+
|
|
13
|
+
// The Deep Research agents are reachable ONLY through the Interactions API,
|
|
14
|
+
// and only with background execution -- which gemcatch always sets. An agent
|
|
15
|
+
// is sent as `agent` on create, INSTEAD of `model`: the two are mutually
|
|
16
|
+
// exclusive, and the CLI rejects the combination before anything is written.
|
|
17
|
+
//
|
|
18
|
+
// This table is the ONE place the full preview ids live. They are preview ids
|
|
19
|
+
// and will be superseded; call sites must resolve through here (or pass an
|
|
20
|
+
// unknown id straight through, so a future agent works without a release).
|
|
21
|
+
const AGENT_ALIASES = Object.freeze({
|
|
22
|
+
'deep-research': 'deep-research-preview-04-2026',
|
|
23
|
+
'deep-research-max': 'deep-research-max-preview-04-2026',
|
|
24
|
+
});
|
|
25
|
+
|
|
26
|
+
// `agent_config.type` for the config block a collaborative-planning turn sends.
|
|
27
|
+
// Both documented Deep Research agents use the same value, and an unknown
|
|
28
|
+
// pass-through id is assumed to be one too -- collaborative planning is a Deep
|
|
29
|
+
// Research feature, so there is nothing else it could be.
|
|
30
|
+
const AGENT_CONFIG_TYPE = 'deep-research';
|
|
31
|
+
|
|
32
|
+
// Documented per-task price bands, in dollars, keyed by the RESOLVED id.
|
|
33
|
+
// The docs' own hedge applies -- "These figures are estimates based on
|
|
34
|
+
// preview rates and are subject to change" -- so the spend guard quotes
|
|
35
|
+
// them as estimates, never as authoritative.
|
|
36
|
+
const AGENT_PRICE_BANDS = Object.freeze({
|
|
37
|
+
'deep-research-preview-04-2026': Object.freeze([1, 3]),
|
|
38
|
+
'deep-research-max-preview-04-2026': Object.freeze([3, 7]),
|
|
39
|
+
});
|
|
40
|
+
|
|
41
|
+
// A known alias resolves to its full preview id; anything else passes through
|
|
42
|
+
// unchanged so a new or newer agent id works without a gemcatch release (a
|
|
43
|
+
// genuinely bad id fails fast: the API 4xxes, and a 4xx never retries).
|
|
44
|
+
function resolveAgent(id) {
|
|
45
|
+
return AGENT_ALIASES[id] || id;
|
|
46
|
+
}
|
|
7
47
|
|
|
8
48
|
// Overridable for tests and for routing via a proxy/gateway.
|
|
9
49
|
const REST_BASE =
|
|
@@ -163,7 +203,11 @@ function collectText(node, acc) {
|
|
|
163
203
|
return acc;
|
|
164
204
|
}
|
|
165
205
|
if (typeof node.text === 'string' && node.text.trim()) acc.push(node.text);
|
|
166
|
-
for (const v of Object.
|
|
206
|
+
for (const [k, v] of Object.entries(node)) {
|
|
207
|
+
// Citations are sources *about* the answer, not answer text: an agent step
|
|
208
|
+
// carries them alongside its content, and a citation's own title/snippet
|
|
209
|
+
// must not be concatenated into the result. They are collected separately.
|
|
210
|
+
if (k === 'citations') continue;
|
|
167
211
|
if (v && typeof v === 'object') collectText(v, acc);
|
|
168
212
|
}
|
|
169
213
|
return acc;
|
|
@@ -173,21 +217,65 @@ function collectText(node, acc) {
|
|
|
173
217
|
// internal reasoning with the actual answer, each tagged by `type`:
|
|
174
218
|
// [ {type:'user_input', ...}, {type:'thought', ...}, {type:'model_output', ...} ]
|
|
175
219
|
// Collecting text indiscriminately prepends the prompt (and any reasoning) to
|
|
176
|
-
// the result, so those step types are skipped.
|
|
177
|
-
//
|
|
178
|
-
//
|
|
220
|
+
// the result, so those step types are skipped.
|
|
221
|
+
//
|
|
222
|
+
// Both kinds of run put the deliverable in the FINAL answer-bearing step. A
|
|
223
|
+
// model run ends [user_input, thought, model_output]; an agent run's steps
|
|
224
|
+
// additionally interleave its plan, searches and interim drafts, and the docs
|
|
225
|
+
// place the finished report at `interaction.steps[-1].content[0].text`. So one
|
|
226
|
+
// rule serves both, with no special-casing on the agent id: take the last step
|
|
227
|
+
// that is not user_input/thought. If that step somehow carries no text -- an
|
|
228
|
+
// unexpected shape, a renamed type -- fall back to collecting across every
|
|
229
|
+
// answer-bearing step, so the failure mode is "too much text", never a
|
|
230
|
+
// silently blank result.
|
|
179
231
|
const NON_ANSWER_STEP = new Set(['user_input', 'thought']);
|
|
180
232
|
|
|
181
233
|
function textFromSteps(steps) {
|
|
182
234
|
if (!Array.isArray(steps)) return '';
|
|
235
|
+
const candidates = steps.filter((s) => !(s && NON_ANSWER_STEP.has(s.type)));
|
|
236
|
+
if (!candidates.length) return '';
|
|
237
|
+
const last = collectText(candidates[candidates.length - 1], []).join('\n').trim();
|
|
238
|
+
if (last) return last;
|
|
183
239
|
const acc = [];
|
|
184
|
-
for (const step of
|
|
185
|
-
if (step && NON_ANSWER_STEP.has(step.type)) continue;
|
|
186
|
-
collectText(step, acc);
|
|
187
|
-
}
|
|
240
|
+
for (const step of candidates) collectText(step, acc);
|
|
188
241
|
return acc.join('\n').trim();
|
|
189
242
|
}
|
|
190
243
|
|
|
244
|
+
// Agent runs carry citations -- the docs explicitly tell users to review them
|
|
245
|
+
// to verify the sources -- so they are gathered rather than discarded. The
|
|
246
|
+
// walk is shape-agnostic (any `citations` array anywhere in the interaction),
|
|
247
|
+
// because the docs do not pin down where they attach; duplicates are dropped.
|
|
248
|
+
function collectCitations(node, acc) {
|
|
249
|
+
if (!node || typeof node !== 'object') return acc;
|
|
250
|
+
if (Array.isArray(node)) {
|
|
251
|
+
for (const n of node) collectCitations(n, acc);
|
|
252
|
+
return acc;
|
|
253
|
+
}
|
|
254
|
+
for (const [k, v] of Object.entries(node)) {
|
|
255
|
+
if (k === 'citations' && Array.isArray(v)) {
|
|
256
|
+
for (const c of v) if (c && typeof c === 'object') acc.push(c);
|
|
257
|
+
continue;
|
|
258
|
+
}
|
|
259
|
+
if (v && typeof v === 'object') collectCitations(v, acc);
|
|
260
|
+
}
|
|
261
|
+
return acc;
|
|
262
|
+
}
|
|
263
|
+
|
|
264
|
+
function citationsOf(interaction) {
|
|
265
|
+
const all = collectCitations(interaction, []);
|
|
266
|
+
if (!all.length) return null;
|
|
267
|
+
const seen = new Set();
|
|
268
|
+
const out = [];
|
|
269
|
+
for (const c of all) {
|
|
270
|
+
const key = JSON.stringify(c);
|
|
271
|
+
if (!seen.has(key)) {
|
|
272
|
+
seen.add(key);
|
|
273
|
+
out.push(c);
|
|
274
|
+
}
|
|
275
|
+
}
|
|
276
|
+
return out;
|
|
277
|
+
}
|
|
278
|
+
|
|
191
279
|
function textOf(interaction) {
|
|
192
280
|
if (interaction && typeof interaction.output_text === 'string' && interaction.output_text) {
|
|
193
281
|
return interaction.output_text;
|
|
@@ -200,6 +288,7 @@ function shape(r) {
|
|
|
200
288
|
interactionId: r.id,
|
|
201
289
|
status: r.status,
|
|
202
290
|
text: textOf(r),
|
|
291
|
+
citations: citationsOf(r),
|
|
203
292
|
usage: r.usage || null,
|
|
204
293
|
raw: r,
|
|
205
294
|
};
|
|
@@ -269,8 +358,29 @@ function restHeaders() {
|
|
|
269
358
|
|
|
270
359
|
async function submit(prompt, opts) {
|
|
271
360
|
const o = opts || {};
|
|
272
|
-
|
|
361
|
+
// `agent` and `model` are mutually exclusive on create: an agent run is sent
|
|
362
|
+
// with `agent` INSTEAD of `model` (the agent picks its own models). `input`
|
|
363
|
+
// stays a plain string and `background` stays true either way -- agents
|
|
364
|
+
// *require* background execution, which gemcatch has always set.
|
|
365
|
+
const body = o.agent
|
|
366
|
+
? { agent: o.agent, input: prompt, background: true }
|
|
367
|
+
: { model: o.model || DEFAULT_MODEL, input: prompt, background: true };
|
|
273
368
|
if (o.systemInstruction) body.system_instruction = o.systemInstruction;
|
|
369
|
+
// collaborative_planning is an `agent_config` field, NOT a top-level one, and
|
|
370
|
+
// the docs send the whole block (type + thinking_summaries) with it. Sent only
|
|
371
|
+
// when a plan turn is involved -- agent_config is optional otherwise, so an
|
|
372
|
+
// ordinary run keeps making exactly the request it always made. Presence, not
|
|
373
|
+
// truthiness: `false` is the approval turn's real value and must reach the API.
|
|
374
|
+
if (o.collaborativePlanning !== undefined) {
|
|
375
|
+
body.agent_config = {
|
|
376
|
+
type: AGENT_CONFIG_TYPE,
|
|
377
|
+
thinking_summaries: 'auto',
|
|
378
|
+
collaborative_planning: !!o.collaborativePlanning,
|
|
379
|
+
};
|
|
380
|
+
}
|
|
381
|
+
// Continues an earlier interaction server-side: the plan is already in that
|
|
382
|
+
// conversation, so this turn sends only what changed.
|
|
383
|
+
if (o.previousInteractionId) body.previous_interaction_id = o.previousInteractionId;
|
|
274
384
|
const r = await call(() => {
|
|
275
385
|
const api = sdkInteractions();
|
|
276
386
|
return api
|
|
@@ -323,6 +433,10 @@ module.exports = {
|
|
|
323
433
|
REST_BASE,
|
|
324
434
|
RPM,
|
|
325
435
|
MAX_RETRIES,
|
|
436
|
+
AGENT_ALIASES,
|
|
437
|
+
AGENT_PRICE_BANDS,
|
|
438
|
+
AGENT_CONFIG_TYPE,
|
|
439
|
+
resolveAgent,
|
|
326
440
|
submit,
|
|
327
441
|
poll,
|
|
328
442
|
cancel,
|
|
@@ -330,6 +444,7 @@ module.exports = {
|
|
|
330
444
|
apiKey,
|
|
331
445
|
textOf,
|
|
332
446
|
collectText,
|
|
447
|
+
citationsOf,
|
|
333
448
|
// Exported for the suite: the retry policy is behaviour worth pinning.
|
|
334
449
|
shouldRetry,
|
|
335
450
|
// Re-exported so callers need only one require.
|