gemcatch 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/CHANGELOG.md +110 -1
  2. package/README.md +137 -7
  3. package/db.js +60 -3
  4. package/gemini.js +125 -10
  5. package/index.js +576 -56
  6. package/package.json +7 -2
package/CHANGELOG.md CHANGED
@@ -7,6 +7,113 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.5.0] - 2026-09-04
11
+
12
+ ### Added
13
+
14
+ - **Collaborative planning.** Until now the only agent flow was a blind one-shot
15
+ bet: gemcatch showed a cost band, asked y/N, and fired. The cost band was
16
+ everything you knew before spending, and there was no way to check the agent
17
+ had read the prompt the way you meant it. A 20-line `batch` against
18
+ `deep-research-max` is a $60–$140 command placed sight unseen. The Deep
19
+ Research docs have always documented a second mode for exactly this, and
20
+ gemcatch could not reach it. Now it can:
21
+
22
+ - `--plan` on `research` and `batch` submits with
23
+ `agent_config.collaborative_planning: true`, so **the agent returns a
24
+ research plan instead of a report**. Still `background: true`, still stored,
25
+ still collected by the daemon. The row is stored with `kind='plan'`.
26
+ - `gemcatch get <id>` on a plan prints the plan and then the literal next
27
+ command, `Approve with: gemcatch approve 8f3a1c04 · Refine with: gemcatch
28
+ refine 8f3a1c04 "..."`, on stderr, so `get <plan> > plan.md` still captures
29
+ just the plan.
30
+ - `gemcatch refine <id> "<instruction>"` sends the instruction back with
31
+ `previous_interaction_id` and planning still on, and stores the revised plan
32
+ linked to the one it came from, inheriting its agent and tag.
33
+ - `gemcatch approve <id>` is the turn that commits: `previous_interaction_id`
34
+ with `collaborative_planning: false`, stored as `kind='report'` linked to the
35
+ plan. It shows the cost band and asks, honours `--yes` when stdin is not a
36
+ TTY, and honours `--dry-run`.
37
+
38
+ **Planning is not cheaper.** The docs give one band per task and do not price a
39
+ planning turn separately, so a planning turn is quoted at the same band and the
40
+ line says so: *"the docs price per task and do not price a planning turn
41
+ separately"*. What you get for it is a look at the plan before you commit to
42
+ the research run, not a discount.
43
+ - `gemcatch list` renders a chain indented under its root, in submission order,
44
+ and grows a `KIND` column when a listing contains plans or reports (a store
45
+ with neither keeps the layout it had). `gemcatch export` follows a chain to its
46
+ report and leaves the intermediate plans out unless `--include-plans` is
47
+ passed; the Markdown sections and the JSON rows now name the turn.
48
+ - `gemcatch stats` now tallies plan and report turns, and totals the estimated
49
+ spend across the agent turns that were actually billed, from the same
50
+ published bands the guard quotes before each one. A chain bills per turn, so a
51
+ release that turns one submission into three owes the user a running total. It
52
+ is the documented bands applied to what was submitted, not a reading of your
53
+ bill, and it errs toward telling you rather than flattering you: only runs
54
+ that reached the server are priced (a submit that failed before it left the
55
+ machine is counted as an attempt but costs nothing), and an agent with no
56
+ published band totals to `unknown`, never to `$0.00`. A store with no agent
57
+ runs prints neither line.
58
+ - Additive schema migration: `collaborative_planning`, `previous_interaction_id`,
59
+ `kind` (defaulting to `'task'`) and `parent_id`. A 0.4.0 `tasks.db` upgrades in
60
+ place, keeps every row, and behaves exactly as it did.
61
+
62
+ ### Fixed
63
+
64
+ - `approve` and `refine` fail before sending anything when the target is not a
65
+ plan, or is a plan that has not completed. A plan whose interaction the free
66
+ tier has already dropped names the retention window (1 day free, 55 days paid)
67
+ instead of sending a `previous_interaction_id` the server would reject. A plan
68
+ that expires *after* it completed locally gets the same explanation rather
69
+ than a bare 404 for an id the user never typed.
70
+ - `--plan` without `--agent` (or with `--model`) is a clean error rather than a
71
+ flag that quietly does nothing: collaborative planning is an agent feature.
72
+ - Approving the same plan twice submits a second run rather than silently
73
+ reusing the first, and `list` shows both under their plan.
74
+
75
+ ## [0.4.0] - 2026-08-08
76
+
77
+ ### Added
78
+
79
+ - **Research agents.** `-a, --agent <id>` on `research` and `batch` submits to a
80
+ Gemini Deep Research agent instead of a model — `interactions.create` is sent
81
+ `agent` *instead of* `model` (they are mutually exclusive, and passing both is
82
+ a clean error). Aliases resolve through one table: `deep-research` →
83
+ `deep-research-preview-04-2026`, `deep-research-max` →
84
+ `deep-research-max-preview-04-2026`; any other value passes through unchanged,
85
+ so a future agent id works without a gemcatch release. Agents *require*
86
+ background execution, which gemcatch has always set — and on the free tier the
87
+ finished report is dropped after 1 day, which is exactly the race the daemon
88
+ exists to win. The agent is recorded per task, shown in `list` (the AGENT
89
+ column appears when a listing contains agent runs) and tallied in `stats`.
90
+ - **Spend guard.** Deep Research is documented at $1.00–$3.00 per task and Deep
91
+ Research Max at $3.00–$7.00 (estimates based on preview rates, per the docs,
92
+ and subject to change). Every agent submission prints its band first —
93
+ `batch` prints N × the band as a total — and asks for an interactive `y/N`
94
+ confirmation. When stdin is not a TTY, `--yes` is required and anything else
95
+ is refused before a row is written; declining writes nothing and exits
96
+ non-zero. `--dry-run` (now on `research` too) prints the full projected spend
97
+ and submits nothing.
98
+ - **Citations.** Agent runs return citations alongside the report; the docs say
99
+ to review them to verify the sources, so they are persisted (new `citations`
100
+ column, JSON) rather than discarded, printed under the result as a `Sources:`
101
+ list, and carried in `--json` output.
102
+ - Result extraction now takes the **final answer-bearing step** — where the
103
+ docs place an agent's completed report (`steps[-1].content[0].text`) and
104
+ where a model run's `model_output` already sits — with a fall-back to the old
105
+ collect-everything behaviour if that step carries no text, so an unexpected
106
+ shape can never silently blank a result. No special-casing on the agent id.
107
+ - Additive schema migration: `agent` and `citations` columns. A pre-0.4.0
108
+ `tasks.db` upgrades in place, keeps every row, and reports `agent` as NULL
109
+ for them.
110
+
111
+ ### Changed
112
+
113
+ - The default model is now **`gemini-3.5-flash-lite`** (GA on 2026-07-21),
114
+ replacing the older `gemini-3.1-flash-lite`. Override with `GEMCATCH_MODEL`
115
+ or `--model` as before.
116
+
10
117
  ## [0.3.0] - 2026-07-19
11
118
 
12
119
  ### Added
@@ -162,7 +269,9 @@ seen a task complete, the text is cached locally and survives that expiry — bu
162
269
  something has to poll inside that window for it to be seen at all, which is what
163
270
  `gemcatch daemon` exists to do.
164
271
 
165
- [Unreleased]: https://github.com/Booyaka101/gemcatch/compare/v0.3.0...HEAD
272
+ [Unreleased]: https://github.com/Booyaka101/gemcatch/compare/v0.5.0...HEAD
273
+ [0.5.0]: https://github.com/Booyaka101/gemcatch/compare/v0.4.0...v0.5.0
274
+ [0.4.0]: https://github.com/Booyaka101/gemcatch/compare/v0.3.0...v0.4.0
166
275
  [0.3.0]: https://github.com/Booyaka101/gemcatch/compare/v0.2.0...v0.3.0
167
276
  [0.2.0]: https://github.com/Booyaka101/gemcatch/compare/v0.1.1...v0.2.0
168
277
  [0.1.1]: https://github.com/Booyaka101/gemcatch/compare/v0.1.0...v0.1.1
package/README.md CHANGED
@@ -24,7 +24,7 @@ The EU AI Act's high-risk obligations phase in from August 2026, whereas...
24
24
 
25
25
  ## Setup
26
26
 
27
- Needs Node.js 22+ and a Gemini API key. **Getting a key needs no billing account and no card.** `gemini-3.1-flash-lite` runs free within the [free tier's](https://ai.google.dev/gemini-api/docs/pricing) daily quota; past that, paid rates apply.
27
+ Needs Node.js 22+ and a Gemini API key. **Getting a key needs no billing account and no card.** `gemini-3.5-flash-lite` (the default model, GA since July 2026) runs free within the [free tier's](https://ai.google.dev/gemini-api/docs/pricing) daily quota; past that, paid rates apply.
28
28
 
29
29
  1. Get a key at **<https://aistudio.google.com/apikey>**
30
30
  2. Put it in your environment:
@@ -81,8 +81,10 @@ This week in AI: ...
81
81
  | `gemcatch research "<prompt>"` | Submits with `background: true`, stores the interaction ID, exits immediately. |
82
82
  | `gemcatch batch <file>` | Submits many prompts from a file at once, tagged as one collectable batch. |
83
83
  | `gemcatch status <id>` | Polls the API and prints the current state. |
84
- | `gemcatch get <id>` | Prints the full response if complete, otherwise the current status. |
85
- | `gemcatch list` | All tasks, newest first: id, age, status, prompt. |
84
+ | `gemcatch get <id>` | Prints the full response if complete, otherwise the current status. On a [plan](#see-the-plan-before-you-pay-for-the-run), the plan plus the approve/refine commands. |
85
+ | `gemcatch refine <id> "<instruction>"` | Sends an instruction back to a plan and returns a revised plan. |
86
+ | `gemcatch approve <id>` | Approves a plan and submits the research run it describes. |
87
+ | `gemcatch list` | All tasks, newest first: id, age, status, prompt. Plan chains are indented under their root. |
86
88
  | `gemcatch export` | Concatenates finished results, each under its prompt, to stdout or a file (Markdown or JSON). |
87
89
  | `gemcatch digest` | Feeds a tag's completed results through one Gemini call into a single summary. |
88
90
  | `gemcatch watch <id>` | Polls until the task finishes, then prints the result. |
@@ -91,7 +93,7 @@ This week in AI: ...
91
93
  | `gemcatch cancel <id>` | Asks the API to stop an in-flight task. |
92
94
  | `gemcatch rm <ids...>` | Forgets tasks locally. `--remote` deletes them server-side too. |
93
95
  | `gemcatch prune` | Drops finished tasks older than `--days` (default 30). |
94
- | `gemcatch stats` | Where the store lives and what's in it. |
96
+ | `gemcatch stats` | Where the store lives, what's in it, and what the agent runs have plausibly cost. |
95
97
 
96
98
  Useful flags:
97
99
 
@@ -99,6 +101,9 @@ Useful flags:
99
101
  | --- | --- | --- |
100
102
  | `--json` | most commands | Machine-readable output. |
101
103
  | `-m, --model <id>` | `research`, `batch` | Override the model. |
104
+ | `-a, --agent <id>` | `research`, `batch` | Submit to a [research agent](#research-agents) instead of a model. Mutually exclusive with `--model`. |
105
+ | `--plan` | `research`, `batch` | Ask the agent for a [research plan](#see-the-plan-before-you-pay-for-the-run) first, to refine and approve. Needs `--agent`. |
106
+ | `--yes` | `research`, `batch`, `refine`, `approve` | Confirm the agent cost without asking. Required for `--agent` when stdin is not a TTY. |
102
107
  | `-s, --system <text>` | `research`, `batch` | Set a system instruction. |
103
108
  | `-f, --file <path>` | `research` | Read the prompt from a file. |
104
109
  | `-t, --tag <tag>` | `research`, `batch`, `list` | Label tasks and filter them. |
@@ -110,7 +115,8 @@ Useful flags:
110
115
  | `-n, --limit <n>` | `list` | Cap the rows (non-negative; `0` shows none). |
111
116
  | `--format <md\|json>` | `export` | Output format. Default `md`. |
112
117
  | `-o, --out <file>` | `export` | Write to a file instead of stdout. |
113
- | `--dry-run` | `batch`, `prune` | Show what would go; submit/delete nothing. |
118
+ | `--include-plans` | `export` | Also emit a chain's plan turns, not just its report. |
119
+ | `--dry-run` | `research`, `batch`, `refine`, `approve`, `prune` | Show what would go — including the projected agent spend; submit/delete nothing. |
114
120
  | `--raw` | `get` | Dump the raw interaction JSON. |
115
121
 
116
122
  IDs are the first 8 characters of a UUID. Any unique prefix works, so `gemcatch get 8f3a` is fine.
@@ -153,6 +159,129 @@ $ id=$(gemcatch research "..." --json | jq -r .id)
153
159
  $ gemcatch watch "$id" --json | jq -r .result
154
160
  ```
155
161
 
162
+ ## Research agents
163
+
164
+ The [Gemini Deep Research agents](https://ai.google.dev/gemini-api/docs/deep-research) are reachable only through the Interactions API, and the docs are explicit: *"You must use background execution (set `background=true`) to run the agent asynchronously and poll for results or stream updates."* That is precisely the half of the job `gemcatch` already does — it always sets `background: true`, owns the polling, and its daemon collects results before the free tier drops interactions after **1 day** (paid tier: 55 days). A Deep Research run takes minutes and you were never going to sit there holding the connection; submit it, and let the daemon catch it.
165
+
166
+ ```console
167
+ $ gemcatch research "map the EU AI Act high-risk obligations against the UK approach" --agent deep-research
168
+ Agent deep-research-preview-04-2026 — estimated $1.00–$3.00 for this task (preview rates, subject to change).
169
+ Submit? [y/N] y
170
+ Task 8f3a1c04 submitted. Run: gemcatch get 8f3a1c04 when ready.
171
+ ```
172
+
173
+ `--agent` takes an alias or a raw agent id:
174
+
175
+ | You type | Sent to the API |
176
+ | --- | --- |
177
+ | `deep-research` | `deep-research-preview-04-2026` |
178
+ | `deep-research-max` | `deep-research-max-preview-04-2026` |
179
+ | anything else | passed through unchanged (future agent ids work without a gemcatch release; a bad id fails fast with the API's own 4xx) |
180
+
181
+ An agent is sent **instead of** a model — the agent picks its own models — so `--model` and `--agent` together is an error, and nothing is submitted.
182
+
183
+ **These agents cost real money, per task.** The docs put Deep Research at **$1.00–$3.00 per task** and Deep Research Max at **$3.00–$7.00 per task** — with their own hedge attached: *"These figures are estimates based on preview rates and are subject to change."* Because `gemcatch batch` fires a whole file at once, a 20-line file against `deep-research-max` is a **$60–$140 command**, so every agent submission shows its band and asks first. In a script (stdin not a TTY) you must pass `--yes`; `--dry-run` prints the full projected spend and submits nothing:
184
+
185
+ ```console
186
+ $ gemcatch batch questions.txt --agent deep-research-max --dry-run
187
+ 20 prompts × deep-research-max-preview-04-2026 — estimated $60.00–$140.00 total. Nothing submitted (--dry-run).
188
+ ```
189
+
190
+ The report lands like any other result — final answer only, none of the agent's interim plan — and its **citations** come with it. The docs tell you to review them to verify the sources, so `gemcatch get` prints them under the report as a `Sources:` list, `--json` carries them as an array, and they live in the store alongside the result.
191
+
192
+ An agent run can also come back `incomplete` — that is what a `max_total_tokens` budget cap produces when the run "safely pauses" — which `gemcatch` treats as terminal, exactly like the API does: the daemon retires it and moves on.
193
+
194
+ ### See the plan before you pay for the run
195
+
196
+ A cost band tells you what a run will cost. It tells you nothing about whether the agent understood the question. Add `--plan` and it doesn't start researching: with `agent_config.collaborative_planning: true`, *"the agent returns a research plan instead of a full report"*. You read it, push back on it, and approve it when it's right.
197
+
198
+ ```console
199
+ $ gemcatch research "map the EU AI Act high-risk obligations against the UK approach" --agent deep-research --plan -t euuk
200
+ Agent deep-research-preview-04-2026 (planning turn) — estimated $1.00–$3.00 for this task (preview rates, subject to change; the docs price per task and do not price a planning turn separately).
201
+ Submit? [y/N] y
202
+ Plan task d014e21b submitted. Run: gemcatch get d014e21b when ready.
203
+
204
+ $ gemcatch get d014e21b
205
+ Research plan
206
+
207
+ 1. Scope the EU AI Act high-risk regime: Annex III use cases, Article 6 classification,
208
+ and the Chapter III obligations (risk management, data governance, logging, human
209
+ oversight, conformity assessment) with their August 2026 / August 2027 dates.
210
+ 2. Scope the UK approach: the five cross-sector principles, the regulator-led model
211
+ (ICO, FCA, MHRA, Ofcom), and what is guidance rather than statute.
212
+ 3. Build an obligation-by-obligation comparison table: EU requirement, nearest UK
213
+ equivalent, whether it is binding, and who enforces it.
214
+ 4. Flag the gaps in both directions and the compliance implications for a firm
215
+ operating in both.
216
+ Approve with: gemcatch approve d014e21b · Refine with: gemcatch refine d014e21b "..."
217
+
218
+ $ gemcatch refine d014e21b "cut the history, and add enforcement penalties on both sides"
219
+ Plan task 2140e699 submitted (refines d014e21b). Run: gemcatch get 2140e699 when ready.
220
+
221
+ $ gemcatch approve 2140e699
222
+ Agent deep-research-preview-04-2026 — estimated $1.00–$3.00 for this task (preview rates, subject to change).
223
+ Submit? [y/N] y
224
+ Task 96e8209b submitted (approves plan 2140e699).
225
+ ```
226
+
227
+ Every turn is an ordinary background task: stored, polled, and collected by the daemon before the 1-day expiry, same as everything else. `refine` chains as many times as you like; each revision inherits the agent and tag of the plan it came from.
228
+
229
+ **A planning turn is a task, and it is billed as one.** The docs publish one band per task and price no planning turn separately, so `gemcatch` quotes the same band for it and says exactly that on the line. What `--plan` buys you is a look at the plan before you commit to the research run, not a discount. Budget for the plan, each refinement, and the run.
230
+
231
+ Since a chain bills per turn, `gemcatch stats` keeps a running total across all of them:
232
+
233
+ ```console
234
+ $ gemcatch stats
235
+ Store: ~/.gemcatch/tasks.db
236
+ Tasks: 3
237
+ completed 3
238
+ Agent runs:
239
+ deep-research-preview-04-2026 3
240
+ Plan chains: 2 plan, 1 report
241
+ Estimated spend: $3.00–$9.00 across 3 billed task(s) (preview rates, subject to change).
242
+ ```
243
+
244
+ That's the chain above: one plan, one refinement, one run, three tasks at the same band. It's the published bands applied to what you actually submitted, not a reading of your bill, and it errs toward telling you rather than flattering you:
245
+
246
+ - Only runs that **reached the server** are priced. A submit that failed before it left the machine (bad key, rejected agent id) is still counted under "Agent runs" as an attempt, but it costs nothing and isn't billed.
247
+ - An agent with **no published band** totals to `unknown`, never to `$0.00`. Quoting zero for a run that costs real money is the one thing a spend guard must not do.
248
+
249
+ `list` shows the chain as one thing, and `export` follows it to the report:
250
+
251
+ ```console
252
+ $ gemcatch list
253
+ ID AGE STATUS KIND AGENT PROMPT
254
+ d014e21b 22s completed plan deep-research map the EU AI Act high-risk obligations against the UK ap...
255
+ 2140e699 12s completed plan deep-research └─ cut the history, and add enforcement penalties on both sides
256
+ 96e8209b 7s completed report deep-research └─ map the EU AI Act high-risk obligations against the UK ap...
257
+
258
+ $ gemcatch export --tag euuk -o report.md
259
+ Wrote 1 result(s) to report.md.
260
+ ```
261
+
262
+ The plans are working notes on the way to the report, so `export` leaves them out; pass `--include-plans` if you want the whole chain in the document. The report is filed under the question that started the chain rather than the one-line approval actually sent to the API, so an exported document reads as research.
263
+
264
+ A few things that will bite otherwise:
265
+
266
+ - `--plan` needs `--agent`. Collaborative planning is an agent feature, so `--plan --model ...` (or `--plan` on its own) is an error, not a no-op.
267
+ - `approve` only works on a plan that has completed. Anything else fails before a request goes out.
268
+ - On the free tier the plan's interaction is dropped after a day. Once that happens the chain cannot be continued. `approve` says so and names the window instead of sending a `previous_interaction_id` the server will reject. Run the daemon, or approve the same day.
269
+ - Approving twice submits twice. There is no dedupe, and both runs show up under the plan in `list`.
270
+ - `--plan` and `--watch` work together: `gemcatch research "..." --agent deep-research --plan -w` waits for the plan and then prints it.
271
+
272
+ The agent recipe, end to end:
273
+
274
+ ```bash
275
+ $ gemcatch batch questions.txt --agent deep-research --plan --yes # a plan per prompt, N × band quoted
276
+ $ gemcatch daemon --exit-when-idle # catch the plans
277
+ $ gemcatch list --tag batch-1a2b3c # read them, approve the good ones
278
+ $ gemcatch approve 8f3a1c04 --yes
279
+ $ gemcatch daemon --exit-when-idle # catch the reports before the 1-day expiry
280
+ $ gemcatch export --tag batch-1a2b3c -o reports.md # every report, with its sources
281
+ ```
282
+
283
+ Drop `--plan` and the first two lines become the 0.4.0 one-shot flow, which still works exactly as it did.
284
+
156
285
  ## How it works
157
286
 
158
287
  Tasks live in SQLite at `~/.gemcatch/tasks.db` (override with `GEMCATCH_HOME`):
@@ -160,7 +289,8 @@ Tasks live in SQLite at `~/.gemcatch/tasks.db` (override with `GEMCATCH_HOME`):
160
289
  ```sql
161
290
  CREATE TABLE tasks (id TEXT PRIMARY KEY, prompt TEXT, interaction_id TEXT,
162
291
  status TEXT DEFAULT 'pending', result TEXT, created_at INTEGER);
163
- -- plus model, system_instruction, tag, error, usage, updated_at
292
+ -- plus model, system_instruction, tag, error, usage, updated_at, agent, citations,
293
+ -- collaborative_planning, previous_interaction_id, kind, parent_id
164
294
  ```
165
295
 
166
296
  `research` calls `interactions.create({model, input, background: true})` via [`@google/genai`](https://www.npmjs.com/package/@google/genai) and keeps the returned `id`. The polling commands call `interactions.get(id)` and write the status back. Once a task completes, the text is cached in the `result` column — `gemcatch get` then answers from disk without touching the network.
@@ -208,7 +338,7 @@ Transient failures are retried with exponential backoff and full jitter, honouri
208
338
  | --- | --- |
209
339
  | `GEMINI_API_KEY` | Your API key. `GOOGLE_API_KEY` also works. |
210
340
  | `GEMCATCH_HOME` | Where `tasks.db` lives. Default `~/.gemcatch`. |
211
- | `GEMCATCH_MODEL` | Default model. Default `gemini-3.1-flash-lite`. |
341
+ | `GEMCATCH_MODEL` | Default model. Default `gemini-3.5-flash-lite`. |
212
342
  | `GEMCATCH_POLL_MS` | `watch` poll interval in ms. Default `10000`. |
213
343
  | `GEMCATCH_DAEMON_S` | `daemon` interval in seconds. Default `300`. |
214
344
  | `GEMCATCH_RPM` | Requests/minute ceiling. Default `15` (the free tier). `0` disables pacing. |
package/db.js CHANGED
@@ -26,6 +26,22 @@ const MIGRATIONS = [
26
26
  ['error', 'TEXT'],
27
27
  ['usage', 'TEXT'],
28
28
  ['updated_at', 'INTEGER'],
29
+ // 0.4.0: agent runs. `agent` is the resolved agent id the task was submitted
30
+ // with (NULL for model runs, including every pre-0.4.0 row); `citations` is
31
+ // the JSON array of sources an agent run returned alongside its report.
32
+ ['agent', 'TEXT'],
33
+ ['citations', 'TEXT'],
34
+ // 0.5.0: collaborative planning. `collaborative_planning` is the agent_config
35
+ // flag the row was submitted with (1 plan turn, 0 report turn, NULL for every
36
+ // run that sent no agent_config at all, including every pre-0.5.0 row);
37
+ // `previous_interaction_id` is the interaction this turn continues;
38
+ // `kind` is 'task' | 'plan' | 'report'; `parent_id` is the local task this
39
+ // turn continues. The DEFAULT backfills 'task' for older rows, so a 0.4.0
40
+ // store keeps behaving exactly as it did.
41
+ ['collaborative_planning', 'INTEGER'],
42
+ ['previous_interaction_id', 'TEXT'],
43
+ ['kind', "TEXT DEFAULT 'task'"],
44
+ ['parent_id', 'TEXT'],
29
45
  ];
30
46
 
31
47
  let _db = null;
@@ -59,8 +75,10 @@ function createTask(fields) {
59
75
  const now = Date.now();
60
76
  db()
61
77
  .prepare(
62
- 'INSERT INTO tasks (id, prompt, status, created_at, updated_at, model, system_instruction, tag) ' +
63
- 'VALUES (@id, @prompt, @status, @now, @now, @model, @system_instruction, @tag)'
78
+ 'INSERT INTO tasks (id, prompt, status, created_at, updated_at, model, system_instruction, tag, agent, ' +
79
+ 'kind, parent_id, collaborative_planning, previous_interaction_id) ' +
80
+ 'VALUES (@id, @prompt, @status, @now, @now, @model, @system_instruction, @tag, @agent, ' +
81
+ '@kind, @parent_id, @collaborative_planning, @previous_interaction_id)'
64
82
  )
65
83
  .run({
66
84
  id,
@@ -70,6 +88,13 @@ function createTask(fields) {
70
88
  model: t.model || null,
71
89
  system_instruction: t.systemInstruction || null,
72
90
  tag: t.tag || null,
91
+ agent: t.agent || null,
92
+ kind: t.kind || 'task',
93
+ parent_id: t.parentId || null,
94
+ // Presence, not truthiness: `false` is the report turn's real flag and
95
+ // must be stored as 0, while a run that sends no agent_config stores NULL.
96
+ collaborative_planning: t.collaborativePlanning === undefined ? null : Number(!!t.collaborativePlanning),
97
+ previous_interaction_id: t.previousInteractionId || null,
73
98
  });
74
99
  return id;
75
100
  }
@@ -101,7 +126,7 @@ function setStatus(id, status, extra) {
101
126
  const e = extra || {};
102
127
  const sets = ['status = @status', 'updated_at = @now'];
103
128
  const params = { id, status, now: Date.now() };
104
- for (const key of ['result', 'error', 'usage']) {
129
+ for (const key of ['result', 'error', 'usage', 'citations']) {
105
130
  if (e[key] !== undefined) {
106
131
  sets.push(`${key} = @${key}`);
107
132
  params[key] = e[key];
@@ -167,6 +192,35 @@ function counts() {
167
192
  return db().prepare('SELECT status, COUNT(*) AS n FROM tasks GROUP BY status').all();
168
193
  }
169
194
 
195
+ // Per-agent totals for `stats`. Model runs (agent IS NULL) are not a row here;
196
+ // they are already accounted for in counts().
197
+ function agentCounts() {
198
+ return db()
199
+ .prepare('SELECT agent, COUNT(*) AS n FROM tasks WHERE agent IS NOT NULL GROUP BY agent')
200
+ .all();
201
+ }
202
+
203
+ // Agent runs that actually reached the server, keyed by agent. A submit that
204
+ // never got an interaction_id (a bad key, a rejected agent id, a network
205
+ // failure) was never billed, so it must not appear in a spend total -- unlike
206
+ // agentCounts(), which tallies every attempt.
207
+ function billedAgentCounts() {
208
+ return db()
209
+ .prepare(
210
+ 'SELECT agent, COUNT(*) AS n FROM tasks WHERE agent IS NOT NULL AND interaction_id IS NOT NULL GROUP BY agent'
211
+ )
212
+ .all();
213
+ }
214
+
215
+ // Plan/report totals for `stats`. Ordinary tasks are not a row here; they are
216
+ // already accounted for in counts(), and a store that has never planned reports
217
+ // nothing at all.
218
+ function kindCounts() {
219
+ return db()
220
+ .prepare("SELECT kind, COUNT(*) AS n FROM tasks WHERE kind IS NOT NULL AND kind != 'task' GROUP BY kind")
221
+ .all();
222
+ }
223
+
170
224
  function close() {
171
225
  if (_db) _db.close();
172
226
  _db = null;
@@ -185,5 +239,8 @@ module.exports = {
185
239
  removeMany,
186
240
  prunableTasks,
187
241
  counts,
242
+ agentCounts,
243
+ billedAgentCounts,
244
+ kindCounts,
188
245
  close,
189
246
  };
package/gemini.js CHANGED
@@ -3,7 +3,47 @@
3
3
  const { isDone, isSuccess } = require('./status');
4
4
 
5
5
  // Free of charge on the Gemini free tier; override per-call with --model.
6
- const DEFAULT_MODEL = process.env.GEMCATCH_MODEL || 'gemini-3.1-flash-lite';
6
+ // gemini-3.5-flash-lite went GA on 2026-07-21 (it replaced 3.1 as the
7
+ // low-latency free-tier workhorse in the same release that deprecated the
8
+ // sampling parameters).
9
+ const DEFAULT_MODEL = process.env.GEMCATCH_MODEL || 'gemini-3.5-flash-lite';
10
+
11
+ // --- agents ---------------------------------------------------------------
12
+
13
+ // The Deep Research agents are reachable ONLY through the Interactions API,
14
+ // and only with background execution -- which gemcatch always sets. An agent
15
+ // is sent as `agent` on create, INSTEAD of `model`: the two are mutually
16
+ // exclusive, and the CLI rejects the combination before anything is written.
17
+ //
18
+ // This table is the ONE place the full preview ids live. They are preview ids
19
+ // and will be superseded; call sites must resolve through here (or pass an
20
+ // unknown id straight through, so a future agent works without a release).
21
+ const AGENT_ALIASES = Object.freeze({
22
+ 'deep-research': 'deep-research-preview-04-2026',
23
+ 'deep-research-max': 'deep-research-max-preview-04-2026',
24
+ });
25
+
26
+ // `agent_config.type` for the config block a collaborative-planning turn sends.
27
+ // Both documented Deep Research agents use the same value, and an unknown
28
+ // pass-through id is assumed to be one too -- collaborative planning is a Deep
29
+ // Research feature, so there is nothing else it could be.
30
+ const AGENT_CONFIG_TYPE = 'deep-research';
31
+
32
+ // Documented per-task price bands, in dollars, keyed by the RESOLVED id.
33
+ // The docs' own hedge applies -- "These figures are estimates based on
34
+ // preview rates and are subject to change" -- so the spend guard quotes
35
+ // them as estimates, never as authoritative.
36
+ const AGENT_PRICE_BANDS = Object.freeze({
37
+ 'deep-research-preview-04-2026': Object.freeze([1, 3]),
38
+ 'deep-research-max-preview-04-2026': Object.freeze([3, 7]),
39
+ });
40
+
41
+ // A known alias resolves to its full preview id; anything else passes through
42
+ // unchanged so a new or newer agent id works without a gemcatch release (a
43
+ // genuinely bad id fails fast: the API 4xxes, and a 4xx never retries).
44
+ function resolveAgent(id) {
45
+ return AGENT_ALIASES[id] || id;
46
+ }
7
47
 
8
48
  // Overridable for tests and for routing via a proxy/gateway.
9
49
  const REST_BASE =
@@ -163,7 +203,11 @@ function collectText(node, acc) {
163
203
  return acc;
164
204
  }
165
205
  if (typeof node.text === 'string' && node.text.trim()) acc.push(node.text);
166
- for (const v of Object.values(node)) {
206
+ for (const [k, v] of Object.entries(node)) {
207
+ // Citations are sources *about* the answer, not answer text: an agent step
208
+ // carries them alongside its content, and a citation's own title/snippet
209
+ // must not be concatenated into the result. They are collected separately.
210
+ if (k === 'citations') continue;
167
211
  if (v && typeof v === 'object') collectText(v, acc);
168
212
  }
169
213
  return acc;
@@ -173,21 +217,65 @@ function collectText(node, acc) {
173
217
  // internal reasoning with the actual answer, each tagged by `type`:
174
218
  // [ {type:'user_input', ...}, {type:'thought', ...}, {type:'model_output', ...} ]
175
219
  // Collecting text indiscriminately prepends the prompt (and any reasoning) to
176
- // the result, so those step types are skipped. Anything else -- model_output,
177
- // an untyped step, a future answer-bearing type -- still contributes, so a
178
- // renamed step never silently blanks the result.
220
+ // the result, so those step types are skipped.
221
+ //
222
+ // Both kinds of run put the deliverable in the FINAL answer-bearing step. A
223
+ // model run ends [user_input, thought, model_output]; an agent run's steps
224
+ // additionally interleave its plan, searches and interim drafts, and the docs
225
+ // place the finished report at `interaction.steps[-1].content[0].text`. So one
226
+ // rule serves both, with no special-casing on the agent id: take the last step
227
+ // that is not user_input/thought. If that step somehow carries no text -- an
228
+ // unexpected shape, a renamed type -- fall back to collecting across every
229
+ // answer-bearing step, so the failure mode is "too much text", never a
230
+ // silently blank result.
179
231
  const NON_ANSWER_STEP = new Set(['user_input', 'thought']);
180
232
 
181
233
  function textFromSteps(steps) {
182
234
  if (!Array.isArray(steps)) return '';
235
+ const candidates = steps.filter((s) => !(s && NON_ANSWER_STEP.has(s.type)));
236
+ if (!candidates.length) return '';
237
+ const last = collectText(candidates[candidates.length - 1], []).join('\n').trim();
238
+ if (last) return last;
183
239
  const acc = [];
184
- for (const step of steps) {
185
- if (step && NON_ANSWER_STEP.has(step.type)) continue;
186
- collectText(step, acc);
187
- }
240
+ for (const step of candidates) collectText(step, acc);
188
241
  return acc.join('\n').trim();
189
242
  }
190
243
 
244
+ // Agent runs carry citations -- the docs explicitly tell users to review them
245
+ // to verify the sources -- so they are gathered rather than discarded. The
246
+ // walk is shape-agnostic (any `citations` array anywhere in the interaction),
247
+ // because the docs do not pin down where they attach; duplicates are dropped.
248
+ function collectCitations(node, acc) {
249
+ if (!node || typeof node !== 'object') return acc;
250
+ if (Array.isArray(node)) {
251
+ for (const n of node) collectCitations(n, acc);
252
+ return acc;
253
+ }
254
+ for (const [k, v] of Object.entries(node)) {
255
+ if (k === 'citations' && Array.isArray(v)) {
256
+ for (const c of v) if (c && typeof c === 'object') acc.push(c);
257
+ continue;
258
+ }
259
+ if (v && typeof v === 'object') collectCitations(v, acc);
260
+ }
261
+ return acc;
262
+ }
263
+
264
+ function citationsOf(interaction) {
265
+ const all = collectCitations(interaction, []);
266
+ if (!all.length) return null;
267
+ const seen = new Set();
268
+ const out = [];
269
+ for (const c of all) {
270
+ const key = JSON.stringify(c);
271
+ if (!seen.has(key)) {
272
+ seen.add(key);
273
+ out.push(c);
274
+ }
275
+ }
276
+ return out;
277
+ }
278
+
191
279
  function textOf(interaction) {
192
280
  if (interaction && typeof interaction.output_text === 'string' && interaction.output_text) {
193
281
  return interaction.output_text;
@@ -200,6 +288,7 @@ function shape(r) {
200
288
  interactionId: r.id,
201
289
  status: r.status,
202
290
  text: textOf(r),
291
+ citations: citationsOf(r),
203
292
  usage: r.usage || null,
204
293
  raw: r,
205
294
  };
@@ -269,8 +358,29 @@ function restHeaders() {
269
358
 
270
359
  async function submit(prompt, opts) {
271
360
  const o = opts || {};
272
- const body = { model: o.model || DEFAULT_MODEL, input: prompt, background: true };
361
+ // `agent` and `model` are mutually exclusive on create: an agent run is sent
362
+ // with `agent` INSTEAD of `model` (the agent picks its own models). `input`
363
+ // stays a plain string and `background` stays true either way -- agents
364
+ // *require* background execution, which gemcatch has always set.
365
+ const body = o.agent
366
+ ? { agent: o.agent, input: prompt, background: true }
367
+ : { model: o.model || DEFAULT_MODEL, input: prompt, background: true };
273
368
  if (o.systemInstruction) body.system_instruction = o.systemInstruction;
369
+ // collaborative_planning is an `agent_config` field, NOT a top-level one, and
370
+ // the docs send the whole block (type + thinking_summaries) with it. Sent only
371
+ // when a plan turn is involved -- agent_config is optional otherwise, so an
372
+ // ordinary run keeps making exactly the request it always made. Presence, not
373
+ // truthiness: `false` is the approval turn's real value and must reach the API.
374
+ if (o.collaborativePlanning !== undefined) {
375
+ body.agent_config = {
376
+ type: AGENT_CONFIG_TYPE,
377
+ thinking_summaries: 'auto',
378
+ collaborative_planning: !!o.collaborativePlanning,
379
+ };
380
+ }
381
+ // Continues an earlier interaction server-side: the plan is already in that
382
+ // conversation, so this turn sends only what changed.
383
+ if (o.previousInteractionId) body.previous_interaction_id = o.previousInteractionId;
274
384
  const r = await call(() => {
275
385
  const api = sdkInteractions();
276
386
  return api
@@ -323,6 +433,10 @@ module.exports = {
323
433
  REST_BASE,
324
434
  RPM,
325
435
  MAX_RETRIES,
436
+ AGENT_ALIASES,
437
+ AGENT_PRICE_BANDS,
438
+ AGENT_CONFIG_TYPE,
439
+ resolveAgent,
326
440
  submit,
327
441
  poll,
328
442
  cancel,
@@ -330,6 +444,7 @@ module.exports = {
330
444
  apiKey,
331
445
  textOf,
332
446
  collectText,
447
+ citationsOf,
333
448
  // Exported for the suite: the retry policy is behaviour worth pinning.
334
449
  shouldRetry,
335
450
  // Re-exported so callers need only one require.