gemcatch 0.1.0

package/CHANGELOG.md

@@ -0,0 +1,68 @@
+# Changelog
+
+All notable changes to this project are documented here.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-07-17
+
+First release. Wraps the background execution capability added to the Gemini
+Interactions API on [2026-07-07](https://blog.google/innovation-and-ai/technology/developers-tools/expanding-managed-agents-gemini-api/).
+
+### Added
+
+- `gemcatch research "<prompt>"` — submits with `background: true` and exits immediately.
+  Reads the prompt from an argument, from `--file`, or from stdin via `-`.
+  Supports `--model`, `--system`, `--tag`, `--watch` and `--json`.
+- `gemcatch status <id>` — polls the API and prints the current state.
+- `gemcatch get <id>` — prints the result if complete, otherwise the current status.
+  Completed results are served from SQLite without a network call. `--raw` dumps
+  the raw interaction JSON.
+- `gemcatch list` — all tasks, newest first. Filters: `--status`, `--tag`, `-n`.
+- `gemcatch watch <id>` — polls until the task finishes, then prints the result.
+  `--interval` tunes the poll rate. Status chatter goes to stderr so the result
+  can be redirected cleanly.
+- `gemcatch sync` — refreshes every in-flight task in one pass, four polls at a time.
+- `gemcatch daemon` — polls in-flight tasks on a loop (default every 300s) so results
+  are cached locally before the free tier drops them at 24h. `--exit-when-idle`
+  stops once nothing is left in flight; `--json` emits newline-delimited events.
+  Results are committed to SQLite as each poll lands, so an abrupt kill loses
+  nothing already collected.
+- Request pacing: every API call is held to `GEMCATCH_RPM` requests/minute (default
+  15, the free-tier allowance; `0` disables). Capping concurrency alone does not
+  cap a rate, which a wide `sync` would otherwise discover the hard way.
+- Retries: transient failures (429, 408, 5xx, network errors) get
+  `GEMCATCH_MAX_RETRIES` further attempts (default 4) with exponential backoff and
+  full jitter, honouring `Retry-After`. A 4xx is surfaced immediately — it will
+  fail identically forever, and retrying it only burns the rate limit.
+- `gemcatch cancel <id>` — asks the API to stop an in-flight task.
+- `gemcatch rm <ids...>` — forgets tasks locally; `--remote` also deletes them server-side.
+- `gemcatch prune` — drops finished tasks older than `--days` (default 30). Never
+  touches in-flight work. `--dry-run` shows what would go.
+- `gemcatch stats` — where the store lives and what's in it.
+- Task IDs are 8-character UUID prefixes, and any unique prefix resolves.
+- Token usage is recorded per task when the API reports it.
+- Colour output on a TTY, honouring `NO_COLOR`.
+- Schema migrations: a v1 `tasks.db` upgrades in place without losing rows.
+- Offline test suite covering every command against a mock Interactions API.
+  No key or network required.
+
+### Notes
+
+Two details in the API differ from what a first reading of the announcement
+suggests, and both are handled here:
+
+- The API key must be sent as `x-goog-api-key`. `Authorization: Bearer <key>`
+  is rejected with `401 ACCESS_TOKEN_TYPE_UNSUPPORTED`.
+- The prompt field is `input` (which accepts a plain string), not `contents`.
+
+Free-tier interactions are retained server-side for 24 hours. Once `gemcatch` has
+seen a task complete, the text is cached locally and survives that expiry — but
+something has to poll inside that window for it to be seen at all, which is what
+`gemcatch daemon` exists to do.
+
+[Unreleased]: https://github.com/Booyaka101/gemcatch/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/Booyaka101/gemcatch/releases/tag/v0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 gemcatch contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,211 @@
+# gemcatch
+
+[![CI](https://github.com/Booyaka101/gemcatch/actions/workflows/ci.yml/badge.svg)](https://github.com/Booyaka101/gemcatch/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/gemcatch.svg)](https://www.npmjs.com/package/gemcatch)
+[![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+Fire-and-forget research tasks for the Gemini API. Submit a long-running prompt, get a task ID back in under a second, close your laptop, collect the answer later.
+
+On [July 7, 2026](https://blog.google/innovation-and-ai/technology/developers-tools/expanding-managed-agents-gemini-api/) Google added background execution to the Gemini Interactions API:
+
+> Holding an HTTP connection open for long-running tasks is fragile. Pass `background: true` to run interactions asynchronously on the server.
+
+That solves the server half. The client half is still on you: you get back an interaction ID and now you own it — polling it, remembering which prompt it belonged to, not losing it when your shell dies, noticing that the free tier throws it away after 24 hours. `gemcatch` is that half. It passes `background: true`, stores the interaction ID in local SQLite next to the prompt that created it, gives you a handful of commands to get results back, and runs a [daemon](#dont-lose-results-gemcatch-daemon) that collects them before the free tier drops them.
+
+```console
+$ gemcatch research "compare the 2026 EU AI Act timelines against the UK approach"
+Task 8f3a1c04 submitted. Run: gemcatch get 8f3a1c04 when ready.
+
+$ # ...close your laptop, come back later...
+
+$ gemcatch get 8f3a1c04
+The EU AI Act's high-risk obligations phase in from August 2026, whereas...
+```
+
+## Setup
+
+Needs Node.js 20+ and a Gemini API key. **Getting a key needs no billing account and no card.** `gemini-3.1-flash-lite` runs free within the [free tier's](https://ai.google.dev/gemini-api/docs/pricing) daily quota; past that, paid rates apply.
+
+1. Get a key at **<https://aistudio.google.com/apikey>**
+2. Put it in your environment:
+
+```powershell
+# PowerShell
+$env:GEMINI_API_KEY = "your-key-here"
+```
+
+```bash
+# bash / zsh
+export GEMINI_API_KEY=your-key-here
+```
+
+To make it permanent, add that line to your shell profile (`$PROFILE` on PowerShell, `~/.bashrc` or `~/.zshrc` on Unix). `GOOGLE_API_KEY` works too.
+
+## Run
+
+```bash
+npx gemcatch research "your question"
+```
+
+Or install it once and use the short name:
+
+```bash
+npm install -g gemcatch
+gemcatch research "your question"
+```
+
+From a clone: `npm install && node index.js research "your question"`.
+
+## Three examples
+
+```bash
+# 1. Submit — returns immediately with a task ID
+$ gemcatch research "summarize this week in AI"
+Task 8f3a1c04 submitted. Run: gemcatch get 8f3a1c04 when ready.
+
+# 2. Check on it — or `gemcatch list` to see everything
+$ gemcatch status 8f3a1c04
+Task 8f3a1c04: in_progress
+
+# 3. Collect the answer (blocks and polls until done)
+$ gemcatch watch 8f3a1c04
+[10:52:31] 8f3a1c04: in_progress
+[10:54:02] 8f3a1c04: completed
+This week in AI: ...
+```
+
+## Commands
+
+| Command | What it does |
+| --- | --- |
+| `gemcatch research "<prompt>"` | Submits with `background: true`, stores the interaction ID, exits immediately. |
+| `gemcatch status <id>` | Polls the API and prints the current state. |
+| `gemcatch get <id>` | Prints the full response if complete, otherwise the current status. |
+| `gemcatch list` | All tasks, newest first: id, age, status, prompt. |
+| `gemcatch watch <id>` | Polls until the task finishes, then prints the result. |
+| `gemcatch sync` | Refreshes every in-flight task in one pass. |
+| `gemcatch daemon` | Keeps polling in-flight tasks on a loop, so results are cached before they expire. |
+| `gemcatch cancel <id>` | Asks the API to stop an in-flight task. |
+| `gemcatch rm <ids...>` | Forgets tasks locally. `--remote` deletes them server-side too. |
+| `gemcatch prune` | Drops finished tasks older than `--days` (default 30). |
+| `gemcatch stats` | Where the store lives and what's in it. |
+
+Useful flags:
+
+| Flag | On | Does |
+| --- | --- | --- |
+| `--json` | most commands | Machine-readable output. |
+| `-m, --model <id>` | `research` | Override the model. |
+| `-s, --system <text>` | `research` | Set a system instruction. |
+| `-f, --file <path>` | `research` | Read the prompt from a file. |
+| `-t, --tag <tag>` | `research`, `list` | Label tasks and filter them. |
+| `-w, --watch` | `research` | Submit and wait, in one command. |
+| `-i, --interval <s>` | `watch`, `daemon` | Poll rate. Default 10s for `watch`, 300s for `daemon`. |
+| `--exit-when-idle` | `daemon` | Stop once nothing is left in flight. |
+| `-n, --limit <n>` | `list` | Cap the rows. |
+| `--dry-run` | `prune` | Show what would go; delete nothing. |
+| `--raw` | `get` | Dump the raw interaction JSON. |
+
+IDs are the first 8 characters of a UUID. Any unique prefix works, so `gemcatch get 8f3a` is fine.
+
+Statuses come straight from the API: `in_progress`, `requires_action`, `completed`, `failed`, `cancelled`, `incomplete`, `budget_exceeded`. Plus `pending`, which is local: the row exists but the submit call hasn't returned yet.
+
+## Recipes
+
+```bash
+# Fire off a batch, then collect later
+$ for q in "topic A" "topic B" "topic C"; do gemcatch research "$q" -t batch1; done
+$ gemcatch sync                       # one pass now...
+$ gemcatch daemon --exit-when-idle    # ...or keep polling until they're all in
+$ gemcatch list --tag batch1 --status completed
+
+# Long prompt from a file, result to a file.
+# Progress goes to stderr, so the redirect captures only the answer.
+$ gemcatch research -f brief.md -w > answer.md
+
+# Pipe a prompt in
+$ cat notes.txt | gemcatch research - -s "extract every open question"
+
+# Script against it
+$ id=$(gemcatch research "..." --json | jq -r .id)
+$ gemcatch watch "$id" --json | jq -r .result
+```
+
+## How it works
+
+Tasks live in SQLite at `~/.gemcatch/tasks.db` (override with `GEMCATCH_HOME`):
+
+```sql
+CREATE TABLE tasks (id TEXT PRIMARY KEY, prompt TEXT, interaction_id TEXT,
+                    status TEXT DEFAULT 'pending', result TEXT, created_at INTEGER);
+-- plus model, system_instruction, tag, error, usage, updated_at
+```
+
+`research` calls `interactions.create({model, input, background: true})` via [`@google/genai`](https://www.npmjs.com/package/@google/genai) and keeps the returned `id`. The polling commands call `interactions.get(id)` and write the status back. Once a task completes, the text is cached in the `result` column — `gemcatch get` then answers from disk without touching the network.
+
+An older `tasks.db` upgrades in place; migrations are additive and never drop a row.
+
+**Free-tier results expire after 24 hours.** The [docs](https://ai.google.dev/gemini-api/docs/interactions-overview) note the system retains interactions for 1 day on the free tier (55 days paid). Once `gemcatch` has seen a task complete, the text is cached locally and survives that expiry — but a task nobody polls inside that window is gone server-side. That is what `gemcatch daemon` is for.
+
+## Don't lose results: `gemcatch daemon`
+
+Caching a result permanently is easy; *noticing* it is the hard part. If nothing polls a finished task within 24 hours, the answer is dropped server-side and no amount of local bookkeeping brings it back. `gemcatch daemon` is the something that looks:
+
+```console
+$ gemcatch daemon
+gemcatch daemon: polling every 300s. Store: ~/.gemcatch/tasks.db. Ctrl-C to stop.
+[10:54:02] 8f3a1c04: completed
+[11:31:20] c7b91e55: completed
+```
+
+It refreshes everything in flight on an interval, writes each result to SQLite the moment it lands, and stays quiet otherwise — only transitions and errors get a line. It's an ordinary foreground process: no forking, no pidfile. Leave it in a terminal, or hand it to whatever supervises long-running jobs on your machine (a systemd user unit, a launchd agent, Task Scheduler at log-on, `nohup`, `pm2`).
+
+**Nothing is lost if it dies.** Every poll is committed to SQLite synchronously, so killing the daemon — Ctrl-C, `kill -9`, a reboot — stops the polling and nothing else. Results it already collected are on disk, and restarting picks up where it left off.
+
+`--exit-when-idle` stops once nothing is in flight, which makes it the "collect this batch, then quit" one-liner:
+
+```bash
+$ for q in "topic A" "topic B" "topic C"; do gemcatch research "$q" -t batch1; done
+$ gemcatch daemon --exit-when-idle -i 30
+$ gemcatch list --tag batch1 --status completed
+```
+
+## Rate limits and retries
+
+The free tier allows roughly 15 requests a minute, which a wide `gemcatch sync` or a busy daemon would otherwise blow straight through. Every outbound call is paced to `GEMCATCH_RPM` (default 15) — set it higher on a paid key, or `0` to disable pacing entirely.
+
+Transient failures are retried with exponential backoff and full jitter, honouring `Retry-After` when the server sends it. A rate limit, a timeout or a 5xx gets `GEMCATCH_MAX_RETRIES` more attempts (default 4); a 4xx does not, because a bad key or a bad model id fails identically forever and retrying it only burns your quota.
+
+## Environment variables
+
+| Variable | Purpose |
+| --- | --- |
+| `GEMINI_API_KEY` | Your API key. `GOOGLE_API_KEY` also works. |
+| `GEMCATCH_HOME` | Where `tasks.db` lives. Default `~/.gemcatch`. |
+| `GEMCATCH_MODEL` | Default model. Default `gemini-3.1-flash-lite`. |
+| `GEMCATCH_POLL_MS` | `watch` poll interval in ms. Default `10000`. |
+| `GEMCATCH_DAEMON_S` | `daemon` interval in seconds. Default `300`. |
+| `GEMCATCH_RPM` | Requests/minute ceiling. Default `15` (the free tier). `0` disables pacing. |
+| `GEMCATCH_MAX_RETRIES` | Extra attempts on a transient failure. Default `4`. `0` disables retries. |
+| `GEMCATCH_BASE_URL` | Override the API endpoint (proxy/gateway/testing). |
+| `GEMCATCH_FORCE_REST` | `1` bypasses the SDK and uses raw `fetch`. |
+| `NO_COLOR` | Disable colour output. |
+
+## Development
+
+```bash
+git clone https://github.com/Booyaka101/gemcatch
+cd gemcatch
+npm install
+npm test
+```
+
+**The tests need no API key and no network.** They run the real CLI as a subprocess against a mock Interactions API on localhost, covering every command, the failure paths, and schema migration. See [CONTRIBUTING.md](CONTRIBUTING.md) for the layout and for the API gotchas worth knowing before touching `gemini.js`.
+
+## Contributing
+
+Issues and PRs welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) and the [Code of Conduct](CODE_OF_CONDUCT.md). Security reports go through [private disclosure](SECURITY.md), not public issues.
+
+## License
+
+[MIT](LICENSE)

package/db.js

@@ -0,0 +1,186 @@
+'use strict';
+
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const crypto = require('crypto');
+const Database = require('better-sqlite3');
+const { PENDING, TERMINAL } = require('./status');
+
+// GEMCATCH_HOME lets tests and power users relocate the store.
+const HOME = process.env.GEMCATCH_HOME || path.join(os.homedir(), '.gemcatch');
+const DB_PATH = path.join(HOME, 'tasks.db');
+
+// The v1 shape. Never change this -- migrations below carry it forward, so an
+// old tasks.db keeps opening cleanly.
+const BASE_SCHEMA =
+  'CREATE TABLE IF NOT EXISTS tasks (id TEXT PRIMARY KEY, prompt TEXT, ' +
+  "interaction_id TEXT, status TEXT DEFAULT 'pending', result TEXT, created_at INTEGER)";
+
+// Columns added after v1. Additive only: SQLite can ALTER TABLE ADD COLUMN but
+// not drop or retype, so anything here must be nullable.
+const MIGRATIONS = [
+  ['model', 'TEXT'],
+  ['system_instruction', 'TEXT'],
+  ['tag', 'TEXT'],
+  ['error', 'TEXT'],
+  ['usage', 'TEXT'],
+  ['updated_at', 'INTEGER'],
+];
+
+let _db = null;
+
+function migrate(d) {
+  const have = new Set(d.prepare('PRAGMA table_info(tasks)').all().map((c) => c.name));
+  for (const [name, type] of MIGRATIONS) {
+    if (!have.has(name)) d.exec(`ALTER TABLE tasks ADD COLUMN ${name} ${type}`);
+  }
+  d.exec('CREATE INDEX IF NOT EXISTS idx_tasks_created_at ON tasks (created_at DESC)');
+  d.exec('CREATE INDEX IF NOT EXISTS idx_tasks_status ON tasks (status)');
+}
+
+function db() {
+  if (_db) return _db;
+  fs.mkdirSync(HOME, { recursive: true });
+  _db = new Database(DB_PATH);
+  _db.pragma('journal_mode = WAL');
+  _db.exec(BASE_SCHEMA);
+  migrate(_db);
+  return _db;
+}
+
+function newId() {
+  return crypto.randomUUID().slice(0, 8);
+}
+
+function createTask(fields) {
+  const t = fields || {};
+  const id = newId();
+  const now = Date.now();
+  db()
+    .prepare(
+      'INSERT INTO tasks (id, prompt, status, created_at, updated_at, model, system_instruction, tag) ' +
+        'VALUES (@id, @prompt, @status, @now, @now, @model, @system_instruction, @tag)'
+    )
+    .run({
+      id,
+      prompt: t.prompt,
+      status: PENDING,
+      now,
+      model: t.model || null,
+      system_instruction: t.systemInstruction || null,
+      tag: t.tag || null,
+    });
+  return id;
+}
+
+// Exact id first, then unique-prefix match so short ids stay forgiving.
+// Returns null for no match; throws on an ambiguous prefix rather than
+// silently picking one.
+function getTask(id) {
+  const exact = db().prepare('SELECT * FROM tasks WHERE id = ?').get(id);
+  if (exact) return exact;
+  const hits = db().prepare('SELECT * FROM tasks WHERE id LIKE ? ORDER BY created_at DESC').all(id + '%');
+  if (hits.length > 1) {
+    const e = new Error(`'${id}' matches ${hits.length} tasks: ${hits.map((h) => h.id).join(', ')}`);
+    e.code = 'AMBIGUOUS_ID';
+    throw e;
+  }
+  return hits[0] || null;
+}
+
+function setInteraction(id, interactionId, status) {
+  db()
+    .prepare('UPDATE tasks SET interaction_id = ?, status = ?, updated_at = ? WHERE id = ?')
+    .run(interactionId, status, Date.now(), id);
+}
+
+// Only overwrites result/error/usage when the caller supplies them, so a poll
+// that returns no text can't blank out a result we already stored.
+function setStatus(id, status, extra) {
+  const e = extra || {};
+  const sets = ['status = @status', 'updated_at = @now'];
+  const params = { id, status, now: Date.now() };
+  for (const key of ['result', 'error', 'usage']) {
+    if (e[key] !== undefined) {
+      sets.push(`${key} = @${key}`);
+      params[key] = e[key];
+    }
+  }
+  db().prepare(`UPDATE tasks SET ${sets.join(', ')} WHERE id = @id`).run(params);
+}
+
+function listTasks(opts) {
+  const o = opts || {};
+  const where = [];
+  const params = {};
+  if (o.status) {
+    where.push('status = @status');
+    params.status = o.status;
+  }
+  if (o.tag) {
+    where.push('tag = @tag');
+    params.tag = o.tag;
+  }
+  let sql = 'SELECT * FROM tasks';
+  if (where.length) sql += ` WHERE ${where.join(' AND ')}`;
+  sql += ' ORDER BY created_at DESC';
+  if (o.limit) {
+    sql += ' LIMIT @limit';
+    params.limit = o.limit;
+  }
+  return db().prepare(sql).all(params);
+}
+
+// In-flight tasks worth re-polling: submitted (we have an id) but not final.
+function activeTasks() {
+  const marks = TERMINAL.map(() => '?').join(', ');
+  return db()
+    .prepare(
+      `SELECT * FROM tasks WHERE interaction_id IS NOT NULL AND status NOT IN (${marks}) ORDER BY created_at ASC`
+    )
+    .all(...TERMINAL);
+}
+
+function removeTask(id) {
+  return db().prepare('DELETE FROM tasks WHERE id = ?').run(id).changes > 0;
+}
+
+// Finished tasks older than `beforeMs`. Never touches in-flight work.
+function prunableTasks(beforeMs) {
+  const marks = TERMINAL.map(() => '?').join(', ');
+  return db()
+    .prepare(`SELECT * FROM tasks WHERE status IN (${marks}) AND created_at < ? ORDER BY created_at ASC`)
+    .all(...TERMINAL, beforeMs);
+}
+
+function removeMany(ids) {
+  if (!ids.length) return 0;
+  const del = db().prepare('DELETE FROM tasks WHERE id = ?');
+  return db().transaction((list) => list.reduce((n, id) => n + del.run(id).changes, 0))(ids);
+}
+
+function counts() {
+  return db().prepare('SELECT status, COUNT(*) AS n FROM tasks GROUP BY status').all();
+}
+
+function close() {
+  if (_db) _db.close();
+  _db = null;
+}
+
+module.exports = {
+  DB_PATH,
+  HOME,
+  createTask,
+  getTask,
+  setInteraction,
+  setStatus,
+  listTasks,
+  activeTasks,
+  removeTask,
+  removeMany,
+  prunableTasks,
+  counts,
+  close,
+};